MongoDBNode.jsdatabase optimizationNoSQL performancequery optimizationMongoDB indexingbackend developmentAPI performance

MongoDB Optimization: 10 Proven Techniques for Node.js APIs

Usama Amjid
12 min read
MongoDB Optimization: 10 Proven Techniques for Node.js APIs

Introduction

Your Node.js API is drowning in database queries. What started as millisecond response times has degraded to multi-second delays. Your MongoDB database, once a performance champion, now buckles under production traffic. Sound familiar?

Database bottlenecks account for over 80% of performance issues in high-traffic Node.js applications. The problem isn't MongoDB itself—it's how you're using it. Most developers implement basic CRUD operations without considering query patterns, index strategies, or connection management at scale.

In this guide, you'll learn 10 battle-tested MongoDB optimization techniques that have helped applications handle millions of requests daily. These aren't theoretical concepts—they're practical strategies you can implement today to transform your database performance from sluggish to lightning-fast. Whether you're struggling with slow aggregations, inefficient queries, or connection pool exhaustion, you'll find actionable solutions here.

1. Strategic Index Design: Beyond Basic Single-Field Indexes

Most developers stop at creating single-field indexes. That's leaving performance on the table.

Compound indexes are your secret weapon for complex queries. When you query multiple fields together, MongoDB can use a single compound index instead of merging multiple single-field indexes—a process that's significantly slower.

typescript
// BAD: Separate single-field indexes
await db.collection('orders').createIndex({ userId: 1 });
await db.collection('orders').createIndex({ status: 1 });
await db.collection('orders').createIndex({ createdAt: -1 });

// GOOD: Strategic compound index
await db.collection('orders').createIndex({ 
  userId: 1, 
  status: 1, 
  createdAt: -1 
});

// Query that leverages the compound index
const userOrders = await db.collection('orders').find({
  userId: new ObjectId(userId),
  status: 'pending'
}).sort({ createdAt: -1 }).limit(20);

The ESR Rule for Compound Indexes:

Follow the Equality-Sort-Range (ESR) rule when designing compound indexes:

  • •Equality fields first (exact matches like userId: value)
  • •Sort fields second (fields used in .sort())
  • •Range fields last (fields using $gt, $lt, $in)

This order maximizes index efficiency because MongoDB can precisely locate documents, then sort them, then filter the range—in that sequence.

Partial Indexes for Sparse Data:

typescript
// Only index active users to reduce index size
await db.collection('users').createIndex(
  { email: 1 },
  { 
    partialFilterExpression: { isActive: true },
    name: 'active_users_email_idx'
  }
);

// This query uses the partial index
const activeUser = await db.collection('users').findOne({
  email: 'user@example.com',
  isActive: true
});

Partial indexes reduce index size by up to 70% when you have large datasets with filtered query patterns.

2. Connection Pooling: Stop Creating Connections on Every Request

Creating a new MongoDB connection for each request is like rebuilding your car before every drive. It's wasteful and slow.

Connection pooling reuses existing connections, dramatically reducing overhead. A properly configured pool can improve throughput by 300-500%.

typescript
// BAD: Creating connection per request (anti-pattern)
app.get('/api/users/:id', async (req, res) => {
  const client = new MongoClient(process.env.MONGODB_URI);
  await client.connect(); // Expensive operation every request!
  const user = await client.db().collection('users').findOne({...});
  await client.close();
  res.json(user);
});

// GOOD: Singleton connection pool
import { MongoClient } from 'mongodb';

class DatabaseConnection {
  private static instance: MongoClient;
  
  static async getInstance(): Promise<MongoClient> {
    if (!this.instance) {
      this.instance = new MongoClient(process.env.MONGODB_URI!, {
        maxPoolSize: 50,        // Max connections in pool
        minPoolSize: 10,        // Keep warm connections
        maxIdleTimeMS: 30000,   // Close idle connections
        serverSelectionTimeoutMS: 5000,
        socketTimeoutMS: 45000,
      });
      await this.instance.connect();
      console.log('MongoDB connection pool established');
    }
    return this.instance;
  }
}

// Usage in route handlers
app.get('/api/users/:id', async (req, res) => {
  const client = await DatabaseConnection.getInstance();
  const user = await client.db().collection('users').findOne({
    _id: new ObjectId(req.params.id)
  });
  res.json(user);
  // Connection returns to pool automatically
});

Pool Size Configuration:

Your optimal pool size depends on your workload:

  • •CPU-bound workloads: Pool size = CPU cores × 2
  • •I/O-bound workloads: Pool size = 50-100 connections
  • •Mixed workloads: Start with 50, monitor, and adjust

Monitor your pool with these metrics: availableConnections, activeConnections, and waitQueueSize. If waitQueueSize grows consistently, increase your pool size.

3. Projection: Fetch Only What You Need

Retrieving entire documents when you need 3 fields is like downloading a 4K movie to watch a 10-second clip.

Projection allows you to specify exactly which fields to return, reducing network transfer and deserialization overhead by up to 90%.

typescript
// BAD: Fetching entire user document (wasteful)
const user = await db.collection('users').findOne({ _id: userId });
return { name: user.name, email: user.email };
// Transferred: ~5KB of unnecessary data

// GOOD: Projection fetches only required fields
const user = await db.collection('users').findOne(
  { _id: userId },
  { projection: { name: 1, email: 1, _id: 0 } }
);
return user;
// Transferred: ~100 bytes

// Advanced: Exclude large embedded documents
const article = await db.collection('articles').findOne(
  { slug: articleSlug },
  { 
    projection: { 
      title: 1, 
      excerpt: 1, 
      author: 1,
      comments: 0,        // Exclude comments array
      fullContent: 0      // Exclude large text field
    } 
  }
);

When Projection Makes the Biggest Impact:

  • •Documents with large embedded arrays or nested objects
  • •List views requiring only summary data
  • •API responses with hundreds/thousands of documents
  • •Mobile applications with bandwidth constraints

Combine projection with pagination for maximum efficiency on list endpoints.

4. Aggregation Pipeline Optimization: Order Matters

Aggregation pipelines are powerful but can become performance killers if stages are poorly ordered.

MongoDB processes pipeline stages sequentially. Each stage receives all documents from the previous stage. Your goal: reduce the document set as early as possible.

typescript
// BAD: Filtering after expensive operations
const pipeline = [
  {
    $lookup: {  // Expensive join operation on ALL documents
      from: 'products',
      localField: 'productId',
      foreignField: '_id',
      as: 'product'
    }
  },
  { $unwind: '$product' },
  { $match: { userId: new ObjectId(userId) } }, // Filter AFTER join
  { $sort: { createdAt: -1 } },
  { $limit: 20 }
];

// GOOD: Filter early, leverage indexes
const pipeline = [
  { 
    $match: { userId: new ObjectId(userId) }  // Filter FIRST
  },
  { $sort: { createdAt: -1 } },  // Index-backed sort
  { $limit: 20 },                 // Reduce dataset early
  {
    $lookup: {  // Join on reduced dataset
      from: 'products',
      localField: 'productId',
      foreignField: '_id',
      as: 'product'
    }
  },
  { $unwind: '$product' },
  {
    $project: {  // Project last to shape output
      'product.name': 1,
      'product.price': 1,
      quantity: 1,
      total: 1
    }
  }
];

const orders = await db.collection('orders').aggregate(pipeline).toArray();

Aggregation Optimization Principles:

  1. $match early: Filter documents before expensive operations
  2. $project early: Remove unnecessary fields to reduce data size
  3. $limit after sort: Reduce documents processed in later stages
  4. $lookup last: Joins are expensive—do them on small datasets
  5. Use indexes: Ensure $match and $sort stages use indexes

Use .explain('executionStats') to verify your pipeline uses indexes and check documents processed at each stage.

5. Implement Read-Through Caching with Redis

Database queries, even optimized ones, are slower than in-memory cache hits. Redis can serve cached data in under 1ms versus 10-50ms for MongoDB queries.

typescript
import { createClient } from 'redis';
import { MongoClient, ObjectId } from 'mongodb';

class CachedRepository {
  private redis = createClient({ url: process.env.REDIS_URL });
  private mongo: MongoClient;
  
  constructor(mongo: MongoClient) {
    this.mongo = mongo;
    this.redis.connect();
  }
  
  async getUserById(userId: string) {
    const cacheKey = `user:${userId}`;
    
    // Try cache first
    const cached = await this.redis.get(cacheKey);
    if (cached) {
      return JSON.parse(cached);
    }
    
    // Cache miss: fetch from MongoDB
    const user = await this.mongo
      .db()
      .collection('users')
      .findOne({ _id: new ObjectId(userId) });
    
    if (user) {
      // Cache for 5 minutes
      await this.redis.setEx(
        cacheKey, 
        300, 
        JSON.stringify(user)
      );
    }
    
    return user;
  }
  
  async updateUser(userId: string, updates: any) {
    // Update database
    await this.mongo
      .db()
      .collection('users')
      .updateOne(
        { _id: new ObjectId(userId) },
        { $set: updates }
      );
    
    // Invalidate cache
    await this.redis.del(`user:${userId}`);
  }
}

Caching Strategies:

  • •Cache-aside (Lazy Loading): Shown above—cache on read, invalidate on write
  • •Write-through: Update cache and database simultaneously
  • •TTL (Time-To-Live): Auto-expire stale data (use for rarely-changing data)

What to Cache:

  • •User profiles and session data
  • •Product catalogs and inventory
  • •API responses for public endpoints
  • •Computed aggregations that don't need real-time accuracy

What NOT to Cache:

  • •Financial transaction data requiring strict consistency
  • •Real-time data (live dashboards, stock prices)
  • •Large binary data (images, files)

For topics like Redis caching patterns and cache invalidation strategies, you can dive deeper into distributed caching architectures.

6. Lean Queries: Use Mongoose Lean Mode

If you're using Mongoose ODM, the .lean() method is a performance game-changer for read-heavy operations.

Mongoose documents are full-featured objects with methods, getters, and change tracking. That overhead costs you 5-10x performance compared to plain JavaScript objects.

typescript
import mongoose from 'mongoose';

// BAD: Full Mongoose document (slow)
const users = await User.find({ isActive: true });
// Returns: Mongoose documents with methods, virtuals, getters
// Memory: ~500 bytes per document
// Speed: ~10ms for 100 docs

// GOOD: Lean query (fast)
const users = await User.find({ isActive: true }).lean();
// Returns: Plain JavaScript objects
// Memory: ~100 bytes per document  
// Speed: ~2ms for 100 docs

// Real-world example: API endpoint
app.get('/api/products', async (req, res) => {
  const products = await Product
    .find({ category: req.query.category })
    .select('name price image')  // Projection
    .lean()                       // Skip Mongoose overhead
    .limit(50);
    
  res.json(products);
});

When to Use Lean:

  • •Read-only operations (GET endpoints)
  • •Bulk data exports
  • •Generating reports or analytics
  • •Any query where you don't need to modify documents

When NOT to Use Lean:

  • •You need to call .save() on documents
  • •You rely on Mongoose virtuals or methods
  • •You need automatic validation on updates

Lean mode can reduce query time by 40-60% and memory usage by 80% for large result sets.

7. Bulk Operations: Batch Your Writes

Executing 1,000 individual insert operations is like making 1,000 separate trips to the grocery store instead of one trip with a cart.

MongoDB's bulk operations batch multiple writes into a single round-trip, reducing network overhead and lock contention.

typescript
// BAD: Individual inserts (1000 round-trips)
for (const order of orders) {
  await db.collection('orders').insertOne(order);
}
// Time: ~5-10 seconds for 1000 documents

// GOOD: Bulk insert (1 round-trip)
await db.collection('orders').insertMany(orders, { ordered: false });
// Time: ~100-200ms for 1000 documents

// Advanced: Bulk write with mixed operations
const bulkOps = [
  {
    insertOne: {
      document: { name: 'Product A', price: 29.99 }
    }
  },
  {
    updateOne: {
      filter: { sku: 'ABC123' },
      update: { $set: { stock: 100 } }
    }
  },
  {
    deleteOne: {
      filter: { _id: obsoleteProductId }
    }
  }
];

const result = await db.collection('products').bulkWrite(bulkOps, {
  ordered: false  // Continue on errors, don't stop at first failure
});

console.log(`Inserted: ${result.insertedCount}`);
console.log(`Modified: ${result.modifiedCount}`);
console.log(`Deleted: ${result.deletedCount}`);

Ordered vs Unordered Bulk Ops:

  • •Ordered (default): Stops on first error, maintains order
  • •Unordered (ordered: false): Processes all operations, better performance

Use unordered for maximum throughput when operation order doesn't matter.

Optimal Batch Sizes:

  • •Small documents (<1KB): 1,000-5,000 per batch
  • •Medium documents (1-10KB): 500-1,000 per batch
  • •Large documents (>10KB): 100-500 per batch

Monitor writeErrors in the result to handle partial failures gracefully.

8. Query Result Pagination: Cursor-Based Over Offset-Based

Traditional offset-based pagination (skip(20).limit(10)) becomes exponentially slower as page numbers increase. MongoDB must scan and discard all skipped documents.

Cursor-based pagination uses indexed fields to efficiently jump to the right position.

typescript
// BAD: Offset-based pagination (slow for large offsets)
app.get('/api/posts', async (req, res) => {
  const page = parseInt(req.query.page) || 1;
  const limit = 20;
  const skip = (page - 1) * limit;
  
  const posts = await db.collection('posts')
    .find({})
    .sort({ createdAt: -1 })
    .skip(skip)      // Scans and discards documents
    .limit(limit)
    .toArray();
  // Page 100: MongoDB scans 2000 documents, returns 20
});

// GOOD: Cursor-based pagination (always fast)
app.get('/api/posts', async (req, res) => {
  const limit = 20;
  const cursor = req.query.cursor; // Last item's createdAt + _id
  
  const query = cursor 
    ? { 
        $or: [
          { createdAt: { $lt: new Date(cursor.createdAt) } },
          { 
            createdAt: new Date(cursor.createdAt),
            _id: { $lt: new ObjectId(cursor.id) }
          }
        ]
      }
    : {};
  
  const posts = await db.collection('posts')
    .find(query)
    .sort({ createdAt: -1, _id: -1 })
    .limit(limit + 1)  // Fetch one extra to check for next page
    .toArray();
  
  const hasNext = posts.length > limit;
  const results = hasNext ? posts.slice(0, limit) : posts;
  
  const nextCursor = hasNext ? {
    createdAt: results[results.length - 1].createdAt,
    id: results[results.length - 1]._id
  } : null;
  
  res.json({ 
    posts: results, 
    nextCursor,
    hasNext 
  });
});

Required Index for Cursor Pagination:

typescript
await db.collection('posts').createIndex({ 
  createdAt: -1, 
  _id: -1 
});

This compound index enables MongoDB to efficiently locate the starting position for each page—regardless of page depth.

Trade-offs:

  • •Cursor-based: Constant performance, no random page access
  • •Offset-based: Supports random pages, performance degrades

For infinite scroll and "Load More" UIs, cursor-based pagination is superior. For traditional page numbers (1, 2, 3...), use offset-based with reasonable limits (max 100 pages).

9. Monitor with Slow Query Logs and Profiling

You can't optimize what you don't measure. MongoDB's profiling system captures slow queries automatically.

typescript
// Enable profiling for queries slower than 100ms
await db.command({ 
  profile: 1,  // 0=off, 1=slow queries, 2=all queries
  slowms: 100  // Threshold in milliseconds
});

// Query the system.profile collection
const slowQueries = await db.collection('system.profile')
  .find({ 
    ns: 'mydb.users',  // Namespace (database.collection)
    millis: { $gt: 100 }
  })
  .sort({ ts: -1 })
  .limit(10)
  .toArray();

slowQueries.forEach(query => {
  console.log(`Operation: ${query.op}`);
  console.log(`Duration: ${query.millis}ms`);
  console.log(`Query: ${JSON.stringify(query.command)}`);
  console.log(`Docs Examined: ${query.docsExamined}`);
  console.log(`Docs Returned: ${query.nreturned}`);
  console.log(`---`);
});

Key Metrics to Monitor:

  • •docsExamined vs nreturned: Ratio should be close to 1:1. High ratio indicates missing indexes
  • •millis: Execution time—target <100ms for most queries
  • •planSummary: Shows if indexes were used (IXSCAN = good, COLLSCAN = bad)

Automated Monitoring Setup:

typescript
import { MongoClient } from 'mongodb';

class QueryMonitor {
  async analyzeSlowQueries(db: Db) {
    const slowQueries = await db.collection('system.profile')
      .aggregate([
        { $match: { millis: { $gt: 100 } } },
        { 
          $group: {
            _id: '$command',
            avgDuration: { $avg: '$millis' },
            count: { $sum: 1 },
            maxDuration: { $max: '$millis' }
          }
        },
        { $sort: { avgDuration: -1 } },
        { $limit: 5 }
      ])
      .toArray();
    
    // Alert or log problematic queries
    slowQueries.forEach(query => {
      if (query.avgDuration > 500) {
        console.error(`CRITICAL: Query averaging ${query.avgDuration}ms`);
        // Send alert to monitoring system
      }
    });
  }
}

Run this analysis hourly or daily to catch performance regressions early.

For deeper insights into application performance monitoring (APM) and database observability, consider exploring distributed tracing tools.

10. Schema Design: Embed vs Reference

MongoDB's flexible schema allows embedding related data or referencing it. The wrong choice can destroy performance.

Embedding: Store related data within the same document
Referencing: Store related data in separate collections with IDs

typescript
// EMBEDDED: One-to-few relationship (user + addresses)
interface User {
  _id: ObjectId;
  name: string;
  email: string;
  addresses: Array<{
    street: string;
    city: string;
    zipCode: string;
  }>;  // Embedded - retrieved in single query
}

// Single query to get user with addresses
const user = await db.collection('users').findOne({ _id: userId });
// Fast: 1 query, all data together

// REFERENCED: One-to-many relationship (user + orders)
interface User {
  _id: ObjectId;
  name: string;
  email: string;
}

interface Order {
  _id: ObjectId;
  userId: ObjectId;  // Reference to user
  items: Array<any>;
  total: number;
}

// Requires two queries or $lookup
const user = await db.collection('users').findOne({ _id: userId });
const orders = await db.collection('orders')
  .find({ userId: user._id })
  .toArray();
// Slower: 2 queries, but better for large/frequently changing data

Decision Framework:

Use Embedding When:

  • •Data is accessed together 90%+ of the time
  • •Related data is small and bounded (e.g., max 10 addresses)
  • •Related data rarely changes independently
  • •You need atomic updates across related data

Use Referencing When:

  • •Related data is large or unbounded (e.g., user's orders over time)
  • •Related data is accessed independently often
  • •Multiple entities reference the same data (product referenced by many orders)
  • •You need to query related data independently

Hybrid Approach:

typescript
// Store summary in main document, details in referenced collection
interface BlogPost {
  _id: ObjectId;
  title: string;
  content: string;
  commentCount: 15,        // Embedded summary
  latestComments: [        // Embed last 3 for quick display
    { author: 'Alice', text: 'Great post!', date: ISODate() }
  ],
  // All comments stored separately for pagination
}

// Full comments in separate collection for pagination
interface Comment {
  _id: ObjectId;
  postId: ObjectId;
  author: string;
  text: string;
  createdAt: Date;
}

This pattern gives you fast initial page loads (embedded summary) while keeping full datasets manageable (referenced).

Conclusion

Database optimization isn't a one-time task—it's an ongoing practice. These 10 techniques form a comprehensive toolkit for tackling MongoDB performance challenges in Node.js applications.

Start with the low-hanging fruit: add strategic indexes, implement connection pooling, and enable lean mode for read queries. Then progress to advanced optimizations like cursor-based pagination and caching layers as your traffic grows.

The key insight: measure before optimizing. Use profiling to identify your actual bottlenecks, then apply the appropriate technique. A well-indexed query with projection will always outperform an unoptimized cached query.

Next Steps: Implement query profiling this week to establish your performance baseline. Identify your top 3 slowest queries and apply the relevant techniques from this guide. Monitor the impact, then iterate.

Your database can handle 10x more traffic with the right optimizations. Now you have the roadmap to get there.

Written by

Usama Amjid

Backend Engineer & Node.js Architect

Get in Touch