Request Collapsing in NestJS

In a high-traffic production environment, basic caching is often not enough to protect your primary database. While adding Redis to your Project monolith is a great first step, it introduces a specific vulnerability known as the Cache Stampede or the Thundering Herd problem.

This guide explores how to design and implement Request Collapsing (also known as Promise Coalescing) to ensure your database remains stable even when cache keys expire during peak traffic.

1. The Problem: The Cache Stampede

In a standard caching strategy, the application follows a simple "Cache-Aside" pattern:

1. The application checks Redis for a key.
2. If the key exists (Cache Hit), it returns the data.
3. If the key does not exist (Cache Miss), it fetches data from PostgreSQL and updates Redis.

The danger arises during a Cache Miss under heavy concurrency. If 50 users request the same resource at the exact millisecond the cache expires, all 50 requests will perceive a Cache Miss simultaneously. Consequently, all 50 requests will attempt to "hydrate" the cache by querying the database at once.

This leads to:

• Connection Exhaustion: Your PostgreSQL connection pool can be quickly depleted.
• Latency Spikes: The database slows down under the sudden load, increasing response times for all users.
• Cascading Failures: If the database becomes unresponsive, the entire monolith may fail.

2. The Solution: Request Collapsing

Request Collapsing is a design pattern that ensures only one request is responsible for fetching data from the source when a cache miss occurs. All other concurrent requests for the same resource "collapse" into the first request and wait for its result.

Once the first request (the "Leader") receives the data from the database and updates Redis, it shares that result with all "Follower" requests. The followers then return the data without ever touching the database.

3. High-Level Design (HLD)

In a monolithic architecture, we can manage this using an in-memory Task Registry. This registry tracks active asynchronous operations.

• Registry Check: Before querying the database, the service checks if a Promise for that specific resource ID is already active.
• Subscription: If a Promise exists, the current request "awaits" that existing Promise.
• Execution: If no Promise exists, the request creates one, adds it to the registry, and proceeds to the database.
• Cleanup: Once the database returns data, the Promise is removed from the registry to allow future updates.

4. Implementation: The Gatekeeper Pattern in NestJS

Below is a practical implementation using a NestJS service. This example demonstrates how to wrap a Prisma query with a collapsing mechanism.

The Service Logic

import { Injectable, Logger } from '@nestjs/common';
import { PrismaService } from './prisma.service';
import { RedisService } from './redis.service';
 
@Injectable()
export class BookingService {
  private readonly logger = new Logger(BookingService.name);
  
  // The Task Registry: Stores ongoing database fetch promises
  private fetchPromises = new Map<string, Promise<any>>();
 
  constructor(
    private prisma: PrismaService,
    private redis: RedisService,
  ) {}
 
  async getBookingDetails(bookingId: string) {
    const cacheKey = `booking:${bookingId}`;
 
    // 1. Attempt to get from Redis
    const cachedData = await this.redis.get(cacheKey);
    if (cachedData) {
      return JSON.parse(cachedData);
    }
 
    // 2. Check if a request for this ID is already in progress
    if (this.fetchPromises.has(cacheKey)) {
      this.logger.log(`Request Collapsed for key: ${cacheKey}`);
      return this.fetchPromises.get(cacheKey);
    }
 
    // 3. I am the "Leader" request. Create the fetch promise.
    const fetchAction = this.performHydration(cacheKey, bookingId);
    
    // 4. Store the promise in the registry
    this.fetchPromises.set(cacheKey, fetchAction);
 
    return fetchAction;
  }
 
  private async performHydration(cacheKey: string, bookingId: string) {
    try {
      this.logger.log(`Database Fetch initiated for key: ${cacheKey}`);
      
      // Fetch from PostgreSQL via Prisma
      const data = await this.prisma.booking.findUnique({
        where: { id: bookingId },
        include: { service: true, provider: true },
      });
 
      if (data) {
        // Hydrate the cache (TTL: 1 hour)
        await this.redis.set(cacheKey, JSON.stringify(data), 'EX', 3600);
      }
 
      return data;
    } finally {
      // 5. Always clean up the registry so subsequent requests can trigger a fresh fetch if needed
      this.fetchPromises.delete(cacheKey);
    }
  }
}

5. Key Technical Considerations

Memory Management

The fetchPromises Map is stored in the application's RAM. In a monolith, this is highly efficient. However, you must ensure the finally block always deletes the key, otherwise, you will encounter a memory leak and "stuck" requests.

Error Handling

If the database query fails, the "Leader" promise will reject. Because all "Follower" requests are awaiting the same promise, they will all receive the same error. This is usually the desired behavior, as it prevents the database from being retried 50 times in a failing state.

Single Instance vs. Distributed

This in-memory Map approach works perfectly for a single-instance monolith. If you scale to multiple horizontal instances (e.g., using Kubernetes or PM2), Instance A will not know Instance B is fetching data. To solve this at a distributed scale, you would use a Redis Distributed Lock (Redlock) instead of an in-memory Map.

6. Conclusion

Request Collapsing is a vital optimization for any on-demand platform . By shifting the responsibility of synchronization from the database to the application layer, you create a system that is:

• Resilient: It cannot be taken down by a simple cache expiration.
• Efficient: It minimizes unnecessary I/O and network overhead.
• Predictable: Database load stays flat even as user concurrency grows.

Implementing this pattern demonstrates a transition from building functional software to building high-performance, production-grade systems.