Schema Design for High-Volume and Unbounded Data

Model growth intentionally

A schema that works for ten records may fail at ten million if it assumes bounded data, ignores access patterns, or creates excessive contention.

Separate event history from current state

For a user activity system, keep the current profile separate from activity events:

// userActivity
{
  _id: ObjectId("65a000000000000000000060"),
  userId: ObjectId("65a000000000000000000010"),
  eventType: "course_completed",
  resourceId: "mongodb-indexes",
  occurredAt: ISODate("2026-05-01T12:00:00Z"),
  metadata: {
    durationSeconds: 1800
  }
}

Useful index for a user's recent activity:

db.userActivity.createIndex({ userId: 1, occurredAt: -1, _id: -1 });

If the application needs global analytics by event type and date, that is a different access pattern and may need another index or an analytics-oriented data pipeline. Avoid adding every conceivable index because indexes consume storage and increase write cost.

Bucket pattern

For high-volume time-series-like events, a bucket document can group a bounded number of measurements by entity and time window. Bucketing reduces document count and can improve range-oriented reads, but it requires careful handling of bucket capacity, concurrent writers, and queries. MongoDB time series collections may be preferable for true time-series workloads.

A simplified bucket document:

{
  sensorId: "sensor-17",
  bucketStart: ISODate("2026-05-01T10:00:00Z"),
  readings: [
    { at: ISODate("2026-05-01T10:00:01Z"), temperature: 21.2 },
    { at: ISODate("2026-05-01T10:00:05Z"), temperature: 21.3 }
  ],
  count: 2
}

Do not allow readings to grow forever. Use a bounded bucket window or maximum reading count.

Subset pattern

If a product has thousands of reviews but the product page needs only the latest three, store the complete reviews separately and maintain a small subset on the product document. This can reduce the cost of the common product-page read while preserving the full review collection.

Computed pattern

If a dashboard repeatedly needs an expensive aggregate, maintain a derived value or summary collection. Define when it is updated, whether stale values are acceptable, and how it can be rebuilt from the source data.

Outlier pattern

If a small number of records have extremely large arrays or relationships compared with the typical record, isolate exceptional values rather than designing every document around the worst case. For example, a post with millions of followers should not necessarily store every follower ID in one document.

Polymorphic pattern

Documents in one collection can have different optional fields when they share common query behavior. Include a discriminator such as type when it helps interpret the shape. If variants require very different indexes, validation, security, or access patterns, separate collections may be clearer.

Specifications & Requirements
  • Bucket pattern:Group bounded sets of similar measurements or events.
  • Subset pattern:Keep a small, frequently used subset near the parent while storing the full data separately.
  • Computed pattern:Persist derived results to avoid repeating expensive calculations.
  • Outlier pattern:Handle exceptional large records without burdening all typical records.
Note

Common patterns such as bucket, subset, computed, and outlier patterns solve different scaling problems. Pick a pattern only after identifying the workload it improves and the consistency costs it introduces.

Maintain a small latest-review subset while retaining full reviews
// Full review is inserted into the reviews collection separately.
// Keep only the three latest reviews in the product's display subset.
db.products.updateOne(
  { _id: productId },
  {
    $push: {
      latestReviews: {
        $each: [{
          reviewId,
          rating: 5,
          excerpt: "Comfortable and well built",
          createdAt: new Date()
        }],
        $sort: { createdAt: -1 },
        $slice: 3
      }
    }
  }
);
Key Takeaways
  • Separate unbounded event history from current entity state.
  • Bound bucket and embedded subset sizes.
  • Account for refresh strategy and consistency when storing derived values.
Knowledge Check

A product page needs the three latest reviews, but the system must retain every review. Which pattern can help?