Design News Feed
A social platform's news feed is the primary stream of content that users see upon opening the app — a personalized timeline of posts from accounts they follow. The core engineering challenge lies in quickly assembling this feed for hundreds of millions of users while keeping it fresh and accurate.
The design tension centers on the trade-off between write amplification and read cost. Pre‑computing feeds makes reads extremely fast, but a single post from a user with millions of followers can trigger an enormous write fan‑out that stalls the entire pipeline.
Clarifying Questions
Q: Is the scope limited to a home feed that shows posts from followed users in reverse chronological order, or does it also encompass discovery feeds, trending topics, and advertisements?
A: Only the home feed, ordered purely by recency. No algorithmic ranking, no discovery content, no ads.
Insight: Restricting the scope to a recency‑sorted social‑graph feed transforms the problem into a fan‑out and pre‑materialization exercise, not a ranking or recommendation pipeline. This justifies a write‑time push strategy for the majority of users.
Q: How quickly must a post from an ordinary user appear in their followers' feeds? Is sub‑second delivery required, or is a delay of a few seconds acceptable?
A: A few seconds of lag is fine for normal users. Sub‑second propagation is not necessary.
Insight: Accepting a few seconds of delay allows asynchronous fan‑out through a queue after the post write is acknowledged, keeping post write latency bounded and decoupled from follower count for typical users.
Q: What happens when a user with millions of followers publishes a post? Could that single action slow down the entire system or degrade freshness for everyone else?
A: No. A post from a highly popular author must not block the write path or hurt feed freshness for other users. The rest of the system should continue to operate normally.
Insight: A hybrid fan‑out approach is required: write‑time materialization applies below a follower‑count threshold, while read‑time assembly is used for the rest. This prevents a single hot‑author post from saturating the fan‑out pipeline or delaying delivery to unrelated users.
Q: If a user's pre‑built feed is lost due to a technical failure, may we show an empty feed, or must we be able to reconstruct it?
A: The feed must be reconstructable. Displaying an empty feed because of a failure is not acceptable.
Insight: The feed store is a performance cache, not the authoritative source of truth. The post store and follow graph must remain independently queryable so that a repair path can rebuild any user's feed on demand after a cache loss or outage.
Q: When someone unfollows another user, do we need to immediately remove all of that user's existing posts from the feed, including posts that were already displayed?
A: Propagating the unfollow within a reasonable window is sufficient. We do not need to retroactively delete posts that were already shown before the unfollow occurred.
Insight: Not requiring retroactive removal simplifies the repair job to a background suppression and forward‑only update, rather than a bulk rewrite of existing materialized feed entries. Follow changes trigger an incremental cache update, not a full feed rebuild.
Functional Requirements
- Users can publish a post containing text and optional media references.
- Users can follow and unfollow other users.
- Users can view a personalized home feed of posts from accounts they follow, sorted by recency.
- Users can paginate through their feed using stable cursor‑based ordering that remains consistent across pages.
- The feed reflects follow and unfollow changes within a reasonable consistency window.
Non‑functional Requirements
- Feed read latency: p95 under 200 ms.
- Post visibility in followers' feeds within a few seconds for normal authors (eventual consistency).
- Support 500M total users, 100M daily active users.
- Handle 300k peak concurrent feed reads per second.
- Handle 5k peak post writes per second.
- Feed reads remain available during partial infrastructure failures.
- Users with millions of followers must not block the write path or degrade feed freshness for other users.
Back‑of‑the‑envelope Estimation
| Metric | Value |
|---|---|
| Total users | 500M |
| Daily active users | 100M |
| Peak feed reads per second | 300k |
| Peak post writes per second | 5k |
| Avg followers per user | ~200 |
| Avg feed page size | 20 posts |
| Feed cache entry (post ID + score) | ~50 bytes |
| Feed cache per user (last 500 posts) | ~25 KB |
| Total feed cache (100M active users) | ~2.5 TB |
| Fan‑out writes per normal post (200 followers) | 200 cache inserts |
| Fan‑out writes for hot author (10M followers) | 10M cache inserts |
| Daily post volume (5k writes/s avg over 12 active hrs) | ~216M posts/day |
| Post storage per day (~1 KB avg per post) | ~216 GB/day |
The read‑to‑write ratio is roughly 60:1 (300k reads vs. 5k writes), indicating a heavily read‑dominant workload. This justifies pre‑materializing feeds at write time for most users. The notable exception is the hot‑author case: a single post by such a user can generate more write work than thousands of normal posts, so the fan‑out strategy must treat these cases differently.
High-level system architecture and data flow for the news feed system, showing the read path, write path, and hybrid fan-out strategy.
[!NOTE] The 2.5 TB feed cache estimate assumes only active users have materialized feeds. Inactive users’ feeds can be rebuilt on demand from the post table and social graph, which keeps the cache footprint bounded.
Below the line (out of scope)
- Full ML‑based ranking and personalization pipeline.
- Media upload, transcoding, and CDN delivery.
- Ads, sponsored content, and engagement features (likes, comments, shares).
- Real‑time push notifications for new posts.
- Multi‑region deployment and geo‑routing.
- Post editing and deletion after publish.