Social & feeds
Home timeline - materialise it, then keep it honest
fan-out on write, the celebrity exception, and the hydrate hop
What this board gets wrong on purpose
The tension
Every timeline here is a denormalised copy that nothing repairs. Deletes, blocks, mutes, unfollows and gone-private accounts never travel back into the 150 million lists that already hold those ids, so the read path fetches rows in order to throw them away - which means a timeline gets SLOWER the longer an account has existed, and a page of twenty tweets can require hydrating sixty. The trim is the second half of it: past eight hundred ids the materialised timeline simply does not exist, so deep scroll falls through to a query path that is barely exercised, has no cache behind it, and is the one nobody load tests.
Requirements
Clarifying questions to ask first
- Is the timeline chronological or ranked? Ranking changes everything: a chronological timeline can be a pure merge, a ranked one needs features at read time and cannot be trimmed by time alone.
- How stale may a timeline be? Seconds, or is a missing tweet a bug?
- Do we owe read-your-own-writes? Seeing your own tweet immediately is the one staleness users notice within a second.
- What is the follower distribution? The p50 and the p99.99 are six orders of magnitude apart and they need different machinery.
- Are there private accounts, blocks and mutes? Each one turns a precomputed list into a list that must be re-checked at read time.
- Is deep scroll a product requirement, or may the timeline simply end?
Functional
- Post a tweet; it appears in the home timeline of every follower.
- Read a home timeline: the most recent tweets from the accounts you follow, ranked, paginated.
- Follow and unfollow; the effect on the timeline may be eventually consistent.
- Delete a tweet; it must stop being served.
- Blocked, muted and protected authors must not appear.
Non-functional
- Timeline read p99 under 200 ms at the API, measured server-side.
- Fan-out lag p99 under 5 seconds from post to a follower list, for ordinary accounts.
- 99.95% availability on the read path. The write path may be less available: failing to post is visible and recoverable, failing to read is the product being down.
- Eventual consistency is acceptable for other people's tweets. Read-your-own-writes is not negotiable.
- Deletes and blocks must take effect within one request, not within one TTL.
Explicitly out of scope
Search, direct messages, notifications, media upload, ads insertion, and the ranking model itself. The ranker is drawn as one box because its internals are a different board.
The rest of this board
Available on Tier Slate
This page publishes the question. The answer — 5 more written pages, an 8-step narrated walkthrough and a 5-table schema — is the board itself, and it opens in Tierslate.
- 5written pages
- 8walkthrough steps
- 5tables22 columns
5 pages behind this one
- Capacity estimation
- Storage estimation
- Availability
- How it works
- Deployment plan
More boards
Timelines, follows, fan-out - the read-heavy classics.