Data & analytics
View counter - counting a million clicks without a million writes
Why a counter on a hot row is batched in memory, and how much of the count you may lose on a crash.
What this board gets wrong on purpose
The tension
Views are summed in each API node's memory and flushed every 5 seconds, so a node that crashes or is killed during a deploy takes up to 5 seconds of its views with it - about 3,500 views per node at peak, 5,000 for a single viral video - and nothing replays them, because the raw event stream is fire-and-forget too and is never reconciled against the counter. The dedupe window lives in one Redis; if it is down the API either counts every refresh or drops every view, and the board has not decided which.
Practise this as an interview
Take View counter - counting a million clicks without a million writes as a timed system design interview on a blank board - on your own, with the requirements to check yourself against, or with a candidate you invite.
Start here
Why this design
Every video, post and article shows a number under it. Writing that number looks like one UPDATE per view, and the interview question is what goes wrong when one item gets 50,000 views a second. The answer is not "the database is too slow" - it is that every increment of the same row waits for the same row lock, so the hottest item is limited to a few thousand increments a second no matter how big the database is. The fix is to stop writing per view, and to be honest about what that costs.
What you will learn
- Why a hot row serialises writes, and why adding database replicas does nothing for it.
- How in-memory batching turns 50,000 increments a second on one row into 10.
- Exactly how many views a crash can lose, computed from the flush window.
- Why dedupe (one view per viewer per 30 seconds) is a separate, cheap store.
- Why the displayed counter and the analytics numbers are two different pipelines.
How to read this board
- Top: COUNT A VIEW. Viewer, player, load balancer, the View API hex, the dedupe store and the red in-memory batcher.
- Right: STORES. The counts table the batcher flushes into, and the count cache the read path uses.
- Bottom: READ THE COUNT. The page asks for the number; it comes from cache, a few seconds stale.
- Bottom right: ANALYTICS. Raw view events, a rollup job, and daily numbers per video.
- Press play, then read pages 1 to 6, then the tables. The conversation to have is the red box.
1 · Requirements
Functional
- Count a view when a video starts playing (or after 3 seconds of play - a product rule, decide it once).
- Count at most one view per viewer per video per 30 seconds, so refresh-spamming does not inflate counts.
- Show the view count on the video page and in listings.
- Daily views per video for the creator dashboard.
Non-functional
- Recording a view must never slow playback: the beacon answers in under 10 ms and the player does not wait for it.
- The displayed count may lag reality by up to ~10 seconds.
- The displayed count may undercount by a small, bounded amount after a crash. It must never overcount.
- Handle 50,000 views a second on a single video without degrading other videos.
- If views are ever billable (paying creators per view), this board is not enough - see page 5.
Out of scope
- Bot and fraud filtering beyond the dedupe window.
- Watch time, likes and other counters - the same design repeated.
The rest of this board
Available on Tier Slate
This page publishes the question. The answer — 5 more written pages, an 8-step narrated walkthrough and a 4-table schema — is the board itself, and it opens in Tierslate.
- 5written pages
- 8walkthrough steps
- 4tables16 columns
5 pages behind this one
- Capacity estimation
- Storage estimation
- Availability
- How it works
- Deployment plan
More boards
Pipelines, metrics and the lag between event and answer.
- Ad serving - an auction inside a page load
- Analytics pipeline - the lag between an event and the answer
- Log aggregation - the index is bigger than the data
- Metrics and monitoring - cardinality is the thing that kills you
- Recommendation engine - retrieve a thousand, rank twenty
- Leaderboard - a sorted set and the rank nobody stores
- Search autocomplete - an answer before the word is finished
- Uptime monitor and status page - checking from outside
- A/B testing platform - assignment is the easy half
- ETL and the data warehouse - yesterday, correctly
- Full-text search - the inverted index and the shard that is slow