Social & feeds
The professional graph - degrees, and why it is not a join
in-memory adjacency, bidirectional BFS, and an offline people-you-may-know
What this board gets wrong on purpose
The tension
The entire architecture rests on the whole edge set living in RAM, and there is no partial-start path on this board: a graph node that restarts loads 1.2 TB from the durable store before it can answer anything, which is tens of minutes per replica, so the real recovery time of the system is a rolling restart measured in hours and nobody has ever timed it under load. The second hole is staleness on both sides of the same fact: the distance cache is expired by TTL rather than invalidated by the edge write, so a connection you just accepted still reads as 2nd degree for up to a minute, while PYMK is recomputed nightly, so accepting five connections today changes none of your suggestions until tomorrow - and the two disagree with each other in the meantime, on the same screen.
Requirements
Clarifying questions to ask first
- How many degrees do we need? Stopping at 3 makes the problem bounded. Unbounded shortest path over a billion nodes is a different, much worse system.
- Are connections symmetric? On this product yes - a connection is mutual and requires acceptance - which halves the storage and makes the traversal undirected. Follows, if we also have them, are a separate directed edge and should not be mixed into the same adjacency.
- Is the degree badge required on every row of every list? If yes, the query rate is set by list rendering, not by profile views, and it is two orders of magnitude higher.
- May PYMK be a day stale? If yes it can be offline, and the entire online system gets simpler.
- What is the maximum connection count? A hard cap (30,000) turns "unbounded fan-out" into "a large but bounded array" and is the single most useful constraint in the design.
- Do we owe privacy controls on who can see connections? A filter on a traversal result is cheap; a filter inside a traversal is not.
Functional
- Send, accept, ignore and withdraw a connection invitation.
- Show the degree (1st, 2nd, 3rd, out of network) between the viewer and any member, everywhere a member is displayed.
- List a member's connections, paginated.
- Show mutual connections with another member.
- Suggest people you may know, ranked.
Non-functional
- Degree lookup p99 under 10 ms, because it happens 10-50 times per rendered page and sits inside somebody else's latency budget.
- Mutual-connection count p99 under 50 ms.
- 99.95% availability on degree lookups. A missing badge degrades to "out of network", it does not fail the page.
- Connection writes may be eventually consistent for other viewers, but the two people involved must see the new connection immediately.
- PYMK may be up to 24 hours stale.
Explicitly out of scope
Messaging, the feed, job matching, search relevance, and the PYMK ranking model itself. The ranker is one box.
The rest of this board
Available on Tier Slate
This page publishes the question. The answer — 5 more written pages, an 8-step narrated walkthrough and a 5-table schema — is the board itself, and it opens in Tierslate.
- 5written pages
- 8walkthrough steps
- 5tables27 columns
5 pages behind this one
- Capacity estimation
- Storage estimation
- Availability
- How it works
- Deployment plan
More boards
Timelines, follows, fan-out - the read-heavy classics.