Messaging & realtime
Encrypted delivery - one tick, two ticks, blue
sent, delivered, read - and what a server that cannot read the message is allowed to claim
What this board gets wrong on purpose
The tension
The ticks are claims the server cannot check. Tick two and tick three are produced by the recipient's device and vouched for by nobody the server can audit, so a patched client shows blue ticks it never earned and a battery-saving OS shows none it did - and the server, holding only ciphertext, has no way to tell the two apart. The offline mailbox is worse: it is undecryptable bytes with no expiry policy drawn, so it can never be deduped, compacted or re-encrypted, and the retention period is a guess worth four orders of magnitude of storage. And because the server never held a key, a device added tomorrow gets no history at all - the fix for that, a sender-side history transfer, is not on this board.
Requirements
Ask these before drawing anything
Every one of these questions moves a box on the diagram. The first one moves all of them.
- Is the server allowed to read a message body? If yes, half this board disappears and search, backup and web clients become easy. If no, fan-out moves onto the sender's phone and stays there.
- One device per person or several? Multi-device is what turns a message into N ciphertexts and makes "history on a new device" a separate product.
- What ordering do we promise, and across what? Per conversation is achievable. Per sender across devices is not, without a sequencer the sender trusts.
- Can a user switch read receipts off? If so the third tick is optional and the UI must be honest about never getting it.
- How long do we hold an undelivered message - 30 days, or forever? This single sentence is worth four orders of magnitude of storage. See the storage page.
- How large may a group be? 1,024 members is a different fan-out problem from a pair, and the sender's uplink pays for it.
- Do we owe anyone server-side search, backup or a browser client? All three need plaintext, so all three become client features or become a compromise.
Functional
- Send a message from one device to every device the recipient owns.
- Deliver immediately to a device that holds a connection; hold it durably for one that does not.
- Report three states to the sender: accepted by the server, received by a device, read by a human.
- Groups up to 1,024 members, with the same guarantees as a 1:1 chat.
- Carry media without putting bytes on the message path.
- Let a new device register keys and begin receiving new messages.
Non-functional
- The server must never hold a key that can open a message body. This is a constraint on the design, not a feature of it.
- p99 send-accepted under 300 ms on a mobile network.
- p50 delivery to a connected device under 500 ms.
- 99.95% availability for accepting a send; 99.9% for delivering to a device that is connected.
- An undelivered message survives the loss of an availability zone.
- At-least-once delivery; the client deduplicates on message id.
Out of scope, and why
- Server-side search and server-side backup: both need plaintext we do not have.
- Voice and video: a different transport with different failure modes.
- History for a newly added device: needs a sender-side transfer, which is a product on its own.
The rest of this board
Available on Tier Slate
This page publishes the question. The answer — 5 more written pages, an 8-step narrated walkthrough and a 5-table schema — is the board itself, and it opens in Tierslate.
- 5written pages
- 8walkthrough steps
- 5tables33 columns
5 pages behind this one
- Capacity estimation
- Storage estimation
- Availability
- How it works
- Deployment plan
More boards
Delivery, presence, ordering, and who is typing.