Link previews - fetching a stranger's page safely

Why this design

Every chat and social product turns a pasted URL into a card with a title, a description and a picture. It looks like a scraping exercise and is really a security boundary: the feature lets any user aim your servers at any address, and the most valuable address on a cloud network - the instance metadata endpoint - answers anyone who can reach it with credentials. Interviewers like it because a candidate who starts with caching has walked past the actual question.

What you will learn

• What server-side request forgery is, concretely: which addresses must never be fetched and why "block 127.0.0.1" is not a defence.
• Why the check and the connection must use one pinned IP, and why every redirect hop is checked again.
• Network isolation as the primary control: a fetcher subnet that can reach nothing, behind an egress proxy that refuses private ranges.
• Why a fetch has hard caps - 5 seconds, 1 MB, 5 redirects, text/html only - and what each one stops.
• Caching by normalised URL, why the recipient must never contact the site, and what a URL-keyed cache gets wrong.

How to read this board

1. Press Play: the walkthrough follows one pasted link through the cache, the guard and the fetch, then shows the two requests that must be refused.
2. THE PASTE (left) is the product; THE FETCH (middle) is a box that can reach only the public internet; THE INTERNET (right) holds the site and the two things that must never be reached.
3. 1 · Requirements, 2 · Capacity and 3 · Storage - including why thumbnails are 92% of the bytes.
4. 4 · Availability - why a preview is the most optional thing in the product, and must be treated that way.
5. 5 · How it works in prose, then 6 · Deployment - including how to test that the guard actually guards.
6. Finally the DATA MODEL, then argue with the weakness: the board trusts the page's own description of itself, and the first unfurl wins for a day.

Ask these before drawing anything

• Unfurl on paste or on send? On paste feels instant and leaks to the site that someone is composing; on send is private and the card arrives a second after the message.
• Does the sender see and approve the card before sending? If yes, the sender can remove a misleading preview; if no, the card is the site's statement in the sender's voice.
• Do we unfurl links in private messages, or only public channels? End-to-end encrypted products cannot unfurl server-side at all - the sender's device has to.
• Should we fetch images, or only text? Images are 92% of the bytes and every one is an untrusted file going through a decoder.
• Are there first-party providers (video sites, maps) with oEmbed endpoints we should prefer over scraping?
• Can a domain owner opt out, and can our trust-and-safety team block a domain from unfurling within minutes?

Functional

• Given a URL, return a card: canonical URL, title, description, site name, thumbnail.
• Never fetch a private, loopback, link-local or internal address, including via redirects or DNS changes.
• Cap every fetch: 5 s total, 1 MB read, 5 redirects, text/html only for the page.
• Proxy and re-encode thumbnails; the recipient never contacts the original site.
• Cache cards by normalised URL; negative-cache failures.
• Let senders remove a card and let moderators block a domain.

Non-functional

• Cache hit: p95 under 50 ms. Miss: card visible within 3 s at p95, or the message goes without one.
• Zero successful requests from the fetcher to any non-public address - measured at the egress proxy, not asserted in code.
• A preview failure never delays or fails a message.
• A malicious page or image must not crash, hang or exhaust a fetcher worker.

Out of scope

• Client-side unfurling for end-to-end encrypted chats.
• Malicious-URL classification (safe browsing lookups) - a separate check the card could carry a warning from.
• Full-page archiving or screenshots.