Feature flags - a config change that ships like code

Why this design

Every team that ships continuously ends up with feature flags, and almost every team builds the first version wrong: a database table and an HTTP endpoint that each service calls to ask "is this on?". That works until the flag check is the most-called endpoint in the company. The real design moves the rules to the caller - a compiled ruleset pushed into every process and evaluated in memory - and the interesting questions become the ones about distribution: how fast a change arrives, what a process does when it has not heard from anyone, and how you know.

What you will learn

• Why 16.8 million evaluations a second have to be local, and why the ruleset that makes that possible is about 140 KB compressed.
• Why a flag change deserves the lifecycle of a deploy - versioned, audited, compiled, rolled back - and why it gets less review than one.
• How a streaming relay plus an immutable snapshot store gets a kill switch to 6,000 pods in about a second, and where the long tail of that second lives.
• Why deterministic bucketing (a hash of flag key and user id) is what makes a 10% rollout the same 10% on every pod and every request.
• What a stale ruleset does during an incident, and why "how old is your copy?" is the question this board cannot answer.

How to read this board

1. Press Play: the walkthrough follows one kill-switch change from the console, through the compiler and relay, into a pod, and out the other side as an exposure event.
2. AUTHORING (left) is the slow, human half; DISTRIBUTION (middle) is the copy; EVALUATION (right) is the request path with no network in it.
3. 1 · Requirements, then 2 · Capacity and 3 · Storage for where 16.8M/s and the 140 KB snapshot come from.
4. 4 · Availability - what each box failing does, and why the flag service may go down without anyone noticing.
5. 5 · How it works in prose, then 6 · Deployment - including how a flag system is itself rolled out without flags.
6. Finally the DATA MODEL, then argue with the weakness: the red ruleset in every pod has no age and no expiry.

Ask these before drawing anything

• Server-side only, or browsers and mobile apps too? A client-side SDK cannot be handed the whole ruleset - it would leak every targeting rule and every unreleased feature name - so it needs a server that evaluates for it. That is a different system.
• How fast must a change arrive? "Within a second" for a kill switch is a streaming design; "within a minute" is polling, and polling is far simpler.
• What does targeting look like - percentages only, or user attributes, segments, plans, regions? Segments of a million user ids change the snapshot size by orders of magnitude.
• Are flags used for experiments? Then evaluation must emit exposure events, and the assignment must be deterministic and sticky.
• Who may change a production flag, and does that need approval? A flag flip has taken down more production systems than most deploys.
• What should a process do if it has never received a ruleset? Code default, last-known file, or refuse to start - each is wrong somewhere.

Functional

• Create flags with typed variants (boolean, string, number, JSON) and a default per environment.
• Target by rules over request context: attribute matches, segment membership, and percentage rollouts by a stable hash.
• Evaluate in-process with no network call on the request path.
• Propagate a change to every connected process, and record who changed what, when and why.
• Emit exposure events for experiment flags, deduplicated.
• Roll back any flag to any previous version in one action.

Non-functional

• Evaluation p99 under 5 µs, zero network calls, zero allocation on the hot path.
• Change propagation to connected SDKs: p50 300 ms, p99 under 2 s.
• The request path must keep working with the entire flag platform down - evaluation degrades to the last ruleset, never to an error.
• Every production change is attributable and reversible.
• Same user, same flag, same answer on every pod, unless the rules changed.

Out of scope

• Client-side (browser/mobile) evaluation and its proxy.
• Experiment statistics - this board stops at the exposure event landing in the warehouse.
• Long-lived configuration (timeouts, limits). It is tempting to put it in the flag system, and it turns a release tool into a config database nobody is allowed to clean up.