Design a Cell-Based Architecture
A Principal prompt about blast radius and sequencing. Strong answers pick the partition key first, keep the router simple enough to trust, and lay out a migration that 60 teams can follow without a company-wide freeze.
1 · Requirements
The Prompt
A consumer platform with 120M accounts runs on one shared stack in each of three regions, and its last two major outages each took down a full region. Design the target cell-based architecture and the two year path to it for 400 services owned by 60 teams.
Functional
- Route every request to the one cell that owns its account
- Provision a new cell from a template in under a day
- Move accounts between cells online, with no user-visible downtime
- Deploy every service cell by cell in waves
Non-Functional
- A failure in one cell affects at most 2.5% of accounts
- Routing keeps working on its last known map when the control plane is down
- Cell overhead stays under 15% of today's infrastructure spend
- The migration lands in two years without a company-wide freeze
2 · Estimates
Drill This Phase First
Show the Answer and Estimates
Answer
40 cells, and 2.5% of accounts
120M divided by 3M is 40 cells. One cell is 1/40 of accounts, which is 2.5%, against 33% when one of three regional stacks fails today.
- Traffic
- 120M accounts and 40M daily actives: about 1.2M requests a second at peak across three regions
- Cells
- Capped at 3M accounts each: 40 cells, about 13 per region, so one failed cell reaches 2.5% of accounts
- Per Cell
- 1.2M divided by 40 is 30,000 requests a second at peak. Each cell is load-tested to 50,000, so it runs at 60% of proven capacity
- Cell Map
- 120M entries at about 40 bytes: about 4.8 GB, held in memory on every router node for lookups under 1 ms
- Cell Moves
- About 2 MB per account is 6 TB per full cell. At 1 GB a second, a full drain takes about 100 minutes
- Cost
- About $30,000 a month of fixed footprint per cell: $1.2M a month for 40 cells, about 12% on a $10M monthly bill
- Services
- About 250 of 400 services move into cells, 110 stay global behind async interfaces, and 40 retire
3 · API
GET /v1/route?account_id= → { cell_id, map_version } (router internal; routers cache the full map)
POST /v1/cells { region, size_class } → { cell_id, state: provisioning }
PUT /v1/cells/:cell_id/state { state: active | draining | quarantined } → 204
POST /v1/migrations { account_ids[], target_cell } → { migration_id, status: copying }
GET /v1/migrations/:id → { copied, cut_over, failed, lag_seconds }
POST /v1/deployments { service, version, waves: [canary, 1, 10%, 50%, all] } → { deployment_id }Data Model
cell_map (KV, key = account_id, replicated to every router):
account_id (PK) : string
cell_id : string
state : enum(active|moving|frozen)
map_version : int64
cells (SQL, control plane):
cell_id (PK) : string
region : string
state : enum(provisioning|active|draining|quarantined)
capacity_accounts: int 3,000,000
assigned_accounts: int
stack_version : string
migrations (SQL, control plane):
migration_id (PK): uuid
source_cell : string
target_cell : string
account_count : int
phase : enum(copying|catching_up|cut_over|done|rolled_back)4 · High-Level Design
5 · Deep Dives
Choosing the Partition Key
Partition by account, since nearly every request carries one and most data belongs to exactly one. Interactions between accounts, such as messages or shared documents, write to the sender's cell and reach the recipient's cell through an outbox and Kafka, which keeps each cell independent. The key and that async contract are the one-way doors: changing either later means moving every byte again, so settle them with the five largest teams before anyone migrates.
A Router Simple Enough to Trust
The router is the one component every cell shares, so it does a map lookup and a forward and nothing else. It holds the full 4.8 GB map in memory, takes versioned updates from the control plane, and keeps serving the last good map if the control plane fails. Deploy the router itself in shards and waves, and change it far less often than anything behind it.
Cell Zero and the Migration Path
Phase one puts the router in front of today's stack and names that stack cell zero, so every request takes the new path with no behavior change. Phase two sends new signups to cells 1 through 5 to prove the template. Phase three makes the 90 request-path services cell-aware through a paved-road template, then the remaining 160. Phase four drains cell zero 3M accounts at a time: copy through change data capture, catch up, and freeze each account's writes for a few seconds at cutover.
Ownership, Standards, and Cost
A cell platform team owns the router, placement, provisioning, and the wave deploy pipeline. Service teams own their code inside every cell and get per-cell dashboards and SLOs. Standardize the cell template, the outbox contract, and deploy waves; leave languages and internal service design to teams. Buy managed databases and queues per cell, and build the router and placement, since those encode the partition key. The 12% overhead is about $14M a year, which a single regional outage at this scale can exceed.
6 · Wrap Up
The Short Version
Partition by account into fixed-size cells behind a thin, statically stable router, and wrap today's stack as cell zero so the migration starts with no behavior change. The partition key and the cross-cell contract are the one-way doors, so settle them before any team moves.