Design Tenant Isolation for B2B SaaS
Every product team builds on the tenancy model, and every enterprise contract leans on it. The interview turns on finding each resource one tenant can exhaust, matching isolation to what customers pay for, and moving a 12 TB tenant while it keeps working.
1 · Requirements
The Prompt
A B2B SaaS platform serves 20,000 tenants on shared infrastructure, and its largest tenant drives 15% of traffic and slows everyone down with Monday batch jobs. Design tenant isolation: noisy-neighbor protection, per-tenant rate limits and quotas, data isolation tiers, and a zero-downtime move of that tenant to dedicated capacity.
Functional
- Rate limit and meter every tenant by plan, with clear 429 responses
- Keep one tenant's batch work from slowing interactive requests for others
- Offer pooled, bridge, and silo data isolation tiers
- Move a tenant between cells or tiers while it stays online
Non-Functional
- No tenant can push another tenant's p99 latency past its SLO
- A query can never return another tenant's rows
- A bad deploy or an overloaded tenant affects at most one cell
- A tenant cutover pauses writes for under 10 seconds
2 · Estimates
Drill This Phase First
Show the Answer and Estimates
Answer
About 17 hours
12 TB is 12,000,000 MB. Divided by 200 MB a second, that is 60,000 seconds, or about 16.7 hours, so changes must stream through CDC during the copy.
- Traffic
- 120,000 requests a second at peak; the largest tenant drives 18,000 of them and the top 50 drive 45%
- Data
- 256 TB across 64 Postgres shards, about 4 TB each; the largest tenant holds 12 TB across 3 shards
- Tiers
- 19,500 pooled tenants in 20 cells of about 975 each, 490 bridge tenants, and 10 silo tenants
- Rate Limits
- 120,000 checks a second from local token buckets on each gateway, reconciled through Redis every second
- Directory
- 20,000 placements at about 100 bytes: roughly 2 MB, cached on every router
- Migration
- 12 TB at 200 MB a second takes about 17 hours, while CDC streams about 3,600 writes a second
- Cutover
- Fence writes, drain CDC lag, and flip the directory in under 10 seconds
3 · API
Every request: the gateway reads tenant_id from the auth token, sets X-Tenant-Id, and strips any client-sent value
Over a limit: 429 { limit: rps|concurrency|storage, retry_after }
GET /internal/tenants/:id/placement → { tier, cell, shards[], limits, version }
PUT /internal/tenants/:id/limits { rps, burst, max_concurrency, batch_share, storage_quota_gb } → 204
POST /internal/migrations { tenant_id, target_cell } → { migration_id, phase: copying }
POST /internal/migrations/:id/cutover → 202 (fences writes, drains CDC lag, flips the directory)
POST /internal/migrations/:id/rollback → 202 (reverse replication keeps the source current)Data Model
tenant_directory (SQL, replicated, cached on every router):
tenant_id (PK) : uuid
tier : enum(pooled|bridge|silo)
cell_id : string
shard_ids : list<int>
kms_key_id : string? bridge and silo tiers
write_fenced : bool true only during cutover
version : bigint bumps on every move
tenant_limits (SQL):
tenant_id (PK) : uuid
plan : string
rps, burst : int
max_concurrency : int per service
batch_share : float weight in the batch scheduler
storage_quota_gb : int
usage_daily (columnar):
tenant_id, date, requests, cpu_ms, db_ms, storage_gb
migrations (SQL):
migration_id (PK) : uuid
tenant_id : uuid
source_cell : string
target_cell : string
phase : enum(copying|streaming|verifying|fenced|cut_over|rolled_back)
cdc_lsn : string
started_at : timestamp4 · High-Level Design
5 · Deep Dives
Noisy Neighbors on Every Shared Resource
List what one tenant can exhaust: gateway capacity, service threads, database connections, shard CPU, and batch workers. Each gets a per-tenant limit: a token bucket at the gateway, a concurrency cap per service, a connection cap and statement timeout in the database pooler, and a weighted fair share in the batch scheduler. Interactive and batch traffic run in separate pools, so the largest tenant's Monday jobs run at their own share and leave interactive capacity alone. Per-tenant usage metrics show who is close to a limit before anyone feels it.
Data Isolation Tiers
Pooled tenants share tables, with tenant_id on every row, a data access library that adds the tenant predicate, and row-level security as the backstop. Bridge tenants get their own database on shared clusters and their own encryption key, which satisfies most enterprise security reviews. Silo tenants get a dedicated cell, for contracts that require it or traffic that warrants it, priced to match. All three tiers run the same code and the same deploys, and only the placement in the tenant directory differs.
Moving the Largest Tenant
Provision a silo cell, then bulk copy the tenant's rows from a consistent snapshot of its 3 shards, which takes about 17 hours. CDC from the snapshot's log position streams every change during and after the copy. Verify with per-table row counts and checksums. At cutover, the router fences the tenant's writes, waits for CDC lag to reach zero, bumps the directory version to the new cell, and resumes, all in under 10 seconds. Reverse replication keeps the old shards current for a week, so rollback is one more directory flip.
Tenant Context Every Team Inherits
Product teams get isolation by default from the platform. The gateway sets the tenant from the auth token, and the shared RPC and data libraries carry it through every call, queue message, and cache key. A CI check fails any query that lacks a tenant predicate, and a synthetic canary tenant probes for cross-tenant reads all day to prove that row-level security holds. Deploys roll out cell by cell, so a bad release or a runaway tenant reaches about 5% of pooled customers at most.
6 · Wrap Up
The Short Version
Put a per-tenant limit on every shared resource, from gateway tokens to database connections and batch slots, and bound the blast radius with cells. Offer pooled, bridge, and silo tiers by contract, and move a tenant with a bulk copy, CDC, verification, and a reversible cutover.