Design Company-Wide Identity and Access
Identity touches every request, every employee, and every service, so this design is mostly about standards and migration order. A Principal answer separates the three kinds of identity, decides the one-way doors up front, and moves off three legacy systems without forcing every team to change at once.
1 · Requirements
The Prompt
Design identity and access for 25,000 employees, 6,000 services, and 90M customers. Three legacy auth systems issue incompatible sessions today, and every service checks permissions its own way. Cover SSO, service identity, authorization policy, and the migration off all three legacy systems.
Functional
- Single sign-on for employees across every internal tool, driven by HR records
- Customer login with passkeys, MFA, and one account across products
- A cryptographic identity for every workload, with mutual TLS between services
- Authorization from centrally reviewed policy, including per-object sharing
- Move every account and service off three legacy auth systems
Non-Functional
- Token validation needs no network call, and authorization decisions take under 1 ms
- A departed employee loses all access within 15 minutes
- Login stays available when a region fails
- Active customers migrate without a forced password reset
2 · Estimates
Drill This Phase First
Show the Answer and Estimates
Answer
About 7,000 a second
30M times 8 is 240M issuances a day. Divided by 86,400 seconds that is about 2,800 a second, and 2.5x is about 7,000. Validation adds no load, since services check signatures locally.
- Customers
- 90M accounts, 60M monthly active, and 30M daily active
- Legacy Accounts
- 70M in system A and 28M in system B, with 8M sharing an email, for 90M unique customers
- Tokens
- 15 minute access tokens refreshed about 8 times a day: 240M a day, about 2,800 a second and 7,000 at peak
- Service Calls
- About 4M internal requests a second, so authorization runs locally in under 1 ms
- Workloads
- About 120,000 workloads on 24 hour certificates: about 3 issuances a second, 200 a second during a mass restart
- Sharing
- About 50 relationship tuples per customer: 4.5B tuples, roughly 450 GB, sharded by object
- Offboarding
- Every session and grant for a departed employee revoked within 15 minutes
- Build or Buy
- A vendor at about $0.015 per monthly active user is about $10.8M a year for 60M users, against about $4M for a 15-engineer team
3 · API
POST /oauth2/token { grant_type: refresh_token | client_credentials } → { access_token, expires_in: 900 }
POST /oauth2/token { grant_type: token-exchange, subject_token: <legacy session> } → { access_token } (migration bridge)
GET /.well-known/jwks.json → { keys[] } (cached by every gateway and service)
POST /v1/authz/check { principal, action, resource, consistency_token? } → { allow, policy_version } (local sidecar)
POST /v1/relationships { object, relation, subject } → { consistency_token }
POST /v1/workforce/offboard { employee_id } → { revoked_sessions, revoked_grants } (triggered by SCIM)Data Model
identities (SQL, sharded by identity_id):
identity_id (PK) : uuid one id per person or workload
kind : enum(customer|employee|workload)
legacy_ids : map<system, string> A, B, and C
status : enum(active|suspended|deleted)
credentials (SQL):
identity_id : uuid
type : enum(password|passkey|totp|federated)
hash_scheme : enum(legacy_a|legacy_b|argon2id)
updated_at : timestamp
relationships (KV, key = object_id, sharded):
object : string doc:123
relation : string viewer | editor | owner
subject : string user:42 or group:eng#member
written_at : hybrid logical clock timestamp
policy_bundles (object storage, versioned):
version : int
rules : compiled policy
signature : bytes4 · High-Level Design
5 · Deep Dives
Three Planes, One Standard
Workforce, customer, and workload identity have different users, risks, and vendors, so each gets its own plane. They share one token format, one identifier scheme, and one policy language, and those three are the one-way doors. Buy the workforce IdP, since employee SSO is a commodity. Build customer identity: a vendor at about $0.015 per monthly active user costs about $10.8M a year at 60M users, against about $4M for a 15-engineer team that also owns the login experience.
Service Identity Without Shared Secrets
The deploy platform attests each workload, and the workload CA issues a 24 hour certificate tied to the service name, rotated at 12 hours. The mesh terminates mutual TLS, so teams get service identity with no code change, and long-lived API keys between services retire. A mass restart of 120,000 workloads peaks near 200 issuances a second, which sizes the CA.
Central Policy, Local Decisions
Policies live in one repository, go through review like code, and compile into signed bundles pushed to a sidecar or library beside every service. At 4M internal calls a second, a remote check per call would add latency and a new single point of failure, so decisions stay local and take under 1 ms. Per-object sharing reads a relationship store; each write returns a consistency token, so a check right after a revoke never sees the old grant.
Retiring Three Legacy Systems
Move the workforce first: it is the smallest population, the highest risk, and a bought IdP drops in through SAML. Service identity follows through the mesh, run by the platform team. Customers migrate on login: verify against the legacy hash, rehash with Argon2id, and link the 8M accounts that share an email once both are verified. Meanwhile the token exchange bridge lets every service accept only new tokens. After 12 months, inactive accounts reset on next login and each legacy system goes read-only, then off.
6 · Wrap Up
The Short Version
Run three identity planes for workforce, customers, and workloads on one token standard and one policy language, with decisions made locally from centrally reviewed policy. A token exchange bridge lets services adopt the new tokens before the legacy login systems retire.