System Design Handbook

Routes by IP/port. Faster, less overhead. Good for simple TCP traffic. Cannot inspect HTTP headers for smart routing.

Routes by URL path, headers, cookies. Slower per-request but enables content-based routing, SSL termination, rate limiting.

Returns multiple IPs for one hostname. Simple but no health awareness — clients may hit dead servers until DNS TTL expires.

ACID transactions, joins, schema enforcement. Best for financial data, user accounts, inventory where consistency matters.

Flexible schema, fast reads on primary key. Best for content catalogs, user profiles, IoT telemetry. Trade-off: no JOINs, weaker consistency.

In-memory, microsecond latency. Best for caching, session storage, real-time leaderboards. Data must fit in RAM.

Append-only log, persists to disk. Best for event streaming, log aggregation, clickstreams. Replays possible. Higher throughput but higher latency.

Smart broker, routing keys, exchanges. Best for task queues, RPC, workloads needing complex routing. Lower throughput than Kafka.

Fully managed, auto-scaling. SQS for simple queues, PubSub for pub/sub fan-out. No broker to manage. Vendor lock-in consideration.

Store blobs (images, videos, backups). 11 nines durability, cheap. Eventual consistency (read-after-write for PUT of new objects only).

Raw disk volumes attached to a VM. Low latency, persists independently. Used for databases, file systems. Limited to single-instance mount.

Shared NFS across instances. POSIX permissions. Good for shared file systems, media processing workflows. Higher latency than block.

Sacrifice availability during partitions. All nodes return same data or error. Examples: Zookeeper, etcd, HBase.

Sacrifice consistency during partitions. Nodes accept writes & return stale reads. Examples: Cassandra, DynamoDB, Riak.

Sacrifice partition tolerance. Cannot survive network splits. Examples: Single-node RDBMS.

Maps both nodes and keys onto a hash ring. Each key is assigned to the next clockwise node. Adding/removing a node only reshuffles keys on its immediate neighbors — not the whole ring.

Each physical node is represented by N virtual nodes on the ring. Prevents hot spots — a heavy key range is spread across physical nodes. Trade-off: more metadata to store.

Decompose app into small, independent services. Each owns its data, communicates via API. Scales independently, enables polyglot tech stacks. Trade-off: distributed complexity, network latency, data consistency challenges.

Services communicate via events (Kafka, EventBridge). Decoupled producers/consumers. Excellent for async workflows, audit trails. Can be hard to debug (event flow across many services).

Separate read models from write models. Reads use optimized views (denormalized, cached), writes go through domain logic. Enables different scaling for reads vs writes. Adds eventual consistency complexity.

Chain of local transactions, each publishing events for the next step. On failure, execute compensating transactions. Choreography: each service listens and acts. Orchestration: central coordinator manages flow.

Wrapper around a remote call. After N failures, trips to OPEN state (fast-fail). After timeout, goes HALF-OPEN to test recovery. Prevents cascading failures, gives downstream time to recover.

Isolate resources (thread pools, connections) per service/endpoint. A failure in one bulkhead can't consume resources from another. Ship compartments analogy — one flooded section doesn't sink the whole ship.

Gradually replace a monolithic system by routing specific functionality to new microservices. Old and new coexist. Eventually the monolith is "strangled" — all traffic goes to new services.

Store all state changes as an append-only event log. Current state = replay all events. Enables audit trail, time travel, full rebuild. Storage grows unbounded — compaction/snapshots needed.

One primary (writes), N replicas (reads). Async: possibly stale reads. Sync: slower writes but consistent reads. Handles read scaling well, writer is bottleneck.

Multiple nodes accept writes. Each replicates to others. Used in multi-DC deployments. Conflict resolution needed (last-write-wins, CRDTs, custom merge).

All nodes accept reads/writes (Cassandra). Quorum: R + W > N for consistency. Tunable: specify R and W per operation. Highest availability, lowest consistency.

Hash shard key → look up range. Even distribution, but range queries hit all shards. Re-sharding requires moving data (consistent hashing helps).

Data divided by key ranges (A-M, N-Z). Good for range scans, but hot spots if keys are not uniformly distributed (timestamps).

Readiness probe (is service ready for traffic?) + Liveness probe (is service alive?). LB auto-detaches unhealthy instances.

When a dependency fails, degrade the feature rather than show an error. E.g., show cached recommendations if ML service is down.

Proactively inject failures (kill instances, add latency, network partitions) to verify the system recovers. Netflix Chaos Monkey is the canonical example.

Aggregated numerical data. Latency (p50/p99), error rate, throughput, saturation. Prometheus + Grafana.

Structured log events with severity, correlation IDs, and context. ELK Stack (Elastic, Logstash, Kibana) or Loki for log aggregation.

End-to-end request lifecycle across services. OpenTelemetry + Jaeger/Zipkin. Shows where time is spent and where errors occur.

Theoretical foundation. Proposer → Acceptors → Learners. Two phases: prepare (promise) and accept (commit). Correct but famously hard to implement correctly.

Designed for understandability. Leaders, logs, term numbers. Leader election (heartbeats + timeouts). Log replication (leader pushes to followers). Etcd, Consul use Raft.

Leader-based, atomic broadcast order. Used by Zookeeper for coordination. Global leader election via FastLeaderElection. ZXID for ordering.

Logical counters: each node increments on events. C(a) < C(b) means a happened before b. But C(a) < C(b) and C(b) < C(a) means concurrent — no causality info.

Each node tracks its view of all others' clocks. V(a) < V(b) (all components ≤) means causal relationship. Incomparable vectors = concurrent changes (conflict detection). Used in Dynamo/Cassandra.

Simplest: set a time-to-live. Cache auto-expires. Trade-off: stale data until TTL expires. Best for data that tolerates staleness (profiles, product catalogs).

On write, delete the cached entry. Next read repopulates. Simple but creates "cold start" on each write.

On write, update both DB and cache atomically. Consistent but more complex — cache write can fail separately from DB write.