---
name: Distributed Systems Architecture
description: Design cloud-native, distributed systems that hold up — making the hard trade-offs of consistency, partitioning, and failure explicit, choosing serverless vs containers vs cluster by the actual load, and automating infrastructure — so the architecture matches the requirement instead of chasing scale it doesn't have.
audience: engineer · architect · founding engineer · platform lead
---

# Distributed Systems Architecture

## What this is
A method for architecting systems that span machines: reasoning honestly about the trade-offs (consistency vs availability under partition, latency vs durability), choosing the right cloud-native building blocks (serverless, containers, managed data stores), and automating the infrastructure so it's reproducible.

## What this is NOT
- **Not scale cosplay.** The first move is to question whether distribution is needed at all; a well-run single node or a managed service often beats a premature microservices/K8s sprawl. The skill right-sizes to the real requirement.
- **Not a magic-consistency promise.** It's explicit that distributed systems force trade-offs (CAP/PACELC): you choose which guarantee to relax, and the choice is stated, not hidden.
- **Not an ops or security substitute.** Deployment pipelines route to the DevOps skill; IAM and the security surface get a dedicated review.

## When to use
Designing a new service or reworking an architecture; choosing serverless vs containers vs cluster; deciding data consistency and partitioning; adding resilience (retries, circuit breakers, idempotency); automating cloud infrastructure (AWS/GCP IaC).

## Operating principle
Every distributed choice trades something away — say what. Design for failure as the normal case (things partition, retry, and arrive twice), make operations idempotent, and pick the weakest consistency the requirement actually tolerates, because strong consistency everywhere is a cost you rarely need.

## Capabilities
- **Architecture & trade-off design** — Goal: the right shape for the requirement. Method: decompose by real boundaries (not fashion), choose sync vs async and the consistency model per data flow, state the CAP/PACELC trade-off explicitly, define idempotency and exactly-once-effect handling. Output: an architecture with the trade-offs named per component. Quality bar: each consistency/availability choice is justified against the requirement; distribution is questioned before it's adopted.
- **Cloud-native building blocks** — Goal: right tool, right load. Method: match compute (serverless for spiky/event-driven, containers for steady, cluster only when justified) and storage (SQL vs document vs KV vs stream) to access patterns and scale; use managed services to cut ops. Output: a component selection with a "why this, at this load" note. Quality bar: serverless/K8s is chosen on load and cost evidence, not reflex; managed beats self-run unless there's a reason.
- **Resilience & infra automation** — Goal: survives failure, reproducible. Method: timeouts, retries with backoff+jitter, circuit breakers, bulkheads, graceful degradation; infrastructure as code (Terraform/CDK) with least-privilege; observability (traces/SLOs). Output: a resilience plan + IaC skeleton. Quality bar: failure paths are designed, not assumed; infra is codified and reproducible, and the SLO defines "healthy."

## A worked example
"We're going to microservices and Kubernetes to be web-scale." → The skill pushes back: current load fits a modular monolith on a managed container platform with a managed database; the one genuinely spiky workload (image processing) becomes a serverless queue-worker with idempotent handling. The consistency choice is stated (eventual for the processing status, strong for billing), failure handling (retry+backoff, dead-letter) is designed in, and the whole thing is Terraformed — matching the requirement, not a scale that isn't there yet.

## Guardrails & escalation
Deployment pipelines & rollout → the DevOps & CI/CD skill. IAM, network security, and the attack surface → security review. Data-residency/compliance on where data lives → legal/compliance. Regulated-scale or safety-critical systems → specialists. Architecture states its trade-offs; it doesn't hide them.

## References
Distributed-systems fundamentals (CAP/PACELC, consensus, idempotency, exactly-once-effect); cloud-native and 12-factor patterns; resilience patterns (retries, circuit breakers, bulkheads); IaC (Terraform/CDK); SRE/SLOs. Right-size to the real requirement and cost.