Why Traditional Transactions Break Down at Scale
You’ve built a monolith. It works. Transactions are beautiful, atomic, and reliable. Then growth happens. You split services. Suddenly, coordinating state changes across multiple databases becomes the kind of problem that keeps you awake at 3 AM debugging partial failures.
Two-phase commit seems like the obvious answer. It worked in your database textbook. But 2PC assumes a world where network partitions are rare and services stay available. In distributed systems, these assumptions crumble fast. When your payment service can’t reach the inventory service during the commit phase, you’re stuck with locks held indefinitely. Scale this across dozens of services and you’ve built a house of cards.
The Saga pattern emerged from this pain. Instead of coordinating a single atomic transaction, you model your business process as a sequence of local transactions. Each step can succeed or fail independently. When something goes wrong, you don’t roll back a global transaction. You compensate.
Orchestration: The Command and Control Approach
Orchestration puts a central coordinator in charge of the entire saga. Think of it as a workflow engine that knows every step and decides what happens next. When a customer places an order, the orchestrator calls the payment service, then inventory, then shipping, handling each response and determining the next action.
This centralized approach has real advantages. The business logic lives in one place. You can see the entire flow by reading the orchestrator code. Debugging becomes manageable because there’s a single place to add logging and monitoring. When something breaks, you know where to look.
But orchestration creates a bottleneck. The orchestrator becomes a single point of failure. Every business process flows through this central component, making it a scaling constraint. Worse, it tends to accumulate business logic that should belong to individual services. Your orchestrator grows into a distributed monolith, knowing too much about every service’s internal behavior.
Netflix’s Conductor and Uber’s Cadence are production examples of orchestration done right. They work because these companies invested heavily in making the orchestrator itself distributed and fault-tolerant. For most teams, this infrastructure burden outweighs the benefits.
Choreography: Dancing Without a Conductor
Choreography takes the opposite approach. No central coordinator. Each service knows its role in the larger dance and reacts to events from other services. When payment completes, it publishes an event. The inventory service listens, reserves items, and publishes its own event. The shipping service picks up that signal and starts fulfillment.
This feels more natural in microservices architectures. Services stay loosely coupled. Each service owns its piece of the business process without external coordination. Adding new steps doesn’t require updating a central orchestrator. You just wire up the appropriate event listeners.
The challenge is observability. The business process exists only as emergent behavior from service interactions. When an order gets stuck, there’s no single place to check its status. You piece together the story from events scattered across multiple services. Debugging requires correlating timestamps and trace IDs across dozens of log files.
Choreography also makes it harder to enforce business rules that span services. In orchestration, the coordinator can validate the entire state before proceeding. In choreography, each service sees only its local view. Global invariants become difficult to maintain without careful design.
Compensation: Undoing What You’ve Done
Both approaches face the same fundamental challenge: handling failures after you’ve started modifying state. Unlike database transactions, you can’t simply abort and pretend nothing happened. The payment went through. The inventory was reserved. Other systems might have already reacted to these changes.
Compensation transactions reverse the effects of previously completed steps. If shipping fails after payment and inventory succeed, you issue a refund and release the reserved items. These compensating actions aren’t simple inverses. Refunding a payment might involve different APIs, fees, or approval workflows than the original charge.
Designing good compensation requires understanding your business deeply. Some actions can’t be perfectly undone. You can’t un-send an email or un-print a shipping label. The best you can do is minimize the impact through careful ordering of operations and graceful degradation strategies.
Time adds another layer of complexity. Compensation might happen minutes or hours after the original action. External state might have changed. The customer’s credit card might have expired. The warehouse might have moved inventory. Your compensation logic needs to handle these temporal inconsistencies gracefully.
Implementation Lessons from the Field
Start with orchestration if you’re building your first saga. The explicit control flow makes it easier to reason about and debug. You can always evolve toward choreography as your system matures and you better understand the interaction patterns.
Make compensation transactions idempotent. Networks are unreliable. Messages get duplicated. Your refund logic might run multiple times for the same transaction. Design it to detect and handle repeated execution safely.
Invest in correlation IDs and distributed tracing early. Whether you choose orchestration or choreography, you’ll need to track requests across service boundaries. A unique saga ID that flows through every related operation becomes invaluable when debugging production issues.
Consider using event sourcing for saga state management. Instead of storing current state, persist the sequence of events that led to that state. This gives you a complete audit trail and makes it easier to replay or analyze saga execution paths.
The Saga pattern isn’t a silver bullet. It trades the complexity of distributed transactions for the complexity of designing proper compensation logic. But when two-phase commit isn’t viable, it gives you a pragmatic path forward. Done right, sagas let you keep the benefits of microservices without sacrificing data consistency entirely.
Have you implemented sagas in production? I’d love to hear about the specific challenges you faced, especially around compensation design and failure recovery strategies.
