APIs, integration & security — in depth

Saga Pattern in Distributed Microservices Transactions

Coordinate distributed transactions across services with compensating actions and careful isolation.

Senior Writer · · 10 min read
Cover illustration for “Saga Pattern in Distributed Microservices Transactions”
API Orchestration · September 21, 2026 · 10 min read · 2,249 words

Picture a bank teller who can freeze half a transaction while the other half goes through. That's what a single ACID transaction gives you in a monolith: one database, one commit, all or nothing. Microservices took that teller and split her into five people in five different buildings, each with their own ledger, and none of them can see what the others are doing. The Saga pattern exists because somebody has to coordinate those five people without making them share a desk.

What the Saga pattern is and where it came from

The idea is older than microservices by decades. Hector Garcia-Molina and Kenneth Salem wrote it up in 1987, back when the problem was long-running database transactions locking up resources, not distributed services. The pattern sat around quietly for twenty-some years until microservices architecture showed up in the 2010s and gave it a second life, because suddenly everyone had Garcia-Molina and Salem's exact problem, just with HTTP calls instead of table locks.

The mechanic is simple to say and harder to live with. Break one big distributed transaction into a chain of small local transactions. Each local transaction updates its own service's database, then fires off a message or event that kicks the next step into motion. No step waits around holding a lock on someone else's data. Each one just does its job, commits, and passes the baton.

Now, every step has an opposite, which is what makes it a saga and not just a chain of hopeful API calls. A compensating transaction. If step four fails, the saga runs the compensations for steps three, two, and one, in reverse, undoing what's been done. A common illustration is a vacation booking: book the flight, book the hotel, book the rental car. If the flight falls through, you don't just shrug at the hotel and the car sitting there charged. You cancel them. That cancellation is the compensating transaction, defined ahead of time, ready to run the moment something breaks.

Sagas keep atomicity, consistency, and durability, but nobody advertises the trade loudly enough. They drop isolation. Completely. While a saga is mid-flight, other transactions can see its half-finished state, because there's no lock holding the world still while it works. Under CAP theorem terms, during a network partition you pick consistency or availability, and Conduktor (2026-09-11) is blunt about which one sagas pick: availability, every time. The system stays up. Consistency is achieved eventually, not instantly. Remember that isolation gap. It's going to matter again later in this piece, and it's the reason half the implementation headaches below exist.

Choreography and orchestration: the two ways to coordinate a saga

There are two ways to run this show, and picking wrong is a slow-motion mistake, not a fast one.

Choreography has no director. Every service publishes events about what it did, and other services listen and react on their own. Conduktor's e-commerce example (2026-09-11) lays it out cleanly: Order Service publishes OrderCreated. Payment Service hears that, charges the card, publishes PaymentProcessed. Inventory Service hears that, reserves the stock, publishes InventoryReserved. Shipping Service hears that and starts moving boxes. Nobody's in charge. Everybody's just listening for their cue.

It's loosely coupled, which is the whole point of event-driven architecture in the first place, and there's no single point of failure sitting in the middle waiting to take the whole system down with it. Once the saga grows past a handful of steps, nobody can look at one screen and see where a transaction is. The flow lives in the collective behavior of five or six services, and debugging it means chasing logs across all of them. Fine for simple, linear sagas. A trap for anything with branches.

Orchestration puts a boss in the room. A central orchestrator holds the saga's state and tells each service what to do next, then decides what compensation to run if something breaks. Microservices.io's example: Order Service kicks off the orchestrator, which sends a Reserve Credit command to Customer Service, waits for the reply, then sends either ApproveOrder or RejectOrder back to Order Service. One place to look. One place to test. One place that knows the whole story.

The cost is coupling of a different flavor: services now depend on the orchestrator, and the orchestrator becomes a single point of failure if it isn't built to survive a crash mid-saga (more on that below). Still, for anything with conditional branches, multiple approval steps, or a compliance officer who wants an audit trail, orchestration wins on visibility alone.

Most mature systems don't pick one and marry it. They run choreography for the simple, reactive stuff and orchestration for the complicated, high-stakes business flows that need a paper trail. A reasonable starting heuristic: choreography while the service landscape is young and flows are short, orchestration as the business logic and the compliance demands both start piling on weight.

How the three hardest implementation problems are typically handled

Three problems appear in nearly every saga implementation, and they don't have clean fixes so much as disciplined ones.

Compensating transactions aren't rollbacks. A database rollback is free, mechanical, and forgets it ever happened. A compensating transaction is a real business action: reserving inventory gets undone by releasing it, charging a card gets undone by issuing a refund. Refunds aren't instant and they aren't guaranteed to work on the first try. Compensations need to be idempotent and retryable, and sometimes they need a human to step in when the automated retries run out. Designing a good compensation means understanding, in precise business terms, what "undo" actually means for that specific action. That's domain knowledge, not plumbing.

Idempotency isn't optional, it's the floor. Networks time out. Retries happen. If a step or its compensation runs twice by accident, running it twice needs to produce the exact same result as running it once, or the whole saga drifts into nonsense (double refunds, double shipments, the works). The sturdiest fix at the application level is unglamorous: store the unique ID of every processed message, check incoming messages against that store, and skip anything already seen.

Isolation is gone, and that produces real anomalies, because each local transaction commits immediately, leaving the in-progress state of a saga visible to anyone else looking at that data. Two named failure modes come out of this. The Lost Update: a second saga overwrites a change the first saga made before it's finished. The Dirty Read: a second transaction reads data a saga wrote, only for that saga to later compensate and erase it, leaving the reader holding stale information. AWS's own guidance for concurrent orchestration recommends semantic locking, which just means flagging a record as "in progress" so other transactions know to tread carefully around it. No tooling fully restores the level of isolation guaranteed by traditional transactions; the isolation gap has to be reasoned through by hand, saga by saga, and semantic locking is a workaround for that. No tooling fully restores the level of isolation guaranteed by traditional transactions here; the isolation gap has to be reasoned through by hand, saga by saga.

All three problems point at the same underlying rule: a saga has to end up in one of two states, fully completed or fully compensated, with nothing left dangling in between. Getting there takes idempotency, durable execution, real observability, and a clear escalation path for the failures the system can't untangle on its own.

Observability requirements for choreography and orchestration in production

A single-service transaction leaves one log trail. A saga leaves five, scattered across services and event streams, with no canonical place to go look. That's not a tooling gap; it's a structural fact about distributed systems, and it means observability for sagas has to be built in from day one rather than bolted on after the first 3am incident.

For choreography, distributed tracing isn't a nice-to-have. Without a Trace ID stitched into every single event, reconstructing what happened to a failed transaction across four or five services is close to impossible once it's in production. OpenTelemetry, Jaeger, and Zipkin are the standard tools for visualizing that request path end to end.

Orchestration has a built-in advantage here: the orchestrator's state store is a natural audit log, since it already knows every step and every decision. But that advantage evaporates the moment the state store isn't durable. Losing orchestrator state halfway through a saga is its own failure mode, and it needs to be designed against just as deliberately as any service outage.

Across both approaches, the minimum bar looks the same: detailed logging at every step, correlation IDs threading through every event and command, explicit timeout handling instead of silent hangs, and a defined escalation path for when compensation itself fails (because it will, eventually). Skip any of these and mid-saga failures stop being incidents you can diagnose and start being unexplained data corruption that nobody can trace back to its cause.

Tooling that implements saga orchestration in practice

Temporal treats sagas as long-running workflows and lets developers write the logic in an actual programming language instead of declarative JSON. It's cloud-agnostic, so it doesn't lock a team into one provider, and it comes with built-in state persistence, which directly solves the problem of the orchestrator crashing mid-saga mentioned above.

AWS Step Functions is the go-to for teams already living inside AWS. AWS's own prescriptive guidance walks through an order-processing saga built in Step Functions with separate success and failure handling at each stage. It's declarative JSON, which trades some flexibility for simplicity, and it's AWS-only, fine if that's already the ecosystem, a lock-in cost if it isn't.

MassTransit brings solid saga support built around state machines, and it's the natural fit for teams already in.NET. Apache Seata targets microservices architectures directly, built primarily in Java with growing support for Go, PHP, and others, and it offers several transaction modes depending on what a given saga needs.

General-purpose messaging and stream processing frameworks can technically be pressed into service for event routing within a saga, but they weren't purpose-built for the full saga lifecycle, compensation, retries, and all. Dedicated orchestration tools handle that lifecycle more completely.

Choosing among these comes down to a short list of real questions: does the team need to stay cloud-portable, does it prefer writing code or writing config, how well does the tool support compensation and retry natively, and how mature is the team's existing stack already.

An emerging application of saga-style compensation: AI agent workflows

AI agents that take multi-step actions against real systems run into the exact same wall. An agent books a flight, then a hotel, then a rental car, and step three fails. Something has to undo steps one and two. That's a saga problem wearing a different hat.

A paper called Robust Agent Compensation (RAC), from Perera, Hapuarachchi, Leymann, and Khalaf at WSO2 and the University of Stuttgart, is set to appear at ACM CAIS '26 (San Jose, CA, May 26 to 29, 2026). It proposes a log-based recovery approach built as an architectural extension that slots into most existing agent frameworks, LangGraph among them, without requiring a rewrite of the agent's own code. Compensation gets added through existing extension points rather than a teardown and rebuild.

The paper's benchmark numbers, run against τ2-bench and REALM-Bench, put RAC at a multiple better, ranging from a modest edge up to 8 times better, in both latency and token cost compared to current LLM-based recovery methods. That's a wide range, which suggests the gain depends heavily on the specific task, but even the low end is a meaningful improvement.

The bigger implication reaches past agents entirely: local action, compensating action, a durable execution log, idempotent retries, none of that is specific to microservices. It's a general answer to a general problem, reliable multi-step execution when any single step might fail. RAC is still a research prototype at this stage, not a production-hardened tool with a track record, so treat the performance numbers as promising early evidence rather than settled fact.

Companion patterns that a saga implementation typically requires

A saga rarely stands alone. It leans on a handful of supporting patterns, and skipping them tends to reintroduce the exact problems the saga was built to avoid.

The Transactional Outbox Pattern solves a nasty little gap: a service needs to update its own database and publish an event as one atomic move, and if it crashes between those two steps, the system ends up out of sync. This is a hard prerequisite for choreography specifically, since choreography lives and dies by events actually getting published.

CQRS (Command Query Responsibility Segregation) shows up constantly alongside sagas because splitting reads from writes lines up naturally with the event-driven flow a saga already produces. It's a pattern that appears naturally in event-driven microservices ecosystems.

The Compensating Transaction Pattern is the formal blueprint for undoing a sequence of steps, the saga pattern is really just this idea applied specifically to distributed transactions. The Retry Pattern handles the transient failures that will happen (a network blip, a timeout, a service that hiccupped for two seconds), letting the system retry automatically instead of giving up, though this only stays safe when paired tightly with idempotency. And the Circuit Breaker Pattern stops a struggling saga from hammering a downstream service that's already on its knees, mid-compensation, mid-retry, wherever.

None of these are garnish. They're the scaffolding that makes the saga's one real promise, that every saga ends up completed or fully compensated, something a production system can actually keep.

Sources

  1. Saga Pattern for Microservices Explained
  2. Microservices Pattern: Pattern: Saga
  3. Robust Agent Compensation (RAC): Teaching AI Agents to Compensate
  4. docs.aws.amazon.com
  5. conduktor.io
  6. arxiv.org
  7. doi.org

More in API Orchestration