System Integration Architecture for Cloud-Native Stacks

CNCF defines cloud-native systems by four properties: distributable, observable, portable, interoperable. That last one trips people up because it sounds like a nice-to-have you bolt on at the end. It's not. It's structural. Build a system that can't talk to itself and to the outside world from day one, and you'll pay for that decision later, usually at 2am, usually right before a release.

AWS splits the stack into four layers, and the map is worth stealing. Infrastructure is what the cloud provider runs for you (compute, storage, network, the OS underneath everything). Provisioning is how you allocate and configure that environment. Runtime is where containerd, or whatever container engine you picked, actually runs your workloads alongside storage and networking. Then there's orchestration and management, where all the separate pieces get wired together so they act like one system instead of a junk drawer. That's where integration architecture lives. Everything below it is plumbing. This layer is the actual product people are paying for.

Walk a single request through a real production system and the pattern jumps out fast. Someone taps a button on their phone. That request lands on an API gateway, which is the first real fork in the road. From there it fans out to a handful of microservices, each sitting in its own container. Kubernetes handles scaling and load balancing in the background, restarting anything that crashed without asking permission. Somewhere else, a CI/CD pipeline just pushed a change, and a monitoring system is tracking the whole trip end to end, quietly judging every millisecond.

Every handoff in that chain is a separate integration problem waiting to happen. The rest of this piece walks through them roughly in the order they show up.

The API gateway as the system's first integration decision

The API gateway is the front door. Every client, mobile app or partner server, comes through one entry point that routes the request to the right microservice, checks who's asking, and decides how fast they're allowed to ask. Part bouncer, part traffic cop, part translator. All in one box.

CNCF's Q3 2025 State of Cloud Native Development report found API gateways paired with microservices are the top integration approach, used by 46% of backend developers. Direct Kubernetes use sits at 30%, well behind it. Teams reach for the gateway before they've even nailed down orchestration, which makes sense once you think about it: it's the layer closest to the thing they're actually shipping to customers.

A gateway ready for real production traffic does more than route requests around. It needs a circuit breaker, so when a backend service starts failing, the gateway stops hammering it and hands back a fallback instead of dragging the whole chain down with it. Per-route timeouts and retries with backoff matter too, but only on routes where retrying is safe, meaning calling them twice doesn't double-charge a customer or ship two of the same order. Rate limiting sliced by user, IP, API key, or tenant is table stakes. And allow/deny lists need to be enforced right at the edge, before a bad request gets anywhere near your services.

This isn't a hypothetical. ANZ Bank runs a Kong-powered gateway to meet Open Banking rules in Australia. Amazon's own API Gateway stitches together Catalog, Inventory, and Pricing services at Prime Day volume, which is not a gentle test of anything. Netflix pairs Zuul with a service mesh to route personalized recommendations to hundreds of millions of screens at once. Kong, AWS API Gateway, Azure API Management, and Traefik are the names you keep bumping into once you start shopping around.

The gateway has a ceiling, and it's worth knowing exactly where it sits. It handles north-south traffic, client to service. It does nothing for east-west traffic, service to service, once the request is already inside your walls. Different problem. Needs a different tool entirely.

What a service mesh adds once service-to-service traffic becomes unmanageable

Picture ten services calling each other directly. Now picture twenty. Every one of those calls needs mutual authentication, retry logic, circuit breaking, and tracing, and if each team builds that plumbing on their own, you end up with twenty slightly different, slightly broken versions of the same thing. A service mesh centralizes that in infrastructure instead, and it does it without touching a single line of application code, which still feels a little like magic even after you've set one up yourself.

What you actually get: mTLS between every pair of services automatically, so nothing inside your cluster talks to anything else without proving who it is first. That's zero-trust baked into the network layer instead of duct-taped onto each app. Automatic retries and circuit breaking on internal calls come standard, along with per-call metrics flowing straight into your observability tooling. You also get traffic policy controls, canary splits, traffic mirroring, fault injection for testing how your system handles chaos before chaos finds it on its own schedule.

Istio and Linkerd are the two names that actually matter. Istio does more but costs more to run and babysit. Linkerd is lighter and faster with a narrower feature set. The choice between them isn't really a technical correctness question, it's a bandwidth question: what can your team actually operate without burning out. A four-person platform team and a forty-person one are not going to land on the same answer, and pretending otherwise is how you end up with a mesh nobody can maintain.

When do you actually need this? Once you're running 20 or more services, once security demands strict service-to-service verification, or once traffic policy needs to be enforced the same way across multiple teams. Run five services for a startup MVP and a mesh is overkill, like installing a home alarm system to guard a lemonade stand.

There's a shift happening under the hood too: sidecar-less mesh designs are gaining ground, aimed at cutting the latency and resource tax the old sidecar model adds at scale. Every pod running its own proxy container adds up fast once you're past a few hundred services.

Even with all that solved, the mesh doesn't fix one basic problem. Synchronous calls create temporal coupling. If Service B is slow, Service A waits. If Service B is down, Service A either fails or waits forever, hoping. Event-driven architecture answers that differently.

Event-driven architecture and the trade-off between loose coupling and distributed consistency

The idea is simple to describe and harder to actually live with. Instead of Service A calling Service B directly, A publishes an event and moves on with its life. Service B, and anyone else who cares, picks it up and reacts on its own schedule. A never needs to know B exists. B never needs to know A exists either. Nobody's exchanging phone numbers.

That decoupling pays off in ways you can point to. Services scale independently, no coordination needed with whoever's producing the events. If a consumer goes down, the producer keeps humming along without noticing. You can add a brand-new consumer to the system without touching a line of the producer's code, and that kind of flexibility is what lets platform teams actually sleep at night.

CNCF's Q3 2025 numbers put event-driven architecture at 27% adoption among backend developers, with streaming and messaging services close behind at 26%. Real, but it trails the API gateway pattern by a wide margin. That gap tells you something: EDA isn't a default. It's a deliberate pick, made for specific workloads where the trade-offs pencil out.

Tools you'll run into: Apache Kafka for high-throughput durable event streams, AWS EventBridge, Azure Service Bus, Google Pub/Sub, and NATS, the lighter option for when Kafka feels like bringing a freight train to deliver a postcard. ING Bank leans on EDA to manage complexity across its European financial platforms. Cloudflare runs event-driven infrastructure at the edge, globally, about as unforgiving an environment as you'll find.

Now the part nobody puts on the slide. EDA doesn't hand you loose coupling for free, it trades that for a much harder consistency problem. When a transaction spans multiple services through events, there's no single moment you can point to and say "committed." Error handling has to be designed on purpose, not assumed: dead-letter queues for messages that keep failing, idempotent consumers so replaying an event twice doesn't cause double damage, and compensating transactions to undo work when something downstream breaks. Debugging an event chain is a genuinely different kind of hard than tracing a synchronous call, too, because the request isn't a straight line anymore. It's more of a scatter pattern, and good luck drawing that on a whiteboard during an incident call.

The Saga pattern is EDA's answer to distributed transactions. It coordinates a sequence of local transactions, either through choreography (services reacting to each other's events) or orchestration (a central coordinator directing traffic), and it defines compensating steps for when any one of those local transactions fails. Generally worth the complexity once you're coordinating 15 or more services in a single business transaction; below that, it's often more scaffolding than the problem calls for.

CQRS tends to ride alongside EDA once things settle down. It splits read and write models apart, pairing well with event sourcing for workloads that lean read-heavy. It's not a day-one decision, and most teams only bring it in after the EDA foundation has proven it can stand on its own two feet.

Where serverless fits as a fourth integration model — and when it undercuts the others

Serverless means the cloud provider owns the infrastructure, full stop. Your function runs on demand, scales to zero when nobody's calling it, and you pay only for the seconds it actually runs. That's not just a pricing trick, it's a genuinely different integration shape than anything covered so far.

It earns its keep on bursty, irregular work: processing an uploaded image, handling a webhook, running a nightly ETL job, firing off a task on a schedule. Standing up a persistent, always-on service for work that happens twice a day is like leaving the lights on in a room nobody's using. Roughly 45% of new applications now use serverless or FaaS patterns for exactly this kind of bursty load, based on the survey data available.

Serverless has outgrown its old job description too. It used to be glue code stitching two other systems together. Now it's a full integration strategy on its own for event-triggered pipelines, with entire apps built as chains of functions reacting to events one after another. One media company rebuilt its event processing pipeline using serverless functions paired with WASM micro-runtimes and cut cloud costs by 29%, without giving up performance. That's not a rounding error, that's a line item someone got to brag about.

Serverless creates its own headaches, and they're worth knowing about before you commit to the pattern. Cold starts add unpredictable latency, a real problem if that function sits inside a synchronous chain where every millisecond stacks on top of the last. State has to live somewhere external entirely, there's no memory hanging around between invocations, so the moment you need to remember anything, you're reaching for a database or cache. Debugging across function boundaries needs deliberate instrumentation, because the out-of-the-box observability here is thinner than what a containerized service gives you for free. Vendor lock-in bites hardest at this layer, too. Function runtimes and their trigger integrations rarely move cleanly from one cloud to another.

Serverless and containers aren't fighting over the same job. Most production systems run both side by side, serverless handling the bursty, event-triggered edges, containers running the services that need to stay up and stay stateful.

Kubernetes as the orchestration layer that makes multi-pattern integration coherent

Kubernetes didn't win because it's the best way to deploy a container. It won because roughly 70% of enterprises have standardized on it as the shared operational model for running containers at scale, and CNCF describes its job as making cloud infrastructure "safe for the enterprise": one consistent way to run workloads, no matter which cloud they sit on.

What does it actually hand you for integration? Service discovery, for one: every service you deploy gets a stable DNS name inside the cluster, so nobody's hardcoding an IP address that changes next Tuesday. Load balancing routes around unhealthy pods automatically, no human paged at 3am to do it by hand. Namespace-based policy boundaries let network policies and access controls carve up services by team or domain without custom infrastructure built from scratch. And underneath all of it, Kubernetes is the ground that service meshes, API gateways, and event brokers actually stand on. Every pattern in this piece depends on Kubernetes staying stable and boring, in the best sense of the word.

Here's the interesting part: mature teams increasingly treat Kubernetes as plumbing. Nobody brags about their plumbing, they just expect it to work when they turn the tap. The real conversation has moved up a level, to developer platforms, self-service environments, and automation that lets application teams ship without filing a ticket to the platform team every single time they need something.

AI and ML workloads are the next stress test. Roughly 48% of organizations haven't deployed AI/ML workloads on Kubernetes yet, and the early movers are mostly running batch jobs, model experiments, and real-time inference. As that grows, it's going to push on integration patterns in ways nobody's fully mapped out.

GitOps is the discipline holding all this configuration together instead of letting it drift into chaos. 93% of organizations plan to keep or increase their GitOps use, adoption crossed two-thirds of organizations by mid-2025, and more than 80% of adopters report better reliability and faster rollbacks because of it. ArgoCD and FluxCD are the tools doing the actual work here. For integration specifically, GitOps means your gateway configs, mesh policies, and event broker schemas all live in version control: auditable, rollback-able, no longer the kind of thing that quietly drifts because someone made a manual change on a Friday afternoon and forgot to tell anyone.

How multi-cloud and hybrid topology change every integration decision already made

Here's the twist nobody mentions until it's their problem: everything covered so far assumes a single cluster, a single provider, a tidy little world. Most companies don't live there. 83% of enterprises run two or more public cloud providers, and 78% use hybrid setups, keeping sensitive or latency-critical systems on private infrastructure while scaling the rest on public cloud. Multi-cloud breaks the tidy assumption immediately, and it doesn't apologize for it.

Every layer feels this differently. Your API gateway can no longer see every service in the system, because some of them sit behind a different provider's front door entirely, so you end up needing federation or a global gateway layer sitting above the individual ones. Your service mesh's mTLS identity doesn't stretch across clusters on its own, so cross-cluster service discovery (Istio's multi-cluster mode is the usual example) has to be configured on purpose rather than assumed to just work. Your event brokers don't replicate for free either. A Kafka cluster running on AWS and a Pub/Sub topic running on GCP won't sync themselves, so schema consistency and delivery guarantees have to be engineered by hand. And your observability traces will simply snap at the provider boundary unless correlation IDs and OpenTelemetry instrumentation are enforced everywhere, no exceptions, no shortcuts.

Portability is the fix, and it's not a nice bonus feature. CNCF's reference architecture names it as a required property. In practice that means favoring open standards, OpenTelemetry for tracing, CloudEvents for event formatting, OpenAPI for service contracts, over provider-native integrations, wherever there's a real chance that service moves across environments down the road.

Vendor lock-in risk concentrates hardest right here, at the integration layer. A provider-native event bus, a managed API gateway, a cloud vendor's own mesh offering: each one buys convenience today and hands you an expensive migration bill later.

Latency stops being background noise the moment calls cross a region or a provider boundary, too. A synchronous call that took two milliseconds inside one cluster might take two hundred once it's hopping between providers, and it inherits every reliability quirk of the public internet along the way. This is exactly where event-driven and async patterns earn their keep over synchronous API calls. You're not holding a phone line open across an ocean. You're dropping a message in a mailbox and trusting the system to deliver it.