hld

Service Boundaries & Event-Driven Architecture: Decoupling at Scale

Master the art of defining service boundaries using domain-driven design principles. Explore event-driven architecture, async patterns, saga implementations, and how to decouple services while maintaining consistency and system observability.

May 22, 202622 min read
microservicesevent-driven-architecturedomain-driven-designdistributed-systemssaga-pattern

Service Boundaries & Event-Driven Architecture: Decoupling at Scale

TLDR: Bad service boundaries are worse than no services. Use domain-driven design (subdomains, bounded contexts) to identify seams. Event-driven architecture lets you decouple services without losing eventual consistency. Sagas handle cross-service transactions.

Service Boundaries via DDD & Events


Part 1: The Boundary Problem

Why Boundaries Matter

When you extract a microservice, you're making a bet:

High cost if wrong:  (Boundary = data synchronization hell)
  ├─ Service A needs data from Service B
  ├─ Service A needs data from Service C
  ├─ Constant network calls (latency)
  ├─ Data inconsistency windows (A sees stale C data)
  ├─ Cascading failures (B fails, A fails)
  └─ Maintenance nightmare (debugging spans 3 services)

Cost of bad boundary: 10x developer overhead

Low cost if right:   (Boundary = clean separation)
  ├─ Each service is self-contained
  ├─ Rare cross-service calls
  ├─ Asynchronous where needed
  ├─ Failures isolated
  └─ Simple debugging

The Naive Approach (What Fails)

❌ Boundary by technology layer:
  ├─ Frontend Service
  ├─ API Service
  ├─ Database Service
  └─ Cache Service
  
  Problem: Every feature touches all 4 layers.
  Each layer changes with every feature.
  "Microservices" become shared libraries.

❌ Boundary by lifecycle:
  ├─ Core Service (stable)
  ├─ Feature Service (frequently changing)
  ├─ Experiment Service (very frequently changing)
  
  Problem: Feature Service likely depends on Core Service.
  Changes propagate everywhere.
  No isolation benefit.

❌ Boundary by team:
  ├─ Team A Service
  ├─ Team B Service
  └─ Team C Service
  
  Problem: If Team A and B share data,
  boundaries are artificial. You just added latency
  without removing coupling.

The Right Approach: Domain-Driven Design (DDD)

Boundary = Bounded Context (autonomous business domain)

A bounded context is:

  • Self-contained (owns its data, logic, rules)
  • Minimal external dependencies (rare cross-context calls)
  • Clear interface (what it exports, what it needs)
  • Meaningful to the business (CEO understands it)

Part 2: Finding Boundaries with DDD

Step 1: Identify Core Subdomains

Map your business:

E-Commerce Company:
├─ Core Domain: Order Management
│  └─ Business differentiator (what makes you money)
│     └─ If you get this wrong, company dies
│
├─ Supporting Domain: Payment Processing
│  └─ Critical but not differentiator
│     └─ Stripe/Adyen handle most of this
│
├─ Generic Domain: User Authentication
│  └─ Table stakes, not differentiator
│     └─ Off-the-shelf (Auth0, Okta)
│
└─ Generic Domain: Email Notifications
   └─ Table stakes
      └─ Off-the-shelf (Sendgrid, Brevo)

Rule: Extract a service only for Core or critical Supporting domains.

Step 2: Identify Bounded Contexts

Within each domain, find contexts:

Order Management (Core Domain):
│
├─ Fulfillment Context
│  └─ Owns: picking, packing, shipping
│     Exports: OrderShipped event
│     Imports: PaymentConfirmed event
│
├─ Inventory Context
│  └─ Owns: stock levels, reservations
│     Exports: StockUpdated event
│     Imports: OrderPlaced event (reserve stock)
│
├─ Pricing Context
│  └─ Owns: discounts, promotions, tax calculation
│     Exports: PriceCalculated (total with tax)
│     Imports: (rarely imports)
│
└─ Order Context
   └─ Owns: order records, state machine
      Exports: OrderPlaced, OrderConfirmed events
      Imports: PaymentConfirmed, StockReserved events

Each context = one microservice

Step 3: Define Context Boundaries

Map relationships between contexts:

┌─────────────────────┐
│   Order Context     │
│ (Core, stateful)    │
└──────────┬──────────┘
           │
           │ (async via events)
           │
    ┌──────┴──────┐
    │             │
    ▼             ▼
┌────────────┐  ┌──────────────┐
│  Inventory │  │  Fulfillment │
│  Context   │  │   Context    │
└────────────┘  └──────────────┘
    │
    │ (sync call only for stock check)
    │
┌────────────────────┐
│   Pricing Context  │
└────────────────────┘

Rule: Each arrow is a dependency.
      Minimize arrows.
      Prefer async (events) over sync (API calls).

Part 3: Event-Driven Architecture

Why Events?

Events decouple services without tight coupling:

Synchronous (Tight Coupling)

┌──────────┐
│  Order   │
│ Service  │
└────┬─────┘
     │ POST /inventory/reserve
     ▼
┌──────────────┐
│  Inventory   │
│  Service     │
└──────────────┘
     │ 
     │ POST /pricing/calculate
     ▼
┌──────────────┐
│  Pricing     │
│  Service     │
└──────────────┘

Problems:
- Order Service knows about Inventory API
- Order Service knows about Pricing API
- If Pricing fails, Order fails
- Hard to add new steps (e.g., Fraud Check)
- Cascading timeouts

Asynchronous (Loose Coupling)

┌──────────┐
│  Order   │
│ Service  │ ──→ Publishes: "OrderPlaced" event
└──────────┘
                ↓ (Event Bus / Message Queue)
     ┌──────────┴──────────┬──────────┐
     │                      │          │
     ▼                      ▼          ▼
┌──────────┐         ┌──────────┐  ┌────────┐
│Inventory │         │ Pricing  │  │Fraud   │
│Service   │         │ Service  │  │Service │
│(listens) │         │(listens) │  │(new!)  │
└──────────┘         └──────────┘  └────────┘

Benefits:
- Order Service publishes one event
- Inventory, Pricing, Fraud all listen independently
- Add Fraud Check without Order Service knowing
- If Pricing is slow, Order still succeeds
- Fault isolation

Event-Driven Patterns

Pattern 1: Event Sourcing

Every state change is an event. State = applying events in order.

Account Service:

Events:
  ├─ AccountCreated(account_id=123, owner="alice", balance=0)
  ├─ DepositMade(account_id=123, amount=100)
  ├─ WithdrawalMade(account_id=123, amount=30)
  ├─ InterestAccrued(account_id=123, amount=0.50)
  └─ AccountFrozen(account_id=123, reason="suspicious")

Current state (balance):
  0 + 100 - 30 + 0.50 = 70.50

Current state (frozen):
  true

Advantages:

  • Complete audit trail (why did balance change?)
  • Temporal queries (what was balance at 2am?)
  • Replay (if you find a bug, replay events with fix)
  • Horizontal scaling (read replicas can rebuild state independently)

Disadvantages:

  • Snapshots needed for large event logs (replaying 100K events is slow)
  • Event versioning (what if DepositMade schema changes?)
  • Complexity (operational overhead)

Pattern 2: Change Data Capture (CDC)

Database publishes events when data changes. External source of truth.

Inventory Database:
  ┌────────────────────────────┐
  │ products table             │
  │ (product_id, stock_level)  │
  └────┬───────────────────────┘
       │
       │ (Change Data Capture)
       │ Listens to WAL/binlog
       │
       ▼
  ┌────────────────────┐
  │  Kafka (CDC topic) │
  └────┬───────────────┘
       │
       ├──→ Fulfillment Service (listens, adjusts packing logic)
       ├──→ Analytics (listens, tracks stock trends)
       └──→ Cache Service (listens, invalidates)

Advantages:

  • Events are guaranteed (every DB change = event)
  • External services always consistent with source
  • No application code changes (CDC is transparent)

Disadvantages:

  • Requires CDC tool (Debezium, Postgres Logical Replication)
  • Operational complexity (monitoring WAL, lag)
  • CDC lag (brief window of inconsistency)

Pattern 3: Outbox Pattern

Service writes to its own database + publishes to event queue atomically.

Problem: What if event publish fails?

Order Service processes order:
  ├─ Write to DB: INSERT INTO orders (id, status, ...)
  ├─ Publish to Kafka: OrderPlaced event
  └─ If Kafka fails → DB has order, event never published
     (Inventory never knows to reserve stock)

Solution: Outbox table

Order Service:
  BEGIN TRANSACTION
    ├─ INSERT INTO orders (...)
    ├─ INSERT INTO outbox (event_id, event_type, payload)
    └─ COMMIT (atomic, both or nothing)

Separate "Outbox Relay" service:
  ├─ Query outbox (events not yet published)
  ├─ Publish to Kafka
  ├─ Mark as published in outbox
  └─ Retry failed publishes (implements exponential backoff)

Benefit: Exactly-once semantics (event published iff DB transaction succeeds)

Pattern 4: CQRS (Command Query Responsibility Segregation)

CQRS splits your data model into two paths: writes (commands) go to a normalized store optimized for consistency; reads (queries) go to a denormalized store optimized for performance.

Without CQRS (same model for reads and writes):

  POST /orders        → INSERT INTO orders (normalized schema)
  GET /order-history  → 6-table JOIN (slow, blocks writes)

With CQRS:

  Command side (write model):
  POST /orders
    ├─ Validate business rules
    ├─ INSERT INTO orders (normalized, ACID)
    └─ Emit: OrderCreated event

  Query side (read model):
  GET /order-history
    ├─ Read from orders_view (denormalized, pre-joined)
    └─ Fast response, no locks

  Read model is rebuilt from events:
  OrderCreated → projector → UPDATE orders_view
  OrderShipped → projector → UPDATE orders_view
  OrderCancelled → projector → UPDATE orders_view

Why separate them?

Reads:
  ├─ High frequency (100x more than writes)
  ├─ Different shape (aggregated, joined, formatted)
  ├─ Tolerate slight staleness (eventual consistency OK)
  └─ Scale independently

Writes:
  ├─ Low frequency but high consistency requirement
  ├─ Must be atomic and validated
  └─ Need strong consistency

CQRS + Event Sourcing = natural pair:

Command flow:
  1. User submits: POST /orders {product_id, qty}
  2. Validate command (inventory, payment eligibility)
  3. Write event: OrderCreated {order_id, user_id, items, total}
  4. Event store is the source of truth

Query flow:
  1. Event projector listens to OrderCreated
  2. Writes denormalized row to orders_read_model table:
     {order_id, user_name, product_names, status, total, formatted_date}
  3. GET /orders/history reads directly from orders_read_model
     (no joins, no locks, sub-millisecond)

Trade-offs:

✓ Scale reads and writes independently
✓ Read models optimized for specific UIs (different models per client)
✓ Audit trail via events
✓ Easy to add new read models without changing write logic

✗ Eventual consistency (read model lags write model by ms–seconds)
✗ Operational complexity (two stores to manage)
✗ Overkill for simple CRUD apps

Use CQRS when: read patterns are complex or high-frequency, write consistency is strict, or you're already using event sourcing.

Pattern 5: Dead Letter Queues (DLQ)

When a consumer fails to process a message after multiple retries, it shouldn't block the queue forever. Dead Letter Queues store failed messages for investigation.

Normal flow:
  Producer → Queue → Consumer (success) ✓

Failure flow:
  Producer → Queue → Consumer (fail)
                         └─ Retry 1 (fail)
                         └─ Retry 2 (fail)
                         └─ Retry 3 (fail)
                         └─ Max retries exceeded
                              → Dead Letter Queue (DLQ)
                                   └─ Alert oncall
                                   └─ Human investigates
                                   └─ Fix & replay or discard

DLQ configuration (Kafka example):

// Kafka consumer with DLQ routing (Spring Cloud Stream)
consumer:
  group: order-service
  max-attempts: 3
  back-off:
    initial-interval: 1000   # 1s
    multiplier: 2             # exponential: 1s, 2s, 4s
  dead-letter-queue:
    topic: order-service.DLQ
    append-original-headers: true  # preserve original metadata

What to track in your DLQ:

Each DLQ message should include:
  ├─ Original message payload
  ├─ Original topic + partition + offset
  ├─ Exception type + stack trace
  ├─ Number of retry attempts
  └─ Timestamp of first failure

Monitoring:
  ├─ Alert when DLQ message count > 0
  ├─ Alert when DLQ growth rate > threshold (spike = systemic failure)
  └─ Dashboard: error rate by message type

DLQ playbook:

On DLQ alert:
  1. Inspect message payload (is it malformed?)
  2. Check consumer logs for exception
  3. If bug: fix consumer, replay DLQ messages
  4. If bad data: discard + notify producer
  5. If transient (DB down): replay after DB recovery

Pattern 6: Event Versioning & Schema Evolution

Events are contracts between services. Consumers depend on them. Changing an event schema can silently break consumers deployed weeks apart.

Problem: OrderPlaced event schema changes

v1 (original):
  {order_id, user_id, product_id, quantity}

v2 (added field):
  {order_id, user_id, product_id, quantity, coupon_code}

Consumer on v1: ignores coupon_code → safe
Consumer on v1 receiving v2 event: OK (extra field ignored)
Consumer on v2 receiving v1 event: coupon_code missing → NPE if not nullable

Safe vs. breaking changes:

Safe (backward compatible):
  ├─ Add an optional field (consumers ignore it)
  ├─ Add a new event type (existing consumers ignore unknown events)
  └─ Widen a field type (int → long)

Breaking (coordinate with all consumers first):
  ├─ Remove a required field
  ├─ Rename an existing field
  ├─ Change field type (string → int)
  └─ Rename the event type itself

Versioning strategies:

Strategy 1: Version in event type name
  OrderPlacedV1, OrderPlacedV2
  ├─ Consumer subscribes to both during migration
  └─ Decommission V1 once all consumers migrated

Strategy 2: Schema Registry (Confluent, AWS Glue)
  ├─ Register schema before publishing
  ├─ Registry enforces compatibility rules
  │  (BACKWARD, FORWARD, FULL compatibility checks)
  └─ Consumer fetches schema at runtime by ID

Strategy 3: Dual publishing (safest migration)
  During migration period:
  ├─ Publish both V1 and V2 events
  ├─ Old consumers continue on V1
  ├─ New consumers use V2
  └─ Retire V1 once old consumers are upgraded

Rule: Treat events like public APIs. Once published and consumed, removing or renaming fields requires a deprecation cycle, not a force-push.


Part 4: The Saga Pattern (Distributed Transactions)

When you need cross-service consistency, sagas coordinate the transaction.

Choreography vs Orchestration

The Problem: Distributed Transaction

Order with payment:

Monolith (easy):
  BEGIN TRANSACTION
    INSERT INTO orders (id, user_id, total)
    UPDATE accounts SET balance = balance - total
  COMMIT

Microservices (hard):
  POST /orders (Order Service)
    ├─ Create order in DB
    └─ Call POST /payments/charge (Payment Service)
       ├─ If success: return
       └─ If fail: ??? order already created, charge failed
  
  Inconsistency: Order exists but wasn't paid.

Saga Pattern: Choreography (Event-Driven)

Services react to events; no central orchestrator.

1. User submits order
   
   ┌──────────────┐
   │ Order Service│ ──→ publishes: OrderPlaced
   └──────────────┘
   
2. Payment Service listens to OrderPlaced
   
   ┌───────────────┐
   │Payment Service│ ──→ publishes: PaymentProcessed
   │               │     (or PaymentFailed)
   └───────────────┘
   
3. Inventory Service listens to PaymentProcessed
   
   ┌─────────────────┐
   │Inventory Service│ ──→ publishes: StockReserved
   │                 │     (or ReservationFailed)
   └─────────────────┘
   
4. Fulfillment Service listens to StockReserved
   
   ┌─────────────────┐
   │Fulfillment Svc  │ ──→ publishes: OrderShipped
   └─────────────────┘

If Payment fails:
   Order Service listens to PaymentFailed
   ├─ Marks order as FAILED
   └─ Publishes: OrderCancelled (compensating transaction)

If Stock not available:
   Inventory Service publishes ReservationFailed
   ├─ Payment Service listens, refunds payment
   ├─ Order Service listens, marks order CANCELLED
   └─ Publishes: OrderCancelled

Advantages:

  • Decoupled (services don't know about each other)
  • Easy to add steps (new service subscribes to events)

Disadvantages:

  • Hard to reason about (transaction logic spread across services)
  • Debugging nightmare (which service failed? why?)
  • Implicit contract (what events do I need to handle?)

Saga Pattern: Orchestration (Coordinator)

Central "Saga Orchestrator" coordinates the transaction.

Saga Orchestrator (simple state machine):

┌─────────────────────────────────────────────────────────┐
│                  OrderSaga (state machine)              │
├─────────────────────────────────────────────────────────┤
│                                                          │
│  State: PENDING                                         │
│  ├─ Call: POST /orders/create                          │
│  └─ Transition: PENDING → ORDER_CREATED                │
│                                                          │
│  State: ORDER_CREATED                                  │
│  ├─ Call: POST /payments/charge                        │
│  ├─ On success: Transition → PAYMENT_DONE              │
│  └─ On failure: Transition → PAYMENT_FAILED            │
│                                                          │
│  State: PAYMENT_DONE                                   │
│  ├─ Call: POST /inventory/reserve                      │
│  ├─ On success: Transition → RESERVED                  │
│  └─ On failure: Call COMPENSATE /payments/refund       │
│                 Transition → CANCELLED                 │
│                                                          │
│  State: RESERVED                                       │
│  ├─ Call: POST /fulfillment/ship                       │
│  ├─ On success: Transition → SHIPPED (end)             │
│  └─ On failure: Compensate /inventory/unreserve        │
│                 Compensate /payments/refund            │
│                 Transition → CANCELLED                 │
│                                                          │
└─────────────────────────────────────────────────────────┘

Request flow:
  POST /orders
    ↓
  ┌──────────────────────┐
  │ OrderSaga Orchestrator│ (implements state machine above)
  └──┬──────────────────┬┘
     │                  │
     ▼                  ▼
  [Order Service] [Payment Service]
     ▼
  [Inventory Service]

Advantages:

  • Clear logic (orchestrator = business process)
  • Easy debugging (one place to understand flow)
  • Explicit compensation (rollback logic in orchestrator)

Disadvantages:

  • Central coordinator (bottleneck, single point of failure)
  • Requires saga framework (Temporal, Cadence, Axon)
  • More infrastructure

Orchestration vs Choreography: When to Use

Use Choreography (events):
  ├─ Few services involved (< 5)
  ├─ Simple flow (linear: A → B → C)
  ├─ Rare compensations
  └─ Team familiar with async patterns

Use Orchestration (coordinator):
  ├─ Many services involved (10+)
  ├─ Complex flow (conditional branches, parallel steps)
  ├─ Complex compensations
  └─ Need explicit audit trail

Part 5: Data Consistency Patterns

Eventual Consistency

Accept that data across services isn't always in sync. Converge to consistency.

Timeline of account balance propagation:

T=0:  Transfer 100 from Account A to Account B
      ├─ Account A: balance -= 100 (immediate)
      └─ Account B: balance += 100 (via async event)

T=5ms: Account A DB has -100, Account B still stale (+0)
       ├─ Inconsistent (reads return different values)
       └─ But... event is in-flight

T=50ms: Event processed, Account B updated (+100)
        └─ Consistent again

Acceptable for: 99% of business cases
              (user transfers, inventory updates, recommendations)

NOT acceptable for: Financial audits
                   Real-time inventory counts
                   Stock trading

Strong Consistency (Consensus)

All replicas agree before returning success.

Distributed transaction:

Client: Debit Account A by 100
  ↓
┌─────────────────────────┐
│ Leader (Replica 1)      │
│ ├─ Write locally        │
│ └─ Propose to other     │
│    replicas             │
└────────┬────────────────┘
         │
    ┌────┴────┐
    ▼         ▼
┌────────┐ ┌────────┐
│ Rep 2  │ │ Rep 3  │
│ ACK    │ │ ACK    │
└────────┘ └────────┘

Result: All replicas consistent before returning OK
Cost: Wait for remote replicas (high latency, complex)
Use cases: Payment settlement (must be atomic across all bookings)

Read-Repair Consistency

Detect inconsistencies on read, fix them.

Account Balance Caching:

Primary DB: balance = 100
Cache (stale): balance = 90

Client reads from cache:
  ├─ Gets 90
  └─ Also reads from primary (background)
     └─ Detects mismatch (90 != 100)
        ├─ Updates cache to 100
        └─ Returns 100 to client (for next request)

Benefit: Cheap reads, eventual consistency
Cost: Occasional wrong reads

Part 6: Service Boundary Anti-Patterns

Anti-Pattern 1: Chatty Services

Every feature requires 10 API calls across 5 services.

❌ User Profile Page:

GET /profile/user/123
  ├─ User Service: get user info (1 call)
  ├─ Profile Service: get bio, photo (1 call)
  ├─ Payment Service: get subscription (1 call)
  ├─ Stats Service: get engagement (1 call)
  ├─ Notification Service: get preference (1 call)
  └─ Social Service: get followers (1 call)

Total: 6 calls × 50ms = 300ms overhead (on top of actual processing)

This is why boundaries failed. These should be one service or one API.

Fix: API Gateway aggregates (BFF - Backend for Frontend)

✓ API Gateway:
  GET /profile/user/123
  ├─ Calls 6 services in parallel (not sequentially)
  ├─ Aggregates responses
  └─ Returns one JSON blob
  
Cost: Duplicates some logic (small price for speed)

Anti-Pattern 2: Shared Database

Multiple services write to the same database.

❌ Shared Orders DB:

┌──────────┐
│ Orders   │
│ Service  │
└────┬─────┘
     │
     │ writes to
     │
┌────▼──────────────┐
│  Shared Orders DB │
│ (can you change   │
│  schema?)         │
└────┬──────────────┘
     │
     │ writes to
     │
┌────▼────────────┐
│ Fulfillment     │
│ Service         │
└─────────────────┘

Problem:
  - Order Service wants to add a column → must coordinate with Fulfillment
  - Fulfillment changes a column → Order Service breaks
  - Not really microservices (they're shared libraries)
  - Circular dependency

Fix: Each service owns its data

✓ Orders Service DB:
  ├─ Owns: order record, status
  └─ Exports: OrderShipped event

✓ Fulfillment Service DB:
  ├─ Owns: fulfillment record, tracking
  └─ Imports: OrderPlaced event

Anti-Pattern 3: God Service

One service that every other service depends on.

❌ Architecture:

All services depend on:
┌─────────────────┐
│  User Service   │ (every feature needs to check user auth, permission)
└─────────────────┘

When User Service fails:
  └─ Everything fails (even unrelated features)

Single point of failure masquerading as microservices.

Fix: Minimize shared dependencies

✓ User Service:
  ├─ Owns: authentication, basic user info
  └─ Cache user data in other services (async sync via events)

Result:
  ├─ User Service can fail
  ├─ Orders still work (with cached user data)
  └─ Graceful degradation

Part 7: Practical Service Design

Service Contract Example

Inventory Service

Owned Data:
  ├─ Product ID
  ├─ Stock Level
  └─ Last Updated

Inbound Events (subscribes):
  ├─ OrderPlaced (reserve stock)
  └─ OrderCancelled (unreserve stock)

Outbound Events (publishes):
  ├─ StockReserved (with reserved_qty, expires_at)
  ├─ ReservationFailed (reason, available_qty)
  └─ StockUpdated (new_level, timestamp)

Sync APIs (rarely used):
  ├─ GET /inventory/{product_id} (check stock before order)
  └─ POST /inventory/reserve (deprecated, use events)

Response Times:
  ├─ Event processing: <100ms (async, best effort)
  └─ Sync API: <50ms p99

Failure Modes:
  ├─ Stock oversold (event race condition, acceptable <1%)
  ├─ Reservation expiration (auto-release after 10 min)
  └─ Service down (clients retry, use cached last-known stock)

SLA:
  ├─ 99.9% availability
  ├─ Events delivered within 10 seconds
  └─ Sync API p99: <100ms

Part 8: Real-World Example: Order Saga

E-Commerce Order Flow:

1. User clicks "Buy" on website
   └─→ POST /api/orders {product_id, qty, user_id}

2. Order Service (Orchestrator)
   ├─ Creates order in DB (status=PENDING)
   ├─ Publishes OrderPlaced event
   └─ Starts OrderSaga state machine

3. Payment Service listens to OrderPlaced
   ├─ Calls payment gateway (Stripe, Razorpay)
   ├─ On success: Publishes PaymentCompleted
   └─ On failure: Publishes PaymentFailed

4. Saga hears PaymentFailed
   ├─ Compensates: Delete order from DB
   ├─ Publishes OrderCancelled
   └─ Returns error to user

5. Saga hears PaymentCompleted
   ├─ Transitions to RESERVED state
   └─ Publishes ReserveStock event

6. Inventory Service listens to ReserveStock
   ├─ Checks stock
   ├─ On success: Decrements stock, Publishes StockReserved
   └─ On failure: Publishes ReservationFailed

7. Saga hears StockReserved
   ├─ Publishes FulfillOrder event
   └─ Notifies Fulfillment Service

8. Fulfillment Service listens to FulfillOrder
   ├─ Creates shipment
   ├─ Publishes OrderShipped
   └─ Updates tracking in fulfillment DB

9. Saga hears OrderShipped
   ├─ Updates order status to COMPLETED
   ├─ Publishes OrderCompleted event
   └─ Returns success to user

10. Analytics Service listens to OrderCompleted
    ├─ Increments daily sales
    ├─ Updates product popularity
    └─ Stores in data warehouse

Failure scenario (stock runs out):

6. Inventory Service hears ReserveStock
   ├─ No stock available
   └─ Publishes ReservationFailed

7. Saga hears ReservationFailed
   ├─ Publishes RefundPayment event
   ├─ Updates order status to FAILED

8. Payment Service hears RefundPayment
   ├─ Calls payment gateway: refund
   ├─ Publishes PaymentRefunded

9. Saga hears PaymentRefunded
   ├─ Publishes OrderCancelled
   └─ Notifies user: "Sorry, out of stock"

Entire transaction:
  ├─ Happens across 5 services
  ├─ No central lock
  ├─ Each service independent
  ├─ Clear audit trail (every state change = event)
  └─ Eventual consistency (all systems converge)

Conclusion

Service boundaries = organizational and technical boundaries.

Get them right:

  • Use domain-driven design (think business domains, not technology)
  • Prefer event-driven over synchronous
  • Use sagas for cross-service transactions
  • Accept eventual consistency

Get them wrong:

  • Services become chatty (defeats the purpose)
  • Every feature touches 10 services
  • Debugging becomes impossible
  • You've added microservices' complexity without benefits

The question isn't "should we use microservices?" It's "where are the natural boundaries in our business?"

Extract services along those boundaries. Services that cross boundaries are warning signs.


Up next: Consistency & Distributed Transactions — your services are decoupled and event-driven, but what happens when you need a guarantee that data is consistent across all of them? CAP theorem, ACID vs BASE, 2PC, and when to just accept eventual consistency.


Further Reading

  • Domain-Driven Design by Eric Evans (2003) — still the canonical reference
  • Building Microservices by Sam Newman (2nd ed.) — practical patterns
  • Designing Data-Intensive Applications by Martin Kleppmann — consistency, distributed systems theory
  • Enterprise Integration Patterns by Hohpe & Woolf — message-driven architecture
  • Temporal.io docs — saga orchestration framework (the tool that makes sagas practical)
R

Ravi Kant Shukla

Senior Java + AI engineer. 9+ years in system design, Kafka, microservices, and LLM/RAG pipelines.

Enjoyed this post?

Get more system design and AWS insights delivered weekly. No spam.

Comments (0)

Loading comments...

Leave a comment

Your email will not be displayed publicly.