← All articles

Microservices That Stay in Sync Without a Shared Database (Redis Streams + Local Replicas)

By Shuvo Haldar5 min read
MicroservicesRedisArchitecturePostgreSQLDistributed Systems
Microservices That Stay in Sync Without a Shared Database (Redis Streams + Local Replicas)

The moment you split a monolith into services, you hit the same wall everyone hits: the exam service needs to know a user's name, the notification service needs their email, but the only service that owns user data is the account service. The lazy answer is to let everyone query the account database. Do that and you don't have microservices — you have a distributed monolith with extra network hops and a single database that can take everything down at once.

On iamwithtrees, a multi-service platform I built solo, I set a hard rule: a service never touches another service's database. Ever. Here's how the services still agree on who a user is.

Animated diagram: the account service publishes a USER_SYNC event to the stream:user Redis stream, which fans out to the study, notification, and moderation services, each upserting its own local user replica.
One published event fans out through the stream; every service upserts its own replica, then acks.

The rule that makes the rest easy

Each service has its own PostgreSQL database. The account service owns the real user record. Every other service that needs user data keeps a local replica — a small users table with just the fields it actually uses — and updates it by listening to events, not by querying across the boundary.

So when the study service renders a leaderboard, it reads names from its own users table. It never phones the account service. That table is a cache that's kept honest by a stream.

Redis Streams, not pub/sub

I use Redis Streams with consumer groups for the event backbone, and the "consumer group" part is the whole reason. Plain pub/sub drops messages if a subscriber is offline — if the notification service is restarting when a user signs up, that event is gone and the replica is permanently wrong. Streams persist the event and track a per-group cursor, so a consumer that was down reads what it missed when it comes back.

// Account service: publish once when a user changes.
await redis.xadd(
  "stream:user",
  "*",
  "type", "USER_SYNC",
  "payload", JSON.stringify({ id: user.id, name: user.name, email: user.email }),
);
// Every other service: one durable consumer per group, reading its own cursor.
async function consumeUserStream(service: string) {
  const group = `${service}:user`;
  while (true) {
    const res = await redis.xreadgroup(
      "GROUP", group, `${service}-1`,
      "COUNT", 50, "BLOCK", 5000,
      "STREAMS", "stream:user", ">",
    );
    for (const [, entries] of res ?? []) {
      for (const [id, fields] of entries) {
        await upsertUserReplica(parse(fields));   // update MY local users table
        await redis.xack("stream:user", group, id); // only ack after it's applied
      }
    }
  }
}
Diagram of the consume loop: XREADGROUP reads from the per-group cursor, the replica is upserted idempotently, then XACK advances the cursor; a crash between the upsert and the ack causes redelivery and reapplication. A side panel lists data safe to replicate (names, profiles) versus data that must never be replicated (authorization, money).
The upsert happens before the ack, so a crash just means the event is redelivered — which is why the write has to be idempotent.

The important line is the order: I upsert the replica first, then xack. If the process dies between reading and acking, the event isn't lost — it's redelivered to the group and reapplied. That's why the upsert has to be idempotent: applying the same USER_SYNC twice must produce the same row. Get that one property right and the whole system tolerates crashes, restarts, and redeploys without drift.

What you trade away

This is eventual consistency, and I want to be honest about the cost. There's a gap — usually tiny — between a user changing their name and every service's replica catching up. For a name on a leaderboard, that's completely fine. For something that must be exactly current at read time (a permission check, say), I don't rely on a replica; the owning service is asked directly, or the check lives where the data lives.

Knowing which data can be eventually consistent and which can't is the actual skill here. Profile fields, display names, avatars: replicate them, let them lag a beat. Money and authorization: don't.

Why I'd choose this again

I built this alone, and the replica-plus-stream pattern is a big reason a one-person team could run company-shaped infrastructure. Each new service I added — study, then notifications, then moderation — plugged into the same stream:user with its own consumer group and its own replica. No service redeployed because another one launched. Adding capability never meant touching the identity core.

The alternative — synchronous cross-service calls for user data — would have coupled every service's uptime to the account service's uptime and turned every page render into a fan-out of network requests. Streams plus local replicas cost me a little consistency lag and bought me genuine independence.

Takeaways

  • No service touches another service's database. Give each one a local replica of the data it needs.
  • Sync replicas over Redis Streams with consumer groups so events survive a consumer being offline — plain pub/sub won't.
  • Apply, then ack, and make the apply idempotent — redelivery is a feature, not a bug.
  • Replicate what can lag (names, profiles); never replicate what must be exact right now (auth, money).

Independence between services isn't free, but eventual consistency on the right data is a bargain — and it's what lets a small team ship a big system.


I build backend and distributed systems end-to-end. If you're working on something similar, let's talk.

Building something similar?

I'm a senior full-stack & DevOps engineer available for full-time roles and projects worldwide.