Modern backend systems inevitably evolve into microservices. But ensuring data consistency across services at scale quickly becomes messy, especially in high-throughput systems like UPI payments processing realtime double digit million payments.
Synchronous communication leads to cascading failures.
Message queues promise decoupling—but introduce delivery guarantees, retries, and operational overhead.
The industry-standard fix? The `Outbox Pattern`.
But in practice, it brings its own challenges:
- Extra workers and cron jobs
- Duplicate handling
- Cleanup and storage overhead
- Increased database load
- How to tie message queue to db transaction?
In this talk, we explore a Postgres-native approach using `pg_logical_emit_message`, a lesser-known capability that allows emitting events directly from the database with ACID guarantees.
We'll cover:
- Why traditional outbox implementations break down at scale
- How logical messages work under the hood
- A production-ready architecture using Postgres as the event source
- Trade-offs, failure modes, and when *not* to use this approach
If you're building high-scale systems and tired of maintaining outbox pipelines, this talk offers a simpler, more reliable pattern - using the good old open source postgres.
How we removed an entire class of infra (outbox workers, retries, cleanup) from our system
The hidden cost of "industry standard" patterns—and how to avoid them
What actually matters for reliability at scale (beyond theory and system design interviews)
Production lessons from handling millions of transaction-driven events
Hey,
The proposal is well-written and shares an interesting idea. We would love to have you at the meetup. Looking forward to seeing you.
Thanks,
Bangalore FOSSUnited team.