What's your current take on queues and event-driven architecture in general?
I have been thinking a lot recently about message queues, Pubsub systems, and event-driven architecture.
The main reason is that our company unfortunately has too many services most of which shouldn't have been a separate service. We created them back when the hype of microservices was at its peak without any good cost analysis.
As a result, those services gradually started to communicate with each other thus introducing potential cascading failure chains.
The immediate lesson here is not to fall prey to the hype of microservices without a good reason. In our case, most of them might have mad a decent modular monolith.
But let's take it as a given. A standard recipe for decoupling your interservice communication is using messaging.
I am a huge fan of Postgres for almost everything and also simplicity so I use Postgres queues and it has worked very well so far.
We also use Google Pubsub for one-to-many cases which makes sense and also works well.
I have zero experience with Kafka or RabbitMQ even though I have read about them and even learned how they work.
I just wasn't lucky enough to have a system big enough that I need a dedicated message queue (not Postgres) or Kafka.
Tell about your experience with event-driven architecture, messaging, queues and how you decide when you must use them.
I am a huge fan of simple architectures, but too much simplicity is also a thing so I'd appreciate your success (or failure) stories with this type of systems.