Our rabbitmq server can’t keep up with connections when we have spikes above 3000 connections/second, and new connections take long to stablish (reaching 3s limit we have for failing). We need publishers to return fast as any delay transmit to user experience.
We are currently looking at coding a load balance of connections to a second rabbitmq server as a fast temporal solution.
We only create the connection when we need to publish, and we reuse the same connection and channel for the same web request/process as best practices suggest. The nature of the PHP process is to die after the web request.
Can I reuse the connection to publish between web requests?
Is a 3000 connection/s spike enough to make the rabbitmq server suffer from a tpc connection overload?
What direction should we take when we have 10x this connections in the future?
Thank you.
Advertisement
Answer
You can use the amqproxy. It will let you re-use connections.
If you want to “10x” this you can create edge-node RabbitMQ servers that uses a shovel to move the messages from the edge nodes to a Central cluster where consumers are connected.