This is a great question. The main uses of messaging are: scaling, offloading work, integration, monitoring, event handling, routing, networking, push, mobility, buffering, queueing, task sharing, alerts, management, logging, batch, data delivery, pubsub, multicast, audit, scheduling, … and more. Basically: anything where you need data but don’t want to make a database request. (Caching is … Read more
Assuming both models are being considered to be implemented using one broker running, there’s little difference that I can see. Option 2 seems more common in the real world for solving this kind of routing problem (at least in my anecdotal experience) and it’s exactly the challenge that Topic Exchanges exist to solve. The only … Read more
You can actually retrieve this via the client. When you perform a queue_declare operation, RabbitMQ returns a tuple with three values: (<queue name>, <message count>, <consumer count>). The passive argument to queue_declare allows you to check whether a queue exists without modifying the server state, so you can use queue_declare with the passive option to … Read more
I have found this on the rabbitmq website it is near the bottom so I have quoted the relevant part below. The tl;dr version is that you should have 1 connection per application and 1 channel per thread. Connections AMQP connections are typically long-lived. AMQP is an application level protocol that uses TCP for reliable … Read more
The difference is that a PUB socket sends the same message to all subscribers, whereas PUSH does a round-robin amongst all its connected PULL sockets. In your example, if you send just a single message from the root, then all the subscribers will receive it (barring slow subscribers, etc.) but only 1 worker. The pub/sub … Read more
I wrote a blog post a while back comparing MSMQ and RabbitMQ (among others): http://mikehadlow.blogspot.co.uk/2011/04/message-queue-shootout.html RabbitMQ gave slightly better performance than MSMQ, but both were comprehensively out performed by ZeroMQ. If performance is your main criteria, you should definitely look at ZeroMQ. It’s worth noting that RabbitMQ and MSMQ are very different beasts. MSMQ is … Read more
The short answer is, no. What you’re looking for is exactly-once processing. While it may often seem feasible, it should never be relied upon because there are always caveats. Even in order to attempt to prevent duplicates you would need to use the simple consumer. How this approach works is for each consumer, when a … Read more
Apache ActiveMQ is a great workhorse full of features and nice stuff. It’s not the fastest MQ software around but fast enough for most use cases. Among features are flexible clustring, fail-over, integrations with different application servers, security etc. Apache Apollo is an attempt to write a new core for ActiveMQ to cope with a … Read more
Using Kafka for a task queue is a bad idea. Use RabbitMQ instead, it does it much better and more elegantly. Although you can use Kafka for a task queue – you will get some issues: Kafka is not allowing to consume a single partition by many consumers (by design), so if for example a … Read more
I would say that Gearman is better for queuing “jobs” and RabbitMQ is better for queuing “data”. Of course, they are both really the same thing, but the way it works out for me is that if you are trying to “fan out” work to be done, and the workers can work independently, Gearman is … Read more