Well, I’ve been investigating these systems for a few months, and I don’t think they differ profoundly in their use cases. I think it’s best to compare them along these lines instead: Age: Storm is the older project, and the original one in this space, so it’s generally more mature and battle-tested. Samza is a … Read more
To answer your question: when shouldn’t you use Trident? Whenever you can afford not to. Trident adds complexity to a Storm topology, lowers performance and generates state. Ask yourself the question: do you need the “exactly once” processing semantics of Trident or can you live with the “at least once” processing semantics of Storm. For … Read more
You use Apache Kafka as a distributed and robust queue that can handle high volume data and enables you to pass messages from one end-point to another. Storm is not a queue. It is a system that has distributed real time processing abilities, meaning you can execute all kind of manipulations on real time data … Read more
Disclaimer: I’m an Apache Flink committer and PMC member and only familiar with Storm’s high-level design, not its internals. Apache Flink is a framework for unified stream and batch processing. Flink’s runtime natively supports both domains due to pipelined data transfers between parallel tasks which includes pipelined shuffles. Records are immediately shipped from producing tasks … Read more