Apache Kafka is an open-source Java/Scala distributed occasion streaming platform for high-performance knowledge pipelines, streaming analytics, knowledge integration, and mission-critical purposes. As I have defined, one draw back of Kafka is that establishing giant Kafka clusters could be tough. One other draw back is that Kafka makes use of the Java digital machine (JVM), which introduces lag due to reminiscence rubbish assortment. Including much more complexity, Kafka has till lately required Apache ZooKeeper for distributed coordination, and it requires a separate schema registry course of.
Redpanda (beforehand known as Vectorized) is a Kafka plug-in substitute written primarily in C++ utilizing the Seastar asynchronous framework, and the Raft consensus algorithm for its distributed log. Redpanda doesn’t require utilizing ZooKeeper or the JVM, and its supply is on the market on GitHub beneath the Enterprise Supply License (BSL). It is not technically open supply as outlined by the Open Software program Basis, however that does not matter to me as a result of I’ve no plans to supply Redpanda as a service.
Redpanda vs. Kafka
As you would possibly anticipate from the reimplementation in C++, Redpanda has considerably decrease latency and better efficiency than Kafka. It is also a lot simpler to put in and tune.
Determine 1 exhibits latency charts for Redpanda and Kafka. The left-hand chart exhibits common latency versus time, and the right-hand chart exhibits latency versus percentile. Redpanda’s caption is not precisely false, however it does exaggerate. I might rephrase it and say that Kafka’s common latency is 6 to 10 occasions larger than Redpanda’s, and that Kafka’s tail latency is as much as 40 occasions larger than Redpanda’s.
