# Introduction to Message Brokers. Part 1: Apache Kafka vs RabbitMQ
URL:: https://freshcodeit.com/blog-introduction-to-message-brokers-part-1-apache-kafka-vs-rabbitmq
Author:: freshcodeit.com
## Highlights
> In previous times, you would use simple "in points" of the machine to machine communication. But nowadays we have special message brokers. They make the process of data exchange simple and reliable. These tools use different protocols that determine the message format. The protocols show how the message should be transmitted, processed, and consumed. ([View Highlight](https://read.readwise.io/read/01gce88hfpfq6txsgakpkkvvh0))
>  ([View Highlight](https://read.readwise.io/read/01gce88rfewdbq6a0nw65s579t))
> In 2010 [LinkedIn engineers](https://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying) faced the problem of integration huge amounts of data from their infrastructure into a lambda architecture. It also included Hadoop and real-time event processing systems.
> As for traditional message brokers, they didn't satisfy Linkedin needs. These solutions were too heavy and slow. So, the engineering team has developed the scalable and fault-tolerant messaging system without lots of bells and whistles. The new queue manager has quickly transformed into a full-fledged event streaming platform. ([View Highlight](https://read.readwise.io/read/01gce89hyprcygerayyhv6vysc))
> distributed commit log. ([View Highlight](https://read.readwise.io/read/01gce8dsw5rk7zdtcse2mf51v5))
> Apache Kafka is based on the *log data structure*. This means a log is a time-ordered, append-only sequence of data inserts. ([View Highlight](https://read.readwise.io/read/01gce8dwded2ee7j2y39bpwc1n))
> Combining the functions of messaging, storage, and processing, Kafka isn't a common message broker. It's a powerful event streaming platform capable of handling trillions of messages a day. Kafka is useful both for storing and processing historical data from the past and for real-time work. You can use it for creating streaming applications, as well as for streaming data pipelines. ([View Highlight](https://read.readwise.io/read/01gce8hj1y80hvsjw6dck97gcc))
> • Kafka doesn't maintain any indexes and doesn't allow random access (it just delivers the messages in order, starting with the offset) ([View Highlight](https://read.readwise.io/read/01gce8hyqt9062gwa9rgq4g6c7))
> I recommend you to use this multiple publish/subscribe and queueing tool, when you need to optimize processing really big amounts of data (100 000 messages per second and more). In this case, Apache Kafka will satisfy your needs. ([View Highlight](https://read.readwise.io/read/01gce8jdgv7dagweyk7vqp3j0b))
> Cons:
> • Non-transactional (by default)
> • Needs Erlang
> • Minimal configuration that can be done through code
> • Issues with processing big amounts of data ([View Highlight](https://read.readwise.io/read/01gce8jt02h040nhnh2h9dbmnt))
> RabbitMQ is known as a "traditional" message broker, which is suitable for a wide range of projects. ([View Highlight](https://read.readwise.io/read/01gce8k5ebj7akcxek7t9gj6az))
> In 2007, Rabbit Technologies Ltd. had developed the system, which originally implemented AMQP. It's an open wire protocol for messaging with complex routing features. AMQP ensured cross-language flexibility of using message broking solutions outside the Java ecosystem. In fact, RabbitMQ perfectly works with Java, Spring, .NET, PHP, Python, Ruby, JavaScript, Go, Elixir, Objective-C, Swift and many other technologies. The numerous plugins and libraries are the main advantage of the software. ([View Highlight](https://read.readwise.io/read/01gce8kdeb8xs9y1pyc2vd5brk))
> Being a broker-centric program, RabbitMQ gives guarantees between producers and consumers. If you choose this software, you should use transient messages, rather than durable. ([View Highlight](https://read.readwise.io/read/01gce8kpsqpw57xzrd3pzajaj6))
> As for me, I like RabbitMQ due to the opportunity to use many plugins. They save time and speed-up work. You can easily adjust filters, priorities, message ordering, etc. Just like Kafka, RabbitMQ requires you to deploy and manage the software. But it has convenient in-built UI and allows using SSL for better security. As for abilities to cope with big data loads, here RabbitMQ is inferior to Kafka. ([View Highlight](https://read.readwise.io/read/01gcep24hst3v348q3800zj1by))
---
Title: Introduction to Message Brokers. Part 1: Apache Kafka vs RabbitMQ
Author: freshcodeit.com
Tags: readwise, articles
date: 2024-01-30
---
# Introduction to Message Brokers. Part 1: Apache Kafka vs RabbitMQ
URL:: https://freshcodeit.com/blog-introduction-to-message-brokers-part-1-apache-kafka-vs-rabbitmq
Author:: freshcodeit.com
## AI-Generated Summary
The development of message brokers is especially important for data analytics and business intelligence. We will look at 2 big data tools: Apache Kafka and Rabbit MQ.
## Highlights
> In previous times, you would use simple "in points" of the machine to machine communication. But nowadays we have special message brokers. They make the process of data exchange simple and reliable. These tools use different protocols that determine the message format. The protocols show how the message should be transmitted, processed, and consumed. ([View Highlight](https://read.readwise.io/read/01gce88hfpfq6txsgakpkkvvh0))
>  ([View Highlight](https://read.readwise.io/read/01gce88rfewdbq6a0nw65s579t))
> In 2010 [LinkedIn engineers](https://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying) faced the problem of integration huge amounts of data from their infrastructure into a lambda architecture. It also included Hadoop and real-time event processing systems.
> As for traditional message brokers, they didn't satisfy Linkedin needs. These solutions were too heavy and slow. So, the engineering team has developed the scalable and fault-tolerant messaging system without lots of bells and whistles. The new queue manager has quickly transformed into a full-fledged event streaming platform. ([View Highlight](https://read.readwise.io/read/01gce89hyprcygerayyhv6vysc))
> distributed commit log. ([View Highlight](https://read.readwise.io/read/01gce8dsw5rk7zdtcse2mf51v5))
> Apache Kafka is based on the *log data structure*. This means a log is a time-ordered, append-only sequence of data inserts. ([View Highlight](https://read.readwise.io/read/01gce8dwded2ee7j2y39bpwc1n))
> Combining the functions of messaging, storage, and processing, Kafka isn't a common message broker. It's a powerful event streaming platform capable of handling trillions of messages a day. Kafka is useful both for storing and processing historical data from the past and for real-time work. You can use it for creating streaming applications, as well as for streaming data pipelines. ([View Highlight](https://read.readwise.io/read/01gce8hj1y80hvsjw6dck97gcc))
> • Kafka doesn't maintain any indexes and doesn't allow random access (it just delivers the messages in order, starting with the offset) ([View Highlight](https://read.readwise.io/read/01gce8hyqt9062gwa9rgq4g6c7))
> I recommend you to use this multiple publish/subscribe and queueing tool, when you need to optimize processing really big amounts of data (100 000 messages per second and more). In this case, Apache Kafka will satisfy your needs. ([View Highlight](https://read.readwise.io/read/01gce8jdgv7dagweyk7vqp3j0b))
> Cons:
> • Non-transactional (by default)
> • Needs Erlang
> • Minimal configuration that can be done through code
> • Issues with processing big amounts of data ([View Highlight](https://read.readwise.io/read/01gce8jt02h040nhnh2h9dbmnt))
> RabbitMQ is known as a "traditional" message broker, which is suitable for a wide range of projects. ([View Highlight](https://read.readwise.io/read/01gce8k5ebj7akcxek7t9gj6az))
> In 2007, Rabbit Technologies Ltd. had developed the system, which originally implemented AMQP. It's an open wire protocol for messaging with complex routing features. AMQP ensured cross-language flexibility of using message broking solutions outside the Java ecosystem. In fact, RabbitMQ perfectly works with Java, Spring, .NET, PHP, Python, Ruby, JavaScript, Go, Elixir, Objective-C, Swift and many other technologies. The numerous plugins and libraries are the main advantage of the software. ([View Highlight](https://read.readwise.io/read/01gce8kdeb8xs9y1pyc2vd5brk))
> Being a broker-centric program, RabbitMQ gives guarantees between producers and consumers. If you choose this software, you should use transient messages, rather than durable. ([View Highlight](https://read.readwise.io/read/01gce8kpsqpw57xzrd3pzajaj6))
> As for me, I like RabbitMQ due to the opportunity to use many plugins. They save time and speed-up work. You can easily adjust filters, priorities, message ordering, etc. Just like Kafka, RabbitMQ requires you to deploy and manage the software. But it has convenient in-built UI and allows using SSL for better security. As for abilities to cope with big data loads, here RabbitMQ is inferior to Kafka. ([View Highlight](https://read.readwise.io/read/01gcep24hst3v348q3800zj1by))