Apache Kafka

This article is part 2 in a series about LinkedIn’s data journey. You can read the first part of LinkedIn’s data infrastructure here.  It requires repeating that LinkedIn is a

Apache Kafka was created at LinkedIn. Kafka was open sourced in 2011, when the company was eight years old. By that time, LinkedIn had developed a social network with millions of users.

A new software product usually starts with a single database. That database manages the tables for user accounts and basic transactions. When a product becomes popular, the database

Ever since Apache Kafka was open sourced from LinkedIn, it has been used to solve a wide variety of problems in distributed systems and data engineering. Kafka is a distributed messaging

Kafka has become a central tool for data at many large organizations. At data-intensive companies like Fiverr and Netflix, Kafka is used simultaneously as: a database a queue for ordered