Kafka Applications with Tim Berglund

Ever since Apache Kafka was open sourced from LinkedIn, it has been used to solve a wide variety of problems in distributed systems and data engineering.

Kafka is a distributed messaging queue that is used by developers to publish messages and subscribe to topics with a certain message type. Kafka allows information to flow throughout a company such that multiple systems can consume the messages from a single sender. 

In previous shows, we have covered design patterns within Kafka, Kafka streams, event sourcing with Kafka, and many other subjects relating to the technology. Kafka is broadly useful, and new strategies for using Kafka continue to emerge as the open source project develops new functionality and becomes a platform for data applications.

In today’s episode, Tim Berglund returns to Software Engineering Daily for a discussion of how applications are built today using Kafka–including systems that are undergoing a refactoring, data engineering applications, and systems with a large number of communicating services.

If you are interested in learning more about how companies are using Kafka, the Kafka Summit in San Francisco is September 30th – October 1st. Companies like LinkedIn, Uber, and Netflix will be talking about how they use Kafka. Full disclosure: Confluent (the company where Tim works) is a sponsor of Software Engineering Daily.

Sponsorship inquiries: sponsor@softwareengineeringdaily.com

Check out our active companies and projects:

• FindCollabs is a place to find collaborators and build projects. Find a project to work on
• Podsheets is an open source podcast hosting platform built with the learnings from Software Engineering Daily. Our goal is to be the best place to host and monetize your podcast. If you have been thinking about starting a podcast, check out podsheets.com.
• The SEDaily app for iOS and Android includes all 1000 of our old episodes, as well as related links, greatest hits, and topics. Subscribe for ad-free episodes.

Transcript

Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.