Kafka in the Cloud with Neha Narkhede

Apache Kafka is an open-source distributed streaming platform. Kafka was originally developed at LinkedIn, and the creators of the project eventually left LinkedIn and started Confluent, a company that is building a streaming platform based on Kafka.

Kafka is very popular, but is not easy to deploy and operationalize. That is why Confluent has built a Kafka-as-a-service product, so that managing Kafka is not the job of an on-call DevOps engineer.

Neha Narkhede is the CTO of Confluent and she has been on the show twice before to discuss Kafka. In our last episode, we discussed event sourcing and CQRS with Kafka. In this episode, we explore more common enterprise uses for Kafka, and Neha talks about the engineering complexities of building a managed Kafka-as-a-service product.

Transcript

Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Sponsors


To build the kinds of things developers want to build today, they need better tools.  Amazon Aurora from Amazon Web Services is a relational database engine that’s compatible with MySQL and PostgreSQL, and provides up to five times the performance on the same hardware—at a tenth of the cost. Check it out at aurora.aws.


Toptal is the best place to find reasonably priced, extremely talented software engineers to build your projects from scratch or scale your workforce. Get a free pair of Apple Airpods when you use Toptal.com/sedaily to work with an engineer for at least 20 hours.


Have you been thinking you’d be happier at a new job? If you’re dreaming about a new job and have been waiting for the right time to make a move, go to hired.com/sedaily. Hired makes finding work enjoyable. Hired uses an algorithmic job-matching tool in combination with a talent advocate who will walk you through the process of finding a better job. Check out hired.com/sedaily to get a special offer for Software Engineering Daily listeners–a $600 signing bonus from Hired when you find that great job that gives you the respect and salary that you deserve as a talented engineer. 


Incapsula is a cloud service that protects applications from attackers and improves performance.  Botnets and denial-of-service attacks are recognized by Incapsula and blocked. This protects your API servers and microservices from responding to unwanted requests. To try Incapsula, go to incapsula.com/sedaily and get a month free for Software Engineering Daily listeners.