Kubernetes Vision with Brendan Burns

Kubernetes has become the standard system for deploying and managing clusters of containers. But the vision of the project goes beyond managing containers. The long-term goal is to democratize the ability to build distributed systems.

Brendan Burns is a co-founder of the Kubernetes project. He recently announced an open source project called Metaparticle, a standard library for cloud-native development:

Metaparticle builds on top of Kubernetes primitives to make distributed synchronization easier… It supplies language independent modules for locking and leader election as easy-to-use abstractions in familiar programming languages.

After decades of distributed systems research and application, patterns have emerged about how we build these systems. We need a way to lock a variable, so that two nodes will not be able to write to that variable in a nondeterministic fashion. We need a way to do master election, so that if the master node dies, the other nodes can pick a new node to orchestrate the system.

We know that just about every distributed application needs locking and leader election–so how can we build these features directly into our programming tools, rather than bolting them on?

With Kubernetes providing a standard operating system for distributed applications, we can start to build standard libraries that assume we have access to underlying Kubernetes primitives. Instead of calling out to external tools like Zookeeper and etcd, a standard library like Metaparticle will abstract them away.

An example: if I am writing a system to do distributed mapreduce, I would like to avoid thinking about node failures and race conditions. Brendan’s idea is to push those problems down into a standard library–so the next developer who comes along with a new idea for a multi-node application has an easier time.

Brendan Burns currently works as a distinguished engineer at Microsoft, and he joins the show to discuss why it is still hard to build distributed systems and what can be done to make it easier. This is the second time we have had Brendan on the show. The first time he came on, he discussed the history of Kubernetes, and some of the design decisions of the system. This episode was more about the future. Full disclosure: Microsoft (where Brendan is employed) is a sponsor of Software Engineering Daily.

Show Notes

Get a free digital copy of Brendan’s new O’Reilly book, Designing Distributed Systemshttp://aka.ms/sedaily


Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.


Azure Container Service simplifies the deployment, management and operations of Kubernetes. Eliminate the complicated planning and deployment of fully orchestrated containerized applications with Kubernetes. You can quickly provision clusters to be up and running in no time, while simplifying your monitoring and cluster management through auto upgrades and a built-in operations console. Avoid being locked into any one vendor or resource. You can continue to work with the tools you already know, such as Helm, and move applications to any Kubernetes deployment. Integrate with your choice of container registry, including Azure Container Registry. Also, quickly and efficiently scale to maximize your resource utilization without having to take your applications offline. Isolate your application from infrastructure failures and transparently scale the underlying infrastructure to meet growing demands—all while increasing the security, reliability, and availability of critical business workloads with Azure. Check out the Azure Container Service at aka.ms/sedaily.

An in-house team of engineers spent thousands of hours developing the Casper mattress. As a software engineer, you know what that kind of development and dedication is like. The result is an exceptional product, one that you’d use and recommend to your friends. You deserve an exceptional night’s rest so you can continue building great software. Casper combines supportive memory foams for a sleep surface that’s got just the right sink and just the right bounce. Plus, its breathable design sleeps cool to help you regulate your temperature through the night. And, buying a Casper mattress is completely risk free. Casper offers free delivery and free returns with a 100-night home trial. If you don’t love it, they’ll pick it up and give you a full refund. As a special offer to Software Engineering Daily listeners, get $50 toward select mattresses by visiting casper.com/sedaily and using code SEDAILY at checkout. Terms and conditions apply.

Simplify continuous delivery with GoCD, the on-premise, open source, continuous delivery tool by ThoughtWorks. With GoCD, you can easily model complex deployment workflows using pipelines and visualize them end-to-end with the Value Stream Map. You get complete visibility into and control of your company’s deployments. At gocd.org/sedaily, find out how to bring continuous delivery to your teams. Say goodbye to deployment panic and hello to consistent, predictable deliveries. Visit gocd.org/sedaily to learn more about GoCD. Commercial support and enterprise add-ons, including disaster recovery, are available.

Software Weekly

Software Weekly

Subscribe to Software Weekly, a curated weekly newsletter featuring the best and newest from the software engineering community.