Container Storage with Jie Yu

A database stores data to an underlying section of storage. If you are an application developer, you might think of your persistent storage system as being the database itself–but at a lower level, that database is writing to block storage, file storage, or object storage.

A container orchestration system manages application containers. If you want to run WordPress (a blogging platform) on Kubernetes, that means you also need to run a database to store your blog posts in a persistent way. To run a database, you need to have an underlying storage medium–which could be a disk that is at your on-prem data center, or block storage on a disk at a cloud provider.

Kubernetes is not the only container orchestrator. There’s also Cloud Foundry, Mesos, Docker Swarm, and several others. Each of these container orchestrators needs to be able to run a variety of persistent workloads (such as a MySQL database or a Kafka cluster). Each of these persistent workloads needs to be able to use different types of backing storage.

With the range of container orchestrators and the range of backing storage types, a problem arises. Every storage type would have to write custom code to connect to each container orchestrator.

The solution to this is the CSI: the container storage interface. The CSI is an interface layer between the container orchestrator and the backing storage system. In today’s episode, Jie Yu from Mesosphere describes the motivation for the CSI, and gives an overview for its design principles. There are great lessons here for anyone working with containers or distributed systems in general.

Transcript

Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Show Notes

Stripe: Designing Robust and Predictable APIs with Idempotency 

Sponsors

Citus is worry-free Postgres that is built to scale out. Made for SaaS and enterprises, Citus is an extension to Postgres that transforms Postgres into a distributed database. Whether you need to scale out a multi-tenant app—or are building real-time analytics dashboards that require sub-second responses—Citus makes it simple to shard Postgres. Go to citusdata.com/sedaily to learn more about how Citus can scale your Postgres database.

Rookout Rapid Production Debugging allows developers to track down issues in production without any additional coding, re-deployment or restarting the app. Insert Rookout ‘non-breaking breakpoints’ to immediately collect any piece of data from your live code and pipeline it anywhere — even if you never thought about it before or didn’t create instrumentation to collect it. Go to rookout.com/sedaily to start a free trial and see how much debugging time you can save.

Azure Container Service simplifies the deployment, management and operations of Kubernetes. Check out the Azure Container Service at aka.ms/sedaily. 

GoCD is a continuous delivery tool created by ThoughtWorks. It’s great to see the continued progress on GoCD with the new Kubernetes integrations–and you can check it out for yourself at gocd.org/sedaily.