PlanetScale: Sharded Database Management with Jiten Vaidya and Dan Kozlowski
In the early days of YouTube, there were scalability problems with the MySQL database that hosted the data model for all of YouTube’s videos. The state of the art solution to scaling MySQL at the time was known as “application-level sharding.”
To scale a database using application-level sharding, you break up the database into shards–disjoint regions of data. When you want to query the database, you need know which shard to query. In your application code, you have to issue the query to a specific shard.
The solution of application-level sharding does scale your database. But the downside is that every application that interfaces with the database now has to include code that is aware of the sharding schema.
If you are an application engineer, you don’t want to have to worry about the way that the database is sharded, because it adds significant complexity to your code. The engineers at YouTube decided to fix this problem with a project called Vitess. Vitess abstracts away the details of sharding by orchestrating reads and writes across the distributed database.
In a previous episode, we covered the architecture, read and write path, and the story of Vitess in detail. In today’s episode, Jiten Vaidya and Dan Kozlowski of PlanetScale Data join the show to give their perspective on MySQL scalability, and their work taking Vitess to market as a solution to scaling relational databases.
Transcript
Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.
