Time Series Databases with Rob Skillington

A time series database is optimized for the storage of high volumes of sequential data across time.

Time series databases are often organized as columnar data stores that can write large volumes of data quickly. These systems can sometimes tolerate data loss, because the data they are gathering is used for monitoring and other applications that require aggregated data sets rather than highly important individual transactions.

The demand for time series databases has grown over the last decade with the rise of mobile devices and the decreasing cost of cloud storage. There has been an increase in the number of systems that require monitoring, and some of those systems produce an incredibly large amount of data, requiring compression, downsampling, and garbage collection.

Rob Skillington is an engineer at Uber, where he helped create M3DB, a time series database. In a previous show, Rob described the basics of M3DB and how it helps Uber with storing data from Prometheus, a monitoring system. In today’s show we discuss the field of time series databases, and Rob’s approach to building M3.

Sponsorship inquiries: sponsor@softwareengineeringdaily.com

ANNOUNCEMENTS

  • FindCollabs is a place to find collaborators and build projects. We recently launched GitHub integrations. It’s easier than ever to find collaborators for your open source projects. And if you are looking for some people to start a project with, FindCollabs we have topic rooms that allow you to find other people who are interested in a particular technology, so that you can find people who are curious about React, or cryptocurrencies, or Kubernetes, or whatever you want to build with.
  • Podsheets is an open source podcast hosting platform that we recently launched. We are building Podsheets with the learnings from Software Engineering Daily, and our goal is to be the best place to host and monetize your podcast. If you have been thinking about starting a podcast, check out podsheets.com.
  • New SEDaily app for iOS and for Android. It includes all 1000 of our old episodes, as well as related links, greatest hits, and topics. You can comment on episodes and have discussions with other members of the community. I’ll be commenting on each episode, so if you hear an episode that you have some commentary on, jump onto the app, or on SoftwareDaily.com to share your thoughts. And you can become a paid subscriber for ad free episodes at softwareengineeringdaily.com/subscribe. Altalogy is the company who has been developing much of the software for the newest app, and if you are looking for a company to help you with your mobile and web development, I recommend checking them out.   

Transcript

Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.