Database Performance and Optimization with Andrew Davidson

When a database gets large, it can start to perform poorly. This can manifest in slow query speed. You can speed up a query by defining an index, which is a data structure that allows for faster access to the data that is being indexed. As a consequence, whenever you update the database, you will now need to update the index with that new piece of data.

The more you index your data, the faster the access time. In order to have more indexes you must pay a write penalty in order to maintain consistency around that data, since the indexes need to be updated with each new entry. This illustrates one simple tradeoff that a developer can make within a database deployment.

Why are there so many different databases in the world? Why do we need SQL databases like Postgres, document databases like MongoDB, key/value systems like Cassandra, and search systems like Elasticsearch? Because each of these each system optimizes for different sets of tradeoffs. Tradeoffs can affect the speed of a read, the speed of a write, the user experience, the consistency of data, and the cost of running the database.

Andrew Davidson is the lead product manager of MongoDB Atlas. Andrew joins the show to talk about how database performance can degrade when a database gets large, and how to measure and optimize performance of a critical database.

Andrew explores the range of distributed systems cases–from a single node database to a multi-geographic distribution of nodes around the world, and describes how the configuration of a database in the cloud can help or hurt the application that the database is serving.
Full disclosure: MongoDB is a sponsor of Software Engineering Daily.

Transcript

Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.


Sponsors

QCon San Francisco 2018 features 18 editorial tracks with 140+ speakers from places like Uber, Google, Dropbox, Slack, Twitter, and more. At QCon, we create a platform for senior software engineers, team leads, architects, and leaders working at innovator and early adopter companies to share their stories. It goes to the heart of who we are. We simply prefer practitioners over evangelists in the speakers we bring to the conference. SED listeners can save $100 off the price of a ticket using the promo code SED100.

Accenture is hiring software engineers and architects skilled in modern cloud native tech. If you’re looking for a job, check out open opportunities at  softwareengineeringdaily.com/accenture. Working with over 90% of the Fortune 100 companies, Accenture is creating innovative, cutting-edge applications for the cloud, and they are the number one integrator for Amazon Web Services, Microsoft Azure, Google Cloud Platform, and more.

Digital Ocean is the easiest cloud platform to run and scale your application. Try it out today and get a free $100 credit–go to do.co/sedaily. Digital Ocean is a complete cloud platform to help developers and teams save time when running and scaling their applications.

Gremlin provides resilience as a service, using chaos engineering techniques pioneered at Netflix and Amazon. Prepare your team for disaster by proactively testing failure scenarios. Check out Gremlin and get a free demo by going to gremlin.com/sedaily.

Software Weekly

Software Weekly

Subscribe to Software Weekly, a curated weekly newsletter featuring the best and newest from the software engineering community.