FaunaDB with Evan Weaver
Twitter’s early engineers faced scalability problems that caused infrastructure failures on a regular basis. The infamous “fail whale” could happen as a result of problems in the application servers, the network, or the database layer.
When Twitter was scaling in its early days, the cloud providers were still immature. Engineers did not have access to the autoscaling cloud infrastructure that is available today. The early Twitter architecture was a combination of open source tools and internally created infrastructure custom built for Twitter’s workloads.
Evan Weaver was an early engineer at Twitter, and he saw the deficiencies of the data tools that the company had access to. Twitter engineers wanted access to a truly reusable data platform that would fit Twitter’s requirements: high availability, globally replicated, and transactionally consistent.
By 2012, Evan had left Twitter and started consulting for other technology companies. He found that databases across the industry were lacking the same properties that Twitter wanted, and the ideas for FaunaDB began to percolate. Around this time, there were two relevant papers about distributed databases that had come out: the Spanner paper from Google and the Calvin paper, a distributed systems paper from Yale.
With inspiration from the literature, his time at Twitter, and his knowledge from consulting, Evan started FaunaDB. Seven years later, FaunaDB is a fully fledged company with a cloud service offering. Fauna is an OLTP database used by companies like Nvidia, Nextdoor, and Capital One.
Evan joins the show to talk about his time spent scaling Twitter and the architecture of FaunaDB.
Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.
MongoDB is the most popular nonrelational database. MongoDB Stitch is a serverless platform from MongoDB, that allows you to build rich interactions with your database. To try it out yourself today, experiment with $10 in free credit by going to mongodb.com/sedaily.
Datadog is a cloud-scale monitoring platform for infrastructure and applications. And with Datadog’s new Live Container view, you can see every container’s health, resource consumption, and running processes in real time. See for yourself by starting a free trial and get a free Datadog T-shirt! softwareengineeringdaily.com/
Logi Analytics provides embedded business intelligence that goes beyond dashboards and reports. Create, deploy, and constantly improve analytic applications that engage users and drive revenue. You focus on building the best applications for your users, while Logi gets you there faster and keeps you competitive. Logi is a powerful yet nearly invisible business intelligence solution. Check it out by going to logianalytics.com/sedaily.