DBT: Data Build Tool with Tristan Handy

A data warehouse serves the purpose of providing low latency queries for high volumes of data. A data warehouse is often part of a data pipeline, which moves data through different areas of infrastructure in order to build applications such as machine learning models, dashboards, and reports.

Modern data pipelines are often associated with the term “ELT” or Extract, Load, Transform. In the “ELT” workflow, data is taken out of a source such as a data lake, loaded into a data warehouse, and then transformed within the data warehouse to create materialized views on the data. Data warehouse queries are usually written in SQL, and for the last 50 years, SQL has been the primary language for executing these kinds of queries. 

DBT is a system for data modeling that allows the user to write queries that involve a mix of SQL and a templating language called Jinja. Jinja allows the analyst to blend imperative code along with the declarative SQL. Tristan Handy is the CEO of Fishtown Analytics, the company that created DBT, and he joins the show to discuss how DBT works, and the role it plays in modern data infrastructure.

Sponsorship inquiries: sponsor@softwareengineeringdaily.com

Transcript

Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.


Sponsors

G2i is a hiring platform run by engineers that matches you with React, React Native, GraphQL, and mobile engineers who you can trust. Whether you are a new company building your first product or an established company that wants additional engineering help, G2i has the talent you need to accomplish your goals. Go to softwareengineeringdaily.com/g2i

X-Team is a company that can help you scale your team with new engineers. X-Team has thousands of proven developers in over 50 countries ready to join your team. X-Team is able and ready to support a full range of team/project needs. If your development team could use some firepower via some of the top engineering talent in the world, visit x-team.com/sedaily.

ExtraHop is a modern security company that provides complete visibility into your infrastructure. ExtraHop lets you detect threats and respond quickly, giving you the deep context you need, as well as the ability to set up automated response workflows. At extrahop.com, you can see a demo for how ExtraHop works.

DataStax provides DataStax Enterprise, a powerful distribution of Cassandra, created by the team that has contributed the most to Cassandra. DataStax Enterprise enables teams to develop faster, scale further, achieve operational simplicity, ensure enterprise security, and run mixed workloads that work with latest Graph, Search, and Analytics technology—all running across the hybrid and multi-cloud. To learn more about Apache Cassandra and DataStax Enterprise, go to datastax.com/sedaily

Software Weekly

Software Weekly

Subscribe to Software Weekly, a curated weekly newsletter featuring the best and newest from the software engineering community.