Iceberg at Netflix and Beyond with Ryan Blue

Mar 7, 2024 by SED

Apache Iceberg is an open source high-performance format for huge data tables. Iceberg enables the use of SQL tables for big data, while making it possible for engines like Spark and Hive to safely work with the same tables, at the same time.

Iceberg was started at Netflix by Ryan Blue and Dan Weeks, and was open-sourced and donated to the Apache Software Foundation in November 2018. It has now been adopted at many other companies including Airbnb, Apple, and Lyft.

Ryan Blue joins the podcast to describe the origins of Iceberg, how it works, the problems it solves, collaborating with Apple and others to open-source it, and more.

This episode is hosted by Lee Atchison. Lee Atchison is a software architect, author, and thought leader on cloud computing and application modernization. His best-selling book, Architecting for Scale (O’Reilly Media), is an essential resource for technical teams looking to maintain high availability and manage risk in their cloud environments.

Lee is the host of his podcast, Modern Digital Business, an engaging and informative podcast produced for people looking to build and grow their digital business with the help of modern applications and processes developed for today’s fast-moving business environment. Listen at mdb.fm. Follow Lee at softwarearchitectureinsights.com, and see all his content at leeatchison.com.

Please click here to see the transcript of this episode.

Sponsorship inquiries: sponsor@softwareengineeringdaily.com

Thanks to our sponsors

This episode is brought to you by:

starburst.io/sedaily.
For data-driven companies, Starburst offers a full-featured data lake analytics platform. Our platform includes the capabilities needed to discover, organize, and consume data on a data lake without the need for time-consuming and costly migrations. Trusted by companies like Comcast, Grubhub, and Priceline, Starburst helps companies make better decisions faster.
Want to see Starburst in action? Try Starburst Galaxy today: starburst.io/sedaily.
miro.com/Podcast
Miro is one of the platforms that I think has been really creative about how it’s incorporating AI. I created a dependency mapping on a Miro board to help think through a new software system. The idea was to visualize direct and indirect dependencies, and refine the connections between them.
The AI features helped summarize and cluster the information, making it easier to understand and work with. I’m also impressed with Miro AI’s ability to auto-generate images from text. This has a lot of value for software and product teams that need to do dynamic brainstorming and idea development.
Find simplicity in your most complex projects with Miro. Your first three Miro boards are free when you sign up today at miro.com/Podcast.
softwareengineeringdaily.com/workos
WorkOS is a developer platform to make your app enterprise-ready. With a few simple APIs, you can immediately add common enterprise features like Single Sign-On, SAML, SCIM user provisioning, and more. Developers will find beautiful docs and SDKs that make integration a breeze. WorkOS is kind of like “Stripe for enterprise features.” WorkOS powers apps like Webflow, Hopin, Vercel, and more than 100 others. The platform is rock solid, fully SOC-2 compliant, and ready for even the largest enterprise environments. So what are you waiting for? Integrate WorkOS today and make your app enterprise-ready. To learn more and get started, go to softwareengineeringdaily.com/workos

starburst.io/sedaily.

miro.com/Podcast

softwareengineeringdaily.com/workos

Subscribe to the newsletter

Related episodes

Subscribe
to the newsletter