Nubank Data Engineering with Sujith Nair

Nubank is a popular bank that is based in Brazil. Nubank has more than 20 million customers, and has accumulated a high volume of data over the six years since it was started.

Mobile computing and cloud computing have given rise to “challenger banks” that operate more like software companies. When a software company reaches the size that Nubank is at today, it needs a data platform.

A data platform is a collection of different technologies that move data into different storage formats and applications, so that different members of an organization can access that data. New data often enters an organization through an OLTP database, which supports user transactions. That data is copied into a data lake, which provides cheap bulk storage. From the data lake, the data is moved into a data warehouse system for fast access. Along the way, tools like Kafka, Spark, and S3 are used to implement the needs of the data platform.

Data platform architecture is not an exact science. Different companies build their data platform based on their own unique requirements. Previous shows have covered the data infrastructure companies like Lyft, Uber, and Facebook. Today’s show is another case study in data infrastructure, with a modern bank.

In a previous episode, we covered the engineering of Nubank. Sujith Nair from Nubank joins today’s show to talk about the data infrastructure of the company. 

If you enjoy the show, you can find all of our past episodes about data infrastructure by going to SoftwareDaily.com and searching for the technologies or companies mentioned. And if there is a subject that you want to hear covered, feel free to leave a comment on the episode, or send us a tweet @software_daily.

Sponsorship inquiries: sponsor@softwareengineeringdaily.com

Transcript

Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.


Sponsors

Seen by Indeed is a tech-focused matching platform. Every Seen candidate also gets free access to technical career coaching, resume reviews, mock interviews, and even salary negotiation tips to seal the deal. If you are ready for a new job, you are ready for Seen by Indeed. You  Join today and get a free resume review when you go to beseen.com/daily.

StrongDM is a system for managing and monitoring access to servers, databases, and kubernetes clusters.  You already treat infrastructure as code; strongDM lets you do the same with access. Start your free 14 day trial of strongDM at: softwareengineeringdaily.com/strongdm

From their recent report on serverless adoption and trends, Datadog found half of their customer base using EC2s have now adopted AWS Lambda. You can easily monitor all your serverless functions in one place and generate serverless metrics straight from Datadog. Check it out yourself by signing up for a free 14-day trial and get a free t-shirt at softwareengineeringdaily.com/datadogtshirt

With Triplebyte, you do one online interview, and then you get to go straight to final interviews at hundreds of companies (from tech giants like Dropbox to exciting startups). It’s like the Common App for software engineers. No resume needed. Apply now at triplebyte.com/sedaily. If you take a job through Triplebyte, you’ll get a $1000 signing bonus.

Software Weekly

Software Weekly

Subscribe to Software Weekly, a curated weekly newsletter featuring the best and newest from the software engineering community.