Podcast: Play in new window | Download
A new programmer learns to build applications using data structures like a queue, a cache, or a database. Modern cloud applications are built using more sophisticated tools like Redis, Kafka, or Amazon S3. These tools do multiple things well, and often have overlapping functionality. Application architecture becomes less straightforward.
The applications we are building today are data-intensive rather than compute-intensive. Netflix needs to know how to store and cache large video files, and stream them to users quickly. Twitter needs to update user news feeds with a fanout of the president’s latest tweet. These operations are simple with small amounts of data, but become complicated with a high volume of users.
Martin Kleppmann is the author of Data Intensive Applications, an O’Reilly book about how to use modern data tools to solve modern data problems. His book includes high-level discussions about architectural strategy, and lower level discussions like how leader election algorithms can create problems for a data intensive application.
If you are interested in hosting a show for Software Engineering Daily, we are looking for engineers, journalists, and hackers who want to work with us on content. It is a paid opportunity. Go to softwareengineeringdaily.com/host to find out more.
The Software Engineering Daily store is now open if you want to buy a Software Engineering Daily branded t-shirt, hoodie, or mug and support the show.
Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.
Pingback: Last week in Stream Processing & Analytics – 10.5.2017 | Enjoy IT - SOA, Java, Event-Driven Computing and Integration()
Hi this was a really excellent episode. Question please the linkedin profile was said to best be modelled by a document because a single linked in profile has multiple jobs. However I see it as relational because they uniquely identify each job so it’s like a reference to a company and a reference to university what do you think?
Document is probably easier in this case. You can’t know beforehand how many jobs a person has, so instead of having to do `job_1`, `job_2`, etc in a relational db, you can just specify a `jobs` array in the document and add as many items as needed.
In this case you would normally (imo, not a dba) have a job table where one field points back to the person table and one field points to company table… Standard stuff really.