Data Warehousing with Mark Rittman

http://traffic.libsyn.com/sedaily/data-warehousing_edited.mp3Podcast: Play in new window | Download In the mid 90s, data warehousing might have meant “using an Oracle database.” Today, it means a wide variety of things. You could be stitching together a big data pipeline using Kafka, Hadoop, and Spark. You could be using managed tools like BigQuery from Google. How did we get from the simple days of Oracle databases to the wealth of options available today?

Continue reading…

Convergence with Haseeb Qureshi

http://traffic.libsyn.com/sedaily/Convergence_edited.mp3Podcast: Play in new window | Download At the first Software Engineering Daily Meetup, the speakers explored a range of topics. A few weeks ago, we published Courtland Allen’s talk about how to build a small software business. In today’s episode, we are publishing Haseeb Qureshi’s talk, called “Everything That Rises Must Converge: Why Engineers Disagree About Everything (And Why Fraudsters Do Too).” This talk explores philosophy, poker, software engineering,

Continue reading…

Security Language with Jean Yang

http://traffic.libsyn.com/sedaily/SecureProgramming.mp3Podcast: Play in new window | Download Security vulnerabilities are an important concern in systems. When we specify that we want certain information hidden, for example our phone number or our date of birth, we expect the system to hide the information. However, this doesn’t always happen due to human error in the code because programmers have to write checks and filters across the program. In this episode, Edaena Salinas

Continue reading…

RealmDB with Brian Munkholm

http://traffic.libsyn.com/sedaily/realmdb_edited.mp3Podcast: Play in new window | Download Expectations for mobile apps have gone up steadily since the iPhone was released. But the choice of databases built for mobile apps has remained limited mostly to SQLite. RealmDB was created as a new option for mobile developers on iOS, Android, or any other mobile platform.   Realm is not just a database. It is a database platform, offering a variety of systems

Continue reading…

Adchain with Ken Brook

http://traffic.libsyn.com/sedaily/ADChain.mp3Podcast: Play in new window | Download Online advertising is a system of transactions that involve many different players. The user visits a publisher’s website; the publisher notifies an exchange that the user is on the website; the exchange presents an opportunity to a marketplace that can buy that opportunity to show the end user the ad. And this a simple example. The transactions in online advertising are as opaque

Continue reading…

Technically Sentient with Rob May

http://traffic.libsyn.com/sedaily/AIwithRobMay.mp3Podcast: Play in new window | Download The impact of artificial intelligence on our everyday lives will be so profound that our modern institutions will change completely. Employment, government, romance, social norms–all of these things will be upended. To see the signs of this coming, you no longer have to read science fiction. Every week, there are blog posts, news stories, and videos chronicling our strange, exciting time. Rob May

Continue reading…

Where Machines Go to Learn with Auren Hoffman

http://traffic.libsyn.com/sedaily/MLwithAuren.mp3Podcast: Play in new window | Download If you wanted to build a machine learning model to understand human health, where would you get the data? A hospital database would be useful, but privacy laws make it difficult to disclose that patient data to the public. In order to publicize the data safely, you would have to anonymize it, so that a patient’s identity could not be derived from data

Continue reading…

Machine Learning is Hard with Zayd Enam

http://traffic.libsyn.com/sedaily/WhyMLisHard.mp3Podcast: Play in new window | Download Machine learning frameworks like Torch and TensorFlow have made the job of a machine learning engineer much easier. But machine learning is still hard. Debugging a machine learning model is a slow, messy process. A bug in a machine learning model does not always mean a complete failure. Your model could continue to deliver usable results even in the presence of a mistaken

Continue reading…

Data Applications With Dave King

http://traffic.libsyn.com/sedaily/DataApps.mp3Podcast: Play in new window | Download Data scientists need flexible interfaces for displaying and manipulating data sets. Data engineers need to be able to visualize how their data pipelines wire together databases and data processing frameworks. DevOps engineers need dashboards to understand their monitoring data at a high level. All of these programmers are building data applications. Data applications let us visualize and manipulate data sets effectively. In today’s

Continue reading…

Service Proxying with Matt Klein

http://traffic.libsyn.com/sedaily/Envoy.mp3Podcast: Play in new window | Download Most tech companies are moving toward a highly distributed microservices architecture. In this architecture, services are decoupled from each other and communicate with a common service language, often JSON over HTTP. This provides some standardization, but these companies are finding that more standardization would come in handy. At the ridesharing company Lyft, every internal service runs a tool called Envoy. Envoy is a

Continue reading…

Infrastructure with Datanauts’ Chris Wahl and Ethan Banks

http://traffic.libsyn.com/sedaily/DataNauts.mp3Podcast: Play in new window | Download Infrastructure is a term that can mean many different things: your physical computer, the data center of your Amazon EC2 cluster, the virtualization layer, the container layer–on and on. In today’s episode, podcasters Chris Wahl and Ethan Banks discuss the past, present, and future of infrastructure with me. Ethan and Chris host Datanauts, a podcast about infrastructure. In each episode, Datanauts goes deep

Continue reading…

Deep Learning with Adam Gibson

http://traffic.libsyn.com/sedaily/DeepLearning.mp3Podcast: Play in new window | Download Deep learning uses neural networks to identify patterns. Neural networks allow us to sequence “layers” of computing, with each layer using learning algorithms such as unsupervised learning, supervised learning, and reinforcement learning. Deep learning has taken off in the last few years, but it has been around for much longer. Adam Gibson founded Skymind, the company behind Deeplearning4j. Deeplearning4j is a distributed deep

Continue reading…

Go Data Science with Daniel Whitenack

http://traffic.libsyn.com/sedaily/Go_Data_Science.mp3Podcast: Play in new window | Download Data science is typically done by engineers writing code in Python, R, or another scripting language. Lots of engineers know these languages, and their ecosystems have great library support. But these languages have some issues around deployment, reproducibility, and other areas. The programming language Golang presents an appealing alternative for data scientists. Daniel Whitenack transitioned from doing most of his data science work

Continue reading…

Engineering Management with Mike Borozdin

http://traffic.libsyn.com/sedaily/managementwithmike_edited.mp3Podcast: Play in new window | Download Engineering managers face a different set of problems than engineers themselves. Whether they are hiring new employees, firing underperformers, or guiding a team of existing engineers, engineering management is all about people. Empathy is paramount because management is like being a professional therapist for the members of your team. Mike Borozdin has been managing engineers for more than a decade and blogs about

Continue reading…

Open Source Contribution with Shubheksha Jalan

http://traffic.libsyn.com/sedaily/2opensource_edited.mp3Podcast: Play in new window | Download Open source software is publicly available code that is worked on in the open by large crowds of developers. Almost all new software today uses some open source software in its code. But most people never contribute to open source themselves. Some people would love to get involved in open source, but they don’t know how to get started. Shubheksha Jalan found herself

Continue reading…

Giphy Engineering with Anthony Johnson

http://traffic.libsyn.com/sedaily/giphy_edited.mp3Podcast: Play in new window | Download Giphy is a search engine for gifs, the short animated graphics that we see around the Internet. Giphy is also a creative platform where people create new gifs. Every search engine requires the construction of a search index, which is a data structure that responds to search queries efficiently. Since Giphy is a search engine for graphics, there is almost no text inherently

Continue reading…

Our Editorial Philosophy

Software Engineering Daily started as a daily podcast about how to build software. Following my own interests, the show evolved toward a discussion of how to create a software company as much as how to build the software itself. Even after 400 episodes of Software Engineering Daily, our coverage continues to inspire me and fill me with a sense of empowerment. Anyone who wants to build a technology company should

Continue reading…

The End of Cloud Computing with Peter Levine

http://traffic.libsyn.com/sedaily/endofcloud_edited.mp3Podcast: Play in new window | Download Cloud computing has pushed computation away from our own private servers and into virtual machines running on a data center. In the world of cloud computing, processing is centralized in these data centers, and our smartphone and laptop application performance suffers from having high latency between the client and the cloud server. As machine learning proliferates, the current model of cloud computing will

Continue reading…

Reality with Donald Hoffman

http://traffic.libsyn.com/sedaily/realitywithdonaldhoffman_edited.mp3Podcast: Play in new window | Download What is the relationship between your brain and your conscious experiences? This is is the fundamental question of the work of Donald Hoffman, a professor of computer science and cognitive science at UC Irvine. When Hoffman was a child, he wondered whether there was a cognitive dividing line between humans and machines, and that curiosity has driven him to his current work–building a

Continue reading…

Developer On Fire with Dave Rael

http://traffic.libsyn.com/sedaily/developeronfire_1.mp3Podcast: Play in new window | Download Software developers succeed by combining technical ability, communication skills, and well-reasoned philosophies to craft information systems. Where this podcast focuses on the information systems, Developer On Fire focuses on the engineers who built them. Dave Rael started his podcast as a way to overcome fear and access an entrepreneurial side of himself. In his interviews with prominent engineers, Dave identifies patterns and strategies

Continue reading…

Twilio Engineering with Pat Malatack

http://traffic.libsyn.com/sedaily/twilio_1.mp3Podcast: Play in new window | Download Back in 2008, the range of tools that engineers could use to connect computer systems together were getting quite good. Cloud computing was democratizing access to servers. But the telephony ecosystem was still inaccessible to the average developer. If you needed your program to make a phone call and connect a user to a customer service representative, there was no easy way to

Continue reading…

Making Money Online for Software Engineers with Courtland Allen

http://traffic.libsyn.com/sedaily/Courtland_Meetup_Talk_mixdown.mp3Podcast: Play in new window | Download Engineers today have a variety of career options. You could go work for a large corporation, you could raise money and start a startup, you could freelance and move from job to job with freedom–or you could start a business with the goal of quickly becoming profitable. Courtland Allen was a guest on Software Engineering Daily a few months ago, when he discussed

Continue reading…

Robots for the Elderly with Itai Mendelsohn

http://traffic.libsyn.com/sedaily/robotics_edited.mp3Podcast: Play in new window | Download Many elderly people live with unhealthy levels of isolation. Social isolation is a problem for anybody, but younger people can use technology to alleviate their isolation with tools like Skype and Facebook. How can we bridge the generational gap and give elderly people access to the same technological tools that younger people find easy to use? Voice interfaces are an important new medium

Continue reading…

Kotlin with Hadi Hariri

http://traffic.libsyn.com/sedaily/kotlin_edited.mp3Podcast: Play in new window | Download Whatever engineering problem you have right now, the solution is probably not to write a new programming language. But sometimes it does make sense. JetBrains makes IDEs–the interactive development environments that many people code in, like IntelliJ and Webstorm. And all of these IDEs are written in Java. So the JetBrains team is very familiar with Java and the JVM. Since JetBrains spends

Continue reading…

Translation with Vasco Pedro

http://traffic.libsyn.com/sedaily/Unbabel_edited.mp3Podcast: Play in new window | Download Translation is a classic problem in computer science. How do you translate a sentence from one human language into another? This seems like a problem that computers are well-suited to solve. Languages follow well-defined rules, we have lots of sample data to train our machine learning models. And yet, the problem has not been solved–largely because languages don’t always follow rules. We have

Continue reading…

Reactive Microservices Development with Markus Eisele

http://traffic.libsyn.com/sedaily/developing_reactive_microservices_edited.mp3Podcast: Play in new window | Download The goals of microservices are the same as what we have pursued in software engineering for decades: isolation, decoupling, maintainability, scalability. The reason that we use the term microservices is not because we have a completely different idea of what a service is than we used to. We use the term microservices because we are signaling that we need to achieve these architectural

Continue reading…

Hybrid Mobile Apps with Adam Bradley

http://traffic.libsyn.com/sedaily/ionic_edited.mp3Podcast: Play in new window | Download Building a mobile app requires developers to build a separate version for Android and iOS. The approval process for app stores makes it difficult to deploy quickly and iterate in small batches for developers who are making native apps. These frictions cause fewer developers to write mobile apps than we would have if the smartphone platform was unified. Since the early days of

Continue reading…

Urbit with Curtis Yarvin and Galen Wolfe-Pauly

http://traffic.libsyn.com/sedaily/urbit_edited.mp3Podcast: Play in new window | Download Urbit is a completely new way of looking at computing. Every user gets a personal server, which runs your apps, wrangles your connected devices, and defines your secure identity. Your urbit presents your whole digital life as a single web service. Urbit feels foreign and confusing for those of us coming from the traditional web because the normal paradigm is to iterate and

Continue reading…

Inferno with Dominic Gannaway

http://traffic.libsyn.com/sedaily/inferno_edited.mp3Podcast: Play in new window | Download Over the past few years, React has become the most popular front end JavaScript framework. As React has matured, the open source community around React has identified areas for improvement. Since React itself is too mature to refactor completely, new projects have been started to take the best aspects of React and start from scratch. Inferno is an extremely fast, React-like JavaScript library

Continue reading…

Product Management with Suzie Prince

http://traffic.libsyn.com/sedaily/productmanagement_edited.mp3Podcast: Play in new window | Download Product managers are responsible for guiding the design and overall functionality of software. The relationship between product managers and engineers is complimentary: a PM is viewing the product from a perspective that is closer to the customer, so the PM often has the responsibility of navigating high-level tradeoffs in the functionality of a product. Suzie Prince is the head of product at ThoughtWorks

Continue reading…

Medical Machine Learning with Razik Yousfi and Leo Grady

http://traffic.libsyn.com/sedaily/heartflow_edited_fixed.mp3Podcast: Play in new window | Download Medical imaging is used to understand what is going on inside the human body and prescribe treatment. With new image processing and machine learning techniques, the traditional medical imaging techniques such as CT scans can be enriched to get a more sophisticated diagnosis. HeartFlow uses data from a standard CT scan to model a human heart and understand blockages of blood flow using

Continue reading…

Python Data Visualization with Jake VanderPlas

http://traffic.libsyn.com/sedaily/python_dataviz_edited_fixed.mp3Podcast: Play in new window | Download Data visualization tools are required to translate the findings of data scientists into charts, graphs, and pictures. Understanding how to utilize these tools and display data is necessary for a data scientist to communicate with people in other domains. In this episode, Srini Kadamati hosts a discussion with Jake VanderPlas about the Python ecosystem for data science and the different attempts at creating

Continue reading…

Columnar Data: Apache Arrow and Parquet with Julien Le Dem and Jacques Nadeau

http://traffic.libsyn.com/sedaily/columnardata_edited_fixed.mp3Podcast: Play in new window | Download Column-oriented data storage allows us to access all of the entries in a database column quickly and efficiently. Columnar storage formats are mostly relevant today for performing large analytics jobs. For example, if you are a bank, and you want to get the sum of all of the financial transactions that took place on your system in the last week, you don’t want

Continue reading…

Architecture of Free Code Camp with Berkeley Martinez

http://traffic.libsyn.com/sedaily/freecodecamp_arch_fixed.mp3Podcast: Play in new window | Download Free Code Camp is an online learning platform that takes people from knowing nothing about code to having enough knowledge to build software for a living. We have already done a show with Quincy Larson, the founder of Free Code Camp, in which we discussed his motivation for starting the organization. The economics of running a free interactive platform with thousands of users

Continue reading…

Ethics of Software with Bill Sourour

http://traffic.libsyn.com/sedaily/ethicsofsoftware_edited.mp3Podcast: Play in new window | Download Software engineers are often one of the last lines of defense against potentially dangerous and unethical practices. Every software company encounters situations where the line between right and wrong is not clearly drawn. Back in the year 2000 Bill Sourour was asked to code a project to help market a pharmaceutical product in a misleading way. Reflecting on this experience, Bill recently wrote

Continue reading…

Email Infrastructure with Chris McFadden

http://traffic.libsyn.com/sedaily/email_infrastructure_edited.mp3Podcast: Play in new window | Download A company like Pinterest has millions of transactional emails to send to people. The scalability challenges of sending high volumes of email mean that it makes more sense for most companies to use an email as a service product rather than building their own. Chris McFadden is the VP of engineering and cloud operations at SparkPost and he joins the show to explain the

Continue reading…

Cyber Warfare with Jared Smith

http://traffic.libsyn.com/sedaily/websecurity_edited.mp3Podcast: Play in new window | Download Vulnerabilities exist in every computer system. As a system gets bigger, the number of vulnerabilities magnifies. The web is the biggest, most complex computer system we have–but fortunately, the steps we can take to secure our web applications are often quite simple. Jared Smith is a cyber security research scientist with Oak Ridge National Laboratory. He joined me on the show to discuss

Continue reading…

Meetup Architecture with Yvette Pasqua

http://traffic.libsyn.com/sedaily/meetup_architecture_edited.mp3Podcast: Play in new window | Download Meetup is an online service that allows people to gather into groups and meet in person. Since 2002, the company has been growing and its technology stack has been changing. Today, they are in the process of migrating to the cloud, using both Amazon Web Services and Google Compute Platform. Yvette Pasqua is the CTO of Meetup and she joins the show to

Continue reading…

Evolutionary Architecture with Neal Ford

http://traffic.libsyn.com/sedaily/architecture_edited.mp3Podcast: Play in new window | Download When a useful new technology comes out, companies that are in a position to adopt that new technology can gain an edge over competitors. As our industry grows and moves faster, these kinds of changes are coming faster–some recent examples are Docker, ReactJS, and Kubernetes. Evolutionary architecture supports incremental, guided change as a first principle along multiple dimensions. A company with an evolutionary

Continue reading…

Security Research with Samy Kamkar

http://traffic.libsyn.com/sedaily/security_edited.mp3Podcast: Play in new window | Download Every digital system has vulnerabilities. Cars can be hacked, locked computers can be exploited, and credit cards can be spoofed. Security researchers make a career out of finding these types of vulnerabilities. Samy Kamkar’s approach to security research is not just about dissection–it’s also about creativity. For many of the technologies he hacks on, Samy open-sources code that summarily describes the vulnerability he

Continue reading…

Self-Contained Systems with Eberhard Wolff

http://traffic.libsyn.com/sedaily/selfcontained_edited.mp3Podcast: Play in new window | Download Self-contained systems is an architectural approach that separates the functionality of a system into many independent systems. Each self-contained system is an autonomous web application, and is owned by one team. Communication with other self-contained systems or 3rd party systems is asynchronous where possible. As Eberhard Wolff explains in this episode, self-contained systems is not the same thing as microservices, but they are

Continue reading…

Breaking Into Startups

http://traffic.libsyn.com/sedaily/breakingintostartups_edited.mp3Podcast: Play in new window | Download Many people find themselves going down a career path that does not bring them satisfaction. A lawyer finds himself constantly working cases he doesn’t care about. A student in medical school gets completely burned out from hospital bureaucracy. An investment banker no longer finds joy in the accounting statements that she used to enthusiastically study. Startups offer a different career path. Within a

Continue reading…

Startup Engineering with Mike Wolfe

http://traffic.libsyn.com/sedaily/mikewolfe_edited.mp3Podcast: Play in new window | Download In the 1990s, the barriers to starting a company were significant. Not only did you need an idea, you needed $200,000 for servers and Oracle licenses. With cloud computing, the up-front financial costs of getting a company off the ground have been mostly eliminated–but the idea of starting a company is still perceived as risky. The process of building software has changed dramatically

Continue reading…

Robot Cloud Lab with Max Hodak

http://traffic.libsyn.com/sedaily/transcriptic_edited.mp3Podcast: Play in new window | Download A biologist wants to study the genetic makeup of an organism. A pharmaceutical researcher wants to test the effects of an experimental drug. These types of experiments require a deep knowledge of the scientific domain as well as the lab techniques to produce the data that will eventually yield a result. Transcriptic is a robotic biology laboratory that allows you to make requests

Continue reading…

Performance Monitoring with Andi Grabner

http://traffic.libsyn.com/sedaily/monitoring_edited.mp3Podcast: Play in new window | Download Application performance monitoring helps an engineer understand what is going on with an application. An application on a single machine is often monitored by inserting bytecode instructions into the application after it has been interpreted. Distributed cloud applications with functionality broken up across multiple servers often use distributed tracing. Andi Grabner from Dynatrace joins today’s show to explain how monitoring software is built,

Continue reading…

Antifraud Architecture with Josh Yudaken

http://traffic.libsyn.com/sedaily/antifraud_architecture_edited.mp3Podcast: Play in new window | Download Online marketplaces and social networks often have a trust and safety team. The trust and safety team helps protect the platform from scams, fraud, and malicious actors. To detect these bad actors at scale requires building a system that classifies every transaction on the platform as safe or potentially malicious. Since every social platform has to build something like this, Smyte decided to

Continue reading…

Sampling with Daniel Trostli

http://traffic.libsyn.com/sedaily/sampling_edited.mp3Podcast: Play in new window | Download Every song you hear on the radio is written with a computer. Computer musicians mostly use synthesizers and samples to compose these songs. A sample is a snippet of recorded sound, sometimes taken from a songs, a movie, or another source. The more samples a musician has access to the better. SampleFocus is a platform where musicians upload and download samples to build

Continue reading…

Data Engineering with Pete Soderling

http://traffic.libsyn.com/sedaily/hakkalabs_edited.mp3Podcast: Play in new window | Download In the last five years, companies started hiring data engineers. A data engineer creates the systems that manage and access the huge volumes of data that are accumulating on cheap cloud servers. As the saying goes, “it’s more expensive to throw out the data than to store it.” Pete Soderling joins the show to discuss the rise of the data engineer, and how

Continue reading…

Ad Fraud Research with Augustine Fou

http://traffic.libsyn.com/sedaily/adfraudresearch_edited.mp3Podcast: Play in new window | Download A huge percentage of online advertisements are never seen by humans. They are viewed by bots–automated scripts that are opening web pages in a browser and pretending to be a human. Advertising scammers set up web pages, embed advertisements on those pages, and then pay for bot traffic to come and view those advertisements. This aspect of the internet is bizarre and alarming.

Continue reading…

Reactive Microservices with Jonas Boner

http://traffic.libsyn.com/sedaily/reactivemicroservices_edited.mp3Podcast: Play in new window | Download For many years, software companies have been breaking up their applications into individual services for the purpose of isolation and maintainability. In the early 2000s, we called this pattern “service-oriented architecture”. Today we call it “microservices”. Why did we change that terminology? Did the services get smaller? Not exactly. Jonas Boner suggests that the movement towards cloud and the increased prevalence of mobile

Continue reading…

Scale API with Lucy Guo and Alexandr Wang

http://traffic.libsyn.com/sedaily/scaleapi_edited1.mp3Podcast: Play in new window | Download Some tasks are simple, but cannot be performed by a computer. Audio transcription, image recognition, survey completion–these are simple procedures that almost any human could execute, but the machine learning models have not gotten consistent enough to do them accurately. Scale is an API for human labor, created by Lucy Guo and Alexandr Wang. Similar to Amazon Mechanical Turk, Scale sends small, simple

Continue reading…

Bots with Jon Bruner

http://traffic.libsyn.com/sedaily/botpodcast_edited.mp3Podcast: Play in new window | Download Over the next few years, bots will pervade our lives more and more. These smart, conversational text interfaces provide a new way of engaging with the computer systems that we have been mostly interacting with through web and mobile app interfaces for the last decade. Bots are a necessary complement to the voice interfaces of the future, because we don’t always want to

Continue reading…

Mesos, Kubernetes, and Infrastructure of the Future with Dharmesh Kakadia

http://traffic.libsyn.com/sedaily/mesos-and-kubernetes_edited_1.mp3Podcast: Play in new window | Download Mesos and Kubernetes are tools for distributed systems management. Kubernetes is built with an emphasis on running services, whereas Mesos is commonly used for a wider variety of workloads, including data infrastructure like Spark and Kafka. Mesos can also be used as a platform to provide resource management for Kubernetes. Dharmesh Kakadia is the author of Apache Mesos Essentials, and has spent time

Continue reading…

Ad Fraud Everywhere with Shailin Dhar

http://traffic.libsyn.com/sedaily/adfraudeverywhere_edited_2.mp3Podcast: Play in new window | Download Advertising fraud is easy, legal, and extremely profitable. A fraudster can set up a website, scrape content from the internet, and run programmatic advertisements against that website. The fraudster can then purchase bot traffic. Those bots will visit the page, consume advertisements, and return profit to the owner of the page. In a past life, Shailin Dhar worked for a company that set

Continue reading…

Cloud Foundry with Cornelia Davis

http://traffic.libsyn.com/sedaily/cloudfountry_edited.mp3Podcast: Play in new window | Download Cloud Foundry is an open-source platform as a service. Companies use Cloud Foundry as a control plane to deploy and manage applications. It provides abstractions for microservices and continuous integration. Cornelia Davis joins the show to discuss Cloud Foundry. I interviewed her at the DevOps Enterprise Summit, where enterprises share their stories of improving their culture and their technology stack. Cornelia explained how

Continue reading…

Netflix Caching with Scott Mansfield

http://traffic.libsyn.com/sedaily/netflixcaching_edited.mp3Podcast: Play in new window | Download Caching is a fundamental concept of computer science. When data is accessed frequently, we put that data in a place where it can be accessed more quickly–we put the data in a cache. When data is accessed less often, we leave it in a place where the access time is slow or expensive. Netflix has a huge variety of data, and a huge

Continue reading…

How Software Engineering Daily Works

http://traffic.libsyn.com/sedaily/how_sed_works_edited.mp3Podcast: Play in new window | Download Software Engineering Daily was started a year and a half ago, based on what I learned from my podcasting experience on Software Engineering Radio. Last week, I interviewed Robert Blumen, the editor of Software Engineering Radio, about how that podcast is produced. In today’s episode, Robert interviews me about this podcast. If you are thinking about starting a podcast about software engineering, this

Continue reading…

Developer Tools with Josh Varty

http://traffic.libsyn.com/sedaily/developertools_edited.mp3Podcast: Play in new window | Download When you are working on a program, a lot of things are going through your head. In some sense, you become part machine when you are programming. Learnable Programming is a concept that facilitates this, by showing developers what the computer is doing in real time, before compiling. In this episode, Josh Varty, co-founder of Code Connect Inc., talks to Edaena Salinas about

Continue reading…

Bot Memorial with Eugenia Kuyda

http://traffic.libsyn.com/sedaily/botmemorial_edited.mp3Podcast: Play in new window | Download When a human passes away, we create a tombstone as a memorial. Friends and family visit a grave to remember the times they had with that person while they were still alive. Memorial bots are another way to celebrate the life of someone who has passed away. A memorial bot is created by taking the messages sent by a deceased person and passing

Continue reading…

Algorithms to Live By with Brian Christian

http://traffic.libsyn.com/sedaily/algorithms_edited.mp3Podcast: Play in new window | Download When you are deciding who to marry, you are using an algorithm. The same is true when you are looking for a parking space, playing a game of poker, or deciding whether or not to organize your closet. Algorithms To Live By is a book about the computer science of human decisions. It offers strategies for how to think through everyday life like

Continue reading…

Adforprize

http://traffic.libsyn.com/sedaily/adforprize_edited_fixed.mp3Podcast: Play in new window | DownloadThere are multiple paths to constructing a piece of software from start to finish. An individual programmer can build an entire product from scratch in a couple days. A giant corporation can commission a project and delegate responsibilities to hundreds of people. An open source community can use the wisdom of the crowds to efficiently build an operating system. Today’s episode is about another

Continue reading…

Robot Lawyer with Joshua Browder

http://traffic.libsyn.com/sedaily/robotlawyer_edited_1.mp3Podcast: Play in new window | Download You have probably received a parking ticket that you felt was unfair,   but instead of fighting it, you paid the expensive price to get rid of it quickly. Fighting a parking ticket sounds like it would be so time consuming that it is a better decision to just pay for it. When Joshua Browder was faced with this situation, his response was different.

Continue reading…

Database as a Service with Eliot Horowitz

http://traffic.libsyn.com/sedaily/mongoservice_editedfixed1.mp3Podcast: Play in new window | Download Eight years ago, MongoDB was an internal project at 10gen, a company that was trying to build a platform-as-a-service out of open-source components. The team at 10gen realized that the platform-as-a-service play would be too complex, and difficult to build. Since MongoDB was the most valuable component of that project, they narrowed their focus to this new document-oriented database. In today’s episode, MongoDB

Continue reading…

Software Podcasting with Robert Blumen

http://traffic.libsyn.com/sedaily/swpodcasting_edited_fixed.mp3Podcast: Play in new window | Download Four years ago, I started volunteering for a popular podcast about software–Software Engineering Radio. For the next two years, I learned about the process of making a quality podcast about engineering. With its emphasis on preparation, timeless engineering principles, and attention to the listener, Software Engineering Radio continues to be one of the most popular shows about engineering. Software Engineering Daily is my

Continue reading…

DotNet Core with Lee Coward and Immo Landwerth

http://traffic.libsyn.com/sedaily/dotnetcore_edited.mp3Podcast: Play in new window | Download C# .NET is the framework that is most often used to write software for the Microsoft Windows operating system. For many years, the C# .NET framework was closed source, but Microsoft’s recent push towards open source has led to the creation of .NET Core, a fork of C# .NET composed of a small subset of features from the original C# .NET stack. This

Continue reading…

Biological Computation with Colin Gravill

http://traffic.libsyn.com/sedaily/biocomputation_edited.mp3Podcast: Play in new window | Download Biology research at Microsoft is focused on three main areas: molecular programming, synthetic biology, and stem cell biology. At the intersection of biology and computing there are implications for health, medicine, and efficient computing techniques. The field of Biological Computation is in its early days, and there is still lots of work to be done. Colin Gravill works in the computational science group

Continue reading…

Rust Concurrency with Alex Crichton

http://traffic.libsyn.com/sedaily/rust_edited_fixed.mp3Podcast: Play in new window | Download Rust is a systems level language that is built to prevent crashes and eliminate data races. A language like C++ gives you high speed and lots of control, but it is easy to have segfaults, data races, and other problems if you aren’t careful. On this spectrum of control versus safety, we can plot other languages like Java, Go, and Haskell–but none of

Continue reading…

Microservices with Rafi Schloming

http://traffic.libsyn.com/sedaily/microservices_tools_edited.mp3Podcast: Play in new window | Download Microservices are a widely adopted pattern for breaking an application up into pieces that can be well-understood by the individual teams within the company. Microservices also allow these individual pieces to be scaled independently and updated in isolation. Past Software Engineering Daily episodes have covered the microservice architectures of Twitter, Netflix, Google, Uber and other companies. In today’s episode, I sit down with

Continue reading…

Apache Geode with Swapnil Bawaskar

http://traffic.libsyn.com/sedaily/geode_edited.mp3Podcast: Play in new window | Download There is a hierarchy of ways to access and store data in a computer system. The cheapest, slowest way to store and retrieve data is disk. On the faster end, we have memory. As we architect systems with increasing complexity, we have additional considerations–network latency, transient compute nodes, and numerous caching layers. Apache Geode is a distributed, in-memory system for the JVM. It

Continue reading…

Debugging Stories with Haseeb Qureshi

http://traffic.libsyn.com/sedaily/debuggingstories_edited.mp3Podcast: Play in new window | Download Everyone has debugging stories. We have all had the experience of wrestling with a seemingly impossible bug for days until we finally come to a solution. In today’s episode, Haseeb Qureshi retells some of his favorite debugging stories: The case of the 500-mile email, Debugging Behind the Iron Curtain, and My Hardest Bug Ever.

Software Small Businesses with Patrick McKenzie

http://traffic.libsyn.com/sedaily/kalzumeus_edited.mp3Podcast: Play in new window | Download Many programmers listening to this podcast are working at a big company, and they would prefer to be running their own software business. Patrick McKenzie has been writing about this topic for several years on his blog Kalzumeus.com. Almost a decade ago, he was working as an enterprise developer at a large company in Japan. Over time, his side projects started making enough

Continue reading…

Botnet Facebook Likes with Derek Muller

http://traffic.libsyn.com/sedaily/likefraud_edited.mp3Podcast: Play in new window | Download Botnets have a massive influence on the Internet. As we have seen recently with the Mirai Botnet, IOT bots can take down companies as big as Netflix. In our recent episodes about advertising fraud, we’ve talked about how bots are being used to take billions of dollars of revenue from advertisers. Derek Muller is one of those advertisers who has spent money on

Continue reading…

Slack Bots with Amir Shevat

http://traffic.libsyn.com/sedaily/slackbots_edited.mp3Podcast: Play in new window | Download Slack is a chat client that has reached wide adoption. The rise of Slack has coincided with the rise of chatbots. A chatbot is a simple, conversational interface into a computer program that may have simple functionality, like telling you some simple statistics, or more complex functionality, like helping you manage your continuous integration pipeline. Bot design and engineering is a new field,

Continue reading…

Winning With Data with Tomasz Tunguz

http://traffic.libsyn.com/sedaily/winningwithdata_edited_2.mp3Podcast: Play in new window | Download Large technology companies have no shortage of data. But raw data itself does not provide a competitive advantage. Many companies are bottlenecked by a shortage of data scientists who can query that data effectively. This results in an organizational dysfunction where people lining up to ask questions of the data science team are unable to move as fast as they want to. Tomasz

Continue reading…

AWS Open Guide with Joshua Levy

http://traffic.libsyn.com/sedaily/awsguide_edited_2.mp3Podcast: Play in new window | Download Amazon Web Services changed the economics of building an internet application. Instead of having to invest tens of thousands of dollars up front for hardware, developers can pay for services over time as their application scales. As AWS has grown to be a gigantic platform, the documentation about how to use cloud infrastructure has become insufficient. As an answer to this, Joshua Levy

Continue reading…

Bot Day

http://traffic.libsyn.com/sedaily/botday_edited.mp3Podcast: Play in new window | DownloadBot Day was an O’Reilly conference featuring talks from some of the leading figures in the bot industry. Before I attended Bot Day, I knew there were lots of applications for chatbots, but I didn’t realize how good the tooling has gotten–it’s very easy to get started with chatbots today so if you are a developer and you have a basic idea, I suggest

Continue reading…

Infrastructure Mistakes with Avi Freedman

http://traffic.libsyn.com/sedaily/infrastructuremistakes_edited.mp3Podcast: Play in new window | Download The blueprint for a typical startup involves investing heavily in cloud services–either from Amazon, Google, or Microsoft. The high costs can quickly eat away at all of the money that startup has raised. In today’s episode, Avi Freedman outlines some of the infrastructure mistakes that can set back a company severely–cloud jail, hipster tools, and lack of monitorability. Avi is the CEO of

Continue reading…

Fraud Prevention with Pete Hunt

http://traffic.libsyn.com/sedaily/antifraud_edited.mp3Podcast: Play in new window | Download When Facebook acquired Instagram, one of the first systems Instagram plugged into was Facebook’s internal spam and fraud prevention system. Pete Hunt was the first Facebook engineer to join the Instagram team. When he joined, the big problems at Instagram were around fake accounts, harassment, and large volumes of spammy comments. After seeing the internal Facebook spam prevention tools clean up Instagram, Pete

Continue reading…

Reflections of an Old Programmer with Ben Northrop

http://traffic.libsyn.com/sedaily/oldprogrammerreflections_edited.mp3Podcast: Play in new window | Download Ben Northrop was sitting at a tech conference, listening to a presentation about a new piece of technology, when he was struck by the sense that history was repeating itself. For the twenty years that Ben has worked as a software engineer, he has been hearing about new technologies that claim they will be able to disrupt everything, and he has relentlessly been

Continue reading…

Julia Language with Jeff Bezanson

http://traffic.libsyn.com/sedaily/julia-language_edited.mp3Podcast: Play in new window | Download Jeff Bezanson’s university thesis described the motivation for a new programming language. He discussed the shortcomings of “array based programming environments” and his desire to create a more performant language with the best qualities of Lisp, Python, Ruby, Perl, Mathematica, R, and C. The Julia Language is a high performance language designed to suit technical users that crave the flexibility to pick their

Continue reading…

Legacy Code with Andrea Goulet

http://traffic.libsyn.com/sedaily/legacycode_edited.mp3Podcast: Play in new window | Download Legacy code is code without automated tests. Most companies have lots of legacy code, and most developers don’t like working on legacy code. Why is that? What is it that makes legacy code so difficult to work with? And why does a large amount of legacy code slow down an organization so severely? Andrea Goulet is the CEO of Corgibytes, a consultancy that

Continue reading…

Indie Hackers with Courtland Allen

http://traffic.libsyn.com/sedaily/indiehackers_edited.mp3Podcast: Play in new window | Download Indie Hackers is a website that profiles independent developers who have made profitable software projects, usually without raising any money. These projects make anywhere from a few hundred dollars a month to more than $100,000 as in the case with park.io, one of the services profiled by Indie Hackers. Courtland Allen is the creator, engineer, and interviewer behind Indie Hackers. For each business

Continue reading…

Ad Tracking with Larry Furr

http://traffic.libsyn.com/sedaily/ghostery_edited.mp3Podcast: Play in new window | Download When you visit a web page, that web page can write data to a file on your computer, known as a cookie. Scripts on that page can also read from your cookie file to understand where you have been in the past. All of this data about you is getting shared between advertising companies like Google, Facebook, and AppNexus. Ghostery is a browser

Continue reading…

ChatOps with Jason Hand

http://traffic.libsyn.com/sedaily/chatops_edited.mp3Podcast: Play in new window | Download Chat bots are your newest co-worker. Slack, HipChat, and other chat clients allow developers and other team members to communicate more dynamically than the limits of email. Companies have started to add bots to their chat rooms. These bots can give you technical information, restart a server, or notify you that a build has finished. Jason Hand is the author of ChatOps: Managing

Continue reading…

Ad Industry with Bob Hoffman

http://traffic.libsyn.com/sedaily/bobhofman_edited.mp3Podcast: Play in new window | Download Online advertising is heavily affected by a set of delusions and fraudulent practices that few people in the adtech industry have an interest in stopping. This is the curious, perverse nature of the world of online media. Some of the online advertising we see fulfills its job, when an ad successfully conveys a meaningful message from a marketer to a consumer. But there

Continue reading…

Musicians’ GitHub with Alan Grow

http://traffic.libsyn.com/sedaily/githubformusicians_edited.mp3Podcast: Play in new window | Download Music collaboration software that works over the Internet is a software challenge that has not been fully tackled. On today’s Internet, users collaborate intensively on programming projects, journalism, and other projects, but the tools for collaborating on music online have not yet become popular. Blend.io is a social music collaboration tool–a github for musicians. I have been using it myself and enjoying it

Continue reading…

Ad Fraud with Ben Trenda

http://traffic.libsyn.com/sedaily/adfraud_edited.mp3Podcast: Play in new window | Download Advertising fraud takes billions of dollars out of the economy every year. We don’t know exactly how much money is being lost, because we don’t know what percentage of Internet users are bots. Are You A Human is a company designed to solve that exact problem, and provide a service for verifying whether a user is real or automated. Ben Trenda is the

Continue reading…

Coin Center with Peter Van Valkenburgh

http://traffic.libsyn.com/sedaily/coincenter_edited.mp3Podcast: Play in new window | Download Blockchain technologies like Bitcoin and Ethereum have not impacted the lives of most consumers today. The theoretical breakthroughs that blockchain enables will eventually happen–I will be able to pay 1 cent to a knowledge worker in Africa without having to pay a 5 cent transaction fee. My servers will be able to pay other servers for small compute jobs. We will have decentralized

Continue reading…

Managed Kafka with Tom Crayford

http://traffic.libsyn.com/sedaily/heroku_kafka_edited.mp3Podcast: Play in new window | Download Kafka is a distributed log for producers and consumers to publish messages to each other. We’ve done many shows about Kafka as a key building block for distributed systems, but we often leave out the discussion of the complexities of setting up Kafka and monitoring it. Kafka deployments can be a complex piece of software to manage. Tom Crayford is an engineer at

Continue reading…

Reactive JavaScript with Ben Lesh

http://traffic.libsyn.com/sedaily/RxJSnetflix_edited.mp3Podcast: Play in new window | Download Netflix has a highly interactive user interface. As I move my mouse around the page, hovering over titles and inspecting movie descriptions, there is a lot going on under the hood. One component of this UI is RxJS, a library for building reactive JavaScript. Reactive programming uses the observer pattern to create objects that emit streams of events. We can compose these streams

Continue reading…

Database Choices and Uber with Markus Winand

http://traffic.libsyn.com/sedaily/uber_database_edited.mp3Podcast: Play in new window | Download When Uber’s engineering team published a blog post about moving to MySQL from Postgres, Markus Winand started receiving lots of email. Markus writes about databases on his blog “Use The Index, Luke,” a guide to database performance for developers. The people emailing Markus wanted to know–if Postgres doesn’t work well for Uber, is it safe to use for anyone? Markus wrote a detailed

Continue reading…

Topic Roundtable with Haseeb Qureshi and Practical Dev’s Ben Halpern

http://traffic.libsyn.com/sedaily/topicroundtable_edited.mp3Podcast: Play in new window | Download Bot fraud, the New York tech scene, RethinkDB and open source; these topics and more are discussed in today’s episode. Two of the most popular guests return to the show to explore a variety of topics. Ben Halpern is the creator of The Practical Dev, a massively popular Twitter account and blog that you may recognize from its parody O’Reilly book covers. Haseeb

Continue reading…

Google Cloudbuilding with Joe Beda

http://traffic.libsyn.com/sedaily/cloudbuilding_edited.mp3Podcast: Play in new window | Download Google Compute Engine is the public cloud built by Google. It provides infrastructure- and platform-as-a-service capabilities that rival Amazon Web Services. Today’s guest Joe Beda was there from the beginning of GCE, and he was also one of the early engineers on the Kubernetes project. Google’s internal systems have made it easy for employees to spin up compute resources, but it was not

Continue reading…

Docker Cloudcasting with Brian Gracely

http://traffic.libsyn.com/sedaily/cloudcasting_edited.mp3Podcast: Play in new window | Download Cloud computing was something much different in 2011, when Brian Gracely and Aaron Delp started The Cloudcast, a podcast I listen to on a regular basis. The Cloudcast features technical discussions about cloud infrastructure technology, and one of the most recent shows was a monologue by Brian Gracely where he explained his perspective on the industry rumblings about a Docker fork. The utility

Continue reading…

Simpsons Data Science with Todd Schneider

http://traffic.libsyn.com/sedaily/simpsons_data_science_edited.mp3Podcast: Play in new window | Download The Simpsons is a classic, beloved television show. The scripts of The Simpsons have been made publicly available, and include dialogue, location, and character information. Todd Schneider used these scripts and other information sources as a corpus to analyze The Simpsons and find interesting statistics–such as who the most important supporting characters were, and how the ratings of the show have trended over

Continue reading…

PANCAKE STACK Data Engineering with Chris Fregly

http://traffic.libsyn.com/sedaily/pancakestack_edited_fixed.mp3Podcast: Play in new window | Download Data engineering is the software engineering that enables data scientists to work effectively. In today’s episode, we explore the different sides of data engineering–the data science algorithms that need to be processed and the implementation of software architectures that enable those algorithms to run smoothly. The PANCAKE STACK is a 12-letter acronym that Chris Fregly gave to a collection of data engineering technologies

Continue reading…

Kafka Event Sourcing with Neha Narkhede

http://traffic.libsyn.com/sedaily/event_sourcing_edited.mp3Podcast: Play in new window | Download When a user of a social network updates her profile, that profile update needs to propagate to several databases that want to know about such an update–search indexes, user databases, caches, and other services. When Neha Narkhede was at LinkedIn, she helped develop Kafka, which was deployed at LinkedIn to help solve this very problem. Using Kafka as an event queue, LinkedIn adopted

Continue reading…

DevOps Handbook with Gene Kim

http://traffic.libsyn.com/sedaily/devops_handbook_edited.mp3Podcast: Play in new window | Download The intent of the DevOps movement is to get organizations moving faster and more effectively by breaking down siloes, and improving communication. Gene Kim’s book The Phoenix Project illustrated this by telling the fictional story of a company adopting a DevOps mentality. Although that book was fiction, Gene is an experienced engineer, having worked as founder and CTO of Tripwire, a software company

Continue reading…

Netflix Scheduling with Sharma Podila

http://traffic.libsyn.com/sedaily/scheduling_at_netflix_edited.mp3Podcast: Play in new window | Download At Netflix, developers write applications with a variety of requirements–from simple requests for a list of movies to more resource-intensive requests like a complex machine learning workflow. Netflix wants developers to be able to request the resources they need from a compute cluster and receive those resources on-demand, without thinking too much about the state of that pool of resources they are drawing

Continue reading…

Monitoring Architecture with Theo Schlossnagle

http://traffic.libsyn.com/sedaily/monitoring_theo_edited_fixed.mp3Podcast: Play in new window | Download Building a monitoring system is a complex distributed systems problem. Events are produced from different points in an application and must be aggregated in order to form metrics. These events are often ingested by a time series database, which forms the backbone of our monitoring system. Theo Schlossnagle is the CEO of Circonus, where he has been working on architecting the company’s monitoring

Continue reading…