LinkedIn Monitoring Infrastructure with Alexander Pucher

Monitoring tools are used by every area of an organization.

Business development teams use monitoring to understand the metrics for product performance. Finance teams need to understand how the costs of cloud computing resources are changing. Site reliability engineers use monitoring dashboards that applications are up and running without problematic latency. Product managers evaluate the results of AB tests based off of the monitoring data of how users are reacting to new features.

A monitoring system needs to be able to handle large volumes of data that are being generated at a high velocity. The data needs to be queryable in an aggregated format, which might require an ETL system for getting data into columnar format.

Alexander Pucher is an engineer at LinkedIn, where he works on a monitoring platform called ThirdEye. ThirdEye is built on top of Apache Pinot, a distributed columnar storage engine that ingests data and serves analytical queries at low latency. Pinot is comparable to Apache Druid.

Alexander joins the show to discuss ThirdEye, and explain why Pinot is a useful building block for monitoring infrastructure.

Transcript

Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.


Software Weekly

Software Weekly

Subscribe to Software Weekly, a curated weekly newsletter featuring the best and newest from the software engineering community.