LinkedIn Monitoring Infrastructure with Alexander Pucher

Monitoring tools are used by every area of an organization.

Business development teams use monitoring to understand the metrics for product performance. Finance teams need to understand how the costs of cloud computing resources are changing. Site reliability engineers use monitoring dashboards that applications are up and running without problematic latency. Product managers evaluate the results of AB tests based off of the monitoring data of how users are reacting to new features.

A monitoring system needs to be able to handle large volumes of data that are being generated at a high velocity. The data needs to be queryable in an aggregated format, which might require an ETL system for getting data into columnar format.

Alexander Pucher is an engineer at LinkedIn, where he works on a monitoring platform called ThirdEye. ThirdEye is built on top of Apache Pinot, a distributed columnar storage engine that ingests data and serves analytical queries at low latency. Pinot is comparable to Apache Druid.

Alexander joins the show to discuss ThirdEye, and explain why Pinot is a useful building block for monitoring infrastructure.

Transcript

Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.


Sponsors

Datadog is a cloud-scale monitoring platform for infrastructure and applications. And with Datadog’s new Live Container view, you can see every container’s health, resource consumption, and running processes in real time. See for yourself by starting a free trial and get a free Datadog T-shirt! softwareengineeringdaily.com/datadog.

Vettery is an online hiring marketplace that connects highly qualified job-seekers with inspiring companies. Once you’re accepted to Vettery, companies reach out directly to you. Sign up on vettery.com/sedaily and get a $500 bonus if you accept a job through Vettery.

GoCD is a continuous delivery tool created by ThoughtWorks. It’s great to see the continued progress on GoCD with the new Kubernetes integrations–and you can check it out for yourself at gocd.org/sedaily.

Get ready to build content-rich websites and professional web applications with Wix Code. Store and manage unlimited data with built-in databases, create dynamic pages, make custom forms and take full control of your site’s functionality with Wix Code APIs and JavaScript. Plus, now you can get 10-percent off your Premium plan. Go to Wix.com/SED.

Software Weekly

Software Weekly

Subscribe to Software Weekly, a curated weekly newsletter featuring the best and newest from the software engineering community.