Accessing Data at Scale with Justin Borgman

Nov 11, 2022 by SED

The Presto/Trino project makes distributed querying easier across a variety of data sources. As the need for machine learning and other high volume data applications has increased, the need for support, tooling, and cloud infrastructure for Presto/Trino has increased with it.
Starburst helps your teams run fast queries on any data source. With Starburst you get a single point of access to your data, no matter where it’s stored and it supports high concurrency. Whether it’s fast SQL queries on your data lake or faster queries across multiple datasets, Starburst helps your teams run analytics anywhere. Justin Borgman is the CEO of Starburst, and he joins us today. Sponsorship inquiries: sponsor@softwareengineeringdaily.com ᐧ

Thanks to our sponsors

This episode is brought to you by:

Dataset
Do you wish your log analytics platform could keep pace with your modernization to containers, Kubernetes, or microservices? Is the high cost of your log solution forcing you to sample and discard data, or do you have developers spending endless cycles to scale traditional tools?
If so, then it’s time to modernize your log analytics solution.
DataSet is a cloud-native log analytics platform that helps DevOps, IT, engineering, and security teams get answers from their event data across all use cases and time periods – streaming or historical. Powered by unique architecture that separates compute from storage and uses a massively parallel query engine, DataSet delivers the unmatched combination of blazing-fast queries and effortless scalability at a fraction of the cost.
Modern companies such as DoorDash, Asana, Shipt, Zalando, and hundreds of others use DataSet to get actionable insights, drive better collaboration, improve mean-time-to-resolution, and deliver exceptional end-user experiences.
So what are you waiting for? Get complete visibility into your entire cloud stack and experience DataSet free. Start today at https://www.dataset.com/sed/.
http://privacydynamics.io/ sedaily
This segment of Software Engineering Daily is brought to you by Privacy Dynamics, the simple solution for anonymizing data intelligently.
Development and testing environments are the number one source of costly data breaches. Privacy Dynamics creates a de-identified copy of your production data in minutes, allowing developers to maintain their velocity and improve testing without taking on unnecessary compliance and security risks.
Using a proprietary approach based on the latest research, Privacy Dynamics automatically detects and eliminates PII while maintaining the schema, format, scale, and analytical utility of the source data. With a simple integration into any data store, Privacy Dynamics anonymizes data in Postgres, MySQL, Snowflake, Big Query, S3 and more.
Are you letting sensitive PII land in development and testing environments? Eliminate the #1 threat of a data breach and start using de-identified data in your development and testing environments by signing up for a free account today at http://privacydynamics.io/sedaily.
download Azul Platform Prime for free
This portion of Software Engineering Daily is brought to you by Azul.
Trusted by engineers and high performing development teams around the world as the only company focused solely on Java. Azul gives you more performance, scale, throughput, and security. More of the Java you love, at the price you can afford. Using an optimized Java Virtual Machine (JVM), Azul Platform Prime turbocharges the performance and scalability of your Java applications with a hyper-optimized runtime that drives down infrastructure costs. Say goodbye to latency lag: download Azul Platform Prime for free.
www.merge.dev/sedaily
Another Monday morning. Sprint stand-up. Your next task is … drumroll… building an app integration. RIP. Excited to spend the next month getting your hands on developer API keys, fiddling with Oauth, testing SOAP requests, and debugging a poorly documented API? We didn’t think so either.
Now, what if there were an easier way?
Meet Merge: a Unified API for B2B apps. Instead of building lots of integrations to widely varying APIs, you can integrate with a single API and support any integration your customers want. Think of Merge as a universal translator, a Roseta Stone for APIs. Merge returns normalized data across 100s of integrations, all synced continuously, and available in REST API. No SOAP or web scraping required.
Engineering teams at Ramp, TripActions, and Drata use Merge to build dozens of integrations in weeks, speed up time to market, and make their customers happy.
If you’re working on integrations with accounting, CRM, HR, project management, recruiting, ticketing apps, and more, check out Merge today at www.merge.dev/sedaily.
https://flatironsdevelopment. com/products/fuse
Flatirons Fuse is the solution to your import challenges. Don’t spend time and money developing a custom solution to import customer CSV and spreadsheet data. Let us provide your customers a simple and intuitive interface, and make onboarding a breeze. Start today for free at https://flatironsdevelopment.com/products/fuse.