OctoML: Automated Deep Learning Engineering with Jason Knight and Luis Ceze

The incredible advances in machine learning research in recent years often take time to propagate out into usage in the field. One reason for this is that such “state-of-the-art” results for machine learning performance rely on the use of handwritten, idiosyncratic optimizations for specific hardware models or operating contexts. When developers are building ML-powered systems to deploy in the cloud and at the edge, their goals to ensure the model delivers the best possible functionality and end-user experience- and importantly, their hardware and software stack may require different optimizations to achieve that goal.

OctoML provides a SaaS product called the Octomizer to help developers and AIOps teams deploy ML models most efficiently on any hardware, in any context. The Octomizer deploys its own ML models to analyze your model topology, and optimize, benchmark, and package the model for deployment. The Octomizer generates insights about model performance over different hardware stacks and helps you choose the deployment format that works best for your organization.

Luis Ceze is the Co-Founder and CEO of OctoML. Luis is a founder of the ApacheTVM project, which is the basis for OctoML’s technology. He is also a professor of Computer Science at the University of Washington. Jason Knight is co-founder and CPO at OctoML. Luis and Jason join the show today to talk about how OctoML is automating deep learning engineering, why it’s so important to consider hardware when building deep learning systems, and how the field of deep learning is evolving.

Sponsorship inquiries: sponsor@softwareengineeringdaily.com

Transcript

Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Sponsors

Indent is a system for simplifying access management across cloud apps and infrastructure. Indent allows users within an organization to request and review access to resources, and keep logs of the changes to who can access those resources. Indent is recruiting product and infrastructure engineers to solve this problem. Want to learn more about working with Indent? Go to keyvalues.com/indent or listen to their interview at indent.com/sedaily.

DataStax Astra does the heavy lifting of managing infrastructure, serverless scaling, operations, and creating data access APIs. It’s the easy button for a scale out, always on, database-as-a-service that spans the globe. Get started on any cloud in 5 minutes or less. Go to datastax.com/sedaily to sign up today and get a free $300 credit with promo code SEDAILY.

Stream provides an easy-to-integrate chat solution for any application. With robust SDKs and an API built for ease of use, scalability, reliability, and security, product teams can focus on what makes their app unique, rather than spending months on building a chat infrastructure. Stream’s feature-rich products include robust client-side SDKs for iOS, Android, React, React Native, Flutter, and support for the most commonly used server-side languages; scalable and secure APIs; and a beautiful UI kit. Check it out at getstream.io/SED. 

strongDM lets you manage and audit access to servers, databases, and Kubernetes clusters, no matter where your employees are. With strongDM, you can easily extend your identity provider to manage infrastructure access. You can automate onboarding, offboarding, and moving people within roles. strongDM. Manage and audit remote access to infrastructure. Start your free 14 day trial today at: strongdm.com/SEDaily