Spot Instances with Amiram Shachar

When a developer provisions a cloud server, that server is called an “instance”. These instances can be used for running whatever workload a developer has, whether it is a web application, a database, or a set of containers.

The cloud is cheap to get started on. New applications with few users can often be hosted on infrastructure that is less than $10 per month. But as an application grows in popularity, there is more demand for CPUs and storage. A company will start to buy more and more servers to scale up to the requirements of their growing user base. The costs of running infrastructure in the cloud will increase, and the company will start to look for ways to save money.

One common method of saving money is to buy “spot instances”. A spot instance is an instance that is cheaper than “reserved instances” or “on-demand” instances. The reason that there are different instance types is because a giant cloud provider has a highly variable amount of work that is being demanded from that cloud provider.

If you are in charge of AWS, you have to make sure that at any given time, you can give server resources to anyone that asks for it. Your data centers need to have physical machines that are ready to go at any time. This means that much of the time, you have server resources that are going unused.

If you are a cloud provider, how can you get people to use your compute resources? You can make them cheaper. So a user can come along and buy your compute at the discounted “spot” price.

But this presents a problem for the cloud provider. If you start to give away your compute at cheaper prices, and then the overall demand for your cloud resources go up once again, you are going to miss out on profits. As the cloud provider, you need to kick people off of your spot instances, so that you can take those same instances and sell them to people at the higher market prices.

And this presents a problem for the user. If you buy a cheap spot instance, that instance is only available until the cloud provider decides to kick you off. You have a tradeoff between cost and availability of your instances. Because of this, spot instances are typically used only for workloads that are not mission critical–workloads that can afford to fail.

Spotinst is a company that allows developers to deploy their workloads reliably onto spot instances. Spotinst works by detecting when a spot instance is going to be reclaimed by a cloud provider and re-scheduling the workload from that cloud provider onto a new spot instance.
Amiram Shachar is the CEO of Spotinst. He joins the show to talk about the different types of instances across cloud providers, the engineering behind Spotinst, and how the usage of containers and the rise of Kubernetes is changing the business landscape of the cloud.

SE Daily

Transcript

Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.

Sponsors