Elastic Load Balancing with Ranga Rajagopalan

Computational load is the amount of demand that is being placed on a computer system. “Load” can take the form of memory, CPU, network bandwidth, disk space, and other finite resources.

When we design systems, we need to prepare for high-load events. On a social network, people are much more active in the mornings. On an e-commerce site, Black Friday causes many more users to come online for discount shopping. Our distributed application must be able to scale in response to these spikes in traffic.

Cloud computing has changed the popular software architecture patterns, and load balancing has changed along with it. With on-demand, infinite infrastructure, we don’t need to worry about ordering servers and provisioning. With infrastructure as code, it becomes simpler to manage lots of deployable units–so we can break up our monolith into microservices, and have hundreds or thousands of virtual machines or containers running.

Enterprises that were started before cloud computing have large on-premise server deployments–but today, many of them also use the cloud. The cloud can be used to augment their classic on-prem deployments with cloud platform-as-a-service features. The cloud can also be used as a reliable way to scale during high load events.

Today, a common architectural pattern is to have your application broken up into services. Each of those services has multiple instances. When the load on a particular service is under lots of demand, you create more instances to handle the increased load. How do you monitor the load on each service? How do you know when to spin up new instances of the service?

Load analysis and load balancing across different services can be implemented by placing “agents” throughout your infrastructure. These agents gather data about services and service instances, and route that data to a centralized place. The centralized “control plane” can be used to make decisions about load-balancing and traffic routing.

Ranga Rajagopalan worked on networking at Cisco for a decade before co-founding Avi Networks as CTO. Avi Networks builds modern load balancing software, and in today’s episode, Ranga describes the requirements of load balancing. We talked about the evolution of network infrastructure, the impact of the cloud, and the technical decisions that his team has made when architecting Avi Networks. Full disclosure: Avi Networks is a sponsor of Software Engineering Daily.


Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.


Spring Framework gives developers an environment for building cloud native projects. On December 4th-7th, SpringOne Platform is coming to San Francisco. SpringOne Platform is a conference where developers congregate to explore the latest technologies in the Spring ecosystem and beyond. Speakers at SpringOne Platform include Eric Brewer (who created the CAP theorem), Vaughn Vernon (who writes extensively about Domain Driven Design), and many thought leaders in the Spring Ecosystem. SpringOne Platform is the premier conference for those who build, deploy, and run cloud-native software. Software Engineering Daily listeners can sign up with the discount code SEDaily100 and receive $100 off of a Spring One Platform conference pass. I will also be at SpringOne reporting on developments in the cloud native ecosystem. Join me December 4th-7th at the SpringOne Platform conference, and use discount code SEDaily100 for $100 off your conference pass.

You are programming a new service for your users. Or, you are hacking on a side project. Whatever you are building, you need to send email. For sending email, developers use SendGrid. SendGrid is the API for email, trusted by developers. Send transactional emails through the SendGrid API. Build marketing campaigns with a beautiful interface for crafting the perfect email. SendGrid is used by Uber, Airbnb, and Spotify–but anybody can start for free and get 100 emails per day. Just go to SendGrid.com/sedaily to get started. Your email is important–make sure it gets delivered properly, with SendGrid, the most reliable email delivery service. Get started with 100 emails per day at SendGrid.com/sedaily.

Simplify continuous delivery with GoCD, the on-premise, open source, continuous delivery tool by ThoughtWorks. With GoCD, you can easily model complex deployment workflows using pipelines and visualize them end-to-end with the Value Stream Map. You get complete visibility into and control of your company’s deployments. At gocd.org/sedaily, find out how to bring continuous delivery to your teams. Say goodbye to deployment panic and hello to consistent, predictable deliveries. Visit gocd.org/sedaily to learn more about GoCD. Commercial support and enterprise add-ons, including disaster recovery, are available.

Every second your cloud servers are running, they are costing you money. Stop paying for idle cloud instances and VMs. Control the cost of your cloud with ParkMyCloud. ParkMyCloud automatically turns off cloud resources when you don’t need them. Whether you are on AWS, Azure, or Google Cloud, it’s easy to start saving money with ParkMyCloud. You sign up for ParkMyCloud, you connect to your cloud provider, and ParkMyCloud gives you a dashboard of all your resources–including their costs. From the dashboard, you can automatically schedule when your different cloud instances get turned on or off – saving you 65% or more. Additionally, you can manage databases, auto scaling groups and set up logical groups of servers to turn off during nights and weekends when you don’t need them–and you can see how much money you are saving. Go to parkmycloud.com/sedaily to get $100 in free credit for ParkMyCloud for SE Daily listeners. ParkMyCloud is used by McDonald’s, CapitalOne, and Fox, and saves customers tens of thousands of dollars every month. Go to parkmycloud.com/sedaily, and cut the cost of your cloud today.




Software Weekly

Software Weekly

Subscribe to Software Weekly, a curated weekly newsletter featuring the best and newest from the software engineering community.