Building on Specialized Hardware
When Steve Jobs stepped on stage with his iconic black turtleneck and introduced the iPhone, it changed the way we interact and live. Many described it as magical at the time, but now we take it for granted. The mobile industry has never been the same since.
Each iterative refresh of the iPhone brings software, hardware, and middleware updates — shipped together as a cohesive, polished and (we hope) secure package. Apple’s control over all aspects of that experience have helped it get an edge against its competition by developing software that runs uniquely well on its hardware. It has also enabled new features like face recognition, avatars, fingerprint readers for payments and integration with other Apple devices.
The idea of building on specialized hardware is nothing new to Apple. Alan Kay, who worked at Apple and was influential in early portable computing concepts and a close collaborator with Steve Jobs, famously said, “People who are really serious about software should make their own hardware.” This was in 1982.
It’s no wonder that the concept of specialized hardware is so impactful in the mobile space, where everything is miniaturized, and efficiency and battery life are key. Traditional and especially cloud-based software development, on the other hand, have focused on the opposite side of the coin, with hardware abstracted away from the application developer whenever possible.
In a local development environment, everything but the overarching features of your laptop are abstracted away. Developing on any laptop is the same as long as they have the same architecture and operating system. In a cloud environment, PaaS and SaaS offerings can abstract away hardware even further, removing the concept of hardware completely from the end developer. However having such a high level of abstraction comes at a cost both in price and development.
Types of Cloud Services. Source: Aalpha
The idea of software being truly modular and independent from hardware is the core tenet of modern software development. The proliferation of Docker, Kubernetes, and the unlimited scale of public clouds has enforced the idea that you can write software without worrying about the underlying hardware.
However upon further inspection this statement breaks down. In application development, many modern programming languages have compilers that poll hardware in an attempt to apply architecture-specific, and in many times, cpu-specific optimizations. Virtualization software interacts with specific hardware to provide a standard execution environment and comes with relatively steep performance penalties. Even the hyperscale clouds such as Google and AWS that famously built their server backend on commercial hardware [Google’s Secret: ‘Cheap and Fast’ Hardware | PCWorld] in the early 2000s and gave credence to the golden standard that servers should be built on “commodity hardware” and are quickly moving to specialized hardware as they chase efficiency gains.
Traditional companies that build homegrown data centers using turnkey server solutions from OEM’s like Dell, HP, and Lenovo have long struggled to match the experience that developers get from public clouds. Private clouds have also struggled to match the efficiency and scale of performance that public clouds provide. While it is true that hyperscalers have talented infrastructure teams and innovative methods to build their data centers, that’s not enough to explain the vast differences between private and public clouds. Increasingly, it’s because public clouds are running optimized software on specialized hardware.
By buying into this idea that software should be truly abstracted from hardware you almost always incur financial cost. In a local development setting, you either have to spend more of your resources executing the code, which could lead to the need to buy a better computer. In a cloud environment, you end up paying the cloud provider to manage interacting with the hardware on your behalf. This leads to a trade-off between utilization, performance, and cost.
Building the hardware to suit your software — instead of the other way around — can increase performance and utilization dramatically.
Private vs Public Cloud Cost per Server. Source: 451 Research
The reality is, optimizing your hardware doesn’t make sense at small scale. However many companies are now coming to the realization that their cloud costs are skyrocketing. If you offer a product that is closely tied to infrastructure like servers and network delivery, it’s clear that these costs not only eat into your revenue, but limit your surface area for innovation. At some point, the advantages of more specialized hardware become apparent.
Companies and the community at large are realizing that building software with the underlying hardware in mind can yield significant advantages. Apple was frustrated with the performance and thermal characteristics they had with Intel CPUs, in part due to the fact that Intel CPUs are not built specifically for Apple’s use case. As a result, they recently released a lineup of new laptops and desktops that use custom Apple M1 chips, built upon the expertise developing chips for the iPhone. Their performance is often times better than their Intel-based offerings when the software is written to take advantage of the new architecture.
Macbook Pro Geekbench 5 Performance Benchmarks. Source: Macworld
Companies are also trying to build specialized server hardware that will enable private data centers to have the similar speed, scalability, and management advantages that hyperscalers like AWS and Google have developed for their own use. Having seen companies like Nebula, a now-defunct company that tried to provide a public cloud-like experience with “commodity hardware” and failed, new “bare-metal” companies are emerging that try to develop custom hardware for specific applications. Oxide took the same idea as Nebula but is building a specialized open source server that is specifically geared for data center users. By doing so they are looking to bring hyperscale-type benefits to smaller companies with specific needs.
Oxide Company Logo. Source: Crunchbase
As developers start looking to specialized hardware for performance gains, public cloud companies and data centers are beginning to offer access at the hardware level to developers. One such example is Equinix Metal with their “metal-as-a-service” offering, launched as Packet in 2014 and acquired by data center giant Equinix in 2020. Built with the end developer in mind, Equinix Metal enables developer-driven companies to build performant software by providing direct access to specialized hardware.
Equinix Metal Logo Source: Equinix
At the very beginning of computing, computers and mainframes often had custom programs or basic operating systems that were unique to their individual hardware. Programs were often written in machine code or as system instructions. With the widespread appeal of operating systems and personal computers in the late 1980s, programs finally had both the capability to and the need to be portable and hardware agnostic. As time has progressed, tools and software was developed and adopted to further divide software from hardware. However as we reach the end of Moore’s Law, we no longer have the ability to increase hardware performance for unoptimized software. In order to build faster and more complex software, we must start optimized first on hardware and eventually build hard for specific software applications.
