Mesosphere DC/OS: The Operating System of the Distributed Cloud
Mesosphere provides the leading products for hybrid and multi-cloud infrastructure automation and data-driven application acceleration, Mesosphere DC/OS – Distributed Cloud Operating System.
The Complexity of Operating Multi- and Hybrid-Cloud Architectures
Enterprises are moving to the cloud to handle their growing scale, accelerate their deployment cycles, and reduce their costs. While cloud usage is growing each year in numbers, most enterprises still have legacy components that have to be hosted on on-prem infrastructure and some services might always need to be hosted on bare metal for security purposes. When thinking of enterprise cloud strategies most enterprises use some form of multi-cloud or hybrid architecture where they utilize both IaaS and on-prem infrastructure. Multi-cloud adoption rate has been increasing and multi-cloud users have doubled in the past two years.
Multi-cloud solutions are growing. To get the best of each public cloud provider’s technology and innovation and to avoid vendor lock-in, enterprises need an architecture that takes advantage of various public cloud provider offerings as well as private clouds for different workloads and different levels of control. Taking into account data privacy laws, especially in Europe which require personal data to be held within the state, enterprises need access to regions that only a combination of public cloud providers can grant. Deploying to multiple regions closer to your customer decreases latency allowing for a better user experience. The multi-cloud strategy also serves as protection against outages of a particular cloud vendor.
However, managing multi-cloud and hybrid architectures is challenging. If your applications use managed services from public cloud providers, such as streaming services, they will have to architect and develop against specific APIs. However, these APIs might not be available on-prem. The APIs for similar streaming services might be completely different between different cloud providers. One solution is using open-source services and managing them yourself across multi- or hybrid-cloud providers. This is requires a lot of operational work. It is expensive and inefficient.
Multi-cloud architectures also have the added problem of capacity planning and cost optimization. Public and private cloud providers have varying fees for the services they provide. Optimizing cost or optimizing resource allocation within different clouds requires careful management and monitoring. Moving applications between public, private, or on-prem infrastructure incurs large costs if parts of applications have to be rewritten to make the move.
Apache Mesos: A Distributed Systems Kernel
Apache Mesos is a distributed systems kernel built to solve the complexity of managing varying infrastructures. Mesos provides a consistent way of managing multiple cloud providers, on-prem, and any combinations. It allows abstracting the underlying infrastructure as a single pool of resources to be presented to frameworks running on top. These frameworks can be Kubernetes or Kafka that have their own scheduling mechanisms for jobs, workloads, and pods. Frameworks and Mesos together construct a two-level scheduling architecture.
Mesosphere DC/OS: Distributed Cloud Operating System
Mesos is in the heart of Mesosphere DC/OS. With Mesos and the two-level scheduling architecture, DC/OS enables multiple workloads with different and possibly varying resource requirements to be allocated efficiently. Since Mesos supports numerous types of distributed workloads like Kubernetes, Spark, Kafka, and Cassandra, organizations can run their distributed workloads without worrying about efficient resource utilization, which is automatically handled by Mesos.
On top of Mesos, with the resources it exposes as a single pool, developers can run diverse applications known as frameworks. These frameworks can be installed upon Mesos with a single click from the DC/OS Service Catalog. Developers can specify aspects of the installation, such as the amount of resources the service will use. It is also possible to implement and build a scheduler from scratch and install it on top of Mesosphere DC/OS.
Mesosphere DC/OS is an integrated platform for modern enterprises that want to use multi-cloud or hybrid infrastructure models. DC/OS works on every infrastructure, whether it’s a public cloud provider, private cloud, bare metal, or on the edge. This provides consistent and scalable operations. Developers can configure their services once and have them running in the same way across all their deployment targets.
On top of the platform-agnostic development freedom Mesos provides, saving many hours and developer effort, DC/OS cuts costs with highly efficient resource scheduling. In the industry, typically servers are over-provisioned leading to low utilization. Presenting all the resources as a single pool, DC/OS eliminates inefficient machine resource use. Diverse workloads can be run on the same DC/OS cluster sharing the resources efficiently. DC/OS scales workloads easily as they grow. Mesosphere DC/OS and Mesos’ efficient resource management can lead to more than 50% savings on infrastructure costs, as it did with Yelp.
While Apache Mesos is at the heart of its architecture, DC/OS provides more features than just what Mesos has to offer. DC/OS solves the problems that come with service orchestration. Applications such as Hadoop File System, Cassandra, TensorFlow, or Spark are necessary for data-driven enterprises and running these often distributed and stateful applications within containers is not desirable. DC/OS simplifies running these services on the same infrastructure, and fixes problems like lifecycle management, resource utilization, and multi-tenancy that usually come along with service orchestration.
DC/OS is truly an operating system for data centers. DC/OS configures storage, enables high availability for clusters, handles networking and routing, offers secrets storage, and multi-tenancy. It provides high security and privacy and a well-rounded data-integrated platform solution for the enterprise.
Running Kubernetes on Mesosphere DC/OS
Kubernetes has become an industry standard. However, Kubernetes is an intricate system and has a steep learning curve for engineers. DC/OS makes presenting infrastructure to Kubernetes and installing Kubernetes easy for developers.
Kubernetes is available from the DC/OS Service Catalog and can be installed as a workload on top of DC/OS. DC/OS in this way offers Kubernetes-as-a-Service and serves the upstream version of Kubernetes. With a single click, Kubernetes can be installed with highly available, self-healing clusters that can be deployed on any infrastructure. Kubernetes is a first-class citizen on DC/OS and can be run alongside other intensive workloads like Spark or Kafka. The tools that are usually installed alongside Kubernetes, such as Prometheus, are available within DC/OS Service Catalog.
Mesosphere Kubernetes Engine: Multi-Kubernetes Management on DC/OS
Mesosphere also provides a specialized Mesosphere Kubernetes Engine (MKE), allowing multiple Kubernetes clusters to be managed from a centralized control plane. MKE eases the management of Kubernetes clusters by improving and tightening security.
Without MKE, only one Kubernetes cluster could be spun up on top of a DC/OS cluster. With the introduction of MKE, multiple Kubernetes clusters can be installed on the same DC/OS cluster, expanding the flexibility and capabilities of DC/OS. MKE allows running multiple development-stage clusters on the same DC/OS cluster as prod clusters. This eases management and reduces costs.
Big Data with Mesosphere
Modern enterprises deal with huge amounts of data and spend a lot of resources extracting insights from their big data. Data analytics, statistical extraction, and machine learning have become integral parts of engineering organizations at the enterprise.
Mesosphere is taking big steps toward improving the development cycles of data-intensive application. The benefits of Mesosphere are inherent in DC/OS: the data-related services an enterprise needs, like message queues, stream processing engines, and persistent storage solutions are available as frameworks that can be added to a DC/OS cluster. Stateful and stateless services and elastic data processing applications can all co-exist and be deployed with a single click within the same DC/OS cluster. This reduces operational headaches, makes management as easy as a click, and allows more time to be spent on developing and improving the applications.
Mesosphere Jupyter Service
Mesosphere Jupyter Service is Jupyter-Notebooks-as-a-Service that allows data science teams to increase their productivity, shorten their development cycles, and iterate more. Mesosphere Jupyter Service provides Jupyter Notebooks as a fit for data scientists’ and machine learning engineers’ needs since it is pre-configured and ready to be used with a few clicks.
Mesosphere Jupyter Service allows teams of data scientists to securely share and work together on Jupyter Notebooks. These notebooks have secure connections to the data sources handled by DC/OS. With the interconnected structure of the notebooks and the shared underlying resource pool, Mesosphere Jupyter Service accelerates both DevOps cycles and machine learning training cycles.
Try Out Mesosphere DC/OS
Providing easier operations, lower infrastructure costs, and accelerated development cycles, Mesosphere DC/OS is the platform that every data-driven enterprise needs.
Mesosphere DC/OS is open source and Mesosphere offers an enterprise edition with security, multi-tenancy, secret management, added support, and easier hybrid-cloud operations. You can compare, see the benefits, and sign up for Mesosphere DC/OS here.
If you want to learn more about Mesosphere architecture, checkout SE Daily episodes Cluster Schedulers with Ben Hindman and Orchestrating Kubernetes with Chris Gaun.
