MongoDB Atlas: Database as a Service
MongoDB is a sponsor of Software Engineering Daily.
Databases are essential to companies. Database technology has been in development for more than 40 years, and numerous options have appeared due to advancements in storage, RAM, cloud architectures, and more. There are a dazzling number of options on the field for database solutions: SQL, NoSQL, NewSQL, relational or graph databases, in-memory, or disk. In this article we will focus on cloud databases, and more specifically, Database-as-a-Service (DBaaS) solutions such as MongoDB Atlas.
Traditionally, before the age of cloud solutions, data was kept locally in servers of a company. This concept is known as on-prem. With the rise of cloud and cloud-based architectures, databases adapted to the change, and storing the data on the cloud rather than local servers created cloud databases. New cloud solutions and architectures emerged throughout the database landscape. Hosting the data on the cloud brought several advantages over the on-prem model:
- Lower entry costs. On-prem databases usually require an upfront capital investment, both in terms of hardware and hiring database administrators. If the roadmap of a company is not designed realistically, over-investing into the on-prem structure can cripple a company financially. The entry level is financially much lower with cloud solutions. A company or a developer can start with a small infrastructure and gradually scale up. The costs are approximately stable for the lifetime of a company. While future planning problems can still apply, in terms of under- or over-provisioning, it is easier to recover from such mistakes.
- Easier scalability. Scalability is an important benefit as with most cloud applications. Not having to deal with the underlying infrastructure issues makes scaling easier and takes less effort from the developer side. While CPU and RAM can be scaled seamlessly in the cloud, scaling storage is harder, and migrating the data between instances for rebalancing might be necessary. However, this same problem can be observed in on-prem solutions along with the added challenge of infrastructure design and configuration.
- Smaller backup intervals and smoother recovery.
- Geographical diversity. It’s a good practice to have the data required for your application closer to the end users to have a more responsive and faster user experience with lower network latency. Public cloud providers allow provisioning servers from multiple regions, e.g., AWS offers 18 regions with 55 availability zones. Besides making the service faster, different regions and availability zones offer resiliency against outages.
However, cloud databases bring challenges as well. Examples include privacy and security concerns, managing configurations and updates, ensuring high availability and consistency, performance, monitoring, scalability issues, and data migration.
There are essentially two ways of leveraging cloud databases: provisioning servers from an Infrastructure-as-a-Service (IaaS) platform and configuring the database on these servers, or using a DBaaS solution. While most IaaS providers offer pre-configured servers dedicated for data storage, it is possible to spin up virtual machines and set up your own database clusters manually as well. On the other end of the spectrum, with a great ease of use, Database-as-a-Service solutions offer automated cloud services managed by the DBaaS provider. A DBaaS product launches and maintains the database, and the developers are only concerned with actually utilizing the database in their applications.
A DBaaS provider that overcomes these challenges is the fully automated cloud service MongoDB Atlas. Atlas is the product of and is currently run by the people who actually built MongoDB. They incorporated the years of experience they have gained from MongoDB and the best practices they have gathered. Atlas can run on top of the three major IaaS providers: AWS, MS Azure, and GCP.
Let’s take a look at how some of these challenges are handled:
- High availability. High availability (HA) is the metric measuring whether a system has been continuously operational for a desirably long time. High availability is very important for companies because every minute of downtime is a lost opportunity for sales. From a broad perspective, achieving HA requires dealing with single points of failure through redundancy and figuring out what happens in the case of failures. The two most widely used architectures are master-slave and multi-master architectures. On a self-managed cloud database, these choices have to be made by the database administrators and possibly third-party tools have to be used. In the case of node failures, one of the slaves might have to be promoted manually or a tool must be used to automate this process. While most IaaS providers offer to have multiple copies of your data in different availability zones, these also need to be managed. MongoDB Atlas, on the other hand, achieves high availability with MongoDB’s replication capability. Atlas clusters are either replica sets or sharded cluster where each shard is a replica set. Atlas provides a fault-tolerant architecture with self-healing capabilities. Atlas ensures minimum downtime during a primary node failure or maintenance jobs such as updates, with retryable writes, and an election-promotion process in the time frame of a few seconds. What’s more, data can be replicated between regions for faster performance and better availability guarantees.
- Privacy and security. Database security covers a very broad area of practices and is an important aspect of database design, configuration, and management. Privacy becomes highly valuable especially if the data contains personal information. With regulations like GDPR and HIPAA, companies need to be compliant with their practices. Ensuring privacy and security can be easier when you have the actual control over the on-prem servers. However, if your data is in some way accessed by users directly, there are still numerous risks shared with cloud databases. With non-automated databases on the cloud, there are various dangers the developers and database administrators need to be careful about including data breaches, fault API endpoints, account hijacking, etc. This requires heavy monitoring, continuous security patches and tests, and a deep understanding of the database that’s being used. On a successful and compliant DBaaS handling personal data, security and privacy are built in the design with powerful authentication and authorization practices, encryption during transactions and in-rest, resilience, and also monitoring abilities for alerting. MongoDB Atlas has inherent security features such as TSL/SSL for encryption and Virtual Private Cloud (VPC) structures that cannot be disabled. There are optional security features as well like 2FA and LDAP for authentication and offers compliance with many regulations.
MongoDB Monitoring Dashboard. - Monitoring. Logs and alerts supported by the cloud provider are the main source of truth for database administrators since physically accessing the infrastructure that holds the data is not possible. Database monitoring mainly involves monitoring queries coming into the database, the performance, and consumption of resources. Monitoring has a high value both from the point of view of optimizing the database to perform well against the incoming queries and analyzing the behaviors of the end-users and finding trends to take as inputs for updates, or further products. While cloud providers like AWS include monitoring tools like CloudWatch, most cloud services require additional tools for more sophisticated monitoring. DBaaS services, like Atlas, provide monitoring out of the box. Atlas also gives MongoDB activity logs alongside performance metrics and alerts that fire when resource usage is closer to the threshold.
These are the technical aspects related to infrastructure and database design. However, for tech companies, how the underlying employee base is utilized is becoming ever more important. As a polar opposite to the tech companies of the 80s and 90s, hardware and software are getting cheaper and the costs of finding talented, suitable engineers are getting higher. DBaaS solutions outsource the work of creating and maintaining databases. In the case of MongoDB Atlas, this outsourcing is forwarded to the same team that built the database thus making it more feasible. More time is left for developers to work on the actual application.
DBaaS offers a great relief both for companies that are just getting started and for companies that want to ease their database management efforts. You can get started with MongoDB Atlas here, starting with a free account with 512 MB storage for experimentation purposes.
