{"id":9931,"date":"2020-09-22T08:00:55","date_gmt":"2020-09-22T15:00:55","guid":{"rendered":"http:\/\/softwareengineeringdaily.com\/?p=9931"},"modified":"2020-09-21T20:49:17","modified_gmt":"2020-09-22T03:49:17","slug":"what-makes-a-database-a-good-fit-to-run-in-kubernetes","status":"publish","type":"post","link":"https:\/\/softwareengineeringdaily.com\/2020\/09\/22\/what-makes-a-database-a-good-fit-to-run-in-kubernetes\/","title":{"rendered":"What Makes a Database a Good Fit to Run in Kubernetes?"},"content":{"rendered":"

As more organizations shift operations to the cloud, the benefits of cloud-native software and infrastructure <\/span>continue to accrue.<\/span><\/a> One critical task in managing a cloud-based microservices architecture is container orchestration. Kubernetes is the container orchestrator of choice for 78% of IT Leaders identified in Red Hat\u2019s <\/span>\u201cState of Open Source 2020\u201d report.<\/span><\/a> It\u2019s a powerful and flexible tool, but certain issues remain that can make <\/span>managing storage difficult.<\/span><\/a> To meet this challenge, NewSQL and, more recently,<\/span> Distributed SQL databases<\/span><\/a> like CockroachDB have sought to reimagine how databases work from the ground up. NewSQL aims to merge the best features of both RDBMS systems and NoSQL systems to create a truly \u201ccloud-native\u201d database. Distributed SQL databases take that a step further for usability, by seamlessly integrating with microservices like Kubernetes while still speaking the language of data: SQL.\u00a0<\/span><\/p>\n

By definition, databases are considered stateful applications, while Kubernetes was intended to facilitate the management of stateless applications that are connected to external services to store state if necessary. The guarantees which Kubernetes offered- for example, high availability of services- were premised on the idea that the underlying pods were effectively ephemeral and interchangeable. In the early days of Kubernetes, it was an <\/span>open question <\/span><\/a>whether it would be convenient, or even possible, to run a database in a cluster.<\/span><\/p>\n

Kubernetes manages containers dynamically, meaning that individual containers may be brought up or taken down at any time to match the needs of the application as a whole. This works fine for stateless services, but a stateful, persistent service like a database cannot afford to be restarted arbitrarily. In the past, developers often chose to run databases outside Kubernetes, which was problematic because it required redundant infrastructure management tools for the external database, such as monitoring and service discovery. Several<\/span> methods<\/span><\/a> have been added to Kubernetes, which can help manage storage, such as Persistent Volumes and Storage Class objects. Since the 1.5 release, Kubernetes has offered <\/span>StatefulSets<\/span><\/a> (<\/span>and the related, but more complex DaemonSets<\/span><\/a>), ensuring that pods retain the same unique ID even if they are rescheduled to another machine. Connecting these pods to a remote \u201c<\/span>Persistent Volume<\/span><\/a>\u201d via their unique ID allows the application to maintain state, even when pods are being created and destroyed. Since version 1.14, Kubernetes has supported \u201c<\/span>Local Persistent Volumes<\/span><\/a>\u201d as well, allowing developers to use the same PersistentVolume APIs to connect to either Remote or Local storage.<\/span><\/p>\n

\"\"<\/a><\/p>\n

The addition of StatefulSets and the ability to connect pods to persistent storage should definitively answer whether it was <\/span>possible<\/span><\/i> to run a database alongside a cluster. But simply adapting technology designed with bare-metal in mind to a cloud environment is a suboptimal solution, and inevitably some of the benefits of the cloud will be compromised. Legacy RDBMS systems still lack certain features that would make them <\/span>ideal<\/span><\/i> for running in Kubernetes. On the other side of the aisle, the trade-offs made by many NoSQL databases to make them more cloud-friendly mean they have lost a degree of usability power that developers may depend on in cloud-native applications (in particular, many have their own querying language). In order to understand why some NoSQL databases made this trade-off, it\u2019s important to align on a single definition of \u201ccloud-native.\u201d Here\u2019s the official one from the CNCF (Cloud Native Computing Foundation)<\/span><\/p>\n

\u201cCloud-native technologies empower organizations to build and run scalable applications in modern, dynamic environments such as public, private, and hybrid clouds. Containers, service meshes, microservices, immutable infrastructure, and declarative APIs exemplify this approach. These techniques enable loosely coupled systems that are resilient, manageable, and observable. Combined with robust automation, they allow engineers to make high-impact changes frequently and predictably with minimal toil.\u201d<\/span><\/p>\n

Source<\/span><\/a>\u00a0<\/span><\/p>\n

Or, more simply, a cloud-native application is \u201c<\/span>architected to take full advantage of cloud platforms.\u201d<\/span><\/a> A cloud-native<\/span> database should make the most of these advantages <\/span>without <\/span><\/i>sacrificing the features which made RDBMS systems dominant for decades. The only databases that do that well are Distributed SQL databases, which bring together the benefits of NoSQL and the features that make RDBMS systems powerful. Now that we know we can run a database in Kubernetes, what criteria can we use to choose <\/span>which<\/span><\/i> database to run? As exemplified below, most databases make you choose a trade-off between ease of use and horizontal scalability.\u00a0<\/span><\/p>\n

\"\"<\/a><\/p>\n

Nate Stewart, the Chief Product Officer at Cockroach Labs, describes \u201cDASH\u201d factors that help align databases with the goals and function of Kubernetes. Quoting at length <\/span>from his article on DASH:<\/span><\/a><\/p>\n