Google Cloud's Anthos Configuration Management

Management of any large-scale application infrastructure has become more difficult over the years. Many companies have moved to leverage Kubernetes in order to provide their development teams with a uniform platform and to empower these teams to handle more of the operational and deployment aspects of their applications. As the proliferation of Kubernetes to more organizations continues and as organizations leverage Kubernetes more often internally, the need to efficiently manage Kubernetes clusters themselves becomes even more critical.

Tantamount to the success of any unified offering for Kubernetes management is the ability to declare, monitor and manage the core elements of one or more clusters. And as Kubernetes extends further into edge computing, managing multiple clusters in disparate locations becomes an ever-more burdensome task. What we need is a way to define how we want a cluster to look, with customization if needed, and have the cluster bring itself into compliance.

Google offers Anthos Configuration Management (ACM) to fulfill this need. ACM (part of the Anthos platform) allows your organization to define Kubernetes objects (roles, security policies, namespaces and deployments) including a way to synchronize the configurations from a central repository. In addition, ACM implements the ability to specify that a particular part of the configuration only applies to specific clusters (such as pushing a particular application only to clusters located in North America). Finally, ACM helps to reduce copying of configurations by allowing a particular object to be applied automatically to multiple namespaces.

ACM is provided as part of Google Cloud's Anthos, Google's modern application platform, helping to run your workloads anywhere — incorporating tooling to help manage, develop, deploy and secure your services wherever they are located, in a public cloud or on-premises. Anthos is intended to provide a consistent working experience across multiple infrastructures, enabling IT staff to work on providing additional value to the organization.

Let's take a look at some of the specific ways Anthos Configuration Management delivers these features.

Kubernetes operator

ACM is deployed as a resident operator within the Kubernetes system, meaning it runs on the cluster it is managing. This allows for simplified networking as the communication is initiated from ACM outbound to the centralized repository for synchronization.

Policy repository

ACM utilizes git to manage the policy repository. The resident operator also uses git to fetch the current status of the repo as the first step in each synchronization cycle. Since the objects are all in plain text (normally YAML or JSON files), almost all tools and processes that can be used on a normal git workflow can also be used to help manage the policy development process. This can include limited push capability on the production branch, pull/merge requests, read-only repos and mirroring capabilities.

Simplifying development

Rather than defining a custom language to express Kubernetes objects, ACM uses Kubernetes' definitions themselves. These definitions are arranged such that a particular object can be automatically deployed into one or more namespaces. For example, you might need to define a common ConfigMap that is copied to all namespaces to set the location of a common assets repository.

Regardless of the location of a configuration, you can further restrict what cluster or namespace it is deployed to by the use of ClusterSelectors and NamespaceSelectors. For both types of selectors, you can use one or more labels on a given cluster or namespace to select it for a given selector. For example, you might define all clusters to have a country label with the country they are located in and choose to deploy different applications based on that label.

Anthos Configuration Management components

Google has introduced additional components to expand the capabilities of ACM even further.

Config Connector can be automatically added to a cluster when installing Anthos Configuration Management, and is used to configure Google Cloud resources using Kubernetes objects. For example, you can create a PubSub Topic and Subscription from within a Kubernetes cluster, which can then be used by a deployed application. This can help to empower employees who are proficient with Kubernetes, but wish to leverage GCP resources.

Policy Controller is an admission controller that leverages the open source Open Policy framework to provide administrators to define a set of constraints on what types of resources can be created. Google provides an initial set of constraint templates to allow users to easily define some of the most common constraints. For organizations that want to track deployments back to a specific cost center, adding a constraint that requires all deployments to define the cost-center label would prevent changes to deployments that do not include that information.

Example implementations

Any organization can leverage ACM to standardize and centralize their Kubernetes policies and deployments, but the components used will vary based on your specific needs. Here we have two specific examples, implementing ACM across several clusters.

Financial Services Company

The Financial Services Company (FSC) is just starting their migration to Kubernetes and have decided to go with a very hierarchical representation of their policies. This allows them to group similar namespaces together to simplify deployment of similar configurations and secrets to a subset of namespaces.

In addition, they are using different external services for their development and test clusters, versus their production cluster. ACM makes this type of separation trivial through the use of ClusterSelectors, and applying those selectors to the different objects. Financial Services Company has decided to store those in a named ConfigMap and store separate configs in production vs the non-production environments:

prod-external-svc-cm.yaml
-------------------------------------------------
kind: ConfigMap
apiVersion: v1
metadata:
  name: external-svc
  annotations:
    configmanagement.gke.io/cluster-selector: selector-env-prod
data:
  EXTERNAL_ROOT_URL: https://api.example.com/v1

nonprod-external-svc-cm.yaml
-------------------------------------------------
kind: ConfigMap
apiVersion: v1
metadata:
  name: external-svc
  annotations:
    configmanagement.gke.io/cluster-selector: selector-env-nonprod
data:
  EXTERNAL_ROOT_URL: http://api-np.example.com/betav2

Since the name of the ConfigMap is the same, only one config will be applied to each cluster. The cluster-selector annotation is used to choose a specific ClusterSelector (defined in a central location) to deploy each configuration. A quick visual representation of this deployment, with a few additional objects can be seen here:

policy repository for financial services company — A portion of the policy repository for Financial Services Company. Since FSC is using a hierarchical repository, the contact-config ConfigMap is copied into each namespace under that directory.

As they move forward, FSC will continue to leverage the auditability and tracking inherent with ACM to fulfill regulatory requirements and ensure that all modifications have the appropriate approvals.

Fast Casual Restaurant Group

The Fast Casual Restaurant Group (FCRG) has been using Kubernetes for a couple of years but wants to centralize and simplify the management of deploying applications to the clusters located at each store. The development teams have become very proficient at generating the Kubernetes manifests for each application as they need using an external templating engine. Therefore, FCRG is using an option in ACM that allows for a more free-form policy organization; each development team can drop new manifests into a directory without making huge changes to their CI/CD processes.

However, since FCRG has chosen to use the free-form style, each object needs to define the namespace it should be placed in or use a NamespaceSelector to deploy to multiple namespaces. NamespaceSelectors are defined in a similar fashion to the ClusterSelectors that Financial Services Company used above but are used for namespaces rather than clusters.

For example, we can use a NamespaceSelector to deploy the same RBAC RoleBinding to multiple namespaces without needing to create each object individually:

boh-rbac-binding.yaml
-------------------------------------------------
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: boh-dev-binding
  annotations:
    configmanagement.gke.io/namespace-selector: sel-ns-boh
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: general-dev-role
subjects:
- kind: Group
  apiGroup: rbac.authorization.k8s.io
  name: boh-devs

Each application is split into its own directory, and the entire repository is evaluated to determine which objects to deploy to which cluster and namespace:

policy repository for Fast Casual Restaurant Group — A portion of the policy repository for FCRG. The general-dev-role is defined with a NamespaceSelector covering all non-k8s namespaces. The boh-dev-binding is only deployed to the 'boh' namespaces, identified by a label on those namespaces.

By leveraging the NamespaceSelectors, FCRG is able to continue using many of their existing CI/CD processes, while leveraging the capabilities of ACM — including the ability to use pull requests to approve upgrades and to rapidly upgrade all locations from a central repository.

Summary

Anthos Configuration Management is intended to make the job of IT professionals easier when managing Kubernetes clusters. Even with the goal of simplicity, there are many ways your particular organization can choose to use ACM as part of your Kubernetes management workflow. WWT is here to help you, whether you are just getting started on your cloud journey or are already well on your way. We can help you organize your policies, implement ACM in your environment and develop appropriate processes and controls for managing your Kubernetes clusters.

Contact your WWT Account team or reach out to the WWT Google Practice for help with your Anthos ACM, Google Cloud Platform or multicloud architecture initiative.