March 11, 2024
Red Hat Advanced Cluster Management (RHACM) enables Open Cluster Management policy driven governance of an entire fleet of Kubernetes clusters. Associating policies with the appropriate clusters is a very flexible operation and requires understanding resources like Placements and ManagedClusterSetBindings. So let’s get familiar!
Open Cluster Management
Red Hat Advanced Cluster Management for Kubernetes implements the Open Cluster Management APIs for the deployment (placement) of governance policies.
Policies
Policy resources are used to enact policies on a managed cluster. There is a large collection of policies that you can draw upon and customize for your particular use cases. This post focused only on the process of associating a policy with a cluster rather than what is in a given policy or how it is enacted.
Policies are scoped to a namespace as you can see in the API resources table above. Policies can be associated to specific clusters in a fleet through a Placement or a PlacementRule resource.
Placement Rules
In the recent past resources called PlacementRule and PlacementBindings were used to associate a policy with a cluster. Those resources still exist but PlacementRule is now deprecated in favor of the Placement resource. This change enables greater control, and brings with it some other resources to facilitate the grouping of clusters into sets. These cluster sets enable more configurable RBAC.
Managed Clusters and Managed Cluster Sets
RHACM can manage OpenShift and other Kubernetes clusters. See my previous posts on RHACM for more background and recorded demonstrations.
Each Kubernetes cluster managed by RHACM is represented by a ManagedCluster resource. This resource is cluster scoped. You can see this in the table below.
Multiple managed clusters may be grouped together into ManagedClusterSets for administrative and RBAC purposes. RHACM has two pre-defined cluster sets named Global and Default. All clusters will be in the Global set, and they can be added to at most only one other set. You might create a cluster set for lifecycle like Dev or cloud like AWS for example.
Managed Cluster Set Bindings
A ManagedClusterSetBinding is a namespaced resource designed to create a relationship between a namespace and a ManagedClusterSet. This resource is prerequisite to a Placement. Without this, a placement has no clusters visible to match against.
The Global managed cluserset set is already bound to the open-cluster-management-global-set by default. You can see this in the console when listing the cluster sets.
Cluster sets and Namespace bindings view
Placement
A Placement exists to make a scheduling decision with regard to Policies and a selection of ManagedClusters. The result of a placement decision is a list of ManagedClusters.
The placement decision begins with a list of managed clusters visibile through the lense of a ManagedClusterSetBinding in the same namespace. Once the placement has this list it may optionally further winnow down that list through predicates like label selectors.
📓 Important
Placementlabel selectors only match clusters that are visible via aManagedClusterSetBinding- No clusters will be selected if no cluster set is bound to the namespace holding the placement resource
⭐ Pro Tip Use
oc explain placementto find the information I reformatted below.
Here is how the placement policy combines with other selection methods to determine a matching list of ManagedClusters:
- Kubernetes clusters are registered with the RHACM hub as cluster-scoped
ManagedClusters ManagedClustersare organized into cluster-scopedManagedClusterSetsManagedClusterSetsare bound to workload namespaces usingManagedClusterSetBindings- Namespace-scoped
Placementsspecify a slice ofManagedClusterSetswhich select a working set of potentialManagedClusters - Then
Placementssubselect from that working set using label/claim selection.
Subselecting from Cluster Sets with Predicates
A placement can explicitly list the ClusterSets of interest or it can simply list the cluster label selectors. BUT the universe of possible clusters is constrained to those bound to the namespace via the managed cluster set bindings.
Here are examples of both use cases. Of course they may also be combined into one.
1️⃣ If a
Placementspecifiesspec.clusterSets[]then it may specify aspec.predicates.requiredClusterSelector{}to filter the set of managed clusters.
apiVersion: cluster.open-cluster-management.io/v1beta1
kind: Placement
metadata:
name: placement-1
namespace: ns1
spec:
clusterSets:
- clusterset1
- clusterset2
2️⃣ If a
Placementdoes not specify anyspec.clusterSets[]it may still specify aspec.predicates.requiredClusterSelector{}which will only match against clusters found in sets bound to the current namespace (by aManagedClusterSetBinding.)
apiVersion: cluster.open-cluster-management.io/v1beta1
kind: Placement
metadata:
name: placement-2
namespace: ns1
spec:
predicates:
- requiredClusterSelector:
labelSelector:
matchExpressions:
- {key: environment, operator: In, values: ["dev"]}
When more than one predicate is defined, they are OR’ed. Refer to the documentation for more detail.
Placement Binding
Once a placement produces a list of managed clusters the policy resource must be associated with and ultimately propagated to the clusters. This is the job of the PlacementBinding.
📓 A
PlacementBindingbinds aPlacementto subjects (eg.Policies,PolicySets)
This example is pulled from a documentation sample.
apiVersion: policy.open-cluster-management.io/v1
kind: PlacementBinding
metadata:
name: binding-policy-role
placementRef:
name: placement-policy-role
kind: Placement
apiGroup: cluster.open-cluster-management.io
subjects:
- name: policy-role
kind: Policy
apiGroup: policy.open-cluster-management.io
Fitting it all together
In this visual representation there are 3 corporate standard policies which we expect to apply to all 4 of our clusters, and there is 1 additional policy which we expect to apply to the production clusters.
This is accomplished by creating a standard (“STD”) Placement that targets all clusters and a PROD Placement that matches only the production cluster. This targetting is permitted by the existance of the ManagedClusterSetBinding in this same namespace.
cluster set by labels"} end
Summary
ManagedClusterSetBindings provide a controlled lense through which ManagedClusterSets may be managed. By controlling the creation of ManagedClusterSetBindings, the Policy management task may be delegated to a namespace administrator while retaining the control over which clusters may be impacted by that namespace administrator.