Adding Taints and Configure Tolerations in Kyvos worker pods

Taints allow you to mark nodes with specific restrictions, so pods will only be scheduled on them if they have corresponding tolerations.

Shared Kubernetes and Dedicated pool

Adding taints and configuring tolerations is needed.

If using a shared Kubernetes cluster with a dedicated user node pool, ensure that the user node pool has dedicated taints.

After creating taints, you must configure tolerations for Kyvos worker pods through Kyvos Manager in the kyvos-compute-worker-job.yaml.template file. This is applicable for all clouds (AWS, Azure and GCP).

Prerequisites for creating taints (GCP)

If you do not have taints created for GCP, perform the steps mentioned below in this section to run Kyvos worker pods on a specific node pool.

1. Ensure the user adding the taints must have the required IAM permissions to add taints

   • Editor or roles/container.admin or a custom role with sufficient permissions.

2. A separate Node pool is required to run system pods. Create a Node pool with the ‘n2-highmem’ configuration with Autoscaling disabled and count as 1.

3. For configuration, two separate node pools are required with different taints added to each node pool. For more information, refer to Adding taints to Node pool.

Add taints to a node pool

Adding Taints to a New Node Pool (GCP)

When creating a new node pool, you can add taints during the creation process through:

• Command-Line:
  gcloud container node-pools create NODE_POOL_NAME \
  --cluster=CLUSTER_NAME \
  --node-taints=KEY=VALUE:NoSchedule \
  --num-nodes=NUMBER_OF_NODES

• Console:

1. Follow steps to create a new node pool.

2. Specify the taints in the Node taints section.

Adding Taints to an Existing Node Pool for GCP

Option 1: Using gcloud Command

1. List your clusters and node pools to identify the target:
   gcloud container clusters list
   gcloud container node-pools list --cluster CLUSTER_NAME

2. Update the node pool to add taints:
   gcloud container node-pools update NODE_POOL_NAME \
   --cluster=CLUSTER_NAME \
   --node-taints=KEY=VALUE:NoSchedule 

3. Replace KEY and VALUE with the required taint details.

4. Verify the added taints using the kubectl describe nodes
    

Option 2: Using the Google Cloud Console

1. Go to the GKE Clusters page.

2. Click your cluster's name.

3. Navigate to the Node Pools tab.

4. Click the pencil/edit icon next to the node pool you want to modify.

5. Add the taints under the Node taints section.

6. Save the changes.

Additional Notes

If you want to add multiple taints, separate them with commas: --node-taints=KEY1=VALUE1:NoSchedule,KEY2=VALUE2:NoSchedule

Configure Tolerations

After adding taints to Node pool, configure tolerations in the kyvos worker's YAML specification. Tolerations allow pods created by the job to be scheduled on nodes with matching taints.

1. Login to Kyvos Manager.

2. Navigate to the Utilities > Manage Configuration Files.

3. To configure taints as per your requirement, modify the kyvos-compute-worker-job.yaml.template file.

   Open

   

4. Download the kyvos-compute-worker-job.yaml.template file. In the kyvos-compute-worker-job.yaml.template file, add the below section at the bottom of the file.
   tolerations:
   Replace KEY and VALUE with the required taint details.

   Open

   

5. Replace it in the kyvos-compute-worker-job.yaml.template file under the KYVOSMANAGER_PATH/cloud/gcp/terraform/kyvos-k8s/ section.

6. After configuring the tolerations, navigate to Kyvos and Ecosystem > Compute Cluster and then click Save to reapply the configuration to make it effective.