Adding Taints and Configure Tolerations in Kyvos worker pods

You need to perform the steps mentioned below in this section if you want to run Kyvos worker pods on a specific node pool.

Prerequisites

1. You must have the required IAM permissions to add taints:

   • Editor or roles/container.admin or a custom role with sufficient permissions.

2. A separate Node pool is required to run system pools. Create a Node pool with the ‘n2-highmem’ configuration with Autoscaling disabled and count as 1.

3. For configuration, two separate node pools are required with different taints added to each node pool. For more information, refer to Adding taints to Node pool

Modify kyvos job yaml

After adding taints to Node pool, configure tolerations in the kyvos worker's YAML specification. Tolerations allow pods created by the job to be scheduled on nodes with matching taints.

1. Download the kyvos-compute-worker-job.yaml.template file. In the kyvos-compute-worker-job.yaml.template file, you will find the below section. In the section, configure the key and value with the taints configured in Step 2.
   tolerations:
   key: "key1"
   operator: "Equal"
   value: "value1"
   effect: "NoSchedule"

2. Copy the tolerations part and replace it in the kyvos-compute-worker-job.yaml.template file under the KYVOSMANAGER_PATH/cloud/gcp/terraform/kyvos-k8s/ section.

3. After configuring the tolerations, reapply the configurations from the Kyvos Native Page using Force Apply.

Add taints to a node pool

Taints allow you to mark nodes with specific restrictions, so pods will only be scheduled on them if they have corresponding tolerations.

Adding Taints to an Existing Node Pool

Option 1: Using gcloud Command

1. List your clusters and node pools to identify the target:
   gcloud container clusters list
   gcloud container node-pools list --cluster CLUSTER_NAME

2. Update the node pool to add taints:
   gcloud container node-pools update NODE_POOL_NAME \
   --cluster=CLUSTER_NAME \
   --node-taints=KEY=VALUE:NoSchedule 

3. Replace KEY and VALUE with the required taint details.

4. Verify the added taints using the kubectl describe nodes
    

Option 2: Using the Google Cloud Console

1. Go to the GKE Clusters page.

2. Click your cluster's name.

3. Navigate to the Node Pools tab.

4. Click the pencil/edit icon next to the node pool you want to modify.

5. Add the taints under the Node taints section.

6. Save the changes.

Adding Taints to a New Node Pool

When creating a new node pool, you can add taints during the creation process through:

• Command-Line:
  gcloud container node-pools create NODE_POOL_NAME \
  --cluster=CLUSTER_NAME \
  --node-taints=KEY=VALUE:NoSchedule \
  --num-nodes=NUMBER_OF_NODES

• Console:

1. Follow steps to create a new node pool.

2. Specify the taints in the Node taints section.

Additional Notes

If you want to add multiple taints, separate them with commas: --node-taints=KEY1=VALUE1:NoSchedule,KEY2=VALUE2:NoSchedule