Prerequisites to deploy Kyvos using Kubernetes

To deploy Kyvos using Kubernetes, read the Common Prerequisites for Dataproc and Kubernetes section for the complete set of permissions required for deploying Kyvos.

You can deploy Kyvos by creating a GKE cluster or using an existing GKE cluster.

• Subnets in which Kubernetes cluster is launched should have connectivity to the subnets in which Kyvos instances are launched.

• When using an existing VPC, ensure that the subnet has two secondary IP ranges with valid mask ranges, as these will be used by the Kubernetes cluster.

Creating a GKE cluster

To create a GKE cluster, complete the following prerequisites.

• Ensure that the GKE service agent’s default service account (service-PROJECT_NUMBER@container-engine-robot.iam.gserviceaccount.com) has the Kubernetes Engine Service Agent role attached to it.

  • Existing Virtual Network

    • If using an existing Virtual Network for creating a GKE Cluster requires two secondary IPV4 addresses in the subnet. Additionally, if using a shared Virtual Network, following roles and permissions are required for by Default service account of Kubernetes (service-PROJECT_NUMBER@container-engine-robot.iam.gserviceaccount.com) on the project of Shared Virtual Network.

      • Compute Network User

      • kubernetes_role: You must create a custom role. To do this, click Roles > Create new role. Provide a name like kubernetes_role; assign the following permissions, and then attach to the service account:

        • compute.firewalls.create

        • compute.firewalls.delete

        • compute.firewalls.get

        • compute.firewalls.list

        • compute.firewalls.update

        • compute.networks.updatePolicy

        • compute.subnetworks.get

        • container.hostServiceAgent.use

    • The 2181,45460,6903 ports must be allowed in the Firewall inbound rules for all internal communication between the Kubernetes cluster and Kyvos.

  •   Existing (IAM) Service account

    1. Add the following predefined roles to the existing IAM service account:

       1. Service Account Token Creator

       2. Kubernetes Engine Developer

       3. Kubernetes Engine Cluster Admin

    2. Add the following permissions to the kubernetes_role custom role that you created above.

       1. compute.instanceGroupManagers.update

       2. Compute.instanceGroupManagers.get

Using existing GKE Cluster

You must have an existing GKE cluster to complete the following prerequisites.

1. Cloud Storage FUSE CSI driver Add-on must be enabled.

2. VNet peering is necessary if the Kyvos VPC differs from the VPC associated with the existing Kubernetes cluster.

3. Firewall rule on GKE Cluster VPC: An inbound rule allows TCP traffic on the 6903 port that is required with the source IP range set to the Kyvos VPC.

4. Permissions required by GKE Service Account: For the GKE Service Account, the following roles and permissions are required:

   • roles/container.clusterAdmin

   • roles/container.developer

   • compute.instanceGroupManagers.update

   • compute.instanceGroupManagers.get

   • roles/iam.serviceAccountTokenCreator

   • roles/iam.workloadIdentityUser: The above permission [roles/iam.workloadIdentityUser] is associated with the Kubernetes namespace service account used for Kyvos deployment.

   • Command:

     gcloud iam service-accounts add-iam-policy-binding IAM_SA_NAME@IAM_SA_PROJECT_ID.iam.gserviceaccount.com --role roles/iam.workloadIdentityUser --member "serviceAccount:PROJECT_ID.svc.id.goog[KYVOS_NAMESPACE/kyvos-sa]"

5. Namespace for Kyvos Compute Worker.

   • Dedicated Node Pool: Kyvos will create a namespace on its own.

   • Shared Node Pool: Users should create this namespace before proceeding with the Kyvos deployment

6. Node pool for Kyvos Compute Worker: A Node pool should be created before proceeding with the Kyvos deployment with existing GKE.

7. Permissions for Kyvos Service Account:

Case 1: Dedicated Node Pool:

For a Dedicated Node Pool in a Shared GKE Cluster, the Kyvos Service Account needs the following permissions:

Roles:

• roles/container.viewer

• container.configMaps.create

• container.jobs.create

• container.persistentVolumeClaims.create

• container.persistentVolumes.create

• container.serviceAccounts.create

• container.serviceAccounts.update

• container.pods.getLogs

• container.jobs.delete

• container.storageClasses.create

• container.namespaces.list

• container.namespaces.create

Additional Permissions for Node Pool Management:

• container.clusters.update

• compute.instanceGroupManagers.get

• compute.instanceGroupManagers.update

Case 2: Shared Node Pool:

For a Shared Node Pool in a Shared GKE Cluster, the Kyvos Service Account requires the following permissions.

Roles:

• roles/container.viewer

• container.configMaps.create

• container.jobs.create

• container.persistentVolumeClaims.create

• container.persistentVolumes.create

• container.serviceAccounts.create

• container.serviceAccounts.update

• container.pods.getLogs

• container.jobs.delete

• container.storageClasses.create

• container.namespaces.list