Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 9 Next »

Kyvos now supports automated resource creation for GCP using Terraform.

To create Kyvos resources, read the following:

Prerequisites to run Terraform from GCP cloud shell

  • You need a valid Google Cloud Platform account. This account will be used to authenticate Terraform to interact with GCP resources.

  • The following permissions must be given to the logged-in user account:

    • Editor

    • Secret Manager Admin

    • Storage Object Admin

    • Cloud Functions Admin

    • Service Account Token Creator

    • Kubernetes Engine Developer

    • Kubernetes Engine Cluster Admin

    • Workload Identity User

    • Create a custom role and assign the below permission to the role.

      • storage.buckets.get

      • storage.buckets.update

      • storage.objects.update

      • compute.instanceGroupManagers.update

      • compute.instanceGroupManagers.get

  • For additional permissions, refer to the Prerequisites for deploying Kyvos in a GCP environment using Deployment Manager section from Step 2 to Step 27.

  • When using an existing VPC, the subnet must have a minimum mask range of /22

  • Subnets in which Kubernetes cluster is launched should have connectivity to the subnets in which Kyvos instances are launched.

  • When using an existing VPC, ensure that the subnet has two secondary IP ranges with valid mask ranges, as these will be used by the Kubernetes cluster.

Prerequisites to run Terraform form local machine

  • Download and install Terraform on your local machine.

  • To install Terraform, refer to the Terraform documentation.

  • Execute Terraform init command to verify successful installation of Terraform.

  • Jq should be installed on your local machine.

  • You need a GCP account to create and manage resources. Ensure that you have the necessary permissions.

  • Configure GCP on your local machine.

  • For gcloud initialization, refer to the Google documentation.

Encryption Key (CMK) support in GCP Terraform

  • To use an existing service account for deployments, the following predefined roles are needed on Kyvos Service Account:

    • Cloud KMS CryptoKey Decrypter

    • Cloud KMS CryptoKey Encrypter

    • Cloud KMS CryptoKey Encrypter/Decrypter

Note

  • Encryption will be enabled for the following components:

    • Disk

    • Cloud storage

    • Secret manager

  • The service agent must be present in the project where the user is going to deploy for Google Cloud Storage and Secret Manager. For more details, refer to Google documentation.

  • Cloud Key Management Service (KMS) API must be enabled in the project before deployment.

  • The existing cmk must be in the same region as deployment.

  • The existing cmk location must be regional; global keys are not supported by GCS buckets. For more details, refer to Google documentation.

  • To use the BYOK (Bring Your Own Key) feature: The service agent must be present in the project where the user is going to deploy for Google Cloud Storage and Secret Manager. For more details, refer to Google documentation.

  • Click Roles > Create new role. Provide a name like Kyvos- role for storage service, and assign the following permissions.

  • deploymentmanager.deployments.list

  • deploymentmanager.resources.list

  • deploymentmanager.manifests.list

  • cloudfunctions.functions.get

  • dataproc.clusters.list

  • dataproc.clusters.get

  • compute.disks.setLabels

  • compute.instances.start

  • compute.instances.stop

  • compute.instances.list

  • compute.instances.setLabels

  • storage.buckets.get

  • storage.buckets.list

  • storage.objects.create

  • storage.objects.delete

  • storage.buckets.update

  • compute.disks.get

  • compute.instances.get

  • dataproc.clusters.update

  • storage.objects.get

  • storage.objects.list

  • storage.objects.update

  • cloudfunctions.functions.update

  • compute.subnetworks.get

  • resourcemanager.projects.getIamPolicy

  • compute.firewalls.list

  • iam.roles.get  

  • compute.machineTypes.get  

  • compute.machineTypes.list  

  • compute.instances.setMachineType

  • compute.instances.setMetadata

  1. Click Edit to add roles in the service account and add the following roles.

    • Kyvos-role (created in step 1)

    • BigQuery data viewer

    • BigQuery user

    • Dataproc Worker

    • Cloud Functions Invoker

    • Cloud Scheduler Admin

    • Cloud Scheduler Service Agent

    • Service Account User

    • Logs Writer

  2. Permissions for Cross-Project Datasets Access with BigQuery:

    1. Use the same service account that is being used by Kyvos VMs.

    2. Give the following roles to the above-created service account on the BigQuery Project.

      • BigQuery Data Viewer

      • BigQuery User

  3. Prerequisites for Cross-Project BigQuery setup and Kyvos VMs.

    1. Use the same service account that is being used by Kyvos VMs.

    2. To the service account used by Kyvos VMs, give the following roles on the BigQuery Project:

      • BigQuery Data Viewer

      • BigQuery User

  4. For accessing BigQuery Views, add the following permissions to the Kyvos custom role (created above).

    • bigquery.tables.create

    • bigquery.tables.delete

    • bigquery.tables.update

    • bigquery.tables.updateData

  5. Permissions to generate Temporary Views in Separate Dataset when performing the validation/preview operation from Kyvos on Google BigQuery.

    • bigquery.tables.create = permissions to create a new table  

    • bigquery.tables.updateData = to write data to a new table, overwrite a table, or append data to a table  

Additional permission required to run Auto scaling for GCP Enterprise

Apart from existing permissions mentioned in the Creating a service account from Google Cloud Console section, you must need the following permissions for GCP Enterprise:

Permissions required in GCP

  • compute.instanceGroups.get

  • compute.instances.create

  • compute.disks.create

  • compute.disks.use

  • compute.subnetworks.use

  • compute.instances.setServiceAccount

  • compute.instances.delete

  • compute.instanceGroups.update

  • compute.instances.use

  • compute.instances.detachDisk

  • compute.disks.delete

Conditional permission if using network in the project other than Kyvos resources

  • compute.subnetworks.use (on the Kyvos service account in the project where your network resides)

Prerequisites to deploy Kyvos using Kubernetes

Prerequisites to deploy Kyvos using Dataproc section for the complete set of permissions required for deploying Kyvos.

Additionally, for creating a GKE cluster, you must complete the following prerequisites.

Create a GKE cluster

  • Ensure that the GKE service agent’s default service account (service-PROJECT_NUMBER@container-engine-robot.iam.gserviceaccount.com) has the Kubernetes Engine Service Agent role attached to it.

  • Existing Virtual Network

    • If using an existing Virtual Network for creating a GKE Cluster requires two secondary IPV4 addresses in the subnet. Additionally, if using a shared Virtual Network, following roles and permissions are required for by Default service account of Kubernetes (service-PROJECT_NUMBER@container-engine-robot.iam.gserviceaccount.com) on the project of Shared Virtual Network.

      • Compute Network User

      • kubernetes_role: You must create a custom role. To do this, click Roles > Create new role. Provide a name like kubernetes_role; assign the following permissions, and then attach to the service account:

        Google documentation.

      • The 2181,45460,6903 ports must be allowed in the Firewall inbound rules for all internal communication between the Kubernetes cluster and Kyvos.

    •   Existing (IAM) Service account

      1. Add the following predefined roles to the existing IAM service account:

        1. Service Account Token Creator

        2. Kubernetes Engine Developer

        3. Kubernetes Engine Cluster Admin

      2. Add the following permissions to the kubernetes_role custom role that you created above.

        1. compute.instanceGroupManagers.update

        2. Compute.instanceGroupManagers.get

  • No labels

Copyright Kyvos, Inc. All rights reserved.