Creating Kyvos resources for AWS wizard-based deployment with EMR

Use Graviton Instances

Select this option to deploy the Kyvos Services (like BI Server, Query Engines, Kyvos Manager, and EMR Master and Core nodes) on Graviton instances.

NOTE: For Kyvos instances, the Graviton CPU is only supported with the RDS repository service.

EC2 instance(s) for BI Server

Number of Node(s)

Provide the number of EC2 instances that you want to create for BI Server nodes.

Instance Type

Select the EC2 instance type to be used for BI Server nodes.

Installation Disk

Select the type and size of the installation disk to be used for BI Server nodes.

EC2 instance(s) for Query Engine

Number of Node(s)

Provide the number of EC2 instances that you want to create for query engine nodes.

Instance Type

Select the EC2 instance type to be used for query engine nodes.

Installation Disk

Select the type and size of the installation disk to be used for query engine nodes.

EBS Volume

Select the number and size of the disk to be used for query engine nodes.

NOTE: Installation disk size must be 100 GB and the cuboid replication disk size must be in the multiplier of 50 or 100.

Disk Type

Select the disk type to be used as EBS Volume.

Web portal high availability 

Select to enable Kyvos web portal high availability.

If you select this, you need to provide the following:

• Number of nodes

• Instance Type

• Subnet for Load Balancer - select Public or Private.

NOTE: To use the Web portal high availability, you must select the EC2 instance for Kyvos Manager checkbox. 

EC2 instance for Kyvos Manager

Instance Type

Clear the check box for wizard-based installation.

Installation Disk

Select the type and size of the installation disk to be used for query engine nodes.

Deploy as Tenant

When using a multi-tenant environment, you can use the    Deploy as Tenant  checkbox to define a tenant instance for Kyvos Manager. Enable this option to deploy the cluster as a tenant.

Repository 

Select one of the following repository types from the  Repository  list.

• Bundled Postgres: To select Postgres as a repository for Kyvos Manager. 

• Create New RDS: To create a new external repository for Kyvos Manager.

• Existing RDS: To use the existing RDS of Kyvos Manager.

Repository Instance Type

Select the instance type to create a new repository instance for Kyvos. 

External Repository for Kyvos

Select the checkbox to create a PostgreSQL-based repository using Azure. Otherwise, the pre-bundled PostgreSQL-based Kyvos repository will be used.

Repository 

To use your existing repository for Kyvos, select the Use Existing Instance option.

To create a new one, select the Create New instance option.

Instance Type

Select the instance type to create a new repository instance for Kyvos.

NOTE: This option is displayed only when you select the Create New instance option.

Output S3 Storage 

Select the checkbox to create an S3 storage bucket for storing Kyvos output.

Bucket Name Prefix

Select this option and specify the bucket name prefix to create an S3 storage bucket for Kyvos output.

EMR Cluster

Select this option to define configurations for the EMR cluster

• Master Node Instance Type: Choose the instance type for the master node

• Core Node Instance Type: Choose the instance type for the core node

• Enable Glue: Select the checkbox to enable Glue (if needed).

NOTE: In case you do not configure the EMR here, you can use a Shared EMR. In this case, you need to add a common Security Group for master and core in the inbound rule of the Security Group of the BI Server.

Use custom image

Select this option to use a custom image of your environment for deploying the Kyvos cluster. Custom images can be used to bootstrap deployment tasks like preloading applications, application configurations, and other configurations.   

You will have to provide the following information  at the time of deploying the cluster through the CloudFormation.

• AmiDefaultUserName:  Enter the AMI default logged in Linux user.

• AmiId:  Enter the AMI ID to be used for launching Kyvos Instances.

NOTE: To use your custom image, templates named oe _custom.json, qe_custom.json,  km_custom.json, and base_custom.json must be available a the $KYVOS_MANAGER_HOME/cloud/aws/templates/ location. 

Create resources for managing credits and usage information

Select this option if you want to use credit-based licensing and your environment does not have pre-created credit information-related resources.

EMR and RDS Versions  

Select  Kyvos  Certified Versions Only to list EMR  and RDS  versions supported by  Kyvos. Else, select All versions option to list all versions of EMR and RDS. 

Deploy with enhanced security

Enhanced security ensures passwords are stored in the AWS Secrets Manager and not in the Kyvos configuration files.

Security Configurations

Select the name of the Key Pair to be used with EC2 instances

A key pair consisting of a public key (stored by AWS), and a private key file (stored by the user). Together, they allow you to connect to your instance securely.

Enter the SSH Private Key text

Provide the text of your SSH key. To get this, open your .pem file in edit mode, and copy the complete text.

Enable SSH for EMR/Databricks cluster 

Set the value as true to enable SSH for the EMR cluster.

NOTE: Currently, the only value for Databricks is false, as SSH is not supported on it. 

S3 Configurations

Enter name of the S3 bucket for storing Kyvos semantic model

Enter the name of your existing bucket, if you selected False above. If you select True, Kyvos will create a bucket with this name. Ensure that the bucket is not already in use.

NOTE: See the AWS rules for naming buckets. Additionally, Kyvos does not allow . (dot) to be used for Bucket Name. The cluster may fail to deploy if you do not meet these requirements.

Network Configurations

VPC

Select the VPC in which EC2 instances will be launched.

NOTE: If you have created your VPC using the NAT Gateway template or the Internet Gateway template, select that VPC here.

Subnet

Select the Subnet to be attached to EC2 instances.

NOTE: In the case of Kyvos Web Portal HA (High Availability) or an RDS repository, you must select at least two subnets from different Availability Zones. Otherwise, you have to select only one subnet.

Availability Zone

Select the Availability Zone for the selected Subnet.

See the list of Supported Availability Zones.

IAM Roles Configurations

Enter the name of the IAM Role that will be attached to all the Kyvos EC2 instances  

If blank, the role will be created automatically. 

Enter the name of the IAM Role that will be attached to all the Lambda functions  

If blank, the role will be created automatically.  

RDS Configurations 

Create a new subnet group for RDS? 

Select True to create a new subnet group for RDS.

Enter the name of existing Subnet group 

If creating a new subnet group for RDS is false, enter the name of the existing subnet group.

Specify whether the database instance is a multiple Availability Zone deployment

Select True to specify the database instance is a multiple availability zone deployment.  

Choose the version of RDS which needs to be launched

Enter the RDS version to be launched. 

EMR Configurations

Enter the number of Core EC2 Instances to be launched with EMR

The number of Core EC2 Instances to be launched with EMR.

Recommended value: 2

Enter the minimum number of Core EC2 instances that should be kept running

The minimum number of core nodes that will keep running during scale-in operation.

Recommended value: 2

Enter the maximum number of Core EC2 instances that should be kept running

The maximum number of core nodes that can be used by the EMR during scale-out operation.

Recommended value: 10

Enter the version of EMR which needs to be launched

Select the EMR version to be used for the Kyvos cluster. 

Allowed values are EMR-5.31.0, EMR-5.35.0, EMR-6.2.0, EMR-6.5.0, EMR-6.6.0, and EMR-6.7.0

NOTE: EMR-6.5.0, EMR-6.6.0, and EMR-6.7.0 are supported only if you enable Livy on the cluster.

You will see the EMR version followed by CERTIFIED indicating if the EMR version is supported by Kyvos.

Enable In-transit encryption for EMR cluster

The value of this parameter will be true if in-transit encryption with TLS is enabled for the EMR cluster.

Enter the S3Object ARN where PEM based certificate is located

Enter the S3Object Amazon Resource Name (ARN) of the KMS certificate.

Kyvos Configurations

Enter the Access key

Access key to access the Kyvos bundle. Contact us if you haven't received it yet.

Enter the Secret key

The secret key to access the Kyvos bundle. Contact us if you haven't received it yet.

Kyvos Work Directory

Enter the work directory path to be used by Kyvos.

Enable Public IP for Kyvos Web Portal

Select true to enable Public IP for the Kyvos Web portal, else select the  false option.

NOTE: If you created your VPC using the Internet Gateway template, select true here. This will ensure that the Kyvos Web Server machine is accessible from your computer network. In this case, you will not need to configure Bastion host or load balancer for the Kyvos Web portal.

Kyvos License Information

Enter the temporary Kyvos license key provided in your onboarding email. Copy the content of the license file here.

Enable Environment validation

Select true to validate the environment information that you provided for where the Kyvos cluster needs to be deployed. 

If you select false, the deployment will continue without validation.

Show hostname for cluster deployment 

Select true to use the hostname for cluster deployment.

PostgresPassword

Provide the password used for Postgres.

Kyvos Credit Load Information

Enter a valid Kyvos credit load key.

Kyvos Shared Credit Information Existing Repository

Enter the endpoint and port of the Shared Credit Information Repository.  

NOTE: You MUST add a semicolon after providing the endpoint and specify the port. For example, endpoint: port

Kyvos Credit Information Existing Secret 

Enter the secret name used for Shared Credit Information Repository. 

SecretName 

Enter the name of your existing Secret Manager. If blank, a new Secret Manager will be created automatically.

Allowed IP Range  

Provide the range of IP addresses allowed to access Kyvos Instances. Use 0.0.0.0/0 to allow all users access.  

Enable Livy Server for EMR cluster

If you have selected the EMR version as 6.5.0 , 6.6.0, or 6.7.0 - you MUST enable Livy .  

PublicSubnets

For deployments with Web portal high availability, if you selected a public subnet while creating the template, please select two public subnets.

AMI Configurations

(Displayed only if you selected the Custom Image option at the time of creating the template)

Enter the AMI default logged in Linux user 

Each Linux instance launches with a default Linux system user account.

For more details, refer to AWS documentation.

Enter the AMI ID to be used for launching Kyvos Instances 

 Provide the AMI ID.