Document toolboxDocument toolbox

Creating Kyvos resources for AWS automated deployment with EMR

Applies to: Kyvos Enterprise  Kyvos Cloud (SaaS on AWS) Kyvos AWS Marketplace

Kyvos Azure Marketplace Kyvos GCP Marketplace Kyvos Single Node Installation (Kyvos SNI)


To create Kyvos resources for creating an AWS CloudFormation template for use in automated deployment, perform the following steps.

  1. Log onto your AWS Console, and create an EC2 instance for the Kyvos Manager.

  2. On this EC2 instance, copy or download the Kyvos installation setup shared by Kyvos Support.

  3. Install Kyvos Manager.

  4. Start the Kyvos Manager and access it using your credentials.

  5. Alternatively, access the installation URL, such as:
    http://clusterIP:port/kyvosmanager/kyvos-installer/aws
    If you do not have the URL and/or credentials, contact Kyvos support. Alternatively, use the default credentials as admin|admin. Kyvos Manager forces password change on first-time login.

  6. To create resources for installing Kyvos services like BI Server and Query Engine on the AWS environment, select the I want to create resources option on the Kyvos Installer page.

  7. Click Create Resources. This displays the Create Resources dialog box.

    Here, select the services for which you want to create resources and provide information as:

Resources will be created only for selected services. If you have pre-created resources for any of the services, you can uncheck them and configure them while installing Kyvos.

Parameter/Field

Remarks/Description

Parameter/Field

Remarks/Description

Use Graviton Instances

Select the checkbox to deploy the Kyvos Services (like BI Server, Query Engine, Kyvos Manager, EMR Master, and Core nodes) on Graviton instances.

NOTE: For Kyvos instances, Graviton CPU is supported with RDS repository service only.

EC2 instance(s) for BI Server

Specify the following:

  • Number of Node(s)

  • Instance Type

  • Installation Disk

EC2 instance(s) for Query Engine

Specify the following:

  • Number of Node(s)

  • Instance Type

  • Installation Disk

  • EBS Volume

NOTE: Installation disk size must be 100 GB and the cuboid replication disk size must be in the multiplier of 50 or 100.  

Web Portal High Availability  

Select to enable web portal high availability, and specify the following:

  • Number of Node(s) 

  • Instance Type 

  • Installation Disk 

  • Subnet for Load Balancer (Possible value list should be Public or Private) 

EC2 instance for Kyvos Manager

The checkbox is selected by default for the automated deployment and an EC2 instance is launched for the Kyvos Manager.

  • Provide the Instance Type and Installation Disk details.
    When using a multi-tenant environment, you can use the Deploy as Tenant checkbox to define a tenant.

Select one of the following repository types from the Repository list.

  • Bundled Postgres: To select bundled Postgres as a repository for Kyvos Manager. 

  • Create New RDS: To create a new external repository for Kyvos Manager.

  • Existing RDS: To use the existing RDS of Kyvos Manager.

  • Select the size of the instance type from the Repository Instance Type. 

External Repository for Kyvos 

  • Select this option to create a PostgreSQL-based repository using the Amazon RDS. Otherwise, the pre-bundled PostgreSQL-based Kyvos repository will be used.

  • Select the Repository Instance Type

Output S3 Storage 

Select this option to create an S3 storage bucket for Kyvos output.

Bucket Name Prefix

Compute Cluster Type

Select the EMR option.

EMR Cluster

Select this option to define configurations for the EMR cluster.

  • Master Node Instance Type

  • Core Node Instance Type

  • Enable Glue

NOTE: In case you do not configure the EMR here, you can use a Shared EMR. In this case, you need to add a common Security Group for master and core in the inbound rule of the Security Group of the BI Server.

Use Graviton Instances for EMR cluster 

The checkbox is selected by default, indicating that the EMR cluster is deployed with Graviton instances.  To deploy the EMR cluster with non-Graviton instances, you must clear the checkbox. 

After selecting this checkbox, the Master Node Instance Type and the Core Node Instance Type are selected by default.  

Use Custom Image

(please see the note below)

 

 

 

Select this option to use a custom image of your environment for deploying the Kyvos cluster. Custom images can be used to bootstrap deployment tasks like preloading applications, application configurations, and other configurations.   

You will have to provide the following information when deploying the cluster through CloudFormation.

  • AmiDefaultUserName: Enter the AMI default logged-in Linux user

  • AmiId: Enter the AMI ID to be used for launching Kyvos Instances .  

NOTE: To use your custom image, templates named oe _custom.json, qe_custom.json,   km_custom.json, and base_custom.json must be available at the $KYVOS_MANAGER_HOME/cloud/ aws /templates/ location.

Deploy with enhanced security

Enhanced security ensures passwords are stored in the AWS Secrets Manager and not in the Kyvos configuration files.

Credit Based Deployment 

Select this checkbox to create or support the credit-based license in Kyvos. 

In the Credit Management Resouces list, select one of the following: 

  • Create New: Select this option to create a shared Credits Database for the primary cluster deployment.

  • Use Existing: Select this option to use the existing shared Credits Database for the secondary or tertiary cluster deployment.  

EMR and RDS Versions  

Select Kyvos Certified Versions Only to list EMR and RDS versions supported by Kyvos .

Else, select the All versions option to view all the versions.

These versions are displayed on the AWS CFT Console at the time of cluster deployment. 

Note

For using Custom Image, Kyvos supports using only customer-provided AMIs for CentOS 7, RHEL 7 and 8, and Amazon Linux 2 with x86 or ARM-based Graviton architecture type.

  • Python 2 is required as a prerequisite for CentOS 7 and RedHat 7 AMIs  

  • Python 3 is required as a prerequisite for RedHat 8 AMIs and CentOS 8 AMIs

  • All operating system commands must be present in the Custom Image machine. See the OS commands section for more details.

  1. Click Next. The Review and Download Template dialog box is displayed with details of all the settings that you selected.

  2. Review all the settings, and click the Back button if you want to make any changes.

  3. To continue, click the Download Template button to download the CloudFormation template and upload this template to your AWS environment to create resources.

Now deploy the Kyvos cluster to create the resources. Once the resources are ready, go to the Installer page to continue installation using the I have all the resources option.

Copyright Kyvos, Inc. All rights reserved.