Deploying Kyvos GCP cluster with Spark through Kyvos Manager

Cluster Name

Provide a name for your cluster.

License

Click the Choose File button to select the license file and then click Upload.

Installation Path

Provide the installation path to deploy Kyvos.

User Name

Username with sufficient rights to deploy cluster.

Authentication Type

• Private Key: Choose and Upload the Private Key file.

• Password: Provide your credentials.

  NOTE: To use password authentication for instances while deployment, ensure that the instances have been configured for using Password-based authentication.

Select Instance by

• IP Address: Select this option for IP Address-based deployment.

• Hostname:  Select this option for hostname-based deployment. In this case, you can deploy the cluster using custom hostnames for nodes to be used for Kyvos services. 

  • Click the Use custom hostname for selected nodes link to specify or map custom hostname for selected BI Server, Query Engine, and Kyvos Manager/repository node.
    NOTE: Ensure that the instances have been configured for using Custom hostnames.

Project

Select your GCP project where your resources/deployment is present.

Deployment

Select the None option (node mapping manually).

Click the Fetch Resources link to get a list of resources available for your user credentials.

Deploy BI Server, Query Engine, and Web Portal Kyvos Services on <IP address> instances

This checkbox is displayed only when you select the  None (node mapping Manually)  option. 

NOTE: By default, the option to deploy instances for BI Server, instances for Query Engine, and instances for Web Portal will be disabled. 

Instance(s) for BI Server

Select the instances that you want to use for the BI Server. Optionally, you can also search the instance with its name.

Instance(s) for Query Engine

Select the instances that you want to use for the Query Engines. Optionally, you can also search the instance with its name.

Instance(s) for Web Portal

Select the instances that you want to use for the Web Portal. Optionally, you can also search the instance with its name.

Repository

Select the instances that you want to use for the Repository. Optionally, you can also search the instance with its name.

Repository Password

Provide the password for repository authentication.

Secrets Manager

• Project: Enter your project name where your secrets are created.

• Secrets: enter or select your secrets name in which you want to store passwords.

NOTE: This option is available only for account-based resources deployments, which means if you selected the None option in the Deployment list and click the Fetch resources link then secrets list will be based on the currently selected project. To use different project, you can change the project for secrets.

Output Storage Bucket Name

Provide the name of the bucket where Kyvos semantic models will be stored.

SSH Preference

Select the checkbox to use SSH for connecting with Dataproc Cluster Master Node.
NOTE: If you do not select this option, you will need to perform manual steps later, in case you need to perform any sync operations from the Kyvos Manager.

User Name

User having requisite permissions for cluster deployment.
NOTE: This option is displayed only if you have selected the SSH Preference option.

Use different private key for Dataproc nodes authentication

Select this option to use a different private key for nodes' authentication.
NOTE: This option is displayed only if you have selected the SSH Preference option.

Private Key

Upload the private key for authentication.
NOTE: This option is displayed only if you have selected the SSH Preference option.

Compute Cluster Type

Select the Dataproc option.

Dataproc Cluster Master Node

Provide the master node for the Dataproc cluster.

If the master node is not fetched from the stack or account resource, then the Fetch Dataproc Details link will be visible below the master node to fetch the cluster master node.

Clicking the link displays the dialog box to provide the following inputs for fetching the master node:

• Project Name:  Enter your project name

• Region: Enter your region

• Cluster Name: Enter your cluster name

External Metastore

Select this option to enable external metastore in the cluster.

NOTE: This option is available and selected only if an external meta store is configured in the selected cluster master node.
If an external metastore is not configured in the selected cluster master node, then the option is disabled and cannot be selected.

Automatically Start or Stop Dataproc Cluster

Select the checkbox to allow Kyvos to start or stop Dataproc cluster as per the cluster schedule defined by the administrator.

Livy

• Enable Livy: Select the checkbox to use the Livy server

• Livy URL: Provide the URL of the Livy server to be used. 

Deployment Function Name

Provide the name of the deployment where your Google Deployment Functions are deployed.

Common

Kyvos Setup Package

Select the Kyvos bundle to be installed. Kyvos Manager contains compatible Kyvos bundles. To use any other Kyvos version, click the Upload button

Semantic Model Local Path

Enter the semantic model local path where cuboids will be copied. For multiple paths, enter a semicolon-separated list.
NOTE: The deployment user must have full permission on the parent of the local semantic model path.

Automatically restart Kyvos services when down

Select this option to enable high availability mode for the Kyvos service. This will ensure Kyvos services are always up and running. The Kyvos Manager will automatically restart the BI server, query engine, repository, and web portal services, in case any of these services go down

BI Server

Listener Port

Enter the listener port for BI Server.

JMX Port

Enter the JMX port for BI Server health monitoring.

Work Directory

Enter the directory location on HDFS, where BI Server has access permissions.

Maximum Heap Memory

Enter the maximum memory available for BI Server.

Additional Java Options

Use this to define custom Java properties for your BI Server.

Query Engine

Thrift Listener Port

Enter the thrift listener port for Query Engine.

JMX Port

Enter the port for Query Engine health monitoring.

Maximum Heap Memory

Enter the maximum memory for Query Engine.

Additional Java Options

Use this to define custom Java properties for your Query Engine.

Web Portal

UI Port

Enter the port on which Kyvos Web client will run.

JMX Port

Enter the port for Web portal health monitoring.

Shut-down Port

Enter the port to be used for shutting the Tomcat server.

AJP Port

Enter the port on which the AJP connector creates a socket and awaits an incoming connection.

Maximum Heap Memory

Enter the maximum memory for the Web portal.

Additional Java Options

Use this to define custom Java properties for the Web portal.

Network Protocol Configuration

Enable TLS

Select the checkbox to enable TLS for the cluster.

HTTP Protocol Version

Select the HTTP Protocol Version for use with TLS.

Kyvos Web Portal on HTTPS

Select the checkbox to use HTTPS for connecting to Kyvos Web Portal.

Kyvos Web Portal Configuration

NOTE: This is displayed only if Kyvos Web Portal on HTTPS checkbox is selected

• Certification Mode: Select the Use Path or Upload File option.

• Keystore: Provide the path or upload the Keystore file for the certificate.

• Keystore Private Key: Provide the Keystore Private Key for the certificate.

• Custom Attributes for Connectors: If needed, provide attributes to be used for the HTTPS connector.

  To update any fields, enter information as: