Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Applies to: (tick) Kyvos Enterprise  (error) Kyvos Cloud (SaaS on AWS) (error) Kyvos AWS Marketplace

...

From Kyvos 2024.3 onwards, you can select the compute cluster on Azure to process semantic model through Databricks or Kyvos Native (Kubernetes or Shared Query engines). For further details, see the Deploying Kyvos through Azure Template Specs with Kyvos Native section.

To deploy Kyvos through Azure Template specs, perform the following steps.

  1. Log in to your Azure Portal, with the user having sufficient permissions and information as mentioned in the prerequisites.

  2. On the Home page, click Cloud Shell.  

    Pic-1.png
  3. Upload the formUiDefinition and mainTemplate templates by clicking the Upload button.  

    Pic-2.png
  4. After uploading the template, in the Cloud Shell Area, execute the following command to create Template Specs.
    az ts create --name <TemplateSpecName> --version <“VersionNumber”> --resource-group <Resource Group name> --location <“Location”> --template-file "MainTemplatePath" --ui-form-definition "FormFilePath"

    For example,

    az ts create --name kyvosspec --version "v1.0.33" --resource-group Kyvos_RG2 --location "Central US" --template-file "./mainTemplate.json" --ui-form-definition "./formUiDefinition.json"  

  5. Close the Cloud Shell and in the Search box, click Template Spec. The Template Specs page is displayed.

  6. On the Template Specs page, search the Template Spec that you have created in Step 4, as mentioned above.

    image-20240507-104935.png
  7. Browse the selected Template Spec and then click the Deploy option to open the Kyvos deployment form.

    Pic-4.png

    The Kyvos Deployment form is displayed with the parameters required for deployment.

    Pic-5.png
  8. Here, enter the details as:
    Fields marked (*) are mandatory.

Aura tab collection
paramsJTdCJTIyZ2VuZXJhbCUyMiUzQSU3QiUyMnRhYlNwYWNpbmclMjIlM0EwJTJDJTIydGFiV2lkdGglMjIlM0ExMDAlMkMlMjJ0YWJIZWlnaHQlMjIlM0E1MCUyQyUyMmRpcmVjdGlvbiUyMiUzQSUyMmhvcml6b250YWwlMjIlN0QlMkMlMjJjb250ZW50JTIyJTNBJTdCJTIyYmFja2dyb3VuZENvbG9yJTIyJTNBJTdCJTIyY29sb3IlMjIlM0ElMjIlMjNmZmYlMjIlN0QlMkMlMjJib3hTaGFkb3clMjIlM0ElN0IlMjJzaGFkb3dzJTIyJTNBJTVCJTdCJTIyY29sb3IlMjIlM0ElMjJyZ2JhKDAlMkMlMjAwJTJDJTIwMCUyQyUyMDAuMDgpJTIyJTJDJTIyaXMlMjIlM0ElMjJkcm9wLXNoYWRvdyUyMiUyQyUyMnglMjIlM0EwJTJDJTIyeSUyMiUzQTAlMkMlMjJibHVyJTIyJTNBMSUyQyUyMnNwcmVhZCUyMiUzQTAlN0QlMkMlN0IlMjJjb2xvciUyMiUzQSUyMnJnYmEoMCUyQyUyMDAlMkMlMjAwJTJDJTIwMC4xNiklMjIlMkMlMjJ4JTIyJTNBMCUyQyUyMnklMjIlM0EzJTJDJTIyYmx1ciUyMiUzQTIlMkMlMjJzcHJlYWQlMjIlM0EtMiU3RCU1RCU3RCUyQyUyMmJvcmRlciUyMiUzQSU3QiUyMnN0eWxlJTIyJTNBJTIyc29saWQlMjIlMkMlMjJ3aWR0aCUyMiUzQTElMkMlMjJ0b3AlMjIlM0F0cnVlJTJDJTIyYm90dG9tJTIyJTNBdHJ1ZSUyQyUyMmxlZnQlMjIlM0F0cnVlJTJDJTIycmlnaHQlMjIlM0F0cnVlJTJDJTIyY29sb3IlMjIlM0ElMjIlMjNmNGY1ZjclMjIlN0QlMkMlMjJwYWRkaW5nJTIyJTNBJTdCJTIydG9wJTIyJTNBMTAlMkMlMjJyaWdodCUyMiUzQTEwJTJDJTIyYm90dG9tJTIyJTNBMTAlMkMlMjJsZWZ0JTIyJTNBMTAlN0QlMkMlMjJzaXplJTIyJTNBJTdCJTIyaGVpZ2h0JTIyJTNBMTUwJTdEJTdEJTJDJTIyYWN0aXZlJTIyJTNBJTdCJTIyYmFja2dyb3VuZENvbG9yJTIyJTNBJTdCJTIyY29sb3IlMjIlM0ElN0IlMjJsaWdodCUyMiUzQSUyMiUyM2Y1ODIyNyUyMiU3RCU3RCUyQyUyMnRleHQlMjIlM0ElN0IlMjJmb250U2l6ZSUyMiUzQTE0JTJDJTIyY29sb3IlMjIlM0ElN0IlMjJsaWdodCUyMiUzQSUyMiUyMzAwMDAwMCUyMiU3RCUyQyUyMnRleHRBbGlnbiUyMiUzQSUyMmxlZnQlMjIlMkMlMjJmb250V2VpZ2h0JTIyJTNBJTIyYm9sZCUyMiU3RCUyQyUyMmJvcmRlciUyMiUzQSU3QiUyMnRvcCUyMiUzQXRydWUlMkMlMjJsZWZ0JTIyJTNBdHJ1ZSUyQyUyMnJpZ2h0JTIyJTNBdHJ1ZSUyQyUyMmJvdHRvbSUyMiUzQXRydWUlMkMlMjJ3aWR0aCUyMiUzQTElMkMlMjJzdHlsZSUyMiUzQSUyMnNvbGlkJTIyJTJDJTIyY29sb3IlMjIlM0ElN0IlMjJsaWdodCUyMiUzQSUyMiUyM2NjY2VjZiUyMiU3RCU3RCU3RCUyQyUyMmhvdmVyJTIyJTNBJTdCJTIyYmFja2dyb3VuZENvbG9yJTIyJTNBJTdCJTIyY29sb3IlMjIlM0ElMjIlMjNkZmUxZTYlMjIlN0QlMkMlMjJ0ZXh0JTIyJTNBJTdCJTIyZm9udFNpemUlMjIlM0ExNCUyQyUyMmNvbG9yJTIyJTNBJTdCJTIybGlnaHQlMjIlM0ElMjIlMjMwMDAwMDAlMjIlN0QlMkMlMjJ0ZXh0QWxpZ24lMjIlM0ElMjJsZWZ0JTIyJTJDJTIyZm9udFdlaWdodCUyMiUzQSUyMmxpZ2h0ZXIlMjIlN0QlMkMlMjJib3JkZXIlMjIlM0ElN0IlMjJ0b3AlMjIlM0F0cnVlJTJDJTIybGVmdCUyMiUzQXRydWUlMkMlMjJyaWdodCUyMiUzQXRydWUlMkMlMjJib3R0b20lMjIlM0F0cnVlJTJDJTIyd2lkdGglMjIlM0ExJTJDJTIyc3R5bGUlMjIlM0ElMjJzb2xpZCUyMiUyQyUyMmNvbG9yJTIyJTNBJTdCJTIybGlnaHQlMjIlM0ElMjIlMjNjY2NlY2YlMjIlN0QlN0QlN0QlMkMlMjJpbmFjdGl2ZSUyMiUzQSU3QiUyMmJhY2tncm91bmRDb2xvciUyMiUzQSU3QiUyMmNvbG9yJTIyJTNBJTIyJTIzZjRmNWY3JTIyJTdEJTJDJTIydGV4dCUyMiUzQSU3QiUyMmZvbnRTaXplJTIyJTNBMTQlMkMlMjJjb2xvciUyMiUzQSU3QiUyMmxpZ2h0JTIyJTNBJTIyJTIzMDAwMDAwJTIyJTdEJTJDJTIydGV4dEFsaWduJTIyJTNBJTIybGVmdCUyMiUyQyUyMmZvbnRXZWlnaHQlMjIlM0ElMjJib2xkJTIyJTdEJTJDJTIyYm9yZGVyJTIyJTNBJTdCJTIydG9wJTIyJTNBdHJ1ZSUyQyUyMmxlZnQlMjIlM0F0cnVlJTJDJTIycmlnaHQlMjIlM0F0cnVlJTJDJTIyYm90dG9tJTIyJTNBdHJ1ZSUyQyUyMndpZHRoJTIyJTNBMSUyQyUyMnN0eWxlJTIyJTNBJTIyc29saWQlMjIlMkMlMjJjb2xvciUyMiUzQSU3QiUyMmxpZ2h0JTIyJTNBJTIyJTIzY2NjZWNmJTIyJTdEJTdEJTdEJTdE
Aura tab
summary1. Basics
paramsJTdCJTIydGl0bGUlMjIlM0ElMjIxLiUyMEJhc2ljcyUyMiU3RA==

Enter details as:

Area 

Parameter

Description

Project Details 

Subscription

Your account subscription.
NOTE: All resources in an Azure subscription are billed together.  

Resource Group*

  • Select the name of your resource group. The resource group is a collection of resources that share the same lifecycle, permissions, and policies.

  • You can also create a new resource group using the Create new option.

NOTE: You must add this role Managed Application Contributor Role at the subscription level if you want to add a new Resource Group. 

Instance details 

Region* 

Select the same region as of Resource group. 

Kyvos Cluster Name*

Enter a unique name that you want to provide for the Kyvos application. 

Use Custom Image

To use customized Operating System Image.

If you select this checkbox, the followings fields are displayed.

  • Gallery Subscription ID:Enter the subscription ID in which gallery resides.

  • Gallery Resource Group Name: Enter the Resource Group name in which gallery resides.

  • Gallery Name:Enter the name of the shared image gallery.

  • Gallery Image Definition Name: Enter the name of the image definition.

  • Gallery Image Version Name: Enter the name of the image version.

Authentication Information

SSH Public Key Source

Select the option to use existing key stored key in Azure or public key from the list.

  • Use existing key stored key in Azure: Select the stored keys from the list.

  • Use existing public key: Enter SSH public key.

SSH Private key

Provide an RSA private key in the single-line format.

Secret Key For Kyvos Bundle Download

Enter the Secret key to access Kyvos bundle.

License File Value

Enter a valid Kyvos license key. It should be in 64-bit encoded format.

Enable Managed Identity Creation

  • Select either Yes or No.

  • If you select Yes, a new managed identity created and attached to the Kyvos VMs.

  • If you select No, the Managed Identity Name list is displayed. Select the name of the user Managed Identity to be attached with all Kyvos VMs.

Compute Cluster

Compute Cluster Type

Select Databricks from the list.

Virtual Machine for Kyvos Manager

Select this checkbox to use Virtual Machine for Kyvos Manager. The Kyvos Manager and Web Server Configurations section will be displayed.

NOTE: For Wizard-based deployment (STACK-based), you must clear the Virtual Machine for Kyvos Manager checkbox. The Kyvos Manager and Web Server Configurations section will not be displayed.

Kyvos Manager and Web Server Configurations

Instance Type

Select the size of Kyvos Manager instance.
This is the default instance type
2x Standard D4s v4
4 vcpus, 16 GB memory

NOTE: You can also change the Instance size using the Change Size link.

Disk Type

Select the disk type applicable for Kyvos Manager Instance.

Disk Size

Enter the required disk size for each Kyvos Manager Instance in GB.

Repository for Kyvos Manager

Select the repository for Kyvos Manager from the list.

  • Bundled Postgres : To select bundled Postgres as a repository for Kyvos Manager. 

  • Create New Azure Postgres : To create a new external Azure Postgres repository for Kyvos Manager. 

  • Use Existing Azure Postgres : To use the existing external Azure Postgres repository for Kyvos Manager. 

Kyvos Manager Repository Username

Enter the user name used for Kyvos Manager Repository.

Kyvos Manager Repository Database

Enter the database name used for Kyvos Manager Repository.

Kyvos Manager Azure Postgres Server Name

Enter the name of the existing Kyvos Manager Azure Postgres Server.

Kyvos Manager Repository Password

Enter the password used for Postgres.
NOTE: Password must contain 8 to 128 characters from three of the following categories: English uppercase letters or English lowercase letters, numbers (0 through 9), and non-alphanumeric characters (for example, !, $, #, %).

Kyvos Manager Azure Postgres Server Resource Group Name

Enter the Resource Group in which above Kyvos Manager Azure Postgres Server resides.

NOTE: This option is only displayed when you select the Use Existing Azure Postgres as Kyvos Manager repository.

Kyvos Manager Postgres TLS Type

Select Yes if Kyvos Manager Postgres is TLS enabled.

External Repository for Kyvos

Select this checkbox to create to create a PostgreSQL based repository using Azure. Otherwise, the Pre-bundled PostgreSQL based Kyvos repository will be used.

Kyvos Repository

Select Kyvos repository from the list. You can create a new instance or use the existing repository.

Virtual Machine Type For Kyvos Postgres

Virtual Machine type which will be launched for Kyvos PostgreSQL.

Click the Change Size link to select the required VM size. Select the Virtual Machine size and click Select.

Enable HA For Kyvos Postgres

Select Yes to enable High Availability for Kyvos PostgreSQL pricing tier.

Backup Retention For Kyvos Postgres

Enter number of days the backup should be kept for Kyvos PostgreSQL. The value can be between 7 to 35.

Kyvos Repository Password

Enter the password used for Postgres. Note: Password must contain 8 to 128 characters from three of the following categories: English uppercase letters or English lowercase letters, numbers (0 through 9), and non-alphanumeric characters (for example, !, $, #, %).

Web Portal High Availability

Selecting this checkbox will enable Web Server Configurations.

NOTE: One additional Web Server instance will be created along with Kyvos Manager instance as you selected Web Portal High Availability.

BI Configurations

Instance Type

Select the size of BI instances.
This is the default instance type
2x Standard D8s v4
8 vcpus, 32 GB memory

NOTE: You can also change the Instance size using the Change Size link.

BI Instance High Availability

Select this to enable High Availability for BI Instances.

Disk Type

Select the disk type applicable for BI Instance.

Disk Size

Enter the disk size applicable for each BI Instance in GB.

Query Engines Configurations

Instance Type*  

Select the size of Query Engine instances.

NOTE: By default, the 1x Standard E4ds v4 Query Engine instance size is displayed. Click the Change Size link to select the required VM size. Select the Virtual Machine size and click Select

Instance Count

Enter the number of Query Engine instances that you want to create in the Kyvos Cluster. or You can use the Instance Count slider to increase or decrease the Query Engine instance count. 

NOTE: You can select a minimum of 3 Query Engine instances and a maximum of 30 Query Engine instances.

Disk Count per Query Engine

Select the disk size applicable for each Query Engine Instance.

Disk Type

Select the disk type applicable for QE Instance.

Disk Size

Enter the disk size applicable for each QE Instance in GB.

Create New Key Vault

Select Yes to create a new key Vault.

Select No to use existing key vault and complete the following fields:

  • Vault URL: Provide the Key Vault URL to use existing Key Vault.
    NOTE: If the field is left blank, a new Key Vault will be created.

  • Vault Resource Group: Enter name of the Resource group in which key vault is deployed.

image-20240718-060609.png

After completing the information on the Basics tab, click Next. The Networking tab is displayed. 

Go to the Networking tab. 

Aura tab
summary2. Networking
paramsJTdCJTIydGl0bGUlMjIlM0ElMjIyLiUyME5ldHdvcmtpbmclMjIlN0Q=

Enter details as:

Parameter

Description

Create (VNet)

  • Select Yes to deploy Kyvos using existing virtual network, and complete the following parameters:  

    • Virtual Network: Select a virtual network where you want to deploy Kyvos cluster with a CIDR range between /16 to /24.

    • Private DNS Zone Resource ID: Resource ID of existing Private DNS Zone which will be used with Flexible Server. If you want to create new Private DNS Zone, leave the value blank.
      NOTE: This field is displayed only when you use the Flexible Server for Kyvos or Kyvos Manager repository.

  • Select No to deploy Kyvos using existing virtual Network, and complete the following parameters:  

    • Select Virtual Network: Select the Virtual Network in which VMs will be running.

    • VM Subnet Name: Select the name of the subnet in which VMs will be running. The Subnet should be a part of above Virtual Network.

    • Application Gateway Subnet Name: Select the name of the subnet in which Application Gateway will be created. The Subnet should be a part of above Virtual Network. A separate subnet is required for Application Gateway.

    • Kubernetes Subnet Name: Enter the name of the subnet in which Azure Kubernetes Service (AKS) will be created. The subnet must be part of the specified Virtual Network and be a separate /23 subnet specifically for AKS.
      NOTE: This option is displayed only when if you select the Kubernetes option from the Kyvos Native Type list.

    • Create Security Group:

      • Yes: Enter the range of IP addresses allowed to access Kyvos Instances. Enter0.0.0.0/0 to allow all users access in the Network Security Group Ip WhiteList option.

      • No: Enter the name of the security group that can be used to access the VMs in the Security Group Name option.

Advanced Network Configuration

Select the Kyvos cluster and Databricks cluster will be in same network or in peered network checkbox if your Kyvos cluster nodes and Databricks cluster are either in the same network or shared using VPC peering. If this is not selected, API gateway methods will be created and used for communication between the Databricks cluster and Kyvos BI server.

The Kyvos Cluster deployment with Load Balancer checkbox option is displayed if Web Portal high availability is enabled or a non peered network exists between Kyvos and Databricks, the Kyvos cluster will be deployed with Load Balancer.

Select the Deploy with Enhanced Security checkbox to enhance Security Enables advance security for Storage Account, Key Vault and Virtual Machines.
Complete the following fields:

  • Firewall IP White List: Enter IP rules to be set for the Key Vault and Azure Storage Account. Value should be comma-separated values passed as list. Exa- ["0.0.0.0"]

  • Enable Firewall on Storage Account: Select this checkbox to enable Firewall rule on Storage Account.

  • Select Databricks Virtual Network: Select the name of the Databricks Virtual Network.

  • Select Databricks private subnet: Select the name of the private Subnet in which Databricks VMs will be running. The Subnet should be part of above Virtual Network.

  • Select Databricks public subnet: Select the name of the public subnet in which Databricks VMs will be running. The Subnet should be part of the above Virtual Network.

image-20240417-103057.png

After completing the information on the Networking tab, click Next. The Storage Account tab is displayed. Go to the Storage Account tab. 

Aura tab
summary3. Storage Account Configurations
paramsJTdCJTIydGl0bGUlMjIlM0ElMjIzLiUyMFN0b3JhZ2UlMjBBY2NvdW50JTIwQ29uZmlndXJhdGlvbnMlMjIlN0Q=

Enter details as:

Parameter

Description

Create New Storage Account*

Select Yes to create new storage account and complete the following fields:

  • Enable Storage Account High Availability: Select this checkbox to create Zone-redundant Storage(ZRS).

  • Storage Account Container Name: Enter the name of Container in Storage Account which will be used for Kyvos.

  • Kyvos Work Directory: Enter the path for the Kyvos work directory.

Select No to use existing storage account and complete the following fields:

  • Storage Account Name: Select the name of storage account to be used for Kyvos.

  • Storage Account Container Name: Enter the name of Container in Storage Account which will be used for Kyvos.

  • Kyvos Work Directory: Enter the path for the Kyvos work directory.

image-20240417-103325.png

After completing the information on the Storage Account tab, click Next. The Databricks Configurations tab is displayed. Go to the Databricks Configurations tab. 

Aura tab
summary4. Build Configurations
paramsJTdCJTIydGl0bGUlMjIlM0ElMjI0LiUyMEJ1aWxkJTIwQ29uZmlndXJhdGlvbnMlMjIlN0Q=
Panel
panelIconIdatlassian-note
panelIcon:note:
bgColor#DEEBFF

Note

You will see the build configurations based on the option (Databricks or Kyvos Native) you select from the Compute Cluster Type list in the Basics tab.

  • If you select the Databricks, the Databricks Configurations will be displayed.

  • If you select the Kubernetes option from the Kyvos Native Type list, the Kubernetes configurations will be displayed.

  • If you select the Shared Query Engines option from the Kyvos Native Type list, this eliminates the need for build configuration, as the build will operate directly on the query engines.

Parameter

Description

Databricks

Databricks URL*

Enter the Databricks Workspace URL where the cluster to be used for data processing by Kyvos is present or will be created.

Databricks Cluster ID * 

Enter the cluster ID of the existing Databricks cluster. 

Databricks Cluster Organization ID*

Enter the Databricks Cluster Organization ID that is used to <>.

Databricks Authentication Type

Select the authentication type for Databricks cluster. AAD token using managed identity option is supported only with premium workspace

Databricks Token*

Enter the value of the token used to connect to Databricks Cluster.

Object ID of Service Principal*

Enter the Object ID assigned to the Service principal. This maps to the ID inside the Active Directory.

Maximum Worker Node Count

Enter the Worker node count up to which the worker should scale.

image-20240417-104932.png

After completing the information on the Databricks Configurations tab, click Next. The Advance Configurations  tab is displayed. 

Go to the Advance Configurations tab. 

Aura tab
summary5. Advance Configurations
paramsJTdCJTIydGl0bGUlMjIlM0ElMjI1LiUyMEFkdmFuY2UlMjBDb25maWd1cmF0aW9ucyUyMiU3RA==
Panel
panelIconIdatlassian-note
panelIcon:note:
bgColor#DEEBFF

Note

This is an optional configuration. If you want to add a prefix to your Kyvos resources, you can specify using the parameters displayed in the Advanced Configurations section. 

Enter details as:

Area 

Parameter

Description

Prefix for Azure Resources

Key Vault

Add a prefix to the name of your Kyvos Key Vault, provide the value here.

Virtual Machines

Enter the value to add a prefix to the name of your Kyvos Virtual Machines. 

Virtual Machine Scale Set

Enter the value to add a prefix to the name of your Kyvos Virtual Machines scale set. 

Create New Log Analytics Workspace

Select Yes to create a new workspace.

Select No to use existing workspace and select existing log analytics workspace from the list.

Enter the Kyvos Version

Enter the desired Kyvos Version to be deployed.

Deploy as Tenant

Select this checkbox to deploy the cluster as a tenant. If enabled, you will see an input field for Management Console URL, which manages all the tenants.

Perform Environment validation

Select the option Yes or No to enable or disable environment validation.

Host Name Based Deployment

Select Yes to use hostnames instead of IP Addresses for instances during cluster deployment.

image-20240417-105152.png

After completing the information on the Advance Configurations tab, click Next. The Tags  tab is displayed. 

Go to the Tags tab. 

Aura tab
summary6. Tags
paramsJTdCJTIydGl0bGUlMjIlM0ElMjI2LiUyMFRhZ3MlMjIlN0Q=
Panel
panelIconIdatlassian-note
panelIcon:note:
bgColor#DEEBFF

Note

This is an optional configuration. If you want to add a tag to your Kyvos resources, you can specify using the parameters displayed in the Tags section. 

Enter details as:

Parameter

Description

Name

Enter a tag name. Tags are case-sensitive and are limited to 512 characters. 
Tags are name/value pairs that enable you to categorize resources and view consolidated billing by applying the same tag to multiple resources and resource groups. Refer to Microsoft documentation to learn more about tags.

NOTE: If you create tags and then change resource settings on other tabs, your tags will be automatically updated.

Value

Enter a tag value. Tags values are case-sensitive and are limited to 256 characters.

Resource

Select the resources on which you want to add tags. 

image-20240417-105317.png

...