Page Comparison

Applies to: Kyvos Enterprise Kyvos Cloud (SaaS on AWS) Kyvos AWS Marketplace

Kyvos Azure Marketplace Kyvos GCP Marketplace Kyvos Single Node Installation (Kyvos SNI)

...

When the cluster usage pattern of a cluster is not fixed, you must use load-based scaling is recommended. This type of scaling allows you to automatically scale up or scale down the automatically adjusts the number of query engines based on the resource utilization of the Query Engine instances. By configuring cluster scaling in this way, you can enhance optimize the utilization of your cloud cluster and save on reduce compute costs.

From Kyvos 2024.2 onwards, scaling is implemented based on the CPU and Memory usage of the Query Engine instances. System resources will be monitored for all BI Servers and Query Engines every 30 seconds, and Query Engines’ scaling will be performed based on this data.

Cluster The cluster will be scaled up step-by-step, for . For instance, the cluster it will be scale from Low to Moderate and then from Moderate to High. Similarly, for when scaling down the cluster will step-by step, as , it will scale from High to moderate Moderate and then from Moderate to Low.

By default, loadLoad-based cluster scaling is enabled and if you want to by default. You can further configure it , you can do this on the Cluster Scaling page by using the Load option.

To set a load-based scaling, from the Toolbox, click Setup > Cluster Scaling. The Cluster Scaling page is displayed.

Anchor

	mode
	mode

...

Managed: In Managed scaling, Kyvos intelligently manages the cluster capacity to scale up and scale down Query Engine instances.
Custom: In Custom scaling , allows you can to set up rules based on your cluster usage patterns. Currently, the custom scaling mode supports This feature presently supports scaling based on CPU and Memory utilization-based scaling. When the CPU or Memory load condition meets the configured parameters, the Query Engine cluster instances can be scaled up or down accordingly.

Panel

panelIconId	atlassian-note
panelIcon	:note:
bgColor	#DEEBFF

Note

The capacity of the BI Servers cannot be changed.
All BI Servers can be shut down except the Coordination Master. If there is only one BI Server, this BI Server is treated as the Coordination Master.
The Settings option and Add Schedule option are disabled on the Load screen.
You can view on-screen notifications that provide you with timely information about the state of the cluster.
When you scale down the Query Engines, you reduce the capacity of the node, including the number of cores and memory. Conversely, when you scale up the Query Engines, you increase the capacity of the node by adding more cores and memory.

Points to know

...

If the cluster is down and a query is executed, the first query triggers the cluster startup process, and all queries fail until the cluster is up and running. In this case, the following messages are displayed:

Message 1: "Could not serve the query as Query Engine Cluster is not available. Query Engine is launched. Please try after some time."
Message 2: "Could not serve the query as Query Engine Cluster is starting. Please try after some time."

...

By default, the queries fail until Query Engines are started. If you want to hold the queries when query engines are down, you can specify the time (in minutes) for holding queries by using the QE_STARTUP_QUERY_HOLD_TIME property.
Ensure that if the query engines become active before the configured time, the query will be served; otherwise, it will fail.

...

The Query Engines do not start for any ROLAP queries.

...

To set the scaling modes for Load-Based scaling, perform the following steps.

On the Cluster Scaling page, the Load option is selected by default.
Image Added
To set the scaling mode, select one of the following:
1. Managed: Select the required capacity from the list to start Query Engine when any query is fired.
2. Custom: Select this option to set the rules for resource utilization.
  Image Added
  To set scale up rules,
  - Select the required capacity from the list to start Query Engine when any query is fired.
  - Enter a percentage to scale up the cluster if CPU and Memory utilization threshold goes above the specified percentage. Also, specify the number of data points and the total number of data points to set.
  - To set scale down rules,
    1. Specify the BI Server and/or Query Engine from the list to shut down when no queries are fired for the specified period of time.
    2. Enter a percentage to scale down the cluster if CPU and memory utilization threshold remains below the specified percentage. Also, specify the number of data points and the total number of data points to set.
Click Save. The load-based scaling mode is set.

Panel

panelIconId	atlassian-note
panelIcon	:note:
bgColor	#DEEBFF

NOTE

A data point is information on resource utilization captured every 30 seconds.

Transitioning period

The following tables specify the approximate time required to complete the process during the transitioning period.

Aura tab collection

params

JTdCJTIyZ2VuZXJhbCUyMiUzQSU3QiUyMnRhYlNwYWNpbmclMjIlM0EwJTJDJTIydGFiV2lkdGglMjIlM0ExMDAlMkMlMjJ0YWJIZWlnaHQlMjIlM0E1MCUyQyUyMmRpcmVjdGlvbiUyMiUzQSUyMmhvcml6b250YWwlMjIlN0QlMkMlMjJjb250ZW50JTIyJTNBJTdCJTIyYmFja2dyb3VuZENvbG9yJTIyJTNBJTdCJTIyY29sb3IlMjIlM0ElMjIlMjNmZmYlMjIlN0QlMkMlMjJib3JkZXIlMjIlM0ElN0IlMjJzdHlsZSUyMiUzQSUyMnNvbGlkJTIyJTJDJTIyd2lkdGglMjIlM0ExJTJDJTIydG9wJTIyJTNBdHJ1ZSUyQyUyMmJvdHRvbSUyMiUzQXRydWUlMkMlMjJsZWZ0JTIyJTNBdHJ1ZSUyQyUyMnJpZ2h0JTIyJTNBdHJ1ZSUyQyUyMmNvbG9yJTIyJTNBJTdCJTIybGlnaHQlMjIlM0ElMjIlMjNjY2NlY2YlMjIlN0QlN0QlMkMlMjJwYWRkaW5nJTIyJTNBJTdCJTIydG9wJTIyJTNBMTAlMkMlMjJyaWdodCUyMiUzQTEwJTJDJTIyYm90dG9tJTIyJTNBMTAlMkMlMjJsZWZ0JTIyJTNBMTAlN0QlN0QlMkMlMjJhY3RpdmUlMjIlM0ElN0IlMjJiYWNrZ3JvdW5kQ29sb3IlMjIlM0ElN0IlMjJjb2xvciUyMiUzQSU3QiUyMmxpZ2h0JTIyJTNBJTIyJTIzZjU4MjI3JTIyJTdEJTdEJTJDJTIydGV4dCUyMiUzQSU3QiUyMmZvbnRTaXplJTIyJTNBMTYlMkMlMjJjb2xvciUyMiUzQSU3QiUyMmxpZ2h0JTIyJTNBJTIyJTIzMDAwMDAwJTIyJTdEJTJDJTIydGV4dEFsaWduJTIyJTNBJTIybGVmdCUyMiUyQyUyMmZvbnRXZWlnaHQlMjIlM0ElMjJib2xkJTIyJTdEJTdEJTJDJTIyaG92ZXIlMjIlM0ElN0IlMjJiYWNrZ3JvdW5kQ29sb3IlMjIlM0ElN0IlMjJjb2xvciUyMiUzQSUyMiUyM2RmZTFlNiUyMiU3RCUyQyUyMnRleHQlMjIlM0ElN0IlMjJmb250U2l6ZSUyMiUzQTE4JTJDJTIyY29sb3IlMjIlM0ElMjIlMjM1ZTZjODQlMjIlMkMlMjJ0ZXh0QWxpZ24lMjIlM0ElMjJsZWZ0JTIyJTJDJTIyZm9udFdlaWdodCUyMiUzQSUyMmxpZ2h0ZXIlMjIlN0QlN0QlMkMlMjJpbmFjdGl2ZSUyMiUzQSU3QiUyMmJhY2tncm91bmRDb2xvciUyMiUzQSU3QiUyMmNvbG9yJTIyJTNBJTIyJTIzZjRmNWY3JTIyJTdEJTJDJTIydGV4dCUyMiUzQSU3QiUyMmZvbnRTaXplJTIyJTNBMTYlMkMlMjJjb2xvciUyMiUzQSUyMiUyMzVlNmM4NCUyMiUyQyUyMnRleHRBbGlnbiUyMiUzQSUyMmxlZnQlMjIlMkMlMjJmb250V2VpZ2h0JTIyJTNBJTIybGlnaHRlciUyMiU3RCUyQyUyMmJvcmRlciUyMiUzQSU3QiUyMnRvcCUyMiUzQXRydWUlMkMlMjJsZWZ0JTIyJTNBdHJ1ZSUyQyUyMnJpZ2h0JTIyJTNBdHJ1ZSUyQyUyMmJvdHRvbSUyMiUzQXRydWUlMkMlMjJ3aWR0aCUyMiUzQTElMkMlMjJzdHlsZSUyMiUzQSUyMnNvbGlkJTIyJTJDJTIyY29sb3IlMjIlM0ElN0IlMjJsaWdodCUyMiUzQSUyMiUyM2NjY2VjZiUyMiU3RCU3RCU3RCU3RA==

Aura tab

summary	AWS
params	JTdCJTIydGl0bGUlMjIlM0ElMjJBV1MlMjIlN0Q=

The process with the number of Query Engine =10	The time required for Load-based scaling
scale up	16 mins.
scale down	12 mins.
startup	4 mins.

Aura tab

summary	Azure
params	JTdCJTIydGl0bGUlMjIlM0ElMjJBenVyZSUyMiU3RA==

The process with the number of Query Engine =7	The previous time required for Load-based scaling	New time required for Load-based scaling
Scale up	15-19 minutes	11-13 minutes
Scale down	15-19 minutes	11-13 minutes
Startup	8-10 minutes	8-10 minutes

Aura tab

summary	GCP
params	JTdCJTIydGl0bGUlMjIlM0ElMjJHQ1AlMjIlN0Q=

The process with the number of Query Engine =10	The time required for Load-based scaling
scale up	4-5 mins.
scale down	5-6 mins.
startup	3-4 mins.

To set the scaling modes for Load-Based scaling, perform the following steps.

On the page, the Load page is displayed by default.
Image Removed
To set the scaling mode, select one of the following:
1. Managed: Select the required capacity from the list to start Query Engine when any query is fired.
2. Custom: Select this option to set the rules for resource utilization.
  - To set scale up rules,
    1. Select the required capacity from the list to start Query Engine when any query is fired.
    2. Enter a percentage to scale up the cluster if CPU and Memory utilization threshold goes above the specified percentage. Also, specify the number of data points and the total number of data points to set.
  - To set scale down rules,
    1. Specify the BI Server and/or Query Engine from the list to shut down when no queries are fired for the specified period of time.
    2. Enter a percentage to scale down the cluster if CPU and memory utilization threshold remains below the specified percentage. Also, specify the number of data points and the total number of data points to set.
Click Save. The load-based scaling mode is set.

Panel

panelIconId	atlassian-note
panelIcon	:note:
bgColor	#DEEBFF

NOTE

A data point is information on resource utilization captured every 30 seconds.

+++++++++

Additionally, you can set rules to automatically shut down or start the BI Servers and Query Engines to further optimize resource utilization.

To implement a scale-down rule, you can configure a setting to automatically reduce the capacity of your Query Engines when the cluster is under usage. Additionally, you can set a rule to automatically shut down the Query Engines and BI Servers if the cluster is not in use for a specified period. Similarly, for a scale-up rule, you can set a rule to automatically start the BI Servers and Query Engines with the required capacity when the cluster utilization exceeds over a defined period.

On this page, you can specify rules to scale down or scale up the query engines, as well as set rules to shut down or start the BI Servers and query engines.

...

Panel

panelIconId	atlassian-info
panelIcon	:info:
bgColor	#FFFAE6

Important

If the cluster is down and a query is executed, the first query triggers the cluster startup process, and all queries fail until the cluster is up and running. In this case, the following messages are displayed:
- Message 1: "Could not serve the query as Query Engine Cluster is not available. Query Engine is launched. Please try after some time."
- Message 2: "Could not serve the query as Query Engine Cluster is starting. Please try after some time."
By default, the queries fail until Query Engines are started. If you want to hold the queries when query engines are down, you can specify the time (in minutes) for holding queries by using the QE_STARTUP_QUERY_HOLD_TIME property.
Ensure that if the query engines become active before the configured time, the query will be served; otherwise, it will fail.
The Query Engines do not start for any ROLAP queries.
During the transitioning period of Query Engines; such as scale-up, scale-down, or shut down; you can design and refine the semantic model because the Coordination Master is always up and running .
The capacity of the BI Servers cannot be changed.
All BI Servers can be shut down except the Coordination Master. If there is only one BI Server, this BI Server is treated as the Coordination Master.
The Settings option and Add Schedule option are disabled on the Load screen.
You can view on-screen notifications that provide you with timely information about the state of the cluster.
When you scale down the Query Engines, you reduce the capacity of the node, including the number of cores and memory. Conversely, when you scale up the Query Engines, you increase the capacity of the node by adding more cores and memory.

Versions Compared

Old Version 9

New Version 10

Key

Transitioning period