Versions Compared

Old Version 25

changes.mady.by.user Sarabjeet Kaur

Saved on

compared with

New Version 26

changes.mady.by.user Sarabjeet Kaur

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

This layer provides efficient data storage and retrieval capabilities that help to optimize both cost and time savings. The Kyvos Analytical Store is specifically designed to support complex, long-running queries that process substantial portions of the stored dataset. It is equipped with a columnar-vectorized query execution engine that processes a large batch of values, referred to as a "vector", in a single operation. This vectorized query execution significantly improves performance, especially in OLAP queries.

Panel
panelIconIdatlassian-note
panelIcon:note:
bgColor#DEEBFF

Note

To use semantic models with Kyvos Analytical store, you must process the full data of the semantic model if it has already been processed before. Additionally, it is recommended not to use the semantic model that was previously processed if you have enabled Kyvos Analytical Store. After enabling Kyvos Analytical Store, the previously processed semantic model may slow down the query execution time due to the new memory configuration.

Certified Providers and supported Operating Systems for Kyvos Analytical Store

...

To use Kyvos Analytical Store, you must ensure the following:

Using Kyvos Analytical Store for semantic models

...

  • on the semantic model

...

  • .

...

Value: Tabular 

Scope: Semantic model advanced property 

Add Kyvos Analytical Store

...

Jar in

...

classpath 

To add Kyvos Analytical Store Jar in class pathclasspath, perform the following steps.

  1. Login to Kyvos and navigate to the Connections page.

  2. On the Connections page, click Properties.

  3. On the Edit Properties dialog box, search “kyvos.connection.classpath.common.jar” and enter the ../lib/duckdb_jdbc.jar value in the existing class path.

    image-20240223-070315.png

  4. Restart Kyvos services.  

Using Kyvos Analytical Store for semantic models

To use the Kyvos Analytical store as data storage layer for a semantic model, use the following property on the semantic model. 

Property: kyvos.build.aggregate.type  

Value: Tabular 

Scope: Semantic model advanced property 

Kyvos Analytical Store memory calculations

Parameter

Memory

...

Calculations

HEAP_MEMORY

VM_TOTAL_MEMORY / 4

OFF_HEAP_MEMORY

VM_TOTAL_MEMORY / 8

KYVOS_ANALYTICAL_STORE_MEMORY

VM_TOTAL_MEMORY / 4

For example, If you have 64 GB QE VM, the following will be the memory calculations:

  • HEAP_MEMORY = 16 GB

  • OFF_HEAP_MEMORY= 8 GB

  • KYVOS_ANALYTICAL_STORE_MEMORY= 16 GB

Memory tuning parameters 

The Heap, Off-heap, and other required memory on Query Engines are automatically configured if memory mode is mode is set to Auto mode.  For

For more details, see the Kyvos Java options page in Kyvos Manager.

...

Aura tab collection
paramsJTdCJTIyZ2VuZXJhbCUyMiUzQSU3QiUyMnRhYlNwYWNpbmclMjIlM0EwJTJDJTIydGFiV2lkdGglMjIlM0ExMDAlMkMlMjJ0YWJIZWlnaHQlMjIlM0E1MCUyQyUyMmRpcmVjdGlvbiUyMiUzQSUyMmhvcml6b250YWwlMjIlN0QlMkMlMjJjb250ZW50JTIyJTNBJTdCJTIyYmFja2dyb3VuZENvbG9yJTIyJTNBJTdCJTIyY29sb3IlMjIlM0ElMjIlMjNmZmYlMjIlN0QlMkMlMjJib3JkZXIlMjIlM0ElN0IlMjJzdHlsZSUyMiUzQSUyMnNvbGlkJTIyJTJDJTIyd2lkdGglMjIlM0ExJTJDJTIydG9wJTIyJTNBdHJ1ZSUyQyUyMmJvdHRvbSUyMiUzQXRydWUlMkMlMjJsZWZ0JTIyJTNBdHJ1ZSUyQyUyMnJpZ2h0JTIyJTNBdHJ1ZSUyQyUyMmNvbG9yJTIyJTNBJTdCJTIybGlnaHQlMjIlM0ElMjIlMjNjY2NlY2YlMjIlN0QlN0QlMkMlMjJwYWRkaW5nJTIyJTNBJTdCJTIydG9wJTIyJTNBMTAlMkMlMjJyaWdodCUyMiUzQTEwJTJDJTIyYm90dG9tJTIyJTNBMTAlMkMlMjJsZWZ0JTIyJTNBMTAlN0QlN0QlMkMlMjJhY3RpdmUlMjIlM0ElN0IlMjJiYWNrZ3JvdW5kQ29sb3IlMjIlM0ElN0IlMjJjb2xvciUyMiUzQSU3QiUyMmxpZ2h0JTIyJTNBJTIyJTIzZjU4MjI3JTIyJTdEJTdEJTJDJTIydGV4dCUyMiUzQSU3QiUyMmZvbnRTaXplJTIyJTNBMTYlMkMlMjJjb2xvciUyMiUzQSU3QiUyMmxpZ2h0JTIyJTNBJTIyJTIzMDAwMDAwJTIyJTdEJTJDJTIydGV4dEFsaWduJTIyJTNBJTIybGVmdCUyMiUyQyUyMmZvbnRXZWlnaHQlMjIlM0ElMjJib2xkJTIyJTdEJTdEJTJDJTIyaG92ZXIlMjIlM0ElN0IlMjJiYWNrZ3JvdW5kQ29sb3IlMjIlM0ElN0IlMjJjb2xvciUyMiUzQSUyMiUyM2RmZTFlNiUyMiU3RCUyQyUyMnRleHQlMjIlM0ElN0IlMjJmb250U2l6ZSUyMiUzQTE4JTJDJTIyY29sb3IlMjIlM0ElMjIlMjM1ZTZjODQlMjIlMkMlMjJ0ZXh0QWxpZ24lMjIlM0ElMjJsZWZ0JTIyJTJDJTIyZm9udFdlaWdodCUyMiUzQSUyMmxpZ2h0ZXIlMjIlN0QlN0QlMkMlMjJpbmFjdGl2ZSUyMiUzQSU3QiUyMmJhY2tncm91bmRDb2xvciUyMiUzQSU3QiUyMmNvbG9yJTIyJTNBJTIyJTIzZjRmNWY3JTIyJTdEJTJDJTIydGV4dCUyMiUzQSU3QiUyMmZvbnRTaXplJTIyJTNBMTYlMkMlMjJjb2xvciUyMiUzQSUyMiUyMzVlNmM4NCUyMiUyQyUyMnRleHRBbGlnbiUyMiUzQSUyMmxlZnQlMjIlMkMlMjJmb250V2VpZ2h0JTIyJTNBJTIybGlnaHRlciUyMiU3RCUyQyUyMmJvcmRlciUyMiUzQSU3QiUyMnRvcCUyMiUzQXRydWUlMkMlMjJsZWZ0JTIyJTNBdHJ1ZSUyQyUyMnJpZ2h0JTIyJTNBdHJ1ZSUyQyUyMmJvdHRvbSUyMiUzQXRydWUlMkMlMjJ3aWR0aCUyMiUzQTElMkMlMjJzdHlsZSUyMiUzQSUyMnNvbGlkJTIyJTJDJTIyY29sb3IlMjIlM0ElN0IlMjJsaWdodCUyMiUzQSUyMiUyM2NjY2VjZiUyMiU3RCU3RCU3RCU3RA==

Aura tab
summaryKyvos versions up to 2024.1
paramsJTdCJTIydGl0bGUlMjIlM0ElMjJLeXZvcyUyMHZlcnNpb25zJTIwdXAlMjB0byUyMDIwMjQuMSUyMiU3RA==

  1. Login to Kyvos Manager and navigate to Properties > queryengine.properties.

  2. In the queryengine.properties, search for the DS_CONNECTION_CONF property.

    • The default value will be Blank  blank. 

    • Set this value as SET memory_limit='10GB'; SET THREADS to 1; and then save it. 

  3. Restart the Query Engine services from the Kyvos Manager dashboard.

For memory configuration, perform the following post-deployment steps.

  1. Open the Kyvos Java options page in Kyvos Manager and navigate to the Query Engine section. 

  2. If Configure Heap memory for Query Engine is set as Auto, no change is required. 

  3. If Configure Heap memory for Query Engine is set as Manual, reduce Off-Heap memory by 10 GB. 
    For Example, Suppose the total machine memory is 128 GB, whereas 45 GB is assigned to Heap, and 66 GB is assigned to Off-Heap. In that case, you need to set Heap as 45 GB and Off-Heap as 56 GB so Kyvos can use the remaining 10 GB internally for the new data store-based architecture. 

Aura tab
summaryKyvos 2024.1.1 onwards
paramsJTdCJTIydGl0bGUlMjIlM0ElMjJLeXZvcyUyMDIwMjQuMS4xJTIwb253YXJkcyUyMiU3RA==
Panel
panelIconIdatlassian-note
panelIcon:note:
bgColor#DEEBFF

Before enabling Kyvos Analytical Store, ensure that Query Engine cluster is running with the required (Full, high, moderate, or Low) capacity, and disable cluster scaling. For this, set the value of ELASTIC_DEPLOYMENT should be set as Disable.

  1. Login to Kyvos Manager and navigate to Properties > global.properties.

  2. Set the value of ENABLE_KYVOS_ANALYTICAL_STORE property as Yes.

  3. Restart all Kyvos services (BI Server, Query Engine, and Kyvos Web Portal) from the Kyvos Manager dashboard.

  4. Open the Kyvos Java options page in Kyvos Manager and navigate to the Query Engine section, and ensure that the Configure Heap memory for Query Engine is set as Auto. Kyvos will now automatically configure memory used for the Analytical Store.

Enhancements in Kyvos Analytical Store

Kyvos 2024.1.x

Kyvos comes with an enhanced Query Engine JVM configuration as per Kyvos Data Store Analytics configurations. To enable this feature, set the value of the ENABLE_KYVOS_ANALYTICAL_STORE property to 'Yes.' After configuring the Kyvos Data Store value, you must disable cluster scaling. Achieve this by setting the ELASTIC_DEPLOYMENT property to 'Disable.

Kyvos 2024.1

Panel
panelIconIdatlassian-note
panelIcon:note:
bgColor#DEEBFF

Note

Kyvos Analytical Store is certified only when the cuboid replication type is set to 'Local' on the semantic model. You can do this through the kyvos.cuboid.replication.type property.

...

Support for Parent-Child Hierarchy (PCH): With this release, Kyvos supports processing and browsing the semantic models having parent-child hierarchy, alternate hierarchy, and custom rollups in the Kyvos Analytical Store.

...

  1. Analytical Store.