The Kyvos 2023.5 has introduced an in-house data storage layer called the Kyvos Analytical Store. The data store is designed to support complex, long-running queries that process significant portions of the stored dataset.
This layer provides efficient data storage and retrieval capabilities that help to optimize both cost and time savings. The Kyvos Analytical Store is specifically designed to support complex, long-running queries that process substantial portions of the stored dataset. It is equipped with a columnar-vectorized query execution engine that processes a large batch of values, referred to as a "vector", in a single operation. This vectorized query execution significantly improves performance, especially in OLAP queries.
Configuration
Kyvos Analytical Store feature is not enabled by default. You can enable it by using the following property on the Semantic model.
Property: kyvos.build.aggregate.type
Value: Tabular
Scope: Semantic model advanced property
Memory tuning parameters
In 2023.5 release, we have smartness to configure Heap, Off-heap and other required memory on Query Engines automatically if memory mode is set to Auto mode. User needs to perform below steps in case they want to use Kyvos Analytical store
Steps to enable
Login to Kyvos Manager and navigate to Properties > queryengine.properties
Search the DS_CONNECTION_CONF property.
Default value will be SET memory_limit='10GB'; SET THREADS to 1;
Edit this value and save it.
Restart the Query Engine services from the Kyvos Manager dashboard.
For manual memory mode, perform the following post-deployment steps.
Open the Kyvos Java options page in Kyvos Manager and Navigate to Query Engine section.
If Configure Heap memory for Query Engine is set as Auto, no need to change anything.
If the Configure Heap memory option for Query Engine is set as Manual, reduce the Off-Heap memory by 10 GB.
For Example, if the Total machine memory is 128 GB, in which 45 GB is assigned to Heap and 66 GB is assigned to Off-Heap. i.e., the user needs to set Heap as 45 GB and Off-Heap as 56 GB so that Kyvos can use the remaining 10 GB internally for new data store-based architecture.