The Kyvos 2023.5 has introduced an in-house data storage layer called the Kyvos Analytical Store. The data store is designed to support complex, long-running queries that process significant portions of the stored dataset.
This layer provides efficient data storage and retrieval capabilities that help to optimize both cost and time savings. The Kyvos Analytical Store is specifically designed to support complex, long-running queries that process substantial portions of the stored dataset. It is equipped with a columnar-vectorized query execution engine that processes a large batch of values, referred to as a "vector", in a single operation. This vectorized query execution significantly improves performance, especially in OLAP queries.
Configuration
Kyvos Analytical Store feature is not enabled by default. You can enable it by using the following property on the Semantic model.
Property: kyvos.build.aggregate.type
Value: Tabular
Scope: Semantic model advanced property
Memory tuning parameters
From Kyvos 2023.5 onwards, Heap, Off-heap, and other required memory on Query Engines are automatically configured if memory mode is set to Auto mode. For more details, see the Kyvos Java options page in Kyvos Manager.
To enable Kyvos Analytical store, perform the following steps.
Login to Kyvos Manager and navigate to Properties > queryengine.properties
In the queryengine.properties, search for the DS_CONNECTION_CONF property.
The default value will be Blank
Set this value as SET memory_limit='10GB'; SET THREADS to 1; and then save it.
Restart the Query Engine services from the Kyvos Manager dashboard.
For memory configuration, perform the following post-deployment steps.
Open the Kyvos Java options page in Kyvos Manager and Navigate to the Query Engine section.
If Configure Heap memory for Query Engine is set as Auto, no change is required.
If Configure Heap memory for Query Engine is set as Manual, reduce Off-Heap memory by 10 GB.
For Example, Suppose the total machine memory is 128 GB, whereas 45 GB is assigned to Heap, and 66 GB is assigned to Off-Heap. In that case, you need to set Heap as 45 GB and Off-Heap as 56 GB so Kyvos can use the remaining 10 GB internally for the new data store-based architecture.
Enhancements in Kyvos Analytical Store
Kyvos 2024.1
Note
Support for Kyvos Analytical Store is certified when the cuboid replication type is set to 'Local' on the semantic model through the kyvos.cuboid.replication.type property.
Support for Decimal type measures: The Kyvos Analytical store efficiently stores, retrieves, and queries big decimal measures created via Kyvos. This release supports reading 128-bit decimal numbers via Parquet files with the help of a fixed-length byte array.
Kyvos Analytical Store statistics: With this release, users can now see Kyvos Analytical Store statistics on the Activity Monitor. To view the Kyvos Analytical Store information, navigate to the Activity Monitor, then click the Query Engine. A detailed view dialog box is displayed. The information displays the total number of available Analytical Store connections and the number of connections currently being used. Users can also check the total amount of memory available and the percentage of memory used.
Support for Parent-Child Hierarchy (PCH): With this release, Kyvos supports processing and browsing the semantic models having parent-child hierarchy, alternate hierarchy, and custom rollups in the Kyvos Analytical Store.
Support for attribute level smart aggregates: This release supports attribute level smart aggregates in Kyvos Analytical Store.