Kyvos Modern Architecture with No-Spark

Kyvos is in the process of transitioning from traditional Spark jobs to interactive SQL workloads. Earlier, ETL-like Spark workloads executed map-reduce transformations, which were intrinsically slow, and there were no significant performance enhancements in ETL-type jobs utilizing these Spark engines. To improve performance, Kyvos has replaced these jobs with interactive SQL workloads. This change will help us to export the data from the underlying source and import it into Kyvos, our high-performance analytical store. Kyvos will utilize its own elastic compute cluster to execute this process.

You will have two configuration options:

Mode 1, which uses the existing Query Engines to minimize TCO
Mode 2, offers a separate elastic compute cluster that runs only for the duration of ingestion. This efficient setup significantly reduces semantic model processing time, providing a reassuring boost to our data processing capabilities.

The Kyvos Architecture consists of several layers. At the bottom, there is the data storage layer, which can be a cloud or on-premise data lake, OLTP systems such as Oracle, SQL server, or Streaming Sources like Kafka. The data is transformed using ETL engines and stored in popular cloud Data warehouses such as Snowflake, Redshift, etc.

Kyvos has developed its own high-performance data store, which can store raw data or even aggregated data transformed with ETL engines. This data store acts as the foundation for Kyvos' semantic layer, which sits between the data and consumption layer. The semantic layer is powered by Kyvos GENAI, and it allows you to connect to various underlying data sources, perform data modeling, and build a single unified business view for consumption. Kyvos allows pre-processing of its semantic model through its elastic compute, which results in smart aggregates that are then stored back in the Kyvos data store, boosting query performance. The consumption layer, which is at the topmost layer, consists of BI tools such as Tableau, Excel, Power BI, AI/ML tools, or custom BI applications. These tools can connect to the Kyvos semantic layer using Kyvos named or certified connectors. Kyvos supports various querying interfaces such as SQL, MDX, DAX, REST API, and Java API. Kyvos is also developing its own langchain connector, which is a new framework in the market that enables AI developers to build their own GenAI apps. With Kyvos langchain connector, organizations can have faster access to their data. Finally, Kyvos offers its own state-of-the-art BI and Reporting capabilities along with its Excel Add-in for advanced financial reporting using Excel.

Organizations often utilize identity providers such as Azure AD, OKTA, or LDAP/Microsoft AD to establish user provisioning and single sign-on (SSO) systems for security and governance purposes. Kyvos, as a semantic layer platform, can integrate with all popular IDPs. In addition, a data governance layer is necessary to ensure data security, auditing, and monitoring. Kyvos offers its own data governance service, AuthKey, which guarantees data security when consuming data through its semantic layer.

Related topics