Applies to: Kyvos Enterprise  Kyvos Cloud (SaaS on AWS) Kyvos AWS Marketplace

Kyvos Azure Marketplace   Kyvos GCP Marketplace Kyvos Single Node Installation (Kyvos SNI)

Kyvos is designed to handle the high availability of components as well as being capable of balancing access to the system. 

The components are shown in the Kyvos architecture diagram.

Open

You can use the Completed Processes tab in the Activity Monitor to view which Kyvos BI server node is serving requests.

The Kyvos BI server is highly available and active/active. There is no downtime switching BI servers allowing Kyvos to scale horizontally without a single point of failure. Any role can be assumed by any engine in the cluster.

Kyvos backend components

• Kyvos Engine Cluster

• Hadoop Cluster

• Coordination Master (One of the nodes in Kyvos Engine Cluster)

• Zookeeper Ensemble

• Repository/Shared Metastore

• HDFS

Each node of a Kyvos cluster can assume one (or more) of the following roles:

• Coordination Master

• BI Servers role

• Query Engine role

The query engine role is assumed by dedicated nodes. Kyvos supports elasticity for the Query Engine, but not for the BI Server.

Kyvos Web Application

The Kyvos web application generally resides on the same edge node as the BI Servers. Kyvos uses Tomcat as its default web server. Tomcat clustering provides both high availability and load balancing of the webserver. Other web servers with similar capabilities can be used in their place.

Kyvos BI Server Cluster

The Kyvos BI Server cluster is processed on a Floating Master approach. Multiple instances of the Kyvos engine form a cluster governed by a coordination master. The coordination master is highly available. If there is a failure, its role is assumed by other nodes in the cluster without downtime. Requests are distributed to the appropriate Kyvos BI server using a round-robin algorithm.

This approach allows Kyvos to scale horizontally without any single point of failure. As a result, each BI Server in Kyvos functions as a BI Server and can function as a coordination master when needed.

Kyvos BI Server

The Kyvos BI Server is designed to serve non‑browsing requests and coordinate query requests to the Query Engines. By using multiple BI Servers, high availability is provided as well as load balancing of the requests coming into Kyvos.

• Non‑browsing requests include dataset and semantic model process requests, coordinating jobs between Kyvos and the “compute” engine (such as Spark or MapReduce), and user authentication requests. User authentication may be handled internally or delegated to another system like LDAP or Kerberos.

• Query delegation handles requests from the Kyvos front end as well as from other applications such as Tableau, Excel, or Python. These application requests are translated into the native Kyvos query language and then passed to the individual Query Engines.

• Once a query engine completes a query, the response is passed back to the BI Server. The BI Server consolidates the responses from the multiple query engines and then packages a single response back to the original application in the proper structure (Native Kyvos, MDX, or SQL).

Coordination Master

The coordination master handles the scheduling and maintenance of the BI Servers, including cuboid balancing and purging threads, in addition to delegating jobs to the other BI Servers.

Kyvos Query Engine Cluster

Query engines are designed to return results to the end-user via the BI Server. They have been designed for optimal performance even when reading from the big data file system. When a Query Engine goes down or becomes unresponsive, the other Query Engines take up the extra load. In addition, when used in a count-based YARN deployment, a replacement query engine will start.

Postgres

The Kyvos Postgres server generally resides on the BI Server node. For high availability, it can be configured to have separate nodes for Active and Standby instances. The Postgres nodes have a live syncing mechanism with automatic failover activated. In case the Active server goes down, the Standby server comes up after the predefined wait interval. There is no data loss and zero failure.

Related topic

Monitoring and configuring and High Availability