Document toolboxDocument toolbox

Managing and Monitoring Kyvos Clusters

Owned by NIDHI BHANDARI
Last updated: May 31, 2024 by DocReviewer
4 min read

Applies to: Kyvos Enterprise  Kyvos Cloud (SaaS on AWS) Kyvos AWS Marketplace

Kyvos Azure Marketplace   Kyvos GCP Marketplace Kyvos Single Node Installation (Kyvos SNI)

Cluster Management services

Kyvos Manager allows you to manage the Kyvos cluster and services. Once you have deployed the cluster, you can also use these settings to update the cluster configuration.

This section provides an overview of the various configurations available on the Kyvos Manager. Please see the corresponding help page for further details.

See the Cluster details section to know more about the details shown on the Kyvos Manager Dashboard. To learn more about the actions that you can perform from the Kyvos Manager, see the Cluster actions section.

Manage cluster

This option on the navigation pane provides access to the Manage Cluster page that shows a cluster overview with details of all the nodes configured in the cluster.

You can use this page to:

  • Configure High Availability

  • Configure instances for BI Server, Web Portal, Postgres Server, and Standby Postgres Server

  • Add and delete nodes

For more information, click the Hadoop Ecosystem Configuration section for more details. 

Hadoop/EMR Ecosystem configurations

On this page, you can specify configurations for the ecosystem components in your Kyvos cluster, such as:

Configuration

Description/Remarks

Configuration

Description/Remarks

EMR and Hadoop Ecosystem Configuration

Here you can provide details for the vendor, file systems, node authentication, paths and versions for configuration files, and Zookeeper configuration.

HCatalog Configuration

Here you can enable HCatalog and provide details for the Hive node authentication, paths and versions for configuration files, and additional HCatalog parameters.

Execution Engine Configuration

Here you can specify the execution engine to be used for performing various jobs and operations in the cluster. If using Spark, you can also provide details for Spark Source node authentication, paths and versions for configuration files, and additional Spark parameters.

Click here to know more.

Common Configuration

Here you can provide details for the Library and Configuration files.

Click here to know more.

Click here to know more.

Security configurations

On this page, you can specify configurations governing the security across your cluster and data.

The options available are:

Configuration

Description/Remarks

Configuration

Description/Remarks

Hadoop authentication

Here you can specify the authentication type for how the user login details will be authenticated.

Kyvos authentication

Here you can specify the Kyvos authentication mechanism. You can further configure:

  • Native: Native Kyvos authentication mechanism.

  • Kerberos: Enables you to use Kerberos network authentication protocol using secret-key for authentication.

  • LDAP: Integrate your organization's LDAP account for Kyvos authentication and importing users. You can integrate multiple accounts. Click here for more details.

  • SSO for Web portal: Single-sign on configuration for accessing the Kyvos Web Portal using any of the following:

    • Waffle

    • Jespa

    • Kerberos
      Click here for more details.

Hadoop authorization

Here you can define configurations for fine-grained Hadoop authorization control for data access. Kyvos supports the following:

  • Sentry

  • Ranger

Network Protocol Configuration

Here you can specify the TSL Configuration for secure communication between client and server.

You can also configure the HTTPS and certificates for Kyvos Web Portal here.

Kyvos Web Portal External Authentication Configuration

Here, you can specify an external authentication mechanism for the Kyvos Web portal. Kyvos supports the following:

  • SAML2.0 based

  • Host Application-based

  • Certificate-based

  • OIDC based

Kyvos Preferences

Here you can specify Kyvos User Preferences for Hadoop access as either KEYTAB PRINCIPAL USER with administrative rights or KYVOS LOGGED IN USER with specific access rights to perform activities as per the user's privileges

Data Security

Here you can specify the Custom Data Security Configurations to limit access to the semantic model data. Kyvos supports:

  • Custom Row-level Security

  • Custom Column-Level Security

  • Custom JDBC Security

Java Options Configuration

This page is used to configure custom Java options for Web Portal, Query Engine, and BI Server after cluster deployment.

Click here to know more.

Query Engine YARN Configuration

This page is used  to  configure the YARN-based Query Engines specifying the number of cores and memory to be used in the cluster if you have deployed the cluster in YARN mode. 

  • You can also switch the cluster mode from YARN to non-YARN.

  • You can choose to add nodes to the cluster or configure existing nodes to assign the role of Query Engine to them.

Resources Configuration

This page is used to show the resources created at the time of cluster creation. You can add and remove resources/disks using the settings on this page.

Click here to know more.

Segmentation Configuration

This page is used to manage and create multiple segments and divide the system resources (such as Query Engines or disk space) among these segments.

Click here to know more.

Properties 

This page is used to view, edit, add, or delete the Kyvos application-level properties governing your cluster environment.

Configuration Files

This page is used to  update and upload the Configuration Files on the cluster.

Instances State Configuration

Use these settings to activate or deactivate the BI Server and Query Engine instances in your cluster.

Click here to know more.

Properties

Use these settings to view, edit, add, or delete the Kyvos application-level properties governing your cluster environment.

Click here to know more.

Sanity Suite

Run the Sanity Suite to check if all the activities (like creating a Registered File, Data Relationship Diagram, and Semantic model processing) for browsing emantic models in the Kyvos application are working fine.

Click here to know more.

Uploading External Libraries

This page is used to upload  any third-party JAR files that you want to use for the Kyvos environment. 

Disaster Recovery

This page is used to restore the system from a disaster. You can also manually recover Kyvos Manager nodes and roles on it. Both guided, and manual recovery are supported.



Cluster Monitoring services

Kyvos Manager allows you to monitor the cluster state through multiple options available on the navigation pane.

You can see:

  • The dashboard with the state of all Kyvos clusters and nodes. The dashboard also shows a graphical representation of the node-level utilization metrics.

  • Utilities options allowing you to monitor:

    • Audit Trail helps to track changes in configuration parameters.

    • Cluster Logs help to track changes made to the cluster.

    • Cluster Snapshot showing the entire cluster configuration settings in one place.

    • Events occurred on the cluster.

    • Reminders and email messages for alerts, warnings, services status, or nodes down

    • Operations performed on the cluster.

    • Agent Logging Configuration allows you to configure agent logging parameters like log level, file size, and backup file count.

Copyright Kyvos, Inc. All rights reserved.