Spark SQL

Applies to: Kyvos Enterprise  Kyvos Cloud (SaaS on AWS) Kyvos AWS Marketplace

Kyvos Azure Marketplace   Kyvos GCP Marketplace Kyvos Single Node Installation (Kyvos SNI)

Kyvos emulates a Spark SQL table to allow connections from BI tools. In these cases, generally, the generic Spark SQL driver will work. As mentioned before, if a tool-specific driver is available, it should be used.

Prerequisites

Download and install the Spark Simba ODBC driver according to your operating system.

Configuration

To configure Spark SQL connection for Kyvos, perform the following steps.

1. Open the ODBC Administrator (64 bit).

2. Add a new System

3. Select the driver from the list.

4. Click Finish.

5. Click the newly created DSN name. This displays the Simba Spark ODBC Driver DSN Setup dialog box.
   

6. Configure the ODBC connection as:

   1. Data Source Name: This is the name you will use to refer to this

   2. Spark Server Type: Select the SparkThriftServer (1.1 and later) option.

   3. Host(s): This is the Kyvos Web server URL without http. For example, use kyvos_install NOT http://kyvos_install

   4. Port: The port your install is running By default, it’s 8081 for HTTP and 8443 for HTTPS

   5. Authentication Mechanism: Select the Username and Password

   6. Enter your credentials in the User Name and Password

   7. Save Password: Select this checkbox to save the password in the Some tools may honor this, others may still ask.

   8. Thrift Transport: Select the HTTP

   9. HTTP Options - HTTP Path: Enter the path as: kyvos/sql. If you are using a custom application name instead of Kyvos, then use the path as <Web application name>/sql

   10. SSL Options: If you are using SSL, you’ll need to configure this based on your IT recommendations.

7. Click Test. Assuming it connects, click the OK button to save.

Points to remember

• Spark SQL connectors do not support the calculated measures and members created in Kyvos semantic models. For the Spark SQL driver, we recommend creating a calculated expression in your BI tool.

• Spark SQL connectors support only Kyvos aggregations. While a tool may support additional functions, such as MEDIAN, if Kyvos does not support the aggregation, it cannot be used.

• Kyvos supports only the aggregations stored in the Kyvos semantic model. Additional aggregations processed on dimensions, such as COUNT (dimension) are not supported. These will need to be created in the Kyvos semantic model first.

• You cannot use measures without aggregation.

• Queries with JOIN, UNION, or UNION ALL clauses are not supported.

• Semi-Additive measures defined in Kyvos are not supported. Tableau does have limited support. Please see the Tableau section for details.

• Only COUNT, MIN, and MAX measures are supported when the aggregation is added in the BI Tool and is not present in Kyvos.