/
Working with Databricks SQL Warehouse on AWS

Working with Databricks SQL Warehouse on AWS

Prerequisites for creating AWS SQL Warehouse Connection

Before creating an AWS SQL Warehouse connection without Spark, complete the following prerequisites.

  1. Unity Catalog must be enabled on your Databricks cluster.

  2. To create a storage credential for connecting to AWS S3, refer to Databricks documentation.

  3. Fetch the S3 path till the parent directory of your respective storage account.

  4. On the Unity Catalog page, go to external locations and create a new external location by providing an external location name, newly created Storage credential, and fetched s3 URL.
    The URL format must be in  s3://bucket name/<path> For example s3://kyvos-output-56555/user/engine_work

  5. Test the newly created external location by clicking the 'Test connection' button. This will validate the connection with the external location path. Through the permissions tab, assign your user the CREATE EXTERNAL TABLE and WRITE FILES role.

  6. Go to semantic model Advanced Properties, and add the following properties:

    1. kyvos.sqlwarehouse.catalog: Enter the temp catalog name to create a parquet table.

    2. kyvos.sqlwarehouse.tempdb: Enter the temp database name to create a parquet table.

  7. You must have 'create table and Use schema' permissions on the temp catalog.

Creating AWS Databricks SQL Warehouse connection

You can create AWS Databricks SQL warehouse connection with no-Spark.

Note

The steps for Working with Databricks SQL Warehouse with Personal Token are the same.

  • Supported for AWS and Azure.

  • Supported only with premium workspace.

  • The OAuth connectivity with Databricks SQL Warehouse for AWS no-Spark is not supported.

  • The serverless type Databricks SQL Warehouse cluster is only supported.

  • Unity Catalog must be enabled.

To create AWS SQL Warehouse connection for processing semantic models with No-Spark, perform the following steps.

  1. From the Toolbox, click Connections.

  2. From the Actions menu (  ) click Add Connection.

  3. Enter the following details: 

Parameter/Field

Comments/Description

Name

Enter a unique name for the connection.

Category

Select the Warehouse option.  There may be more than one warehouse connection.

Provider

Select DATABRICKSSQL from the list.

Driver class

This field will be auto selected. The Driver class (com.databricks.client.jdbc.Driver ) is required to connect to the Databricks SQL.

URL

In the URL field, enter the Databricks SQL Warehouse JDBC URL.

For more information, see Microsoft documentation.

Authentication Type

Select Personal Access Token.

User Name

Enter the text token for username.

Password

Enter Databricks SQL Personal Access Token’s value for the Databricks SQL workspace

Use as Source

This checkbox is auto selected. Enter Spark Read Method as JDBC

Is default SQL Engine

By default, the checkbox is selected as this connection can only be used to read data (creating datasets) on which the semantic model will be created. 

Catalog Enabled

Select this checkbox to list different catalog created in the workspace.

Properties

Click Properties to view or set properties.

  1. After you finish configuring the settings using the table shown below the screenshot, click the Test button from the top left to validate the connection settings.

  2. If the connection is valid, click the Save button. 

image-20241220-122712.png

Connection Details in Kyvos for Sanity Suite

To create AWS SQL Warehouse connection for sanity suite, perform the following steps.

Parameter

Description

Parameter

Description

Name

Enter SanityConnection as a name.

Category

Select the Warehouse option.

Provider

Select the Generic option.

Driver

Enter the Driver class as com.databricks.client.jdbc.Driver 

URL

Enter JDBC Databricks URL.

Username

Enter Databricks token.

Password

Enter Databricks token.

Authentication Type

Select Personal Access Token.

Use as Source

This checkbox is auto selected. Enter Spark Read Method as JDBC.

Is Default SQL Engine

To enable the connection for raw data, click the Is Default SQL Engine checkbox to set this connection to run the default SQL engine.

Properties

Click Properties to view or set properties.

Catalog Enabled

Select this checkbox to list different catalog created in the workspace.

 

Related content

Copyright Kyvos, Inc. All rights reserved.