Versions Compared

Version Old Version 3 New Version Current
Changes made by

Sarabjeet Kaur

Sarabjeet Kaur

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. Unity Catalog must be enabled on your Databricks cluster.

  2. In Unity Catalog, create storage credentials and an external location with appropriate access permissions for both the source and destination locations. You must have permission to create storage credentials and external locations.

  3. To create storage credentials, first, create an access connector for Azure Databricks. Assign that access connector while creating storage credentials.

  4. Grant the managed identity access to the storage account. You must have the Owner or User Access Administrator Azure RBAC role to grant permission to access your storage account.

    1. Log in to your Azure Data Lake Storage Gen2 account.

    2. Go to Access Control (IAM), click + Add, and select Add role assignment.

    3. Select the Storage Blob Data Contributor role and click Next.

    4. Under Assign access, select a Managed identity.

    5. Click +Select Members, and select Access connector for Azure Databricks or User-assigned managed identity.

    6. Search for your connector name or user-assigned identity, select it, and click Review and Assign.

  5. Fetch the ABFSS path till the parent directory of your respective storage account.

  6. On the Unity Catalog page, go to external locations and create a new external location by providing an external location name, newly created Storage credential, and fetched ABFSS URL. The URL format should be in abfss://my-container-name@my-storage-account.dfs.core.windows.net/<path>

  7. Test the newly created external location by clicking the 'Test connection' button. This will validate the connection with the external location path. Through the permissions tab, assign your user the CREATE EXTERNAL TABLE and WRITE FILESrole.

  8. Go to semantic model Advanced Properties, and add the following properties:

    • kyvos.sqlwarehouse.catalog

    : This property is used to create a temp catalog name for creating a parquet table.

    • = <catalog name for temporary tables created while cubes building>

    • kyvos.sqlwarehouse.tempdb

    : This property is used to create a temp database name for creating a parquet table

    • = <temporary database for temporary tables created while cubes building>

  9. You must have 'create table and Use schema' permissions on the temp catalog.

...