Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Applies to: Image Removed Kyvos Enterprise    Image Removed Kyvos Cloud (Managed Services on AWS)     Image Removed Kyvos Azure Marketplace

Image Removed Kyvos AWS Marketplace    Image Removed Kyvos Applies to: (tick) Kyvos Enterprise  (error) Kyvos Cloud (SaaS on AWS) (error) Kyvos AWS Marketplace

(error) Kyvos Azure Marketplace   (error) Kyvos GCP Marketplace (error) Kyvos Single Node Installation (Kyvos SNI)     Image Removed Kyvos Free ( Limited offering for AWS)

...

Azure Key Vault is a cloud service that provides a secure store for secrets. You can securely store keys, passwords, certificates, and other secrets in the vault.

You can create and manage the key vaults through the Azure portal.

Creating a vault

Create a vault as explained at Quickstart - Create an Azure Key Vault with the Azure portal | Microsoft Learn

...

Panel
panelIconIdatlassian-note
panelIcon:note:
bgColor#DEEBFF

Note the Vault Name and Vault URL for use later.

...

Once you have created the Key Vault, navigate to the Access Configuration page, and perform the following steps:

  1. From the Permission Model, select the option.

  2. From the Resource Access, choose the following permissions (as shown in the figure below).

    1. Azure Virtual Machines for deployment

    2. Azure Resource Manager for template deployment

    3. Azure Disk Encryption for volume encryption

...

Adding secret to Key Vault 

To add a secret to the vault, you need to perform a few additional steps. In this case, we add a password that could be used by an application. The password is called ExamplePassword, and we store the value of hVFkk965BuUv in it. 

  1. On the Key Vault properties page, select Secrets.

  2. Click on Generate/Import

  3. On the Create a secret screen, select the following values: 

    1. Upload options: Manual

    2. Name: ExamplePassword

    3. Value: hVFkk965BuUv

  4. Leave the other values to their defaults. Click Create.

  5. By clicking the Show Secret Value  button in the right pane, you can see the hidden value.

...

Panel
panelIconIdatlassian-note
panelIcon:note:
bgColor#DEEBFF

Note

Please note down the Name of Secret given in the <secret-name> as you will need it later.

Create an Azure Key Vault-backed secret scope

  1. Verify that you have Contributor permission on the Azure Key Vault instance that you want to use to back the secret scope.

  2. Go to https://<your_azure_databricks_url>#secrets/createScope (for example, https://westus.azuredatabricks.net#secrets/createScope ). This URL is case sensitive; the scope in createScope must be uppercase.

  3. Enter the name of the secret scope. Secret scope names are case insensitive. 

  4. Use the Manage Principal drop-down to specify whether All Users have to  MANAGE permission for this secret scope or only the Creator of the secret scope (that is to say, you).

    1. MANAGE permission allows users to read and write to this secret scope, and, in the case of accounts on the Azure Databricks Premium Plan , to change permissions for the scope.

    2. Your account must have the Azure Databricks Premium Plan for you to be able to select Creator . This is the recommended approach: grant MANAGE permission to the Creator When you create the secret scope, and then assign more granular access permissions after you have tested the scope. For example workflow, see Secret workflow example .

    3. If your account has the Standard Plan, you must set the MANAGE permission to the “All Users” group. If you select Creator here, you will see an error message when you try to save the scope.

    4. For more information about the MANAGE permission, see Secret access control

  5. Enter the DNS Name (for example, https://databrickskv.vault.azure.net/ ) and Resource ID, for example: /subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourcegroups/databricks-rg/providers/Microsoft.KeyVault/vaults/databricksKV.

  6. These properties are available from the Properties tab of an Azure Key Vault in your Azure portal.

    1. Click the Create button.

    2. Use the Databricks CLI Databricks secrets list-scopes command to verify that the scope was created successfully.

...

Panel
panelIconIdatlassian-note
panelIcon:note:
bgColor#DEEBFF

Note

Please note down the Name of the Secret given in the <secret-name> as you will need it later.

Using  Key Vault Secret in Azure Databricks

...


Panel
panelIconIdatlassian-note
panelIcon:note:
bgColor#DEEBFF

Note

Available in Databricks Runtime 6.1 and above.

The Key Vault secret value is referred to in Databricks using the below syntax.
{{secrets/<scope-name>/<secret-name>}}

The value must start with {{secrets/ and end with }} . The variable portions of the property or environment variable are: 

  • <scope-name> : The name of the scope in which the secret is associated. 

  • <secret-name> : The unique name of the secret in the scope. 

Use the secret value with Spark property as:  spark.hadoop.fs.azure.account.oauth2.client.secret {{secrets /<scope-name>/<secret-name>}}