Applies to: Kyvos Enterprise  Kyvos Cloud (SaaS on AWS) Kyvos AWS Marketplace

Kyvos Azure Marketplace   Kyvos Azure Marketplace Kyvos Single Node Installation (Kyvos SNI)

You can use a data connection to connect to a single Amazon Redshift database. You can create multiple connections if needed, each going to a separate Redshift database.

Prerequisites

You must complete the following: 

1. Create the following VPC Endpoints to use the Redshift connection.

   com.amazonaws.<region>.redshift
   com.amazonaws.<region>.redshift-data

2. Complete the following configuration for the Spark Unload mechanism.

   1. Provide Redshift IAM Role access rights on the Kyvos S3 bucket to avoid AccessDenied exception. This is required for Redshift unloading data to Amazon S3. 

   2. Attach an IAM Role to the Redshift cluster that has the rights to write on the Kyvos S3 location.

   3. Copy the ARN of the IAM role attached to Redshift.

   4. Add the below policy statement to the Kyvos S3 bucket policy. Refer to AWS documentation for details.

      "Sid": "Redshift-role-access", "Effect": "Allow", "Principal": { "AWS": "<ARN of the IAM role>" } "Action": [ "s3:PutAnalyticsConfiguration", "s3:GetObjectVersionTagging", "s3:ReplicateObject", "s3:GetObjectAcl", "s3:GetBucketObjectLockConfiguration", "s3:DeleteBucketWebsite", "s3:PutLifecycleConfiguration", "s3:GetObjectVersionAcl", "s3:DeleteObject", "s3:GetBucketPolicyStatus", "s3:GetObjectRetention", "s3:GetBucketWebsite", "s3:PutReplicationConfiguration", "s3:PutObjectLegalHold", "s3:GetObjectLegalHold", "s3:GetBucketNotification", "s3:PutBucketCORS", "s3:GetReplicationConfiguration", "s3:ListMultipartUploadParts", "s3:PutObject", "s3:GetObject", "s3:PutBucketNotification", "s3:PutBucketLogging", "s3:GetAnalyticsConfiguration", "s3:PutBucketObjectLockConfiguration", "s3:GetObjectVersionForReplication", "s3:GetLifecycleConfiguration", "s3:GetInventoryConfiguration", "s3:GetBucketTagging", "s3:PutAccelerateConfiguration", "s3:DeleteObjectVersion", "s3:GetBucketLogging", "s3:ListBucketVersions", "s3:RestoreObject", "s3:ListBucket", "s3:GetAccelerateConfiguration", "s3:GetBucketPolicy", "s3:PutEncryptionConfiguration", "s3:GetEncryptionConfiguration", "s3:GetObjectVersionTorrent", "s3:AbortMultipartUpload", "s3:GetBucketRequestPayment", "s3:GetObjectTagging", "s3:GetMetricsConfiguration", "s3:DeleteBucket", "s3:PutBucketVersioning", "s3:GetBucketPublicAccessBlock", "s3:ListBucketMultipartUploads", "s3:PutMetricsConfiguration", "s3:GetBucketVersioning", "s3:GetBucketAcl", "s3:PutInventoryConfiguration", "s3:GetObjectTorrent", "s3:PutBucketWebsite", "s3:PutBucketRequestPayment", "s3:PutObjectRetention", "s3:GetBucketCORS", "s3:PutBucketAcl", "s3:GetBucketLocation", "s3:ReplicateDelete", "s3:GetObjectVersion", "s3:PutBucketTagging" ], "Resource": [ "arn:aws:s3:::<kyvos S3 bucket name>/_kyvos_app_intermediate_dir_/_kyvos_app_load_/*", "arn:aws:s3:::<kyvos S3 bucket name>" ] }

3. Change bucket ownership of Kyvos S3 bucket to Bucket Owner Enforced in case of existing Kyvos deployment. Refer to AWS Documentation for details. 

4. If Redshift and Kyvos resources are in different VPCs, and Redshift is not publicly accessible, do the following:

• Using Redshift-managed VPC endpoints

• Managing Redshift-managed VPC endpoints using the Amazon Redshift console

• Granting access to a cluster

• Creating a Redshift-managed VPC endpoint

Considerations when using Redshift-managed VPC endpoints

Before creating a Redshift-managed VPC endpoint (Points to be shared with Client), ensure the following: 

• The cluster to access is an RA3 node type.

• The cluster to access has cluster relocation turned on. For information about requirements to turn on cluster relocation, see Managing cluster relocation in Amazon Redshift .

• The cluster to access is available through port 5439.

• Modify the VPC security groups associated with an existing Redshift-managed VPC endpoint, if needed. 
  To modify other settings, delete the current Redshift-managed VPC endpoint and create a new one.

• The number of Redshift-managed VPC endpoints that you can create is limited to your VPC endpoint quota.

• The Redshift-managed VPC endpoints aren't accessible from the internet.
  A Redshift-managed VPC endpoint is accessible only within the VPC where the endpoint is provisioned.
   or
   Any VPCs peered with the VPC where the endpoint is provisioned as permitted by the route tables and security groups.

• You can't use the Amazon VPC console to manage Redshift-managed VPC endpoints.

Managing Redshift-managed VPC endpoints using the Amazon Redshift console

You can configure the use of Redshift-managed VPC endpoints by using the Amazon Redshift console.

Granting access to a cluster

Ensure that the Client (Account A) performs the following steps on the Redshift Cluster:

To allow a VPC in another AWS account to have access to your cluster, perform the following steps: 

1. Sign in to the AWS Management Console and open the Amazon Redshift console.

2. On the navigation menu, click Clusters.

3. For the cluster that you want to allow access to, view the cluster details by choosing the cluster name. Click the Properties tab of the cluster.
   The Granted accounts section displays the accounts and corresponding VPCs that have access to your cluster.

4. Click Grant access to display a form to enter Grantee information to add an account.

5. For AWS account ID, enter the ID of the account you are granting access. You can grant access to specific VPCs or all VPCs in the specified account.

6. Click Grant access to grant access.

Creating a Redshift-managed VPC endpoint

Ensure that the Kyvos team (Account B) performs the following steps on the Redshift AWS console. 

1. Sign in to the AWS Management Console and open the Amazon Redshift console.

2. On the navigation menu, click Configurations.
   The Configuration page displays the Redshift-managed VPC endpoints that have been created.

3. To view details for an endpoint, select its name.

4. Click Create endpoint to display a form to enter information about the endpoint to add.

5. Enter values for Endpoint name, AWS account ID, Cluster identifier, Virtual private cloud (VPC), Subnet group, and other properties of the endpoint.
   The subnet group in the Subnet group defines the subnets and IP addresses where Amazon Redshift deploys the endpoint.
   Amazon Redshift chooses a subnet that has IP addresses available for the network interface associated with the endpoint.
   The optional security group in the Security group defines the ports, protocols, and sources for inbound traffic that you are authorizing for your endpoint. Commonly, you allow access to port 5439 to the security group or the CIDR range where your workloads run.

6. Click Create endpoint to create the endpoint.

After your endpoint is created, you can access the cluster through the URL shown in the Endpoint URL in the configuration settings for your Redshift-managed VPC endpoint.

Policy definition

The policy gives rights to the Redshift resource. To restrict the resource, replace * with these resource details:

Set up or view Redshift connection

You can set up a Redshift connection using a user name and password or IAM Instance Profile Credentials. For both connection types, you need to:

• Provide the iam_role

• Set the  LoginTimeout value

To set up or view a Redshift warehouse connection, perform the following steps. 

1. From the Toolbox, click Setup, then Connections.

2. From the Actions menu (  ⋮  ) click Add Connection.

3. Enter a name or select it from the Connection list.

4. After you finish configuring the settings using the table shown below the screenshot, click the Test button from the top left to validate the connection settings.

5. If the connection is valid, click the Save button.

6. To refresh connections, click the Actions menu (  ⋮  ) at the top of the Connections column and select Refresh.