Cross-Account Glue access

Applies to:Kyvos Enterprise  Kyvos Cloud (Managed Services on AWS)  Kyvos Azure Marketplace

Kyvos AWS Marketplace  Kyvos Single Node Installation (Kyvos SNI)  Kyvos Free (Limited offering for AWS)

To enable cross-account Glue access in AWS environment, you need to provide certain roles and permissions. 

Configuration

1. hive.metastore.glue.catalogid this property will be given by the user at the time of deployment in the CF template.
2. Using this KM will copy another property spark.hadoop.hive.metastore.glue.catalogid in the DefaultHadoopCluster.xml file.
3. Kyvos will use this property to further fetch the metadata from the customer’s Glue.

List of roles and permissions

You can use any of the following AWS Glue methods for granting cross-account access to a resource:

1. Use a Data Catalog resource policy
2. Use an IAM role

Currently, Kyvos supports access by using a Data Catalog resource policy

To grant cross-account access using a Data Catalog resource policy, perform the following steps.

1. An administrator (or other authorized identity) in Account A (e.g. customer’s account) attaches a resource policy to the Data Catalog in Account A. This policy grants specific cross-account permissions to Account B (account where the Kyvos application is deployed) for performing operations on a resource in Account A's catalog.
2. An administrator in Account B attaches an IAM policy to a user or other IAM identity in Account B that delegates the permissions received from Account A.
   The user or other identity in Account B now has access to the specified resource in Account A.

For example, to give IAM Role created for Kyvos service in Account B access to database db1 in Customer Account (Account A), attach the following resource policy to the catalog in Account A.

1. To give role cloudformation_ec2_role created in Kyvos service in MS Account (Account B) access to database db1 in Customer Account(Account A), attach the following resource policy to the Data Catalog Settings on the AWS Glue Console in Customer Account (Account A).

   {
   "Version" : "2012-10-17",
   "Statement" : [ {
   "Effect" : "Allow",
   "Principal" : {
   "AWS" : "arn:aws:iam:: Account_B:role/cloudformation_ec2_role"
   },
   "Action" : "glue:Get*",
   "Resource" : [ "arn:aws:glue:us-east-1: Account_A:database/db1,
   "arn:aws:glue:us-east-1: Account_A:table/db1/*" ]
   } ]
   }

2. In addition, Account B would have to attach the following IAM policy to cloudformation_ec2_role role prior to getting access to db1 in Customer Account (Account A).

   {
   "Version": "2012-10-17",
   "Statement": [
   {
   "Effect": "Allow",
   "Action": [
   "glue:Get*"
   ],
   "Resource" :
   [ "arn:aws:glue:us-east-1:Account_A:database/default",
   "arn:aws:glue:us-east-1:Account_A:database/global_temp",
   "arn:aws:glue:us-east-1:Account_A:catalog",
   "arn:aws:glue:us-east-1:Account_A:database/db1",
   "arn:aws:glue:us-east-1:Account_A:table/db1/*" ]
   }
   ]
   }

3. Add a bucket policy for the destination bucket (Account_A_Bucket) in Customer Account(Account A) from which we can grant access to the MS Account (Account B).

   {
   "Sid": "AddCannedAcl",
   "Effect": "Allow",
   "Principal": {
   "AWS": "arn:aws:iam:: Account_B:role/cloudformation_ec2_role"
   },
   "Action": [
   "s3:GetObject", "s3:GetObjectAcl", "s3:ListBucket"
   ],
   "Resource": [
   "arn:aws:s3::: Account_A_Bucket ",
   "arn:aws:s3::: Account_A_Bucket/*"
   ]