/
Creating Resources on AWS
Document toolboxDocument toolbox

Creating Resources on AWS

Owned by Sarabjeet Kaur
Last updated: Jan 09, 2025
6 min read

You can create Kyvos resources, such as Kyvos Manager, Query Engine, BI Server, and Kyvos Web Portal on AWS.

To create a resource on AWS, perform the following steps.

  1. Login to AWS console.

  2. In the search bar, enter EC2.

  3. On the EC2 Instances page, in the Resources section, click the Instances (running) option.

  4. Click Launch Instances. The Launch an Instance page is displayed.

    1. Enter the name of the instance/resource.

    2. Click Add Additional Tags link, and then click Add new tag. You must add two additional tags with the name as Jira and Owner.

    3. Now, navigate to the Application and OS Image (Amazon Machine Image) section and then select the required AMI from the Amazon Machine Image (AMI) list.

    4. In the Instance Type section, search and select the required instance type.

    5. Enter the Key Pair.

    6. In the Network Settings section, select the required VPC and Subnet as per your region.

    7. For the Firewall (Security Groups), either create a new security group or use an existing security group.

    8. Enable inbound security ports.

    9. Configure storage as needed. By default, the root volume is created. You can also create storage (data disk) as needed. For more details about configuring storage, see AWS documentation.

  5. In the Advanced Details section, select the required IAM role from the IAM Instance Profile list.

    1. Navigate further down to the User data section. Click Upload to select the Kyvos Manager, BI Server, or Query Engine script file. Note that the parameter for AWS_REGION in the script file will depend upon the region of deployment.

      • Kyvos Manager Script file.

        #!/bin/bash KYVOS_BASE_PATH="/data/kyvos" KYVOS_INSTALLATION_PATH="$KYVOS_BASE_PATH/app" KM_INSTALLATION_PATH="$KYVOS_BASE_PATH/installs" OUT_FILENAME="/tmp/bootstrap_logs.out" AWS_REGION="us-east-1" LOGIN_USER="ec2-user" adduser kyvos sudo su - kyvos -c "mkdir ~/.aws" echo sudo su - kyvos -c "echo '[default] region = $AWS_REGION' > ~/.aws/config" sudo su - kyvos -c "echo '[default] region = $AWS_REGION' > ~/.aws/config" ln -s /bin/python3 /bin/python checkLastCommand() { if [ $? -eq 0 ]; then echo "Command Executed" else printf "Error while executing command" >>$OUT_FILENAME exit 1 fi } echo sudo mkdir -p /home/kyvos/.ssh >>$OUT_FILENAME sudo mkdir -p /home/kyvos/.ssh >>$OUT_FILENAME 2>&1 checkLastCommand #For Centos echo sudo cp /home/$LOGIN_USER/.ssh/authorized_keys /home/kyvos/.ssh/ >>$OUT_FILENAME sudo cp /home/$LOGIN_USER/.ssh/authorized_keys /home/kyvos/.ssh/ >>$OUT_FILENAME 2>&1 checkLastCommand echo sudo chown -R kyvos:kyvos /home/kyvos/.ssh >>$OUT_FILENAME sudo chown -R kyvos:kyvos /home/kyvos/.ssh >>$OUT_FILENAME 2>&1 checkLastCommand echo sudo chmod 600 /home/kyvos/.ssh/authorized_keys >>$OUT_FILENAME sudo chmod 600 /home/kyvos/.ssh/authorized_keys >>$OUT_FILENAME 2>&1 checkLastCommand echo "kyvos soft nofile 10240 kyvos hard nofile 10240" >> /etc/security/limits.conf checkLastCommand echo "kyvos soft nofile 10240 kyvos hard nofile 10240" >> /etc/security/limits.d/20-nproc.conf checkLastCommand ALL_DISKS=`lsblk --noheadings --raw | grep 'disk' | awk '{print "/dev/"$1}'` ALL_DISKS_ARRAY=($ALL_DISKS) MOUNTED_DISKS=`df -hx tmpfs --output=source` MOUNTED_DISKS_ARRAY=($MOUNTED_DISKS) for disks1 in "${ALL_DISKS_ARRAY[@]}"; do for disks2 in "${MOUNTED_DISKS_ARRAY[@]}"; do [[ $disks1 == "$disks2" ]] && continue 2 done if [[ $disks1 != "/dev/xvda" && $disks1 != "/dev/sda" && $disks1 != "/dev/sdb" && $disks1 != "/dev/fd0" && $disks1 != "/dev/nvme0n1" ]];then UMOUNTED_DISKS+=( "$disks1" ) fi done if [ "${#UMOUNTED_DISKS[@]}" -ne 0 ]; then echo -e "Disks to be mounted:" >>$OUT_FILENAME printf '%s\n' "${UMOUNTED_DISKS[@]}" >>$OUT_FILENAME else echo "All disks are mounted" >>$OUT_FILENAME fi count=0 for disks in "${UMOUNTED_DISKS[@]}"; do echo -e "\nMounting disk:" $disks >>$OUT_FILENAME if [[ $disks == "/dev/xvdb" || $disks == "/dev/nvme1n1" ]]; then echo sudo mkfs.ext4 $disks >>$OUT_FILENAME yes | sudo mkfs.ext4 $disks 2>>$OUT_FILENAME checkLastCommand echo sudo mkdir -p /data/ >>$OUT_FILENAME sudo mkdir -p /data/ >>$OUT_FILENAME 2>&1 echo sudo mount $disks /data >>$OUT_FILENAME sudo mount $disks /data >>$OUT_FILENAME 2>&1 checkLastCommand uuid=`sudo blkid -s UUID -o value $disks` checkLastCommand echo "Disk uuid:" $uuid >>$OUT_FILENAME sudo echo "UUID=$uuid /data ext4 defaults 0 0" >>$OUT_FILENAME sudo echo "UUID=$uuid /data ext4 defaults 0 0" >> /etc/fstab checkLastCommand echo sudo mkdir -p $KYVOS_INSTALLATION_PATH >>$OUT_FILENAME sudo mkdir -p $KYVOS_INSTALLATION_PATH >>$OUT_FILENAME 2>&1 checkLastCommand echo sudo mkdir -p $KM_INSTALLATION_PATH >>$OUT_FILENAME sudo mkdir -p $KM_INSTALLATION_PATH >>$OUT_FILENAME checkLastCommand echo sudo chown -R kyvos:kyvos /data/kyvos/ >>$OUT_FILENAME sudo chown -R kyvos:kyvos /data/kyvos/ >>$OUT_FILENAME 2>&1 checkLastCommand else count=$((count+1)) echo mkdir /data$count >>$OUT_FILENAME >>$OUT_FILENAME mkdir /data$count >>$OUT_FILENAME checkLastCommand echo sudo mkfs.ext4 $disks >>$OUT_FILENAME yes | sudo mkfs.ext4 $disks 2>>$OUT_FILENAME checkLastCommand echo sudo mount $disks /data$count >>$OUT_FILENAME sudo mount $disks /data$count >>$OUT_FILENAME checkLastCommand uuid=`sudo blkid -s UUID -o value $disks` checkLastCommand echo "Disk uuid:" $uuid >>$OUT_FILENAME sudo echo "UUID=$uuid /data$count ext4 defaults 0 0" >>$OUT_FILENAME sudo echo "UUID=$uuid /data$count ext4 defaults 0 0" >> /etc/fstab checkLastCommand echo sudo mkdir -p /data$count/kyvos/app/cube_local_cache >>$OUT_FILENAME sudo mkdir -p /data$count/kyvos/app/cube_local_cache >>$OUT_FILENAME 2>&1 checkLastCommand echo sudo chmod -R 777 /data$count/kyvos/app/cube_local_cache >>$OUT_FILENAME sudo chmod -R 777 /data$count/kyvos/app/cube_local_cache >>$OUT_FILENAME 2>&1 checkLastCommand fi done

      • Query Engine or BI Server Script File.

        #!/bin/bash KYVOS_INSTALLATION_PATH="/data/kyvos/app" OUT_FILENAME="/tmp/bootstrap_logs.out" AWS_REGION="us-east-1" LOGIN_USER="ec2-user" adduser kyvos sudo su - kyvos -c "mkdir ~/.aws" echo sudo su - kyvos -c "echo '[default] region = $AWS_REGION' > ~/.aws/config" sudo su - kyvos -c "echo '[default] region = $AWS_REGION' > ~/.aws/config" ln -s /bin/python3 /bin/python checkLastCommand() { if [ $? -eq 0 ]; then echo "Command Executed" else printf "\nError while executing command" >>$OUT_FILENAME exit 1 fi } echo "Script Started at :" `date` >$OUT_FILENAME echo sudo mkdir -p /home/kyvos/.ssh >>$OUT_FILENAME sudo mkdir -p /home/kyvos/.ssh >>$OUT_FILENAME 2>&1 checkLastCommand echo sudo cp /home/$LOGIN_USER/.ssh/authorized_keys /home/kyvos/.ssh/ >>$OUT_FILENAME sudo cp /home/$LOGIN_USER/.ssh/authorized_keys /home/kyvos/.ssh/ >>$OUT_FILENAME 2>&1 checkLastCommand echo sudo chown -R kyvos:kyvos /home/kyvos/.ssh >>$OUT_FILENAME sudo chown -R kyvos:kyvos /home/kyvos/.ssh >>$OUT_FILENAME 2>&1 checkLastCommand echo sudo chmod 600 /home/kyvos/.ssh/authorized_keys >>$OUT_FILENAME sudo chmod 600 /home/kyvos/.ssh/authorized_keys >>$OUT_FILENAME 2>&1 checkLastCommand echo "kyvos soft nofile 10240 kyvos hard nofile 10240" >> /etc/security/limits.conf checkLastCommand echo "kyvos soft nofile 10240 kyvos hard nofile 10240" >> /etc/security/limits.d/20-nproc.conf checkLastCommand dataVolumeMount() { echo sudo mkfs.ext4 $disks >>$OUT_FILENAME yes | sudo mkfs.ext4 $disks 2>>$OUT_FILENAME checkLastCommand echo sudo mkdir -p /data/ >>$OUT_FILENAME sudo mkdir -p /data/ >>$OUT_FILENAME 2>&1 echo sudo mount $disks /data >>$OUT_FILENAME sudo mount $disks /data >>$OUT_FILENAME 2>&1 checkLastCommand uuid=`sudo blkid -s UUID -o value $disks` checkLastCommand echo "Disk uuid:" $uuid >>$OUT_FILENAME sudo echo "UUID=$uuid /data ext4 defaults 0 0" >>$OUT_FILENAME sudo echo "UUID=$uuid /data ext4 defaults 0 0" >> /etc/fstab checkLastCommand echo sudo mkdir -p $KYVOS_INSTALLATION_PATH >>$OUT_FILENAME sudo mkdir -p $KYVOS_INSTALLATION_PATH >>$OUT_FILENAME 2>&1 checkLastCommand echo sudo chown -R kyvos:kyvos /data/kyvos/ >>$OUT_FILENAME sudo chown -R kyvos:kyvos /data/kyvos/ >>$OUT_FILENAME 2>&1 checkLastCommand } metadataDiskMount() { count=$((count+1)) echo mkdir /data$count >>$OUT_FILENAME >>$OUT_FILENAME mkdir /data$count >>$OUT_FILENAME checkLastCommand echo sudo mkfs.ext4 $disks >>$OUT_FILENAME yes | sudo mkfs.ext4 $disks 2>>$OUT_FILENAME checkLastCommand echo sudo mount $disks /data$count >>$OUT_FILENAME sudo mount $disks /data$count >>$OUT_FILENAME checkLastCommand uuid=`sudo blkid -s UUID -o value $disks` checkLastCommand echo "Disk uuid:" $uuid >>$OUT_FILENAME sudo echo "UUID=$uuid /data$count ext4 defaults 0 0" >>$OUT_FILENAME sudo echo "UUID=$uuid /data$count ext4 defaults 0 0" >> /etc/fstab checkLastCommand echo sudo mkdir -p /data$count/kyvos/app/cube_local_cache >>$OUT_FILENAME sudo mkdir -p /data$count/kyvos/app/cube_local_cache >>$OUT_FILENAME 2>&1 checkLastCommand echo sudo chmod -R 755 /data$count/kyvos/app/cube_local_cache >>$OUT_FILENAME sudo chmod -R 755 /data$count/kyvos/app/cube_local_cache >>$OUT_FILENAME 2>&1 checkLastCommand } ALL_DISKS=`lsblk --noheadings --raw | grep 'disk' | awk '{print "/dev/"$1}'` ALL_DISKS_ARRAY=($ALL_DISKS) MOUNTED_DISKS=`df -hx tmpfs --output=source` MOUNTED_DISKS_ARRAY=($MOUNTED_DISKS) for disks1 in "${ALL_DISKS_ARRAY[@]}"; do for disks2 in "${MOUNTED_DISKS_ARRAY[@]}"; do [[ $disks1 == "$disks2" ]] && continue 2 done if [[ $disks1 != "/dev/xvda" && $disks1 != "/dev/sda" && $disks1 != "/dev/sdb" && $disks1 != "/dev/fd0" && $disks1 != "/dev/nvme0n1" ]];then UMOUNTED_DISKS+=( "$disks1" ) fi done if [ "${#UMOUNTED_DISKS[@]}" -ne 0 ]; then echo -e "Disks to be mounted:" >>$OUT_FILENAME printf '%s ' "${UMOUNTED_DISKS[@]}" >>$OUT_FILENAME else echo "All disks are mounted" >>$OUT_FILENAME fi count=0 for disks in "${UMOUNTED_DISKS[@]}"; do echo -e "Mounting disk:" $disks >>$OUT_FILENAME if [[ $disks == /dev/nvm* ]]; then echo "The instance is Nitro based instance" >>$OUT_FILENAME Original_device_name=`sudo /sbin/ebsnvme-id $disks | tail -1` Volume_id=`sudo /sbin/ebsnvme-id $disks | head -1` echo "Disk:" $disks "with Volume id:" $Volume_id "is originally mounted on:" $Original_device_name >>$OUT_FILENAME if [[ $Original_device_name =~ .*sdb ]]; then dataVolumeMount else metadataDiskMount fi else echo "The instance is Non-Nitro based instance" >>$OUT_FILENAME if [[ $disks == "/dev/xvdb" ]]; then dataVolumeMount else metadataDiskMount fi fi done echo "Script Completed :" `date` >>$OUT_FILENAM

  6. Click Launch Instance.

  7. After launching the Kyvos Manager node, run the following scripts as root user:

    KyvosBundleName=`sudo su - kyvos -c "export AWS_ACCESS_KEY_ID=<Enter your Access Key>;export AWS_SECRET_ACCESS_KEY=<Enter your secret key>;aws s3 ls s3://us-east-1.kyvos/2024.12/latest/ | sort | grep ".tar.gz" | tail -1"` KyvosBundleName=`echo $KyvosBundleName | awk '{print $4}'` sudo su - kyvos -c "export AWS_ACCESS_KEY_ID=<Enter your Access Key>;export AWS_SECRET_ACCESS_KEY=<Enter your secret key>;aws s3 cp s3://us-east-1.kyvos/2024.12/latest/$KyvosBundleName /data/kyvos/installs"

  8. Configure the following ports on the Kyvos Manager, BI Server and Query Engine nodes.

    image-20250108-114623.png

Related content

Prerequisites for Wizard-based Deployment on AWS with Kyvos Native
Prerequisites for Wizard-based Deployment on AWS with Kyvos Native
Kyvos 2024.12
Read with this
Creating Resources on AWS
Creating Resources on AWS
Kyvos 2025.1
More like this
Prerequisites for Manually Creating Kyvos Resources for GCP
Prerequisites for Manually Creating Kyvos Resources for GCP
Kyvos 2024.12
Read with this
Creating Kyvos resources for AWS automated deployment with EMR
Creating Kyvos resources for AWS automated deployment with EMR
Kyvos 2025.1
More like this
Creating GCP Instances for use with Kyvos
Creating GCP Instances for use with Kyvos
Kyvos 2024.12
Read with this
Creating Kyvos resources for AWS automated deployment with EMR
Creating Kyvos resources for AWS automated deployment with EMR
Kyvos 2023.3
More like this

Copyright Kyvos, Inc. All rights reserved.