Versions Compared

Version Old Version 1 New Version Current
Changes made by

Sarabjeet Kaur

Sarabjeet Kaur

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. Login to AWS console.

  2. In the search bar, enter EC2.

  3. On the EC2 Instances page, in the Resources section, click the Instances (running) option.

  4. Click Launch Instances. The Launch an Instance page is displayed.

    1. Enter the name of the instance/resource.

    2. Click Add Additional Tags link, and then click Add new tag. You must add two additional tags with the name as Jira and Owner.

    3. Now, navigate to the Application and OS Image (Amazon Machine Image) section and then select the required AMI from the Amazon Machine Image (AMI) list.

    4. In the Instance Type section, search and select the required instance type.

    5. Enter the Key Pair.

    6. In the Network Settings section, select the required VPC and Subnet as per your region.

    7. For the Firewall (Security Groups), either create a new security group or use an existing security group.

    8. Enable inbound security ports.

    9. Configure storage as needed. By default, the root volume is created. You can also create storage (data disk) as needed. For more details about configuring storage, see AWS documentation.

  5. In the Advanced Details section, select the required IAM role from the IAM Instance Profile list.

    1. Navigate further down to the User data section. Click Upload to select the Kyvos Manager, BI Server, or Query Engine script file. Note that the parameter for AWS_REGION in the script file will depend upon the region of deployment.

      • Kyvos Manager Script file.

        Code Block
        #!/bin/bash

KYVOS_BASE_PATH="/data/kyvos"
KYVOS_INSTALLATION_PATH="$KYVOS_BASE_PATH/app"
KM_INSTALLATION_PATH="$KYVOS_BASE_PATH/installs"
OUT_FILENAME="/tmp/bootstrap_logs.out"

AWS_REGION="us-east-1" 

LOGIN_USER="ec2-user" 

adduser kyvos 

sudo su - kyvos -c "mkdir ~/.aws" 

echo sudo su - kyvos -c "echo '[default] 
region = $AWS_REGION' > ~/.aws/config"

sudo su - kyvos -c "echo '[default] 
region = $AWS_REGION' > ~/.aws/config" 

ln -s /bin/python3 /bin/python

checkLastCommand() {
if [ $? -eq 0 ]; then
echo "Command Executed"
else
printf "Error while executing command" >>$OUT_FILENAME
exit 1
fi
}

echo sudo mkdir -p /home/kyvos/.ssh >>$OUT_FILENAME 
sudo mkdir -p /home/kyvos/.ssh >>$OUT_FILENAME 2>&1 
checkLastCommand 

#For Centos 
echo sudo cp /home/$LOGIN_USER/.ssh/authorized_keys /home/kyvos/.ssh/ >>$OUT_FILENAME 
sudo cp /home/$LOGIN_USER/.ssh/authorized_keys /home/kyvos/.ssh/ >>$OUT_FILENAME 2>&1 
checkLastCommand 

echo sudo chown -R kyvos:kyvos /home/kyvos/.ssh >>$OUT_FILENAME 
sudo chown -R kyvos:kyvos /home/kyvos/.ssh >>$OUT_FILENAME 2>&1 
checkLastCommand 

echo sudo chmod 600 /home/kyvos/.ssh/authorized_keys >>$OUT_FILENAME 
sudo chmod 600 /home/kyvos/.ssh/authorized_keys >>$OUT_FILENAME 2>&1 
checkLastCommand 

echo "kyvos soft nofile 10240
kyvos hard nofile 10240" >> /etc/security/limits.conf 
checkLastCommand 

echo "kyvos soft nofile 10240 
kyvos hard nofile 10240" >> /etc/security/limits.d/20-nproc.conf 
checkLastCommand 

ALL_DISKS=`lsblk --noheadings --raw | grep 'disk' | awk '{print "/dev/"$1}'` 
ALL_DISKS_ARRAY=($ALL_DISKS) 
MOUNTED_DISKS=`df -hx tmpfs --output=source` 
MOUNTED_DISKS_ARRAY=($MOUNTED_DISKS) 

for disks1 in "${ALL_DISKS_ARRAY[@]}"; do 
for disks2 in "${MOUNTED_DISKS_ARRAY[@]}"; do [[ $disks1 == "$disks2" ]] && continue 2 
done 

if [[ $disks1 != "/dev/xvda" && $disks1 != "/dev/sda" && $disks1 != "/dev/sdb" && $disks1 != "/dev/fd0" && $disks1 != "/dev/nvme0n1" ]];then 
UMOUNTED_DISKS+=( "$disks1" ) 
fi 
done 

if [ "${#UMOUNTED_DISKS[@]}" -ne 0 ]; then 
echo -e "Disks to be mounted:" >>$OUT_FILENAME 
printf '%s\n' "${UMOUNTED_DISKS[@]}" >>$OUT_FILENAME 
else 
echo "All disks are mounted" >>$OUT_FILENAME 
fi 

count=0 

for disks in "${UMOUNTED_DISKS[@]}"; do 
echo -e "\nMounting disk:" $disks >>$OUT_FILENAME 

if [[ $disks == "/dev/xvdb" || $disks == "/dev/nvme1n1" ]]; then 
echo sudo mkfs.ext4 $disks >>$OUT_FILENAME 
yes | sudo mkfs.ext4 $disks 2>>$OUT_FILENAME 
checkLastCommand 

echo sudo mkdir -p /data/ >>$OUT_FILENAME 
sudo mkdir -p /data/ >>$OUT_FILENAME 2>&1 

echo sudo mount $disks /data >>$OUT_FILENAME 
sudo mount $disks /data >>$OUT_FILENAME 2>&1 
checkLastCommand 

uuid=`sudo blkid -s UUID -o value $disks` 
checkLastCommand 

echo "Disk uuid:" $uuid >>$OUT_FILENAME 

sudo echo "UUID=$uuid /data ext4 defaults 0 0" >>$OUT_FILENAME 
sudo echo "UUID=$uuid /data ext4 defaults 0 0" >> /etc/fstab 
checkLastCommand 

echo sudo mkdir -p $KYVOS_INSTALLATION_PATH >>$OUT_FILENAME 
sudo mkdir -p $KYVOS_INSTALLATION_PATH >>$OUT_FILENAME 2>&1 
checkLastCommand 

echo sudo mkdir -p $KM_INSTALLATION_PATH >>$OUT_FILENAME 
sudo mkdir -p $KM_INSTALLATION_PATH >>$OUT_FILENAME 
checkLastCommand 

echo sudo chown -R kyvos:kyvos /data/kyvos/ >>$OUT_FILENAME 
sudo chown -R kyvos:kyvos /data/kyvos/ >>$OUT_FILENAME 2>&1 
checkLastCommand 

else 
count=$((count+1)) 
echo mkdir /data$count >>$OUT_FILENAME >>$OUT_FILENAME 
mkdir /data$count >>$OUT_FILENAME 
checkLastCommand 

echo sudo mkfs.ext4 $disks >>$OUT_FILENAME 
yes | sudo mkfs.ext4 $disks 2>>$OUT_FILENAME 
checkLastCommand 

echo sudo mount $disks /data$count >>$OUT_FILENAME 
sudo mount $disks /data$count >>$OUT_FILENAME 
checkLastCommand 

uuid=`sudo blkid -s UUID -o value $disks` 
checkLastCommand 
echo "Disk uuid:" $uuid >>$OUT_FILENAME 

sudo echo "UUID=$uuid /data$count ext4 defaults 0 0" >>$OUT_FILENAME 
sudo echo "UUID=$uuid /data$count ext4 defaults 0 0" >> /etc/fstab 
checkLastCommand 

echo sudo mkdir -p /data$count/kyvos/app/cube_local_cache >>$OUT_FILENAME 
sudo mkdir -p /data$count/kyvos/app/cube_local_cache >>$OUT_FILENAME 2>&1 
checkLastCommand 

echo sudo chmod -R 777 /data$count/kyvos/app/cube_local_cache >>$OUT_FILENAME 
sudo chmod -R 777 /data$count/kyvos/app/cube_local_cache >>$OUT_FILENAME 2>&1 
checkLastCommand 
fi 
done

      • Query Engine or BI Server Script File.

        Code Block
        #!/bin/bash

KYVOS_INSTALLATION_PATH="/data/kyvos/app" 
OUT_FILENAME="/tmp/bootstrap_logs.out" 
AWS_REGION="us-east-1" 
LOGIN_USER="ec2-user" 

adduser kyvos 

sudo su - kyvos -c "mkdir ~/.aws" 
echo sudo su - kyvos -c "echo '[default] region = $AWS_REGION' > ~/.aws/config" 
sudo su - kyvos -c "echo '[default] 
region = $AWS_REGION' > ~/.aws/config" 

ln -s /bin/python3 /bin/python

checkLastCommand() { 
if [ $? -eq 0 ]; then 
echo "Command Executed" 
else 
printf "\nError while executing command" >>$OUT_FILENAME 
exit 1 
fi 
} 

echo "Script Started at :" `date` >$OUT_FILENAME 

echo sudo mkdir -p /home/kyvos/.ssh >>$OUT_FILENAME 
sudo mkdir -p /home/kyvos/.ssh >>$OUT_FILENAME 2>&1 
checkLastCommand

echo sudo cp /home/$LOGIN_USER/.ssh/authorized_keys /home/kyvos/.ssh/ >>$OUT_FILENAME 
sudo cp /home/$LOGIN_USER/.ssh/authorized_keys /home/kyvos/.ssh/ >>$OUT_FILENAME 2>&1 
checkLastCommand 

echo sudo chown -R kyvos:kyvos /home/kyvos/.ssh >>$OUT_FILENAME 
sudo chown -R kyvos:kyvos /home/kyvos/.ssh >>$OUT_FILENAME 2>&1 
checkLastCommand 

echo sudo chmod 600 /home/kyvos/.ssh/authorized_keys >>$OUT_FILENAME 
sudo chmod 600 /home/kyvos/.ssh/authorized_keys >>$OUT_FILENAME 2>&1 
checkLastCommand 

echo "kyvos soft nofile 10240
kyvos hard nofile 10240" >> /etc/security/limits.conf 
checkLastCommand 

echo "kyvos soft nofile 10240 
kyvos hard nofile 10240" >> /etc/security/limits.d/20-nproc.conf 
checkLastCommand 

dataVolumeMount() { 
echo sudo mkfs.ext4 $disks >>$OUT_FILENAME 
yes | sudo mkfs.ext4 $disks 2>>$OUT_FILENAME 
checkLastCommand 

echo sudo mkdir -p /data/ >>$OUT_FILENAME 
sudo mkdir -p /data/ >>$OUT_FILENAME 2>&1 

echo sudo mount $disks /data >>$OUT_FILENAME 
sudo mount $disks /data >>$OUT_FILENAME 2>&1 
checkLastCommand 

uuid=`sudo blkid -s UUID -o value $disks` 
checkLastCommand 
echo "Disk uuid:" $uuid >>$OUT_FILENAME 

sudo echo "UUID=$uuid /data ext4 defaults 0 0" >>$OUT_FILENAME 
sudo echo "UUID=$uuid /data ext4 defaults 0 0" >> /etc/fstab 
checkLastCommand 

echo sudo mkdir -p $KYVOS_INSTALLATION_PATH >>$OUT_FILENAME 
sudo mkdir -p $KYVOS_INSTALLATION_PATH >>$OUT_FILENAME 2>&1 
checkLastCommand 

echo sudo chown -R kyvos:kyvos /data/kyvos/ >>$OUT_FILENAME 
sudo chown -R kyvos:kyvos /data/kyvos/ >>$OUT_FILENAME 2>&1 
checkLastCommand 

} 

metadataDiskMount() { 
count=$((count+1)) 

echo mkdir /data$count >>$OUT_FILENAME >>$OUT_FILENAME 
mkdir /data$count >>$OUT_FILENAME 
checkLastCommand 

echo sudo mkfs.ext4 $disks >>$OUT_FILENAME 
yes | sudo mkfs.ext4 $disks 2>>$OUT_FILENAME 
checkLastCommand 

echo sudo mount $disks /data$count >>$OUT_FILENAME 
sudo mount $disks /data$count >>$OUT_FILENAME 
checkLastCommand 

uuid=`sudo blkid -s UUID -o value $disks` 
checkLastCommand 
echo "Disk uuid:" $uuid >>$OUT_FILENAME 

sudo echo "UUID=$uuid /data$count ext4 defaults 0 0" >>$OUT_FILENAME 
sudo echo "UUID=$uuid /data$count ext4 defaults 0 0" >> /etc/fstab 
checkLastCommand 

echo sudo mkdir -p /data$count/kyvos/app/cube_local_cache >>$OUT_FILENAME 
sudo mkdir -p /data$count/kyvos/app/cube_local_cache >>$OUT_FILENAME 2>&1 
checkLastCommand 

echo sudo chmod -R 755 /data$count/kyvos/app/cube_local_cache >>$OUT_FILENAME 
sudo chmod -R 755 /data$count/kyvos/app/cube_local_cache >>$OUT_FILENAME 2>&1 
checkLastCommand 

} 

ALL_DISKS=`lsblk --noheadings --raw | grep 'disk' | awk '{print "/dev/"$1}'` 
ALL_DISKS_ARRAY=($ALL_DISKS) 
MOUNTED_DISKS=`df -hx tmpfs --output=source` 
MOUNTED_DISKS_ARRAY=($MOUNTED_DISKS) 

for disks1 in "${ALL_DISKS_ARRAY[@]}"; do 
for disks2 in "${MOUNTED_DISKS_ARRAY[@]}"; do [[ $disks1 == "$disks2" ]] && continue 2 
done 
if [[ $disks1 != "/dev/xvda" && $disks1 != "/dev/sda" && $disks1 != "/dev/sdb" && $disks1 != "/dev/fd0" && $disks1 != "/dev/nvme0n1" ]];then 
UMOUNTED_DISKS+=( "$disks1" ) 
fi 
done 

if [ "${#UMOUNTED_DISKS[@]}" -ne 0 ]; then 
echo -e "Disks to be mounted:" >>$OUT_FILENAME 
printf '%s 
' "${UMOUNTED_DISKS[@]}" >>$OUT_FILENAME 

else
echo "All disks are mounted" >>$OUT_FILENAME 
fi 

count=0 

for disks in "${UMOUNTED_DISKS[@]}"; do 
echo -e "Mounting disk:" $disks >>$OUT_FILENAME 

if [[ $disks == /dev/nvm* ]]; then 
echo "The instance is Nitro based instance" >>$OUT_FILENAME 
Original_device_name=`sudo /sbin/ebsnvme-id $disks | tail -1` 
Volume_id=`sudo /sbin/ebsnvme-id $disks | head -1` 
echo "Disk:" $disks "with Volume id:" $Volume_id "is originally mounted on:" $Original_device_name >>$OUT_FILENAME 

if [[ $Original_device_name =~ .*sdb ]]; then 
dataVolumeMount 
else 
metadataDiskMount 
fi 
else 
echo "The instance is Non-Nitro based instance" >>$OUT_FILENAME

if [[ $disks == "/dev/xvdb" ]]; then 
dataVolumeMount 
else 
metadataDiskMount 
fi 
fi 
done 
echo "Script Completed :" `date` >>$OUT_FILENAM

  6. Click Launch Instance.

  7. After launching the Kyvos Manager node, run the following scripts as root user:

    Code Block
    KyvosBundleName=`sudo su - kyvos  -c "export AWS_ACCESS_KEY_ID=<Enter your Access Key>;export AWS_SECRET_ACCESS_KEY=<Enter your secret key>;aws s3 ls s3://us-east-1.kyvos/2024.12/latest/ | sort | grep ".tar.gz" | tail -1"`
KyvosBundleName=`echo $KyvosBundleName |  awk '{print $4}'`
sudo su - kyvos  -c "export AWS_ACCESS_KEY_ID=<Enter your Access Key>;export AWS_SECRET_ACCESS_KEY=<Enter your secret key>;aws s3 cp s3://us-east-1.kyvos/2024.12/latest/$KyvosBundleName /data/kyvos/installs"

  8. Configure the following ports on the Kyvos Manager, BI Server and Query Engine nodes.

    image-20250108-114623.png