Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

As a part of the constant endeavor to enhance the ease of working with Kyvos, the Continuous Integration and Continuous Deployment (CI/CD) Utility is created to automate the versioning of Kyvos entities like Semantic Models, Relationships, Datasets, etc., using Git as a versioning tool. It can be used to migrate entities from one Kyvos deployment (Source) to another Kyvos deployment (Target).

...

Parameter

Description

EXPORT_SEMANTIC_MODELS 

Defines whether to export semantic model entities or not.  

  • Y to allow 

  • N to deny/refrain 

SEMANTIC_MODEL_DETAILS 

Provides details of the semantic model that needs to be exported in the following format:

"FolderName":["SemanticModelName"], 

“FolderName” should be the name of the Kyvos entity folder name to be exported. 

"SemanticModelName": Comma separated entity names existing within the folder 

Example 

{"Folder1": [ “Model1”,” Model2”], “Folder2”: [“Model3”]} 

EXPORT_DATASET_RELATIONSHIPS 

 

Defines whether to export Dataset Relationship entities or not.  

  • Y to allow 

  • N to deny/refrain 

DATASET_RELATIONSHIP_DETAILS 

 

Provides details of the Dataset Relationship that must be exported in the following format. 

"FolderName":["DatasetRelationshipName"], 

“FolderName” should be the name of the Kyvos entity folder name to be exported. 

"DatasetRelationshipName": Comma separated entity names existing within the folder 

For example, 

{"Folder1": [ “DRelationship1”,” DRelationship2”], “Folder2”: [“DRelationship3”]} 

EXPORT_DATASETS 

Defines whether to export Dataset entities or not.  

  • Y to allow 

  • N to deny/refrain 

DATASET_DETAILS 

Provides details of the Dataset that must be exported in the following format. 

"FolderName": ["DatasetName"], 

“FolderName” should be the name of the Kyvos entity folder name to be exported. 

"DatasetName": Comma separated entity names existing within the folder 

Example 

{"Folder1": [ “Dataset 1”,” Dataset 2”], “Folder2”: [“Dataset 3”]} 

EXPORT_WB 

Defines whether to export Workbooks or not.  

  • Y to allow 

  • N to deny/refrain 

WB_DETAILS 

Provides details of the Workbooks that must be exported in the following format. 

"FolderName": ["Workbook"], 

“FolderName” should be the name of the Kyvos entity folder name to be exported. 

"Workbook": Comma separated entity names existing within the folder 

For example, 

{"Folder1": [ “workbook1”,” workbook2”], “Folder2”: [“workbook3”]} 

GIT_PUSH 

Defines whether to push the exported entities into GitLab or not.  

  • Y to push in GitLab as well as in GIT_LOCAL_REPO_PATH both. 

  • N to not push in Gitlab, but just only in the output folder. 

GIT_URL 

Server URL

GIT_LOCAL_REPO_PATH 

Provides a folder path to be used as a staging location. 

For example, “C:/CICDUtility/Export/GITRepo” 

NOTE: Use the forward-slash (/) in the repo path. 

GIT_FEATURE_BRANCH_NAME 

Specifies the GitLab feature branch name where entities will get exported.  

GIT_FEATURE_BRANCH_CLEAN 

Defines whether to clean the GIT Feature branch before a fresh export. 

The default value is N

  • Y to clean 

  • N to do nothing 

GIT_REPONAME 

Specifies the GitLab repository name where the entities will get exported. 

GIT_USERNAME 

Specifies the GitLab user name to connect with GIT Repo. 

GIT_TOKEN 

The Personal Access Token for authentication with the user’s GIT account. 

If you do not have the Personal Access token, you can create it using the steps explained in the Appendix

GIT_URL

Git url to be mentioned

Panel
panelIconIdatlassian-note
panelIcon:note:
bgColor#DEEBFF

Note

To export an entity from the root folder, users must specify the folder name as ‘ROOT’ in the input.json file.

...

Example configurations for different scenarios

...

  1. If the GITPUSH parameter is set as Y, all the entity XML files will be exported to the GitLab Feature branch in the following order.

    1. A GIT PULL is performed from the feature branch specified in the input file to sync the remote and local repository (GITRepo).

    2. The entities are exported to the GITRepo folder from Kyvos, a local repository clone of the GitLab repository.

    3. Entities are then pushed to the GitLab feature branch.

  2. The entities are directly exported to the Output folder if the GITPUSH parameter is set as N in the input file. No GIT PULL and GIT PUSH will be performed in this case.

Import Utility

Panel
panelIconIdatlassian-note
panelIcon:note:
bgColor#DEEBFF

Note

Upon the initial setup configuration, users are required to execute the following commands:

git config --global user.email "email"
git config --global user.name "username"

Import Utility

The import utility works in two steps.

...

If the Jenkins pipeline is not configured, Step 2 can still launch ImportKyvosEntities.sh independently.

Prerequisites

  • Git Bash: It is a package that installs Bash, some common bash utilities, and Git on an operating system. This is needed to execute the Git commands on your system.

  • GitLab: Git server with main and feature branches.

  • Jenkins: For automated execution of the Import utility script, based on a trigger from GitLab.

...

  1. Extract the setup zip. 

  2. Configure source target Kyvos deployment details in the serverDetailImportUtility.properties file in the Conf folder. This file needs the details below to be filled in. If the details are not provided in this file, then alternatively, these details can be provided as an argument to the import utility script.

    • KYVOS_URL = <YourKyvosURL>/rest/

    • USERNAME = <KyvosUserName>

    • PASSWORD = <KyvosUserPassword>= <KyvosUserPassword>

    • KyvosEntitiesPath=<Path of KyvosEntities store in local machine>
      NOTE: It’s value should represent the directory path leading up to the KyvosEntities folder.
      For example:
      /data/Automation_Data/mayank/CICD/CICD_Utility_2023.5/EntitiesInMultipleFolders/CICD_Testing/KyvosEntities

  3. Execute the chmod +x ImportKyvosEntities.sh permission on the ImportKyvosEntities.sh file, if not already provided.

...

  • KyvosUsername is the name of the native user with admin access to the target Kyvos deployment.

  • KyvosPassword is the password of the native user with admin access to the target Kyvos deployment.

  • KyvosURL is the URL of the target Kyvos deployment.

...

  • image (32).pngImage Added
Code Block
pipeline {
    agent any
    stages {
        stage ('Environment Details') {
            steps {
                sh '''
                env | grep -e PATH -e JAVA_HOME
                which java
                java -version
                mvn –version"” }}
             stage('Checkout') {
            steps {
                checkout (
                    [$class: 'GitSCM',
                    branches: [[name: '*/your_branch_name']],
                    extensions: [], userRemoteConfigs:
                    [[credentialsId: 'your_credential_id',
                    url: 'your_git_url']]]   ) } }  
        stage('Build') {
            steps {
                script{
                 
                    dir('jenkinsWorkSpacePath/cicdUtilityPath') {
                        sh "pwd"
                sh " jenkinsWorkSpacePath /cicdUtilityPath/ImportKyvosEntities.sh ${kyvosEntitiesPath} ${KyvosUsername} ${kyvosPassword} ${KyvosURL}"     }  } } } } }

...

  1. Login to your Jenkins account.

  2. Install the following plugins:

    1. Git client plugin

    2. GitHub API

    3. GitHub Branch Source Plugin

    4. GitHub plugin

    5. Genric Webhook plugin

  3. If the plugins are not installed while initializing Jenkins, go to Manage Jenkins >Manage Plugins and install the plugins.

  4. Click New Item from the top left of your Dashboard.

  5. Enter the name of the item you want to create, select Pipeline,and click Ok. The project is created.

    image-20240201-115827.png
  6. Navigate to the newly created project and select the Configure option, as shown in the following figure.

    image-20240201-115911.png
  7. On the General tab, select the GitHub Project check box and provide the URL of the GitHub/GitLab repository in the Project url field, as shown in the following figure.

    image-20240201-115953.png
  8. On the Build Triggers tab, select Generic Webhook Trigger. This triggers a Jenkins job when the GitLab webhook relays any payload. If you are using a generic webhook, you can select the generic webhook trigger option.

    image-20240212-170041.png
  9. Provide Jenkins credential ID in the Token section or generate a token ID in the Token Credential section by clicking the Add button and provide credentials.

    image-20240212-170216.png
  10. As soon as entities are merged from the feature branch to the master branch in the GitHub repository mentioned in Jenkins, the Import Utility is executed.

  11. On the Build tab, select the Pipeline Script command and create the pipeline (explained in the above section) to execute the Utility.

  12. Click Save.

Panel
panelIconIdatlassian-note
panelIcon:note:
bgColor#DEEBFF

Note

Default values for parameters (kyvosEntitiesPath, KyvosUsername, kyvosPassword, KyvosURL) should be specified during Jenkins configuration. Otherwise, merging the feature branch with the main branch will trigger the Jenkins job with null parameters

GitLab settings to integrate with Jenkins
Anchor
GITLABJenkins
GITLABJenkins
isMissingRequiredParameterstrue

...