As a part of the constant endeavor to enhance the ease of working with Kyvos, the Continuous Integration and Continuous Deployment (CI/CD) Utility is created to automate the versioning of Kyvos entities like Semantic Models, Relationships, Datasets, etc., using Git as a versioning tool. It can be used to migrate entities from one Kyvos deployment (Source) to another Kyvos deployment (Target).
...
Parameter | Description |
EXPORT_SEMANTIC_MODELS | Defines whether to export semantic model entities or not.
|
SEMANTIC_MODEL_DETAILS | Provides details of the semantic model that needs to be exported in the following format: "FolderName":["SemanticModelName"], “FolderName” should be the name of the Kyvos entity folder name to be exported. "SemanticModelName": Comma separated entity names existing within the folder Example {"Folder1": [ “Model1”,” Model2”], “Folder2”: [“Model3”]} |
EXPORT_DATASET_RELATIONSHIPS
| Defines whether to export Dataset Relationship entities or not.
|
DATASET_RELATIONSHIP_DETAILS
| Provides details of the Dataset Relationship that must be exported in the following format. "FolderName":["DatasetRelationshipName"], “FolderName” should be the name of the Kyvos entity folder name to be exported. "DatasetRelationshipName": Comma separated entity names existing within the folder For example, {"Folder1": [ “DRelationship1”,” DRelationship2”], “Folder2”: [“DRelationship3”]} |
EXPORT_DATASETS | Defines whether to export Dataset entities or not.
|
DATASET_DETAILS | Provides details of the Dataset that must be exported in the following format. "FolderName": ["DatasetName"], “FolderName” should be the name of the Kyvos entity folder name to be exported. "DatasetName": Comma separated entity names existing within the folder Example {"Folder1": [ “Dataset 1”,” Dataset 2”], “Folder2”: [“Dataset 3”]} |
EXPORT_WB | Defines whether to export Workbooks or not.
|
WB_DETAILS | Provides details of the Workbooks that must be exported in the following format. "FolderName": ["Workbook"], “FolderName” should be the name of the Kyvos entity folder name to be exported. "Workbook": Comma separated entity names existing within the folder For example, {"Folder1": [ “workbook1”,” workbook2”], “Folder2”: [“workbook3”]} |
GIT_PUSH | Defines whether to push the exported entities into GitLab or not.
|
GIT_URL | Server URL |
GIT_LOCAL_REPO_PATH | Provides a folder path to be used as a staging location. For example, “C:/CICDUtility/Export/GITRepo” NOTE: Use the forward-slash (/) in the repo path. |
GIT_FEATURE_BRANCH_NAME | Specifies the GitLab feature branch name where entities will get exported. |
GIT_FEATURE_BRANCH_CLEAN | Defines whether to clean the GIT Feature branch before a fresh export. The default value is N.
|
GIT_REPONAME | Specifies the GitLab repository name where the entities will get exported. |
GIT_USERNAME | Specifies the GitLab user name to connect with GIT Repo. |
GIT_TOKEN | The Personal Access Token for authentication with the user’s GIT account. If you do not have the Personal Access token, you can create it using the steps explained in the Appendix. |
GIT_URL | Git url to be mentioned |
Panel | ||||||
---|---|---|---|---|---|---|
| ||||||
Note To export an entity from the root folder, users must specify the folder name as ‘ROOT’ in the input.json file. |
...
Example configurations for different scenarios
...
If the GITPUSH parameter is set as Y, all the entity XML files will be exported to the GitLab Feature branch in the following order.
A GIT PULL is performed from the feature branch specified in the input file to sync the remote and local repository (GITRepo).
The entities are exported to the GITRepo folder from Kyvos, a local repository clone of the GitLab repository.
Entities are then pushed to the GitLab feature branch.
The entities are directly exported to the Output folder if the GITPUSH parameter is set as N in the input file. No GIT PULL and GIT PUSH will be performed in this case.
Import Utility
Panel | ||||||
---|---|---|---|---|---|---|
| ||||||
Note Upon the initial setup configuration, users are required to execute the following commands:
|
Import Utility
The import utility works in two steps.
...
If the Jenkins pipeline is not configured, Step 2 can still launch ImportKyvosEntities.sh independently.
Prerequisites
Git Bash: It is a package that installs Bash, some common bash utilities, and Git on an operating system. This is needed to execute the Git commands on your system.
GitLab: Git server with main and feature branches.
Jenkins: For automated execution of the Import utility script, based on a trigger from GitLab.
...
Extract the setup zip.
Configure source target Kyvos deployment details in the serverDetailImportUtility.properties file in the Conf folder. This file needs the details below to be filled in. If the details are not provided in this file, then alternatively, these details can be provided as an argument to the import utility script.
KYVOS_URL = <YourKyvosURL>/rest/
USERNAME = <KyvosUserName>
PASSWORD = <KyvosUserPassword>= <KyvosUserPassword>
KyvosEntitiesPath=<Path of KyvosEntities store in local machine>
NOTE: It’s value should represent the directory path leading up to the KyvosEntities folder.
For example:/data/Automation_Data/mayank/CICD/CICD_Utility_2023.5/EntitiesInMultipleFolders/CICD_Testing/KyvosEntities
Execute the chmod +x ImportKyvosEntities.sh permission on the ImportKyvosEntities.sh file, if not already provided.
...
KyvosUsername is the name of the native user with admin access to the target Kyvos deployment.
KyvosPassword is the password of the native user with admin access to the target Kyvos deployment.
KyvosURL is the URL of the target Kyvos deployment.
...
Code Block |
---|
pipeline { agent any stages { stage ('Environment Details') { steps { sh ''' env | grep -e PATH -e JAVA_HOME which java java -version mvn –version"” }} stage('Checkout') { steps { checkout ( [$class: 'GitSCM', branches: [[name: '*/your_branch_name']], extensions: [], userRemoteConfigs: [[credentialsId: 'your_credential_id', url: 'your_git_url']]] ) } } stage('Build') { steps { script{ dir('jenkinsWorkSpacePath/cicdUtilityPath') { sh "pwd" sh " jenkinsWorkSpacePath /cicdUtilityPath/ImportKyvosEntities.sh ${kyvosEntitiesPath} ${KyvosUsername} ${kyvosPassword} ${KyvosURL}" } } } } } } |
...
Login to your Jenkins account.
Install the following plugins:
Git client plugin
GitHub API
GitHub Branch Source Plugin
GitHub plugin
Genric Webhook plugin
If the plugins are not installed while initializing Jenkins, go to Manage Jenkins >Manage Plugins and install the plugins.
Click New Item from the top left of your Dashboard.
Enter the name of the item you want to create, select Pipeline,and click Ok. The project is created.
Navigate to the newly created project and select the Configure option, as shown in the following figure.
On the General tab, select the GitHub Project check box and provide the URL of the GitHub/GitLab repository in the Project url field, as shown in the following figure.
On the Build Triggers tab, select Generic Webhook Trigger. This triggers a Jenkins job when the GitLab webhook relays any payload. If you are using a generic webhook, you can select the generic webhook trigger option.
Provide Jenkins credential ID in the Token section or generate a token ID in the Token Credential section by clicking the Add button and provide credentials.
As soon as entities are merged from the feature branch to the master branch in the GitHub repository mentioned in Jenkins, the Import Utility is executed.
On the Build tab, select the Pipeline Script command and create the pipeline (explained in the above section) to execute the Utility.
Click Save.
Panel | ||||||
---|---|---|---|---|---|---|
| ||||||
Note Default values for parameters (kyvosEntitiesPath, KyvosUsername, kyvosPassword, KyvosURL) should be specified during Jenkins configuration. Otherwise, merging the feature branch with the main branch will trigger the Jenkins job with null parameters |
GitLab settings to integrate with Jenkins
Anchor | ||||||
---|---|---|---|---|---|---|
|
...