As a part of the constant endeavor to enhance the ease of working with Kyvos, the Continuous Integration and Continuous Deployment (CI/CD) Utility is was created to automate the versioning of Kyvos entities like Semantic Models, Relationships, Datasets, etc., using Git as a versioning tool. It can be used to migrate a versioning tool.
This document provides details of the utility and the procedures to work with it.The Kyvos CI/CD utility is a Java-based utility that uses Kyvos REST APIs to enable you to export entities from one Kyvos deployment (Source) to source) and import them into another Kyvos deployment (Targettarget).
...
The utility is designed for use in a Linux system. It uses Gitlab to store versions of Kyvos entities in XML format
...
and provides integration with Bitbucket repositories.
...
...
| Panel | ||||||
|---|---|---|---|---|---|---|
| ||||||
Note It is recommended to use Kyvos user credentials with administrative privileges to import and export all entities. |
The CI/CD Utility comprises the following two utilities:
Export Utilityutility: It fetches entities from Kyvos deployment in XML format, stores them in a local folder, and then pushes them to the configured Git repository. It has The export utility comes with a configuration file to define all the entities to be fetched along with source the source Kyvos deployment and git repository details.
Import Utilityutility: It uses Jenkins to automate the import of entities from the Git the Git repository to the target Kyvos deployment. The Jenkins pipeline is configured in the Gitlab the Gitlab server to pull the entities from Gitlab to the local file system. Jenkins then launches the import utility shell script, which finally imports the entities from the file system to target the target Kyvos deployment.
Utility bundle details
The utility bundle contains a single zip file having containing the following folders and files within it:
Conf folderConf folder: It This folder contains serverDetailExportUtility.properties, serverDetailImportUtility.properties, and cicdLogs.properties files.
Input folderInput folder: It This folder contains an the input.json file to configure the exported entities and the for configuring the entities to be exported and Gitlab server details.
Lib folderLib folder: This folder contains the required libraries for the utility.
Output folder: This initially blank folder will folder is initially blank and is used to place exported entities' XML files when they are not pushing them being pushed into the Gitlab server.
ExportKyvosEntities.sh: This is a the script file to launch the Export utility.
ImportKyvosEntities.sh: This is a the script file to launch the Import utility.
Export Utility
Prerequisites
...
Extract the setup zip file.
Configure source Kyvos deployment details in Navigate to the Conf folder and edit the serverDetailExportUtility.properties file located in the Conf folder. Provide the following details in this file. If you don't provide the details in the file, you can alternatively provide them as an argument to the export utility script.to configure source Kyvos deployment. For this, provide the following details:
KYVOS_URL = <YourKyvosURL>/rest/
USERNAME =
<KyvosUserName><KyvosUserName>
PASSWORD =
<KyvosUserPassword><KyvosUserPassword>
If the details are not provided in this file, you can alternatively provide these as arguments in the Export utility script.
Configure input.json within the input folder to define the entities to be exported. The following table provides details of the parameters in the input.json file.
Provide execute permission on ExportKyvosEntities.sh file, using chmod +x ExportKyvosEntities.sh, if not already provided.
Parameter | Description |
EXPORT_SEMANTIC_MODELS | Defines whether to export semantic model entities or not.
|
SEMANTIC_MODEL_DETAILS | Provides details of the semantic model that needs to be exported in the following format: "FolderName":["SemanticModelName"], “FolderName” should be the name of the Kyvos entity folder name to be exported. "SemanticModelName": Comma separated entity names existing within the folder Example {"Folder1": [ “Model1”,” Model2”], “Folder2”: [“Model3”]} |
EXPORT_DATASET_RELATIONSHIPS
| Defines whether to export Dataset Relationship entities or not.
|
DATASET_RELATIONSHIP_DETAILS
| Provides details of the Dataset Relationship that must be exported in the following format. "FolderName":["DatasetRelationshipName"], “FolderName” should be the name of the Kyvos entity folder name to be exported. "DatasetRelationshipName": Comma separated entity names existing within the folder For example, {"Folder1": [ “DRelationship1”,” DRelationship2”], “Folder2”: [“DRelationship3”]} |
EXPORT_DATASETS | Defines whether to export Dataset entities or not.
|
DATASET_DETAILS | Provides details of the Dataset that must be exported in the following format. "FolderName": ["DatasetName"], “FolderName” should be the name of the Kyvos entity folder name to be exported. "DatasetName": Comma separated entity names existing within the folder Example {"Folder1": [ “Dataset 1”,” Dataset 2”], “Folder2”: [“Dataset 3”]} |
EXPORT_WB | Defines whether to export Workbooks or not.
|
WB_DETAILS | Provides details of the Workbooks that must be exported in the following format. "FolderName": ["Workbook"], “FolderName” should be the name of the Kyvos entity folder name to be exported. "Workbook": Comma separated entity names existing within the folder For example, {"Folder1": [ “workbook1”,” workbook2”], “Folder2”: [“workbook3”]} |
GIT_PUSH | Defines whether to push the exported entities into GitLab or not.
|
GIT_URL | Server URL |
GIT_LOCAL_REPO_PATH | Provides a folder path to be used as a staging location. For example, “C:/CICDUtility/Export/GITRepo” NOTE: Use the forward-slash (/) in the repo path. |
GIT_FEATURE_BRANCH_NAME | Specifies the GitLab feature branch name where entities will get exported. |
GIT_FEATURE_BRANCH_CLEAN | Defines whether to clean the GIT Feature branch before a fresh export. The default value is N.
|
GIT_REPONAME | Specifies the GitLab repository name where the entities will get exported. |
GIT_USERNAME | Specifies the GitLab user name to connect with GIT Repo. |
GIT_TOKEN | The Personal Access Token for authentication with the user’s GIT account. If you do not have the Personal Access token, you can create it using the steps explained in the Appendix. |
GIT_URL | Git url to be mentioned |
...
Execute the following Import Utility separately without Jenkins to perform step 2.
| Code Block |
|---|
Run ./ImportKyvosEntities.sh $KyvosEntitiesPath $Username $Password $KyvosUrl |
...
If the Jenkins job is not created and configured, see the Creating new Project in Jenkinssection.
If the Jenkins job is not Integrated with the GitLab Repository, see the GitLab settings to integrate with Jenkinssection.
Once the above steps are completed, you can execute the Import Utility via Jenkins.
...
The Personal Access Token is required for authentication of Export Utility with Git. You are required to provide this in the Export Utility config file, as explained in the Export Utility section. To create a Personal Access Token in GitLab, perform the following steps.
...