Document toolboxDocument toolbox

Java API for Designing Datasets

Owned by NIDHI BHANDARI
Last updated: Jun 03, 2024 by DocReviewer
8 min read

Applies to: Kyvos Enterprise  Kyvos Cloud (SaaS on AWS) Kyvos AWS Marketplace

Kyvos Azure Marketplace   Kyvos GCP Marketplace Kyvos Single Node Installation (Kyvos SNI)

Overview

  1. Create a dataset.
    You will need to specify the registered file name as input (could be a Dataset or a Registered File). This will add a Fetch step by default with provided registered file.
    Add different steps to the created dataset.

    1. Format step

    2. Filter step

    3. GroupBy step

    4. Formula step

    5. Lookup step

    6. ExternalTask step

    7. Join step

    8. Union step

  2. To add a second Fetch step, you will need to specify a registered file name as input.

  3. Preview dataset at any specified step.

  4. Save the dataset object in the repository.

  5. Retrieve the existing dataset object by name from the repository.

  6. Delete any step in the dataset by name.

  7. Modify materialize/other properties at the output step.

  8. Update dataset object in the repository.

  9. Process dataset-object.

    1. Process dataset with NOW option

    2. Process dataset with ONCE option

    3. Cancel dataset process, if currently running

Example

  1. Create the Dataset Object.
    This API fetches the Dataset object and sets column details in the Fetch step.

    /* * method to create new Dataset object in Repository Manager. * @param doName: Dataset/Registered File to be associated with Input step. * @param requestorUserInfo: Object of type UserInfo. * @return Dataset: Object of type Dataset. */ public Dataset createDatasetObject(String doName, UserInfo requestorUserInfo); /** * method to create new Dataset object to use dataset inside a folder in Repository Manager. * @param doName: Dataset/Registered File to be associated with Input step. * @param folderName: Name of the folder that contains dataset/registered file to be    associated with Input step. * @param requestorUserInfo: Object of type UserInfo containing information about logged in user. * @return datasetObj: Object of type Dataset. */ public Dataset createDatasetObject(String doName, String folderName, UserInfo requestorUserInfo);

  2. Add different steps in dataset.

    1. Add Format step after specified step to:

      1. Add new column

      2. Modify existing column

        /** * method to add Format step in transformation. * @param inputStepName: Name of the step after which Format step has to be added. * @return formatStep: Object of type FormatStep. */ public FormatStep addFormatStep(String inputStepName);

    2. Add new column

      /** * method to add new column in Format step. * @param columnName: Name of the column. * @param dataType: Data type of the column. * @param staticValue: Static value to be used. * @return formatColumn: Object of type FormatColumn. */ public FormatColumn addNewColumn(String columnName, String dataType, String staticValue);code

    3. Modify existing column

    4. Update column caption

    5. Update column name

    6. Update column data type

    7. Update column input format

    8. Update column output format

      1. Add Filter step to filter out the data by specifying filter criteria

        1. Add filter step after the specified step

        2. Add filter condition with a single filter value. Can also be used in the case of INLIST and NOTINLIST operation by specifying comma (,) separated values.

        3. Add filter condition for BETWEEN operation by specifying firstValue (from) and secondValue (to)

        4. Add GroupBy step after specified step for grouping the data based upon GroupBy fields

          1. Add GroupBy step after specified step

          2. Add GroupBy field by specifying the field name

          3. Add GroupBy field by specifying field name, function name, and filed alias name

          4. Add output field by specifying field name

          5. Add output field by specifying field name, function name, and field alias name

      2. Add formula step to add any java formula

      3. Add Formula step after specified step

      4. Add Lookup step to select any lookup type dataset with reference field to get location related data for current dataset field

      5. Add formula by specifying formula name, formula return type and formula expression

        1. Add External Task step to run any external script

        2. Add External Task step after any specified step

        3. Set class name property value

        4. Set parameters property value

        5. Add new property by specifying property name with value

        6. Add join type to join two different steps

          1. Add join step by specifying step names to be joined together

          2. Add join condition by specifying columns names for both input steps

        7. Add Union step by specifying two different steps as Input

          1. Add Union step after specified step

  3. API for creating Fetch step, passing positions, and Dataset Object ID.

  4. API to preview dataset at any specified step.

  5. Add the Dataset Object to the repository.

  6. API to retrieve the existing dataset object by name from the repository.

  7. API to delete any step in dataset by name.

  8. API to modify materialize/other properties at output step.

    1. Get output step of dataset

    2. Add column fields at output steps

      1. Set materialize property at output step

      2. Add KML information at output step

  9. API to update the dataset object in the repository.

  10. Process a dataset object.
    Process dataset with NOW option

    1. Process dataset with ONCE option

    2. Cancel dataset process if currently running



Copyright Kyvos, Inc. All rights reserved.