Java API for Register File

Applies to: Kyvos Enterprise  Kyvos Cloud (SaaS on AWS) Kyvos AWS Marketplace

Kyvos Azure Marketplace   Kyvos GCP Marketplace Kyvos Single Node Installation (Kyvos SNI)

Overview

1. Create a RegisteredFile object and set the HDFS folder path to where the CSV file is stored.

2. Create a RegisteredFile object using the HCatalog data source.

3. Iterate the columns in RegisteredFile to format (formatting may include changing the caption, input format, data type, and so on).

4. Add the RegisteredFile object in the repository.

5. Retrieve existing RegisteredFile object from repository.

6. Update CSV properties of existing RegisteredFile object:

   1. path of CSV file

   2. line separator character

   3. field separator character

   4. escape character

   5. enclosed by character

   6. skip top lines

   7. encoding style

   8. if file contains header

   9. if need to ignore empty rows

   10. to include file name as a column

7. Refresh RegisteredFile columns.

8. Update RegisteredFile object in the repository.

9. Data profile for File with ONCE option

10. Data profile for File with NOW option

Example

1. Create the RegisteredFile object (This API fetches the Registered File object and sets column details in the Fetch step)

   /** * method to create Registered File object in Repository Manager. * @param folderPath: Full path of the datasource file. * @param userInfo: Object of type UserInfo containing information about logged in user. * @return RegisteredFile: Object of type RegisteredFile. */ public RegisteredFile registerCSVFile(String folderPath, UserInfo userInfo);

2. Create the RegisteredFile object using HCatalogSource.

   /** * method to create Registered File object in Repository Manager. * @param hcatalog: Object of type HCatalogSource. * @param userInfo: Object of type UserInfo containing information about logged in user. * @return RegisteredFile: Object of type RegisteredFile. */ public RegisteredFile registerHCatalogDataSource(HCatalogSource hcatalog, UserInfo userInfo); /** * method to create Registered File object using HCatalog database and table name. * @param dbName: Name of the hCatalog database. * @param tableName: Name of the hCatalog table. * @param userInfo: Object of type UserInfo containing information about logged in user * @return RegisteredFile: Object of type RegisteredFile. * @throws ReportObjectException. */ public RegisteredFile registerHCatalogDataSource(String dbName, String tableName, UserInfo userInfo);

3. Fetch columns from the RegisteredFile object.

   /* * @return ArrayList<FetchColumn>: ArrayList of FetchColumn objects. */ public ArrayList<FetchColumn> getColumnDetails(); /*  * On iterating this ArrayList, user will get a FetchColumn object for each column. */ public void setName(String columnName); public void setDataType(String dataType); public void setFieldFormat(String fieldFormat); public void setColumnDetails(ArrayList <FetchColumn> columnList);

4. Add the RegisteredFile object to the repository.

5. Retrieve existing RegisteredFile object from repository.

6. Update CSV properties of existing RegisteredFile object.

   1. Update CSV file path

   2. Update line separator character.

   3. Update field separator character.

   4. Update escape character

   5. Update enclosed by character.

   6. Update skip toplines.

   7. Update encoding style.

   8. Update if file contains header.

   9. Update if need to ignore empty rows

   10. Update to include file name as column.

7. Refresh RegisteredFile columns upon updating CSV properties.

8. Update the RegisteredFile object in the repository.

9. Preview on Register Files.

10. Data profile for File with ONCE option.

11. Data profile for File with NOW option. 

12.  Constructor to create a blank file containing a fetch step and a filter step linked together.

13. Set an ID of the registered file.

14. Get a registered file id.   

15.  Set a folder id of the registered file.   

16. Set a folder name of the registered file.

17. Set a connection name in the registered file to connect with the data source.

18.  Set input type in a file.

19. Add a filter in the registered file. 

20. Add or update a user parameter. 

21. Remove a user parameter from the file. 

22. Set column metadata into the file.

23. Get the input type of the file.

24. Get FilterStep from the registered file.

25. Set and get partition column details on the registered File.

    1. Get incremental identifier column name.

    2. Set partition table name.

    3. Set the partition column's metadata mode. 

    4. Set the partition column's minimum value.

    5. Set the partition column's maximum value.

    6. Set the partition column's record count.

    7. Set the number of partitions.

26. Set incremental identifier column in the registered file.

Sample Code