Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Applies to: (tick) Kyvos Enterprise  (tick) Kyvos Cloud (SaaS on AWS) (tick) Kyvos AWS Marketplace

...

Your files or Hive tables may have columns used by your Kyvos semantic model as well as unused columns. The registration process allows you to select the columns which should be read by Kyvos. Similarly, if you are using SQL to resister input data in Kyvos, then it is always recommended to get the required columns in the select clause of the SQL.Selecting only the useful columns increases the performance at various stages of transformation and the semantic model process by reducing the memory footprint and network bandwidth usage required for the transformation and semantic model process.

Specify input format for date columns.

...

The meta information on the dataset can change on many occasions. There might be a change in the data type, column name, or location of the underlying file on HDFS. In any of these cases, the format for the file must be re-validated to avoid the chances of metadata mismatch and subsequent failure in transformation building or semantic model buildingprocessing.

However, in case of changes in the HCatalog table, especially in scenarios where the dataset is imported through a .cab file – you must be very careful while overwriting metadata. This may cause changes to the imported design.

...

Panel
panelIconIdatlassian-note
panelIcon:note:
bgColor#DEEBFF

Note

There is no limit to the number of character or lines in the dataset SQL editor. We have successfully verified with 440+ lines with approximately the same number of columns in the Kyvos QA lab.

Incremental identifiers should be used when incoming data is in increasing order

...