Versions Compared

Old Version 5

changes.mady.by.user Sarabjeet Kaur

Saved on

compared with

New Version Current

changes.mady.by.user DocReviewer

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Applies to: (tick) Kyvos Enterprise  (tick) Kyvos Cloud (SaaS on AWS) (tick) Kyvos AWS Marketplace

...

The sanity suite is a small set of integration tests to check the activities after receiving the software build process and to ensure that all the changes introduced are working as expected.

...

  • Create database kyvosrepo_$clustername if the KYVOS_QUERY_ANALYZER_DATABASE is not present in the olapengine.properties file.

  • Create database $value of KYVOS_QUERY_ANALYZER_DATABASE if KYVOS_QUERY_ANALYZER_DATABASE is present in the olapengine.properties file.

  • Ensure that the Sanity_table table must exist when you are creating a database. 
    NOTE: If AWS Glue is enabled, the Sanity_table tableis not created for AWS cluster deployment. Similarly, if Dataproc Metastore is enabled, this table is not created for GCP cluster deployment. 

Table Syntax

Following are the different types of table Table syntax to be used in the database for cluster deployment if AWS Glue is enabled.

...

paramsJTdCJTIyZ2VuZXJhbCUyMiUzQSU3QiUyMnRhYlNwYWNpbmclMjIlM0EwJTJDJTIydGFiV2lkdGglMjIlM0ExMDAlMkMlMjJ0YWJIZWlnaHQlMjIlM0E1MCUyQyUyMmRpcmVjdGlvbiUyMiUzQSUyMmhvcml6b250YWwlMjIlN0QlMkMlMjJjb250ZW50JTIyJTNBJTdCJTIyYmFja2dyb3VuZENvbG9yJTIyJTNBJTdCJTIyY29sb3IlMjIlM0ElMjIlMjNmZmYlMjIlN0QlMkMlMjJib3JkZXIlMjIlM0ElN0IlMjJzdHlsZSUyMiUzQSUyMnNvbGlkJTIyJTJDJTIyd2lkdGglMjIlM0ExJTJDJTIydG9wJTIyJTNBdHJ1ZSUyQyUyMmJvdHRvbSUyMiUzQXRydWUlMkMlMjJsZWZ0JTIyJTNBdHJ1ZSUyQyUyMnJpZ2h0JTIyJTNBdHJ1ZSUyQyUyMmNvbG9yJTIyJTNBJTdCJTIybGlnaHQlMjIlM0ElMjIlMjNjY2NlY2YlMjIlN0QlN0QlMkMlMjJwYWRkaW5nJTIyJTNBJTdCJTIydG9wJTIyJTNBMTAlMkMlMjJyaWdodCUyMiUzQTEwJTJDJTIyYm90dG9tJTIyJTNBMTAlMkMlMjJsZWZ0JTIyJTNBMTAlN0QlN0QlMkMlMjJhY3RpdmUlMjIlM0ElN0IlMjJiYWNrZ3JvdW5kQ29sb3IlMjIlM0ElN0IlMjJjb2xvciUyMiUzQSU3QiUyMmxpZ2h0JTIyJTNBJTIyJTIzZjU4MjI3JTIyJTdEJTdEJTJDJTIydGV4dCUyMiUzQSU3QiUyMmZvbnRTaXplJTIyJTNBMTYlMkMlMjJjb2xvciUyMiUzQSU3QiUyMmxpZ2h0JTIyJTNBJTIyJTIzMDAwMDAwJTIyJTdEJTJDJTIydGV4dEFsaWduJTIyJTNBJTIybGVmdCUyMiUyQyUyMmZvbnRXZWlnaHQlMjIlM0ElMjJib2xkJTIyJTdEJTdEJTJDJTIyaG92ZXIlMjIlM0ElN0IlMjJiYWNrZ3JvdW5kQ29sb3IlMjIlM0ElN0IlMjJjb2xvciUyMiUzQSUyMiUyM2RmZTFlNiUyMiU3RCUyQyUyMnRleHQlMjIlM0ElN0IlMjJmb250U2l6ZSUyMiUzQTE4JTJDJTIyY29sb3IlMjIlM0ElMjIlMjM1ZTZjODQlMjIlMkMlMjJ0ZXh0QWxpZ24lMjIlM0ElMjJsZWZ0JTIyJTJDJTIyZm9udFdlaWdodCUyMiUzQSUyMmxpZ2h0ZXIlMjIlN0QlN0QlMkMlMjJpbmFjdGl2ZSUyMiUzQSU3QiUyMmJhY2tncm91bmRDb2xvciUyMiUzQSU3QiUyMmNvbG9yJTIyJTNBJTIyJTIzZjRmNWY3JTIyJTdEJTJDJTIydGV4dCUyMiUzQSU3QiUyMmZvbnRTaXplJTIyJTNBMTYlMkMlMjJjb2xvciUyMiUzQSUyMiUyMzVlNmM4NCUyMiUyQyUyMnRleHRBbGlnbiUyMiUzQSUyMmxlZnQlMjIlMkMlMjJmb250V2VpZ2h0JTIyJTNBJTIybGlnaHRlciUyMiU3RCUyQyUyMmJvcmRlciUyMiUzQSU3QiUyMnRvcCUyMiUzQXRydWUlMkMlMjJsZWZ0JTIyJTNBdHJ1ZSUyQyUyMnJpZ2h0JTIyJTNBdHJ1ZSUyQyUyMmJvdHRvbSUyMiUzQXRydWUlMkMlMjJ3aWR0aCUyMiUzQTElMkMlMjJzdHlsZSUyMiUzQSUyMnNvbGlkJTIyJTJDJTIyY29sb3IlMjIlM0ElN0IlMjJsaWdodCUyMiUzQSUyMiUyM2NjY2VjZiUyMiU3RCU3RCU3RCU3RA==
Aura tab
summaryAzure
paramsJTdCJTIydGl0bGUlMjIlM0ElMjJBenVyZSUyMCUyMiU3RA==

These are the steps

  1. Step 1 for editing

  2. Step 2 for editing

Aura tab
summaryGCP
paramsJTdCJTIydGl0bGUlMjIlM0ElMjJHQ1AlMjAlMjIlN0Q=

  1. Step 1 for editing

  2. Step 2 for editing

Aura tab
summaryOn-prem
paramsJTdCJTIydGl0bGUlMjIlM0ElMjJPbi1wcmVtJTIyJTdE

...

Step 1 for editing

...

Code Block
languagejson
CREATE EXTERNAL TABLE `sanity_table`(
  `product_id` int,
  `product_category` string,
  `manufacturer` string,
  `product_type` string,
  `product_name` string,
  `order_priority` string,
  `sale_amount` double,
  `order_quantity` int,
  `discount` double,
  `continent` string,
  `country` string,
  `region` string,
  `state` string,
  `city` string,
  `profit` double,
  `customer_name` string,
  `customer_segment` string,
  `shipping_cost` double,
  `date` string,
  `quarter` string,
  `month` string,
  `week` string,
  `day` string)
PARTITIONED BY (
  `year` int)
ROW FORMAT SERDE
  'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe'
STORED AS INPUTFORMAT
  'org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat'
OUTPUTFORMAT
  'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat'
LOCATION
  's3://cftautomationdeployment/user/engine_work_cftAutomationCluster/data/Reference/parquet'
TBLPROPERTIES (
  'bucketing_version'='2',
  'transient_lastDdlTime'='1694802287')
Time taken: 0.426 seconds, Fetched: 37 row(s)CREATE EXTERNAL TABLE `sanity_table`(
  `product_id` int,
  `product_category` string,
  `manufacturer` string,
  `product_type` string,
  `product_name` string,
  `order_priority` string,
  `sale_amount` double,
  `order_quantity` int,
  `discount` double,
  `continent` string,
  `country` string,
  `region` string,
  `state` string,
  `city` string,
  `profit` double,
  `customer_name` string,
  `customer_segment` string,
  `shipping_cost` double,
  `date` string,
  `quarter` string,
  `month` string,
  `week` string,
  `day` string)
PARTITIONED BY (
  `year` int)
ROW FORMAT SERDE
  'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe'
STORED AS INPUTFORMAT
  'org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat'
OUTPUTFORMAT
  'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat'
LOCATION
  's3://cftautomationdeployment/user/engine_work_cftAutomationCluster/data/Reference/parquet'
TBLPROPERTIES (
  'bucketing_version'='2',
  'transient_lastDdlTime'='1694802287')
Time taken: 0.426 seconds, Fetched: 37 row(s)

Using Sanity Suite

After cluster deployment, you can run the Sanity Suite to check if all the activities (like creating a Datasets, Data Relationship Diagram, and Semantic model processing) for browsing semantic models in the Kyvos application are working fine.

...