Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Applies to: (tick) Kyvos Enterprise  (tick) Kyvos Cloud (SaaS on AWS) (tick) Kyvos AWS Marketplace

...

Panel
panelIconIdatlassian-note
panelIcon:note:
bgColor#DEEBFF

Note

  • From Kyvos 2024.11 onwards, you can use the semantic model in a tabular way.

  • From Kyvos Manager, you can set the duration (in hours) to limit the semantic model process time using the BUILD_TIME_THRESHOLD (OlapEngine.property). Once the semantic model process limit is reached, the process will fail. If you resume the semantic model process, it will use the same duration that you set during the initial semantic model process. 

There are two options for scheduling the different job types. See Semantic model process types. Click the i icon next to the job type list to get a description of each process type. 

...

Aura tab collection
paramsJTdCJTIyZ2VuZXJhbCUyMiUzQSU3QiUyMnRhYlNwYWNpbmclMjIlM0EwJTJDJTIydGFiV2lkdGglMjIlM0ExMDAlMkMlMjJ0YWJIZWlnaHQlMjIlM0E1MCUyQyUyMmRpcmVjdGlvbiUyMiUzQSUyMmhvcml6b250YWwlMjIlN0QlMkMlMjJjb250ZW50JTIyJTNBJTdCJTIyYmFja2dyb3VuZENvbG9yJTIyJTNBJTdCJTIyY29sb3IlMjIlM0ElMjIlMjNmZmYlMjIlN0QlMkMlMjJib3JkZXIlMjIlM0ElN0IlMjJzdHlsZSUyMiUzQSUyMnNvbGlkJTIyJTJDJTIyd2lkdGglMjIlM0ExJTJDJTIydG9wJTIyJTNBdHJ1ZSUyQyUyMmJvdHRvbSUyMiUzQXRydWUlMkMlMjJsZWZ0JTIyJTNBdHJ1ZSUyQyUyMnJpZ2h0JTIyJTNBdHJ1ZSUyQyUyMmNvbG9yJTIyJTNBJTdCJTIybGlnaHQlMjIlM0ElMjIlMjNjY2NlY2YlMjIlN0QlN0QlMkMlMjJwYWRkaW5nJTIyJTNBJTdCJTIydG9wJTIyJTNBMTAlMkMlMjJyaWdodCUyMiUzQTEwJTJDJTIyYm90dG9tJTIyJTNBMTAlMkMlMjJsZWZ0JTIyJTNBMTAlN0QlN0QlMkMlMjJhY3RpdmUlMjIlM0ElN0IlMjJiYWNrZ3JvdW5kQ29sb3IlMjIlM0ElN0IlMjJjb2xvciUyMiUzQSU3QiUyMmxpZ2h0JTIyJTNBJTIyJTIzZjU4MjI3JTIyJTdEJTdEJTJDJTIydGV4dCUyMiUzQSU3QiUyMmZvbnRTaXplJTIyJTNBMTYlMkMlMjJjb2xvciUyMiUzQSU3QiUyMmxpZ2h0JTIyJTNBJTIyJTIzMDAwMDAwJTIyJTdEJTJDJTIydGV4dEFsaWduJTIyJTNBJTIybGVmdCUyMiUyQyUyMmZvbnRXZWlnaHQlMjIlM0ElMjJib2xkJTIyJTdEJTdEJTJDJTIyaG92ZXIlMjIlM0ElN0IlMjJiYWNrZ3JvdW5kQ29sb3IlMjIlM0ElN0IlMjJjb2xvciUyMiUzQSUyMiUyM2RmZTFlNiUyMiU3RCUyQyUyMnRleHQlMjIlM0ElN0IlMjJmb250U2l6ZSUyMiUzQTE4JTJDJTIyY29sb3IlMjIlM0ElMjIlMjM1ZTZjODQlMjIlMkMlMjJ0ZXh0QWxpZ24lMjIlM0ElMjJsZWZ0JTIyJTJDJTIyZm9udFdlaWdodCUyMiUzQSUyMmxpZ2h0ZXIlMjIlN0QlN0QlMkMlMjJpbmFjdGl2ZSUyMiUzQSU3QiUyMmJhY2tncm91bmRDb2xvciUyMiUzQSU3QiUyMmNvbG9yJTIyJTNBJTIyJTIzZjRmNWY3JTIyJTdEJTJDJTIydGV4dCUyMiUzQSU3QiUyMmZvbnRTaXplJTIyJTNBMTYlMkMlMjJjb2xvciUyMiUzQSUyMiUyMzVlNmM4NCUyMiUyQyUyMnRleHRBbGlnbiUyMiUzQSUyMmxlZnQlMjIlMkMlMjJmb250V2VpZ2h0JTIyJTNBJTIybGlnaHRlciUyMiU3RCUyQyUyMmJvcmRlciUyMiUzQSU3QiUyMnRvcCUyMiUzQXRydWUlMkMlMjJsZWZ0JTIyJTNBdHJ1ZSUyQyUyMnJpZ2h0JTIyJTNBdHJ1ZSUyQyUyMmJvdHRvbSUyMiUzQXRydWUlMkMlMjJ3aWR0aCUyMiUzQTElMkMlMjJzdHlsZSUyMiUzQSUyMnNvbGlkJTIyJTJDJTIyY29sb3IlMjIlM0ElN0IlMjJsaWdodCUyMiUzQSUyMiUyM2NjY2VjZiUyMiU3RCU3RCU3RCU3RA==
Aura tab
summaryProcess Full Data
paramsJTdCJTIydGl0bGUlMjIlM0ElMjJQcm9jZXNzJTIwRnVsbCUyMERhdGElMjIlN0Q=
  1. From the Toolbox, choose Semantic Model.

  2. Select a semantic model from the list and click Process

  3. From the top right of the window, click Schedule Job.

  4. From the Job Type list, select the Process Full Data option.

  5. Select the job connection that you want to use. In case there are multiple job connections in the underlying registered files in the semantic model, semantic model validation will be triggered on the default Hadoop connection.

  6. You can review and alter the design using recommendations.

  7. If the last job failed, you can resume from the last failure state. Click Resume to confirm. Use the job Source Datasets options to specify whether to materialize datasets with every job or not.

  8. Depending upon the type of Aggregation Strategy selected in the semantic model design, you can add additional query patterns or analyze query patterns using the corresponding links. 

    1. Optionally, click filter queries to select the queries to be used for the aggregation strategy.

    2. The system will automatically generate aggregates definitions based on the latest data profiles, all available historical queries, and additional query patterns.

  9. Select Process as Metadata and Data or Metadata only. This is applicable only when Kyvos compute -based processing and raw data querying is enabled on the semantic model.

    • Metadata and Data: Selecting this option will populate both the dimension metadata cache and data (Kyvos Analytical Store). Queries will be served from Kyvos Analytical Store.Metadata Only

      • Aggregated Data: The system will

      populate only dimension metadata cache. Your queries will be submitted directly to the underlying data source.
    Click one of the following options toselect your materialization strategy.
    NOTE: This is applicable only when Kyvos compute and raw data querying is enabled on the semantic model. While scheduling Process Full Data for a semantic model, the Both option must be selected.
    1. Aggregated Data: The system will keep the aggregated data (and update it in subsequent semantic model processing as per the query patterns) in Kyvos Analytical store. This data will be used to serve the aggregates queries.

    2. Raw Data: The system will keep the raw data in Kyvos Analytical store. This data will be used to serve those queries for which smart aggregates are not available in the Kyvos Analytical store.

    3. Both: The system will keep aggregated and raw data both.

    Choose to run once, now, or set up a recurring schedule. With each option, you can also view or change the properties currently in use.

    • If you choose Once, select a date, time, and time zone.

    • To set up
      • keep the aggregated data (and update it in subsequent semantic model processing as per the query patterns) in Kyvos Analytical store. This data will be used to serve the aggregates queries.

      • Raw Data: The system will keep the raw data in Kyvos Analytical store. This data will be used to serve those queries for which smart aggregates are not available in the Kyvos Analytical store.

      • Both: The system will keep aggregated and raw data both.

    • Metadata Only: The system will populate only dimension metadata cache. Your queries will be submitted directly to the underlying data source.

  10. Choose to run once, now, or set up a recurring schedule. With each option, you can also view or change the properties currently in use.

    • If you choose Once, select a date, time, and time zone.

    • To set up recurring process, enter the start and end times, the frequency, date, days, week, moths, hours, minutes, and time zone.
      Specify whether to job once per day or repeat. Kyvos picks the time zone from where the report server is running, as the default time zone.

    Click
    • You can also specify configure semantic model job schedules for specific days of the week using a recurring schedule. This includes flexible options to schedule jobs on non-consecutive days, such as alternating days, specific weekdays, or any custom combination of days.

  11. Click Schedule to start the process.

image-20241125-185558.pngImage Removedimage-20250103-113844.pngImage Added

Aura tab
summaryIncremental Process
paramsJTdCJTIydGl0bGUlMjIlM0ElMjJJbmNyZW1lbnRhbCUyMFByb2Nlc3MlMjAlMjIlN0Q=

    in a day. The adjustment details are also displayed in the job summary.
  1. Use the Replace Partition option to update the already processed data. This option drops out old data and adds current data - keeping the partition size constant. See Replace Partitions.

  2. Depending upon the type of Aggregation Strategy selected in the semantic model design, you can select any of the following:

    1. Generate additional aggregates, keeping the existing ones (if any): Additional aggregates definition recommendation will be taken keeping existing aggregates as it is.

    2. Use existing aggregates definition: No new aggregates definition recommendation will be taken, aggregates will be created using the existing definition only.

  3. Add additional query patterns or analyze query patterns using the corresponding links. 

  4. Optionally, click filter queries to select the queries to be used for the aggregation strategy.

  5. Select Process as Metadata and Data or Metadata only. This is applicable only when Kyvos compute-based processing is enabled on the semantic model.

    • Metadata and Data: Selecting this option will populate both the dimension metadata cache and data (Kyvos Analytical Store). Queries will be served from Kyvos Analytical Store.

    • Metadata Only: The system will populate only dimension metadata cache. Your queries will be submitted directly to the underlying data source.

  6. Click one of the following options toselect your materialization strategy.
    NOTE: This is applicable only when Kyvos compute and raw data querying is enabled on the semantic model. While scheduling Process Full Data for a semantic model, the Both option must be selected.
  7. Aggregated Data: The system will keep the aggregated data (and update it in subsequent semantic model processing as per the query patterns) in Kyvos Analytical store. This data will be used to serve the aggregates queries.

  8. Raw Data: The system will keep the raw data in Kyvos Analytical store. This data will be used to serve those queries for which smart aggregates are not available in the Kyvos Analytical store.

  9. Both: The system will keep aggregated and raw data both

    From the Toolbox, choose Semantic Model.

  10. Select a semantic model from the list and click Process.

  11. From the top right of the window, click Schedule Job.

  12. From the Job Type list, select the Process Incremental Data option.

  13. Select the Job connection that you want to use. In case there are there are multiple job connections in the underlying datasets in the semantic model, semantic model validation will be triggered on the default Hadoop connection.

  14. From the Process list, select any of the following:

  15. All Newer Data: Detects and processes all the new data in the files used in semantic model design.

  16. Only Fact Adjustment: Detects and processes only adjustable measures in fact files used in semantic model design. In this case, the old data is not overridden, and the adjustment data can be incrementally loaded in the semantic model without touching any previous data. The adjustments will be made at the lowest granularity of the measure data. Adjustment job can be completedquickly and there can be multiple adjustments
  17. Semantic Model.

  18. Select a semantic model from the list and click Process.

  19. From the top right of the window, click Schedule Job.

  20. From the Job Type list, select the Process Incremental Data option.

  21. Select the Job connection that you want to use. In case there are there are multiple job connections in the underlying datasets in the semantic model, semantic model validation will be triggered on the default Hadoop connection.

  22. From the Process list, select any of the following:

    1. All Newer Data: Detects and processes all the new data in the files used in semantic model design.

    2. Only Fact Adjustment: Detects and processes only adjustable measures in fact files used in semantic model design. In this case, the old data is not overridden, and the adjustment data can be incrementally loaded in the semantic model without touching any previous data. The adjustments will be made at the lowest granularity of the measure data. Adjustment job can be completedquickly and there can be multiple adjustments in a day. The adjustment details are also displayed in the job summary.

  23. Use the Replace Partition option to update the already processed data. This option drops out old data and adds current data - keeping the partition size constant. See Replace Partitions.

  24. Depending upon the type of Aggregation Strategy selected in the semantic model design, you can select any of the following:

    1. Generate additional aggregates, keeping the existing ones (if any): Additional aggregates definition recommendation will be taken keeping existing aggregates as it is.

    2. Use existing aggregates definition: No new aggregates definition recommendation will be taken, aggregates will be created using the existing definition only.

  25. Add additional query patterns or analyze query patterns using the corresponding links. 

  26. Optionally, click filter queries to select the queries to be used for the aggregation strategy.

  27. Choose to run once, now, or set up a recurring schedule. With each option, you can also view or change the properties currently in use.

    • If you choose Once, select a date, time, and time zone.

    • To set up a recurring process, enter the start and end times, the frequency, date, days, week, moths, hours, minutes, and time zone.
      Specify whether to job once per day or repeat. Kyvos picks the time zone from where the report server is running, as the default time zoneis running, as the default time zone. You can also specify configure semantic model job schedules for specific days of the week using a recurring schedule. This includes flexible options to schedule jobs on non-consecutive days, such as alternating days, specific weekdays, or any custom combination of days.

    • Click Schedule to start the process.

image-20241125-185716.pngImage Removed
Aura tab
summaryProcess Test Data
paramsJTdCJTIydGl0bGUlMjIlM0ElMjJQcm9jZXNzJTIwVGVzdCUyMERhdGElMjIlN0Q=
  • From the Toolbox, choose Semantic Model.

  • Select a semantic model from the list and click Process

  • From the top right of the window, click Schedule Job.

  • From the Job Type list, select the Process Test Data option.

  • Select the Job connection that you want to use. In case there are there are multiple process connections in the underlying registered files in the semantic model, semantic model validation will be triggered on the default Hadoop connection.

  • From Input Data, you can specify the number of last fact data partitions or the number of records to be processed for the test process.

    Use the Process Source Datasets options to specify whether to materialize datasets with every process or not.

  • Select Process as Metadata and Data or Metadata only. This is applicable only when Kyvos compute-based processing is enabled on the semantic model.

    • Metadata and Data: Selecting this option will populate both the dimension metadata cache and data (Kyvos Analytical Store). Queries will be served from Kyvos Analytical Store.

    • Metadata Only: The system will populate only dimension metadata cache. Your queries will be submitted directly to the underlying data source.

  • Click one of the following options toselect your materialization strategy.
    NOTE:
    paramsJTdCJTIydGl0bGUlMjIlM0ElMjJQcm9jZXNzJTIwVGVzdCUyMERhdGElMjIlN0Q=
    1. From the Toolbox, choose Semantic Model.

    2. Select a semantic model from the list and click Process

    3. From the top right of the window, click Schedule Job.

    4. From the Job Type list, select the Process Test Data option.

    5. Select the Job connection that you want to use. In case there are there are multiple process connections in the underlying registered files in the semantic model, semantic model validation will be triggered on the default Hadoop connection.

    6. From Input Data, you can specify the number of last fact data partitions or the number of records to be processed for the test process.

      Use the Process Source Datasets options to specify whether to materialize datasets with every process or not.

    7. Select Process as Metadata and Data or Metadata only. This is applicable only when Kyvos compute and raw data querying is enabled on the semantic model. While scheduling Process Full Data for a semantic model, the Both option must be selected

      • Metadata and Data: Selecting this option will populate both the dimension metadata cache and data (Kyvos Analytical Store). Queries will be served from Kyvos Analytical Store.

        • Aggregated Data: The system will keep the aggregated data (and update it in subsequent semantic model processing as per the query patterns

      ) in Kyvos Analytical store. This data will be used to serve the aggregates queries.Raw Data: The system will keep the raw data
        • ) in Kyvos Analytical store. This data will be used to serve

      those queries for which smart aggregates are not available in the Kyvos Analytical store.Both
        • the aggregates queries.

        • Raw Data: The system will keep

      aggregated and
        • the raw data

      both.
    8. Depending upon the type of Aggregation Strategy selected  in the semantic model design, you can select any of the following:

      1. Generate minimal aggregates: Semantic model will be processed with minimal aggregation strategy (lowest level materialization only). Use this option to compute values for only the lowest levels in the hierarchy.

      2. Generate fresh aggregates: Fresh aggregates definition recommendation will be taken and existing aggregates will be deleted (if any).

      3. Use existing aggregates definition: No new aggregates definition recommendation will be taken, aggregates will be created using the existing definition only.

    9. Add additional query patterns or analyze query patterns using the corresponding links. 

    10. Optionally, click filter queries to select the queries to be used for the aggregation strategy.
      NOTE: This job will overwrite existing semantic model data (if any).

    11. Choose to run once, now, or set up a recurring schedule. With each option, you can also view or change the properties currently in use.

      • If you choose Once, select a date, time, and time zone.

      • To set up a recurring process, enter the start and end times, the frequency, date, hours, minutes, and time zone.
        Specify whether to job once per day or repeat. Kyvos picks the time zone from where the report server is running, as the default time zone.

    12. Click Schedule to start the process.

    image-20241125-185824.pngImage Removed
    Aura tab
    summaryUpdate Aggregates
    paramsJTdCJTIydGl0bGUlMjIlM0ElMjJVcGRhdGUlMjBBZ2dyZWdhdGVzJTIyJTdE
    1. From the Toolbox, choose Semantic Model.

    2. Select a semantic model from the list and click Process

    3. From the top right of the window, click Schedule Job.

    4. From the Job Type list, select the Upgrade Aggregates option.

    5. Select the job connection that you want to use. In case there are multiple job connections in the underlying registered files in the semantic model, semantic model validation will be triggered on the default Hadoop connection.

    6. You can review and alter the design using recommendations.

    7. Select partitions to update aggregates. 

    8. Select Process as Metadata and Data or Metadata only. This is applicable only when Kyvos compute-based processing is enabled on the semantic model.

      • Metadata and Data: Selecting this option will populate both the dimension metadata cache and data (Kyvos Analytical Store). Queries will be served from Kyvos Analytical Store.

      • Metadata Only: The system will populate only dimension metadata cache. Your queries will be submitted directly to the underlying data source.

    9. Click one of the following options toselect your materialization strategy.
      NOTE: This is applicable only when Kyvos compute and raw data querying is enabled on the semantic model. While scheduling Process Full Data for a semantic model, the Both option must be selected.
      1. Aggregated Data: The system will keep the aggregated data (and update it in subsequent semantic model processing as per the query patterns) in Kyvos Analytical store. This data will be used to serve the aggregates queries.

      2. Raw Data: The system will keep the raw data in Kyvos Analytical store. This data will be used to serve those queries for which smart aggregates are not available in the Kyvos Analytical store.

      3. Both: The system will keep aggregated and raw data both.

        • in Kyvos Analytical store. This data will be used to serve those queries for which smart aggregates are not available in the Kyvos Analytical store.

        • Both: The system will keep aggregated and raw data both.

      • Metadata Only: The system will populate only dimension metadata cache. Your queries will be submitted directly to the underlying data source.

    10. Depending upon the type of Aggregation Strategy selected  in the semantic model design, you can select any of the following:

      1. Generate minimal aggregates: Semantic model will be processed with minimal aggregation strategy (lowest level materialization only). Use this option to compute values for only the lowest levels in the hierarchy.

      2. Generate fresh aggregates: Fresh aggregates definition recommendation will be taken and existing aggregates will be deleted (if any).

      3. Use existing aggregates definition: No new aggregates definition recommendation will be taken, aggregates will be created using the existing definition only.

    11. Add additional query patterns or analyze query patterns using the corresponding links. 

    12. Optionally, click filter queries to select the queries to be used for the aggregation strategy.
      NOTE: This job will overwrite existing semantic model data (if any).

    13. Choose to run once, now, or set up a recurring schedule. With each option, you can also view or change the properties currently in use.

      • If you choose Once, select a date, time, and time zone.

      • To set up a recurring process, enter the start and end times, the frequency, date, days, week, moths, hours, minutes, and time zone.
        Specify whether to job once per day or repeat. Kyvos picks the time zone from where the report server is running, as the default time zone. You can also specify configure semantic model job schedules for specific days of the week using a recurring schedule. This includes flexible options to schedule jobs on non-consecutive days, such as alternating days, specific weekdays, or any custom combination of days.

    14. Click Schedule to start the process.

    image-20241125-185824.pngImage Added
    Aura tab
    summaryUpdate Aggregates
    paramsJTdCJTIydGl0bGUlMjIlM0ElMjJVcGRhdGUlMjBBZ2dyZWdhdGVzJTIyJTdE
    1. From the Toolbox, choose Semantic Model.

    2. Select a semantic model from the list and click Process

    3. From the top right of the window, click Schedule Job.

    4. From the Job Type list, select the Upgrade Aggregates option.

    5. Select the job connection that you want to use. In case there are multiple job connections in the underlying registered files in the semantic model, semantic model validation will be triggered on the default Hadoop connection.

    6. You can review and alter the design using recommendations.

    7. Select partitions to update aggregates. 

    8. Depending upon the type of Aggregation Strategy selected in the semantic model design, you can select any of the following:

      1. Generate additional aggregates, keeping the existing ones (if any): Additional aggregates definition recommendation will be taken keeping existing aggregates as it is.

      2. Use existing aggregates definition: No new aggregates definition recommendation will be taken, aggregates will be created using the existing definition only.

    9. Add additional query patterns or analyze query patterns using the corresponding links. 

    10. Optionally, click filter queries to select the queries to be used for the aggregation strategy.
      NOTE: This jobwill overwrite existing semantic model data (if any).

    11. Choose to run once, now, or set up a recurring job schedule. With each option, you can also view or change the properties currently in use.

      • Now: This will immediately start the job.

      • Once: This will run the job at specified time. If you choose Once, select a date, time, and time zone.

      • Recurring: This will run the job regularly at the specified time. To set up a recurring process, enter the start and end times, the frequency, date, hours, minutes, and time zone.
        Specify whether to process job once per day or repeat. Kyvos picks the time zone from where the report server is running, as the default time zonethe report server is running, as the default time zone. You can also specify configure semantic model job schedules for specific days of the week using a recurring schedule. This includes flexible options to schedule jobs on non-consecutive days, such as alternating days, specific weekdays, or any custom combination of days.

    12. Click Schedule to start the process.

    image-20241125-185917.pngImage Removed

    ...

    Other job types

    To schedule other job types, click the ... menu on the Process tab and choose Add other jobs option.

    ...

    Frequency
    Anchor

    ...

    85fbab9c-79e4-4229-b650-bb0c7e0d085f
    85fbab9c-79e4-4229-b650-bb0c7e0d085f

    You can specify the frequency of when your semantic model gets updated using the following options:

    ...