Semantic model process types

Applies to: Kyvos Enterprise  Kyvos Cloud (SaaS on AWS) Kyvos AWS Marketplace

Kyvos Azure Marketplace   Kyvos GCP Marketplace Kyvos Single Node Installation (Kyvos SNI)

When you process a semantic model, you can set up a schedule for when it gets refreshed, and you can specify whether you want to create an incremental or full process. Use a data profile process to help optimize the processing of the semantic model. When you use semantic model recommendations on an already processed semantic model, you can use the update aggregates option to update only the aggregates in the semantic model.

When you are viewing a semantic model, click the Process tab. There are two ways to schedule a process:

• Click Schedule Job. Use this option to create a full, incremental, or test data process and update aggregates.

• Click the Actions menu (...) menu and click Add Other Jobs. Process jobs include dropping partitions, cache, and data profile.

These are all of the available process types.

Schedule Process

• Full data process: Recreates the semantic model with all available data, including recalculating all aggregations. This option can be slow, depending on the size of your data.

• Incremental data process: Updates only the latest changes to an existing semantic model. This option is faster than a full process. For example, use it to update the semantic model with daily changes. You can also replace already processed data using the replace partition option.
  NOTE: You can run an incremental process only on a successfully processed semantic model.
  In an incremental process, you can process the following:

  • All new data: Detects new incremental data in all the files used in the semantic model design and processes it.

  • Only fact adjustments: Detects incremental data only in the fact files used in the semantic model design and processes only the adjustable measure.

• Test data process: Processes the semantic model on the sample data. You can choose the number of rows, aggregation, and data partitions to be considered in this process.

• Update Aggregates: Updates only the aggregates in a process to improve query time. You can also specify the partition to use for updating aggregates. 
  NOTE: You can schedule this process only on a successfully processed semantic model.

Other Process Jobs

• Drop Partition: Drops a whole partition from the process. This option is only available if you are using partitions.

• Cache process: Sets up cache rules and specifies actions for historical and metadata queries.

• Data Profile: Sets profiling options, sampling options, and properties to help you optimize processing a semantic model. 

Deprecated process types

Use consolidation processes to merge incremental processes into a consolidated semantic model that improves query performance. For example, if you do an incremental process daily, you may want to do a monthly consolidation process.

• Consolidation process: Consolidates the original full process and subsequent incremental data processes to create a single full current semantic model. You can combine or combine and balance processes. If there are query performance issues with incremental data processes, use the Combine and balance process option to improve query performance.

• Forced Update Partition :  Replaces a whole partition in the process. This option is only available if you are using partitions. In its place, use the incremental process option and the replace partition option.

Process Status

After the process job is completed, the status is displayed as Success or Failed on the semantic model designer, and the Job summary

The process job can be displayed as Success, Failed, or Partial Success for incremental processes. 
There are scenarios in which the semantic model process is partially completed, such as:

• During the incremental data process of a multifact semantic model, the incremental data is not provided for all the facts. In such cases, no data will be processed for a few facts.

• During the incremental data process, all the records are skipped for a few facts in the multifact semantic model design.

• During fact adjustment processes, a few of the facts do not perform the summary functions which are supported by fact adjustment.
  For all such scenarios, the process status is shown as a partial success.

Related topics

• Scheduling semantic model processes

• Job summary