Quick Data Modeling

Applies to: Kyvos Enterprise  Kyvos Cloud (SaaS on AWS) Kyvos AWS Marketplace

Kyvos Azure Marketplace   Kyvos GCP Marketplace Kyvos Single Node Installation (Kyvos SNI)

Workflow-based data modeling

As a self-serve tool, Kyvos allows you to define your business case without the need for any coding expertise. To further enhance the user experience, the Quick Data Modeling feature is introduced. It allows you to create data connections quickly, identify your data (available in tables and SQL), and design an OLAP model without deep learning of the Kyvos interface. It enables you to build a smart skeleton of your business use case with minimal interaction and bootstrapping work.

Typically, to start your analysis using Kyvos, you need to perform the following five steps.

1. Connect to data

2. Create datasets

3. Define relationships

4. Design a semantic model

5. Create visualization/connect to BI tool

With the Quick Data Modeling feature, you can connect to the data source and select the data to register; Kyvos automatically validates your registered data and creates relationships and semantic model design, thereby eliminating the effort and time needed for designing the OLAP model.

On the Quick Data Modeling page, you can further select your semantic model type:

• Multidimensional Semantic Model: Choose this option if you work with large datasets stored in multiple tables. This option creates a dataset, establishes relationships, and processes the semantic models.

• Single Table Semantic Model: Choose this option if all your data is stored in a single dataset. This option creates a dataset and generates a semantic model. You can also define Semantic Model view type as flat or smart.

  • Flat: The semantic model will be designed as a flat table, where all source data columns are included as attributes within a single dimension.

  • Smart: The system will intelligently identify measures and dimensions and create a more business-friendly semantic model view. This model will organize source data fields into measures, attributes, and hierarchies based on their data types.

Using Multidimensional Semantic Model

Let us consider a manufacturing use case for analyzing publicly available SSB data.

1. On the Kyvos home page, click Quick Data Modeling from the top-right of the home page.

2. On the displayed wizard, provide an Analysis Name. Kyvos will create all the folders, relationships, and semantic model with this name. In this example, we have named it Kyvos.

3. Click Start.

4. On the displayed Create Dataset tab, select your data Connection from the Datasource column.

   

5. To modify an existing connection according to your environment, select it from the list and click the Edit link. 

6. You can also modify the default Hadoop connection or click a new connection using the Create New Connection link to define a new Snowflake or Redshift data warehouse connection. Refer to the Working with data connections section to know more.

7. Select the Input Type from File, Table, or SQL, and proceed as explained below:
   

8. Choose the Preview Table icon to see sample data and click Filter data to refine the data that you want to bring in for your analysis.

9. Click the Next button. The system validates all your files and tables and displays errors if your data is invalid.

10. The Define Relationships tab page is displayed with a set of relationships automatically created by the system.
    These relationships are created on the basis of data selected in the previous step and information about master data, fact data, primary key, and foreign key.

    

11. If the system does not generate any relationships or to define your custom relationships, you can manually define relationships. For this, click the Add Relationship link.
    

    

12. Click Apply.

13. The system validates relationships for correctness, and if everything is valid, and generates a semantic model design. The validation status is displayed at the bottom of the screen.

14. The Design semantic model tab displays the dimensions and measures created by the system.
    By default, the name of the data source table from which the first measure is created in the measure group is taken as the measure group name. However, if you have created a measure group with a custom name, then dragging a measure in it would not impact the name of the measure group.
    

    

15. You can further modify the design to add dimensions, measures, define dimensions and measure properties, and so on. Refer to Working with Semantic Models to know more.

16. Click the Next button.

17. The Review and Process tab shows the entities designed till now.

18. Here, you can choose any of the following options:

    • I want to execute the test process now: Select this option to run the test build and click Process Now.
      You can further choose:

      1. Job Connection: Select a connection for job execution.

      2. Input Data: Here, you can specify the number of fact data partitions to be used for the test data process.

      3. Process: Select Metadata and Data or Metadata only. This is applicable only when the semantic model is processed with Kyvos Compute.

         • Metadata and Data: Selecting this option will populate both the dimension metadata cache and data (Kyvos Analytical Store). Queries will be served from Kyvos Analytical Store.

         • Metadata Only: Selecting this option will populate both the dimension metadata cache and data (Kyvos Analytical Store). Queries will be submitted directly to the underlying data source.

      4. Keep: Click one of the following options to keep the Aggregated data or Raw Data. This is applicable only when the semantic model is processed with Kyvos compute and raw data querying is enabled. Additionally, the Both option must be selected when you schedule Process Full Data for a semantic model.

      5. Aggregated Data: The system will keep the aggregated data (and update it in subsequent semantic model processing as per the query patterns) in Kyvos Analytical store. This data will be used to serve the aggregates queries.

      6. Raw Data: The system will keep the aggregated data (and update it in subsequent semantic model processing as per the query patterns) in Kyvos Analytical store. This data will be used to serve the aggregates queries.

      7. Both: The system will keep aggregated and raw data both.

    • I want to execute the build later: Select this option if you want to review your design and entities before proceeding with the build. In this case, your design is saved, and you can access the semantic model from Toolbox at any time.

19. Click Process Now.

20. Once you have launched the test data process data, click View Job Progress to see the job status.
    The system shows the job status in the semantic model designer, as shown in the following figure.

    

    At the successful completion of your test data process, you can start creating visualization and then generate recommendations to further optimize your design.

Using Single Semantic Model

You can now quickly process the semantic model with minimal input, enabling you to generate a semantic design by simply selecting the required table.

To use single semantic model, perform the following steps.

1. On the Kyvos home page, click Quick Data Modeling from the top-right of the home page.

   

2. Select semantic model type as Flat or Smart.

3. On the displayed wizard, provide an Analysis Name. Kyvos will create all the folders, relationships, and semantic model with this name. In this example, we have named it Kyvos.

4. Click Start.

5. On the displayed Create Dataset tab, select your data Connections from the Datasource column.

   

6. By default, the Table is selected as Input Type.

7. From the Database drop-down list, select the data source. The databases available in the data source are listed. 

8. Click the Choose tables as datasource link. The tables available in the database are listed. 

   

9. Click Next, and the system has automatically created dimensions and measures. Kyvos recommends reviewing the semantic model design before moving forward.

10. In the Properties section, click Conversational Analytics Metadata link. Additionally, you hide or show metadata summary.

11. On the displayed Design Semantic Model tab, by default, Raw Data Querying is enabled, and the System Default option is selected for Raw Data SQL Connection to select the default SQL connection for raw data querying.

12. Click Review.

    

13. On the displayed Review tab, click the datasets or semantic models link to open in a new window.

    

14. Here, you can process semantic model to create workbooks or connect semantic model to BI tools. Choose any of the following options:

    • I want to execute the test process now: Select this option to run the test build and click Process Now.
      You can further choose:

      1. Job Connection: Select a connection for job execution.

      2. Input Data: Here, you can specify the number of fact data partitions to be used for the test data process.

      3. Process: Select Metadata and Data or Metadata only. This is applicable only when the semantic model is processed with Kyvos Compute.

         • Metadata and Data: Selecting this option will populate both the dimension metadata cache and data (Kyvos Analytical Store). Queries will be served from Kyvos Analytical Store.

         • Metadata Only: Selecting this option will populate both the dimension metadata cache and data (Kyvos Analytical Store). Queries will be submitted directly to the underlying data source.

      4. Keep: Click one of the following options to keep the Aggregated data or Raw Data. This is applicable only when the semantic model is processed with Kyvos compute and raw data querying is enabled. Additionally, the Both option must be selected when you schedule Process Full Data for a semantic model.

      5. Aggregated Data: The system will keep the aggregated data (and update it in subsequent semantic model processing as per the query patterns) in Kyvos Analytical store. This data will be used to serve the aggregates queries.

      6. Raw Data: The system will keep the aggregated data (and update it in subsequent semantic model processing as per the query patterns) in Kyvos Analytical store. This data will be used to serve the aggregates queries.

      7. Both: The system will keep aggregated and raw data both.

    • I want to execute the build later: Select this option if you want to review your design and entities before proceeding with the build. In this case, your design is saved, and you can access the semantic model from Toolbox at any time.

15. Click Done.

16. Once you have launched the test data process data, click View Job Progress to see the job status.