Semantic Model: kyvos.query.shuffle.sortBeforeShuffle.enabled

Description:

Kyvos' distributed query engines process queries on several query engine nodes. Query engines may need to exchange tuples during query execution to ensure correctness and maintain a high degree of parallelism. One query engine node may receive multiple chunks of shuffled tuples from different query engines, where it needs to merge these tuples.

This property governs the behavior of what method should be used for merging these tuples. In the default method, the tuples are merged using a disk-backed (swap-able) tree-map. In the other method, if the tuples are sorted, they can be merged via a “sorted merge” method which guarantees a tuple would be completed before being swapped to disk.

Values and behavior:

• true: If this property is true, tuples are sorted by the sender query engine before shuffle happens and a “sorted merge” is used at the receiver query engine as explained above. 

• false: If set to false, it will keep running as earlier.

Default Value:

false

Scope:

This is applicable to MDX queries only.

Recommendation:

A sorted merge method would be more optimal for cases where the resultant tuples cannot be kept in memory. Like an MDX query involving high cardinality distinct count or involving an MDX calculation requiring operation over a large set of values (where the calculation is a supported push-down type calculation).

Related Properties:

Connection Property: kyvos.query.shuffle.mode