Page Comparison

Description:

This property specifies the container memory for the map tasks in the MapReduce jobs launched during the semantic model process.

...

If this property is not applied at any of the levels, it would fall back to the default YARN job configuration.

Default value:
8192

Scope:

• Connection: If the property is set at the connection, then the property value is applicable for all transformation processes, semantic model processes, or data profile jobs launched using Spark.

• Semantic model: If the property is set on a semantic model, then the value will override the connection level value for that semantic model’s process job.

Comes into effect:

This property comes into effect in all the semantic model process jobs.  The value of the property can be changed at any time and will be respected in the next process.

Dependencies and related properties:

• The value of the  mapreduce.map.memory.opts

   property

   property should be 55-65% of the memory specified in this property.

• The property comes into effect only

  when 

  when  kyvos.build.execution.engine

   is

   is set as MapReduce

• Semantic Model: kyvos.indexing.map.distributed.mb

Recommendation:

The default value of the property should work fine for most scenarios. If the semantic model process jobs fail due to memory-related failures, then the job memory can be increased using this property. Recommended range:  5 120  to  1 5360