Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. Databricks > Advance configuration > Spark

    • In the Spark Configurations, add the spark.local.dir /local_disk0 property where the local_disk0 is the managed disk.
      This can be verified using the df -h command from a notebook

  2. Environment variables

    Add the SPARK_WORKER_DIR=/local_disk0 as environment variables.