Skip to content
Snippets Groups Projects
Commit 00d92617 authored by Jeff Zhang's avatar Jeff Zhang Committed by Marcelo Vanzin
Browse files

[DOCUMENTATION] doc fix of job scheduling

spark.shuffle.service.enabled is spark application related configuration, it is not necessary to set it in yarn-site.xml

Author: Jeff Zhang <zjffdu@apache.org>

Closes #10657 from zjffdu/doc-fix.
parent ea104b8f
No related branches found
No related tags found
No related merge requests found
......@@ -91,7 +91,7 @@ pre-packaged distribution.
2. Add this jar to the classpath of all `NodeManager`s in your cluster.
3. In the `yarn-site.xml` on each node, add `spark_shuffle` to `yarn.nodemanager.aux-services`,
then set `yarn.nodemanager.aux-services.spark_shuffle.class` to
`org.apache.spark.network.yarn.YarnShuffleService` and `spark.shuffle.service.enabled` to true.
`org.apache.spark.network.yarn.YarnShuffleService`.
4. Restart all `NodeManager`s in your cluster.
All other relevant configurations are optional and under the `spark.dynamicAllocation.*` and
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment