Skip to content
Snippets Groups Projects
Commit c2204436 authored by James Lohse's avatar James Lohse Committed by Sean Owen
Browse files

Provide same info as in spark-submit --help

this is stated for --packages and --repositories. Without stating it for --jars, people expect a standard java classpath to work, with expansion and using a different delimiter than a comma. Currently this is only state in the --help for spark-submit "Comma-separated list of local jars to include on the driver and executor classpaths."

Author: James Lohse <jimlohse@users.noreply.github.com>

Closes #10890 from jimlohse/patch-1.
parent 4a091232
No related branches found
No related tags found
No related merge requests found
...@@ -177,8 +177,9 @@ debugging information by running `spark-submit` with the `--verbose` option. ...@@ -177,8 +177,9 @@ debugging information by running `spark-submit` with the `--verbose` option.
# Advanced Dependency Management # Advanced Dependency Management
When using `spark-submit`, the application jar along with any jars included with the `--jars` option When using `spark-submit`, the application jar along with any jars included with the `--jars` option
will be automatically transferred to the cluster. Spark uses the following URL scheme to allow will be automatically transferred to the cluster. URLs supplied after `--jars` must be separated by commas. That list is included on the driver and executor classpaths. Directory expansion does not work with `--jars`.
different strategies for disseminating jars:
Spark uses the following URL scheme to allow different strategies for disseminating jars:
- **file:** - Absolute paths and `file:/` URIs are served by the driver's HTTP file server, and - **file:** - Absolute paths and `file:/` URIs are served by the driver's HTTP file server, and
every executor pulls the file from the driver HTTP server. every executor pulls the file from the driver HTTP server.
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment