From 2b1e94b9add82b30bc94f639fa97492624bf0dce Mon Sep 17 00:00:00 2001
From: hyukjinkwon <gurwls223@gmail.com>
Date: Tue, 4 Jul 2017 12:18:42 +0100
Subject: [PATCH] [MINOR][SPARK SUBMIT] Print out R file usage in spark-submit
## What changes were proposed in this pull request?
Currently, running the shell below:
```bash
$ ./bin/spark-submit tmp.R a b c
```
with R file, `tmp.R` as below:
```r
#!/usr/bin/env Rscript
library(SparkR)
sparkRSQL.init(sparkR.init(master = "local"))
collect(createDataFrame(list(list(1))))
print(commandArgs(trailingOnly = TRUE))
```
working fine as below:
```bash
_1
1 1
[1] "a" "b" "c"
```
However, it looks not printed in usage documentation as below:
```bash
$ ./bin/spark-submit
```
```
Usage: spark-submit [options] <app jar | python file> [app arguments]
...
```
For `./bin/sparkR`, it looks fine as below:
```bash
$ ./bin/sparkR tmp.R
```
```
Running R applications through 'sparkR' is not supported as of Spark 2.0.
Use ./bin/spark-submit <R file>
```
Running the script below:
```bash
$ ./bin/spark-submit
```
**Before**
```
Usage: spark-submit [options] <app jar | python file> [app arguments]
...
```
**After**
```
Usage: spark-submit [options] <app jar | python file | R file> [app arguments]
...
```
## How was this patch tested?
Manually tested.
Author: hyukjinkwon <gurwls223@gmail.com>
Closes #18505 from HyukjinKwon/minor-doc-summit.
---
.../scala/org/apache/spark/deploy/SparkSubmitArguments.scala | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala b/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala
index 3d9a14c516..7800d3d624 100644
--- a/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala
+++ b/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala
@@ -504,7 +504,7 @@ private[deploy] class SparkSubmitArguments(args: Seq[String], env: Map[String, S
outStream.println("Unknown/unsupported param " + unknownParam)
}
val command = sys.env.get("_SPARK_CMD_USAGE").getOrElse(
- """Usage: spark-submit [options] <app jar | python file> [app arguments]
+ """Usage: spark-submit [options] <app jar | python file | R file> [app arguments]
|Usage: spark-submit --kill [submission ID] --master [spark://...]
|Usage: spark-submit --status [submission ID] --master [spark://...]
|Usage: spark-submit run-example [options] example-class [example args]""".stripMargin)
--
GitLab