Skip to content
Snippets Groups Projects
Commit 48d760d0 authored by Takeshi Yamamuro's avatar Takeshi Yamamuro Committed by Xiao Li
Browse files

[SPARK-20281][SQL] Print the identical Range parameters of SparkContext APIs and SQL in explain

## What changes were proposed in this pull request?
This pr modified code to print the identical `Range` parameters of SparkContext APIs and SQL in `explain` output. In the current master, they internally use `defaultParallelism` for `splits` by default though, they print different strings in explain output;

```
scala> spark.range(4).explain
== Physical Plan ==
*Range (0, 4, step=1, splits=Some(8))

scala> sql("select * from range(4)").explain
== Physical Plan ==
*Range (0, 4, step=1, splits=None)
```

## How was this patch tested?
Added tests in `SQLQuerySuite` and modified some results in the existing tests.

Author: Takeshi Yamamuro <yamamuro@apache.org>

Closes #17670 from maropu/SPARK-20281.
parent 760c8d08
No related branches found
No related tags found
No related merge requests found
......@@ -332,6 +332,7 @@ case class RangeExec(range: org.apache.spark.sql.catalyst.plans.logical.Range)
extends LeafExecNode with CodegenSupport {
def start: Long = range.start
def end: Long = range.end
def step: Long = range.step
def numSlices: Int = range.numSlices.getOrElse(sparkContext.defaultParallelism)
def numElements: BigInt = range.numElements
......@@ -538,7 +539,7 @@ case class RangeExec(range: org.apache.spark.sql.catalyst.plans.logical.Range)
}
}
override def simpleString: String = range.simpleString
override def simpleString: String = s"Range ($start, $end, step=$step, splits=$numSlices)"
}
/**
......
......@@ -88,7 +88,7 @@ Project [coalesce(cast(id#xL as string), x) AS ifnull(`id`, 'x')#x, id#xL AS nul
== Physical Plan ==
*Project [coalesce(cast(id#xL as string), x) AS ifnull(`id`, 'x')#x, id#xL AS nullif(`id`, 'x')#xL, coalesce(cast(id#xL as string), x) AS nvl(`id`, 'x')#x, x AS nvl2(`id`, 'x', 'y')#x]
+- *Range (0, 2, step=1, splits=None)
+- *Range (0, 2, step=1, splits=2)
-- !query 9
......
......@@ -102,4 +102,4 @@ EXPLAIN select * from RaNgE(2)
struct<plan:string>
-- !query 8 output
== Physical Plan ==
*Range (0, 2, step=1, splits=None)
*Range (0, 2, step=1, splits=2)
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment