Skip to content
Snippets Groups Projects
Commit 985d5328 authored by Sean Zhong's avatar Sean Zhong Committed by Cheng Lian
Browse files

[SPARK-15734][SQL] Avoids printing internal row in explain output

## What changes were proposed in this pull request?

This PR avoids printing internal rows in explain output for some operators.

**Before change:**

```
scala> (1 to 10).toSeq.map(_ => (1,2,3)).toDF().createTempView("df3")
scala> spark.sql("select * from df3 where 1=2").explain(true)
...
== Analyzed Logical Plan ==
_1: int, _2: int, _3: int
Project [_1#37,_2#38,_3#39]
+- Filter (1 = 2)
   +- SubqueryAlias df3
      +- LocalRelation [_1#37,_2#38,_3#39], [[0,1,2,3],[0,1,2,3],[0,1,2,3],[0,1,2,3],[0,1,2,3],[0,1,2,3],[0,1,2,3],[0,1,2,3],[0,1,2,3],[0,1,2,3]]
...
== Physical Plan ==
LocalTableScan [_1#37,_2#38,_3#39]
```

**After change:**

```
scala> spark.sql("select * from df3 where 1=2").explain(true)
...
== Analyzed Logical Plan ==
_1: int, _2: int, _3: int
Project [_1#58,_2#59,_3#60]
+- Filter (1 = 2)
   +- SubqueryAlias df3
      +- LocalRelation [_1#58,_2#59,_3#60]
...
== Physical Plan ==
LocalTableScan <empty>, [_1#58,_2#59,_3#60]
```

## How was this patch tested?
Manual test.

Author: Sean Zhong <seanzhong@databricks.com>

Closes #13471 from clockfly/verbose_breakdown_5.
parent 43154276
No related branches found
No related tags found
No related merge requests found
......@@ -57,7 +57,13 @@ case class LocalRelation(output: Seq[Attribute], data: Seq[InternalRow] = Nil)
LocalRelation(output.map(_.newInstance()), data).asInstanceOf[this.type]
}
override protected def stringArgs = Iterator(output)
override protected def stringArgs: Iterator[Any] = {
if (data.isEmpty) {
Iterator("<empty>", output)
} else {
Iterator(output)
}
}
override def sameResult(plan: LogicalPlan): Boolean = plan match {
case LocalRelation(otherOutput, otherData) =>
......
......@@ -91,6 +91,8 @@ private[sql] case class LogicalRDD(
case _ => false
}
override protected def stringArgs: Iterator[Any] = Iterator(output)
override def producedAttributes: AttributeSet = outputSet
@transient override lazy val statistics: Statistics = Statistics(
......
......@@ -48,6 +48,14 @@ private[sql] case class LocalTableScanExec(
}
}
override protected def stringArgs: Iterator[Any] = {
if (rows.isEmpty) {
Iterator("<empty>", output)
} else {
Iterator(output)
}
}
override def executeCollect(): Array[InternalRow] = {
unsafeRows
}
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment