Skip to content
Snippets Groups Projects
Commit c0dbe08d authored by gatorsmile's avatar gatorsmile Committed by Herman van Hovell
Browse files

[SPARK-18415][SQL] Weird Plan Output when CTE used in RunnableCommand


### What changes were proposed in this pull request?
Currently, when CTE is used in RunnableCommand, the Analyzer does not replace the logical node `With`. The child plan of RunnableCommand is not resolved. Thus, the output of the `With` plan node looks very confusing.
For example,
```
sql(
  """
    |CREATE VIEW cte_view AS
    |WITH w AS (SELECT 1 AS n), cte1 (select 2), cte2 as (select 3)
    |SELECT n FROM w
  """.stripMargin).explain()
```
The output is like
```
ExecutedCommand
   +- CreateViewCommand `cte_view`, WITH w AS (SELECT 1 AS n), cte1 (select 2), cte2 as (select 3)
SELECT n FROM w, false, false, PersistedView
         +- 'With [(w,SubqueryAlias w
+- Project [1 AS n#16]
   +- OneRowRelation$
), (cte1,'SubqueryAlias cte1
+- 'Project [unresolvedalias(2, None)]
   +- OneRowRelation$
), (cte2,'SubqueryAlias cte2
+- 'Project [unresolvedalias(3, None)]
   +- OneRowRelation$
)]
            +- 'Project ['n]
               +- 'UnresolvedRelation `w`
```
After the fix, the output is as shown below.
```
ExecutedCommand
   +- CreateViewCommand `cte_view`, WITH w AS (SELECT 1 AS n), cte1 (select 2), cte2 as (select 3)
SELECT n FROM w, false, false, PersistedView
         +- CTE [w, cte1, cte2]
            :  :- SubqueryAlias w
            :  :  +- Project [1 AS n#16]
            :  :     +- OneRowRelation$
            :  :- 'SubqueryAlias cte1
            :  :  +- 'Project [unresolvedalias(2, None)]
            :  :     +- OneRowRelation$
            :  +- 'SubqueryAlias cte2
            :     +- 'Project [unresolvedalias(3, None)]
            :        +- OneRowRelation$
            +- 'Project ['n]
               +- 'UnresolvedRelation `w`
```

BTW, this PR also fixes the output of the view type.

### How was this patch tested?
Manual

Author: gatorsmile <gatorsmile@gmail.com>

Closes #15854 from gatorsmile/cteName.

(cherry picked from commit 608ecc51)
Signed-off-by: default avatarHerman van Hovell <hvanhovell@databricks.com>
parent b0ae8712
No related branches found
No related tags found
No related merge requests found
......@@ -26,6 +26,7 @@ import org.apache.spark.sql.catalyst.expressions._
import org.apache.spark.sql.catalyst.expressions.aggregate.AggregateExpression
import org.apache.spark.sql.catalyst.plans._
import org.apache.spark.sql.types._
import org.apache.spark.util.Utils
/**
* When planning take() or collect() operations, this special node that is inserted at the top of
......@@ -405,6 +406,13 @@ case class InsertIntoTable(
*/
case class With(child: LogicalPlan, cteRelations: Seq[(String, SubqueryAlias)]) extends UnaryNode {
override def output: Seq[Attribute] = child.output
override def simpleString: String = {
val cteAliases = Utils.truncatedString(cteRelations.map(_._1), "[", ", ", "]")
s"CTE $cteAliases"
}
override def innerChildren: Seq[QueryPlan[_]] = cteRelations.map(_._2)
}
case class WithWindowDefinition(
......
......@@ -33,7 +33,9 @@ import org.apache.spark.sql.types.MetadataBuilder
* ViewType is used to specify the expected view type when we want to create or replace a view in
* [[CreateViewCommand]].
*/
sealed trait ViewType
sealed trait ViewType {
override def toString: String = getClass.getSimpleName.stripSuffix("$")
}
/**
* LocalTempView means session-scoped local temporary views. Its lifetime is the lifetime of the
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment