Skip to content
Snippets Groups Projects
Commit f0315437 authored by Keuntae Park's avatar Keuntae Park Committed by Michael Armbrust
Browse files

[SPARK-8783] [SQL] CTAS with WITH clause does not work

Currently, CTESubstitution only handles the case that WITH is on the top of the plan.
I think it SHOULD handle the case that WITH is child of CTAS.
This patch simply changes 'match' to 'transform' for recursive search of WITH in the plan.

Author: Keuntae Park <sirpkt@apache.org>

Closes #7180 from sirpkt/SPARK-8783 and squashes the following commits:

e4428f0 [Keuntae Park] Merge remote-tracking branch 'upstream/master' into CTASwithWITH
1671c77 [Keuntae Park] WITH clause can be inside CTAS
parent 2b40365d
No related branches found
No related tags found
No related merge requests found
......@@ -85,7 +85,7 @@ class Analyzer(
*/
object CTESubstitution extends Rule[LogicalPlan] {
// TODO allow subquery to define CTE
def apply(plan: LogicalPlan): LogicalPlan = plan match {
def apply(plan: LogicalPlan): LogicalPlan = plan transform {
case With(child, relations) => substituteCTE(child, relations)
case other => other
}
......
......@@ -159,6 +159,24 @@ class SQLQuerySuite extends QueryTest {
checkAnswer(query, Row(1, 1) :: Nil)
}
test("CTAS with WITH clause") {
val df = Seq((1, 1)).toDF("c1", "c2")
df.registerTempTable("table1")
sql(
"""
|CREATE TABLE with_table1 AS
|WITH T AS (
| SELECT *
| FROM table1
|)
|SELECT *
|FROM T
""".stripMargin)
val query = sql("SELECT * FROM with_table1")
checkAnswer(query, Row(1, 1) :: Nil)
}
test("explode nested Field") {
Seq(NestedArray1(NestedArray2(Seq(1, 2, 3)))).toDF.registerTempTable("nestedArray")
checkAnswer(
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment