-
- Downloads
[SPARK-17751][SQL] Remove spark.sql.eagerAnalysis and Output the Plan if...
[SPARK-17751][SQL] Remove spark.sql.eagerAnalysis and Output the Plan if Existed in AnalysisException ### What changes were proposed in this pull request? Dataset always does eager analysis now. Thus, `spark.sql.eagerAnalysis` is not used any more. Thus, we need to remove it. This PR also outputs the plan. Without the fix, the analysis error is like ``` cannot resolve '`k1`' given input columns: [k, v]; line 1 pos 12 ``` After the fix, the analysis error becomes: ``` org.apache.spark.sql.AnalysisException: cannot resolve '`k1`' given input columns: [k, v]; line 1 pos 12; 'Project [unresolvedalias(CASE WHEN ('k1 = 2) THEN 22 WHEN ('k1 = 4) THEN 44 ELSE 0 END, None), v#6] +- SubqueryAlias t +- Project [_1#2 AS k#5, _2#3 AS v#6] +- LocalRelation [_1#2, _2#3] ``` ### How was this patch tested? N/A Author: gatorsmile <gatorsmile@gmail.com> Closes #15316 from gatorsmile/eagerAnalysis.
Showing
- sql/catalyst/src/main/scala/org/apache/spark/sql/AnalysisException.scala 7 additions, 0 deletions...c/main/scala/org/apache/spark/sql/AnalysisException.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/debug/package.scala 0 additions, 9 deletions.../scala/org/apache/spark/sql/execution/debug/package.scala
- sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala 0 additions, 10 deletions...rc/main/scala/org/apache/spark/sql/internal/SQLConf.scala
- sql/core/src/test/scala/org/apache/spark/sql/SQLQueryTestSuite.scala 3 additions, 0 deletions...c/test/scala/org/apache/spark/sql/SQLQueryTestSuite.scala
Loading
Please register or sign in to comment