-
- Downloads
[SPARK-19544][SQL] Improve error message when some column types are compatible...
[SPARK-19544][SQL] Improve error message when some column types are compatible and others are not in set operations ## What changes were proposed in this pull request? This PR proposes to fix the error message when some data types are compatible and others are not in set/union operation. Currently, the code below: ```scala Seq((1,("a", 1))).toDF.union(Seq((1L,("a", "b"))).toDF) ``` throws an exception saying `LongType` and `IntegerType` are incompatible types. It should say something about `StructType`s with more readable format as below: **Before** ``` Union can only be performed on tables with the compatible column types. LongType <> IntegerType at the first column of the second table;; ``` **After** ``` Union can only be performed on tables with the compatible column types. struct<_1:string,_2:string> <> struct<_1:string,_2:int> at the second column of the second table;; ``` *I manually inserted a newline in the messages above for readability only in this PR description. ## How was this patch tested? Unit tests in `AnalysisErrorSuite`, manual tests and build wth Scala 2.10. Author: hyukjinkwon <gurwls223@gmail.com> Closes #16882 from HyukjinKwon/SPARK-19544.
Showing
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala 3 additions, 3 deletions...rg/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala 13 additions, 11 deletions...org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisErrorSuite.scala 15 additions, 0 deletions...ache/spark/sql/catalyst/analysis/AnalysisErrorSuite.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/TestRelations.scala 7 additions, 0 deletions...rg/apache/spark/sql/catalyst/analysis/TestRelations.scala
Loading
Please register or sign in to comment