-
- Downloads
[SPARK-18053][SQL] compare unsafe and safe complex-type values correctly
## What changes were proposed in this pull request? In Spark SQL, some expression may output safe format values, e.g. `CreateArray`, `CreateStruct`, `Cast`, etc. When we compare 2 values, we should be able to compare safe and unsafe formats. The `GreaterThan`, `LessThan`, etc. in Spark SQL already handles it, but the `EqualTo` doesn't. This PR fixes it. ## How was this patch tested? new unit test and regression test Author: Wenchen Fan <wenchen@databricks.com> Closes #15929 from cloud-fan/type-aware. (cherry picked from commit 84284e8c) Signed-off-by:Herman van Hovell <hvanhovell@databricks.com>
Showing
- sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeRow.java 1 addition, 5 deletions.../org/apache/spark/sql/catalyst/expressions/UnsafeRow.java
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala 18 additions, 2 deletions...park/sql/catalyst/expressions/codegen/CodeGenerator.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala 4 additions, 28 deletions...rg/apache/spark/sql/catalyst/expressions/predicates.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/PredicateSuite.scala 29 additions, 0 deletions...pache/spark/sql/catalyst/expressions/PredicateSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala 7 additions, 0 deletions...e/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala
Loading
Please register or sign in to comment