[SPARK-21344][SQL] BinaryType comparison does signed byte array comparison
## What changes were proposed in this pull request? This PR fixes a wrong comparison for `BinaryType`. This PR enables unsigned comparison and unsigned prefix generation for an array for `BinaryType`. Previous implementations uses signed operations. ## How was this patch tested? Added a test suite in `OrderingSuite`. Author: Kazuaki Ishizaki <ishizaki@jp.ibm.com> Closes #18571 from kiszk/SPARK-21344. (cherry picked from commit ac5d5d79) Signed-off-by:gatorsmile <gatorsmile@gmail.com>
Showing
- common/unsafe/src/main/java/org/apache/spark/unsafe/types/ByteArray.java 1 addition, 1 deletion...rc/main/java/org/apache/spark/unsafe/types/ByteArray.java
- core/src/test/scala/org/apache/spark/util/collection/unsafe/sort/PrefixComparatorsSuite.scala 16 additions, 1 deletion.../util/collection/unsafe/sort/PrefixComparatorsSuite.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/TypeUtils.scala 3 additions, 1 deletion.../scala/org/apache/spark/sql/catalyst/util/TypeUtils.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/OrderingSuite.scala 19 additions, 0 deletions...apache/spark/sql/catalyst/expressions/OrderingSuite.scala
- sql/core/src/test/resources/sql-tests/inputs/comparator.sql 3 additions, 0 deletionssql/core/src/test/resources/sql-tests/inputs/comparator.sql
- sql/core/src/test/resources/sql-tests/results/comparator.sql.out 18 additions, 0 deletions...e/src/test/resources/sql-tests/results/comparator.sql.out
Loading
Please register or sign in to comment