-
- Downloads
[SPARK-10747][SQL] Support NULLS FIRST|LAST clause in ORDER BY
## What changes were proposed in this pull request? Currently, ORDER BY clause returns nulls value according to sorting order (ASC|DESC), considering null value is always smaller than non-null values. However, SQL2003 standard support NULLS FIRST or NULLS LAST to allow users to specify whether null values should be returned first or last, regardless of sorting order (ASC|DESC). This PR is to support this new feature. ## How was this patch tested? New test cases are added to test NULLS FIRST|LAST for regular select queries and windowing queries. (If this patch involves UI changes, please attach a screenshot; otherwise, remove this) Author: Xin Wu <xinwu@us.ibm.com> Closes #14842 from xwu0226/SPARK-10747.
Showing
- core/src/main/java/org/apache/spark/util/collection/unsafe/sort/PrefixComparators.java 54 additions, 4 deletions.../spark/util/collection/unsafe/sort/PrefixComparators.java
- core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeInMemorySorter.java 6 additions, 5 deletions...ark/util/collection/unsafe/sort/UnsafeInMemorySorter.java
- core/src/test/scala/org/apache/spark/util/collection/unsafe/sort/RadixSortSuite.scala 21 additions, 6 deletions...he/spark/util/collection/unsafe/sort/RadixSortSuite.scala
- sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 5 additions, 2 deletions...in/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala 2 additions, 2 deletions...ala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/SubstituteUnresolvedOrdinals.scala 1 addition, 1 deletion.../sql/catalyst/analysis/SubstituteUnresolvedOrdinals.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/dsl/package.scala 2 additions, 1 deletion...ain/scala/org/apache/spark/sql/catalyst/dsl/package.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/SortOrder.scala 55 additions, 10 deletions...org/apache/spark/sql/catalyst/expressions/SortOrder.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateOrdering.scala 12 additions, 4 deletions...k/sql/catalyst/expressions/codegen/GenerateOrdering.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ordering.scala 3 additions, 3 deletions.../org/apache/spark/sql/catalyst/expressions/ordering.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala 11 additions, 3 deletions...ala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/SortPrefixUtils.scala 58 additions, 10 deletions...cala/org/apache/spark/sql/execution/SortPrefixUtils.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/SparkPlan.scala 1 addition, 1 deletion...main/scala/org/apache/spark/sql/execution/SparkPlan.scala
- sql/core/src/test/resources/sql-tests/inputs/orderby-nulls-ordering.sql 83 additions, 0 deletions...est/resources/sql-tests/inputs/orderby-nulls-ordering.sql
- sql/core/src/test/resources/sql-tests/results/orderby-nulls-ordering.sql.out 254 additions, 0 deletions...esources/sql-tests/results/orderby-nulls-ordering.sql.out
- sql/core/src/test/scala/org/apache/spark/sql/execution/SortSuite.scala 2 additions, 1 deletion...test/scala/org/apache/spark/sql/execution/SortSuite.scala
- sql/hive/src/test/resources/sqlgen/agg2.sql 1 addition, 1 deletionsql/hive/src/test/resources/sqlgen/agg2.sql
- sql/hive/src/test/resources/sqlgen/agg3.sql 1 addition, 1 deletionsql/hive/src/test/resources/sqlgen/agg3.sql
- sql/hive/src/test/resources/sqlgen/broadcast_join_subquery.sql 1 addition, 1 deletion...ive/src/test/resources/sqlgen/broadcast_join_subquery.sql
- sql/hive/src/test/resources/sqlgen/generate_with_other_1.sql 1 addition, 1 deletionsql/hive/src/test/resources/sqlgen/generate_with_other_1.sql
Loading
Please register or sign in to comment