-
- Downloads
[SPARK-21608][SPARK-9221][SQL] Window rangeBetween() API should allow literal boundary
## What changes were proposed in this pull request? Window rangeBetween() API should allow literal boundary, that means, the window range frame can calculate frame of double/date/timestamp. Example of the use case can be: ``` SELECT val_timestamp, cate, avg(val_timestamp) OVER(PARTITION BY cate ORDER BY val_timestamp RANGE BETWEEN CURRENT ROW AND interval 23 days 4 hours FOLLOWING) FROM testData ``` This PR refactors the Window `rangeBetween` and `rowsBetween` API, while the legacy user code should still be valid. ## How was this patch tested? Add new test cases both in `DataFrameWindowFunctionsSuite` and in `window.sql`. Author: Xingbo Jiang <xingbo.jiang@databricks.com> Closes #18814 from jiangxb1987/literal-boundary.
Showing
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala 6 additions, 3 deletions...org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/windowExpressions.scala 6 additions, 2 deletions...he/spark/sql/catalyst/expressions/windowExpressions.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/window/WindowExec.scala 8 additions, 1 deletion...la/org/apache/spark/sql/execution/window/WindowExec.scala
- sql/core/src/main/scala/org/apache/spark/sql/expressions/Window.scala 57 additions, 6 deletions.../main/scala/org/apache/spark/sql/expressions/Window.scala
- sql/core/src/main/scala/org/apache/spark/sql/expressions/WindowSpec.scala 61 additions, 8 deletions...n/scala/org/apache/spark/sql/expressions/WindowSpec.scala
- sql/core/src/main/scala/org/apache/spark/sql/functions.scala 27 additions, 0 deletionssql/core/src/main/scala/org/apache/spark/sql/functions.scala
- sql/core/src/test/resources/sql-tests/inputs/window.sql 18 additions, 4 deletionssql/core/src/test/resources/sql-tests/inputs/window.sql
- sql/core/src/test/resources/sql-tests/results/window.sql.out 100 additions, 41 deletionssql/core/src/test/resources/sql-tests/results/window.sql.out
- sql/core/src/test/scala/org/apache/spark/sql/DataFrameWindowFunctionsSuite.scala 52 additions, 1 deletion.../org/apache/spark/sql/DataFrameWindowFunctionsSuite.scala
Loading
Please register or sign in to comment