-
- Downloads
[SPARK-21580][SQL] Integers in aggregation expressions are wrongly taken as group-by ordinal
## What changes were proposed in this pull request? create temporary view data as select * from values (1, 1), (1, 2), (2, 1), (2, 2), (3, 1), (3, 2) as data(a, b); `select 3, 4, sum(b) from data group by 1, 2;` `select 3 as c, 4 as d, sum(b) from data group by c, d;` When running these two cases, the following exception occurred: `Error in query: GROUP BY position 4 is not in select list (valid range is [1, 3]); line 1 pos 10` The cause of this failure: If an aggregateExpression is integer, after replaced with this aggregateExpression, the groupExpression still considered as an ordinal. The solution: This bug is due to re-entrance of an analyzed plan. We can solve it by using `resolveOperators` in `SubstituteUnresolvedOrdinals`. ## How was this patch tested? Added unit test case Author: liuxian <liu.xian3@zte.com.cn> Closes #18779 from 10110346/groupby.
Showing
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/SubstituteUnresolvedOrdinals.scala 1 addition, 1 deletion.../sql/catalyst/analysis/SubstituteUnresolvedOrdinals.scala
- sql/core/src/test/resources/sql-tests/inputs/group-by-ordinal.sql 1 addition, 1 deletion.../src/test/resources/sql-tests/inputs/group-by-ordinal.sql
- sql/core/src/test/scala/org/apache/spark/sql/DataFrameAggregateSuite.scala 16 additions, 0 deletions.../scala/org/apache/spark/sql/DataFrameAggregateSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala 6 additions, 0 deletions.../src/test/scala/org/apache/spark/sql/DataFrameSuite.scala
Loading
Please register or sign in to comment