-
- Downloads
[SPARK-19509][SQL] Grouping Sets do not respect nullable grouping columns
## What changes were proposed in this pull request? The analyzer currently does not check if a column used in grouping sets is actually nullable itself. This can cause the nullability of the column to be incorrect, which can cause null pointer exceptions down the line. This PR fixes that by also consider the nullability of the column. This is only a problem for Spark 2.1 and below. The latest master uses a different approach. Closes https://github.com/apache/spark/pull/16874 ## How was this patch tested? Added a regression test to `SQLQueryTestSuite.grouping_set`. Author: Herman van Hovell <hvanhovell@databricks.com> Closes #16873 from hvanhovell/SPARK-19509.
Showing
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala 2 additions, 1 deletion...ala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
- sql/core/src/test/resources/sql-tests/inputs/grouping_set.sql 10 additions, 2 deletions...core/src/test/resources/sql-tests/inputs/grouping_set.sql
- sql/core/src/test/resources/sql-tests/results/grouping_set.sql.out 44 additions, 9 deletions...src/test/resources/sql-tests/results/grouping_set.sql.out
Loading
Please register or sign in to comment