Skip to content
Snippets Groups Projects
Commit 0f37d1d7 authored by Liang-Chi Hsieh's avatar Liang-Chi Hsieh Committed by Yin Huai
Browse files

[SPARK-11949][SQL] Check bitmasks to set nullable property

Following up #10038.

We can use bitmasks to determine which grouping expressions need to be set as nullable.

cc yhuai

Author: Liang-Chi Hsieh <viirya@appier.com>

Closes #10067 from viirya/fix-cube-following.
parent 8a75a304
No related branches found
No related tags found
No related merge requests found
...@@ -224,10 +224,15 @@ class Analyzer( ...@@ -224,10 +224,15 @@ class Analyzer(
case other => Alias(other, other.toString)() case other => Alias(other, other.toString)()
} }
// TODO: We need to use bitmasks to determine which grouping expressions need to be val nonNullBitmask = x.bitmasks.reduce(_ & _)
// set as nullable. For example, if we have GROUPING SETS ((a,b), a), we do not need
// to change the nullability of a. val attributeMap = groupByAliases.zipWithIndex.map { case (a, idx) =>
val attributeMap = groupByAliases.map(a => (a -> a.toAttribute.withNullability(true))).toMap if ((nonNullBitmask & 1 << idx) == 0) {
(a -> a.toAttribute.withNullability(true))
} else {
(a -> a.toAttribute)
}
}.toMap
val aggregations: Seq[NamedExpression] = x.aggregations.map { val aggregations: Seq[NamedExpression] = x.aggregations.map {
// If an expression is an aggregate (contains a AggregateExpression) then we dont change // If an expression is an aggregate (contains a AggregateExpression) then we dont change
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment