Skip to content
Snippets Groups Projects
Commit 5b8b324f authored by Liang-Chi Hsieh's avatar Liang-Chi Hsieh Committed by Michael Armbrust
Browse files

[SPARK-6303][SQL] Remove unnecessary Average in GeneratedAggregate

Because `Average` is a `PartialAggregate`, we never get a `Average` node when reaching `HashAggregation` to prepare `GeneratedAggregate`.

That is why in SQLQuerySuite there is already a test for `avg` with codegen. And it works.

But we can find a case in `GeneratedAggregate` to deal with `Average`. Based on the above, we actually never execute this case.

So we can remove this case from `GeneratedAggregate`.

Author: Liang-Chi Hsieh <viirya@gmail.com>

Closes #4996 from viirya/add_average_codegened and squashes the following commits:

621c12f [Liang-Chi Hsieh] Merge remote-tracking branch 'upstream/master' into add_average_codegened
368cfbc [Liang-Chi Hsieh] Merge remote-tracking branch 'upstream/master' into add_average_codegened
74926d1 [Liang-Chi Hsieh] Add Average in canBeCodeGened lists.
parent d7f2c198
No related branches found
No related tags found
No related merge requests found
......@@ -153,51 +153,6 @@ case class GeneratedAggregate(
AggregateEvaluation(currentSum :: Nil, initialValue :: Nil, updateFunction :: Nil, result)
case a @ Average(expr) =>
val calcType =
expr.dataType match {
case DecimalType.Fixed(_, _) =>
DecimalType.Unlimited
case _ =>
expr.dataType
}
val currentCount = AttributeReference("currentCount", LongType, nullable = false)()
val currentSum = AttributeReference("currentSum", calcType, nullable = false)()
val initialCount = Literal(0L)
val initialSum = Cast(Literal(0L), calcType)
// If we're evaluating UnscaledValue(x), we can do Count on x directly, since its
// UnscaledValue will be null if and only if x is null; helps with Average on decimals
val toCount = expr match {
case UnscaledValue(e) => e
case _ => expr
}
val updateCount = If(IsNotNull(toCount), Add(currentCount, Literal(1L)), currentCount)
val updateSum = Coalesce(Add(Cast(expr, calcType), currentSum) :: currentSum :: Nil)
val result =
expr.dataType match {
case DecimalType.Fixed(_, _) =>
If(EqualTo(currentCount, Literal(0L)),
Literal.create(null, a.dataType),
Cast(Divide(
Cast(currentSum, DecimalType.Unlimited),
Cast(currentCount, DecimalType.Unlimited)), a.dataType))
case _ =>
If(EqualTo(currentCount, Literal(0L)),
Literal.create(null, a.dataType),
Divide(Cast(currentSum, a.dataType), Cast(currentCount, a.dataType)))
}
AggregateEvaluation(
currentCount :: currentSum :: Nil,
initialCount :: initialSum :: Nil,
updateCount :: updateSum :: Nil,
result
)
case m @ Max(expr) =>
val currentMax = AttributeReference("currentMax", expr.dataType, nullable = true)()
val initialValue = Literal.create(null, expr.dataType)
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment