Skip to content
Snippets Groups Projects
Commit 1dab63d8 authored by Tom Magrino's avatar Tom Magrino Committed by Sean Owen
Browse files

[SPARK-16837][SQL] TimeWindow incorrectly drops slideDuration in constructors

## What changes were proposed in this pull request?

Fix of incorrect arguments (dropping slideDuration and using windowDuration) in constructors for TimeWindow.

The JIRA this addresses is here: https://issues.apache.org/jira/browse/SPARK-16837

## How was this patch tested?

Added a test to TimeWindowSuite to check that the results of TimeWindow object apply and TimeWindow class constructor are equivalent.

Author: Tom Magrino <tmagrino@fb.com>

Closes #14441 from tmagrino/windowing-fix.
parent 36827dda
No related branches found
No related tags found
No related merge requests found
......@@ -45,12 +45,12 @@ case class TimeWindow(
slideDuration: Expression,
startTime: Expression) = {
this(timeColumn, TimeWindow.parseExpression(windowDuration),
TimeWindow.parseExpression(windowDuration), TimeWindow.parseExpression(startTime))
TimeWindow.parseExpression(slideDuration), TimeWindow.parseExpression(startTime))
}
def this(timeColumn: Expression, windowDuration: Expression, slideDuration: Expression) = {
this(timeColumn, TimeWindow.parseExpression(windowDuration),
TimeWindow.parseExpression(windowDuration), 0)
TimeWindow.parseExpression(slideDuration), 0)
}
def this(timeColumn: Expression, windowDuration: Expression) = {
......
......@@ -108,4 +108,16 @@ class TimeWindowSuite extends SparkFunSuite with ExpressionEvalHelper with Priva
TimeWindow.invokePrivate(parseExpression(Rand(123)))
}
}
test("SPARK-16837: TimeWindow.apply equivalent to TimeWindow constructor") {
val slideLength = "1 second"
for (windowLength <- Seq("10 second", "1 minute", "2 hours")) {
val applyValue = TimeWindow(Literal(10L), windowLength, slideLength, "0 seconds")
val constructed = new TimeWindow(Literal(10L),
Literal(windowLength),
Literal(slideLength),
Literal("0 seconds"))
assert(applyValue == constructed)
}
}
}
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment