Skip to content
Snippets Groups Projects
Commit beb8bc1e authored by zsxwing's avatar zsxwing Committed by Josh Rosen
Browse files

[SPARK-11126][SQL] Fix the potential flaky test

The unit test added in #9132 is flaky. This is a follow up PR to add `listenerBus.waitUntilEmpty` to fix it.

Author: zsxwing <zsxwing@gmail.com>

Closes #9163 from zsxwing/SPARK-11126-follow-up.
parent d3180c25
No related branches found
No related tags found
No related merge requests found
......@@ -316,10 +316,12 @@ class SQLListenerSuite extends SparkFunSuite with SharedSQLContext {
test("SPARK-11126: no memory leak when running non SQL jobs") {
val previousStageNumber = sqlContext.listener.stageIdToStageMetrics.size
sqlContext.sparkContext.parallelize(1 to 10).foreach(i => ())
sqlContext.sparkContext.listenerBus.waitUntilEmpty(10000)
// listener should ignore the non SQL stage
assert(sqlContext.listener.stageIdToStageMetrics.size == previousStageNumber)
sqlContext.sparkContext.parallelize(1 to 10).toDF().foreach(i => ())
sqlContext.sparkContext.listenerBus.waitUntilEmpty(10000)
// listener should save the SQL stage
assert(sqlContext.listener.stageIdToStageMetrics.size == previousStageNumber + 1)
}
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment