Skip to content
Snippets Groups Projects
Commit 90787de8 authored by Wenchen Fan's avatar Wenchen Fan Committed by Reynold Xin
Browse files

[SPARK-15033][SQL] fix a flaky test in CachedTableSuite

## What changes were proposed in this pull request?

This is caused by https://github.com/apache/spark/pull/12776, which removes the `synchronized` from all methods in `AccumulatorContext`.

However, a test in `CachedTableSuite` synchronize on `AccumulatorContext` and expecting no one else can change it, which is not true anymore.

This PR update that test to not require to lock on `AccumulatorContext`.

## How was this patch tested?

N/A

Author: Wenchen Fan <wenchen@databricks.com>

Closes #12811 from cloud-fan/flaky.
parent 507bea5c
No related branches found
No related tags found
No related merge requests found
......@@ -68,7 +68,7 @@ private[sql] case class InMemoryRelation(
override def producedAttributes: AttributeSet = outputSet
private val batchStats: Accumulable[ArrayBuffer[InternalRow], InternalRow] =
private[sql] val batchStats: Accumulable[ArrayBuffer[InternalRow], InternalRow] =
if (_batchStats == null) {
child.sqlContext.sparkContext.accumulableCollection(ArrayBuffer.empty[InternalRow])
} else {
......
......@@ -333,12 +333,19 @@ class CachedTableSuite extends QueryTest with SQLTestUtils with SharedSQLContext
sql("SELECT * FROM t1").count()
sql("SELECT * FROM t2").count()
AccumulatorContext.synchronized {
val accsSize = AccumulatorContext.numAccums
sqlContext.uncacheTable("t1")
sqlContext.uncacheTable("t2")
assert((accsSize - 2) == AccumulatorContext.numAccums)
}
val accId1 = sqlContext.table("t1").queryExecution.withCachedData.collect {
case i: InMemoryRelation => i.batchStats.id
}.head
val accId2 = sqlContext.table("t1").queryExecution.withCachedData.collect {
case i: InMemoryRelation => i.batchStats.id
}.head
sqlContext.uncacheTable("t1")
sqlContext.uncacheTable("t2")
assert(AccumulatorContext.get(accId1).isEmpty)
assert(AccumulatorContext.get(accId2).isEmpty)
}
test("SPARK-10327 Cache Table is not working while subquery has alias in its project list") {
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment