-
- Downloads
[HOTFIX][CORE] fix a concurrence issue in NewAccumulator
## What changes were proposed in this pull request? `AccumulatorContext` is not thread-safe, that's why all of its methods are synchronized. However, there is one exception: the `AccumulatorContext.originals`. `NewAccumulator` use it to check if it's registered, which is wrong as it's not synchronized. This PR mark `AccumulatorContext.originals` as `private` and now all access to `AccumulatorContext` is synchronized. ## How was this patch tested? I verified it locally. To be safe, we can let jenkins test it many times to make sure this problem is gone. Author: Wenchen Fan <wenchen@databricks.com> Closes #12773 from cloud-fan/debug.
Showing
- core/src/main/scala/org/apache/spark/NewAccumulator.scala 8 additions, 2 deletionscore/src/main/scala/org/apache/spark/NewAccumulator.scala
- core/src/test/scala/org/apache/spark/AccumulatorSuite.scala 1 addition, 1 deletioncore/src/test/scala/org/apache/spark/AccumulatorSuite.scala
- core/src/test/scala/org/apache/spark/InternalAccumulatorSuite.scala 3 additions, 3 deletions...est/scala/org/apache/spark/InternalAccumulatorSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/CachedTableSuite.scala 2 additions, 2 deletions...rc/test/scala/org/apache/spark/sql/CachedTableSuite.scala
Loading
Please register or sign in to comment