-
- Downloads
[SPARK-11047] Internal accumulators miss the internal flag when replaying...
[SPARK-11047] Internal accumulators miss the internal flag when replaying events in the history server Internal accumulators don't write the internal flag to event log. So on the history server Web UI, all accumulators are not internal. This causes incorrect peak execution memory and unwanted accumulator table displayed on the stage page. To fix it, I add the "internal" property of AccumulableInfo when writing the event log. Author: Carson Wang <carson.wang@intel.com> Closes #9061 from carsonwang/accumulableBug.
Showing
- core/src/main/scala/org/apache/spark/scheduler/AccumulableInfo.scala 9 additions, 0 deletions...in/scala/org/apache/spark/scheduler/AccumulableInfo.scala
- core/src/main/scala/org/apache/spark/util/JsonProtocol.scala 4 additions, 2 deletionscore/src/main/scala/org/apache/spark/util/JsonProtocol.scala
- core/src/test/scala/org/apache/spark/util/JsonProtocolSuite.scala 66 additions, 30 deletions.../test/scala/org/apache/spark/util/JsonProtocolSuite.scala
Loading
Please register or sign in to comment