-
- Downloads
[SPARK-15703][SCHEDULER][CORE][WEBUI] Make ListenerBus event queue size configurable
## What changes were proposed in this pull request? This change adds a new configuration entry to specify the size of the spark listener bus event queue. The value for this config ("spark.scheduler.listenerbus.eventqueue.size") is set to a default to 10000. Note: I haven't currently documented the configuration entry. We can decide whether it would be appropriate to make it a public configuration or keep it as an undocumented one. Refer JIRA for more details. ## How was this patch tested? Ran existing jobs and verified the event queue size with debug logs and from the Spark WebUI Environment tab. Author: Dhruve Ashar <dhruveashar@gmail.com> Closes #14269 from dhruve/bug/SPARK-15703.
Showing
- core/src/main/scala/org/apache/spark/SparkContext.scala 2 additions, 2 deletionscore/src/main/scala/org/apache/spark/SparkContext.scala
- core/src/main/scala/org/apache/spark/internal/config/package.scala 5 additions, 0 deletions...main/scala/org/apache/spark/internal/config/package.scala
- core/src/main/scala/org/apache/spark/scheduler/LiveListenerBus.scala 14 additions, 9 deletions...in/scala/org/apache/spark/scheduler/LiveListenerBus.scala
- core/src/test/scala/org/apache/spark/scheduler/EventLoggingListenerSuite.scala 2 additions, 2 deletions...rg/apache/spark/scheduler/EventLoggingListenerSuite.scala
- core/src/test/scala/org/apache/spark/scheduler/SparkListenerSuite.scala 16 additions, 14 deletions...scala/org/apache/spark/scheduler/SparkListenerSuite.scala
- core/src/test/scala/org/apache/spark/storage/BlockManagerReplicationSuite.scala 7 additions, 2 deletions...g/apache/spark/storage/BlockManagerReplicationSuite.scala
- core/src/test/scala/org/apache/spark/storage/BlockManagerSuite.scala 4 additions, 2 deletions...st/scala/org/apache/spark/storage/BlockManagerSuite.scala
- core/src/test/scala/org/apache/spark/ui/storage/StorageTabSuite.scala 6 additions, 5 deletions...t/scala/org/apache/spark/ui/storage/StorageTabSuite.scala
- streaming/src/test/scala/org/apache/spark/streaming/ReceivedBlockHandlerSuite.scala 4 additions, 1 deletion...rg/apache/spark/streaming/ReceivedBlockHandlerSuite.scala
Loading
Please register or sign in to comment