Skip to content
Snippets Groups Projects
Commit 2a23cdd0 authored by Josh Rosen's avatar Josh Rosen Committed by Wenchen Fan
Browse files

[SPARK-20863] Add metrics/instrumentation to LiveListenerBus

## What changes were proposed in this pull request?

This patch adds Coda Hale metrics for instrumenting the `LiveListenerBus` in order to track the number of events received, dropped, and processed. In addition, it adds per-SparkListener-subclass timers to track message processing time. This is useful for identifying when slow third-party SparkListeners cause performance bottlenecks.

See the new `LiveListenerBusMetrics` for a complete description of the new metrics.

## How was this patch tested?

New tests in SparkListenerSuite, including a test to ensure proper counting of dropped listener events.

Author: Josh Rosen <joshrosen@databricks.com>

Closes #18083 from JoshRosen/listener-bus-metrics.
parent 6e95897e
No related branches found
No related tags found
No related merge requests found
Showing
with 220 additions and 40 deletions
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment