-
- Downloads
[SPARK-14670] [SQL] allow updating driver side sql metrics
## What changes were proposed in this pull request? On the SparkUI right now we have this SQLTab that displays accumulator values per operator. However, it only displays metrics updated on the executors, not on the driver. It is useful to also include driver metrics, e.g. broadcast time. This is a different version from https://github.com/apache/spark/pull/12427. This PR sends driver side accumulator updates right after the updating happens, not at the end of execution, by a new event. ## How was this patch tested? new test in `SQLListenerSuite`  Author: Wenchen Fan <wenchen@databricks.com> Closes #13189 from cloud-fan/metrics.
Showing
- sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/BroadcastExchangeExec.scala 9 additions, 0 deletions.../spark/sql/execution/exchange/BroadcastExchangeExec.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLListener.scala 23 additions, 5 deletions...scala/org/apache/spark/sql/execution/ui/SQLListener.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLListenerSuite.scala 53 additions, 3 deletions.../org/apache/spark/sql/execution/ui/SQLListenerSuite.scala
Loading
Please register or sign in to comment