-
- Downloads
[SPARK-14628][CORE][FOLLLOW-UP] Always tracking read/write metrics
## What changes were proposed in this pull request? This PR is a follow up for https://github.com/apache/spark/pull/12417, now we always track input/output/shuffle metrics in spark JSON protocol and status API. Most of the line changes are because of re-generating the gold answer for `HistoryServerSuite`, and we add a lot of 0 values for read/write metrics. ## How was this patch tested? existing tests. Author: Wenchen Fan <wenchen@databricks.com> Closes #12462 from cloud-fan/follow.
Showing
- core/src/main/scala/org/apache/spark/executor/InputMetrics.scala 0 additions, 5 deletions...c/main/scala/org/apache/spark/executor/InputMetrics.scala
- core/src/main/scala/org/apache/spark/executor/OutputMetrics.scala 0 additions, 5 deletions.../main/scala/org/apache/spark/executor/OutputMetrics.scala
- core/src/main/scala/org/apache/spark/executor/ShuffleReadMetrics.scala 0 additions, 5 deletions.../scala/org/apache/spark/executor/ShuffleReadMetrics.scala
- core/src/main/scala/org/apache/spark/executor/ShuffleWriteMetrics.scala 0 additions, 5 deletions...scala/org/apache/spark/executor/ShuffleWriteMetrics.scala
- core/src/main/scala/org/apache/spark/status/api/v1/AllStagesResource.scala 45 additions, 93 deletions...la/org/apache/spark/status/api/v1/AllStagesResource.scala
- core/src/main/scala/org/apache/spark/status/api/v1/api.scala 9 additions, 9 deletionscore/src/main/scala/org/apache/spark/status/api/v1/api.scala
- core/src/main/scala/org/apache/spark/util/JsonProtocol.scala 5 additions, 17 deletionscore/src/main/scala/org/apache/spark/util/JsonProtocol.scala
- core/src/test/resources/HistoryServerExpectations/application_list_json_expectation.json 32 additions, 32 deletions...ServerExpectations/application_list_json_expectation.json
- core/src/test/resources/HistoryServerExpectations/completed_app_list_json_expectation.json 32 additions, 32 deletions...rverExpectations/completed_app_list_json_expectation.json
- core/src/test/resources/HistoryServerExpectations/executor_list_json_expectation.json 1 addition, 1 deletion...oryServerExpectations/executor_list_json_expectation.json
- core/src/test/resources/HistoryServerExpectations/job_list_from_multi_attempt_app_json_1__expectation.json 1 addition, 1 deletion.../job_list_from_multi_attempt_app_json_1__expectation.json
- core/src/test/resources/HistoryServerExpectations/job_list_from_multi_attempt_app_json_2__expectation.json 1 addition, 1 deletion.../job_list_from_multi_attempt_app_json_2__expectation.json
- core/src/test/resources/HistoryServerExpectations/job_list_json_expectation.json 1 addition, 1 deletion.../HistoryServerExpectations/job_list_json_expectation.json
- core/src/test/resources/HistoryServerExpectations/maxDate2_app_list_json_expectation.json 4 additions, 4 deletions...erverExpectations/maxDate2_app_list_json_expectation.json
- core/src/test/resources/HistoryServerExpectations/maxDate_app_list_json_expectation.json 8 additions, 8 deletions...ServerExpectations/maxDate_app_list_json_expectation.json
- core/src/test/resources/HistoryServerExpectations/minDate_app_list_json_expectation.json 34 additions, 36 deletions...ServerExpectations/minDate_app_list_json_expectation.json
- core/src/test/resources/HistoryServerExpectations/one_app_json_expectation.json 4 additions, 4 deletions...s/HistoryServerExpectations/one_app_json_expectation.json
- core/src/test/resources/HistoryServerExpectations/one_app_multi_attempt_json_expectation.json 8 additions, 8 deletions...rExpectations/one_app_multi_attempt_json_expectation.json
- core/src/test/resources/HistoryServerExpectations/one_job_json_expectation.json 1 addition, 1 deletion...s/HistoryServerExpectations/one_job_json_expectation.json
- core/src/test/resources/HistoryServerExpectations/one_rdd_storage_json_expectation.json 0 additions, 64 deletions...yServerExpectations/one_rdd_storage_json_expectation.json
Loading
Please register or sign in to comment