-
- Downloads
[SPARK-17671][WEBUI] Spark 2.0 history server summary page is slow even set...
[SPARK-17671][WEBUI] Spark 2.0 history server summary page is slow even set spark.history.ui.maxApplications ## What changes were proposed in this pull request? Return Iterator of applications internally in history server, for consistency and performance. See https://github.com/apache/spark/pull/15248 for some back-story. The code called by and calling HistoryServer.getApplicationList wants an Iterator, but this method materializes an Iterable, which potentially causes a performance problem. It's simpler too to make this internal method also pass through an Iterator. ## How was this patch tested? Existing tests. Author: Sean Owen <sowen@cloudera.com> Closes #15321 from srowen/SPARK-17671.
Showing
- core/src/main/scala/org/apache/spark/deploy/history/ApplicationHistoryProvider.scala 1 addition, 1 deletion...che/spark/deploy/history/ApplicationHistoryProvider.scala
- core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala 1 addition, 1 deletion...a/org/apache/spark/deploy/history/FsHistoryProvider.scala
- core/src/main/scala/org/apache/spark/deploy/history/HistoryPage.scala 1 addition, 4 deletions...n/scala/org/apache/spark/deploy/history/HistoryPage.scala
- core/src/main/scala/org/apache/spark/deploy/history/HistoryServer.scala 2 additions, 2 deletions...scala/org/apache/spark/deploy/history/HistoryServer.scala
- core/src/main/scala/org/apache/spark/status/api/v1/ApplicationListResource.scala 13 additions, 25 deletions.../apache/spark/status/api/v1/ApplicationListResource.scala
- core/src/test/scala/org/apache/spark/deploy/history/HistoryServerSuite.scala 2 additions, 2 deletions.../org/apache/spark/deploy/history/HistoryServerSuite.scala
- project/MimaExcludes.scala 2 additions, 0 deletionsproject/MimaExcludes.scala
Loading
Please register or sign in to comment