-
- Downloads
[SPARK-18692][BUILD][DOCS] Test Java 8 unidoc build on Jenkins
## What changes were proposed in this pull request? This PR proposes to run Spark unidoc to test Javadoc 8 build as Javadoc 8 is easily re-breakable. There are several problems with it: - It introduces little extra bit of time to run the tests. In my case, it took 1.5 mins more (`Elapsed :[94.8746569157]`). How it was tested is described in "How was this patch tested?". - > One problem that I noticed was that Unidoc appeared to be processing test sources: if we can find a way to exclude those from being processed in the first place then that might significantly speed things up. (see joshrosen's [comment](https://issues.apache.org/jira/browse/SPARK-18692?focusedCommentId=15947627&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15947627)) To complete this automated build, It also suggests to fix existing Javadoc breaks / ones introduced by test codes as described above. There fixes are similar instances that previously fixed. Please refer https://github.com/apache/spark/pull/15999 and https://github.com/apache/spark/pull/16013 Note that this only fixes **errors** not **warnings**. Please see my observation https://github.com/apache/spark/pull/17389#issuecomment-288438704 for spurious errors by warnings. ## How was this patch tested? Manually via `jekyll build` for building tests. Also, tested via running `./dev/run-tests`. This was tested via manually adding `time.time()` as below: ```diff profiles_and_goals = build_profiles + sbt_goals print("[info] Building Spark unidoc (w/Hive 1.2.1) using SBT with these arguments: ", " ".join(profiles_and_goals)) + import time + st = time.time() exec_sbt(profiles_and_goals) + print("Elapsed :[%s]" % str(time.time() - st)) ``` produces ``` ... ======================================================================== Building Unidoc API Documentation ======================================================================== ... [info] Main Java API documentation successful. ... Elapsed :[94.8746569157] ... Author: hyukjinkwon <gurwls223@gmail.com> Closes #17477 from HyukjinKwon/SPARK-18692.
Showing
- core/src/main/scala/org/apache/spark/rpc/RpcEndpoint.scala 5 additions, 5 deletionscore/src/main/scala/org/apache/spark/rpc/RpcEndpoint.scala
- core/src/main/scala/org/apache/spark/rpc/RpcTimeout.scala 1 addition, 1 deletioncore/src/main/scala/org/apache/spark/rpc/RpcTimeout.scala
- core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala 2 additions, 2 deletions.../main/scala/org/apache/spark/scheduler/DAGScheduler.scala
- core/src/main/scala/org/apache/spark/scheduler/ExternalClusterManager.scala 1 addition, 1 deletion...a/org/apache/spark/scheduler/ExternalClusterManager.scala
- core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala 4 additions, 4 deletions.../scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala
- core/src/main/scala/org/apache/spark/storage/BlockManager.scala 1 addition, 1 deletion...rc/main/scala/org/apache/spark/storage/BlockManager.scala
- core/src/test/scala/org/apache/spark/AccumulatorSuite.scala 2 additions, 2 deletionscore/src/test/scala/org/apache/spark/AccumulatorSuite.scala
- core/src/test/scala/org/apache/spark/ExternalShuffleServiceSuite.scala 1 addition, 1 deletion.../scala/org/apache/spark/ExternalShuffleServiceSuite.scala
- core/src/test/scala/org/apache/spark/LocalSparkContext.scala 1 addition, 1 deletioncore/src/test/scala/org/apache/spark/LocalSparkContext.scala
- core/src/test/scala/org/apache/spark/scheduler/SchedulerIntegrationSuite.scala 2 additions, 2 deletions...rg/apache/spark/scheduler/SchedulerIntegrationSuite.scala
- core/src/test/scala/org/apache/spark/serializer/SerializerPropertiesSuite.scala 1 addition, 1 deletion...g/apache/spark/serializer/SerializerPropertiesSuite.scala
- dev/run-tests.py 15 additions, 0 deletionsdev/run-tests.py
- graphx/src/test/scala/org/apache/spark/graphx/LocalSparkContext.scala 1 addition, 1 deletion...est/scala/org/apache/spark/graphx/LocalSparkContext.scala
- mllib/src/main/scala/org/apache/spark/ml/classification/Classifier.scala 1 addition, 1 deletion...scala/org/apache/spark/ml/classification/Classifier.scala
- mllib/src/test/scala/org/apache/spark/ml/PipelineSuite.scala 6 additions, 2 deletionsmllib/src/test/scala/org/apache/spark/ml/PipelineSuite.scala
- mllib/src/test/scala/org/apache/spark/ml/feature/LSHTest.scala 8 additions, 4 deletions.../src/test/scala/org/apache/spark/ml/feature/LSHTest.scala
- mllib/src/test/scala/org/apache/spark/ml/param/ParamsSuite.scala 1 addition, 1 deletion...rc/test/scala/org/apache/spark/ml/param/ParamsSuite.scala
- mllib/src/test/scala/org/apache/spark/ml/tree/impl/TreeTests.scala 4 additions, 2 deletions.../test/scala/org/apache/spark/ml/tree/impl/TreeTests.scala
- mllib/src/test/scala/org/apache/spark/ml/util/DefaultReadWriteTest.scala 9 additions, 9 deletions...scala/org/apache/spark/ml/util/DefaultReadWriteTest.scala
- mllib/src/test/scala/org/apache/spark/ml/util/StopwatchSuite.scala 2 additions, 2 deletions.../test/scala/org/apache/spark/ml/util/StopwatchSuite.scala
Loading
Please register or sign in to comment