-
- Downloads
SPARK-1093: Annotate developer and experimental API's
This patch marks some existing classes as private[spark] and adds two types of API annotations: - `EXPERIMENTAL API` = experimental user-facing module - `DEVELOPER API - UNSTABLE` = developer-facing API that might change There is some discussion of the different mechanisms for doing this here: https://issues.apache.org/jira/browse/SPARK-1081 I was pretty aggressive with marking things private. Keep in mind that if we want to open something up in the future we can, but we can never reduce visibility. A few notes here: - In the past we've been inconsistent with the visiblity of the X-RDD classes. This patch marks them private whenever there is an existing function in RDD that can directly creat them (e.g. CoalescedRDD and rdd.coalesce()). One trade-off here is users can't subclass them. - Noted that compression and serialization formats don't have to be wire compatible across versions. - Compression codecs and serialization formats are semi-private as users typically don't instantiate them directly. - Metrics sources are made private - user only interacts with them through Spark's reflection Author: Patrick Wendell <pwendell@gmail.com> Author: Andrew Or <andrewor14@gmail.com> Closes #274 from pwendell/private-apis and squashes the following commits: 44179e4 [Patrick Wendell] Merge remote-tracking branch 'apache-github/master' into private-apis 042c803 [Patrick Wendell] spark.annotations -> spark.annotation bfe7b52 [Patrick Wendell] Adding experimental for approximate counts 8d0c873 [Patrick Wendell] Warning in SparkEnv 99b223a [Patrick Wendell] Cleaning up annotations e849f64 [Patrick Wendell] Merge pull request #2 from andrewor14/annotations 982a473 [Andrew Or] Generalize jQuery matching for non Spark-core API docs a01c076 [Patrick Wendell] Merge pull request #1 from andrewor14/annotations c1bcb41 [Andrew Or] DeveloperAPI -> DeveloperApi 0d48908 [Andrew Or] Comments and new lines (minor) f3954e0 [Andrew Or] Add identifier tags in comments to work around scaladocs bug 99192ef [Andrew Or] Dynamically add badges based on annotations 824011b [Andrew Or] Add support for injecting arbitrary JavaScript to API docs 037755c [Patrick Wendell] Some changes after working with andrew or f7d124f [Patrick Wendell] Small fixes c318b24 [Patrick Wendell] Use CSS styles e4c76b9 [Patrick Wendell] Logging f390b13 [Patrick Wendell] Better visibility for workaround constructors d6b0afd [Patrick Wendell] Small chang to existing constructor 403ba52 [Patrick Wendell] Style fix 870a7ba [Patrick Wendell] Work around for SI-8479 7fb13b2 [Patrick Wendell] Changes to UnionRDD and EmptyRDD 4a9e90c [Patrick Wendell] EXPERIMENTAL API --> EXPERIMENTAL c581dce [Patrick Wendell] Changes after building against Shark. 8452309 [Patrick Wendell] Style fixes 1ed27d2 [Patrick Wendell] Formatting and coloring of badges cd7a465 [Patrick Wendell] Code review feedback 2f706f1 [Patrick Wendell] Don't use floats 542a736 [Patrick Wendell] Small fixes cf23ec6 [Patrick Wendell] Marking GraphX as alpha d86818e [Patrick Wendell] Another naming change 5a76ed6 [Patrick Wendell] More visiblity clean-up 42c1f09 [Patrick Wendell] Using better labels 9d48cbf [Patrick Wendell] Initial pass
Showing
- core/src/main/scala/org/apache/spark/Aggregator.scala 3 additions, 0 deletionscore/src/main/scala/org/apache/spark/Aggregator.scala
- core/src/main/scala/org/apache/spark/Dependency.scala 11 additions, 0 deletionscore/src/main/scala/org/apache/spark/Dependency.scala
- core/src/main/scala/org/apache/spark/FutureAction.scala 7 additions, 0 deletionscore/src/main/scala/org/apache/spark/FutureAction.scala
- core/src/main/scala/org/apache/spark/InterruptibleIterator.scala 1 addition, 1 deletion...c/main/scala/org/apache/spark/InterruptibleIterator.scala
- core/src/main/scala/org/apache/spark/Logging.scala 7 additions, 0 deletionscore/src/main/scala/org/apache/spark/Logging.scala
- core/src/main/scala/org/apache/spark/SerializableWritable.scala 3 additions, 0 deletions...rc/main/scala/org/apache/spark/SerializableWritable.scala
- core/src/main/scala/org/apache/spark/SparkContext.scala 73 additions, 13 deletionscore/src/main/scala/org/apache/spark/SparkContext.scala
- core/src/main/scala/org/apache/spark/SparkEnv.scala 7 additions, 1 deletioncore/src/main/scala/org/apache/spark/SparkEnv.scala
- core/src/main/scala/org/apache/spark/TaskContext.scala 6 additions, 0 deletionscore/src/main/scala/org/apache/spark/TaskContext.scala
- core/src/main/scala/org/apache/spark/TaskEndReason.scala 22 additions, 9 deletionscore/src/main/scala/org/apache/spark/TaskEndReason.scala
- core/src/main/scala/org/apache/spark/annotation/AlphaComponent.java 26 additions, 0 deletions...ain/scala/org/apache/spark/annotation/AlphaComponent.java
- core/src/main/scala/org/apache/spark/annotation/DeveloperApi.java 30 additions, 0 deletions.../main/scala/org/apache/spark/annotation/DeveloperApi.java
- core/src/main/scala/org/apache/spark/annotation/Experimental.java 31 additions, 0 deletions.../main/scala/org/apache/spark/annotation/Experimental.java
- core/src/main/scala/org/apache/spark/api/java/JavaDoubleRDD.scala 16 additions, 3 deletions.../main/scala/org/apache/spark/api/java/JavaDoubleRDD.scala
- core/src/main/scala/org/apache/spark/api/java/JavaPairRDD.scala 8 additions, 3 deletions...rc/main/scala/org/apache/spark/api/java/JavaPairRDD.scala
- core/src/main/scala/org/apache/spark/api/java/JavaRDDLike.scala 7 additions, 2 deletions...rc/main/scala/org/apache/spark/api/java/JavaRDDLike.scala
- core/src/main/scala/org/apache/spark/api/java/JavaSparkContext.scala 1 addition, 1 deletion...in/scala/org/apache/spark/api/java/JavaSparkContext.scala
- core/src/main/scala/org/apache/spark/api/java/JavaUtils.scala 1 addition, 1 deletion.../src/main/scala/org/apache/spark/api/java/JavaUtils.scala
- core/src/main/scala/org/apache/spark/broadcast/BroadcastFactory.scala 5 additions, 2 deletions...n/scala/org/apache/spark/broadcast/BroadcastFactory.scala
- core/src/main/scala/org/apache/spark/executor/ExecutorSource.scala 1 addition, 1 deletion...main/scala/org/apache/spark/executor/ExecutorSource.scala
Loading
Please register or sign in to comment