-
- Downloads
[MINOR] Remove inappropriate type notation and extra anonymous closure within...
[MINOR] Remove inappropriate type notation and extra anonymous closure within functional transformations ## What changes were proposed in this pull request? This PR removes - Inappropriate type notations For example, from ```scala words.foreachRDD { (rdd: RDD[String], time: Time) => ... ``` to ```scala words.foreachRDD { (rdd, time) => ... ``` - Extra anonymous closure within functional transformations. For example, ```scala .map(item => { ... }) ``` which can be just simply as below: ```scala .map { item => ... } ``` and corrects some obvious style nits. ## How was this patch tested? This was tested after adding rules in `scalastyle-config.xml`, which ended up with not finding all perfectly. The rules applied were below: - For the first correction, ```xml <check customId="NoExtraClosure" level="error" class="org.scalastyle.file.RegexChecker" enabled="true"> <parameters><parameter name="regex">(?m)\.[a-zA-Z_][a-zA-Z0-9]*\(\s*[^,]+s*=>\s*\{[^\}]+\}\s*\)</parameter></parameters> </check> ``` ```xml <check customId="NoExtraClosure" level="error" class="org.scalastyle.file.RegexChecker" enabled="true"> <parameters><parameter name="regex">\.[a-zA-Z_][a-zA-Z0-9]*\s*[\{|\(]([^\n>,]+=>)?\s*\{([^()]|(?R))*\}^[,]</parameter></parameters> </check> ``` - For the second correction ```xml <check customId="TypeNotation" level="error" class="org.scalastyle.file.RegexChecker" enabled="true"> <parameters><parameter name="regex">\.[a-zA-Z_][a-zA-Z0-9]*\s*[\{|\(]\s*\([^):]*:R))*\}^[,]</parameter></parameters> </check> ``` **Those rules were not added** Author: hyukjinkwon <gurwls223@gmail.com> Closes #12413 from HyukjinKwon/SPARK-style.
Showing
- core/src/main/scala/org/apache/spark/TaskEndReason.scala 1 addition, 3 deletionscore/src/main/scala/org/apache/spark/TaskEndReason.scala
- core/src/main/scala/org/apache/spark/api/java/JavaRDDLike.scala 5 additions, 5 deletions...rc/main/scala/org/apache/spark/api/java/JavaRDDLike.scala
- core/src/main/scala/org/apache/spark/deploy/master/ui/ApplicationPage.scala 2 additions, 3 deletions...a/org/apache/spark/deploy/master/ui/ApplicationPage.scala
- core/src/main/scala/org/apache/spark/deploy/worker/DriverRunner.scala 5 additions, 2 deletions...n/scala/org/apache/spark/deploy/worker/DriverRunner.scala
- core/src/main/scala/org/apache/spark/rdd/CoalescedRDD.scala 4 additions, 4 deletionscore/src/main/scala/org/apache/spark/rdd/CoalescedRDD.scala
- core/src/main/scala/org/apache/spark/rdd/JdbcRDD.scala 2 additions, 2 deletionscore/src/main/scala/org/apache/spark/rdd/JdbcRDD.scala
- core/src/main/scala/org/apache/spark/rdd/ParallelCollectionRDD.scala 4 additions, 5 deletions...in/scala/org/apache/spark/rdd/ParallelCollectionRDD.scala
- examples/src/main/scala/org/apache/spark/examples/mllib/StreamingTestExample.scala 2 additions, 2 deletions...rg/apache/spark/examples/mllib/StreamingTestExample.scala
- examples/src/main/scala/org/apache/spark/examples/streaming/RecoverableNetworkWordCount.scala 3 additions, 5 deletions...park/examples/streaming/RecoverableNetworkWordCount.scala
- examples/src/main/scala/org/apache/spark/examples/streaming/SqlNetworkWordCount.scala 1 addition, 1 deletion...apache/spark/examples/streaming/SqlNetworkWordCount.scala
- mllib/src/main/scala/org/apache/spark/mllib/api/python/Word2VecModelWrapper.scala 3 additions, 1 deletion.../apache/spark/mllib/api/python/Word2VecModelWrapper.scala
- mllib/src/main/scala/org/apache/spark/mllib/evaluation/BinaryClassificationMetrics.scala 1 addition, 2 deletions.../spark/mllib/evaluation/BinaryClassificationMetrics.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala 3 additions, 3 deletions...ala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Projection.scala 1 addition, 3 deletions...rg/apache/spark/sql/catalyst/expressions/Projection.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala 4 additions, 6 deletions...a/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/joins/ShuffledHashJoin.scala 1 addition, 3 deletions...g/apache/spark/sql/execution/joins/ShuffledHashJoin.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/stat/StatFunctions.scala 1 addition, 1 deletion...a/org/apache/spark/sql/execution/stat/StatFunctions.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUDFs.scala 3 additions, 5 deletions...e/src/main/scala/org/apache/spark/sql/hive/hiveUDFs.scala
- streaming/src/main/scala/org/apache/spark/streaming/dstream/DStream.scala 8 additions, 8 deletions...in/scala/org/apache/spark/streaming/dstream/DStream.scala
Loading
Please register or sign in to comment