-
- Downloads
[SPARK-13423][WIP][CORE][SQL][STREAMING] Static analysis fixes for 2.x
## What changes were proposed in this pull request? Make some cross-cutting code improvements according to static analysis. These are individually up for discussion since they exist in separate commits that can be reverted. The changes are broadly: - Inner class should be static - Mismatched hashCode/equals - Overflow in compareTo - Unchecked warnings - Misuse of assert, vs junit.assert - get(a) + getOrElse(b) -> getOrElse(a,b) - Array/String .size -> .length (occasionally, -> .isEmpty / .nonEmpty) to avoid implicit conversions - Dead code - tailrec - exists(_ == ) -> contains find + nonEmpty -> exists filter + size -> count - reduce(_+_) -> sum map + flatten -> map The most controversial may be .size -> .length simply because of its size. It is intended to avoid implicits that might be expensive in some places. ## How was the this patch tested? Existing Jenkins unit tests. Author: Sean Owen <sowen@cloudera.com> Closes #11292 from srowen/SPARK-13423.
Showing
- common/network-common/src/test/java/org/apache/spark/network/ChunkFetchIntegrationSuite.java 1 addition, 1 deletion.../org/apache/spark/network/ChunkFetchIntegrationSuite.java
- common/network-common/src/test/java/org/apache/spark/network/RequestTimeoutIntegrationSuite.java 6 additions, 6 deletions.../apache/spark/network/RequestTimeoutIntegrationSuite.java
- common/network-common/src/test/java/org/apache/spark/network/RpcIntegrationSuite.java 1 addition, 1 deletion...st/java/org/apache/spark/network/RpcIntegrationSuite.java
- common/network-common/src/test/java/org/apache/spark/network/TransportClientFactorySuite.java 4 additions, 3 deletions...org/apache/spark/network/TransportClientFactorySuite.java
- common/network-common/src/test/java/org/apache/spark/network/protocol/MessageWithHeaderSuite.java 3 additions, 3 deletions...apache/spark/network/protocol/MessageWithHeaderSuite.java
- common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/protocol/mesos/RegisterDriver.java 8 additions, 0 deletions.../spark/network/shuffle/protocol/mesos/RegisterDriver.java
- common/network-shuffle/src/test/java/org/apache/spark/network/shuffle/ExternalShuffleIntegrationSuite.java 1 addition, 1 deletion...park/network/shuffle/ExternalShuffleIntegrationSuite.java
- common/network-shuffle/src/test/java/org/apache/spark/network/shuffle/RetryingBlockFetcherSuite.java 1 addition, 1 deletion...ache/spark/network/shuffle/RetryingBlockFetcherSuite.java
- common/unsafe/src/test/scala/org/apache/spark/unsafe/types/UTF8StringPropertyCheckSuite.scala 1 addition, 1 deletion...che/spark/unsafe/types/UTF8StringPropertyCheckSuite.scala
- core/src/main/java/org/apache/spark/shuffle/sort/ShuffleInMemorySorter.java 3 additions, 1 deletion.../org/apache/spark/shuffle/sort/ShuffleInMemorySorter.java
- core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeExternalSorter.java 1 addition, 1 deletion...ark/util/collection/unsafe/sort/UnsafeExternalSorter.java
- core/src/main/scala/org/apache/spark/Dependency.scala 1 addition, 1 deletioncore/src/main/scala/org/apache/spark/Dependency.scala
- core/src/main/scala/org/apache/spark/Partitioner.scala 4 additions, 4 deletionscore/src/main/scala/org/apache/spark/Partitioner.scala
- core/src/main/scala/org/apache/spark/SparkContext.scala 4 additions, 3 deletionscore/src/main/scala/org/apache/spark/SparkContext.scala
- core/src/main/scala/org/apache/spark/TaskEndReason.scala 0 additions, 1 deletioncore/src/main/scala/org/apache/spark/TaskEndReason.scala
- core/src/main/scala/org/apache/spark/deploy/ClientArguments.scala 2 additions, 0 deletions.../main/scala/org/apache/spark/deploy/ClientArguments.scala
- core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala 3 additions, 0 deletions.../src/main/scala/org/apache/spark/deploy/SparkSubmit.scala
- core/src/main/scala/org/apache/spark/deploy/history/HistoryServerArguments.scala 3 additions, 0 deletions.../apache/spark/deploy/history/HistoryServerArguments.scala
- core/src/main/scala/org/apache/spark/deploy/master/MasterArguments.scala 3 additions, 0 deletions...cala/org/apache/spark/deploy/master/MasterArguments.scala
- core/src/main/scala/org/apache/spark/deploy/master/MasterSource.scala 2 additions, 2 deletions...n/scala/org/apache/spark/deploy/master/MasterSource.scala
Loading
Please register or sign in to comment