-
- Downloads
[SPARK-9479] [STREAMING] [TESTS] Fix ReceiverTrackerSuite failure for maven...
[SPARK-9479] [STREAMING] [TESTS] Fix ReceiverTrackerSuite failure for maven build and other potential test failures in Streaming See https://issues.apache.org/jira/browse/SPARK-9479 for the failure cause. The PR includes the following changes: 1. Make ReceiverTrackerSuite create StreamingContext in the test body. 2. Fix places that don't stop StreamingContext. I verified no SparkContext was stopped in the shutdown hook locally after this fix. 3. Fix an issue that `ReceiverTracker.endpoint` may be null. 4. Make sure stopping SparkContext in non-main thread won't fail other tests. Author: zsxwing <zsxwing@gmail.com> Closes #7797 from zsxwing/fix-ReceiverTrackerSuite and squashes the following commits: 3a4bb98 [zsxwing] Fix another potential NPE d7497df [zsxwing] Fix ReceiverTrackerSuite; make sure StreamingContext in tests is closed
Showing
- mllib/src/test/scala/org/apache/spark/mllib/classification/StreamingLogisticRegressionSuite.scala 15 additions, 6 deletions...lib/classification/StreamingLogisticRegressionSuite.scala
- mllib/src/test/scala/org/apache/spark/mllib/clustering/StreamingKMeansSuite.scala 13 additions, 4 deletions.../apache/spark/mllib/clustering/StreamingKMeansSuite.scala
- mllib/src/test/scala/org/apache/spark/mllib/regression/StreamingLinearRegressionSuite.scala 15 additions, 6 deletions...ark/mllib/regression/StreamingLinearRegressionSuite.scala
- streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala 9 additions, 3 deletions...rg/apache/spark/streaming/scheduler/ReceiverTracker.scala
- streaming/src/test/java/org/apache/spark/streaming/JavaAPISuite.java 1 addition, 0 deletions...rc/test/java/org/apache/spark/streaming/JavaAPISuite.java
- streaming/src/test/scala/org/apache/spark/streaming/BasicOperationsSuite.scala 30 additions, 28 deletions...ala/org/apache/spark/streaming/BasicOperationsSuite.scala
- streaming/src/test/scala/org/apache/spark/streaming/InputStreamsSuite.scala 21 additions, 17 deletions.../scala/org/apache/spark/streaming/InputStreamsSuite.scala
- streaming/src/test/scala/org/apache/spark/streaming/MasterFailureTest.scala 7 additions, 1 deletion.../scala/org/apache/spark/streaming/MasterFailureTest.scala
- streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala 17 additions, 5 deletions...la/org/apache/spark/streaming/StreamingContextSuite.scala
- streaming/src/test/scala/org/apache/spark/streaming/StreamingListenerSuite.scala 11 additions, 2 deletions...a/org/apache/spark/streaming/StreamingListenerSuite.scala
- streaming/src/test/scala/org/apache/spark/streaming/scheduler/ReceiverTrackerSuite.scala 30 additions, 26 deletions...ache/spark/streaming/scheduler/ReceiverTrackerSuite.scala
- streaming/src/test/scala/org/apache/spark/streaming/ui/StreamingJobProgressListenerSuite.scala 14 additions, 5 deletions...park/streaming/ui/StreamingJobProgressListenerSuite.scala
Loading
Please register or sign in to comment