Skip to content
  • Tathagata Das's avatar
    30040741
    [SPARK-3169] Removed dependency on spark streaming test from spark flume sink · 30040741
    Tathagata Das authored
    Due to maven bug https://jira.codehaus.org/browse/MNG-1378, maven could not resolve spark streaming classes required by the spark-streaming test-jar dependency of external/flume-sink. There is no particular reason that the external/flume-sink has to depend on Spark Streaming at all, so I am eliminating this dependency. Also I have removed the exclusions present in the Flume dependencies, as there is no reason to exclude them (they were excluded in the external/flume module to prevent dependency collisions with Spark).
    
    Since Jenkins will test the sbt build and the unit test, I only tested maven compilation locally.
    
    Author: Tathagata Das <tathagata.das1565@gmail.com>
    
    Closes #2101 from tdas/spark-sink-pom-fix and squashes the following commits:
    
    8f42621 [Tathagata Das] Added Flume sink exclusions back, and added netty to test dependencies
    93b559f [Tathagata Das] Removed dependency on spark streaming test from spark flume sink
    30040741
    [SPARK-3169] Removed dependency on spark streaming test from spark flume sink
    Tathagata Das authored
    Due to maven bug https://jira.codehaus.org/browse/MNG-1378, maven could not resolve spark streaming classes required by the spark-streaming test-jar dependency of external/flume-sink. There is no particular reason that the external/flume-sink has to depend on Spark Streaming at all, so I am eliminating this dependency. Also I have removed the exclusions present in the Flume dependencies, as there is no reason to exclude them (they were excluded in the external/flume module to prevent dependency collisions with Spark).
    
    Since Jenkins will test the sbt build and the unit test, I only tested maven compilation locally.
    
    Author: Tathagata Das <tathagata.das1565@gmail.com>
    
    Closes #2101 from tdas/spark-sink-pom-fix and squashes the following commits:
    
    8f42621 [Tathagata Das] Added Flume sink exclusions back, and added netty to test dependencies
    93b559f [Tathagata Das] Removed dependency on spark streaming test from spark flume sink
Loading