Skip to content
Snippets Groups Projects
  1. Apr 24, 2017
  2. Dec 02, 2016
  3. Jul 19, 2016
  4. Jul 11, 2016
    • Reynold Xin's avatar
      [SPARK-16477] Bump master version to 2.1.0-SNAPSHOT · ffcb6e05
      Reynold Xin authored
      ## What changes were proposed in this pull request?
      After SPARK-16476 (committed earlier today as #14128), we can finally bump the version number.
      
      ## How was this patch tested?
      N/A
      
      Author: Reynold Xin <rxin@databricks.com>
      
      Closes #14130 from rxin/SPARK-16477.
      ffcb6e05
  5. Mar 25, 2016
    • Shixiong Zhu's avatar
      [SPARK-14073][STREAMING][TEST-MAVEN] Move flume back to Spark · 24587ce4
      Shixiong Zhu authored
      ## What changes were proposed in this pull request?
      
      This PR moves flume back to Spark as per the discussion in the dev mail-list.
      
      ## How was this patch tested?
      
      Existing Jenkins tests.
      
      Author: Shixiong Zhu <shixiong@databricks.com>
      
      Closes #11895 from zsxwing/move-flume-back.
      24587ce4
  6. Mar 14, 2016
    • Shixiong Zhu's avatar
      [SPARK-13843][STREAMING] Remove streaming-flume, streaming-mqtt,... · 06dec374
      Shixiong Zhu authored
      [SPARK-13843][STREAMING] Remove streaming-flume, streaming-mqtt, streaming-zeromq, streaming-akka, streaming-twitter to Spark packages
      
      ## What changes were proposed in this pull request?
      
      Currently there are a few sub-projects, each for integrating with different external sources for Streaming.  Now that we have better ability to include external libraries (spark packages) and with Spark 2.0 coming up, we can move the following projects out of Spark to https://github.com/spark-packages
      
      - streaming-flume
      - streaming-akka
      - streaming-mqtt
      - streaming-zeromq
      - streaming-twitter
      
      They are just some ancillary packages and considering the overhead of maintenance, running tests and PR failures, it's better to maintain them out of Spark. In addition, these projects can have their different release cycles and we can release them faster.
      
      I have already copied these projects to https://github.com/spark-packages
      
      ## How was this patch tested?
      
      Jenkins tests
      
      Author: Shixiong Zhu <shixiong@databricks.com>
      
      Closes #11672 from zsxwing/remove-external-pkg.
      06dec374
  7. Jan 30, 2016
    • Josh Rosen's avatar
      [SPARK-6363][BUILD] Make Scala 2.11 the default Scala version · 289373b2
      Josh Rosen authored
      This patch changes Spark's build to make Scala 2.11 the default Scala version. To be clear, this does not mean that Spark will stop supporting Scala 2.10: users will still be able to compile Spark for Scala 2.10 by following the instructions on the "Building Spark" page; however, it does mean that Scala 2.11 will be the default Scala version used by our CI builds (including pull request builds).
      
      The Scala 2.11 compiler is faster than 2.10, so I think we'll be able to look forward to a slight speedup in our CI builds (it looks like it's about 2X faster for the Maven compile-only builds, for instance).
      
      After this patch is merged, I'll update Jenkins to add new compile-only jobs to ensure that Scala 2.10 compilation doesn't break.
      
      Author: Josh Rosen <joshrosen@databricks.com>
      
      Closes #10608 from JoshRosen/SPARK-6363.
      289373b2
  8. Dec 19, 2015
  9. Sep 15, 2015
  10. Aug 24, 2015
    • zsxwing's avatar
      [SPARK-10168] [STREAMING] Fix the issue that maven publishes wrong artifact jars · 4e0395dd
      zsxwing authored
      This PR removed the `outputFile` configuration from pom.xml and updated `tests.py` to search jars for both sbt build and maven build.
      
      I ran ` mvn -Pkinesis-asl -DskipTests clean install` locally, and verified the jars in my local repository were correct. I also checked Python tests for maven build, and it passed all tests.
      
      Author: zsxwing <zsxwing@gmail.com>
      
      Closes #8373 from zsxwing/SPARK-10168 and squashes the following commits:
      
      e0b5818 [zsxwing] Fix the sbt build
      c697627 [zsxwing] Add the jar pathes to the exception message
      be1d8a5 [zsxwing] Fix the issue that maven publishes wrong artifact jars
      4e0395dd
  11. Aug 18, 2015
  12. Jul 09, 2015
    • Marcelo Vanzin's avatar
      [SPARK-8852] [FLUME] Trim dependencies in flume assembly. · 0e78e40c
      Marcelo Vanzin authored
      Also, add support for the *-provided profiles. This avoids repackaging
      things that are already in the Spark assembly, or, in the case of the
      *-provided profiles, are provided by the distribution.
      
      The flume-ng-auth dependency was also excluded since it's not really
      used by Spark.
      
      Author: Marcelo Vanzin <vanzin@cloudera.com>
      
      Closes #7247 from vanzin/SPARK-8852 and squashes the following commits:
      
      298a7d5 [Marcelo Vanzin] Feedback.
      c962082 [Marcelo Vanzin] [SPARK-8852] [flume] Trim dependencies in flume assembly.
      0e78e40c
  13. Jul 01, 2015
    • zsxwing's avatar
      [SPARK-8378] [STREAMING] Add the Python API for Flume · 75b9fe4c
      zsxwing authored
      Author: zsxwing <zsxwing@gmail.com>
      
      Closes #6830 from zsxwing/flume-python and squashes the following commits:
      
      78dfdac [zsxwing] Fix the compile error in the test code
      f1bf3c0 [zsxwing] Address TD's comments
      0449723 [zsxwing] Add sbt goal streaming-flume-assembly/assembly
      e93736b [zsxwing] Fix the test case for determine_modules_to_test
      9d5821e [zsxwing] Fix pyspark_core dependencies
      f9ee681 [zsxwing] Merge branch 'master' into flume-python
      7a55837 [zsxwing] Add streaming_flume_assembly to run-tests.py
      b96b0de [zsxwing] Merge branch 'master' into flume-python
      ce85e83 [zsxwing] Fix incompatible issues for Python 3
      01cbb3d [zsxwing] Add import sys
      152364c [zsxwing] Fix the issue that StringIO doesn't work in Python 3
      14ba0ff [zsxwing] Add flume-assembly for sbt building
      b8d5551 [zsxwing] Merge branch 'master' into flume-python
      4762c34 [zsxwing] Fix the doc
      0336579 [zsxwing] Refactor Flume unit tests and also add tests for Python API
      9f33873 [zsxwing] Add the Python API for Flume
      75b9fe4c
Loading