Skip to content
Snippets Groups Projects
  1. Dec 08, 2016
  2. Nov 28, 2016
  3. Jul 19, 2016
  4. Jul 11, 2016
    • Reynold Xin's avatar
      [SPARK-16477] Bump master version to 2.1.0-SNAPSHOT · ffcb6e05
      Reynold Xin authored
      ## What changes were proposed in this pull request?
      After SPARK-16476 (committed earlier today as #14128), we can finally bump the version number.
      
      ## How was this patch tested?
      N/A
      
      Author: Reynold Xin <rxin@databricks.com>
      
      Closes #14130 from rxin/SPARK-16477.
      ffcb6e05
  5. Jun 12, 2016
    • Sean Owen's avatar
      [SPARK-15086][CORE][STREAMING] Deprecate old Java accumulator API · f51dfe61
      Sean Owen authored
      ## What changes were proposed in this pull request?
      
      - Deprecate old Java accumulator API; should use Scala now
      - Update Java tests and examples
      - Don't bother testing old accumulator API in Java 8 (too)
      - (fix a misspelling too)
      
      ## How was this patch tested?
      
      Jenkins tests
      
      Author: Sean Owen <sowen@cloudera.com>
      
      Closes #13606 from srowen/SPARK-15086.
      f51dfe61
  6. May 31, 2016
    • Marcelo Vanzin's avatar
      [SPARK-15451][BUILD] Use jdk7's rt.jar when available. · 57adb77e
      Marcelo Vanzin authored
      This helps with preventing jdk8-specific calls being checked in,
      because PR builders are running the compiler with the wrong settings.
      
      If the JAVA_7_HOME env variable is set, assume it points at
      a jdk7 and use its rt.jar when invoking javac. For zinc, just run
      it with jdk7, and disable it when building jdk8-specific code.
      
      A big note for sbt usage: adding the bootstrap options forces sbt
      to fork the compiler, and that disables incremental compilation.
      That means that it's really not convenient to use for normal
      development, but should be ok for automated builds.
      
      Tested with JAVA_HOME=jdk8 and JAVA_7_HOME=jdk7:
      - mvn + zinc
      - mvn sans zinc
      - sbt
      
      Verified that in all cases, jdk8-specific library calls fail to
      compile.
      
      Author: Marcelo Vanzin <vanzin@cloudera.com>
      
      Closes #13272 from vanzin/SPARK-15451.
      57adb77e
  7. May 27, 2016
    • Reynold Xin's avatar
      [SPARK-15633][MINOR] Make package name for Java tests consistent · 73178c75
      Reynold Xin authored
      ## What changes were proposed in this pull request?
      This is a simple patch that makes package names for Java 8 test suites consistent. I moved everything to test.org.apache.spark to we can test package private APIs properly. Also added "java8" as the package name so we can easily run all the tests related to Java 8.
      
      ## How was this patch tested?
      This is a test only change.
      
      Author: Reynold Xin <rxin@databricks.com>
      
      Closes #13364 from rxin/SPARK-15633.
      73178c75
  8. May 17, 2016
  9. May 10, 2016
    • Subhobrata Dey's avatar
      [SPARK-14642][SQL] import org.apache.spark.sql.expressions._ breaks udf under functions · 89f73f67
      Subhobrata Dey authored
      ## What changes were proposed in this pull request?
      
      PR fixes the import issue which breaks udf functions.
      
      The following code snippet throws an error
      
      ```
      scala> import org.apache.spark.sql.functions._
      import org.apache.spark.sql.functions._
      
      scala> import org.apache.spark.sql.expressions._
      import org.apache.spark.sql.expressions._
      
      scala> udf((v: String) => v.stripSuffix("-abc"))
      <console>:30: error: No TypeTag available for String
             udf((v: String) => v.stripSuffix("-abc"))
      ```
      
      This PR resolves the issue.
      
      ## How was this patch tested?
      
      patch tested with unit tests.
      
      (If this patch involves UI changes, please attach a screenshot; otherwise, remove this)
      
      Author: Subhobrata Dey <sbcd90@gmail.com>
      
      Closes #12458 from sbcd90/udfFuncBreak.
      89f73f67
  10. Apr 28, 2016
  11. Apr 09, 2016
    • Dongjoon Hyun's avatar
      [SPARK-14465][BUILD] Checkstyle should check all Java files · aea30a1a
      Dongjoon Hyun authored
      ## What changes were proposed in this pull request?
      
      Currently, `checkstyle` is configured to check the files under `src/main/java`. However, Spark has Java files in `src/main/scala`, too. This PR fixes the following configuration in `pom.xml` and the unchecked-so-far violations on those files.
      ```xml
      -<sourceDirectory>${basedir}/src/main/java</sourceDirectory>
      +<sourceDirectories>${basedir}/src/main/java,${basedir}/src/main/scala</sourceDirectories>
      ```
      
      ## How was this patch tested?
      
      After passing the Jenkins build and manually `dev/lint-java`. (Note that Jenkins does not run `lint-java`)
      
      Author: Dongjoon Hyun <dongjoon@apache.org>
      
      Closes #12242 from dongjoon-hyun/SPARK-14465.
      aea30a1a
  12. Apr 06, 2016
    • Marcelo Vanzin's avatar
      [SPARK-14134][CORE] Change the package name used for shading classes. · 21d5ca12
      Marcelo Vanzin authored
      The current package name uses a dash, which is a little weird but seemed
      to work. That is, until a new test tried to mock a class that references
      one of those shaded types, and then things started failing.
      
      Most changes are just noise to fix the logging configs.
      
      For reference, SPARK-8815 also raised this issue, although at the time it
      did not cause any issues in Spark, so it was not addressed.
      
      Author: Marcelo Vanzin <vanzin@cloudera.com>
      
      Closes #11941 from vanzin/SPARK-14134.
      21d5ca12
  13. Apr 05, 2016
  14. Mar 31, 2016
    • Josh Rosen's avatar
      [SPARK-14281][TESTS] Fix java8-tests and simplify their build · a7af6cd2
      Josh Rosen authored
      This patch fixes a compilation / build break in Spark's `java8-tests` and refactors their POM to simplify the build. See individual commit messages for more details.
      
      Author: Josh Rosen <joshrosen@databricks.com>
      
      Closes #12073 from JoshRosen/fix-java8-tests.
      a7af6cd2
  15. Mar 09, 2016
    • Sean Owen's avatar
      [SPARK-13595][BUILD] Move docker, extras modules into external · 256704c7
      Sean Owen authored
      ## What changes were proposed in this pull request?
      
      Move `docker` dirs out of top level into `external/`; move `extras/*` into `external/`
      
      ## How was this patch tested?
      
      This is tested with Jenkins tests.
      
      Author: Sean Owen <sowen@cloudera.com>
      
      Closes #11523 from srowen/SPARK-13595.
      256704c7
Loading