Skip to content
Snippets Groups Projects
  1. Sep 01, 2013
  2. Aug 29, 2013
    • Matei Zaharia's avatar
      Update Maven build to create assemblies expected by new scripts · 666d93c2
      Matei Zaharia authored
      This includes the following changes:
      - The "assembly" package now builds in Maven by default, and creates an
        assembly containing both hadoop-client and Spark, unlike the old
        BigTop distribution assembly that skipped hadoop-client
      - There is now a bigtop-dist package to build the old BigTop assembly
      - The repl-bin package is no longer built by default since the scripts
        don't reply on it; instead it can be enabled with -Prepl-bin
      - Py4J is now included in the assembly/lib folder as a local Maven repo,
        so that the Maven package can link to it
      - run-example now adds the original Spark classpath as well because the
        Maven examples assembly lists spark-core and such as provided
      - The various Maven projects add a spark-yarn dependency correctly
      666d93c2
    • Matei Zaharia's avatar
    • Matei Zaharia's avatar
      Change build and run instructions to use assemblies · 53cd50c0
      Matei Zaharia authored
      This commit makes Spark invocation saner by using an assembly JAR to
      find all of Spark's dependencies instead of adding all the JARs in
      lib_managed. It also packages the examples into an assembly and uses
      that as SPARK_EXAMPLES_JAR. Finally, it replaces the old "run" script
      with two better-named scripts: "run-examples" for examples, and
      "spark-class" for Spark internal classes (e.g. REPL, master, etc). This
      is also designed to minimize the confusion people have in trying to use
      "run" to run their own classes; it's not meant to do that, but now at
      least if they look at it, they can modify run-examples to do a decent
      job for them.
      
      As part of this, Bagel's examples are also now properly moved to the
      examples package instead of bagel.
      53cd50c0
  3. Aug 18, 2013
  4. Aug 16, 2013
  5. Aug 15, 2013
  6. Aug 11, 2013
  7. Aug 10, 2013
  8. Aug 08, 2013
  9. Aug 07, 2013
  10. Aug 06, 2013
    • Shivaram Venkataraman's avatar
      Refactor GLM algorithms and add Java tests · 7db69d56
      Shivaram Venkataraman authored
      This change adds Java examples and unit tests for all GLM algorithms
      to make sure the MLLib interface works from Java. Changes include
      - Introduce LabeledPoint and avoid using Doubles in train arguments
      - Rename train to run in class methods
      - Make the optimizer a member variable of GLM to make sure the builder
        pattern works
      7db69d56
    • Shivaram Venkataraman's avatar
      Java examples, tests for KMeans and ALS · 471fbadd
      Shivaram Venkataraman authored
      - Changes ALS to accept RDD[Rating] instead of (Int, Int, Double) making it
        easier to call from Java
      - Renames class methods from `train` to `run` to enable static methods to be
        called from Java.
      - Add unit tests which check if both static / class methods can be called.
      - Also add examples which port the main() function in ALS, KMeans to the
        examples project.
      
      Couple of minor changes to existing code:
      - Add a toJavaRDD method in RDD to convert scala RDD to java RDD easily
      - Workaround a bug where using double[] from Java leads to class cast exception in
        KMeans init
      471fbadd
    • stayhf's avatar
      Got rid of unnecessary map function · 882baee4
      stayhf authored
      882baee4
    • stayhf's avatar
      changes as reviewer requested · 326a7a82
      stayhf authored
      326a7a82
  11. Aug 04, 2013
  12. Aug 03, 2013
  13. Jul 16, 2013
  14. Jul 08, 2013
  15. Jul 01, 2013
  16. Jun 25, 2013
  17. Jun 13, 2013
  18. Jun 03, 2013
  19. Jun 02, 2013
  20. Jun 01, 2013
  21. May 20, 2013
  22. May 09, 2013
Loading