- Sep 01, 2013
-
-
Matei Zaharia authored
* RDD, *RDDFunctions -> org.apache.spark.rdd * Utils, ClosureCleaner, SizeEstimator -> org.apache.spark.util * JavaSerializer, KryoSerializer -> org.apache.spark.serializer
-
Matei Zaharia authored
-
Matei Zaharia authored
-
- Aug 29, 2013
-
-
Matei Zaharia authored
This includes the following changes: - The "assembly" package now builds in Maven by default, and creates an assembly containing both hadoop-client and Spark, unlike the old BigTop distribution assembly that skipped hadoop-client - There is now a bigtop-dist package to build the old BigTop assembly - The repl-bin package is no longer built by default since the scripts don't reply on it; instead it can be enabled with -Prepl-bin - Py4J is now included in the assembly/lib folder as a local Maven repo, so that the Maven package can link to it - run-example now adds the original Spark classpath as well because the Maven examples assembly lists spark-core and such as provided - The various Maven projects add a spark-yarn dependency correctly
-
Matei Zaharia authored
-
Matei Zaharia authored
This commit makes Spark invocation saner by using an assembly JAR to find all of Spark's dependencies instead of adding all the JARs in lib_managed. It also packages the examples into an assembly and uses that as SPARK_EXAMPLES_JAR. Finally, it replaces the old "run" script with two better-named scripts: "run-examples" for examples, and "spark-class" for Spark internal classes (e.g. REPL, master, etc). This is also designed to minimize the confusion people have in trying to use "run" to run their own classes; it's not meant to do that, but now at least if they look at it, they can modify run-examples to do a decent job for them. As part of this, Bagel's examples are also now properly moved to the examples package instead of bagel.
-
- Aug 18, 2013
-
-
Jey Kottalam authored
-
- Aug 16, 2013
-
-
Jey Kottalam authored
-
Jey Kottalam authored
-
Jey Kottalam authored
-
Jey Kottalam authored
-
Jey Kottalam authored
-
- Aug 15, 2013
-
-
Jey Kottalam authored
-
- Aug 11, 2013
-
-
Alexander Pivovarov authored
-
- Aug 10, 2013
-
-
Matei Zaharia authored
-
- Aug 08, 2013
-
-
Matei Zaharia authored
-
Shivaram Venkataraman authored
Also remove java-specific constructor for LabeledPoint.
-
Shivaram Venkataraman authored
The scala constructor works for native type java types. Modify examples to match this.
-
Nick Pentreath authored
-
- Aug 07, 2013
-
-
Nick Pentreath authored
-
- Aug 06, 2013
-
-
Shivaram Venkataraman authored
This change adds Java examples and unit tests for all GLM algorithms to make sure the MLLib interface works from Java. Changes include - Introduce LabeledPoint and avoid using Doubles in train arguments - Rename train to run in class methods - Make the optimizer a member variable of GLM to make sure the builder pattern works
-
Shivaram Venkataraman authored
- Changes ALS to accept RDD[Rating] instead of (Int, Int, Double) making it easier to call from Java - Renames class methods from `train` to `run` to enable static methods to be called from Java. - Add unit tests which check if both static / class methods can be called. - Also add examples which port the main() function in ALS, KMeans to the examples project. Couple of minor changes to existing code: - Add a toJavaRDD method in RDD to convert scala RDD to java RDD easily - Workaround a bug where using double[] from Java leads to class cast exception in KMeans init
-
stayhf authored
-
stayhf authored
-
- Aug 04, 2013
-
-
stayhf authored
-
- Aug 03, 2013
-
-
stayhf authored
-
- Jul 16, 2013
-
-
Matei Zaharia authored
-
- Jul 08, 2013
-
-
Mark Hamstra authored
-
Mark Hamstra authored
-
- Jul 01, 2013
-
-
Konstantin Boudnik authored
-
- Jun 25, 2013
-
-
James Phillpotts authored
-
James Phillpotts authored
-
- Jun 13, 2013
-
-
Rohit Rai authored
-
- Jun 03, 2013
- Jun 02, 2013
- Jun 01, 2013
-
-
Rohit Rai authored
-
- May 20, 2013
-
-
Ethan Jewett authored
-
- May 09, 2013
-
-
Ethan Jewett authored
-