Commits · f08db010d3ba4eff132f5c06a087b0c7c2e58576 · cs525-sp18-g07 / spark

Feb 26, 2013
- Small hack to work around multiple JARs being built by sbt package · 434a1ce7
  Matei Zaharia authored 12 years ago
  
  434a1ce7
Feb 25, 2013
- Pass a code JAR to SparkContext in our examples. Fixes SPARK-594. · 5d7b591c
  Matei Zaharia authored 12 years ago
  
  5d7b591c
- Change tabs to spaces · 25f73780
  Matei Zaharia authored 12 years ago
  
  25f73780
Feb 24, 2013
- Fixed class paths and dependencies based on Matei's comments. · 5ab37be9
  Tathagata Das authored 12 years ago
  
  5ab37be9
Feb 16, 2013
- support customized java options for master, worker, executor, repl shell · 85878445
  haitao.yao authored 12 years ago
  
  85878445
Feb 10, 2013
- Use a separate memory setting for standalone cluster daemons · 05d2e948
  Matei Zaharia authored 12 years ago
  
  Conflicts: docs/_config.yml
  05d2e948
Jan 20, 2013
- Update run script to deal with change to build of REPL shaded JAR · 4750907c
  Matei Zaharia authored 12 years ago
  
  4750907c
Jan 18, 2013
- kafka jar wasn't being included by run script · 1db119a0
  seanm authored 12 years ago
  
  1db119a0
Jan 17, 2013
- Warn users if they run pyspark or spark-shell without compiling Spark · 892c32a1
  Matei Zaharia authored 12 years ago
  
  892c32a1
Jan 08, 2013
- Retrieve jars to a flat directory so * can be used for the classpath. · c3f1675f
  Stephen Haberman authored 12 years ago
  
  c3f1675f
Jan 06, 2013
- Removed streaming-env.sh.template · 934ecc82
  Tathagata Das authored 12 years ago
  
  934ecc82
Jan 01, 2013
- Rename top-level 'pyspark' directory to 'python' · b58340db
  Josh Rosen authored 12 years ago
  
  b58340db
Dec 28, 2012

Simplify PySpark installation. · 665466df

Josh Rosen authored 12 years ago

- Bundle Py4J binaries, since it's hard to install
- Uses Spark's `run` script to launch the Py4J
  gateway, inheriting the settings in spark-env.sh

With these changes, (hopefully) nothing more than
running `sbt/sbt package` will be necessary to run
PySpark.

665466df

Dec 10, 2012
- Make "run" script work with Maven builds · 01c1f97e
  Matei Zaharia authored 12 years ago
  
  01c1f97e
Nov 12, 2012

Fixed bugs in RawNetworkInputDStream and in its examples. Made the... · ae61ebae

Tathagata Das authored 12 years ago

Fixed bugs in RawNetworkInputDStream and in its examples. Made the ReducedWindowedDStream persist RDDs to MEMOERY_SER_ONLY by default. Removed unncessary examples. Added streaming-env.sh.template to add recommended setting for streaming.

ae61ebae

Oct 22, 2012
- Tweaked run file to live more happily with typesafe's debian package · f595bb53
  Thomas Dudziak authored 12 years ago
  
  f595bb53
Oct 13, 2012
- Document how to configure SPARK_MEM & co on a per-job basis · 4a3e9cf6
  Matei Zaharia authored 12 years ago
  
  4a3e9cf6
Oct 07, 2012
- Changes to run script and README to deal with updated Scala version and · f31b6f92
  Matei Zaharia authored 12 years ago
  
  with Mesos being in Maven
  f31b6f92
Oct 06, 2012
- Made run script add test-classes onto the classpath only if SPARK_TESTING is set; fixes #216 · ce915cad
  root authored 12 years ago
  
  ce915cad
Oct 04, 2012
- Don't check for JARs in core/lib anymore · c535762d
  Matei Zaharia authored 12 years ago
  
  c535762d
Sep 24, 2012
- Update Scala version dependency to 2.9.2 · 1f539aa4
  Matei Zaharia authored 12 years ago
  
  1f539aa4
Sep 10, 2012
- Added a unit test for local-cluster mode and simplified some of the code involved in that · a1378067
  Matei Zaharia authored 12 years ago
  
  a1378067
Sep 07, 2012
- Added a unit test for local-cluster mode and simplified some of the code involved in that · 995982b3
  Matei Zaharia authored 12 years ago
  
  995982b3
Aug 04, 2012
- Renamed EXAMPLES_JAR to SPARK_EXAMPLES_JAR · 48cac417
  Denny authored 12 years ago
  
  48cac417
Jul 28, 2012
- Added the Spark Streaing code, ported to Akka 2 · 47b7ebad
  Matei Zaharia authored 12 years ago
  
  47b7ebad
- Fixed SPARK_MEM not being passed when runner is java · dc8763fc
  Matei Zaharia authored 12 years ago
  
  dc8763fc
Jul 18, 2012
- Use extended constructor in the examples. · 1d988845
  Denny authored 12 years ago
  
  1d988845
Jul 08, 2012
- More work to allow Spark to run on the standalone deploy cluster. · 0a472840
  Matei Zaharia authored 12 years ago
  
  0a472840
Jun 30, 2012
- More work on deploy code (adding Worker class) · 408b5a13
  Matei Zaharia authored 12 years ago
  
  408b5a13
Mar 17, 2012
- Further fixes to how Mesos is found and used · 08cda89e
  Matei Zaharia authored 13 years ago
  
  08cda89e
Sep 26, 2011
- Set SCALA_VERSION to 2.9.1 (from 2.9.1.final) to match expectation of SBT 0.11.0 · 4019305a
  Ismael Juma authored 13 years ago
  
  4019305a
Aug 31, 2011

Upgrade to Scala 2.9.1. · 483f724d

Ismael Juma authored 13 years ago

Interestingly, the version in Maven is 2.9.1, but SBT outputs file to the 2.9.1.final
directory inside target.

A couple of small changes in SparkIMain were also required.

All tests pass and ./spark-shell launches successfully.

483f724d

Aug 30, 2011
- Removed a debugging line · 0c965a10
  Matei Zaharia authored 13 years ago
  
  0c965a10
Aug 01, 2011
- Updated to newest Mesos API, which includes better memory accounting · 4050d661
  Matei Zaharia authored 13 years ago
  
  by specifying per-executor memory.
  4050d661
Jul 14, 2011
- Initial work on converting build to SBT 0.10.1 · f686e3da
  Ismael Juma authored 13 years ago
  
  f686e3da
Jun 01, 2011
- Pass quoted arguments properly to run · 96daa31a
  Matei Zaharia authored 13 years ago
  
  96daa31a
- Pass quoted arguments properly to run · 3854d23d
  Matei Zaharia authored 13 years ago
  
  3854d23d
May 27, 2011
- SCALA_VERSION in run should be 2.9.0-1 · 2c691295
  Ismael Juma authored 13 years ago
  
  2c691295
- Update to Scala 2.9.0-1 and disable repl module for now. · 1d75c606
  Ismael Juma authored 13 years ago
  
  The repl module requires more complex work.
  1d75c606
- Update run to work with SBT managed dependencies and the newly introduced repl module. · 59f1f42a
  Ismael Juma authored 13 years ago
  
  59f1f42a