Skip to content
Snippets Groups Projects
  1. Jan 28, 2015
    • Nicholas Chammas's avatar
      [SPARK-5434] [EC2] Preserve spaces in EC2 path · d44ee436
      Nicholas Chammas authored
      Fixes [SPARK-5434](https://issues.apache.org/jira/browse/SPARK-5434).
      
      Simple demonstration of the problem and the fix:
      
      ```
      $ spacey_path="/path/with some/spaces"
      $ dirname $spacey_path
      usage: dirname path
      $ echo $?
      1
      $ dirname "$spacey_path"
      /path/with some
      $ echo $?
      0
      ```
      
      Author: Nicholas Chammas <nicholas.chammas@gmail.com>
      
      Closes #4224 from nchammas/patch-1 and squashes the following commits:
      
      960711a [Nicholas Chammas] [EC2] Preserve spaces in EC2 path
      d44ee436
  2. Dec 19, 2014
    • Josh Rosen's avatar
      [SPARK-4890] Upgrade Boto to 2.34.0; automatically download Boto from PyPi instead of packaging it · c28083f4
      Josh Rosen authored
      This patch upgrades `spark-ec2`'s Boto version to 2.34.0, since this is blocking several features.  Newer versions of Boto don't work properly when they're loaded from a zipfile since they try to read a JSON file from a path relative to the Boto library sources.
      
      Therefore, this patch also changes spark-ec2 to automatically download Boto from PyPi if it's not present in `SPARK_EC2_DIR/lib`, similar to what we do in the `sbt/sbt` script. This shouldn't ben an issue for users since they already need to have an internet connection to launch an EC2 cluster.  By performing the downloading in spark_ec2.py instead of the Bash script, this should also work for Windows users.
      
      I've tested this with Python 2.6, too.
      
      Author: Josh Rosen <joshrosen@databricks.com>
      
      Closes #3737 from JoshRosen/update-boto and squashes the following commits:
      
      0aa43cc [Josh Rosen] Remove unused setup_standalone_cluster() method.
      f02935d [Josh Rosen] Enable Python deprecation warnings and fix one Boto warning:
      587ae89 [Josh Rosen] [SPARK-4890] Upgrade Boto to 2.34.0; automatically download Boto from PyPi instead of packaging it
      c28083f4
  3. Nov 05, 2014
  4. Jun 22, 2014
    • Ori Kremer's avatar
      SPARK-2241: quote command line args in ec2 script · 9fc373e3
      Ori Kremer authored
      To preserve quoted command line args (in case options have space in them).
      
      Author: Ori Kremer <ori.kremer@gmail.com>
      
      Closes #1169 from orikremer/quote_cmd_line_args and squashes the following commits:
      
      67e2aa1 [Ori Kremer] quote command line args
      9fc373e3
  5. Jul 16, 2013
  6. Jun 11, 2012
Loading