Skip to content
Snippets Groups Projects
  1. Feb 01, 2013
    • Josh Rosen's avatar
      Do not launch JavaGateways on workers (SPARK-674). · 9cc6ff9c
      Josh Rosen authored
      The problem was that the gateway was being initialized whenever the
      pyspark.context module was loaded.  The fix uses lazy initialization
      that occurs only when SparkContext instances are actually constructed.
      
      I also made the gateway and jvm variables private.
      
      This change results in ~3-4x performance improvement when running the
      PySpark unit tests.
      9cc6ff9c
  2. Jan 23, 2013
  3. Jan 21, 2013
Loading