-
- Downloads
[SPARK-14945][PYTHON] SparkSession Python API
## What changes were proposed in this pull request? ``` Welcome to ____ __ / __/__ ___ _____/ /__ _\ \/ _ \/ _ `/ __/ '_/ /__ / .__/\_,_/_/ /_/\_\ version 2.0.0-SNAPSHOT /_/ Using Python version 2.7.5 (default, Mar 9 2014 22:15:05) SparkSession available as 'spark'. >>> spark <pyspark.sql.session.SparkSession object at 0x101f3bfd0> >>> spark.sql("SHOW TABLES").show() ... +---------+-----------+ |tableName|isTemporary| +---------+-----------+ | src| false| +---------+-----------+ >>> spark.range(1, 10, 2).show() +---+ | id| +---+ | 1| | 3| | 5| | 7| | 9| +---+ ``` **Note**: This API is NOT complete in its current state. In particular, for now I left out the `conf` and `catalog` APIs, which were added later in Scala. These will be added later before 2.0. ## How was this patch tested? Python tests. Author: Andrew Or <andrew@databricks.com> Closes #12746 from andrewor14/python-spark-session.
Showing
- python/pyspark/shell.py 6 additions, 5 deletionspython/pyspark/shell.py
- python/pyspark/sql/__init__.py 3 additions, 2 deletionspython/pyspark/sql/__init__.py
- python/pyspark/sql/context.py 49 additions, 229 deletionspython/pyspark/sql/context.py
- python/pyspark/sql/readwriter.py 1 addition, 1 deletionpython/pyspark/sql/readwriter.py
- python/pyspark/sql/session.py 525 additions, 0 deletionspython/pyspark/sql/session.py
- python/pyspark/sql/tests.py 1 addition, 3 deletionspython/pyspark/sql/tests.py
Loading
Please register or sign in to comment