Skip to content
Snippets Groups Projects
  • Andrew Or's avatar
    89addd40
    [SPARK-14945][PYTHON] SparkSession Python API · 89addd40
    Andrew Or authored
    ## What changes were proposed in this pull request?
    
    ```
    Welcome to
          ____              __
         / __/__  ___ _____/ /__
        _\ \/ _ \/ _ `/ __/  '_/
       /__ / .__/\_,_/_/ /_/\_\   version 2.0.0-SNAPSHOT
          /_/
    
    Using Python version 2.7.5 (default, Mar  9 2014 22:15:05)
    SparkSession available as 'spark'.
    >>> spark
    <pyspark.sql.session.SparkSession object at 0x101f3bfd0>
    >>> spark.sql("SHOW TABLES").show()
    ...
    +---------+-----------+
    |tableName|isTemporary|
    +---------+-----------+
    |      src|      false|
    +---------+-----------+
    
    >>> spark.range(1, 10, 2).show()
    +---+
    | id|
    +---+
    |  1|
    |  3|
    |  5|
    |  7|
    |  9|
    +---+
    ```
    **Note**: This API is NOT complete in its current state. In particular, for now I left out the `conf` and `catalog` APIs, which were added later in Scala. These will be added later before 2.0.
    
    ## How was this patch tested?
    
    Python tests.
    
    Author: Andrew Or <andrew@databricks.com>
    
    Closes #12746 from andrewor14/python-spark-session.
    89addd40
    History
    [SPARK-14945][PYTHON] SparkSession Python API
    Andrew Or authored
    ## What changes were proposed in this pull request?
    
    ```
    Welcome to
          ____              __
         / __/__  ___ _____/ /__
        _\ \/ _ \/ _ `/ __/  '_/
       /__ / .__/\_,_/_/ /_/\_\   version 2.0.0-SNAPSHOT
          /_/
    
    Using Python version 2.7.5 (default, Mar  9 2014 22:15:05)
    SparkSession available as 'spark'.
    >>> spark
    <pyspark.sql.session.SparkSession object at 0x101f3bfd0>
    >>> spark.sql("SHOW TABLES").show()
    ...
    +---------+-----------+
    |tableName|isTemporary|
    +---------+-----------+
    |      src|      false|
    +---------+-----------+
    
    >>> spark.range(1, 10, 2).show()
    +---+
    | id|
    +---+
    |  1|
    |  3|
    |  5|
    |  7|
    |  9|
    +---+
    ```
    **Note**: This API is NOT complete in its current state. In particular, for now I left out the `conf` and `catalog` APIs, which were added later in Scala. These will be added later before 2.0.
    
    ## How was this patch tested?
    
    Python tests.
    
    Author: Andrew Or <andrew@databricks.com>
    
    Closes #12746 from andrewor14/python-spark-session.
context.py 19.89 KiB