Skip to content
Snippets Groups Projects
  • Dongjoon Hyun's avatar
    c8d0aba1
    [SPARK-21278][PYSPARK] Upgrade to Py4J 0.10.6 · c8d0aba1
    Dongjoon Hyun authored
    ## What changes were proposed in this pull request?
    
    This PR aims to bump Py4J in order to fix the following float/double bug.
    Py4J 0.10.5 fixes this (https://github.com/bartdag/py4j/issues/272) and the latest Py4J is 0.10.6.
    
    **BEFORE**
    ```
    >>> df = spark.range(1)
    >>> df.select(df['id'] + 17.133574204226083).show()
    +--------------------+
    |(id + 17.1335742042)|
    +--------------------+
    |       17.1335742042|
    +--------------------+
    ```
    
    **AFTER**
    ```
    >>> df = spark.range(1)
    >>> df.select(df['id'] + 17.133574204226083).show()
    +-------------------------+
    |(id + 17.133574204226083)|
    +-------------------------+
    |       17.133574204226083|
    +-------------------------+
    ```
    
    ## How was this patch tested?
    
    Manual.
    
    Author: Dongjoon Hyun <dongjoon@apache.org>
    
    Closes #18546 from dongjoon-hyun/SPARK-21278.
    c8d0aba1
    History
    [SPARK-21278][PYSPARK] Upgrade to Py4J 0.10.6
    Dongjoon Hyun authored
    ## What changes were proposed in this pull request?
    
    This PR aims to bump Py4J in order to fix the following float/double bug.
    Py4J 0.10.5 fixes this (https://github.com/bartdag/py4j/issues/272) and the latest Py4J is 0.10.6.
    
    **BEFORE**
    ```
    >>> df = spark.range(1)
    >>> df.select(df['id'] + 17.133574204226083).show()
    +--------------------+
    |(id + 17.1335742042)|
    +--------------------+
    |       17.1335742042|
    +--------------------+
    ```
    
    **AFTER**
    ```
    >>> df = spark.range(1)
    >>> df.select(df['id'] + 17.133574204226083).show()
    +-------------------------+
    |(id + 17.133574204226083)|
    +-------------------------+
    |       17.133574204226083|
    +-------------------------+
    ```
    
    ## How was this patch tested?
    
    Manual.
    
    Author: Dongjoon Hyun <dongjoon@apache.org>
    
    Closes #18546 from dongjoon-hyun/SPARK-21278.
This project is licensed under the Apache License 2.0. Learn more