Skip to content
Snippets Groups Projects
  • Liang-Chi Hsieh's avatar
    056883e0
    [SPARK-13266] [SQL] None read/writer options were not transalated to "null" · 056883e0
    Liang-Chi Hsieh authored
    ## What changes were proposed in this pull request?
    
    In Python, the `option` and `options` method of `DataFrameReader` and `DataFrameWriter` were sending the string "None" instead of `null` when passed `None`, therefore making it impossible to send an actual `null`. This fixes that problem.
    
    This is based on #11305 from mathieulongtin.
    
    ## How was this patch tested?
    
    Added test to readwriter.py.
    
    Author: Liang-Chi Hsieh <simonh@tw.ibm.com>
    Author: mathieu longtin <mathieu.longtin@nuance.com>
    
    Closes #12494 from viirya/py-df-none-option.
    056883e0
    History
    [SPARK-13266] [SQL] None read/writer options were not transalated to "null"
    Liang-Chi Hsieh authored
    ## What changes were proposed in this pull request?
    
    In Python, the `option` and `options` method of `DataFrameReader` and `DataFrameWriter` were sending the string "None" instead of `null` when passed `None`, therefore making it impossible to send an actual `null`. This fixes that problem.
    
    This is based on #11305 from mathieulongtin.
    
    ## How was this patch tested?
    
    Added test to readwriter.py.
    
    Author: Liang-Chi Hsieh <simonh@tw.ibm.com>
    Author: mathieu longtin <mathieu.longtin@nuance.com>
    
    Closes #12494 from viirya/py-df-none-option.