-
- Downloads
[SPARK-13266] [SQL] None read/writer options were not transalated to "null"
## What changes were proposed in this pull request? In Python, the `option` and `options` method of `DataFrameReader` and `DataFrameWriter` were sending the string "None" instead of `null` when passed `None`, therefore making it impossible to send an actual `null`. This fixes that problem. This is based on #11305 from mathieulongtin. ## How was this patch tested? Added test to readwriter.py. Author: Liang-Chi Hsieh <simonh@tw.ibm.com> Author: mathieu longtin <mathieu.longtin@nuance.com> Closes #12494 from viirya/py-df-none-option.
Showing
- python/pyspark/sql/readwriter.py 6 additions, 3 deletionspython/pyspark/sql/readwriter.py
- python/pyspark/sql/tests.py 3 additions, 0 deletionspython/pyspark/sql/tests.py
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVOptions.scala 5 additions, 1 deletion...ache/spark/sql/execution/datasources/csv/CSVOptions.scala
Loading
Please register or sign in to comment