-
- Downloads
[SPARK-23448][SQL] Clarify JSON and CSV parser behavior in document
## What changes were proposed in this pull request? Clarify JSON and CSV reader behavior in document. JSON doesn't support partial results for corrupted records. CSV only supports partial results for the records with more or less tokens. ## How was this patch tested? Pass existing tests. Author: Liang-Chi Hsieh <viirya@gmail.com> Closes #20666 from viirya/SPARK-23448-2.
Showing
- python/pyspark/sql/readwriter.py 16 additions, 14 deletionspython/pyspark/sql/readwriter.py
- python/pyspark/sql/streaming.py 16 additions, 14 deletionspython/pyspark/sql/streaming.py
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonParser.scala 3 additions, 0 deletions...la/org/apache/spark/sql/catalyst/json/JacksonParser.scala
- sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala 12 additions, 10 deletions...src/main/scala/org/apache/spark/sql/DataFrameReader.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala 5 additions, 0 deletions...spark/sql/execution/datasources/csv/UnivocityParser.scala
- sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamReader.scala 12 additions, 10 deletions...ala/org/apache/spark/sql/streaming/DataStreamReader.scala
Loading
Please register or sign in to comment