-
- Downloads
[SPARK-19949][SQL][FOLLOW-UP] Clean up parse modes and update related comments
## What changes were proposed in this pull request? This PR proposes to make `mode` options in both CSV and JSON to use `cass object` and fix some related comments related previous fix. Also, this PR modifies some tests related parse modes. ## How was this patch tested? Modified unit tests in both `CSVSuite.scala` and `JsonSuite.scala`. Author: hyukjinkwon <gurwls223@gmail.com> Closes #17377 from HyukjinKwon/SPARK-19949.
Showing
- python/pyspark/sql/readwriter.py 2 additions, 4 deletionspython/pyspark/sql/readwriter.py
- python/pyspark/sql/streaming.py 2 additions, 0 deletionspython/pyspark/sql/streaming.py
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala 2 additions, 2 deletions...ache/spark/sql/catalyst/expressions/jsonExpressions.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JSONOptions.scala 2 additions, 10 deletions...cala/org/apache/spark/sql/catalyst/json/JSONOptions.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/FailureSafeParser.scala 9 additions, 6 deletions...rg/apache/spark/sql/catalyst/util/FailureSafeParser.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/ParseMode.scala 56 additions, 0 deletions.../scala/org/apache/spark/sql/catalyst/util/ParseMode.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/JsonExpressionsSuite.scala 2 additions, 2 deletions...spark/sql/catalyst/expressions/JsonExpressionsSuite.scala
- sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala 1 addition, 3 deletions...src/main/scala/org/apache/spark/sql/DataFrameReader.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVOptions.scala 2 additions, 12 deletions...ache/spark/sql/execution/datasources/csv/CSVOptions.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JsonInferSchema.scala 2 additions, 1 deletion...park/sql/execution/datasources/json/JsonInferSchema.scala
- sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamReader.scala 1 addition, 1 deletion...ala/org/apache/spark/sql/streaming/DataStreamReader.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala 4 additions, 3 deletions...apache/spark/sql/execution/datasources/csv/CSVSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala 45 additions, 68 deletions...ache/spark/sql/execution/datasources/json/JsonSuite.scala
Loading
Please register or sign in to comment