-
- Downloads
[SPARK-13503][SQL] Support to specify the (writing) option for compression codec for TEXT
## What changes were proposed in this pull request? https://issues.apache.org/jira/browse/SPARK-13503 This PR makes the TEXT datasource can compress output by option instead of manually setting Hadoop configurations. For reflecting codec by names, it is similar with https://github.com/apache/spark/pull/10805 and https://github.com/apache/spark/pull/10858. ## How was this patch tested? This was tested with unittests and with `dev/run_tests` for coding style Author: hyukjinkwon <gurwls223@gmail.com> Closes #11384 from HyukjinKwon/SPARK-13503.
Showing
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/CompressionCodecs.scala 14 additions, 0 deletions...e/spark/sql/execution/datasources/CompressionCodecs.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVRelation.scala 16 additions, 20 deletions...che/spark/sql/execution/datasources/csv/CSVRelation.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JSONRelation.scala 1 addition, 5 deletions...e/spark/sql/execution/datasources/json/JSONRelation.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/text/DefaultSource.scala 11 additions, 2 deletions.../spark/sql/execution/datasources/text/DefaultSource.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/text/TextSuite.scala 15 additions, 0 deletions...ache/spark/sql/execution/datasources/text/TextSuite.scala
Loading
Please register or sign in to comment