-
- Downloads
[SPARK-21839][SQL] Support SQL config for ORC compression
## What changes were proposed in this pull request? This PR aims to support `spark.sql.orc.compression.codec` like Parquet's `spark.sql.parquet.compression.codec`. Users can use SQLConf to control ORC compression, too. ## How was this patch tested? Pass the Jenkins with new and updated test cases. Author: Dongjoon Hyun <dongjoon@apache.org> Closes #19055 from dongjoon-hyun/SPARK-21839.
Showing
- python/pyspark/sql/readwriter.py 3 additions, 2 deletionspython/pyspark/sql/readwriter.py
- sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala 10 additions, 0 deletions...rc/main/scala/org/apache/spark/sql/internal/SQLConf.scala
- sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala 5 additions, 3 deletions...src/main/scala/org/apache/spark/sql/DataFrameWriter.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcFileFormat.scala 1 addition, 1 deletion...n/scala/org/apache/spark/sql/hive/orc/OrcFileFormat.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcOptions.scala 11 additions, 7 deletions...main/scala/org/apache/spark/sql/hive/orc/OrcOptions.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/OrcSourceSuite.scala 27 additions, 2 deletions.../scala/org/apache/spark/sql/hive/orc/OrcSourceSuite.scala
Loading
Please register or sign in to comment