-
- Downloads
[SPARK-15719][SQL] Disables writing Parquet summary files by default
## What changes were proposed in this pull request? This PR disables writing Parquet summary files by default (i.e., when Hadoop configuration "parquet.enable.summary-metadata" is not set). Please refer to [SPARK-15719][1] for more details. ## How was this patch tested? New test case added in `ParquetQuerySuite` to check no summary files are written by default. [1]: https://issues.apache.org/jira/browse/SPARK-15719 Author: Cheng Lian <lian@databricks.com> Closes #13455 from liancheng/spark-15719-disable-parquet-summary-files.
Showing
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala 6 additions, 1 deletion...sql/execution/datasources/parquet/ParquetFileFormat.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetIOSuite.scala 11 additions, 9 deletions...rk/sql/execution/datasources/parquet/ParquetIOSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetQuerySuite.scala 26 additions, 5 deletions...sql/execution/datasources/parquet/ParquetQuerySuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaSuite.scala 3 additions, 16 deletions...ql/execution/datasources/parquet/ParquetSchemaSuite.scala
- sql/hive/src/test/scala/org/apache/spark/sql/sources/ParquetHadoopFsRelationSuite.scala 16 additions, 13 deletions...ache/spark/sql/sources/ParquetHadoopFsRelationSuite.scala
Loading
Please register or sign in to comment