-
- Downloads
[MINOR][SQL] Add missing functions for some options in SQLConf and use them where applicable
## What changes were proposed in this pull request? I first thought they are missing because they are kind of hidden options but it seems they are just missing. For example, `spark.sql.parquet.mergeSchema` is documented in [sql-programming-guide.md](https://github.com/apache/spark/blob/master/docs/sql-programming-guide.md) but this function is missing whereas many options such as `spark.sql.join.preferSortMergeJoin` are not documented but have its own function individually. So, this PR suggests making them consistent by adding the missing functions for some options in `SQLConf` and use them where applicable, in order to make them more readable. ## How was this patch tested? Existing tests should cover this. Author: hyukjinkwon <gurwls223@gmail.com> Closes #14678 from HyukjinKwon/sqlconf-cleanup.
Showing
- sql/core/src/main/scala/org/apache/spark/sql/RelationalGroupedDataset.scala 1 addition, 1 deletion...scala/org/apache/spark/sql/RelationalGroupedDataset.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala 1 addition, 1 deletion...scala/org/apache/spark/sql/execution/QueryExecution.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala 1 addition, 1 deletion...g/apache/spark/sql/execution/datasources/DataSource.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala 1 addition, 1 deletion...ution/datasources/InsertIntoHadoopFsRelationCommand.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningAwareFileCatalog.scala 1 addition, 1 deletion.../execution/datasources/PartitioningAwareFileCatalog.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala 4 additions, 4 deletions...sql/execution/datasources/parquet/ParquetFileFormat.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetOptions.scala 1 addition, 1 deletion...rk/sql/execution/datasources/parquet/ParquetOptions.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/FileStreamSinkLog.scala 3 additions, 3 deletions...che/spark/sql/execution/streaming/FileStreamSinkLog.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala 1 addition, 1 deletion...pache/spark/sql/execution/streaming/StreamExecution.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StateStoreConf.scala 2 additions, 4 deletions.../spark/sql/execution/streaming/state/StateStoreConf.scala
- sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala 31 additions, 11 deletions...rc/main/scala/org/apache/spark/sql/internal/SQLConf.scala
- sql/core/src/main/scala/org/apache/spark/sql/streaming/StreamingQueryManager.scala 2 additions, 2 deletions...rg/apache/spark/sql/streaming/StreamingQueryManager.scala
Loading
Please register or sign in to comment