-
- Downloads
[SPARK-17720][SQL] introduce static SQL conf
## What changes were proposed in this pull request? SQLConf is session-scoped and mutable. However, we do have the requirement for a static SQL conf, which is global and immutable, e.g. the `schemaStringThreshold` in `HiveExternalCatalog`, the flag to enable/disable hive support, the global temp view database in https://github.com/apache/spark/pull/14897. Actually we've already implemented static SQL conf implicitly via `SparkConf`, this PR just make it explicit and expose it to users, so that they can see the config value via SQL command or `SparkSession.conf`, and forbid users to set/unset static SQL conf. ## How was this patch tested? new tests in SQLConfSuite Author: Wenchen Fan <wenchen@databricks.com> Closes #15295 from cloud-fan/global-conf.
Showing
- R/pkg/inst/tests/testthat/test_sparkSQL.R 1 addition, 1 deletionR/pkg/inst/tests/testthat/test_sparkSQL.R
- core/src/main/scala/org/apache/spark/internal/config/package.scala 0 additions, 14 deletions...main/scala/org/apache/spark/internal/config/package.scala
- python/pyspark/sql/session.py 1 addition, 1 deletionpython/pyspark/sql/session.py
- repl/scala-2.11/src/main/scala/org/apache/spark/repl/Main.scala 1 addition, 1 deletion...cala-2.11/src/main/scala/org/apache/spark/repl/Main.scala
- repl/scala-2.11/src/test/scala/org/apache/spark/repl/ReplSuite.scala 1 addition, 1 deletion...2.11/src/test/scala/org/apache/spark/repl/ReplSuite.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala 1 addition, 2 deletions...rg/apache/spark/sql/catalyst/catalog/SessionCatalog.scala
- sql/core/src/main/scala/org/apache/spark/sql/RuntimeConfig.scala 10 additions, 1 deletion...e/src/main/scala/org/apache/spark/sql/RuntimeConfig.scala
- sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala 4 additions, 4 deletions...re/src/main/scala/org/apache/spark/sql/SparkSession.scala
- sql/core/src/main/scala/org/apache/spark/sql/api/r/SQLUtils.scala 2 additions, 2 deletions.../src/main/scala/org/apache/spark/sql/api/r/SQLUtils.scala
- sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala 39 additions, 17 deletions...rc/main/scala/org/apache/spark/sql/internal/SQLConf.scala
- sql/core/src/main/scala/org/apache/spark/sql/internal/SharedState.scala 1 addition, 0 deletions...ain/scala/org/apache/spark/sql/internal/SharedState.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala 1 addition, 1 deletion...ala/org/apache/spark/sql/execution/command/DDLSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/internal/SQLConfSuite.scala 22 additions, 1 deletion...st/scala/org/apache/spark/sql/internal/SQLConfSuite.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala 2 additions, 5 deletions...scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala 1 addition, 2 deletions.../src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/test/TestHive.scala 1 addition, 1 deletion.../main/scala/org/apache/spark/sql/hive/test/TestHive.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/MetastoreDataSourcesSuite.scala 22 additions, 23 deletions...org/apache/spark/sql/hive/MetastoreDataSourcesSuite.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala 1 addition, 1 deletion...la/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala
Loading
Please register or sign in to comment