-
- Downloads
[SPARK-19157][SQL] should be able to change spark.sql.runSQLOnFiles at runtime
## What changes were proposed in this pull request? The analyzer rule that supports to query files directly will be added to `Analyzer.extendedResolutionRules` when SparkSession is created, according to the `spark.sql.runSQLOnFiles` flag. If the flag is off when we create `SparkSession`, this rule is not added and we can not query files directly even we turn on the flag later. This PR fixes this bug by always adding that rule to `Analyzer.extendedResolutionRules`. ## How was this patch tested? new regression test Author: Wenchen Fan <wenchen@databricks.com> Closes #16531 from cloud-fan/sql-on-files.
Showing
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/rules.scala 12 additions, 9 deletions...la/org/apache/spark/sql/execution/datasources/rules.scala
- sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala 1 addition, 1 deletion...in/scala/org/apache/spark/sql/internal/SessionState.scala
- sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala 23 additions, 0 deletions...e/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionState.scala 1 addition, 1 deletion...in/scala/org/apache/spark/sql/hive/HiveSessionState.scala
Loading
Please register or sign in to comment