-
- Downloads
[SPARK-14014][SQL] Replace existing catalog with SessionCatalog
## What changes were proposed in this pull request? `SessionCatalog`, introduced in #11750, is a catalog that keeps track of temporary functions and tables, and delegates metastore operations to `ExternalCatalog`. This functionality overlaps a lot with the existing `analysis.Catalog`. As of this commit, `SessionCatalog` and `ExternalCatalog` will no longer be dead code. There are still things that need to be done after this patch, namely: - SPARK-14013: Properly implement temporary functions in `SessionCatalog` - SPARK-13879: Decide which DDL/DML commands to support natively in Spark - SPARK-?????: Implement the ones we do want to support through `SessionCatalog`. - SPARK-?????: Merge SQL/HiveContext ## How was this patch tested? This is largely a refactoring task so there are no new tests introduced. The particularly relevant tests are `SessionCatalogSuite` and `ExternalCatalogSuite`. Author: Andrew Or <andrew@databricks.com> Author: Yin Huai <yhuai@databricks.com> Closes #11836 from andrewor14/use-session-catalog.
Showing
- R/pkg/inst/tests/testthat/test_sparkSQL.R 2 additions, 1 deletionR/pkg/inst/tests/testthat/test_sparkSQL.R
- project/MimaExcludes.scala 3 additions, 0 deletionsproject/MimaExcludes.scala
- python/pyspark/sql/context.py 1 addition, 1 deletionpython/pyspark/sql/context.py
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala 10 additions, 10 deletions...ala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Catalog.scala 0 additions, 218 deletions...cala/org/apache/spark/sql/catalyst/analysis/Catalog.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/unresolved.scala 1 addition, 1 deletion...a/org/apache/spark/sql/catalyst/analysis/unresolved.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/InMemoryCatalog.scala 18 additions, 17 deletions...g/apache/spark/sql/catalyst/catalog/InMemoryCatalog.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala 90 additions, 33 deletions...rg/apache/spark/sql/catalyst/catalog/SessionCatalog.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala 2 additions, 0 deletions...ala/org/apache/spark/sql/catalyst/catalog/interface.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisSuite.scala 1 addition, 5 deletions...rg/apache/spark/sql/catalyst/analysis/AnalysisSuite.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisTest.scala 9 additions, 14 deletions...org/apache/spark/sql/catalyst/analysis/AnalysisTest.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/DecimalPrecisionSuite.scala 13 additions, 12 deletions...e/spark/sql/catalyst/analysis/DecimalPrecisionSuite.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/CatalogTestCases.scala 2 additions, 1 deletion.../apache/spark/sql/catalyst/catalog/CatalogTestCases.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalogSuite.scala 19 additions, 1 deletion...ache/spark/sql/catalyst/catalog/SessionCatalogSuite.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/BooleanSimplificationSuite.scala 6 additions, 5 deletions...k/sql/catalyst/optimizer/BooleanSimplificationSuite.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/EliminateSortsSuite.scala 3 additions, 2 deletions...he/spark/sql/catalyst/optimizer/EliminateSortsSuite.scala
- sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala 35 additions, 38 deletions...core/src/main/scala/org/apache/spark/sql/SQLContext.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/command/commands.scala 5 additions, 3 deletions...ala/org/apache/spark/sql/execution/command/commands.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/ddl.scala 18 additions, 6 deletions...cala/org/apache/spark/sql/execution/datasources/ddl.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/rules.scala 7 additions, 3 deletions...la/org/apache/spark/sql/execution/datasources/rules.scala
Loading
Please register or sign in to comment