Skip to content
Snippets Groups Projects
  • Andrew Or's avatar
    5dfc0197
    [SPARK-14014][SQL] Replace existing catalog with SessionCatalog · 5dfc0197
    Andrew Or authored
    ## What changes were proposed in this pull request?
    
    `SessionCatalog`, introduced in #11750, is a catalog that keeps track of temporary functions and tables, and delegates metastore operations to `ExternalCatalog`. This functionality overlaps a lot with the existing `analysis.Catalog`.
    
    As of this commit, `SessionCatalog` and `ExternalCatalog` will no longer be dead code. There are still things that need to be done after this patch, namely:
    - SPARK-14013: Properly implement temporary functions in `SessionCatalog`
    - SPARK-13879: Decide which DDL/DML commands to support natively in Spark
    - SPARK-?????: Implement the ones we do want to support through `SessionCatalog`.
    - SPARK-?????: Merge SQL/HiveContext
    
    ## How was this patch tested?
    
    This is largely a refactoring task so there are no new tests introduced. The particularly relevant tests are `SessionCatalogSuite` and `ExternalCatalogSuite`.
    
    Author: Andrew Or <andrew@databricks.com>
    Author: Yin Huai <yhuai@databricks.com>
    
    Closes #11836 from andrewor14/use-session-catalog.
    5dfc0197
    History
    [SPARK-14014][SQL] Replace existing catalog with SessionCatalog
    Andrew Or authored
    ## What changes were proposed in this pull request?
    
    `SessionCatalog`, introduced in #11750, is a catalog that keeps track of temporary functions and tables, and delegates metastore operations to `ExternalCatalog`. This functionality overlaps a lot with the existing `analysis.Catalog`.
    
    As of this commit, `SessionCatalog` and `ExternalCatalog` will no longer be dead code. There are still things that need to be done after this patch, namely:
    - SPARK-14013: Properly implement temporary functions in `SessionCatalog`
    - SPARK-13879: Decide which DDL/DML commands to support natively in Spark
    - SPARK-?????: Implement the ones we do want to support through `SessionCatalog`.
    - SPARK-?????: Merge SQL/HiveContext
    
    ## How was this patch tested?
    
    This is largely a refactoring task so there are no new tests introduced. The particularly relevant tests are `SessionCatalogSuite` and `ExternalCatalogSuite`.
    
    Author: Andrew Or <andrew@databricks.com>
    Author: Yin Huai <yhuai@databricks.com>
    
    Closes #11836 from andrewor14/use-session-catalog.
context.py 25.52 KiB