-
- Downloads
[SPARK-17974] Refactor FileCatalog classes to simplify the inheritance tree
## What changes were proposed in this pull request? This renames `BasicFileCatalog => FileCatalog`, combines `SessionFileCatalog` with `PartitioningAwareFileCatalog`, and removes the old `FileCatalog` trait. In summary, ``` MetadataLogFileCatalog extends PartitioningAwareFileCatalog ListingFileCatalog extends PartitioningAwareFileCatalog PartitioningAwareFileCatalog extends FileCatalog TableFileCatalog extends FileCatalog ``` cc cloud-fan mallman ## How was this patch tested? Existing tests Author: Eric Liang <ekl@databricks.com> Closes #15518 from ericl/refactor-session-file-catalog.
Showing
- sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala 1 addition, 1 deletionsql/core/src/main/scala/org/apache/spark/sql/Dataset.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala 2 additions, 2 deletions...a/org/apache/spark/sql/execution/DataSourceScanExec.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileCatalog.scala 66 additions, 0 deletions.../apache/spark/sql/execution/datasources/FileCatalog.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormat.scala 0 additions, 61 deletions...g/apache/spark/sql/execution/datasources/FileFormat.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/HadoopFsRelation.scala 2 additions, 2 deletions...he/spark/sql/execution/datasources/HadoopFsRelation.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningAwareFileCatalog.scala 205 additions, 12 deletions.../execution/datasources/PartitioningAwareFileCatalog.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala 6 additions, 6 deletions...e/spark/sql/execution/datasources/PartitioningUtils.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/SessionFileCatalog.scala 0 additions, 225 deletions.../spark/sql/execution/datasources/SessionFileCatalog.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/TableFileCatalog.scala 5 additions, 6 deletions...he/spark/sql/execution/datasources/TableFileCatalog.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/FileCatalogSuite.scala 10 additions, 0 deletions...he/spark/sql/execution/datasources/FileCatalogSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetPartitionDiscoverySuite.scala 5 additions, 4 deletions.../datasources/parquet/ParquetPartitionDiscoverySuite.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala 1 addition, 1 deletion...cala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala
Loading
Please register or sign in to comment