-
- Downloads
[SPARK-17410][SPARK-17284] Move Hive-generated Stats Info to HiveClientImpl
### What changes were proposed in this pull request? After we adding a new field `stats` into `CatalogTable`, we should not expose Hive-specific Stats metadata to `MetastoreRelation`. It complicates all the related codes. It also introduces a bug in `SHOW CREATE TABLE`. The statistics-related table properties should be skipped by `SHOW CREATE TABLE`, since it could be incorrect in the newly created table. See the Hive JIRA: https://issues.apache.org/jira/browse/HIVE-13792 Also fix the issue to fill Hive-generated RowCounts to our stats. This PR is to handle Hive-specific Stats metadata in `HiveClientImpl`. ### How was this patch tested? Added a few test cases. Author: Xiao Li <gatorsmile@gmail.com> Closes #14971 from gatorsmile/showCreateTableNew.
Showing
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala 4 additions, 1 deletion...ala/org/apache/spark/sql/catalyst/catalog/interface.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/trees/TreeNodeSuite.scala 2 additions, 1 deletion...a/org/apache/spark/sql/catalyst/trees/TreeNodeSuite.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala 1 addition, 14 deletions...main/scala/org/apache/spark/sql/hive/HiveStrategies.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala 63 additions, 4 deletions...ala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
- sql/hive/src/test/java/org/apache/spark/sql/hive/test/TestHiveSingleton.scala 5 additions, 1 deletion...va/org/apache/spark/sql/hive/test/TestHiveSingleton.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveDDLCommandSuite.scala 0 additions, 1 deletion...scala/org/apache/spark/sql/hive/HiveDDLCommandSuite.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogBackwardCompatibilitySuite.scala 0 additions, 4 deletions.../hive/HiveExternalCatalogBackwardCompatibilitySuite.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/MetastoreDataSourcesSuite.scala 0 additions, 5 deletions...org/apache/spark/sql/hive/MetastoreDataSourcesSuite.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/ShowCreateTableSuite.scala 4 additions, 10 deletions...cala/org/apache/spark/sql/hive/ShowCreateTableSuite.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/StatisticsSuite.scala 211 additions, 4 deletions...est/scala/org/apache/spark/sql/hive/StatisticsSuite.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/client/VersionsSuite.scala 1 addition, 1 deletion...cala/org/apache/spark/sql/hive/client/VersionsSuite.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveComparisonTest.scala 0 additions, 5 deletions.../apache/spark/sql/hive/execution/HiveComparisonTest.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala 0 additions, 5 deletions...la/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveSQLViewSuite.scala 0 additions, 2 deletions...rg/apache/spark/sql/hive/execution/HiveSQLViewSuite.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/OrcSourceSuite.scala 0 additions, 1 deletion.../scala/org/apache/spark/sql/hive/orc/OrcSourceSuite.scala
Loading
Please register or sign in to comment