-
- Downloads
[SPARK-18464][SQL] support old table which doesn't store schema in metastore
## What changes were proposed in this pull request? Before Spark 2.1, users can create an external data source table without schema, and we will infer the table schema at runtime. In Spark 2.1, we decided to infer the schema when the table was created, so that we don't need to infer it again and again at runtime. This is a good improvement, but we should still respect and support old tables which doesn't store table schema in metastore. ## How was this patch tested? regression test. Author: Wenchen Fan <wenchen@databricks.com> Closes #15900 from cloud-fan/hive-catalog. (cherry picked from commit 07b3f045) Signed-off-by:Reynold Xin <rxin@databricks.com>
Showing
- sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala 7 additions, 1 deletion...scala/org/apache/spark/sql/execution/command/tables.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala 5 additions, 0 deletions...scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala 3 additions, 1 deletion...cala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/MetastoreDataSourcesSuite.scala 22 additions, 0 deletions...org/apache/spark/sql/hive/MetastoreDataSourcesSuite.scala
Loading
Please register or sign in to comment