-
- Downloads
[SPARK-18647][SQL] do not put provider in table properties for Hive serde table
## What changes were proposed in this pull request? In Spark 2.1, we make Hive serde tables case-preserving by putting the table metadata in table properties, like what we did for data source table. However, we should not put table provider, as it will break forward compatibility. e.g. if we create a Hive serde table with Spark 2.1, using `sql("create table test stored as parquet as select 1")`, we will fail to read it with Spark 2.0, as Spark 2.0 mistakenly treat it as data source table because there is a `provider` entry in table properties. Logically Hive serde table's provider is always hive, we don't need to store it in table properties, this PR removes it. ## How was this patch tested? manually test the forward compatibility issue. Author: Wenchen Fan <wenchen@databricks.com> Closes #16080 from cloud-fan/hive.
Showing
- sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala 41 additions, 39 deletions...scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogSuite.scala 18 additions, 0 deletions.../org/apache/spark/sql/hive/HiveExternalCatalogSuite.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveMetastoreCatalogSuite.scala 0 additions, 2 deletions...org/apache/spark/sql/hive/HiveMetastoreCatalogSuite.scala
Loading
Please register or sign in to comment