-
- Downloads
[SPARK-19152][SQL] DataFrameWriter.saveAsTable support hive append
## What changes were proposed in this pull request? After [SPARK-19107](https://issues.apache.org/jira/browse/SPARK-19107), we now can treat hive as a data source and create hive tables with DataFrameWriter and Catalog. However, the support is not completed, there are still some cases we do not support. This PR implement: DataFrameWriter.saveAsTable work with hive format with append mode ## How was this patch tested? unit test added Author: windpiger <songjun@outlook.com> Closes #16552 from windpiger/saveAsTableWithHiveAppend.
Showing
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/ddl.scala 6 additions, 1 deletion...cala/org/apache/spark/sql/execution/datasources/ddl.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/rules.scala 1 addition, 6 deletions...la/org/apache/spark/sql/execution/datasources/rules.scala
- sql/hive/src/main/resources/META-INF/services/org.apache.spark.sql.sources.DataSourceRegister 1 addition, 0 deletions.../services/org.apache.spark.sql.sources.DataSourceRegister
- sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala 1 addition, 8 deletions...main/scala/org/apache/spark/sql/hive/HiveStrategies.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/CreateHiveTableAsSelectCommand.scala 14 additions, 8 deletions...k/sql/hive/execution/CreateHiveTableAsSelectCommand.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveFileFormat.scala 8 additions, 1 deletion.../org/apache/spark/sql/hive/execution/HiveFileFormat.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/MetastoreDataSourcesSuite.scala 4 additions, 9 deletions...org/apache/spark/sql/hive/MetastoreDataSourcesSuite.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala 31 additions, 5 deletions...la/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala 17 additions, 0 deletions...a/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala
Loading
Please register or sign in to comment