-
- Downloads
[SPARK-18567][SQL] Simplify CreateDataSourceTableAsSelectCommand
## What changes were proposed in this pull request? The `CreateDataSourceTableAsSelectCommand` is quite complex now, as it has a lot of work to do if the table already exists: 1. throw exception if we don't want to ignore it. 2. do some check and adjust the schema if we want to append data. 3. drop the table and create it again if we want to overwrite. The work 2 and 3 should be done by analyzer, so that we can also apply it to hive tables. ## How was this patch tested? existing tests. Author: Wenchen Fan <wenchen@databricks.com> Closes #15996 from cloud-fan/append.
Showing
- sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala 46 additions, 32 deletions...src/main/scala/org/apache/spark/sql/DataFrameWriter.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/command/createDataSourceTables.scala 42 additions, 125 deletions.../spark/sql/execution/command/createDataSourceTables.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/rules.scala 124 additions, 40 deletions...la/org/apache/spark/sql/execution/datasources/rules.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/MetastoreDataSourcesSuite.scala 1 addition, 1 deletion...org/apache/spark/sql/hive/MetastoreDataSourcesSuite.scala
Loading
Please register or sign in to comment