-
- Downloads
[SPARK-17693][SQL] Fixed Insert Failure To Data Source Tables when the Schema has the Comment Field
### What changes were proposed in this pull request? ```SQL CREATE TABLE tab1(col1 int COMMENT 'a', col2 int) USING parquet INSERT INTO TABLE tab1 SELECT 1, 2 ``` The insert attempt will fail if the target table has a column with comments. The error is strange to the external users: ``` assertion failed: No plan for InsertIntoTable Relation[col1#15,col2#16] parquet, false, false +- Project [1 AS col1#19, 2 AS col2#20] +- OneRowRelation$ ``` This PR is to fix the above bug by checking the metadata when comparing the schema between the table and the query. If not matched, we also copy the metadata. This is an alternative to https://github.com/apache/spark/pull/15266 ### How was this patch tested? Added a test case Author: gatorsmile <gatorsmile@gmail.com> Closes #15615 from gatorsmile/insertDataSourceTableWithCommentSolution2.
Showing
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/rules.scala 8 additions, 2 deletions...la/org/apache/spark/sql/execution/datasources/rules.scala
- sql/core/src/test/scala/org/apache/spark/sql/sources/InsertSuite.scala 42 additions, 0 deletions...test/scala/org/apache/spark/sql/sources/InsertSuite.scala
Please register or sign in to comment