-
- Downloads
[SPARK-15312][SQL] Detect Duplicate Key in Partition Spec and Table Properties
#### What changes were proposed in this pull request? When there are duplicate keys in the partition specs or table properties, we always use the last value and ignore all the previous values. This is caused by the function call `toMap`. partition specs or table properties are widely used in multiple DDL statements. This PR is to detect the duplicates and issue an exception if found. #### How was this patch tested? Added test cases in DDLSuite Author: gatorsmile <gatorsmile@gmail.com> Closes #13095 from gatorsmile/detectDuplicate.
Showing
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala 6 additions, 9 deletions...ala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ParserUtils.scala 7 additions, 0 deletions...la/org/apache/spark/sql/catalyst/parser/ParserUtils.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/PlanParserSuite.scala 1 addition, 1 deletion...rg/apache/spark/sql/catalyst/parser/PlanParserSuite.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala 5 additions, 2 deletions...scala/org/apache/spark/sql/execution/SparkSqlParser.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLCommandSuite.scala 15 additions, 0 deletions.../apache/spark/sql/execution/command/DDLCommandSuite.scala
Loading
Please register or sign in to comment