-
- Downloads
[SPARK-3007][SQL]Add Dynamic Partition support to Spark Sql hive
a new PR base on new master. changes are the same as https://github.com/apache/spark/pull/1919 Author: baishuo(白硕) <vc_java@hotmail.com> Author: baishuo <vc_java@hotmail.com> Author: Cheng Lian <lian.cs.zju@gmail.com> Closes #2226 from baishuo/patch-3007 and squashes the following commits: e69ce88 [Cheng Lian] Adds tests to verify dynamic partitioning folder layout b20a3dc [Cheng Lian] Addresses @yhuai's comments 096bbbc [baishuo(白硕)] Merge pull request #1 from liancheng/refactor-dp 1093c20 [Cheng Lian] Adds more tests 5004542 [Cheng Lian] Minor refactoring fae9eff [Cheng Lian] Refactors InsertIntoHiveTable to a Command 528e84c [Cheng Lian] Fixes typo in test name, regenerated golden answer files c464b26 [Cheng Lian] Refactors dynamic partitioning support 5033928 [baishuo] pass check style 2201c75 [baishuo] use HiveConf.DEFAULTPARTITIONNAME to replace hive.exec.default.partition.name b47c9bf [baishuo] modify according micheal's advice c3ab36d [baishuo] modify for some bad indentation 7ce2d9f [baishuo] modify code to pass scala style checks 37c1c43 [baishuo] delete a empty else branch 66e33fc [baishuo] do a little modify 88d0110 [baishuo] update file after test a3961d9 [baishuo(白硕)] Update Cast.scala f7467d0 [baishuo(白硕)] Update InsertIntoHiveTable.scala c1a59dd [baishuo(白硕)] Update Cast.scala 0e18496 [baishuo(白硕)] Update HiveQuerySuite.scala 60f70aa [baishuo(白硕)] Update InsertIntoHiveTable.scala 0a50db9 [baishuo(白硕)] Update HiveCompatibilitySuite.scala 491c7d0 [baishuo(白硕)] Update InsertIntoHiveTable.scala a2374a8 [baishuo(白硕)] Update InsertIntoHiveTable.scala 701a814 [baishuo(白硕)] Update SparkHadoopWriter.scala dc24c41 [baishuo(白硕)] Update HiveQl.scala
Showing
- sql/hive/compatibility/src/test/scala/org/apache/spark/sql/hive/execution/HiveCompatibilitySuite.scala 17 additions, 0 deletions...che/spark/sql/hive/execution/HiveCompatibilitySuite.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveQl.scala 0 additions, 5 deletions...ive/src/main/scala/org/apache/spark/sql/hive/HiveQl.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala 114 additions, 93 deletions...apache/spark/sql/hive/execution/InsertIntoHiveTable.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveWriterContainers.scala 217 additions, 0 deletions...cala/org/apache/spark/sql/hive/hiveWriterContainers.scala
- sql/hive/src/test/resources/golden/dynamic_partition-0-be33aaa7253c8f248ff3921cd7dae340 0 additions, 0 deletions...lden/dynamic_partition-0-be33aaa7253c8f248ff3921cd7dae340
- sql/hive/src/test/resources/golden/dynamic_partition-1-640552dd462707563fd255a713f83b41 0 additions, 0 deletions...lden/dynamic_partition-1-640552dd462707563fd255a713f83b41
- sql/hive/src/test/resources/golden/dynamic_partition-2-36456c9d0d2e3ef72ab5ba9ba48e5493 1 addition, 0 deletions...lden/dynamic_partition-2-36456c9d0d2e3ef72ab5ba9ba48e5493
- sql/hive/src/test/resources/golden/dynamic_partition-3-b7f7fa7ebf666f4fee27e149d8c6961f 0 additions, 0 deletions...lden/dynamic_partition-3-b7f7fa7ebf666f4fee27e149d8c6961f
- sql/hive/src/test/resources/golden/dynamic_partition-4-8bdb71ad8cb3cc3026043def2525de3a 0 additions, 0 deletions...lden/dynamic_partition-4-8bdb71ad8cb3cc3026043def2525de3a
- sql/hive/src/test/resources/golden/dynamic_partition-5-c630dce438f3792e7fb0f523fbbb3e1e 0 additions, 0 deletions...lden/dynamic_partition-5-c630dce438f3792e7fb0f523fbbb3e1e
- sql/hive/src/test/resources/golden/dynamic_partition-6-7abc9ec8a36cdc5e89e955265a7fd7cf 0 additions, 0 deletions...lden/dynamic_partition-6-7abc9ec8a36cdc5e89e955265a7fd7cf
- sql/hive/src/test/resources/golden/dynamic_partition-7-be33aaa7253c8f248ff3921cd7dae340 0 additions, 0 deletions...lden/dynamic_partition-7-be33aaa7253c8f248ff3921cd7dae340
- sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveQuerySuite.scala 94 additions, 6 deletions.../org/apache/spark/sql/hive/execution/HiveQuerySuite.scala
Loading
Please register or sign in to comment