-
- Downloads
[SQL] SPARK-1800 Add broadcast hash join operator & associated hints.
This PR is based off Michael's [PR 734](https://github.com/apache/spark/pull/734) and includes a bunch of cleanups. Moreover, this PR also - makes `SparkLogicalPlan` take a `tableName: String`, which facilitates testing. - moves join-related tests to a single file. Author: Zongheng Yang <zongheng.y@gmail.com> Author: Michael Armbrust <michael@databricks.com> Closes #1163 from concretevitamin/auto-broadcast-hash-join and squashes the following commits: d0f4991 [Zongheng Yang] Fix bug in broadcast hash join & add test to cover it. af080d7 [Zongheng Yang] Fix in joinIterators()'s next(). 440d277 [Zongheng Yang] Fixes to imports; add back requiredChildDistribution (lost when merging) 208d5f6 [Zongheng Yang] Make LeftSemiJoinHash mix in HashJoin. ad6c7cc [Zongheng Yang] Minor cleanups. 814b3bf [Zongheng Yang] Merge branch 'master' into auto-broadcast-hash-join a8a093e [Zongheng Yang] Minor cleanups. 6fd8443 [Zongheng Yang] Cut down size estimation related stuff. a4267be [Zongheng Yang] Add test for broadcast hash join and related necessary refactorings: 0e64b08 [Zongheng Yang] Scalastyle fix. 91461c2 [Zongheng Yang] Merge branch 'master' into auto-broadcast-hash-join 7c7158b [Zongheng Yang] Prototype of auto conversion to broadcast hash join. 0ad122f [Zongheng Yang] Merge branch 'master' into auto-broadcast-hash-join 3e5d77c [Zongheng Yang] WIP: giant and messy WIP. a92ed0c [Michael Armbrust] Formatting. 76ca434 [Michael Armbrust] A simple strategy that broadcasts tables only when they are found in a configuration hint. cf6b381 [Michael Armbrust] Split out generic logic for hash joins and create two concrete physical operators: BroadcastHashJoin and ShuffledHashJoin. a8420ca [Michael Armbrust] Copy records in executeCollect to avoid issues with mutable rows.
Showing
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Projection.scala 5 additions, 3 deletions...rg/apache/spark/sql/catalyst/expressions/Projection.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/BaseRelation.scala 0 additions, 1 deletion...pache/spark/sql/catalyst/plans/logical/BaseRelation.scala
- sql/core/src/main/scala/org/apache/spark/sql/SQLConf.scala 17 additions, 0 deletionssql/core/src/main/scala/org/apache/spark/sql/SQLConf.scala
- sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala 5 additions, 1 deletion...core/src/main/scala/org/apache/spark/sql/SQLContext.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/SparkPlan.scala 8 additions, 7 deletions...main/scala/org/apache/spark/sql/execution/SparkPlan.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala 46 additions, 8 deletions...cala/org/apache/spark/sql/execution/SparkStrategies.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/basicOperators.scala 0 additions, 1 deletion...scala/org/apache/spark/sql/execution/basicOperators.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/joins.scala 133 additions, 86 deletions...src/main/scala/org/apache/spark/sql/execution/joins.scala
- sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetRelation.scala 3 additions, 2 deletions.../scala/org/apache/spark/sql/parquet/ParquetRelation.scala
- sql/core/src/test/scala/org/apache/spark/sql/DslQuerySuite.scala 0 additions, 99 deletions...e/src/test/scala/org/apache/spark/sql/DslQuerySuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/JoinSuite.scala 173 additions, 0 deletionssql/core/src/test/scala/org/apache/spark/sql/JoinSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/QueryTest.scala 2 additions, 2 deletionssql/core/src/test/scala/org/apache/spark/sql/QueryTest.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/PlannerSuite.scala 0 additions, 17 deletions...t/scala/org/apache/spark/sql/execution/PlannerSuite.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveContext.scala 1 addition, 1 deletion...rc/main/scala/org/apache/spark/sql/hive/HiveContext.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala 2 additions, 5 deletions...cala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala
Loading
Please register or sign in to comment