-
- Downloads
[SPARK-12950] [SQL] Improve lookup of BytesToBytesMap in aggregate
This PR improve the lookup of BytesToBytesMap by: 1. Generate code for calculate the hash code of grouping keys. 2. Do not use MemoryLocation, fetch the baseObject and offset for key and value directly (remove the indirection). Author: Davies Liu <davies@databricks.com> Closes #11010 from davies/gen_map.
Showing
- core/src/main/java/org/apache/spark/unsafe/map/BytesToBytesMap.java 61 additions, 47 deletions...ain/java/org/apache/spark/unsafe/map/BytesToBytesMap.java
- core/src/test/java/org/apache/spark/unsafe/map/AbstractBytesToBytesMapSuite.java 35 additions, 29 deletions...apache/spark/unsafe/map/AbstractBytesToBytesMapSuite.java
- project/MimaExcludes.scala 1 addition, 0 deletionsproject/MimaExcludes.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala 0 additions, 1 deletion...cala/org/apache/spark/sql/catalyst/expressions/misc.scala
- sql/core/src/main/java/org/apache/spark/sql/execution/UnsafeFixedWidthAggregationMap.java 18 additions, 16 deletions...e/spark/sql/execution/UnsafeFixedWidthAggregationMap.java
- sql/core/src/main/java/org/apache/spark/sql/execution/UnsafeKVExternalSorter.java 2 additions, 2 deletions...rg/apache/spark/sql/execution/UnsafeKVExternalSorter.java
- sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegen.scala 1 addition, 5 deletions...la/org/apache/spark/sql/execution/WholeStageCodegen.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/TungstenAggregate.scala 8 additions, 2 deletions...che/spark/sql/execution/aggregate/TungstenAggregate.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala 8 additions, 9 deletions...org/apache/spark/sql/execution/joins/HashedRelation.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/BenchmarkWholeStageCodegen.scala 48 additions, 16 deletions...ache/spark/sql/execution/BenchmarkWholeStageCodegen.scala
Loading
Please register or sign in to comment