-
- Downloads
[SPARK-17702][SQL] Code generation including too many mutable states exceeds JVM size limit.
## What changes were proposed in this pull request? Code generation including too many mutable states exceeds JVM size limit to extract values from `references` into fields in the constructor. We should split the generated extractions in the constructor into smaller functions. ## How was this patch tested? I added some tests to check if the generated codes for the expressions exceed or not. Author: Takuya UESHIN <ueshin@happy-camper.st> Closes #15275 from ueshin/issues/SPARK-17702.
Showing
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala 13 additions, 5 deletions...park/sql/catalyst/expressions/codegen/CodeGenerator.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateMutableProjection.scala 2 additions, 1 deletion...alyst/expressions/codegen/GenerateMutableProjection.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateOrdering.scala 2 additions, 1 deletion...k/sql/catalyst/expressions/codegen/GenerateOrdering.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GeneratePredicate.scala 3 additions, 1 deletion.../sql/catalyst/expressions/codegen/GeneratePredicate.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateSafeProjection.scala 3 additions, 1 deletion...catalyst/expressions/codegen/GenerateSafeProjection.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateUnsafeProjection.scala 2 additions, 1 deletion...talyst/expressions/codegen/GenerateUnsafeProjection.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CodeGenerationSuite.scala 20 additions, 1 deletion.../spark/sql/catalyst/expressions/CodeGenerationSuite.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala 3 additions, 1 deletion...rg/apache/spark/sql/execution/WholeStageCodegenExec.scala
Loading
Please register or sign in to comment