-
- Downloads
[SPARK-15285][SQL] Generated SpecificSafeProjection.apply method grows beyond 64 KB
## What changes were proposed in this pull request? This PR splits the generated code for ```SafeProjection.apply``` by using ```ctx.splitExpressions()```. This is because the large code body for ```NewInstance``` may grow beyond 64KB bytecode size for ```apply()``` method. Here is [the original PR](https://github.com/apache/spark/pull/13243) for SPARK-15285. However, it breaks a build with Scala 2.10 since Scala 2.10 does not a case class with large number of members. Thus, it was reverted by [this commit](https://github.com/apache/spark/commit/fa244e5a90690d6a31be50f2aa203ae1a2e9a1cf). ## How was this patch tested? Added new tests by using `DefinedByConstructorParams` instead of case class for scala-2.10 Author: Kazuaki Ishizaki <ishizaki@jp.ibm.com> Closes #14670 from kiszk/SPARK-15285-2.
Showing
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala 26 additions, 6 deletions...ache/spark/sql/catalyst/expressions/objects/objects.scala
- sql/core/src/test/scala/org/apache/spark/sql/DataFrameComplexTypeSuite.scala 40 additions, 0 deletions...cala/org/apache/spark/sql/DataFrameComplexTypeSuite.scala
Loading
Please register or sign in to comment