Skip to content
  • Davies Liu's avatar
    c8709dcf
    [SPARK-7956] [SQL] Use Janino to compile SQL expressions into bytecode · c8709dcf
    Davies Liu authored
    In order to reduce the overhead of codegen, this PR switch to use Janino to compile SQL expressions into bytecode.
    
    After this, the time used to compile a SQL expression is decreased from 100ms to 5ms, which is necessary to turn on codegen for general workload, also tests.
    
    cc rxin
    
    Author: Davies Liu <davies@databricks.com>
    
    Closes #6479 from davies/janino and squashes the following commits:
    
    cc689f5 [Davies Liu] remove globalLock
    262d848 [Davies Liu] Merge branch 'master' of github.com:apache/spark into janino
    eec3a33 [Davies Liu] address comments from Josh
    f37c8c3 [Davies Liu] fix DecimalType and cast to String
    202298b [Davies Liu] Merge branch 'master' of github.com:apache/spark into janino
    a21e968 [Davies Liu] fix style
    0ed3dc6 [Davies Liu] Merge branch 'master' of github.com:apache/spark into janino
    551a851 [Davies Liu] fix tests
    c3bdffa [Davies Liu] remove print
    6089ce5 [Davies Liu] change logging level
    7e46ac3 [Davies Liu] fix style
    d8f0f6c [Davies Liu] Merge branch 'master' of github.com:apache/spark into janino
    da4926a [Davies Liu] fix tests
    03660f3 [Davies Liu] WIP: use Janino to compile Java source
    f2629cd [Davies Liu] Merge branch 'master' of github.com:apache/spark into janino
    f7d66cf [Davies Liu] use template based string for codegen
    c8709dcf
    [SPARK-7956] [SQL] Use Janino to compile SQL expressions into bytecode
    Davies Liu authored
    In order to reduce the overhead of codegen, this PR switch to use Janino to compile SQL expressions into bytecode.
    
    After this, the time used to compile a SQL expression is decreased from 100ms to 5ms, which is necessary to turn on codegen for general workload, also tests.
    
    cc rxin
    
    Author: Davies Liu <davies@databricks.com>
    
    Closes #6479 from davies/janino and squashes the following commits:
    
    cc689f5 [Davies Liu] remove globalLock
    262d848 [Davies Liu] Merge branch 'master' of github.com:apache/spark into janino
    eec3a33 [Davies Liu] address comments from Josh
    f37c8c3 [Davies Liu] fix DecimalType and cast to String
    202298b [Davies Liu] Merge branch 'master' of github.com:apache/spark into janino
    a21e968 [Davies Liu] fix style
    0ed3dc6 [Davies Liu] Merge branch 'master' of github.com:apache/spark into janino
    551a851 [Davies Liu] fix tests
    c3bdffa [Davies Liu] remove print
    6089ce5 [Davies Liu] change logging level
    7e46ac3 [Davies Liu] fix style
    d8f0f6c [Davies Liu] Merge branch 'master' of github.com:apache/spark into janino
    da4926a [Davies Liu] fix tests
    03660f3 [Davies Liu] WIP: use Janino to compile Java source
    f2629cd [Davies Liu] Merge branch 'master' of github.com:apache/spark into janino
    f7d66cf [Davies Liu] use template based string for codegen
Loading