Skip to content
Snippets Groups Projects
  • Davies Liu's avatar
    32fbd297
    [SPARK-6216] [PYSPARK] check python version of worker with driver · 32fbd297
    Davies Liu authored
    This PR revert #5404, change to pass the version of python in driver into JVM, check it in worker before deserializing closure, then it can works with different major version of Python.
    
    Author: Davies Liu <davies@databricks.com>
    
    Closes #6203 from davies/py_version and squashes the following commits:
    
    b8fb76e [Davies Liu] fix test
    6ce5096 [Davies Liu] use string for version
    47c6278 [Davies Liu] check python version of worker with driver
    32fbd297
    History
    [SPARK-6216] [PYSPARK] check python version of worker with driver
    Davies Liu authored
    This PR revert #5404, change to pass the version of python in driver into JVM, check it in worker before deserializing closure, then it can works with different major version of Python.
    
    Author: Davies Liu <davies@databricks.com>
    
    Closes #6203 from davies/py_version and squashes the following commits:
    
    b8fb76e [Davies Liu] fix test
    6ce5096 [Davies Liu] use string for version
    47c6278 [Davies Liu] check python version of worker with driver
functions.py 14.84 KiB