Skip to content
Snippets Groups Projects
  • Davies Liu's avatar
    712679a7
    [SPARK-6294] fix hang when call take() in JVM on PythonRDD · 712679a7
    Davies Liu authored
    The Thread.interrupt() can not terminate the thread in some cases, so we should not wait for the writerThread of PythonRDD.
    
    This PR also ignore some exception during clean up.
    
    cc JoshRosen mengxr
    
    Author: Davies Liu <davies@databricks.com>
    
    Closes #4987 from davies/fix_take and squashes the following commits:
    
    4488f1a [Davies Liu] fix hang when call take() in JVM on PythonRDD
    712679a7
    History
    [SPARK-6294] fix hang when call take() in JVM on PythonRDD
    Davies Liu authored
    The Thread.interrupt() can not terminate the thread in some cases, so we should not wait for the writerThread of PythonRDD.
    
    This PR also ignore some exception during clean up.
    
    cc JoshRosen mengxr
    
    Author: Davies Liu <davies@databricks.com>
    
    Closes #4987 from davies/fix_take and squashes the following commits:
    
    4488f1a [Davies Liu] fix hang when call take() in JVM on PythonRDD