Skip to content
Snippets Groups Projects
  • Davies Liu's avatar
    28dcbb53
    [SPARK-2898] [PySpark] fix bugs in deamon.py · 28dcbb53
    Davies Liu authored
    1. do not use signal handler for SIGCHILD, it's easy to cause deadlock
    2. handle EINTR during accept()
    3. pass errno into JVM
    4. handle EAGAIN during fork()
    
    Now, it can pass 50k tasks tests in 180 seconds.
    
    Author: Davies Liu <davies.liu@gmail.com>
    
    Closes #1842 from davies/qa and squashes the following commits:
    
    f0ea451 [Davies Liu] fix lint
    03a2e8c [Davies Liu] cleanup dead children every seconds
    32cb829 [Davies Liu] fix lint
    0cd0817 [Davies Liu] fix bugs in deamon.py
    28dcbb53
    History
    [SPARK-2898] [PySpark] fix bugs in deamon.py
    Davies Liu authored
    1. do not use signal handler for SIGCHILD, it's easy to cause deadlock
    2. handle EINTR during accept()
    3. pass errno into JVM
    4. handle EAGAIN during fork()
    
    Now, it can pass 50k tasks tests in 180 seconds.
    
    Author: Davies Liu <davies.liu@gmail.com>
    
    Closes #1842 from davies/qa and squashes the following commits:
    
    f0ea451 [Davies Liu] fix lint
    03a2e8c [Davies Liu] cleanup dead children every seconds
    32cb829 [Davies Liu] fix lint
    0cd0817 [Davies Liu] fix bugs in deamon.py