Skip to content
  • Scott Taylor's avatar
    6e1c7e27
    [SPARK-7735] [PYSPARK] Raise Exception on non-zero exit from pipe commands · 6e1c7e27
    Scott Taylor authored
    This will allow problems with piped commands to be detected.
    This will also allow tasks to be retried where errors are rare (such as network problems in piped commands).
    
    Author: Scott Taylor <github@megatron.me.uk>
    
    Closes #6262 from megatron-me-uk/patch-2 and squashes the following commits:
    
    04ae1d5 [Scott Taylor] Remove spurious empty line
    98fa101 [Scott Taylor] fix blank line style error
    574b564 [Scott Taylor] Merge pull request #2 from megatron-me-uk/patch-4
    0c1e762 [Scott Taylor] Update rdd pipe method for checkCode
    ab9a2e1 [Scott Taylor] Update rdd pipe tests for checkCode
    eb4801c [Scott Taylor] fix fail_condition
    b0ac3a4 [Scott Taylor] Merge pull request #1 from megatron-me-uk/megatron-me-uk-patch-1
    a307d13 [Scott Taylor] update rdd tests to test pipe modes
    34fcdc3 [Scott Taylor] add optional argument 'mode' for rdd.pipe
    a0c0161 [Scott Taylor] fix generator issue
    8a9ef9c [Scott Taylor] make check_return_code an iterator
    0486ae3 [Scott Taylor] style fixes
    8ed89a6 [Scott Taylor] Chain generators to prevent potential deadlock
    4153b02 [Scott Taylor] fix list.sort returns None
    491d3fc [Scott Taylor] Pass a function handle to assertRaises
    3344a21 [Scott Taylor] wrap assertRaises with QuietTest
    3ab8c7a [Scott Taylor] remove whitespace for style
    cc1a73d [Scott Taylor] fix style issues in pipe test
    8db4073 [Scott Taylor] Add a test for rdd pipe functions
    1b3dc4e [Scott Taylor] fix missing space around operator style
    0974f98 [Scott Taylor] add space between words in multiline string
    45f4977 [Scott Taylor] fix line too long style error
    5745d85 [Scott Taylor] Remove space to fix style
    f552d49 [Scott Taylor] Catch non-zero exit from pipe commands
    6e1c7e27
    [SPARK-7735] [PYSPARK] Raise Exception on non-zero exit from pipe commands
    Scott Taylor authored
    This will allow problems with piped commands to be detected.
    This will also allow tasks to be retried where errors are rare (such as network problems in piped commands).
    
    Author: Scott Taylor <github@megatron.me.uk>
    
    Closes #6262 from megatron-me-uk/patch-2 and squashes the following commits:
    
    04ae1d5 [Scott Taylor] Remove spurious empty line
    98fa101 [Scott Taylor] fix blank line style error
    574b564 [Scott Taylor] Merge pull request #2 from megatron-me-uk/patch-4
    0c1e762 [Scott Taylor] Update rdd pipe method for checkCode
    ab9a2e1 [Scott Taylor] Update rdd pipe tests for checkCode
    eb4801c [Scott Taylor] fix fail_condition
    b0ac3a4 [Scott Taylor] Merge pull request #1 from megatron-me-uk/megatron-me-uk-patch-1
    a307d13 [Scott Taylor] update rdd tests to test pipe modes
    34fcdc3 [Scott Taylor] add optional argument 'mode' for rdd.pipe
    a0c0161 [Scott Taylor] fix generator issue
    8a9ef9c [Scott Taylor] make check_return_code an iterator
    0486ae3 [Scott Taylor] style fixes
    8ed89a6 [Scott Taylor] Chain generators to prevent potential deadlock
    4153b02 [Scott Taylor] fix list.sort returns None
    491d3fc [Scott Taylor] Pass a function handle to assertRaises
    3344a21 [Scott Taylor] wrap assertRaises with QuietTest
    3ab8c7a [Scott Taylor] remove whitespace for style
    cc1a73d [Scott Taylor] fix style issues in pipe test
    8db4073 [Scott Taylor] Add a test for rdd pipe functions
    1b3dc4e [Scott Taylor] fix missing space around operator style
    0974f98 [Scott Taylor] add space between words in multiline string
    45f4977 [Scott Taylor] fix line too long style error
    5745d85 [Scott Taylor] Remove space to fix style
    f552d49 [Scott Taylor] Catch non-zero exit from pipe commands
Loading