Skip to content
Snippets Groups Projects
  • Shixiong Zhu's avatar
    be7a2cfd
    [SPARK-11870][STREAMING][PYSPARK] Rethrow the exceptions in TransformFunction... · be7a2cfd
    Shixiong Zhu authored
    [SPARK-11870][STREAMING][PYSPARK] Rethrow the exceptions in TransformFunction and TransformFunctionSerializer
    
    TransformFunction and TransformFunctionSerializer don't rethrow the exception, so when any exception happens, it just return None. This will cause some weird NPE and confuse people.
    
    Author: Shixiong Zhu <shixiong@databricks.com>
    
    Closes #9847 from zsxwing/pyspark-streaming-exception.
    be7a2cfd
    History
    [SPARK-11870][STREAMING][PYSPARK] Rethrow the exceptions in TransformFunction...
    Shixiong Zhu authored
    [SPARK-11870][STREAMING][PYSPARK] Rethrow the exceptions in TransformFunction and TransformFunctionSerializer
    
    TransformFunction and TransformFunctionSerializer don't rethrow the exception, so when any exception happens, it just return None. This will cause some weird NPE and confuse people.
    
    Author: Shixiong Zhu <shixiong@databricks.com>
    
    Closes #9847 from zsxwing/pyspark-streaming-exception.