-
- Downloads
[SPARK-20886][CORE] HadoopMapReduceCommitProtocol to handle FileOutputCommitter.getWorkPath==null
## What changes were proposed in this pull request? Handles the situation where a `FileOutputCommitter.getWorkPath()` returns `null` by downgrading to the supplied `path` argument. The existing code does an `Option(workPath.toString).getOrElse(path)`, which triggers an NPE in the `toString()` operation if the workPath == null. The code apparently was meant to handle this (hence the getOrElse() clause, but as the NPE has already occurred at that point the else-clause never gets invoked. ## How was this patch tested? Manually, with some later code review. Author: Steve Loughran <stevel@hortonworks.com> Closes #18111 from steveloughran/cloud/SPARK-20886-committer-NPE.
Please register or sign in to comment