Skip to content
Snippets Groups Projects
Commit f4346f61 authored by jhu-chang's avatar jhu-chang Committed by Shixiong Zhu
Browse files

[SPARK-11749][STREAMING] Duplicate creating the RDD in file stream when... 

[SPARK-11749][STREAMING] Duplicate creating the RDD in file stream when recovering from checkpoint data

Add a transient flag `DStream.restoredFromCheckpointData` to control the restore processing in DStream to avoid duplicate works:  check this flag first in `DStream.restoreCheckpointData`, only when `false`, the restore process will be executed.

Author: jhu-chang <gt.hu.chang@gmail.com>

Closes #9765 from jhu-chang/SPARK-11749.
parent 658f66e6
No related branches found
No related tags found
Hide whitespace changes
Inline Side-by-side
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment