Skip to content
Snippets Groups Projects
Commit 48ee16d8 authored by Ernest's avatar Ernest Committed by Josh Rosen
Browse files

[SPARK-14055] writeLocksByTask need to be update when removeBlock

## What changes were proposed in this pull request?

https://issues.apache.org/jira/browse/SPARK-14055

## How was this patch tested?

manual tests by running LiveJournalPageRank on a large dataset ( the dataset must larger enough to incure RDD partition eviction).

Author: Ernest <earneyzxl@gmail.com>

Closes #11875 from Earne/issue-14055.
parent 3de24ae2
No related branches found
No related tags found
No related merge requests found
......@@ -421,6 +421,7 @@ private[storage] class BlockInfoManager extends Logging {
infos.remove(blockId)
blockInfo.readerCount = 0
blockInfo.writerTask = BlockInfo.NO_WRITER
writeLocksByTask.removeBinding(currentTaskAttemptId, blockId)
}
case None =>
throw new IllegalArgumentException(
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment