Skip to content
Snippets Groups Projects
Commit b2a45606 authored by Liwei Lin's avatar Liwei Lin Committed by Davies Liu
Browse files

[SPARK-14911] [CORE] Fix a potential data race in TaskMemoryManager

## What changes were proposed in this pull request?

[[SPARK-13210][SQL] catch OOM when allocate memory and expand array](https://github.com/apache/spark/commit/37bc203c8dd5022cb11d53b697c28a737ee85bcc) introduced an `acquiredButNotUsed` field, but it might not be correctly synchronized:
- the write `acquiredButNotUsed += acquired` is guarded by `this` lock (see [here](https://github.com/apache/spark/blame/master/core/src/main/java/org/apache/spark/memory/TaskMemoryManager.java#L271));
- the read `memoryManager.releaseExecutionMemory(acquiredButNotUsed, taskAttemptId, tungstenMemoryMode)` (see [here](https://github.com/apache/spark/blame/master/core/src/main/java/org/apache/spark/memory/TaskMemoryManager.java#L400)) might not be correctly synchronized, and thus might not see `acquiredButNotUsed`'s most recent value.

This patch makes `acquiredButNotUsed` volatile to fix this.

## How was this patch tested?

This should be covered by existing suits.

Author: Liwei Lin <lwlin7@gmail.com>

Closes #12681 from lw-lin/fix-acquiredButNotUsed.
parent 8fda5a73
No related branches found
No related tags found
No related merge requests found
...@@ -114,7 +114,7 @@ public class TaskMemoryManager { ...@@ -114,7 +114,7 @@ public class TaskMemoryManager {
/** /**
* The amount of memory that is acquired but not used. * The amount of memory that is acquired but not used.
*/ */
private long acquiredButNotUsed = 0L; private volatile long acquiredButNotUsed = 0L;
/** /**
* Construct a new TaskMemoryManager. * Construct a new TaskMemoryManager.
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment