Skip to content
Snippets Groups Projects
Commit 54e61df2 authored by Sean Owen's avatar Sean Owen
Browse files

[SPARK-16599][CORE] java.util.NoSuchElementException: None.get at at...

[SPARK-16599][CORE] java.util.NoSuchElementException: None.get at at org.apache.spark.storage.BlockInfoManager.releaseAllLocksForTask

## What changes were proposed in this pull request?

Avoid None.get exception in (rare?) case that no readLocks exist
Note that while this would resolve the immediate cause of the exception, it's not clear it is the root problem.

## How was this patch tested?

Existing tests

Author: Sean Owen <sowen@cloudera.com>

Closes #17290 from srowen/SPARK-16599.
parent ccba622e
No related branches found
No related tags found
No related merge requests found
......@@ -23,7 +23,7 @@ import scala.collection.JavaConverters._
import scala.collection.mutable
import scala.reflect.ClassTag
import com.google.common.collect.ConcurrentHashMultiset
import com.google.common.collect.{ConcurrentHashMultiset, ImmutableMultiset}
import org.apache.spark.{SparkException, TaskContext}
import org.apache.spark.internal.Logging
......@@ -340,7 +340,7 @@ private[storage] class BlockInfoManager extends Logging {
val blocksWithReleasedLocks = mutable.ArrayBuffer[BlockId]()
val readLocks = synchronized {
readLocksByTask.remove(taskAttemptId).get
readLocksByTask.remove(taskAttemptId).getOrElse(ImmutableMultiset.of[BlockId]())
}
val writeLocks = synchronized {
writeLocksByTask.remove(taskAttemptId).getOrElse(Seq.empty)
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment