Skip to content
Snippets Groups Projects
Commit 24db3582 authored by Shixiong Zhu's avatar Shixiong Zhu
Browse files

[SPARK-20940][CORE] Replace IllegalAccessError with IllegalStateException

## What changes were proposed in this pull request?

`IllegalAccessError` is a fatal error (a subclass of LinkageError) and its meaning is `Thrown if an application attempts to access or modify a field, or to call a method that it does not have access to`. Throwing a fatal error for AccumulatorV2 is not necessary and is pretty bad because it usually will just kill executors or SparkContext ([SPARK-20666](https://issues.apache.org/jira/browse/SPARK-20666) is an example of killing SparkContext due to `IllegalAccessError`). I think the correct type of exception in AccumulatorV2 should be `IllegalStateException`.

## How was this patch tested?

Jenkins

Author: Shixiong Zhu <shixiong@databricks.com>

Closes #18168 from zsxwing/SPARK-20940.
parent 2bc32728
No related branches found
No related tags found
No related merge requests found
...@@ -68,7 +68,7 @@ abstract class AccumulatorV2[IN, OUT] extends Serializable { ...@@ -68,7 +68,7 @@ abstract class AccumulatorV2[IN, OUT] extends Serializable {
private def assertMetadataNotNull(): Unit = { private def assertMetadataNotNull(): Unit = {
if (metadata == null) { if (metadata == null) {
throw new IllegalAccessError("The metadata of this accumulator has not been assigned yet.") throw new IllegalStateException("The metadata of this accumulator has not been assigned yet.")
} }
} }
...@@ -265,7 +265,7 @@ private[spark] object AccumulatorContext { ...@@ -265,7 +265,7 @@ private[spark] object AccumulatorContext {
// Since we are storing weak references, we must check whether the underlying data is valid. // Since we are storing weak references, we must check whether the underlying data is valid.
val acc = ref.get val acc = ref.get
if (acc eq null) { if (acc eq null) {
throw new IllegalAccessError(s"Attempted to access garbage collected accumulator $id") throw new IllegalStateException(s"Attempted to access garbage collected accumulator $id")
} }
acc acc
} }
......
...@@ -210,7 +210,7 @@ class AccumulatorSuite extends SparkFunSuite with Matchers with LocalSparkContex ...@@ -210,7 +210,7 @@ class AccumulatorSuite extends SparkFunSuite with Matchers with LocalSparkContex
assert(ref.get.isEmpty) assert(ref.get.isEmpty)
// Getting a garbage collected accum should throw error // Getting a garbage collected accum should throw error
intercept[IllegalAccessError] { intercept[IllegalStateException] {
AccumulatorContext.get(accId) AccumulatorContext.get(accId)
} }
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment