Skip to content
Snippets Groups Projects
Commit bf40cf89 authored by Cheng Hao's avatar Cheng Hao Committed by Michael Armbrust
Browse files

[SPARK-4713] [SQL] SchemaRDD.unpersist() should not raise exception if it is not persisted

Unpersist a uncached RDD, will not raise exception, for example:
```
val data = Array(1, 2, 3, 4, 5)
val distData = sc.parallelize(data)
distData.unpersist(true)
```

But the `SchemaRDD` will raise exception if the `SchemaRDD` is not cached. Since `SchemaRDD` is the subclasses of the `RDD`, we should follow the same behavior.

Author: Cheng Hao <hao.cheng@intel.com>

Closes #3572 from chenghao-intel/try_uncache and squashes the following commits:

50a7a89 [Cheng Hao] SchemaRDD.unpersist() should not raise exception if it is not persisted
parent b004150a
No related branches found
No related tags found
No related merge requests found
......@@ -501,7 +501,7 @@ class SchemaRDD(
}
override def unpersist(blocking: Boolean): this.type = {
sqlContext.uncacheQuery(this, blocking)
sqlContext.tryUncacheQuery(this, blocking)
this
}
}
......@@ -49,6 +49,20 @@ class CachedTableSuite extends QueryTest {
uncacheTable("tempTable")
}
test("unpersist an uncached table will not raise exception") {
assert(None == lookupCachedData(testData))
testData.unpersist(true)
assert(None == lookupCachedData(testData))
testData.unpersist(false)
assert(None == lookupCachedData(testData))
testData.persist()
assert(None != lookupCachedData(testData))
testData.unpersist(true)
assert(None == lookupCachedData(testData))
testData.unpersist(false)
assert(None == lookupCachedData(testData))
}
test("cache table as select") {
sql("CACHE TABLE tempTable AS SELECT key FROM testData")
assertCached(sql("SELECT COUNT(*) FROM tempTable"))
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment