Skip to content
Snippets Groups Projects
Commit ae9e4247 authored by gatorsmile's avatar gatorsmile
Browse files

[SQL][MINOR][TEST] Set spark.unsafe.exceptionOnMemoryLeak to true

## What changes were proposed in this pull request?
When running IntelliJ, we are unable to capture the exception of memory leak detection.
> org.apache.spark.executor.Executor: Managed memory leak detected

Explicitly setting `spark.unsafe.exceptionOnMemoryLeak` in SparkConf when building the SparkSession, instead of reading it from system properties.

## How was this patch tested?
N/A

Author: gatorsmile <gatorsmile@gmail.com>

Closes #18967 from gatorsmile/setExceptionOnMemoryLeak.
parent b83b502c
No related branches found
No related tags found
No related merge requests found
......@@ -31,7 +31,9 @@ import org.apache.spark.sql.{SparkSession, SQLContext}
trait SharedSQLContext extends SQLTestUtils with BeforeAndAfterEach with Eventually {
protected def sparkConf = {
new SparkConf().set("spark.hadoop.fs.file.impl", classOf[DebugFilesystem].getName)
new SparkConf()
.set("spark.hadoop.fs.file.impl", classOf[DebugFilesystem].getName)
.set("spark.unsafe.exceptionOnMemoryLeak", "true")
}
/**
......
......@@ -55,7 +55,8 @@ object TestHive
"org.apache.spark.sql.hive.execution.PairSerDe")
.set("spark.sql.warehouse.dir", TestHiveContext.makeWarehouseDir().toURI.getPath)
// SPARK-8910
.set("spark.ui.enabled", "false")))
.set("spark.ui.enabled", "false")
.set("spark.unsafe.exceptionOnMemoryLeak", "true")))
case class TestHiveVersion(hiveClient: HiveClient)
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment