Skip to content
Snippets Groups Projects
Commit 330bf5c9 authored by caoxuewen's avatar caoxuewen Committed by Sean Owen
Browse files

[SPARK-20609][MLLIB][TEST] manually cleared 'spark.local.dir' before/after a...

[SPARK-20609][MLLIB][TEST] manually cleared 'spark.local.dir' before/after a test in ALSCleanerSuite

## What changes were proposed in this pull request?

This PR is similar to #17869.
Once` 'spark.local.dir'` is set. Unless this is manually cleared before/after a test. it could return the same directory even if this property is configured.
and add before/after for each likewise in ALSCleanerSuite.

## How was this patch tested?
existing test.

Author: caoxuewen <cao.xuewen@zte.com.cn>

Closes #18537 from heary-cao/ALSCleanerSuite.
parent 01f183e8
No related branches found
No related tags found
No related merge requests found
...@@ -29,6 +29,7 @@ import scala.language.existentials ...@@ -29,6 +29,7 @@ import scala.language.existentials
import com.github.fommil.netlib.BLAS.{getInstance => blas} import com.github.fommil.netlib.BLAS.{getInstance => blas}
import org.apache.commons.io.FileUtils import org.apache.commons.io.FileUtils
import org.apache.commons.io.filefilter.TrueFileFilter import org.apache.commons.io.filefilter.TrueFileFilter
import org.scalatest.BeforeAndAfterEach
import org.apache.spark._ import org.apache.spark._
import org.apache.spark.internal.Logging import org.apache.spark.internal.Logging
...@@ -777,7 +778,20 @@ class ALSSuite ...@@ -777,7 +778,20 @@ class ALSSuite
} }
} }
class ALSCleanerSuite extends SparkFunSuite { class ALSCleanerSuite extends SparkFunSuite with BeforeAndAfterEach {
override def beforeEach(): Unit = {
super.beforeEach()
// Once `Utils.getOrCreateLocalRootDirs` is called, it is cached in `Utils.localRootDirs`.
// Unless this is manually cleared before and after a test, it returns the same directory
// set before even if 'spark.local.dir' is configured afterwards.
Utils.clearLocalRootDirs()
}
override def afterEach(): Unit = {
Utils.clearLocalRootDirs()
super.afterEach()
}
test("ALS shuffle cleanup standalone") { test("ALS shuffle cleanup standalone") {
val conf = new SparkConf() val conf = new SparkConf()
val localDir = Utils.createTempDir() val localDir = Utils.createTempDir()
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment