Skip to content
Snippets Groups Projects
Commit 5743352a authored by Xin Ren's avatar Xin Ren Committed by Josh Rosen
Browse files

[SPARK-14935][CORE] DistributedSuite "local-cluster format" shouldn't actually launch clusters

https://issues.apache.org/jira/browse/SPARK-14935

In DistributedSuite, the "local-cluster format" test actually launches a bunch of clusters, but this doesn't seem necessary for what should just be a unit test of a regex. We should clean up the code so that this is testable without actually launching a cluster, which should buy us about 20 seconds per build.

Passed unit test on my local machine

Author: Xin Ren <iamshrek@126.com>

Closes #12744 from keypointt/SPARK-14935.
parent bed0b002
No related branches found
No related tags found
No related merge requests found
......@@ -51,18 +51,21 @@ class DistributedSuite extends SparkFunSuite with Matchers with LocalSparkContex
}
test("local-cluster format") {
sc = new SparkContext("local-cluster[2,1,1024]", "test")
assert(sc.parallelize(1 to 2, 2).count() == 2)
resetSparkContext()
sc = new SparkContext("local-cluster[2 , 1 , 1024]", "test")
assert(sc.parallelize(1 to 2, 2).count() == 2)
resetSparkContext()
sc = new SparkContext("local-cluster[2, 1, 1024]", "test")
assert(sc.parallelize(1 to 2, 2).count() == 2)
resetSparkContext()
sc = new SparkContext("local-cluster[ 2, 1, 1024 ]", "test")
assert(sc.parallelize(1 to 2, 2).count() == 2)
resetSparkContext()
import SparkMasterRegex._
val masterStrings = Seq(
"local-cluster[2,1,1024]",
"local-cluster[2 , 1 , 1024]",
"local-cluster[2, 1, 1024]",
"local-cluster[ 2, 1, 1024 ]"
)
masterStrings.foreach {
case LOCAL_CLUSTER_REGEX(numSlaves, coresPerSlave, memoryPerSlave) =>
assert(numSlaves.toInt == 2)
assert(coresPerSlave.toInt == 1)
assert(memoryPerSlave.toInt == 1024)
}
}
test("simple groupByKey") {
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment