Skip to content
Snippets Groups Projects
Commit be99a99f authored by Dongjoon Hyun's avatar Dongjoon Hyun Committed by Andrew Or
Browse files

[MINOR][CORE][TEST] Update obsolete `takeSample` test case.

## What changes were proposed in this pull request?

This PR fixes some obsolete comments and assertion in `takeSample` testcase of `RDDSuite.scala`.

## How was this patch tested?

This fixes the testcase only.

Author: Dongjoon Hyun <dongjoon@apache.org>

Closes #13260 from dongjoon-hyun/SPARK-15481.
parent 784cc07d
No related branches found
No related tags found
No related merge requests found
......@@ -678,27 +678,26 @@ class RDDSuite extends SparkFunSuite with SharedSparkContext {
}
{
val sample = data.takeSample(withReplacement = true, num = 20)
assert(sample.size === 20) // Got exactly 100 elements
assert(sample.toSet.size <= 20, "sampling with replacement returned all distinct elements")
assert(sample.size === 20) // Got exactly 20 elements
assert(sample.forall(x => 1 <= x && x <= n), s"elements not in [1, $n]")
}
{
val sample = data.takeSample(withReplacement = true, num = n)
assert(sample.size === n) // Got exactly 100 elements
// Chance of getting all distinct elements is astronomically low, so test we got < 100
assert(sample.size === n) // Got exactly n elements
// Chance of getting all distinct elements is astronomically low, so test we got < n
assert(sample.toSet.size < n, "sampling with replacement returned all distinct elements")
assert(sample.forall(x => 1 <= x && x <= n), s"elements not in [1, $n]")
}
for (seed <- 1 to 5) {
val sample = data.takeSample(withReplacement = true, n, seed)
assert(sample.size === n) // Got exactly 100 elements
// Chance of getting all distinct elements is astronomically low, so test we got < 100
assert(sample.size === n) // Got exactly n elements
// Chance of getting all distinct elements is astronomically low, so test we got < n
assert(sample.toSet.size < n, "sampling with replacement returned all distinct elements")
}
for (seed <- 1 to 5) {
val sample = data.takeSample(withReplacement = true, 2 * n, seed)
assert(sample.size === 2 * n) // Got exactly 200 elements
// Chance of getting all distinct elements is still quite low, so test we got < 100
assert(sample.size === 2 * n) // Got exactly 2 * n elements
// Chance of getting all distinct elements is still quite low, so test we got < n
assert(sample.toSet.size < n, "sampling with replacement returned all distinct elements")
}
}
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment