Skip to content
Snippets Groups Projects
Commit 59577df1 authored by Aaron Davidson's avatar Aaron Davidson
Browse files

SPARK-1770: Revert accidental(?) fix

Looks like this change was accidentally committed here: https://github.com/apache/spark/commit/06b15baab25951d124bbe6b64906f4139e037deb
but the change does not show up in the PR itself (#704).

Other than not intending to go in with that PR, this also broke the test JavaAPISuite.repartition.

Author: Aaron Davidson <aaron@databricks.com>

Closes #716 from aarondav/shufflerand and squashes the following commits:

b1cf70b [Aaron Davidson] SPARK-1770: Revert accidental(?) fix
parent bd67551e
No related branches found
No related tags found
No related merge requests found
......@@ -330,9 +330,9 @@ abstract class RDD[T: ClassTag](
if (shuffle) {
// include a shuffle step so that our upstream tasks are still distributed
new CoalescedRDD(
new ShuffledRDD[Int, T, (Int, T)](map(x => (Utils.random.nextInt(), x)),
new ShuffledRDD[T, Null, (T, Null)](map(x => (x, null)),
new HashPartitioner(numPartitions)),
numPartitions).values
numPartitions).keys
} else {
new CoalescedRDD(this, numPartitions)
}
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment