-
- Downloads
[SPARK-19525][CORE] Add RDD checkpoint compression support
## What changes were proposed in this pull request? This PR adds RDD checkpoint compression support and add a new config `spark.checkpoint.compress` to enable/disable it. Credit goes to aramesh117 Closes #17024 ## How was this patch tested? The new unit test. Author: Shixiong Zhu <shixiong@databricks.com> Author: Aaditya Ramesh <aramesh@conviva.com> Closes #17789 from zsxwing/pr17024.
Showing
- core/src/main/scala/org/apache/spark/internal/config/package.scala 6 additions, 0 deletions...main/scala/org/apache/spark/internal/config/package.scala
- core/src/main/scala/org/apache/spark/rdd/ReliableCheckpointRDD.scala 22 additions, 2 deletions...in/scala/org/apache/spark/rdd/ReliableCheckpointRDD.scala
- core/src/test/scala/org/apache/spark/CheckpointSuite.scala 41 additions, 0 deletionscore/src/test/scala/org/apache/spark/CheckpointSuite.scala
Loading
Please register or sign in to comment