-
- Downloads
[SPARK-3824][SQL] Sets in-memory table default storage level to MEMORY_AND_DISK
Using `MEMORY_AND_DISK` as default storage level for in-memory table caching. Due to the in-memory columnar representation, recomputing an in-memory cached table partitions can be very expensive. Author: Cheng Lian <lian.cs.zju@gmail.com> Closes #2686 from liancheng/spark-3824 and squashes the following commits: 35d2ed0 [Cheng Lian] Removes extra space 1ab7967 [Cheng Lian] Reduces test data size to fit DiskStore.getBytes() ba565f0 [Cheng Lian] Maks CachedBatch serializable 07f0204 [Cheng Lian] Sets in-memory table default storage level to MEMORY_AND_DISK
Showing
- sql/core/src/main/scala/org/apache/spark/sql/CacheManager.scala 7 additions, 3 deletions...re/src/main/scala/org/apache/spark/sql/CacheManager.scala
- sql/core/src/main/scala/org/apache/spark/sql/columnar/InMemoryColumnarTableScan.scala 5 additions, 4 deletions...apache/spark/sql/columnar/InMemoryColumnarTableScan.scala
- sql/core/src/test/scala/org/apache/spark/sql/CachedTableSuite.scala 5 additions, 5 deletions...rc/test/scala/org/apache/spark/sql/CachedTableSuite.scala
Loading
Please register or sign in to comment