Skip to content
Snippets Groups Projects
Commit ce0051d6 authored by Calvin Jia's avatar Calvin Jia Committed by Reynold Xin
Browse files

[SPARK-6391][DOCS] Document Tachyon compatibility.

Adds a section in the RDD persistence section of the programming-guide docs detailing Spark-Tachyon version compatibility as discussed in [[SPARK-6391]](https://issues.apache.org/jira/browse/SPARK-6391).

Author: Calvin Jia <jia.calvin@gmail.com>

Closes #6382 from calvinjia/spark-6391 and squashes the following commits:

113e863 [Calvin Jia] Move compatibility info to the offheap storage level section.
7942dc5 [Calvin Jia] Add a section in the programming-guide docs for Tachyon compatibility.
parent 8af1bf10
No related branches found
No related tags found
No related merge requests found
......@@ -1214,9 +1214,11 @@ storage levels is:
Compared to MEMORY_ONLY_SER, OFF_HEAP reduces garbage collection overhead and allows executors
to be smaller and to share a pool of memory, making it attractive in environments with
large heaps or multiple concurrent applications. Furthermore, as the RDDs reside in Tachyon,
the crash of an executor does not lead to losing the in-memory cache. In this mode, the memory
the crash of an executor does not lead to losing the in-memory cache. In this mode, the memory
in Tachyon is discardable. Thus, Tachyon does not attempt to reconstruct a block that it evicts
from memory.
from memory. If you plan to use Tachyon as the off heap store, Spark is compatible with Tachyon
out-of-the-box. Please refer to this <a href="http://tachyon-project.org/master/Running-Spark-on-Tachyon.html">page</a>
for the suggested version pairings.
</td>
</tr>
</table>
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment