Skip to content
Snippets Groups Projects
Commit c5a66356 authored by Hervé's avatar Hervé Committed by Reynold Xin
Browse files

Encryption of shuffle files

Hello

According to my understanding of commits 4b4e329e & 8b325b17, one may now encrypt shuffle files regardless of the cluster manager in use.

However I have limited understanding of the code, I'm not able to find out whether theses changes also comprise all "temporary local storage, such as shuffle files, cached data, and other application files".

Please feel free to amend or reject my PR if I'm wrong.

dud

Author: Hervé <dud225@users.noreply.github.com>

Closes #16885 from dud225/patch-1.
parent 8640dc08
No related branches found
No related tags found
No related merge requests found
......@@ -28,11 +28,7 @@ If your applications are using event logging, the directory where the event logs
## Encryption
Spark supports SSL for HTTP protocols. SASL encryption is supported for the block transfer service
and the RPC endpoints.
Encryption is not yet supported for data stored by Spark in temporary local storage, such as shuffle
files, cached data, and other application files. If encrypting this data is desired, a workaround is
to configure your cluster manager to store application data on encrypted disks.
and the RPC endpoints. Shuffle files can also be encrypted if desired.
### SSL Configuration
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment