Skip to content
  • Jacek Lewandowski's avatar
    cfea3003
    Spark 3883: SSL support for HttpServer and Akka · cfea3003
    Jacek Lewandowski authored
    SPARK-3883: SSL support for Akka connections and Jetty based file servers.
    
    This story introduced the following changes:
    - Introduced SSLOptions object which holds the SSL configuration and can build the appropriate configuration for Akka or Jetty. SSLOptions can be created by parsing SparkConf entries at a specified namespace.
    - SSLOptions is created and kept by SecurityManager
    - All Akka actor address creation snippets based on interpolated strings were replaced by a dedicated methods from AkkaUtils. Those methods select the proper Akka protocol - whether akka.tcp or akka.ssl.tcp
    - Added tests cases for AkkaUtils, FileServer, SSLOptions and SecurityManager
    - Added a way to use node local SSL configuration by executors and driver in standalone mode. It can be done by specifying spark.ssl.useNodeLocalConf in SparkConf.
    - Made CoarseGrainedExecutorBackend not overwrite the settings which are executor startup configuration - they are passed anyway from Worker
    
    Refer to https://github.com/apache/spark/pull/3571 for discussion and details
    
    Author: Jacek Lewandowski <lewandowski.jacek@gmail.com>
    Author: Jacek Lewandowski <jacek.lewandowski@datastax.com>
    
    Closes #3571 from jacek-lewandowski/SPARK-3883-master and squashes the following commits:
    
    9ef4ed1 [Jacek Lewandowski] Merge pull request #2 from jacek-lewandowski/SPARK-3883-docs2
    fb31b49 [Jacek Lewandowski] SPARK-3883: Added SSL setup documentation
    2532668 [Jacek Lewandowski] SPARK-3883: Refactored AkkaUtils.protocol method to not use Try
    90a8762 [Jacek Lewandowski] SPARK-3883: Refactored methods to resolve Akka address and made it possible to easily configure multiple communication layers for SSL
    72b2541 [Jacek Lewandowski] SPARK-3883: A reference to the fallback SSLOptions can be provided when constructing SSLOptions
    93050f4 [Jacek Lewandowski] SPARK-3883: SSL support for HttpServer and Akka
    cfea3003
    Spark 3883: SSL support for HttpServer and Akka
    Jacek Lewandowski authored
    SPARK-3883: SSL support for Akka connections and Jetty based file servers.
    
    This story introduced the following changes:
    - Introduced SSLOptions object which holds the SSL configuration and can build the appropriate configuration for Akka or Jetty. SSLOptions can be created by parsing SparkConf entries at a specified namespace.
    - SSLOptions is created and kept by SecurityManager
    - All Akka actor address creation snippets based on interpolated strings were replaced by a dedicated methods from AkkaUtils. Those methods select the proper Akka protocol - whether akka.tcp or akka.ssl.tcp
    - Added tests cases for AkkaUtils, FileServer, SSLOptions and SecurityManager
    - Added a way to use node local SSL configuration by executors and driver in standalone mode. It can be done by specifying spark.ssl.useNodeLocalConf in SparkConf.
    - Made CoarseGrainedExecutorBackend not overwrite the settings which are executor startup configuration - they are passed anyway from Worker
    
    Refer to https://github.com/apache/spark/pull/3571 for discussion and details
    
    Author: Jacek Lewandowski <lewandowski.jacek@gmail.com>
    Author: Jacek Lewandowski <jacek.lewandowski@datastax.com>
    
    Closes #3571 from jacek-lewandowski/SPARK-3883-master and squashes the following commits:
    
    9ef4ed1 [Jacek Lewandowski] Merge pull request #2 from jacek-lewandowski/SPARK-3883-docs2
    fb31b49 [Jacek Lewandowski] SPARK-3883: Added SSL setup documentation
    2532668 [Jacek Lewandowski] SPARK-3883: Refactored AkkaUtils.protocol method to not use Try
    90a8762 [Jacek Lewandowski] SPARK-3883: Refactored methods to resolve Akka address and made it possible to easily configure multiple communication layers for SSL
    72b2541 [Jacek Lewandowski] SPARK-3883: A reference to the fallback SSLOptions can be provided when constructing SSLOptions
    93050f4 [Jacek Lewandowski] SPARK-3883: SSL support for HttpServer and Akka
Loading