Skip to content
  • Michael Gummelt's avatar
    266b92fa
    [SPARK-16637] Unified containerizer · 266b92fa
    Michael Gummelt authored
    ## What changes were proposed in this pull request?
    
    New config var: spark.mesos.docker.containerizer={"mesos","docker" (default)}
    
    This adds support for running docker containers via the Mesos unified containerizer: http://mesos.apache.org/documentation/latest/container-image/
    
    The benefit is losing the dependency on `dockerd`, and all the costs which it incurs.
    
    I've also updated the supported Mesos version to 0.28.2 for support of the required protobufs.
    
    This is blocked on: https://github.com/apache/spark/pull/14167
    
    ## How was this patch tested?
    
    - manually testing jobs submitted with both "mesos" and "docker" settings for the new config var.
    - spark/mesos integration test suite
    
    Author: Michael Gummelt <mgummelt@mesosphere.io>
    
    Closes #14275 from mgummelt/unified-containerizer.
    266b92fa
    [SPARK-16637] Unified containerizer
    Michael Gummelt authored
    ## What changes were proposed in this pull request?
    
    New config var: spark.mesos.docker.containerizer={"mesos","docker" (default)}
    
    This adds support for running docker containers via the Mesos unified containerizer: http://mesos.apache.org/documentation/latest/container-image/
    
    The benefit is losing the dependency on `dockerd`, and all the costs which it incurs.
    
    I've also updated the supported Mesos version to 0.28.2 for support of the required protobufs.
    
    This is blocked on: https://github.com/apache/spark/pull/14167
    
    ## How was this patch tested?
    
    - manually testing jobs submitted with both "mesos" and "docker" settings for the new config var.
    - spark/mesos integration test suite
    
    Author: Michael Gummelt <mgummelt@mesosphere.io>
    
    Closes #14275 from mgummelt/unified-containerizer.
Loading