Skip to content
Snippets Groups Projects
  • Nicholas Chammas's avatar
    31f48e5a
    [SPARK-8576] Add spark-ec2 options to set IAM roles and instance-initiated shutdown behavior · 31f48e5a
    Nicholas Chammas authored
    Both of these options are useful when spark-ec2 is being used as part of an automated pipeline and the engineers want to minimize the need to pass around AWS keys for access to things like S3 (keys are replaced by the IAM role) and to be able to launch a cluster that can terminate itself cleanly.
    
    Author: Nicholas Chammas <nicholas.chammas@gmail.com>
    
    Closes #6962 from nchammas/additional-ec2-options and squashes the following commits:
    
    fcf252e [Nicholas Chammas] PEP8 fixes
    efba9ee [Nicholas Chammas] add help for --instance-initiated-shutdown-behavior
    598aecf [Nicholas Chammas] option to launch instances into IAM role
    2743632 [Nicholas Chammas] add option for instance initiated shutdown
    31f48e5a
    History
    [SPARK-8576] Add spark-ec2 options to set IAM roles and instance-initiated shutdown behavior
    Nicholas Chammas authored
    Both of these options are useful when spark-ec2 is being used as part of an automated pipeline and the engineers want to minimize the need to pass around AWS keys for access to things like S3 (keys are replaced by the IAM role) and to be able to launch a cluster that can terminate itself cleanly.
    
    Author: Nicholas Chammas <nicholas.chammas@gmail.com>
    
    Closes #6962 from nchammas/additional-ec2-options and squashes the following commits:
    
    fcf252e [Nicholas Chammas] PEP8 fixes
    efba9ee [Nicholas Chammas] add help for --instance-initiated-shutdown-behavior
    598aecf [Nicholas Chammas] option to launch instances into IAM role
    2743632 [Nicholas Chammas] add option for instance initiated shutdown
spark_ec2.py 59.15 KiB