Skip to content
Snippets Groups Projects
  • Jeffrey Naisbitt's avatar
    28132ceb
    [SPARK-11264] bin/spark-class can't find assembly jars with certain GREP_OPTIONS set · 28132ceb
    Jeffrey Naisbitt authored
    Temporarily remove GREP_OPTIONS if set in bin/spark-class.
    
    Some GREP_OPTIONS will modify the output of the grep commands that are looking for the assembly jars.
    For example, if the -n option is specified, the grep output will look like:
    5:spark-assembly-1.5.1-hadoop2.4.0.jar
    
    This will not match the regular expressions, and so the jar files will not be found.  We could improve the regular expression to handle this case and trim off extra characters, but it is difficult to know which options may or may not be set.  Unsetting GREP_OPTIONS within the script handles all the cases and gives the desired output.
    
    Author: Jeffrey Naisbitt <jnaisbitt@familysearch.org>
    
    Closes #9231 from naisbitt/unset-GREP_OPTIONS.
    28132ceb
    History
    [SPARK-11264] bin/spark-class can't find assembly jars with certain GREP_OPTIONS set
    Jeffrey Naisbitt authored
    Temporarily remove GREP_OPTIONS if set in bin/spark-class.
    
    Some GREP_OPTIONS will modify the output of the grep commands that are looking for the assembly jars.
    For example, if the -n option is specified, the grep output will look like:
    5:spark-assembly-1.5.1-hadoop2.4.0.jar
    
    This will not match the regular expressions, and so the jar files will not be found.  We could improve the regular expression to handle this case and trim off extra characters, but it is difficult to know which options may or may not be set.  Unsetting GREP_OPTIONS within the script handles all the cases and gives the desired output.
    
    Author: Jeffrey Naisbitt <jnaisbitt@familysearch.org>
    
    Closes #9231 from naisbitt/unset-GREP_OPTIONS.