Skip to content
  • azagrebin's avatar
    16687651
    [SPARK-3340] Deprecate ADD_JARS and ADD_FILES · 16687651
    azagrebin authored
    I created a patch that disables the environment variables.
    Thereby scala or python shell log a warning message to notify user about the deprecation
    with the following message:
    scala: "ADD_JARS environment variable is deprecated, use --jar spark submit argument instead"
    python: "Warning: ADD_FILES environment variable is deprecated, use --py-files argument instead"
    
    Is it what is expected or the code associated with the variables should be just completely removed?
    Should it be somewhere documented?
    
    Author: azagrebin <azagrebin@gmail.com>
    
    Closes #4616 from azagrebin/master and squashes the following commits:
    
    bab1aa9 [azagrebin] [SPARK-3340] Deprecate ADD_JARS and ADD_FILES: minor readability issue
    0643895 [azagrebin] [SPARK-3340] Deprecate ADD_JARS and ADD_FILES: add warning messages
    42f0107 [azagrebin] [SPARK-3340] Deprecate ADD_JARS and ADD_FILES
    16687651
    [SPARK-3340] Deprecate ADD_JARS and ADD_FILES
    azagrebin authored
    I created a patch that disables the environment variables.
    Thereby scala or python shell log a warning message to notify user about the deprecation
    with the following message:
    scala: "ADD_JARS environment variable is deprecated, use --jar spark submit argument instead"
    python: "Warning: ADD_FILES environment variable is deprecated, use --py-files argument instead"
    
    Is it what is expected or the code associated with the variables should be just completely removed?
    Should it be somewhere documented?
    
    Author: azagrebin <azagrebin@gmail.com>
    
    Closes #4616 from azagrebin/master and squashes the following commits:
    
    bab1aa9 [azagrebin] [SPARK-3340] Deprecate ADD_JARS and ADD_FILES: minor readability issue
    0643895 [azagrebin] [SPARK-3340] Deprecate ADD_JARS and ADD_FILES: add warning messages
    42f0107 [azagrebin] [SPARK-3340] Deprecate ADD_JARS and ADD_FILES
Loading