Skip to content
Snippets Groups Projects
  • Andrew Or's avatar
    258d154c
    [SPARK-6048] SparkConf should not translate deprecated configs on set · 258d154c
    Andrew Or authored
    There are multiple issues with translating on set outlined in the JIRA.
    
    This PR reverts the translation logic added to `SparkConf`. In the future, after the 1.3.0 release we will figure out a way to reorganize the internal structure more elegantly. For now, let's preserve the existing semantics of `SparkConf` since it's a public interface. Unfortunately this means duplicating some code for now, but this is all internal and we can always clean it up later.
    
    Author: Andrew Or <andrew@databricks.com>
    
    Closes #4799 from andrewor14/conf-set-translate and squashes the following commits:
    
    11c525b [Andrew Or] Move warning to driver
    10e77b5 [Andrew Or] Add documentation for deprecation precedence
    a369cb1 [Andrew Or] Merge branch 'master' of github.com:apache/spark into conf-set-translate
    c26a9e3 [Andrew Or] Revert all translate logic in SparkConf
    fef6c9c [Andrew Or] Restore deprecation logic for spark.executor.userClassPathFirst
    94b4dfa [Andrew Or] Translate on get, not set
    258d154c
    History
    [SPARK-6048] SparkConf should not translate deprecated configs on set
    Andrew Or authored
    There are multiple issues with translating on set outlined in the JIRA.
    
    This PR reverts the translation logic added to `SparkConf`. In the future, after the 1.3.0 release we will figure out a way to reorganize the internal structure more elegantly. For now, let's preserve the existing semantics of `SparkConf` since it's a public interface. Unfortunately this means duplicating some code for now, but this is all internal and we can always clean it up later.
    
    Author: Andrew Or <andrew@databricks.com>
    
    Closes #4799 from andrewor14/conf-set-translate and squashes the following commits:
    
    11c525b [Andrew Or] Move warning to driver
    10e77b5 [Andrew Or] Add documentation for deprecation precedence
    a369cb1 [Andrew Or] Merge branch 'master' of github.com:apache/spark into conf-set-translate
    c26a9e3 [Andrew Or] Revert all translate logic in SparkConf
    fef6c9c [Andrew Or] Restore deprecation logic for spark.executor.userClassPathFirst
    94b4dfa [Andrew Or] Translate on get, not set