Skip to content
Snippets Groups Projects
  • hyukjinkwon's avatar
    0c0ad436
    [SPARK-17719][SPARK-17776][SQL] Unify and tie up options in a single place in... · 0c0ad436
    hyukjinkwon authored
    [SPARK-17719][SPARK-17776][SQL] Unify and tie up options in a single place in JDBC datasource package
    
    ## What changes were proposed in this pull request?
    
    This PR proposes to fix arbitrary usages among `Map[String, String]`, `Properties` and `JDBCOptions` instances for options in `execution/jdbc` package and make the connection properties exclude Spark-only options.
    
    This PR includes some changes as below:
    
      - Unify `Map[String, String]`, `Properties` and `JDBCOptions` in `execution/jdbc` package to `JDBCOptions`.
    
    - Move `batchsize`, `fetchszie`, `driver` and `isolationlevel` options into `JDBCOptions` instance.
    
    - Document `batchSize` and `isolationlevel` with marking both read-only options and write-only options. Also, this includes minor types and detailed explanation for some statements such as url.
    
    - Throw exceptions fast by checking arguments first rather than in execution time (e.g. for `fetchsize`).
    
    - Exclude Spark-only options in connection properties.
    
    ## How was this patch tested?
    
    Existing tests should cover this.
    
    Author: hyukjinkwon <gurwls223@gmail.com>
    
    Closes #15292 from HyukjinKwon/SPARK-17719.
    0c0ad436
    History
    [SPARK-17719][SPARK-17776][SQL] Unify and tie up options in a single place in...
    hyukjinkwon authored
    [SPARK-17719][SPARK-17776][SQL] Unify and tie up options in a single place in JDBC datasource package
    
    ## What changes were proposed in this pull request?
    
    This PR proposes to fix arbitrary usages among `Map[String, String]`, `Properties` and `JDBCOptions` instances for options in `execution/jdbc` package and make the connection properties exclude Spark-only options.
    
    This PR includes some changes as below:
    
      - Unify `Map[String, String]`, `Properties` and `JDBCOptions` in `execution/jdbc` package to `JDBCOptions`.
    
    - Move `batchsize`, `fetchszie`, `driver` and `isolationlevel` options into `JDBCOptions` instance.
    
    - Document `batchSize` and `isolationlevel` with marking both read-only options and write-only options. Also, this includes minor types and detailed explanation for some statements such as url.
    
    - Throw exceptions fast by checking arguments first rather than in execution time (e.g. for `fetchsize`).
    
    - Exclude Spark-only options in connection properties.
    
    ## How was this patch tested?
    
    Existing tests should cover this.
    
    Author: hyukjinkwon <gurwls223@gmail.com>
    
    Closes #15292 from HyukjinKwon/SPARK-17719.