Skip to content
Snippets Groups Projects
user avatar
hyukjinkwon authored
[SPARK-17719][SPARK-17776][SQL] Unify and tie up options in a single place in JDBC datasource package

## What changes were proposed in this pull request?

This PR proposes to fix arbitrary usages among `Map[String, String]`, `Properties` and `JDBCOptions` instances for options in `execution/jdbc` package and make the connection properties exclude Spark-only options.

This PR includes some changes as below:

  - Unify `Map[String, String]`, `Properties` and `JDBCOptions` in `execution/jdbc` package to `JDBCOptions`.

- Move `batchsize`, `fetchszie`, `driver` and `isolationlevel` options into `JDBCOptions` instance.

- Document `batchSize` and `isolationlevel` with marking both read-only options and write-only options. Also, this includes minor types and detailed explanation for some statements such as url.

- Throw exceptions fast by checking arguments first rather than in execution time (e.g. for `fetchsize`).

- Exclude Spark-only options in connection properties.

## How was this patch tested?

Existing tests should cover this.

Author: hyukjinkwon <gurwls223@gmail.com>

Closes #15292 from HyukjinKwon/SPARK-17719.
0c0ad436
History
Name Last commit Last update
..
_data
_includes
_layouts
_plugins
css
img
js
README.md
_config.yml
api.md
building-spark.md
cluster-overview.md
configuration.md
contributing-to-spark.md
ec2-scripts.md
graphx-programming-guide.md
hadoop-provided.md
hardware-provisioning.md
index.md
java-programming-guide.md
job-scheduling.md
ml-advanced.md
ml-ann.md
ml-classification-regression.md
ml-clustering.md
ml-collaborative-filtering.md
ml-decision-tree.md
ml-ensembles.md
ml-features.md
ml-guide.md
ml-linear-methods.md
ml-migration-guides.md
ml-pipeline.md
ml-survival-regression.md
ml-tuning.md
mllib-classification-regression.md
mllib-clustering.md
mllib-collaborative-filtering.md
mllib-data-types.md
mllib-decision-tree.md
mllib-dimensionality-reduction.md
mllib-ensembles.md
mllib-evaluation-metrics.md
mllib-feature-extraction.md
mllib-frequent-pattern-mining.md
mllib-guide.md
mllib-isotonic-regression.md
mllib-linear-methods.md
mllib-migration-guides.md
mllib-naive-bayes.md
mllib-optimization.md
mllib-pmml-model-export.md
mllib-statistics.md
monitoring.md
programming-guide.md
python-programming-guide.md
quick-start.md
running-on-mesos.md
running-on-yarn.md
scala-programming-guide.md
security.md
spark-standalone.md
sparkr.md
sql-programming-guide.md
storage-openstack-swift.md
streaming-custom-receivers.md
streaming-flume-integration.md
streaming-kafka-0-10-integration.md
streaming-kafka-0-8-integration.md
streaming-kafka-integration.md
streaming-kinesis-integration.md
streaming-programming-guide.md
structured-streaming-kafka-integration.md
structured-streaming-programming-guide.md
submitting-applications.md
tuning.md