-
- Downloads
[SPARK-15648][SQL] Add teradataDialect for JDBC connection to Teradata
The contribution is my original work and I license the work to the project under the project’s open source license. Note: the Teradata JDBC connector limits the row size to 64K. The default string datatype equivalent I used is a 255 character/byte length varchar. This effectively limits the max number of string columns to 250 when using the Teradata jdbc connector. ## What changes were proposed in this pull request? Added a teradataDialect for JDBC connection to Teradata. The Teradata dialect uses VARCHAR(255) in place of TEXT for string datatypes, and CHAR(1) in place of BIT(1) for boolean datatypes. ## How was this patch tested? I added two unit tests to double check that the types get set correctly for a teradata jdbc url. I also ran a couple manual tests to make sure the jdbc connector worked with teradata and to make sure that an error was thrown if a row could potentially exceed 64K (this error comes from the teradata jdbc connector, not from the spark code). I did not check how string columns longer than 255 characters are handled. Author: Kirby Linvill <kirby.linvill@teradata.com> Author: klinvill <kjlinvill@gmail.com> Closes #16746 from klinvill/master.
Showing
- sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala 1 addition, 0 deletions...c/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala
- sql/core/src/main/scala/org/apache/spark/sql/jdbc/TeradataDialect.scala 34 additions, 0 deletions...ain/scala/org/apache/spark/sql/jdbc/TeradataDialect.scala
- sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala 12 additions, 0 deletions.../src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala
Loading
Please register or sign in to comment