-
- Downloads
[SPARK-4508] [SQL] build native date type to conform behavior to Hive
Store daysSinceEpoch as an Int value(4 bytes) to represent DateType, instead of using java.sql.Date(8 bytes as Long) in catalyst row. This ensures the same comparison behavior of Hive and Catalyst. Subsumes #3381 I thinks there are already some tests in JavaSQLSuite, and for python it will not affect python's datetime class. Author: Daoyuan Wang <daoyuan.wang@intel.com> Closes #3732 from adrian-wang/datenative and squashes the following commits: 0ed0fdc [Daoyuan Wang] fix test data a2fdd4e [Daoyuan Wang] getDate c37832b [Daoyuan Wang] row to catalyst f0005b1 [Daoyuan Wang] add date in sql parser and java type conversion 024c9a6 [Daoyuan Wang] clean some import order d6715fc [Daoyuan Wang] refactoring Date as Primitive Int internally 374abd5 [Daoyuan Wang] spark native date type support
Showing
- sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala 1 addition, 1 deletionsql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala 5 additions, 4 deletions...scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/SqlParser.scala 2 additions, 0 deletions.../main/scala/org/apache/spark/sql/catalyst/SqlParser.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala 20 additions, 33 deletions...cala/org/apache/spark/sql/catalyst/expressions/Cast.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala 3 additions, 0 deletions...park/sql/catalyst/expressions/codegen/CodeGenerator.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/literals.scala 1 addition, 1 deletion.../org/apache/spark/sql/catalyst/expressions/literals.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/types/DateUtils.scala 60 additions, 0 deletions...src/main/scala/org/apache/spark/sql/types/DateUtils.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/types/dataTypes.scala 5 additions, 7 deletions...src/main/scala/org/apache/spark/sql/types/dataTypes.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvaluationSuite.scala 14 additions, 14 deletions.../sql/catalyst/expressions/ExpressionEvaluationSuite.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/types/DataTypeSuite.scala 1 addition, 1 deletion...test/scala/org/apache/spark/sql/types/DataTypeSuite.scala
- sql/core/src/main/scala/org/apache/spark/sql/columnar/ColumnStats.scala 2 additions, 17 deletions...ain/scala/org/apache/spark/sql/columnar/ColumnStats.scala
- sql/core/src/main/scala/org/apache/spark/sql/columnar/ColumnType.scala 6 additions, 7 deletions...main/scala/org/apache/spark/sql/columnar/ColumnType.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/pythonUdfs.scala 3 additions, 1 deletion...ain/scala/org/apache/spark/sql/execution/pythonUdfs.scala
- sql/core/src/main/scala/org/apache/spark/sql/json/JsonRDD.scala 4 additions, 2 deletions...re/src/main/scala/org/apache/spark/sql/json/JsonRDD.scala
- sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala 7 additions, 0 deletions...e/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/ScalaReflectionRelationSuite.scala 2 additions, 1 deletion...a/org/apache/spark/sql/ScalaReflectionRelationSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/columnar/ColumnStatsSuite.scala 1 addition, 1 deletion...cala/org/apache/spark/sql/columnar/ColumnStatsSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/columnar/ColumnTypeSuite.scala 3 additions, 3 deletions...scala/org/apache/spark/sql/columnar/ColumnTypeSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/columnar/ColumnarTestUtils.scala 3 additions, 3 deletions...ala/org/apache/spark/sql/columnar/ColumnarTestUtils.scala
- sql/core/src/test/scala/org/apache/spark/sql/json/JsonSuite.scala 4 additions, 3 deletions.../src/test/scala/org/apache/spark/sql/json/JsonSuite.scala
Loading
Please register or sign in to comment