Skip to content
  • Davies Liu's avatar
    f0d880e2
    [SPARK-2674] [SQL] [PySpark] support datetime type for SchemaRDD · f0d880e2
    Davies Liu authored
    Datetime and time in Python will be converted into java.util.Calendar after serialization, it will be converted into java.sql.Timestamp during inferSchema().
    
    In javaToPython(), Timestamp will be converted into Calendar, then be converted into datetime in Python after pickling.
    
    Author: Davies Liu <davies.liu@gmail.com>
    
    Closes #1601 from davies/date and squashes the following commits:
    
    f0599b0 [Davies Liu] remove tests for sets and tuple in sql, fix list of list
    c9d607a [Davies Liu] convert datetype for runtime
    709d40d [Davies Liu] remove brackets
    96db384 [Davies Liu] support datetime type for SchemaRDD
    f0d880e2
    [SPARK-2674] [SQL] [PySpark] support datetime type for SchemaRDD
    Davies Liu authored
    Datetime and time in Python will be converted into java.util.Calendar after serialization, it will be converted into java.sql.Timestamp during inferSchema().
    
    In javaToPython(), Timestamp will be converted into Calendar, then be converted into datetime in Python after pickling.
    
    Author: Davies Liu <davies.liu@gmail.com>
    
    Closes #1601 from davies/date and squashes the following commits:
    
    f0599b0 [Davies Liu] remove tests for sets and tuple in sql, fix list of list
    c9d607a [Davies Liu] convert datetype for runtime
    709d40d [Davies Liu] remove brackets
    96db384 [Davies Liu] support datetime type for SchemaRDD
Loading