Skip to content
Snippets Groups Projects
  1. Mar 27, 2015
    • Davies Liu's avatar
      [SPARK-6574] [PySpark] fix sql example · 887e1b72
      Davies Liu authored
      Fix the import in sql example.
      
      Author: Davies Liu <davies@databricks.com>
      
      Closes #5230 from davies/fix_sql_example and squashes the following commits:
      
      7ecc5f4 [Davies Liu] fix sql example
      887e1b72
  2. Feb 10, 2015
    • Davies Liu's avatar
      [SPARK-5704] [SQL] [PySpark] createDataFrame from RDD with columns · ea602840
      Davies Liu authored
      Deprecate inferSchema() and applySchema(), use createDataFrame() instead, which could take an optional `schema` to create an DataFrame from an RDD. The `schema` could be StructType or list of names of columns.
      
      Author: Davies Liu <davies@databricks.com>
      
      Closes #4498 from davies/create and squashes the following commits:
      
      08469c1 [Davies Liu] remove Scala/Java API for now
      c80a7a9 [Davies Liu] fix hive test
      d1bd8f2 [Davies Liu] cleanup applySchema
      9526e97 [Davies Liu] createDataFrame from RDD with columns
      ea602840
  3. Jan 27, 2015
    • Reynold Xin's avatar
      [SPARK-5097][SQL] DataFrame · 119f45d6
      Reynold Xin authored
      This pull request redesigns the existing Spark SQL dsl, which already provides data frame like functionalities.
      
      TODOs:
      With the exception of Python support, other tasks can be done in separate, follow-up PRs.
      - [ ] Audit of the API
      - [ ] Documentation
      - [ ] More test cases to cover the new API
      - [x] Python support
      - [ ] Type alias SchemaRDD
      
      Author: Reynold Xin <rxin@databricks.com>
      Author: Davies Liu <davies@databricks.com>
      
      Closes #4173 from rxin/df1 and squashes the following commits:
      
      0a1a73b [Reynold Xin] Merge branch 'df1' of github.com:rxin/spark into df1
      23b4427 [Reynold Xin] Mima.
      828f70d [Reynold Xin] Merge pull request #7 from davies/df
      257b9e6 [Davies Liu] add repartition
      6bf2b73 [Davies Liu] fix collect with UDT and tests
      e971078 [Reynold Xin] Missing quotes.
      b9306b4 [Reynold Xin] Remove removeColumn/updateColumn for now.
      a728bf2 [Reynold Xin] Example rename.
      e8aa3d3 [Reynold Xin] groupby -> groupBy.
      9662c9e [Davies Liu] improve DataFrame Python API
      4ae51ea [Davies Liu] python API for dataframe
      1e5e454 [Reynold Xin] Fixed a bug with symbol conversion.
      2ca74db [Reynold Xin] Couple minor fixes.
      ea98ea1 [Reynold Xin] Documentation & literal expressions.
      2b22684 [Reynold Xin] Got rid of IntelliJ problems.
      02bbfbc [Reynold Xin] Tightening imports.
      ffbce66 [Reynold Xin] Fixed compilation error.
      59b6d8b [Reynold Xin] Style violation.
      b85edfb [Reynold Xin] ALS.
      8c37f0a [Reynold Xin] Made MLlib and examples compile
      6d53134 [Reynold Xin] Hive module.
      d35efd5 [Reynold Xin] Fixed compilation error.
      ce4a5d2 [Reynold Xin] Fixed test cases in SQL except ParquetIOSuite.
      66d5ef1 [Reynold Xin] SQLContext minor patch.
      c9bcdc0 [Reynold Xin] Checkpoint: SQL module compiles!
      119f45d6
  4. Oct 17, 2014
  5. Oct 01, 2014
    • jyotiska's avatar
      Python SQL Example Code · 17333c7a
      jyotiska authored
      SQL example code for Python, as shown on [SQL Programming Guide](https://spark.apache.org/docs/1.0.2/sql-programming-guide.html)
      
      Author: jyotiska <jyotiska123@gmail.com>
      
      Closes #2521 from jyotiska/sql_example and squashes the following commits:
      
      1471dcb [jyotiska] added imports for sql
      b25e436 [jyotiska] pep 8 compliance
      43fd10a [jyotiska] lines broken to maintain 80 char limit
      b4fdf4e [jyotiska] removed blank lines
      83d5ab7 [jyotiska] added inferschema and applyschema to the demo
      306667e [jyotiska] replaced blank line with end line
      c90502a [jyotiska] fixed new line
      4939a70 [jyotiska] added new line at end for python style
      0b46148 [jyotiska] fixed appname for python sql example
      8f67b5b [jyotiska] added python sql example
      17333c7a
Loading