Skip to content
Snippets Groups Projects
Commit 70f88148 authored by Reynold Xin's avatar Reynold Xin
Browse files

[Docs] Replace references to SchemaRDD with DataFrame

Author: Reynold Xin <rxin@databricks.com>

Closes #4952 from rxin/schemardd-df-reference and squashes the following commits:

b2b1dbe [Reynold Xin] [Docs] Replace references to SchemaRDD with DataFrame
parent f7c79920
No related branches found
No related tags found
No related merge requests found
......@@ -39,7 +39,7 @@ class Estimator(Params):
Fits a model to the input dataset with optional parameters.
:param dataset: input dataset, which is an instance of
:py:class:`pyspark.sql.SchemaRDD`
:py:class:`pyspark.sql.DataFrame`
:param params: an optional param map that overwrites embedded
params
:returns: fitted model
......@@ -62,7 +62,7 @@ class Transformer(Params):
Transforms the input dataset with optional parameters.
:param dataset: input dataset, which is an instance of
:py:class:`pyspark.sql.SchemaRDD`
:py:class:`pyspark.sql.DataFrame`
:param params: an optional param map that overwrites embedded
params
:returns: transformed dataset
......
......@@ -102,7 +102,7 @@ class JavaEstimator(Estimator, JavaWrapper):
"""
Fits a Java model to the input dataset.
:param dataset: input dataset, which is an instance of
:py:class:`pyspark.sql.SchemaRDD`
:py:class:`pyspark.sql.DataFrame`
:param params: additional params (overwriting embedded values)
:return: fitted Java model
"""
......
......@@ -263,14 +263,14 @@ class ReplSuite extends FunSuite {
assertDoesNotContain("Exception", output)
}
test("SPARK-2576 importing SQLContext.createSchemaRDD.") {
test("SPARK-2576 importing SQLContext.createDataFrame.") {
// We need to use local-cluster to test this case.
val output = runInterpreter("local-cluster[1,1,512]",
"""
|val sqlContext = new org.apache.spark.sql.SQLContext(sc)
|import sqlContext.createSchemaRDD
|import sqlContext.implicits._
|case class TestCaseClass(value: Int)
|sc.parallelize(1 to 10).map(x => TestCaseClass(x)).toSchemaRDD.collect
|sc.parallelize(1 to 10).map(x => TestCaseClass(x)).toDF.collect
""".stripMargin)
assertDoesNotContain("error:", output)
assertDoesNotContain("Exception", output)
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment