Skip to content
Snippets Groups Projects
Commit 488863d8 authored by Holden Karau's avatar Holden Karau Committed by Marcelo Vanzin
Browse files

[SPARK-13382][DOCS][PYSPARK] Update pyspark testing notes in build docs

## What changes were proposed in this pull request?

The current build documents don't specify that for PySpark tests we need to include Hive in the assembly otherwise the ORC tests fail.

## How was the this patch tested?

Manually built the docs locally. Ran the provided build command follow by the PySpark SQL tests.

![pyspark2](https://cloud.githubusercontent.com/assets/59893/13190008/8829cde4-d70f-11e5-8ff5-a88b7894d2ad.png)

Author: Holden Karau <holden@us.ibm.com>

Closes #11278 from holdenk/SPARK-13382-update-pyspark-testing-notes-r2.
parent 26462653
No related branches found
No related tags found
No related merge requests found
......@@ -123,6 +123,21 @@ To produce a Spark package compiled with Scala 2.10, use the `-Dscala-2.10` prop
./dev/change-scala-version.sh 2.10
mvn -Pyarn -Phadoop-2.4 -Dscala-2.10 -DskipTests clean package
# PySpark Tests with Maven
If you are building PySpark and wish to run the PySpark tests you will need to build Spark with hive support.
{% highlight bash %}
build/mvn -DskipTests clean package -Phive
./python/run-tests
{% endhighlight %}
The run-tests script also can be limited to a specific Python version or a specific module
./python/run-tests --python-executables=python --modules=pyspark-sql
**Note:** You can also run Python tests with an sbt build, provided you build Spark with hive support.
# Spark Tests in Maven
Tests are run by default via the [ScalaTest Maven plugin](http://www.scalatest.org/user_guide/using_the_scalatest_maven_plugin).
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment