-
- Downloads
[SPARK-14744][EXAMPLES] Clean up examples packaging, remove outdated examples.
First, make all dependencies in the examples module provided, and explicitly list a couple of ones that somehow are promoted to compile by maven. This means that to run streaming examples, the streaming connector package needs to be provided to run-examples using --packages or --jars, just like regular apps. Also, remove a couple of outdated examples. HBase has had Spark bindings for a while and is even including them in the HBase distribution in the next version, making the examples obsolete. The same applies to Cassandra, which seems to have a proper Spark binding library already. I just tested the build, which passes, and ran SparkPi. The examples jars directory now has only two jars: ``` $ ls -1 examples/target/scala-2.11/jars/ scopt_2.11-3.3.0.jar spark-examples_2.11-2.0.0-SNAPSHOT.jar ``` Author: Marcelo Vanzin <vanzin@cloudera.com> Closes #12544 from vanzin/SPARK-14744.
Showing
- examples/pom.xml 15 additions, 214 deletionsexamples/pom.xml
- examples/src/main/python/cassandra_inputformat.py 0 additions, 84 deletionsexamples/src/main/python/cassandra_inputformat.py
- examples/src/main/python/cassandra_outputformat.py 0 additions, 88 deletionsexamples/src/main/python/cassandra_outputformat.py
- examples/src/main/python/hbase_inputformat.py 0 additions, 90 deletionsexamples/src/main/python/hbase_inputformat.py
- examples/src/main/python/hbase_outputformat.py 0 additions, 73 deletionsexamples/src/main/python/hbase_outputformat.py
- examples/src/main/scala/org/apache/spark/examples/CassandraCQLTest.scala 0 additions, 135 deletions...in/scala/org/apache/spark/examples/CassandraCQLTest.scala
- examples/src/main/scala/org/apache/spark/examples/CassandraTest.scala 0 additions, 215 deletions.../main/scala/org/apache/spark/examples/CassandraTest.scala
- examples/src/main/scala/org/apache/spark/examples/HBaseTest.scala 0 additions, 65 deletions.../src/main/scala/org/apache/spark/examples/HBaseTest.scala
- examples/src/main/scala/org/apache/spark/examples/pythonconverters/CassandraConverters.scala 0 additions, 70 deletions...spark/examples/pythonconverters/CassandraConverters.scala
- examples/src/main/scala/org/apache/spark/examples/pythonconverters/HBaseConverters.scala 0 additions, 84 deletions...che/spark/examples/pythonconverters/HBaseConverters.scala
- pom.xml 1 addition, 7 deletionspom.xml
Loading
Please register or sign in to comment