Skip to content
Snippets Groups Projects
Commit ef82bddc authored by tedyu's avatar tedyu Committed by Reynold Xin
Browse files

SPARK-7107 Add parameter for zookeeper.znode.parent to hbase_inputformat...

....py

Author: tedyu <yuzhihong@gmail.com>

Closes #5673 from tedyu/master and squashes the following commits:

ab7c72b [tedyu] SPARK-7107 Adjust indentation to pass Python style tests
6e25939 [tedyu] Adjust line length to be shorter than 100 characters
18d172a [tedyu] SPARK-7107 Add parameter for zookeeper.znode.parent to hbase_inputformat.py
parent 7078f602
No related branches found
No related tags found
No related merge requests found
......@@ -54,8 +54,9 @@ if __name__ == "__main__":
Run with example jar:
./bin/spark-submit --driver-class-path /path/to/example/jar \
/path/to/examples/hbase_inputformat.py <host> <table>
/path/to/examples/hbase_inputformat.py <host> <table> [<znode>]
Assumes you have some data in HBase already, running on <host>, in <table>
optionally, you can specify parent znode for your hbase cluster - <znode>
""", file=sys.stderr)
exit(-1)
......@@ -64,6 +65,9 @@ if __name__ == "__main__":
sc = SparkContext(appName="HBaseInputFormat")
conf = {"hbase.zookeeper.quorum": host, "hbase.mapreduce.inputtable": table}
if len(sys.argv) > 3:
conf = {"hbase.zookeeper.quorum": host, "zookeeper.znode.parent": sys.argv[3],
"hbase.mapreduce.inputtable": table}
keyConv = "org.apache.spark.examples.pythonconverters.ImmutableBytesWritableToStringConverter"
valueConv = "org.apache.spark.examples.pythonconverters.HBaseResultToStringConverter"
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment