Skip to content
Snippets Groups Projects
  • GenTang's avatar
    4583cf4b
    [SPARK-5090] [EXAMPLES] The improvement of python converter for hbase · 4583cf4b
    GenTang authored
    Hi,
    
    Following the discussion in http://apache-spark-developers-list.1001551.n3.nabble.com/python-converter-in-HBaseConverter-scala-spark-examples-td10001.html. I made some modification in three files in package examples:
    1. HBaseConverters.scala: the new converter will converts all the records in an hbase results into a single string
    2. hbase_input.py: as the value string may contain several records, we can use ast package to convert the string into dict
    3. HBaseTest.scala: as the package examples use hbase 0.98.7 the original constructor HTableDescriptor is deprecated. The updation to new constructor is made
    
    Author: GenTang <gen.tang86@gmail.com>
    
    Closes #3920 from GenTang/master and squashes the following commits:
    
    d2153df [GenTang] import JSONObject precisely
    4802481 [GenTang] dump the result into a singl String
    62df7f0 [GenTang] remove the comment
    21de653 [GenTang] return the string in json format
    15b1fe3 [GenTang] the modification of comments
    5cbbcfc [GenTang] the improvement of pythonconverter
    ceb31c5 [GenTang] the modification for adapting updation of hbase
    3253b61 [GenTang] the modification accompanying the improvement of pythonconverter
    4583cf4b
    History
    [SPARK-5090] [EXAMPLES] The improvement of python converter for hbase
    GenTang authored
    Hi,
    
    Following the discussion in http://apache-spark-developers-list.1001551.n3.nabble.com/python-converter-in-HBaseConverter-scala-spark-examples-td10001.html. I made some modification in three files in package examples:
    1. HBaseConverters.scala: the new converter will converts all the records in an hbase results into a single string
    2. hbase_input.py: as the value string may contain several records, we can use ast package to convert the string into dict
    3. HBaseTest.scala: as the package examples use hbase 0.98.7 the original constructor HTableDescriptor is deprecated. The updation to new constructor is made
    
    Author: GenTang <gen.tang86@gmail.com>
    
    Closes #3920 from GenTang/master and squashes the following commits:
    
    d2153df [GenTang] import JSONObject precisely
    4802481 [GenTang] dump the result into a singl String
    62df7f0 [GenTang] remove the comment
    21de653 [GenTang] return the string in json format
    15b1fe3 [GenTang] the modification of comments
    5cbbcfc [GenTang] the improvement of pythonconverter
    ceb31c5 [GenTang] the modification for adapting updation of hbase
    3253b61 [GenTang] the modification accompanying the improvement of pythonconverter