Skip to content
  • medale's avatar
    75fdccca
    [SPARK-3039] [BUILD] Spark assembly for new hadoop API (hadoop 2) contai... · 75fdccca
    medale authored
    ...ns avro-mapred for
    
    hadoop 1 API had been marked as resolved but did not work for at least some
    builds due to version conflicts using avro-mapred-1.7.5.jar and
    avro-mapred-1.7.6-hadoop2.jar (the correct version) when building for hadoop2.
    
    sql/hive/pom.xml org.spark-project.hive:hive-exec's depends on 1.7.5:
    
    Building Spark Project Hive 1.2.0
    [INFO] ------------------------------------------------------------------------
    [INFO]
    [INFO] --- maven-dependency-plugin:2.4:tree (default-cli)  spark-hive_2.10 ---
    [INFO] org.apache.spark:spark-hive_2.10:jar:1.2.0
    [INFO] +- org.spark-project.hive:hive-exec:jar:0.13.1a:compile
    [INFO] |  \- org.apache.avro:avro-mapred:jar:1.7.5:compile
    [INFO] \- org.apache.avro:avro-mapred:jar:hadoop2:1.7.6:compile
    [INFO]
    
    Excluding this dependency allows the explicitly listed avro-mapred dependency
    to be picked up.
    
    Author: medale <medale94@yahoo.com>
    
    Closes #4315 from medale/avro-hadoop2 and squashes the following commits:
    
    1ab4fa3 [medale] Merge branch 'master' into avro-hadoop2
    9d85e2a [medale] Merge remote-tracking branch 'upstream/master' into avro-hadoop2
    51b9c2a [medale] [SPARK-3039] [BUILD] Spark assembly for new hadoop API (hadoop 2) contains avro-mapred for hadoop 1 API had been marked as resolved but did not work for at least some builds due to version conflicts using avro-mapred-1.7.5.jar and avro-mapred-1.7.6-hadoop2.jar (the correct version) when building for hadoop2.
    75fdccca
    [SPARK-3039] [BUILD] Spark assembly for new hadoop API (hadoop 2) contai...
    medale authored
    ...ns avro-mapred for
    
    hadoop 1 API had been marked as resolved but did not work for at least some
    builds due to version conflicts using avro-mapred-1.7.5.jar and
    avro-mapred-1.7.6-hadoop2.jar (the correct version) when building for hadoop2.
    
    sql/hive/pom.xml org.spark-project.hive:hive-exec's depends on 1.7.5:
    
    Building Spark Project Hive 1.2.0
    [INFO] ------------------------------------------------------------------------
    [INFO]
    [INFO] --- maven-dependency-plugin:2.4:tree (default-cli)  spark-hive_2.10 ---
    [INFO] org.apache.spark:spark-hive_2.10:jar:1.2.0
    [INFO] +- org.spark-project.hive:hive-exec:jar:0.13.1a:compile
    [INFO] |  \- org.apache.avro:avro-mapred:jar:1.7.5:compile
    [INFO] \- org.apache.avro:avro-mapred:jar:hadoop2:1.7.6:compile
    [INFO]
    
    Excluding this dependency allows the explicitly listed avro-mapred dependency
    to be picked up.
    
    Author: medale <medale94@yahoo.com>
    
    Closes #4315 from medale/avro-hadoop2 and squashes the following commits:
    
    1ab4fa3 [medale] Merge branch 'master' into avro-hadoop2
    9d85e2a [medale] Merge remote-tracking branch 'upstream/master' into avro-hadoop2
    51b9c2a [medale] [SPARK-3039] [BUILD] Spark assembly for new hadoop API (hadoop 2) contains avro-mapred for hadoop 1 API had been marked as resolved but did not work for at least some builds due to version conflicts using avro-mapred-1.7.5.jar and avro-mapred-1.7.6-hadoop2.jar (the correct version) when building for hadoop2.
Loading