Skip to content
  • Steve Loughran's avatar
    ee11be25
    SPARK-6433 hive tests to import spark-sql test JAR for QueryTest access · ee11be25
    Steve Loughran authored
    1. Test JARs are built & published
    1. log4j.resources is explicitly excluded. Without this, downstream test run logging depends on the order the JARs are listed/loaded
    1. sql/hive pulls in spark-sql &...spark-catalyst for its test runs
    1. The copied in test classes were rm'd, and a test edited to remove its now duplicate assert method
    1. Spark streaming is now build with the same plugin/phase as the rest, but its shade plugin declaration is kept in (so different from the rest of the test plugins). Due to (#2), this means the test JAR no longer includes its log4j file.
    
    Outstanding issues:
    * should the JARs be shaded? `spark-streaming-test.jar` does, but given these are test jars for developers only, especially in the same spark source tree, it's hard to justify.
    * `maven-jar-plugin` v 2.6 was explicitly selected; without this the apache-1.4 parent template JAR version (2.4) chosen.
    * Are there any other resources to exclude?
    
    Author: Steve Loughran <stevel@hortonworks.com>
    
    Closes #5119 from steveloughran/stevel/patches/SPARK-6433-test-jars and squashes the following commits:
    
    81ceb01 [Steve Loughran] SPARK-6433 add a clearer comment explaining what the plugin is doing & why
    a6dca33 [Steve Loughran] SPARK-6433 : pull configuration section form archive plugin
    c2b5f89 [Steve Loughran] SPARK-6433 omit "jar" goal from jar plugin
    fdac51b [Steve Loughran] SPARK-6433 -002; indentation & delegate plugin version to parent
    650f442 [Steve Loughran] SPARK-6433 patch 001: test JARs are built; sql/hive pulls in spark-sql & spark-catalyst for its test runs
    ee11be25
    SPARK-6433 hive tests to import spark-sql test JAR for QueryTest access
    Steve Loughran authored
    1. Test JARs are built & published
    1. log4j.resources is explicitly excluded. Without this, downstream test run logging depends on the order the JARs are listed/loaded
    1. sql/hive pulls in spark-sql &...spark-catalyst for its test runs
    1. The copied in test classes were rm'd, and a test edited to remove its now duplicate assert method
    1. Spark streaming is now build with the same plugin/phase as the rest, but its shade plugin declaration is kept in (so different from the rest of the test plugins). Due to (#2), this means the test JAR no longer includes its log4j file.
    
    Outstanding issues:
    * should the JARs be shaded? `spark-streaming-test.jar` does, but given these are test jars for developers only, especially in the same spark source tree, it's hard to justify.
    * `maven-jar-plugin` v 2.6 was explicitly selected; without this the apache-1.4 parent template JAR version (2.4) chosen.
    * Are there any other resources to exclude?
    
    Author: Steve Loughran <stevel@hortonworks.com>
    
    Closes #5119 from steveloughran/stevel/patches/SPARK-6433-test-jars and squashes the following commits:
    
    81ceb01 [Steve Loughran] SPARK-6433 add a clearer comment explaining what the plugin is doing & why
    a6dca33 [Steve Loughran] SPARK-6433 : pull configuration section form archive plugin
    c2b5f89 [Steve Loughran] SPARK-6433 omit "jar" goal from jar plugin
    fdac51b [Steve Loughran] SPARK-6433 -002; indentation & delegate plugin version to parent
    650f442 [Steve Loughran] SPARK-6433 patch 001: test JARs are built; sql/hive pulls in spark-sql & spark-catalyst for its test runs
Loading