Skip to content
Snippets Groups Projects
Commit 93861a5e authored by aniketbhatnagar's avatar aniketbhatnagar Committed by Josh Rosen
Browse files

SPARK-3638 | Forced a compatible version of http client in kinesis-asl profile

This patch forces use of commons http client 4.2 in Kinesis-asl profile so that the AWS SDK does not run into dependency conflicts

Author: aniketbhatnagar <aniket.bhatnagar@gmail.com>

Closes #2535 from aniketbhatnagar/Kinesis-HttpClient-Dep-Fix and squashes the following commits:

aa2079f [aniketbhatnagar] Merge branch 'Kinesis-HttpClient-Dep-Fix' of https://github.com/aniketbhatnagar/spark into Kinesis-HttpClient-Dep-Fix
73f55f6 [aniketbhatnagar] SPARK-3638 | Forced a compatible version of http client in kinesis-asl profile
70cc75b [aniketbhatnagar] deleted merge files
725dbc9 [aniketbhatnagar] Merge remote-tracking branch 'origin/Kinesis-HttpClient-Dep-Fix' into Kinesis-HttpClient-Dep-Fix
4ed61d8 [aniketbhatnagar] SPARK-3638 | Forced a compatible version of http client in kinesis-asl profile
9cd6103 [aniketbhatnagar] SPARK-3638 | Forced a compatible version of http client in kinesis-asl profile
parent 1b9f0d67
No related branches found
No related tags found
No related merge requests found
...@@ -349,5 +349,15 @@ ...@@ -349,5 +349,15 @@
</plugins> </plugins>
</build> </build>
</profile> </profile>
<profile>
<id>kinesis-asl</id>
<dependencies>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>${commons.httpclient.version}</version>
</dependency>
</dependencies>
</profile>
</profiles> </profiles>
</project> </project>
...@@ -43,6 +43,11 @@ ...@@ -43,6 +43,11 @@
<artifactId>spark-streaming-kinesis-asl_${scala.binary.version}</artifactId> <artifactId>spark-streaming-kinesis-asl_${scala.binary.version}</artifactId>
<version>${project.version}</version> <version>${project.version}</version>
</dependency> </dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>${commons.httpclient.version}</version>
</dependency>
</dependencies> </dependencies>
</profile> </profile>
</profiles> </profiles>
......
...@@ -138,6 +138,7 @@ ...@@ -138,6 +138,7 @@
<jets3t.version>0.7.1</jets3t.version> <jets3t.version>0.7.1</jets3t.version>
<aws.java.sdk.version>1.8.3</aws.java.sdk.version> <aws.java.sdk.version>1.8.3</aws.java.sdk.version>
<aws.kinesis.client.version>1.1.0</aws.kinesis.client.version> <aws.kinesis.client.version>1.1.0</aws.kinesis.client.version>
<commons.httpclient.version>4.2.6</commons.httpclient.version>
<PermGen>64m</PermGen> <PermGen>64m</PermGen>
<MaxPermGen>512m</MaxPermGen> <MaxPermGen>512m</MaxPermGen>
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment