[SPARK-19517][SS] KafkaSource fails to initialize partition offsets
## What changes were proposed in this pull request? This patch fixes a bug in `KafkaSource` with the (de)serialization of the length of the JSON string that contains the initial partition offsets. ## How was this patch tested? I ran the test suite for spark-sql-kafka-0-10. Author: Roberto Agostino Vitillo <ra.vitillo@gmail.com> Closes #16857 from vitillo/kafka_source_fix.
Showing
- dev/.rat-excludes 1 addition, 0 deletionsdev/.rat-excludes
- external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSource.scala 25 additions, 7 deletions...ain/scala/org/apache/spark/sql/kafka010/KafkaSource.scala
- external/kafka-0-10-sql/src/test/resources/kafka-source-initial-offset-version-2.1.0.bin 1 addition, 0 deletions...t/resources/kafka-source-initial-offset-version-2.1.0.bin
- external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSourceSuite.scala 104 additions, 0 deletions...cala/org/apache/spark/sql/kafka010/KafkaSourceSuite.scala
Loading
Please register or sign in to comment