Skip to content
Snippets Groups Projects
Commit 08eac356 authored by Shixiong Zhu's avatar Shixiong Zhu
Browse files

[SPARK-17834][SQL] Fetch the earliest offsets manually in KafkaSource instead... 

[SPARK-17834][SQL] Fetch the earliest offsets manually in KafkaSource instead of counting on KafkaConsumer

## What changes were proposed in this pull request?

Because `KafkaConsumer.poll(0)` may update the partition offsets, this PR just calls `seekToBeginning` to manually set the earliest offsets for the KafkaSource initial offsets.

## How was this patch tested?

Existing tests.

Author: Shixiong Zhu <shixiong@databricks.com>

Closes #15397 from zsxwing/SPARK-17834.
parent 84f149e4
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment