Class KafkaSourceEnumerator
- java.lang.Object
-
- org.apache.flink.connector.kafka.source.enumerator.KafkaSourceEnumerator
-
- All Implemented Interfaces:
AutoCloseable,org.apache.flink.api.common.state.CheckpointListener,org.apache.flink.api.connector.source.SplitEnumerator<KafkaPartitionSplit,KafkaSourceEnumState>
@Internal public class KafkaSourceEnumerator extends Object implements org.apache.flink.api.connector.source.SplitEnumerator<KafkaPartitionSplit,KafkaSourceEnumState>
The enumerator class for Kafka source.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static classKafkaSourceEnumerator.PartitionOffsetsRetrieverImplThe implementation for offsets retriever with a consumer and an admin client.
-
Constructor Summary
Constructors Constructor Description KafkaSourceEnumerator(KafkaSubscriber subscriber, OffsetsInitializer startingOffsetInitializer, OffsetsInitializer stoppingOffsetInitializer, Properties properties, org.apache.flink.api.connector.source.SplitEnumeratorContext<KafkaPartitionSplit> context, org.apache.flink.api.connector.source.Boundedness boundedness)KafkaSourceEnumerator(KafkaSubscriber subscriber, OffsetsInitializer startingOffsetInitializer, OffsetsInitializer stoppingOffsetInitializer, Properties properties, org.apache.flink.api.connector.source.SplitEnumeratorContext<KafkaPartitionSplit> context, org.apache.flink.api.connector.source.Boundedness boundedness, Set<org.apache.kafka.common.TopicPartition> assignedPartitions)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description voidaddReader(int subtaskId)voidaddSplitsBack(List<KafkaPartitionSplit> splits, int subtaskId)voidclose()voidhandleSplitRequest(int subtaskId, String requesterHostname)KafkaSourceEnumStatesnapshotState(long checkpointId)voidstart()Start the enumerator.-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
-
-
-
Constructor Detail
-
KafkaSourceEnumerator
public KafkaSourceEnumerator(KafkaSubscriber subscriber, OffsetsInitializer startingOffsetInitializer, OffsetsInitializer stoppingOffsetInitializer, Properties properties, org.apache.flink.api.connector.source.SplitEnumeratorContext<KafkaPartitionSplit> context, org.apache.flink.api.connector.source.Boundedness boundedness)
-
KafkaSourceEnumerator
public KafkaSourceEnumerator(KafkaSubscriber subscriber, OffsetsInitializer startingOffsetInitializer, OffsetsInitializer stoppingOffsetInitializer, Properties properties, org.apache.flink.api.connector.source.SplitEnumeratorContext<KafkaPartitionSplit> context, org.apache.flink.api.connector.source.Boundedness boundedness, Set<org.apache.kafka.common.TopicPartition> assignedPartitions)
-
-
Method Detail
-
start
public void start()
Start the enumerator.Depending on
partitionDiscoveryIntervalMs, the enumerator will trigger a one-time partition discovery, or schedule a callable for discover partitions periodically.The invoking chain of partition discovery would be:
getSubscribedTopicPartitions()in worker threadcheckPartitionChanges(java.util.Set<org.apache.kafka.common.TopicPartition>, java.lang.Throwable)in coordinator threadinitializePartitionSplits(org.apache.flink.connector.kafka.source.enumerator.KafkaSourceEnumerator.PartitionChange)in worker threadhandlePartitionSplitChanges(org.apache.flink.connector.kafka.source.enumerator.KafkaSourceEnumerator.PartitionSplitChange, java.lang.Throwable)in coordinator thread
- Specified by:
startin interfaceorg.apache.flink.api.connector.source.SplitEnumerator<KafkaPartitionSplit,KafkaSourceEnumState>
-
handleSplitRequest
public void handleSplitRequest(int subtaskId, @Nullable String requesterHostname)- Specified by:
handleSplitRequestin interfaceorg.apache.flink.api.connector.source.SplitEnumerator<KafkaPartitionSplit,KafkaSourceEnumState>
-
addSplitsBack
public void addSplitsBack(List<KafkaPartitionSplit> splits, int subtaskId)
- Specified by:
addSplitsBackin interfaceorg.apache.flink.api.connector.source.SplitEnumerator<KafkaPartitionSplit,KafkaSourceEnumState>
-
addReader
public void addReader(int subtaskId)
- Specified by:
addReaderin interfaceorg.apache.flink.api.connector.source.SplitEnumerator<KafkaPartitionSplit,KafkaSourceEnumState>
-
snapshotState
public KafkaSourceEnumState snapshotState(long checkpointId) throws Exception
- Specified by:
snapshotStatein interfaceorg.apache.flink.api.connector.source.SplitEnumerator<KafkaPartitionSplit,KafkaSourceEnumState>- Throws:
Exception
-
close
public void close()
- Specified by:
closein interfaceAutoCloseable- Specified by:
closein interfaceorg.apache.flink.api.connector.source.SplitEnumerator<KafkaPartitionSplit,KafkaSourceEnumState>
-
-