| Modifier and Type | Method and Description |
|---|---|
HashMap<StreamShardMetadata,SequenceNumber> |
KinesisDataFetcher.snapshotState()
Creates a snapshot of the current last processed sequence numbers of each subscribed shard.
|
| Modifier and Type | Method and Description |
|---|---|
protected RecordPublisher |
DynamoDBStreamsDataFetcher.createRecordPublisher(SequenceNumber sequenceNumber,
Properties configProps,
org.apache.flink.metrics.MetricGroup metricGroup,
StreamShardHandle subscribedShard) |
protected RecordPublisher |
KinesisDataFetcher.createRecordPublisher(SequenceNumber sequenceNumber,
Properties configProps,
org.apache.flink.metrics.MetricGroup metricGroup,
StreamShardHandle subscribedShard) |
protected ShardConsumer<T> |
KinesisDataFetcher.createShardConsumer(Integer subscribedShardStateIndex,
StreamShardHandle subscribedShard,
SequenceNumber lastSequenceNum,
org.apache.flink.metrics.MetricGroup metricGroup,
KinesisDeserializationSchema<T> shardDeserializer)
Create a new shard consumer.
|
protected void |
KinesisDataFetcher.emitRecordAndUpdateState(T record,
long recordTimestamp,
int shardStateIndex,
SequenceNumber lastSequenceNumber)
Prepare a record and hand it over to the
RecordEmitter, which may collect it asynchronously. |
protected void |
KinesisDataFetcher.updateState(int shardStateIndex,
SequenceNumber lastSequenceNumber)
Update the shard to last processed sequence number state.
|
| Constructor and Description |
|---|
ShardConsumer(KinesisDataFetcher<T> fetcherRef,
RecordPublisher recordPublisher,
Integer subscribedShardStateIndex,
StreamShardHandle subscribedShard,
SequenceNumber lastSequenceNum,
ShardConsumerMetricsReporter shardConsumerMetricsReporter,
KinesisDeserializationSchema<T> shardDeserializer)
Creates a shard consumer.
|
| Modifier and Type | Method and Description |
|---|---|
SequenceNumber |
RecordPublisher.RecordBatchConsumer.accept(RecordBatch recordBatch) |
| Modifier and Type | Method and Description |
|---|---|
SequenceNumber |
SentinelSequenceNumber.get() |
SequenceNumber |
KinesisStreamShardState.getLastProcessedSequenceNum() |
| Modifier and Type | Method and Description |
|---|---|
static StartingPosition |
StartingPosition.continueFromSequenceNumber(SequenceNumber sequenceNumber)
Returns the starting position for the next record to consume from the given sequence number.
|
static boolean |
SentinelSequenceNumber.isSentinelSequenceNumber(SequenceNumber candidateSequenceNumber)
Returns
true if the given SequenceNumber is a sentinel. |
static StartingPosition |
StartingPosition.restartFromSequenceNumber(SequenceNumber sequenceNumber)
Returns the starting position to restart record consumption from the given sequence number after failure.
|
void |
KinesisStreamShardState.setLastProcessedSequenceNum(SequenceNumber update) |
| Constructor and Description |
|---|
KinesisStreamShardState(StreamShardMetadata streamShardMetadata,
StreamShardHandle streamShardHandle,
SequenceNumber lastProcessedSequenceNum) |
| Modifier and Type | Method and Description |
|---|---|
static org.apache.flink.api.java.typeutils.runtime.TupleSerializer<org.apache.flink.api.java.tuple.Tuple2<StreamShardMetadata,SequenceNumber>> |
KinesisStateUtil.createShardsStateSerializer(org.apache.flink.api.common.ExecutionConfig executionConfig)
Creates state serializer for kinesis shard sequence number.
|
| Modifier and Type | Method and Description |
|---|---|
static StartingPosition |
AWSUtil.getStartingPosition(SequenceNumber sequenceNumber,
Properties configProps)
|
Copyright © 2014–2022 The Apache Software Foundation. All rights reserved.