software.amazon.kinesis.connectors.flink.internals

Class DynamoDBStreamsDataFetcher<T>

java.lang.Object

software.amazon.kinesis.connectors.flink.internals.KinesisDataFetcher<T>

software.amazon.kinesis.connectors.flink.internals.DynamoDBStreamsDataFetcher<T>

Type Parameters:

T - type of fetched data.

public class DynamoDBStreamsDataFetcher<T>
extends KinesisDataFetcher<T>

Dynamodb streams data fetcher.

Nested Class Summary

Nested classes/interfaces inherited from class software.amazon.kinesis.connectors.flink.internals.KinesisDataFetcher

KinesisDataFetcher.FlinkKinesisProxyFactory, KinesisDataFetcher.FlinkKinesisProxyV2Factory

Field Summary

Fields inherited from class software.amazon.kinesis.connectors.flink.internals.KinesisDataFetcher

DEFAULT_SHARD_ASSIGNER

Constructor Summary

Constructors

Constructor and Description
DynamoDBStreamsDataFetcher(List<String> streams, org.apache.flink.streaming.api.functions.source.SourceFunction.SourceContext<T> sourceContext, org.apache.flink.api.common.functions.RuntimeContext runtimeContext, Properties configProps, KinesisDeserializationSchema<T> deserializationSchema, KinesisShardAssigner shardAssigner) Constructor.

Method Summary

Modifier and Type	Method and Description
protected RecordPublisher	createRecordPublisher(SequenceNumber sequenceNumber, Properties configProps, org.apache.flink.metrics.MetricGroup metricGroup, StreamShardHandle subscribedShard)
protected boolean	shouldAdvanceLastDiscoveredShardId(String shardId, String lastSeenShardIdOfStream) Given lastSeenShardId, check if last discovered shardId should be advanced.

Methods inherited from class software.amazon.kinesis.connectors.flink.internals.KinesisDataFetcher

advanceLastDiscoveredShardOfStream, awaitTermination, closeRecordPublisherFactory, convertToStreamShardHandle, convertToStreamShardMetadata, createInitialSubscribedStreamsToLastDiscoveredShardsState, createRecordPublisherFactory, createShardConsumer, createShardConsumersThreadPool, deregisterStreamConsumer, discoverNewShardsToSubscribe, emitRecordAndUpdateState, emitWatermark, getConsumerConfiguration, getCurrentTimeMillis, getSubscribedShardsState, isThisSubtaskShouldSubscribeTo, registerNewSubscribedShardState, runFetcher, shutdownFetcher, snapshotState, stopWithError, updateState

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DynamoDBStreamsDataFetcher

public DynamoDBStreamsDataFetcher(List<String> streams,
                                  org.apache.flink.streaming.api.functions.source.SourceFunction.SourceContext<T> sourceContext,
                                  org.apache.flink.api.common.functions.RuntimeContext runtimeContext,
                                  Properties configProps,
                                  KinesisDeserializationSchema<T> deserializationSchema,
                                  KinesisShardAssigner shardAssigner)

Constructor.

Parameters:

streams - list of streams to fetch data

sourceContext - source context

runtimeContext - runtime context

configProps - config properties

deserializationSchema - deserialization schema

shardAssigner - shard assigner

Method Detail

shouldAdvanceLastDiscoveredShardId

protected boolean shouldAdvanceLastDiscoveredShardId(String shardId,
                                                     String lastSeenShardIdOfStream)

Description copied from class: KinesisDataFetcher

Given lastSeenShardId, check if last discovered shardId should be advanced.

Overrides:

shouldAdvanceLastDiscoveredShardId in class KinesisDataFetcher<T>

createRecordPublisher

protected RecordPublisher createRecordPublisher(SequenceNumber sequenceNumber,
                                                Properties configProps,
                                                org.apache.flink.metrics.MetricGroup metricGroup,
                                                StreamShardHandle subscribedShard)
                                         throws InterruptedException

Overrides:

createRecordPublisher in class KinesisDataFetcher<T>

Throws:

InterruptedException