software.amazon.kinesis.connectors.flink.internals.publisher.polling

Class AdaptivePollingRecordPublisher

java.lang.Object

software.amazon.kinesis.connectors.flink.internals.publisher.polling.PollingRecordPublisher

software.amazon.kinesis.connectors.flink.internals.publisher.polling.AdaptivePollingRecordPublisher

All Implemented Interfaces:

RecordPublisher

@Internal
public class AdaptivePollingRecordPublisher
extends PollingRecordPublisher

An adaptive record publisher to add a dynamic batch read size for PollingRecordPublisher. Kinesis Streams have quotas on the transactions per second, and throughout. This class attempts to balance quotas and mitigate back off errors.

Nested Class Summary

Nested classes/interfaces inherited from interface software.amazon.kinesis.connectors.flink.internals.publisher.RecordPublisher

RecordPublisher.RecordBatchConsumer, RecordPublisher.RecordPublisherRunResult

Method Summary

Modifier and Type	Method and Description
RecordPublisher.RecordPublisherRunResult	run(RecordPublisher.RecordBatchConsumer consumer) Run the record publisher.

Methods inherited from class software.amazon.kinesis.connectors.flink.internals.publisher.polling.PollingRecordPublisher

run

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

run

public RecordPublisher.RecordPublisherRunResult run(RecordPublisher.RecordBatchConsumer consumer)
                                             throws InterruptedException

Description copied from interface: RecordPublisher

Run the record publisher. Records will be consumed from the stream and published to the consumer. The number of batches retrieved by a single invocation will vary based on implementation.

Specified by:

run in interface RecordPublisher

Overrides:

run in class PollingRecordPublisher

Parameters:

consumer - the record batch consumer in which to output records

Returns:

a status enum to represent whether a shard has been fully consumed

Throws:

InterruptedException