org.apache.flink.streaming.api.operators.source

Class ProgressiveTimestampsAndWatermarks<T>

java.lang.Object

org.apache.flink.streaming.api.operators.source.ProgressiveTimestampsAndWatermarks<T>

Type Parameters:

T - The type of the emitted records.

All Implemented Interfaces:

TimestampsAndWatermarks<T>

@Internal
public class ProgressiveTimestampsAndWatermarks<T>
extends Object
implements TimestampsAndWatermarks<T>

An implementation of TimestampsAndWatermarks that does periodic watermark emission and keeps track of watermarks on a per-split basis. This should be used in execution contexts where watermarks are important for efficiency/correctness, for example in STREAMING execution mode.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.streaming.api.operators.source.TimestampsAndWatermarks

TimestampsAndWatermarks.TimestampsAndWatermarksContextProvider, TimestampsAndWatermarks.WatermarkUpdateListener

Constructor Summary

Constructors

Constructor and Description

ProgressiveTimestampsAndWatermarks(org.apache.flink.api.common.eventtime.TimestampAssigner<T> timestampAssigner, org.apache.flink.api.common.eventtime.WatermarkGeneratorSupplier<T> watermarksFactory, TimestampsAndWatermarks.TimestampsAndWatermarksContextProvider watermarksContextProvider, ProcessingTimeService timeService, Duration periodicWatermarkInterval, org.apache.flink.util.clock.RelativeClock mainInputActivityClock, org.apache.flink.util.clock.Clock clock, org.apache.flink.runtime.metrics.groups.TaskIOMetricGroup taskIOMetricGroup)

Method Summary

Modifier and Type	Method and Description
org.apache.flink.api.connector.source.ReaderOutput<T>	createMainOutput(PushingAsyncDataInput.DataOutput<T> output, TimestampsAndWatermarks.WatermarkUpdateListener watermarkUpdateListener) Creates the ReaderOutput for the source reader, than internally runs the timestamp extraction and watermark generation.
void	emitImmediateWatermark(long wallClockTimestamp) Emit a watermark immediately.
void	pauseOrResumeSplits(Collection<String> splitsToPause, Collection<String> splitsToResume)
void	startPeriodicWatermarkEmits() Starts emitting periodic watermarks, if this implementation produces watermarks, and if periodic watermarks are configured.
void	stopPeriodicWatermarkEmits() Stops emitting periodic watermarks.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.streaming.api.operators.source.TimestampsAndWatermarks

createNoOpEventTimeLogic, createProgressiveEventTimeLogic

Constructor Detail

ProgressiveTimestampsAndWatermarks

public ProgressiveTimestampsAndWatermarks(org.apache.flink.api.common.eventtime.TimestampAssigner<T> timestampAssigner,
                                          org.apache.flink.api.common.eventtime.WatermarkGeneratorSupplier<T> watermarksFactory,
                                          TimestampsAndWatermarks.TimestampsAndWatermarksContextProvider watermarksContextProvider,
                                          ProcessingTimeService timeService,
                                          Duration periodicWatermarkInterval,
                                          org.apache.flink.util.clock.RelativeClock mainInputActivityClock,
                                          org.apache.flink.util.clock.Clock clock,
                                          org.apache.flink.runtime.metrics.groups.TaskIOMetricGroup taskIOMetricGroup)

Method Detail

createMainOutput

public org.apache.flink.api.connector.source.ReaderOutput<T> createMainOutput(PushingAsyncDataInput.DataOutput<T> output,
                                                                              TimestampsAndWatermarks.WatermarkUpdateListener watermarkUpdateListener)

Description copied from interface: TimestampsAndWatermarks

Creates the ReaderOutput for the source reader, than internally runs the timestamp extraction and watermark generation.

Specified by:

createMainOutput in interface TimestampsAndWatermarks<T>

startPeriodicWatermarkEmits

public void startPeriodicWatermarkEmits()

Description copied from interface: TimestampsAndWatermarks

Starts emitting periodic watermarks, if this implementation produces watermarks, and if periodic watermarks are configured.

Periodic watermarks are produced by periodically calling the WatermarkGenerator.onPeriodicEmit(WatermarkOutput) method of the underlying Watermark Generators.

Specified by:

startPeriodicWatermarkEmits in interface TimestampsAndWatermarks<T>

stopPeriodicWatermarkEmits

public void stopPeriodicWatermarkEmits()

Description copied from interface: TimestampsAndWatermarks

Stops emitting periodic watermarks.

Specified by:

stopPeriodicWatermarkEmits in interface TimestampsAndWatermarks<T>

emitImmediateWatermark

public void emitImmediateWatermark(long wallClockTimestamp)

Description copied from interface: TimestampsAndWatermarks

Emit a watermark immediately.

Specified by:

emitImmediateWatermark in interface TimestampsAndWatermarks<T>

pauseOrResumeSplits

public void pauseOrResumeSplits(Collection<String> splitsToPause,
                                Collection<String> splitsToResume)

Specified by:

pauseOrResumeSplits in interface TimestampsAndWatermarks<T>