A B C D G I M O P R S T W

A

AbstractPartFileWriter<IN,BucketID> - Class in org.apache.flink.streaming.api.functions.sink.filesystem

An abstract writer for the currently open part file in a specific Bucket.

AbstractPartFileWriter(BucketID, long) - Constructor for class org.apache.flink.streaming.api.functions.sink.filesystem.AbstractPartFileWriter

B

BasePathBucketAssigner<T> - Class in org.apache.flink.streaming.api.functions.sink.filesystem.bucketassigners

A BucketAssigner that does not perform any bucketing of files.

BasePathBucketAssigner() - Constructor for class org.apache.flink.streaming.api.functions.sink.filesystem.bucketassigners.BasePathBucketAssigner

BucketAssigner<IN,BucketID> - Interface in org.apache.flink.streaming.api.functions.sink.filesystem

A BucketAssigner is used with a file sink to determine the bucket each incoming element should be put into.

BucketAssigner.Context - Interface in org.apache.flink.streaming.api.functions.sink.filesystem

Context that the BucketAssigner can use for getting additional data about an input record.

BucketWriter<IN,BucketID> - Interface in org.apache.flink.streaming.api.functions.sink.filesystem

An interface for factories that create the different writers.

BucketWriter.PendingFile - Interface in org.apache.flink.streaming.api.functions.sink.filesystem

This represents the file that can not write any data to.

build() - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.OutputFileConfig.OutputFileConfigBuilder

build() - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies.CheckpointRollingPolicy.PolicyBuilder

build() - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies.DefaultRollingPolicy.PolicyBuilder

Creates the actual policy.

build() - Static method in class org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies.OnCheckpointRollingPolicy

builder() - Static method in class org.apache.flink.streaming.api.functions.sink.filesystem.OutputFileConfig

builder() - Static method in class org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies.DefaultRollingPolicy

Creates a new DefaultRollingPolicy.PolicyBuilder that is used to configure and build an instance of DefaultRollingPolicy.

BulkBucketWriter<IN,BucketID> - Class in org.apache.flink.streaming.api.functions.sink.filesystem

A factory that creates BulkPartWriters.

BulkBucketWriter(RecoverableWriter, BulkWriter.Factory<IN>) - Constructor for class org.apache.flink.streaming.api.functions.sink.filesystem.BulkBucketWriter

C

CheckpointRollingPolicy<IN,BucketID> - Class in org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies

An abstract RollingPolicy which rolls on every checkpoint.

CheckpointRollingPolicy() - Constructor for class org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies.CheckpointRollingPolicy

CheckpointRollingPolicy.PolicyBuilder<IN,BucketID,T extends CheckpointRollingPolicy.PolicyBuilder<IN,BucketID,T>> - Class in org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies

The base abstract builder class for CheckpointRollingPolicy.

cleanupInProgressFileRecoverable(InProgressFileWriter.InProgressFileRecoverable) - Method in interface org.apache.flink.streaming.api.functions.sink.filesystem.BucketWriter

Frees up any resources that were previously occupied in order to be able to recover from a (potential) failure.

closeForCommit() - Method in interface org.apache.flink.streaming.api.functions.sink.filesystem.InProgressFileWriter

closeForCommit() - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.OutputStreamBasedPartFileWriter

commit() - Method in interface org.apache.flink.streaming.api.functions.sink.filesystem.BucketWriter.PendingFile

Commits the pending file, making it visible.

commitAfterRecovery() - Method in interface org.apache.flink.streaming.api.functions.sink.filesystem.BucketWriter.PendingFile

Commits the pending file, making it visible.

create() - Static method in class org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies.DefaultRollingPolicy

Deprecated.

currentProcessingTime() - Method in interface org.apache.flink.streaming.api.functions.sink.filesystem.BucketAssigner.Context

Returns the current processing time.

currentWatermark() - Method in interface org.apache.flink.streaming.api.functions.sink.filesystem.BucketAssigner.Context

Returns the current event-time watermark.

D

DateTimeBucketAssigner<IN> - Class in org.apache.flink.streaming.api.functions.sink.filesystem.bucketassigners

A BucketAssigner that assigns to buckets based on current system time.

DateTimeBucketAssigner() - Constructor for class org.apache.flink.streaming.api.functions.sink.filesystem.bucketassigners.DateTimeBucketAssigner

Creates a new DateTimeBucketAssigner with format string "yyyy-MM-dd--HH".

DateTimeBucketAssigner(String) - Constructor for class org.apache.flink.streaming.api.functions.sink.filesystem.bucketassigners.DateTimeBucketAssigner

Creates a new DateTimeBucketAssigner with the given date/time format string.

DateTimeBucketAssigner(ZoneId) - Constructor for class org.apache.flink.streaming.api.functions.sink.filesystem.bucketassigners.DateTimeBucketAssigner

Creates a new DateTimeBucketAssigner with format string "yyyy-MM-dd--HH" using the given timezone.

DateTimeBucketAssigner(String, ZoneId) - Constructor for class org.apache.flink.streaming.api.functions.sink.filesystem.bucketassigners.DateTimeBucketAssigner

Creates a new DateTimeBucketAssigner with the given date/time format string using the given timezone.

DefaultRollingPolicy<IN,BucketID> - Class in org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies

The default implementation of the RollingPolicy.

DefaultRollingPolicy.PolicyBuilder - Class in org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies

A helper class that holds the configuration properties for the DefaultRollingPolicy.

deserialize(int, byte[]) - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.bucketassigners.SimpleVersionedStringSerializer

deserialize(int, byte[]) - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.OutputStreamBasedPartFileWriter.OutputStreamBasedInProgressFileRecoverableSerializer

deserialize(int, byte[]) - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.OutputStreamBasedPartFileWriter.OutputStreamBasedPendingFileRecoverableSerializer

dispose() - Method in interface org.apache.flink.streaming.api.functions.sink.filesystem.InProgressFileWriter

Dispose the part file.

dispose() - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.OutputStreamBasedPartFileWriter

G

getBucketId() - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.AbstractPartFileWriter

getBucketId(IN, BucketAssigner.Context) - Method in interface org.apache.flink.streaming.api.functions.sink.filesystem.BucketAssigner

Returns the identifier of the bucket the provided element should be put into.

getBucketId(T, BucketAssigner.Context) - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.bucketassigners.BasePathBucketAssigner

getBucketId(IN, BucketAssigner.Context) - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.bucketassigners.DateTimeBucketAssigner

getBucketId() - Method in interface org.apache.flink.streaming.api.functions.sink.filesystem.PartFileInfo

getCommitSerializer() - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.OutputStreamBasedPartFileWriter.OutputStreamBasedPendingFileRecoverableSerializer

getCreationTime() - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.AbstractPartFileWriter

getCreationTime() - Method in interface org.apache.flink.streaming.api.functions.sink.filesystem.PartFileInfo

getInactivityInterval() - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies.DefaultRollingPolicy

Returns time duration of allowed inactivity after which a part file will have to roll.

getInProgressFileRecoverableSerializer() - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.WriterProperties

getLastUpdateTime() - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.AbstractPartFileWriter

getLastUpdateTime() - Method in interface org.apache.flink.streaming.api.functions.sink.filesystem.PartFileInfo

getMaxPartSize() - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies.DefaultRollingPolicy

Returns the maximum part file size before rolling.

getPartPrefix() - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.OutputFileConfig

The prefix for the part name.

getPartSuffix() - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.OutputFileConfig

The suffix for the part name.

getPendingFileRecoverableSerializer() - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.WriterProperties

getProperties() - Method in interface org.apache.flink.streaming.api.functions.sink.filesystem.BucketWriter

getResumeSerializer() - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.OutputStreamBasedPartFileWriter.OutputStreamBasedInProgressFileRecoverableSerializer

getRolloverInterval() - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies.DefaultRollingPolicy

Returns the maximum time duration a part file can stay open before rolling.

getSerializer() - Method in interface org.apache.flink.streaming.api.functions.sink.filesystem.BucketAssigner

getSerializer() - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.bucketassigners.BasePathBucketAssigner

getSerializer() - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.bucketassigners.DateTimeBucketAssigner

getSize() - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.OutputStreamBasedPartFileWriter

getSize() - Method in interface org.apache.flink.streaming.api.functions.sink.filesystem.PartFileInfo

getVersion() - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.bucketassigners.SimpleVersionedStringSerializer

getVersion() - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.OutputStreamBasedPartFileWriter.OutputStreamBasedInProgressFileRecoverableSerializer

getVersion() - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.OutputStreamBasedPartFileWriter.OutputStreamBasedPendingFileRecoverableSerializer

I

InProgressFileWriter<IN,BucketID> - Interface in org.apache.flink.streaming.api.functions.sink.filesystem

The Bucket uses the InProgressFileWriter to write element to a part file.

InProgressFileWriter.InProgressFileRecoverable - Interface in org.apache.flink.streaming.api.functions.sink.filesystem

A handle can be used to recover in-progress file..

InProgressFileWriter.PendingFileRecoverable - Interface in org.apache.flink.streaming.api.functions.sink.filesystem

The handle can be used to recover pending file.

INSTANCE - Static variable in class org.apache.flink.streaming.api.functions.sink.filesystem.bucketassigners.SimpleVersionedStringSerializer

M

markWrite(long) - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.AbstractPartFileWriter

O

OnCheckpointRollingPolicy<IN,BucketID> - Class in org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies

A RollingPolicy which rolls (ONLY) on every checkpoint.

openNew(BucketID, RecoverableFsDataOutputStream, Path, long) - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.BulkBucketWriter

openNew(BucketID, RecoverableFsDataOutputStream, Path, long) - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.RowWiseBucketWriter

openNewInProgressFile(BucketID, Path, long) - Method in interface org.apache.flink.streaming.api.functions.sink.filesystem.BucketWriter

Used to create a new InProgressFileWriter.

org.apache.flink.streaming.api.functions.sink.filesystem - package org.apache.flink.streaming.api.functions.sink.filesystem

org.apache.flink.streaming.api.functions.sink.filesystem.bucketassigners - package org.apache.flink.streaming.api.functions.sink.filesystem.bucketassigners

org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies - package org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies

OutputFileConfig - Class in org.apache.flink.streaming.api.functions.sink.filesystem

Part file name configuration.

OutputFileConfig(String, String) - Constructor for class org.apache.flink.streaming.api.functions.sink.filesystem.OutputFileConfig

Initiates the PartFileConfig with values passed as parameters.

OutputFileConfig.OutputFileConfigBuilder - Class in org.apache.flink.streaming.api.functions.sink.filesystem

A builder to create the part file configuration.

OutputStreamBasedInProgressFileRecoverable(RecoverableWriter.ResumeRecoverable) - Constructor for class org.apache.flink.streaming.api.functions.sink.filesystem.OutputStreamBasedPartFileWriter.OutputStreamBasedInProgressFileRecoverable

OutputStreamBasedPartFileWriter<IN,BucketID> - Class in org.apache.flink.streaming.api.functions.sink.filesystem

The base class for all the part file writer that use RecoverableFsDataOutputStream.

OutputStreamBasedPartFileWriter.OutputStreamBasedInProgressFileRecoverable - Class in org.apache.flink.streaming.api.functions.sink.filesystem

The InProgressFileRecoverable implementation for OutputStreamBasedBucketWriter.

OutputStreamBasedPartFileWriter.OutputStreamBasedInProgressFileRecoverableSerializer - Class in org.apache.flink.streaming.api.functions.sink.filesystem

The serializer for OutputStreamBasedPartFileWriter.OutputStreamBasedInProgressFileRecoverable.

OutputStreamBasedPartFileWriter.OutputStreamBasedPendingFileRecoverable - Class in org.apache.flink.streaming.api.functions.sink.filesystem

The PendingFileRecoverable implementation for OutputStreamBasedBucketWriter.

OutputStreamBasedPartFileWriter.OutputStreamBasedPendingFileRecoverableSerializer - Class in org.apache.flink.streaming.api.functions.sink.filesystem

The serializer for OutputStreamBasedPartFileWriter.OutputStreamBasedPendingFileRecoverable.

OutputStreamBasedPendingFileRecoverable(RecoverableWriter.CommitRecoverable) - Constructor for class org.apache.flink.streaming.api.functions.sink.filesystem.OutputStreamBasedPartFileWriter.OutputStreamBasedPendingFileRecoverable

P

PartFileInfo<BucketID> - Interface in org.apache.flink.streaming.api.functions.sink.filesystem

An interface exposing the information concerning the current (open) part file that is necessary to the RollingPolicy in order to determine if it should roll the part file or not.

persist() - Method in interface org.apache.flink.streaming.api.functions.sink.filesystem.InProgressFileWriter

persist() - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.OutputStreamBasedPartFileWriter

PolicyBuilder() - Constructor for class org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies.CheckpointRollingPolicy.PolicyBuilder

R

recoverPendingFile(InProgressFileWriter.PendingFileRecoverable) - Method in interface org.apache.flink.streaming.api.functions.sink.filesystem.BucketWriter

Recovers a pending file for finalizing and committing.

resumeFrom(BucketID, RecoverableFsDataOutputStream, RecoverableWriter.ResumeRecoverable, long) - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.BulkBucketWriter

resumeFrom(BucketID, RecoverableFsDataOutputStream, RecoverableWriter.ResumeRecoverable, long) - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.RowWiseBucketWriter

resumeInProgressFileFrom(BucketID, InProgressFileWriter.InProgressFileRecoverable, long) - Method in interface org.apache.flink.streaming.api.functions.sink.filesystem.BucketWriter

Used to resume a InProgressFileWriter from a InProgressFileWriter.InProgressFileRecoverable.

RollingPolicy<IN,BucketID> - Interface in org.apache.flink.streaming.api.functions.sink.filesystem

The policy based on which a Bucket in the Filesystem Sink rolls its currently open part file and opens a new one.

RowWiseBucketWriter<IN,BucketID> - Class in org.apache.flink.streaming.api.functions.sink.filesystem

A factory that creates RowWisePartWriters.

RowWiseBucketWriter(RecoverableWriter, Encoder<IN>) - Constructor for class org.apache.flink.streaming.api.functions.sink.filesystem.RowWiseBucketWriter

S

self() - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies.CheckpointRollingPolicy.PolicyBuilder

serialize(String) - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.bucketassigners.SimpleVersionedStringSerializer

serialize(InProgressFileWriter.InProgressFileRecoverable) - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.OutputStreamBasedPartFileWriter.OutputStreamBasedInProgressFileRecoverableSerializer

serialize(InProgressFileWriter.PendingFileRecoverable) - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.OutputStreamBasedPartFileWriter.OutputStreamBasedPendingFileRecoverableSerializer

shouldRollOnCheckpoint(PartFileInfo<BucketID>) - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies.CheckpointRollingPolicy

shouldRollOnCheckpoint(PartFileInfo<BucketID>) - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies.DefaultRollingPolicy

shouldRollOnCheckpoint(PartFileInfo<BucketID>) - Method in interface org.apache.flink.streaming.api.functions.sink.filesystem.RollingPolicy

Determines if the in-progress part file for a bucket should roll on every checkpoint.

shouldRollOnEvent(PartFileInfo<BucketID>, IN) - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies.CheckpointRollingPolicy

shouldRollOnEvent(PartFileInfo<BucketID>, IN) - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies.DefaultRollingPolicy

shouldRollOnEvent(PartFileInfo<BucketID>, IN) - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies.OnCheckpointRollingPolicy

shouldRollOnEvent(PartFileInfo<BucketID>, IN) - Method in interface org.apache.flink.streaming.api.functions.sink.filesystem.RollingPolicy

Determines if the in-progress part file for a bucket should roll based on its current state, e.g.

shouldRollOnProcessingTime(PartFileInfo<BucketID>, long) - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies.CheckpointRollingPolicy

shouldRollOnProcessingTime(PartFileInfo<BucketID>, long) - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies.DefaultRollingPolicy

shouldRollOnProcessingTime(PartFileInfo<BucketID>, long) - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies.OnCheckpointRollingPolicy

shouldRollOnProcessingTime(PartFileInfo<BucketID>, long) - Method in interface org.apache.flink.streaming.api.functions.sink.filesystem.RollingPolicy

Determines if the in-progress part file for a bucket should roll based on a time condition.

SimpleVersionedStringSerializer - Class in org.apache.flink.streaming.api.functions.sink.filesystem.bucketassigners

A SimpleVersionedSerializer implementation for Strings.

supportsResume() - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.WriterProperties

T

timestamp() - Method in interface org.apache.flink.streaming.api.functions.sink.filesystem.BucketAssigner.Context

Returns the timestamp of the current input record or null if the element does not have an assigned timestamp.

toString() - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.bucketassigners.BasePathBucketAssigner

toString() - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.bucketassigners.DateTimeBucketAssigner

W

withInactivityInterval(long) - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies.DefaultRollingPolicy.PolicyBuilder

Sets the interval of allowed inactivity after which a part file will have to roll.

withMaxPartSize(long) - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies.DefaultRollingPolicy.PolicyBuilder

Sets the part size above which a part file will have to roll.

withPartPrefix(String) - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.OutputFileConfig.OutputFileConfigBuilder

withPartSuffix(String) - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.OutputFileConfig.OutputFileConfigBuilder

withRolloverInterval(long) - Method in class org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies.DefaultRollingPolicy.PolicyBuilder

Sets the max time a part file can stay open before having to roll.

write(IN, long) - Method in interface org.apache.flink.streaming.api.functions.sink.filesystem.InProgressFileWriter

Write a element to the part file.

WriterProperties - Class in org.apache.flink.streaming.api.functions.sink.filesystem

This class describes the property of the BucketWriter.

WriterProperties(SimpleVersionedSerializer<InProgressFileWriter.InProgressFileRecoverable>, SimpleVersionedSerializer<InProgressFileWriter.PendingFileRecoverable>, boolean) - Constructor for class org.apache.flink.streaming.api.functions.sink.filesystem.WriterProperties

A B C D G I M O P R S T W