org.apache.flink.streaming.api.functions.sink.filesystem.bucketassigners

Class DateTimeBucketAssigner<IN>

java.lang.Object

org.apache.flink.streaming.api.functions.sink.filesystem.bucketassigners.DateTimeBucketAssigner<IN>

All Implemented Interfaces:

Serializable, BucketAssigner<IN,String>

@PublicEvolving
public class DateTimeBucketAssigner<IN>
extends Object
implements BucketAssigner<IN,String>

A BucketAssigner that assigns to buckets based on current system time.

The DateTimeBucketer will create directories of the following form: /{basePath}/{dateTimePath}/. The basePath is the path that was specified as a base path when creating the StreamingFileSink. The dateTimePath is determined based on the current system time and the user provided format string.

SimpleDateFormat is used to derive a date string from the current system time and the date format string. The default format string is "yyyy-MM-dd--HH" so the rolling files will have a granularity of hours.

Example:

     Bucketer buck = new DateTimeBucketer("yyyy-MM-dd--HH");
 

This will create for example the following bucket path: /base/1976-12-31-14/

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.streaming.api.functions.sink.filesystem.BucketAssigner

BucketAssigner.Context

Constructor Summary

Constructors

Constructor and Description
DateTimeBucketAssigner() Creates a new DateTimeBucketer with format string "yyyy-MM-dd--HH".
DateTimeBucketAssigner(String formatString) Creates a new DateTimeBucketer with the given date/time format string.

Method Summary

Modifier and Type	Method and Description
String	getBucketId(IN element, BucketAssigner.Context context) Returns the identifier of the bucket the provided element should be put into.
org.apache.flink.core.io.SimpleVersionedSerializer<String>	getSerializer()
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

DateTimeBucketAssigner

public DateTimeBucketAssigner()

Creates a new DateTimeBucketer with format string "yyyy-MM-dd--HH".

DateTimeBucketAssigner

public DateTimeBucketAssigner(String formatString)

Creates a new DateTimeBucketer with the given date/time format string.

Parameters:

formatString - The format string that will be given to SimpleDateFormat to determine the bucket path.

Method Detail

getBucketId

public String getBucketId(IN element,
                          BucketAssigner.Context context)

Description copied from interface: BucketAssigner

Returns the identifier of the bucket the provided element should be put into.

Specified by:

getBucketId in interface BucketAssigner<IN,String>

Parameters:

element - The current element being processed.

context - The context used by the sink.

Returns:

A string representing the identifier of the bucket the element should be put into. The actual path to the bucket will result from the concatenation of the returned string and the base path provided during the initialization of the sink.

getSerializer

public org.apache.flink.core.io.SimpleVersionedSerializer<String> getSerializer()

Specified by:

getSerializer in interface BucketAssigner<IN,String>

Returns:

A SimpleVersionedSerializer capable of serializing/deserializing the elements of type BucketID. That is the type of the objects returned by the BucketAssigner.getBucketId(Object, BucketAssigner.Context).

toString

public String toString()

Overrides:

toString in class Object