Package org.apache.flink.streaming.api.functions.sink.filesystem.bucketassigners

Class DateTimeBucketAssigner<IN>

  • All Implemented Interfaces:
    Serializable, BucketAssigner<IN,​String>
    @PublicEvolving
public class DateTimeBucketAssigner<IN>
extends Object
implements BucketAssigner<IN,​String>
    A BucketAssigner that assigns to buckets based on current system time.

    The DateTimeBucketer will create directories of the following form: /{basePath}/{dateTimePath}/. The basePath is the path that was specified as a base path when creating the new bucket. The dateTimePath is determined based on the current system time and the user provided format string.

    DateTimeFormatter is used to derive a date string from the current system time and the date format string. The default format string is "yyyy-MM-dd--HH" so the rolling files will have a granularity of hours.

    Example: 

    
 BucketAssigner bucketAssigner = new DateTimeBucketAssigner("yyyy-MM-dd--HH");

    This will create for example the following bucket path: /base/1976-12-31-14/

    See Also:
    Serialized Form

    • Constructor Detail

      • DateTimeBucketAssigner

        public DateTimeBucketAssigner()
        Creates a new DateTimeBucketAssigner with format string "yyyy-MM-dd--HH".

      • DateTimeBucketAssigner

        public DateTimeBucketAssigner​(String formatString)
        Creates a new DateTimeBucketAssigner with the given date/time format string.
        Parameters:
        formatString - The format string that will be given to SimpleDateFormat to determine the bucket id.

      • DateTimeBucketAssigner

        public DateTimeBucketAssigner​(ZoneId zoneId)
        Creates a new DateTimeBucketAssigner with format string "yyyy-MM-dd--HH" using the given timezone.
        Parameters:
        zoneId - The timezone used to format DateTimeFormatter for bucket id.

      • DateTimeBucketAssigner

        public DateTimeBucketAssigner​(String formatString,
                              ZoneId zoneId)
        Creates a new DateTimeBucketAssigner with the given date/time format string using the given timezone.
        Parameters:
        formatString - The format string that will be given to DateTimeFormatter to determine the bucket path.
        zoneId - The timezone used to format DateTimeFormatter for bucket id.

    • Method Detail

      • getBucketId

        public String getBucketId​(IN element,
                          BucketAssigner.Context context)
        Description copied from interface: BucketAssigner
        Returns the identifier of the bucket the provided element should be put into.
        Specified by:
        getBucketId in interface BucketAssigner<IN,​String>
        Parameters:
        element - The current element being processed.
        context - The context used by the current bucket assigner.
        Returns:
        A string representing the identifier of the bucket the element should be put into. The actual path to the bucket will result from the concatenation of the returned string and the base path provided during the initialization of the file sink.