Package org.apache.flink.streaming.runtime.operators.sink.committables

Interface CheckpointCommittableManager<CommT>

  • Type Parameters:
    CommT - type of the committable
    @Internal
public interface CheckpointCommittableManager<CommT>
    A CheckpointCommittableManager collects committables for one checkpoint across potentially multiple upstream subtasks.

    While it collects committables from multiple upstream subtasks, it belongs to exactly one committer subtask.

    Each upstream subtask of this particular checkpoint is represented by a SubtaskCommittableManager.

    • Method Summary

      All Methods Instance Methods Abstract Methods 
      Modifier and Type Method Description
      void commit​(org.apache.flink.api.connector.sink2.Committer<CommT> committer, int maxRetries)
      Commits all due committables if all respective committables of the specific subtask and checkpoint have been received.
      long getCheckpointId()
      Returns the checkpoint id in which the committables were created.
      int getNumberOfSubtasks()
      Returns the number of upstream subtasks belonging to the checkpoint.
      int getNumFailed()
      Returns the number of committables that have failed with a known error.
      Collection<CommT> getSuccessfulCommittables()
      Returns the number of committables that have been successfully committed; that is, the corresponding Committer.CommitRequest was not used to signal an error of any kind (retryable or not).
      boolean hasGloballyReceivedAll()
      Returns true if all committables of all upstream subtasks arrived, which is only guaranteed to happen if the DOP of the caller is 1.
      boolean isFinished()  

    • Method Detail

      • getCheckpointId

        long getCheckpointId()
        Returns the checkpoint id in which the committables were created.
        Returns:
        checkpoint id

      • getNumberOfSubtasks

        int getNumberOfSubtasks()
        Returns the number of upstream subtasks belonging to the checkpoint.

      • isFinished

        boolean isFinished()

      • hasGloballyReceivedAll

        boolean hasGloballyReceivedAll()
        Returns true if all committables of all upstream subtasks arrived, which is only guaranteed to happen if the DOP of the caller is 1.

      • commit

        void commit​(org.apache.flink.api.connector.sink2.Committer<CommT> committer,
            int maxRetries)
     throws IOException,
            InterruptedException
        Commits all due committables if all respective committables of the specific subtask and checkpoint have been received.
        Parameters:
        committer - used to commit to the external system
        maxRetries -
        Throws:
        IOException
        InterruptedException

      • getSuccessfulCommittables

        Collection<CommT> getSuccessfulCommittables()
        Returns the number of committables that have been successfully committed; that is, the corresponding Committer.CommitRequest was not used to signal an error of any kind (retryable or not).
        Returns:
        number of successful committables

      • getNumFailed

        int getNumFailed()
        Returns the number of committables that have failed with a known error. By the current semantics of Committer.CommitRequest.signalFailedWithKnownReason(Throwable) discards the committable but proceeds processing. The returned number should be emitted downstream in a CommittableSummary, such that downstream can assess if all committables have been processed.
        Returns:
        number of failed committables