Package org.apache.flink.streaming.runtime.tasks.mailbox

Class MailboxProcessor

  • All Implemented Interfaces:
    Closeable, AutoCloseable
    @Internal
public class MailboxProcessor
extends Object
implements Closeable
    This class encapsulates the logic of the mailbox-based execution model. At the core of this model runMailboxLoop() that continuously executes the provided MailboxDefaultAction in a loop. On each iteration, the method also checks if there are pending actions in the mailbox and executes such actions. This model ensures single-threaded execution between the default action (e.g. record processing) and mailbox actions (e.g. checkpoint trigger, timer firing, ...).

    The MailboxDefaultAction interacts with this class through the MailboxProcessor.MailboxController to communicate control flow changes to the mailbox loop, e.g. that invocations of the default action are temporarily or permanently exhausted.

    The design of runMailboxLoop() is centered around the idea of keeping the expected hot path (default action, no mail) as fast as possible. This means that all checking of mail and other control flags (mailboxLoopRunning, suspendedDefaultAction) are always connected to #hasMail indicating true. This means that control flag changes in the mailbox thread can be done directly, but we must ensure that there is at least one action in the mailbox so that the change is picked up. For control flag changes by all other threads, that must happen through mailbox actions, this is automatically the case.

    This class has an open-prepareClose-close lifecycle that is connected with and maps to the lifecycle of the encapsulated TaskMailbox (which is open-quiesce-close).

    • Field Detail

      • mailbox

        protected final TaskMailbox mailbox
        The mailbox data-structure that manages request for special actions, like timers, checkpoints, ...

      • mailboxDefaultAction

        protected final MailboxDefaultAction mailboxDefaultAction
        Action that is repeatedly executed if no action request is in the mailbox. Typically record processing.

    • Method Detail

      • getMainMailboxExecutor

        public org.apache.flink.api.common.operators.MailboxExecutor getMainMailboxExecutor()

      • getMailboxExecutor

        public org.apache.flink.api.common.operators.MailboxExecutor getMailboxExecutor​(int priority)
        Returns an executor service facade to submit actions to the mailbox.
        Parameters:
        priority - the priority of the MailboxExecutor.

      • prepareClose

        public void prepareClose()
        Lifecycle method to close the mailbox for action submission.

      • close

        public void close()
        Lifecycle method to close the mailbox for action submission/retrieval. This will cancel all instances of RunnableFuture that are still contained in the mailbox.
        Specified by:
        close in interface AutoCloseable
        Specified by:
        close in interface Closeable

      • drain

        public void drain()
           throws Exception
        Finishes running all mails in the mailbox. If no concurrent write operations occurred, the mailbox must be empty after this method.
        Throws:
        Exception

      • runMailboxLoop

        public void runMailboxLoop()
                    throws Exception
        Runs the mailbox processing loop. This is where the main work is done. This loop can be suspended at any time by calling suspend(). For resuming the loop this method should be called again.
        Throws:
        Exception

      • suspend

        public void suspend()
        Suspend the running of the loop which was started by runMailboxLoop()}.

      • runSingleMailboxLoop

        @VisibleForTesting
public boolean runSingleMailboxLoop()
                             throws Exception
        Execute a single (as small as possible) step of the mailbox.
        Returns:
        true if something was processed.
        Throws:
        Exception

      • runMailboxStep

        @VisibleForTesting
public boolean runMailboxStep()
                       throws Exception
        Execute a single (as small as possible) step of the mailbox.
        Returns:
        true if something was processed.
        Throws:
        Exception

      • isMailboxThread

        public boolean isMailboxThread()
        Check if the current thread is the mailbox thread.
        Returns:
        only true if called from the mailbox thread.

      • reportThrowable

        public void reportThrowable​(Throwable throwable)
        Reports a throwable for rethrowing from the mailbox thread. This will clear and cancel all other pending mails.
        Parameters:
        throwable - to report by rethrowing from the mailbox loop.

      • allActionsCompleted

        public void allActionsCompleted()
        This method must be called to end the stream task when all actions for the tasks have been performed.

      • isDefaultActionAvailable

        @VisibleForTesting
public boolean isDefaultActionAvailable()

      • isMailboxLoopRunning

        @VisibleForTesting
public boolean isMailboxLoopRunning()

      • hasMail

        public boolean hasMail()