org.apache.flink.runtime.jobmaster.event

Class FileSystemJobEventStore

java.lang.Object

org.apache.flink.runtime.jobmaster.event.FileSystemJobEventStore

All Implemented Interfaces:

JobEventStore

public class FileSystemJobEventStore
extends Object
implements JobEventStore

Implementation of JobEventStore that stores all JobEvent instances in a FileSystem. Events are written and read sequentially, ensuring a consistent order of events.

Write operations to the file system are primarily asynchronous, leveraging a ScheduledExecutorService to periodically flush the buffered output stream, which executes write tasks at an interval determined by the configure option JobEventStoreOptions.FLUSH_INTERVAL.

Read operations are performed synchronously, with the calling thread directly interacting with the file system to fetch event data.

Constructor Summary

Constructors

Constructor and Description
FileSystemJobEventStore(org.apache.flink.api.common.JobID jobID, org.apache.flink.configuration.Configuration configuration)
FileSystemJobEventStore(org.apache.flink.core.fs.Path workingDir, org.apache.flink.configuration.Configuration configuration)

Method Summary

Modifier and Type	Method and Description
protected ScheduledExecutorService	createJobEventWriterExecutor()
boolean	isEmpty() Returns whether the store is empty.
JobEvent	readEvent() Read a job event.
void	start() Start the store.
void	stop(boolean clearEventLogs) Stop the store.
void	writeEvent(JobEvent event, boolean cutBlock) Write a job event.
protected void	writeEventRunnable(JobEvent event, boolean cutBlock)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FileSystemJobEventStore

public FileSystemJobEventStore(org.apache.flink.api.common.JobID jobID,
                               org.apache.flink.configuration.Configuration configuration)
                        throws IOException

Throws:

IOException

FileSystemJobEventStore

@VisibleForTesting
public FileSystemJobEventStore(org.apache.flink.core.fs.Path workingDir,
                                                  org.apache.flink.configuration.Configuration configuration)
                                           throws IOException

Throws:

IOException

Method Detail

start

public void start()
           throws IOException

Description copied from interface: JobEventStore

Start the store. This method should be called before any other operations.

Specified by:

start in interface JobEventStore

Throws:

IOException

createJobEventWriterExecutor

protected ScheduledExecutorService createJobEventWriterExecutor()

stop

public void stop(boolean clearEventLogs)

Description copied from interface: JobEventStore

Stop the store.

Specified by:

stop in interface JobEventStore

Parameters:

clearEventLogs - Whether to clear the job events that have been recorded in the store.

writeEvent

public void writeEvent(JobEvent event,
                       boolean cutBlock)

Description copied from interface: JobEventStore

Write a job event.

Specified by:

writeEvent in interface JobEventStore

Parameters:

event - The job event that will be recorded.

cutBlock - If set to true, the current output file will be closed after writing this event, and a new output file will be created for subsequent events. This parameter effectively controls the segmentation of event data into separate files.

writeEventRunnable

@VisibleForTesting
protected void writeEventRunnable(JobEvent event,
                                                     boolean cutBlock)

readEvent

public JobEvent readEvent()
                   throws Exception

Description copied from interface: JobEventStore

Read a job event.

Specified by:

readEvent in interface JobEventStore

Returns:

job event.

Throws:

Exception

isEmpty

public boolean isEmpty()
                throws Exception

Description copied from interface: JobEventStore

Returns whether the store is empty.

Specified by:

isEmpty in interface JobEventStore

Returns:

false if the store contains any job events, true otherwise.

Throws:

Exception