org.apache.flink.connector.testframe.container

Class FlinkContainers

java.lang.Object

org.apache.flink.connector.testframe.container.FlinkContainers

All Implemented Interfaces:

org.junit.jupiter.api.extension.AfterAllCallback, org.junit.jupiter.api.extension.BeforeAllCallback, org.junit.jupiter.api.extension.Extension

public class FlinkContainers
extends Object
implements org.junit.jupiter.api.extension.BeforeAllCallback, org.junit.jupiter.api.extension.AfterAllCallback

A Flink cluster running JM and TMs on containers.

This containerized Flink cluster is based on Testcontainers, which simulates a truly distributed environment for E2E tests. This class can also be used as an Extension of JUnit 5 so that the lifecycle of the cluster can be easily managed by JUnit Jupiter engine.

Example usage

{@code
 public class E2ETest {
     // Create a Flink cluster using default configurations.
     // Remember to declare it as "static" as required by
     // JUnit 5.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	FlinkContainers.Builder The FlinkContainers builder.

Field Summary

Fields

Modifier and Type	Field and Description
static java.time.Duration	DEFAULT_TIMEOUT

Method Summary

Modifier and Type	Method and Description
void	afterAll(org.junit.jupiter.api.extension.ExtensionContext context)
void	beforeAll(org.junit.jupiter.api.extension.ExtensionContext context)
static FlinkContainers.Builder	builder() Creates a builder for FlinkContainers.
org.testcontainers.containers.GenericContainer<?>	getJobManager() Gets JobManager container.
String	getJobManagerHost() Gets JobManager's hostname on the host machine.
int	getJobManagerPort() Gets JobManager's port on the host machine.
org.apache.flink.client.program.rest.RestClusterClient<org.apache.flink.client.deployment.StandaloneClusterId>	getRestClusterClient() Gets REST client connected to JobManager.
List<org.testcontainers.containers.GenericContainer<?>>	getTaskManagers() Gets TaskManager containers.
boolean	isStarted() Gets the running state of the cluster.
void	restartJobManager(org.apache.flink.util.function.RunnableWithException afterFailAction) Restarts JobManager container.
void	restartTaskManager(org.apache.flink.util.function.RunnableWithException afterFailAction) Restarts all TaskManager containers.
void	start() Starts all containers.
void	stop() Stops all containers.
org.apache.flink.api.common.JobID	submitJob(org.apache.flink.test.util.JobSubmission job) Submits the given job to the cluster.
void	submitSQLJob(org.apache.flink.test.util.SQLJobSubmission job) Submits an SQL job to the running cluster.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_TIMEOUT

public static final java.time.Duration DEFAULT_TIMEOUT

Method Detail

builder

public static FlinkContainers.Builder builder()

Creates a builder for FlinkContainers.

start

public void start()
           throws Exception

Starts all containers.

Throws:

Exception

stop

public void stop()

Stops all containers.

isStarted

public boolean isStarted()

Gets the running state of the cluster.

getJobManager

public org.testcontainers.containers.GenericContainer<?> getJobManager()

Gets JobManager container.

getTaskManagers

public List<org.testcontainers.containers.GenericContainer<?>> getTaskManagers()

Gets TaskManager containers.

getJobManagerHost

public String getJobManagerHost()

Gets JobManager's hostname on the host machine.

getJobManagerPort

public int getJobManagerPort()

Gets JobManager's port on the host machine.

getRestClusterClient

@Nullable
public org.apache.flink.client.program.rest.RestClusterClient<org.apache.flink.client.deployment.StandaloneClusterId> getRestClusterClient()

Gets REST client connected to JobManager.

restartJobManager

public void restartJobManager(org.apache.flink.util.function.RunnableWithException afterFailAction)
                       throws Exception

Restarts JobManager container.

Note that the REST port will be changed because the new JM container will be mapped to another random port. Please make sure to get the REST cluster client again after this method is invoked.

Throws:

Exception

restartTaskManager

public void restartTaskManager(org.apache.flink.util.function.RunnableWithException afterFailAction)
                        throws Exception

Restarts all TaskManager containers.

Throws:

Exception

submitSQLJob

public void submitSQLJob(org.apache.flink.test.util.SQLJobSubmission job)
                  throws IOException,
                         InterruptedException

Submits an SQL job to the running cluster.

NOTE: You should not use '\t'.

Throws:

IOException

InterruptedException

submitJob

public org.apache.flink.api.common.JobID submitJob(org.apache.flink.test.util.JobSubmission job)
                                            throws IOException,
                                                   InterruptedException

Submits the given job to the cluster.

Parameters:

job - job to submit

Throws:

IOException

InterruptedException

beforeAll

public void beforeAll(org.junit.jupiter.api.extension.ExtensionContext context)
               throws Exception

Specified by:

beforeAll in interface org.junit.jupiter.api.extension.BeforeAllCallback

Throws:

Exception

afterAll

public void afterAll(org.junit.jupiter.api.extension.ExtensionContext context)

Specified by:

afterAll in interface org.junit.jupiter.api.extension.AfterAllCallback