org.apache.flink.client.program.rest

Class RestClusterClient<T>

java.lang.Object

org.apache.flink.client.program.ClusterClient<T>

org.apache.flink.client.program.rest.RestClusterClient<T>

All Implemented Interfaces:

NewClusterClient

public class RestClusterClient<T>
extends ClusterClient<T>
implements NewClusterClient

A ClusterClient implementation that communicates via HTTP REST requests.

Field Summary

Fields inherited from class org.apache.flink.client.program.ClusterClient

flinkConfig, highAvailabilityServices, lastJobExecutionResult, log, timeout

Constructor Summary

Constructors

Constructor and Description
RestClusterClient(org.apache.flink.configuration.Configuration config, T clusterId)
RestClusterClient(org.apache.flink.configuration.Configuration config, T clusterId, org.apache.flink.runtime.leaderretrieval.LeaderRetrievalService webMonitorRetrievalService)

Method Summary

Modifier and Type	Method and Description
void	cancel(org.apache.flink.api.common.JobID jobID) Cancels a job identified by the job id.
String	cancelWithSavepoint(org.apache.flink.api.common.JobID jobId, String savepointDirectory) Cancels a job identified by the job id and triggers a savepoint.
CompletableFuture<org.apache.flink.runtime.messages.Acknowledge>	disposeSavepoint(String savepointPath)
Map<String,org.apache.flink.util.OptionalFailure<Object>>	getAccumulators(org.apache.flink.api.common.JobID jobID, ClassLoader loader) Requests and returns the accumulators for the given job identifier.
T	getClusterId() Returns the cluster id identifying the cluster to which the client is connected.
CompletableFuture<org.apache.flink.runtime.rest.messages.job.JobDetailsInfo>	getJobDetails(org.apache.flink.api.common.JobID jobId) Requests the job details.
CompletableFuture<org.apache.flink.runtime.jobgraph.JobStatus>	getJobStatus(org.apache.flink.api.common.JobID jobId) Requests the JobStatus of the job with the given JobID.
String	getWebInterfaceURL() Returns an URL (as a string) to the JobManager web interface.
CompletableFuture<Collection<org.apache.flink.runtime.client.JobStatusMessage>>	listJobs() Lists the currently running and finished jobs on the cluster.
CompletableFuture<org.apache.flink.runtime.jobmaster.JobResult>	requestJobResult(org.apache.flink.api.common.JobID jobId) Requests the JobResult for the given JobID.
<M extends org.apache.flink.runtime.rest.messages.MessageHeaders<R,P,U>,U extends org.apache.flink.runtime.rest.messages.MessageParameters,R extends org.apache.flink.runtime.rest.messages.RequestBody,P extends org.apache.flink.runtime.rest.messages.ResponseBody> CompletableFuture<P>	sendRequest(M messageHeaders, U messageParameters, R request)
void	shutdown() Shuts down the client.
void	shutDownCluster()
String	stopWithSavepoint(org.apache.flink.api.common.JobID jobId, boolean advanceToEndOfTime, String savepointDirectory) Stops a program on Flink cluster whose job-manager is configured in this client's configuration.
CompletableFuture<org.apache.flink.api.common.JobSubmissionResult>	submitJob(org.apache.flink.runtime.jobgraph.JobGraph jobGraph) Submits the given JobGraph to the dispatcher.
org.apache.flink.api.common.JobSubmissionResult	submitJob(org.apache.flink.runtime.jobgraph.JobGraph jobGraph, ClassLoader classLoader) Calls the subclasses' submitJob method.
CompletableFuture<String>	triggerSavepoint(org.apache.flink.api.common.JobID jobId, String savepointDirectory) Triggers a savepoint for the job identified by the job id.

Methods inherited from class org.apache.flink.client.program.ClusterClient

getAccumulators, getClusterConnectionInfo, getFlinkConfiguration, getJobGraph, getJobGraph, getOptimizedPlan, getOptimizedPlan, getOptimizedPlanAsJson, getPrintStatusDuringExecution, isDetached, run, run, run, run, run, setDetached, setPrintStatusDuringExecution

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

RestClusterClient

public RestClusterClient(org.apache.flink.configuration.Configuration config,
                         T clusterId)
                  throws Exception

Throws:

Exception

RestClusterClient

public RestClusterClient(org.apache.flink.configuration.Configuration config,
                         T clusterId,
                         org.apache.flink.runtime.leaderretrieval.LeaderRetrievalService webMonitorRetrievalService)
                  throws Exception

Throws:

Exception

Method Detail

shutdown

public void shutdown()

Description copied from class: ClusterClient

Shuts down the client. This stops the internal actor system and actors.

Overrides:

shutdown in class ClusterClient<T>

submitJob

public org.apache.flink.api.common.JobSubmissionResult submitJob(org.apache.flink.runtime.jobgraph.JobGraph jobGraph,
                                                                 ClassLoader classLoader)
                                                          throws ProgramInvocationException

Description copied from class: ClusterClient

Calls the subclasses' submitJob method. It may decide to simply call one of the run methods or it may perform some custom job submission logic.

Specified by:

submitJob in class ClusterClient<T>

Parameters:

jobGraph - The JobGraph to be submitted

Returns:

JobSubmissionResult

Throws:

ProgramInvocationException

getJobDetails

public CompletableFuture<org.apache.flink.runtime.rest.messages.job.JobDetailsInfo> getJobDetails(org.apache.flink.api.common.JobID jobId)

Requests the job details.

Parameters:

jobId - The job id

Returns:

Job details

getJobStatus

public CompletableFuture<org.apache.flink.runtime.jobgraph.JobStatus> getJobStatus(org.apache.flink.api.common.JobID jobId)

Description copied from class: ClusterClient

Requests the JobStatus of the job with the given JobID.

Specified by:

getJobStatus in class ClusterClient<T>

requestJobResult

public CompletableFuture<org.apache.flink.runtime.jobmaster.JobResult> requestJobResult(@Nonnull
                                                                                        org.apache.flink.api.common.JobID jobId)

Requests the JobResult for the given JobID. The method retries multiple times to poll the JobResult before giving up.

Specified by:

requestJobResult in interface NewClusterClient

Parameters:

jobId - specifying the job for which to retrieve the JobResult

Returns:

Future which is completed with the JobResult once the job has completed or with a failure if the JobResult could not be retrieved.

submitJob

public CompletableFuture<org.apache.flink.api.common.JobSubmissionResult> submitJob(@Nonnull
                                                                                    org.apache.flink.runtime.jobgraph.JobGraph jobGraph)

Submits the given JobGraph to the dispatcher.

Specified by:

submitJob in interface NewClusterClient

Parameters:

jobGraph - to submit

Returns:

Future which is completed with the submission response

cancel

public void cancel(org.apache.flink.api.common.JobID jobID)
            throws Exception

Description copied from class: ClusterClient

Cancels a job identified by the job id.

Specified by:

cancel in class ClusterClient<T>

Parameters:

jobID - the job id

Throws:

Exception - In case an error occurred.

stopWithSavepoint

public String stopWithSavepoint(org.apache.flink.api.common.JobID jobId,
                                boolean advanceToEndOfTime,
                                @Nullable
                                String savepointDirectory)
                         throws Exception

Description copied from class: ClusterClient

Stops a program on Flink cluster whose job-manager is configured in this client's configuration. Stopping works only for streaming programs. Be aware, that the program might continue to run for a while after sending the stop command, because after sources stopped to emit data all operators need to finish processing.

Specified by:

stopWithSavepoint in class ClusterClient<T>

Parameters:

jobId - the job ID of the streaming program to stop

advanceToEndOfTime - flag indicating if the source should inject a MAX_WATERMARK in the pipeline

savepointDirectory - directory the savepoint should be written to

Returns:

a CompletableFuture containing the path where the savepoint is located

Throws:

Exception - If the job ID is invalid (ie, is unknown or refers to a batch job) or if sending the stop signal failed. That might be due to an I/O problem, ie, the job-manager is unreachable.

cancelWithSavepoint

public String cancelWithSavepoint(org.apache.flink.api.common.JobID jobId,
                                  @Nullable
                                  String savepointDirectory)
                           throws Exception

Description copied from class: ClusterClient

Cancels a job identified by the job id and triggers a savepoint.

Specified by:

cancelWithSavepoint in class ClusterClient<T>

Parameters:

jobId - the job id

savepointDirectory - directory the savepoint should be written to

Returns:

path where the savepoint is located

Throws:

Exception - In case an error occurred.

triggerSavepoint

public CompletableFuture<String> triggerSavepoint(org.apache.flink.api.common.JobID jobId,
                                                  @Nullable
                                                  String savepointDirectory)

Description copied from class: ClusterClient

Triggers a savepoint for the job identified by the job id. The savepoint will be written to the given savepoint directory, or CheckpointingOptions.SAVEPOINT_DIRECTORY if it is null.

Specified by:

triggerSavepoint in class ClusterClient<T>

Parameters:

jobId - job id

savepointDirectory - directory the savepoint should be written to

Returns:

path future where the savepoint is located

getAccumulators

public Map<String,org.apache.flink.util.OptionalFailure<Object>> getAccumulators(org.apache.flink.api.common.JobID jobID,
                                                                                 ClassLoader loader)
                                                                          throws Exception

Description copied from class: ClusterClient

Requests and returns the accumulators for the given job identifier. Accumulators can be requested while a is running or after it has finished.

Specified by:

getAccumulators in class ClusterClient<T>

Parameters:

jobID - The job identifier of a job.

loader - The class loader for deserializing the accumulator results.

Returns:

A Map containing the accumulator's name and its value.

Throws:

Exception

listJobs

public CompletableFuture<Collection<org.apache.flink.runtime.client.JobStatusMessage>> listJobs()

Description copied from class: ClusterClient

Lists the currently running and finished jobs on the cluster.

Specified by:

listJobs in class ClusterClient<T>

Returns:

future collection of running and finished jobs

getClusterId

public T getClusterId()

Description copied from class: ClusterClient

Returns the cluster id identifying the cluster to which the client is connected.

Specified by:

getClusterId in class ClusterClient<T>

Returns:

cluster id of the connected cluster

disposeSavepoint

public CompletableFuture<org.apache.flink.runtime.messages.Acknowledge> disposeSavepoint(String savepointPath)

Specified by:

disposeSavepoint in class ClusterClient<T>

shutDownCluster

public void shutDownCluster()

Overrides:

shutDownCluster in class ClusterClient<T>

getWebInterfaceURL

public String getWebInterfaceURL()

Description copied from class: ClusterClient

Returns an URL (as a string) to the JobManager web interface.

Specified by:

getWebInterfaceURL in class ClusterClient<T>

sendRequest

@VisibleForTesting
public <M extends org.apache.flink.runtime.rest.messages.MessageHeaders<R,P,U>,U extends org.apache.flink.runtime.rest.messages.MessageParameters,R extends org.apache.flink.runtime.rest.messages.RequestBody,P extends org.apache.flink.runtime.rest.messages.ResponseBody> CompletableFuture<P> sendRequest(M messageHeaders,
                                                                                                                                                                                                                                                                                                                                  U messageParameters,
                                                                                                                                                                                                                                                                                                                                  R request)