public abstract class ClusterClient extends Object
| Modifier and Type | Class and Description |
|---|---|
protected static class |
ClusterClient.LazyActorSystemLoader |
| Modifier and Type | Field and Description |
|---|---|
protected ClusterClient.LazyActorSystemLoader |
actorSystemLoader
The actor system used to communicate with the JobManager.
|
protected org.apache.flink.configuration.Configuration |
flinkConfig
Configuration of the client
|
protected scala.concurrent.duration.FiniteDuration |
timeout
Timeout for futures
|
| Constructor and Description |
|---|
ClusterClient(org.apache.flink.configuration.Configuration flinkConfig)
Creates a instance that submits the programs to the JobManager defined in the
configuration.
|
| Modifier and Type | Method and Description |
|---|---|
void |
cancel(org.apache.flink.api.common.JobID jobId)
Cancels a job identified by the job id.
|
void |
endSession(org.apache.flink.api.common.JobID jobId)
Tells the JobManager to finish the session (job) defined by the given ID.
|
void |
endSessions(List<org.apache.flink.api.common.JobID> jobIds)
Tells the JobManager to finish the sessions (jobs) defined by the given IDs.
|
protected abstract void |
finalizeCluster()
Request the cluster to shut down or disconnect.
|
Map<String,Object> |
getAccumulators(org.apache.flink.api.common.JobID jobID)
Requests and returns the accumulators for the given job identifier.
|
Map<String,Object> |
getAccumulators(org.apache.flink.api.common.JobID jobID,
ClassLoader loader)
Requests and returns the accumulators for the given job identifier.
|
abstract String |
getClusterIdentifier()
Returns a string representation of the cluster.
|
abstract org.apache.flink.runtime.clusterframework.messages.GetClusterStatusResponse |
getClusterStatus()
Returns the latest cluster status, with number of Taskmanagers and slots
|
org.apache.flink.configuration.Configuration |
getFlinkConfiguration()
Return the Flink configuration object
|
org.apache.flink.runtime.jobgraph.JobGraph |
getJobGraph(PackagedProgram prog,
org.apache.flink.optimizer.plan.FlinkPlan optPlan) |
org.apache.flink.runtime.jobgraph.JobGraph |
getJobGraph(PackagedProgram prog,
org.apache.flink.optimizer.plan.FlinkPlan optPlan,
org.apache.flink.runtime.jobgraph.SavepointRestoreSettings savepointSettings) |
InetSocketAddress |
getJobManagerAddress()
Gets the current JobManager address (may change in case of a HA setup).
|
org.apache.flink.runtime.instance.ActorGateway |
getJobManagerGateway()
Returns the
ActorGateway of the current job manager leader using
the LeaderRetrievalService. |
abstract int |
getMaxSlots()
The client may define an upper limit on the number of slots to use
|
protected abstract List<String> |
getNewMessages()
May return new messages from the cluster.
|
static org.apache.flink.optimizer.plan.FlinkPlan |
getOptimizedPlan(org.apache.flink.optimizer.Optimizer compiler,
PackagedProgram prog,
int parallelism) |
static org.apache.flink.optimizer.plan.OptimizedPlan |
getOptimizedPlan(org.apache.flink.optimizer.Optimizer compiler,
org.apache.flink.api.common.Plan p,
int parallelism) |
static String |
getOptimizedPlanAsJson(org.apache.flink.optimizer.Optimizer compiler,
PackagedProgram prog,
int parallelism) |
boolean |
getPrintStatusDuringExecution() |
abstract String |
getWebInterfaceURL()
Returns an URL (as a string) to the JobManager web interface
|
boolean |
isDetached()
A flag to indicate whether this clients submits jobs detached.
|
protected void |
logAndSysout(String message)
Logs and prints to sysout if printing to stdout is enabled.
|
org.apache.flink.api.common.JobSubmissionResult |
run(org.apache.flink.optimizer.plan.FlinkPlan compiledPlan,
List<URL> libraries,
List<URL> classpaths,
ClassLoader classLoader) |
org.apache.flink.api.common.JobSubmissionResult |
run(org.apache.flink.optimizer.plan.FlinkPlan compiledPlan,
List<URL> libraries,
List<URL> classpaths,
ClassLoader classLoader,
org.apache.flink.runtime.jobgraph.SavepointRestoreSettings savepointSettings) |
org.apache.flink.api.common.JobExecutionResult |
run(org.apache.flink.runtime.jobgraph.JobGraph jobGraph,
ClassLoader classLoader)
Submits a JobGraph blocking.
|
org.apache.flink.api.common.JobSubmissionResult |
run(JobWithJars program,
int parallelism) |
org.apache.flink.api.common.JobSubmissionResult |
run(JobWithJars program,
int parallelism,
org.apache.flink.runtime.jobgraph.SavepointRestoreSettings savepointSettings)
Runs a program on the Flink cluster to which this client is connected.
|
org.apache.flink.api.common.JobSubmissionResult |
run(PackagedProgram prog,
int parallelism)
General purpose method to run a user jar from the CliFrontend in either blocking or detached mode, depending
on whether
setDetached(true) or setDetached(false). |
org.apache.flink.api.common.JobSubmissionResult |
runDetached(org.apache.flink.runtime.jobgraph.JobGraph jobGraph,
ClassLoader classLoader)
Submits a JobGraph detached.
|
void |
setDetached(boolean isDetached)
Set the mode of this client (detached or blocking job execution).
|
void |
setPrintStatusDuringExecution(boolean print)
Configures whether the client should print progress updates during the execution to
System.out. |
void |
shutdown()
Shuts down the client.
|
void |
stop(org.apache.flink.api.common.JobID jobId)
Stops a program on Flink cluster whose job-manager is configured in this client's configuration.
|
protected abstract org.apache.flink.api.common.JobSubmissionResult |
submitJob(org.apache.flink.runtime.jobgraph.JobGraph jobGraph,
ClassLoader classLoader)
Calls the subclasses' submitJob method.
|
abstract void |
waitForClusterToBeReady()
Blocks until the client has determined that the cluster is ready for Job submission.
|
protected final ClusterClient.LazyActorSystemLoader actorSystemLoader
protected final org.apache.flink.configuration.Configuration flinkConfig
protected final scala.concurrent.duration.FiniteDuration timeout
public ClusterClient(org.apache.flink.configuration.Configuration flinkConfig)
throws IOException
flinkConfig - The config used to obtain the job-manager's address, and used to configure the optimizer.IOException - Thrown, if the client's actor system could not be started.public void shutdown()
public void setPrintStatusDuringExecution(boolean print)
System.out.
All updates are logged via the SLF4J loggers regardless of this setting.print - True to print updates to standard out during execution, false to not print them.public boolean getPrintStatusDuringExecution()
System.outpublic InetSocketAddress getJobManagerAddress()
public static String getOptimizedPlanAsJson(org.apache.flink.optimizer.Optimizer compiler, PackagedProgram prog, int parallelism) throws org.apache.flink.optimizer.CompilerException, ProgramInvocationException
org.apache.flink.optimizer.CompilerExceptionProgramInvocationExceptionpublic static org.apache.flink.optimizer.plan.FlinkPlan getOptimizedPlan(org.apache.flink.optimizer.Optimizer compiler,
PackagedProgram prog,
int parallelism)
throws org.apache.flink.optimizer.CompilerException,
ProgramInvocationException
org.apache.flink.optimizer.CompilerExceptionProgramInvocationExceptionpublic static org.apache.flink.optimizer.plan.OptimizedPlan getOptimizedPlan(org.apache.flink.optimizer.Optimizer compiler,
org.apache.flink.api.common.Plan p,
int parallelism)
throws org.apache.flink.optimizer.CompilerException
org.apache.flink.optimizer.CompilerExceptionpublic org.apache.flink.api.common.JobSubmissionResult run(PackagedProgram prog, int parallelism) throws ProgramInvocationException
setDetached(true) or setDetached(false).prog - the packaged programparallelism - the parallelism to execute the contained Flink jobProgramInvocationExceptionpublic org.apache.flink.api.common.JobSubmissionResult run(JobWithJars program, int parallelism) throws ProgramInvocationException
ProgramInvocationExceptionpublic org.apache.flink.api.common.JobSubmissionResult run(JobWithJars program, int parallelism, org.apache.flink.runtime.jobgraph.SavepointRestoreSettings savepointSettings) throws org.apache.flink.optimizer.CompilerException, ProgramInvocationException
program - The program to be executed.parallelism - The default parallelism to use when running the program. The default parallelism is used
when the program does not set a parallelism by itself.org.apache.flink.optimizer.CompilerException - Thrown, if the compiler encounters an illegal situation.ProgramInvocationException - Thrown, if the program could not be instantiated from its jar file,
or if the submission failed. That might be either due to an I/O problem,
i.e. the job-manager is unreachable, or due to the fact that the
parallel execution failed.public org.apache.flink.api.common.JobSubmissionResult run(org.apache.flink.optimizer.plan.FlinkPlan compiledPlan,
List<URL> libraries,
List<URL> classpaths,
ClassLoader classLoader)
throws ProgramInvocationException
ProgramInvocationExceptionpublic org.apache.flink.api.common.JobSubmissionResult run(org.apache.flink.optimizer.plan.FlinkPlan compiledPlan,
List<URL> libraries,
List<URL> classpaths,
ClassLoader classLoader,
org.apache.flink.runtime.jobgraph.SavepointRestoreSettings savepointSettings)
throws ProgramInvocationException
ProgramInvocationExceptionpublic org.apache.flink.api.common.JobExecutionResult run(org.apache.flink.runtime.jobgraph.JobGraph jobGraph,
ClassLoader classLoader)
throws ProgramInvocationException
jobGraph - The JobGraphclassLoader - User code class loader to deserialize the results and errors (may contain custom classes).ProgramInvocationExceptionpublic org.apache.flink.api.common.JobSubmissionResult runDetached(org.apache.flink.runtime.jobgraph.JobGraph jobGraph,
ClassLoader classLoader)
throws ProgramInvocationException
jobGraph - The JobGraphclassLoader - User code class loader to deserialize the results and errors (may contain custom classes).ProgramInvocationExceptionpublic void cancel(org.apache.flink.api.common.JobID jobId)
throws Exception
jobId - the job idException - In case an error occurred.public void stop(org.apache.flink.api.common.JobID jobId)
throws Exception
jobId - the job ID of the streaming program to stopException - If the job ID is invalid (ie, is unknown or refers to a batch job) or if sending the stop signal
failed. That might be due to an I/O problem, ie, the job-manager is unreachable.public Map<String,Object> getAccumulators(org.apache.flink.api.common.JobID jobID) throws Exception
jobID - The job identifier of a job.Exceptionpublic Map<String,Object> getAccumulators(org.apache.flink.api.common.JobID jobID, ClassLoader loader) throws Exception
jobID - The job identifier of a job.loader - The class loader for deserializing the accumulator results.Exceptionpublic void endSession(org.apache.flink.api.common.JobID jobId)
throws Exception
jobId - The ID that identifies the session.Exceptionpublic void endSessions(List<org.apache.flink.api.common.JobID> jobIds) throws Exception
jobIds - The IDs that identify the sessions.Exceptionpublic org.apache.flink.runtime.jobgraph.JobGraph getJobGraph(PackagedProgram prog, org.apache.flink.optimizer.plan.FlinkPlan optPlan) throws ProgramInvocationException
ProgramInvocationExceptionpublic org.apache.flink.runtime.jobgraph.JobGraph getJobGraph(PackagedProgram prog, org.apache.flink.optimizer.plan.FlinkPlan optPlan, org.apache.flink.runtime.jobgraph.SavepointRestoreSettings savepointSettings) throws ProgramInvocationException
ProgramInvocationExceptionpublic org.apache.flink.runtime.instance.ActorGateway getJobManagerGateway()
throws Exception
ActorGateway of the current job manager leader using
the LeaderRetrievalService.Exceptionprotected void logAndSysout(String message)
message - The message to log/printpublic abstract void waitForClusterToBeReady()
public abstract String getWebInterfaceURL()
public abstract org.apache.flink.runtime.clusterframework.messages.GetClusterStatusResponse getClusterStatus()
protected abstract List<String> getNewMessages()
public abstract String getClusterIdentifier()
protected abstract void finalizeCluster()
public void setDetached(boolean isDetached)
isDetached - If true, the client will submit programs detached via the run methodpublic boolean isDetached()
public org.apache.flink.configuration.Configuration getFlinkConfiguration()
public abstract int getMaxSlots()
protected abstract org.apache.flink.api.common.JobSubmissionResult submitJob(org.apache.flink.runtime.jobgraph.JobGraph jobGraph,
ClassLoader classLoader)
throws ProgramInvocationException
jobGraph - The JobGraph to be submittedProgramInvocationExceptionCopyright © 2014–2017 The Apache Software Foundation. All rights reserved.