org.apache.flink.client.program

Class Client

java.lang.Object

org.apache.flink.client.program.Client

public class Client
extends Object

Encapsulates the functionality necessary to submit a program to a remote cluster.

Constructor Summary

Constructors

Constructor and Description
Client(Configuration config) Creates a instance that submits the programs to the JobManager defined in the configuration.
Client(Configuration config, int maxSlots) Creates a new instance of the class that submits the jobs to a job-manager.

Method Summary

Methods

Modifier and Type	Method and Description
void	cancel(JobID jobId) Cancels a job identified by the job id.
void	endSession(JobID jobId) Tells the JobManager to finish the session (job) defined by the given ID.
void	endSessions(List<JobID> jobIds) Tells the JobManager to finish the sessions (jobs) defined by the given IDs.
Map<String,Object>	getAccumulators(JobID jobID) Requests and returns the accumulators for the given job identifier.
Map<String,Object>	getAccumulators(JobID jobID, ClassLoader loader) Requests and returns the accumulators for the given job identifier.
JobGraph	getJobGraph(PackagedProgram prog, FlinkPlan optPlan)
int	getMaxSlots()
static FlinkPlan	getOptimizedPlan(Optimizer compiler, PackagedProgram prog, int parallelism)
static OptimizedPlan	getOptimizedPlan(Optimizer compiler, Plan p, int parallelism)
static String	getOptimizedPlanAsJson(Optimizer compiler, PackagedProgram prog, int parallelism)
boolean	getPrintStatusDuringExecution()
JobExecutionResult	runBlocking(JobGraph jobGraph, ClassLoader classLoader)
JobExecutionResult	runBlocking(JobWithJars program, int parallelism) Runs a program on the Flink cluster to which this client is connected.
JobExecutionResult	runBlocking(OptimizedPlan compiledPlan, List<URL> libraries, List<URL> classpaths, ClassLoader classLoader)
JobSubmissionResult	runBlocking(PackagedProgram prog, int parallelism)
JobSubmissionResult	runDetached(JobGraph jobGraph, ClassLoader classLoader)
JobSubmissionResult	runDetached(JobWithJars program, int parallelism) Submits a program to the Flink cluster to which this client is connected.
JobSubmissionResult	runDetached(OptimizedPlan compiledPlan, List<URL> libraries, List<URL> classpaths, ClassLoader classLoader)
JobSubmissionResult	runDetached(PackagedProgram prog, int parallelism)
void	setPrintStatusDuringExecution(boolean print) Configures whether the client should print progress updates during the execution to System.out.
void	shutdown() Shuts down the client.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Client

public Client(Configuration config)
       throws IOException

Creates a instance that submits the programs to the JobManager defined in the configuration. This method will try to resolve the JobManager hostname and throw an exception if that is not possible.

Parameters:

config - The config used to obtain the job-manager's address, and used to configure the optimizer.

Throws:

IOException - Thrown, if the client's actor system could not be started.

UnknownHostException - Thrown, if the JobManager's hostname could not be resolved.

Client

public Client(Configuration config,
      int maxSlots)
       throws IOException

Creates a new instance of the class that submits the jobs to a job-manager. at the given address using the default port.

Parameters:

config - The configuration for the client-side processes, like the optimizer.

maxSlots - maxSlots The number of maxSlots on the cluster if != -1.

Throws:

IOException - Thrown, if the client's actor system could not be started.

UnknownHostException - Thrown, if the JobManager's hostname could not be resolved.

Method Detail

shutdown

public void shutdown()

Shuts down the client. This stops the internal actor system and actors.

setPrintStatusDuringExecution

public void setPrintStatusDuringExecution(boolean print)

Configures whether the client should print progress updates during the execution to System.out. All updates are logged via the SLF4J loggers regardless of this setting.

Parameters:

print - True to print updates to standard out during execution, false to not print them.

getPrintStatusDuringExecution

public boolean getPrintStatusDuringExecution()

Returns:

whether the client will print progress updates during the execution to System.out

getMaxSlots

public int getMaxSlots()

Returns:

-1 if unknown. The maximum number of available processing slots at the Flink cluster connected to this client.

getOptimizedPlanAsJson

public static String getOptimizedPlanAsJson(Optimizer compiler,
                            PackagedProgram prog,
                            int parallelism)
                                     throws CompilerException,
                                            ProgramInvocationException

Throws:

CompilerException

ProgramInvocationException

getOptimizedPlan

public static FlinkPlan getOptimizedPlan(Optimizer compiler,
                         PackagedProgram prog,
                         int parallelism)
                                  throws CompilerException,
                                         ProgramInvocationException

Throws:

CompilerException

ProgramInvocationException

getOptimizedPlan

public static OptimizedPlan getOptimizedPlan(Optimizer compiler,
                             Plan p,
                             int parallelism)
                                      throws CompilerException

Throws:

CompilerException

runBlocking

public JobSubmissionResult runBlocking(PackagedProgram prog,
                              int parallelism)
                                throws ProgramInvocationException

Throws:

ProgramInvocationException

runDetached

public JobSubmissionResult runDetached(PackagedProgram prog,
                              int parallelism)
                                throws ProgramInvocationException

Throws:

ProgramInvocationException

runBlocking

public JobExecutionResult runBlocking(JobWithJars program,
                             int parallelism)
                               throws CompilerException,
                                      ProgramInvocationException

Runs a program on the Flink cluster to which this client is connected. The call blocks until the execution is complete, and returns afterwards.

Parameters:

program - The program to be executed.

parallelism - The default parallelism to use when running the program. The default parallelism is used when the program does not set a parallelism by itself.

Throws:

CompilerException - Thrown, if the compiler encounters an illegal situation.

ProgramInvocationException - Thrown, if the program could not be instantiated from its jar file, or if the submission failed. That might be either due to an I/O problem, i.e. the job-manager is unreachable, or due to the fact that the parallel execution failed.

runDetached

public JobSubmissionResult runDetached(JobWithJars program,
                              int parallelism)
                                throws CompilerException,
                                       ProgramInvocationException

Submits a program to the Flink cluster to which this client is connected. The call returns after the program was submitted and does not wait for the program to complete.

Parameters:

program - The program to be executed.

parallelism - The default parallelism to use when running the program. The default parallelism is used when the program does not set a parallelism by itself.

Throws:

CompilerException - Thrown, if the compiler encounters an illegal situation.

ProgramInvocationException - Thrown, if the program could not be instantiated from its jar file, or if the submission failed. That might be either due to an I/O problem, i.e. the job-manager is unreachable.

runBlocking

public JobExecutionResult runBlocking(OptimizedPlan compiledPlan,
                             List<URL> libraries,
                             List<URL> classpaths,
                             ClassLoader classLoader)
                               throws ProgramInvocationException

Throws:

ProgramInvocationException

runDetached

public JobSubmissionResult runDetached(OptimizedPlan compiledPlan,
                              List<URL> libraries,
                              List<URL> classpaths,
                              ClassLoader classLoader)
                                throws ProgramInvocationException

Throws:

ProgramInvocationException

runBlocking

public JobExecutionResult runBlocking(JobGraph jobGraph,
                             ClassLoader classLoader)
                               throws ProgramInvocationException

Throws:

ProgramInvocationException

runDetached

public JobSubmissionResult runDetached(JobGraph jobGraph,
                              ClassLoader classLoader)
                                throws ProgramInvocationException

Throws:

ProgramInvocationException

cancel

public void cancel(JobID jobId)
            throws Exception

Cancels a job identified by the job id.

Parameters:

jobId - the job id

Throws:

Exception - In case an error occurred.

getAccumulators

public Map<String,Object> getAccumulators(JobID jobID)
                                   throws Exception

Requests and returns the accumulators for the given job identifier. Accumulators can be requested while a is running or after it has finished. The default class loader is used to deserialize the incoming accumulator results.

Parameters:

jobID - The job identifier of a job.

Returns:

A Map containing the accumulator's name and its value.

Throws:

Exception

getAccumulators

public Map<String,Object> getAccumulators(JobID jobID,
                                 ClassLoader loader)
                                   throws Exception

Requests and returns the accumulators for the given job identifier. Accumulators can be requested while a is running or after it has finished.

Parameters:

jobID - The job identifier of a job.

loader - The class loader for deserializing the accumulator results.

Returns:

A Map containing the accumulator's name and its value.

Throws:

Exception

endSession

public void endSession(JobID jobId)
                throws Exception

Tells the JobManager to finish the session (job) defined by the given ID.

Parameters:

jobId - The ID that identifies the session.

Throws:

Exception

endSessions

public void endSessions(List<JobID> jobIds)
                 throws Exception

Tells the JobManager to finish the sessions (jobs) defined by the given IDs.

Parameters:

jobIds - The IDs that identify the sessions.

Throws:

Exception

getJobGraph

public JobGraph getJobGraph(PackagedProgram prog,
                   FlinkPlan optPlan)
                     throws ProgramInvocationException

Throws:

ProgramInvocationException