com.edmunds.rest.databricks.service

Class ClusterServiceImpl

java.lang.Object

com.edmunds.rest.databricks.service.DatabricksService

com.edmunds.rest.databricks.service.ClusterServiceImpl

All Implemented Interfaces:

ClusterService

public final class ClusterServiceImpl
extends DatabricksService
implements ClusterService

A basic implementation of the ClusterService.

Field Summary

Fields inherited from class com.edmunds.rest.databricks.service.DatabricksService

client, mapper

Constructor Summary

Constructors

Constructor and Description
ClusterServiceImpl(DatabricksRestClient client)

Method Summary

Modifier and Type	Method and Description
String	create(CreateClusterRequest createClusterRequest) Deprecated.
String	create(NewClusterDTO clusterDTO) Creates a databricks cluster.
void	delete(String clusterId) Terminates a cluster.
void	edit(EditClusterRequest editClusterRequest) Deprecated.
void	edit(UpsertClusterDTO clusterDTO) Edits the configurations of a databricks cluster.
List<ClusterInfoDTO>	findByName(String clusterName) Look for clusters with a given name.
Optional<ClusterInfoDTO>	findUniqueByName(String clusterName) Same as findByName, but it will throw an exception if more then one cluster has the name.
ClusterInfoDTO	getInfo(String clusterId) Gets information about a given cluster.
boolean	isClusterRunning(String clusterId) Returns true if the cluster is running.
ClusterInfoDTO[]	list() Lists all of the clusters on a given databricks instance.
List<ClusterEventDTO>	listEvents(String clusterId, ClusterEventTypeDTO[] eventsToFilter, int offset, int limit) Lists events of a specific cluster on a given databricks instance.
List<NodeTypeDTO>	listNodeTypes() Return a list of supported Spark node types.
List<SparkVersionDTO>	listSparkVersions() Return the list of available runtime versions.
List<String>	listZones() Return a list of availability zones where clusters can be created in.
void	permanentDelete(String clusterId) Permanently terminates a cluster.
void	pin(String clusterId) Pin a cluster with a given id.
void	resize(AutoScaleDTO autoscale, String clusterId) Resizes a cluster.
void	resize(int numWorkers, String clusterId) Resizes a cluster.
void	restart(String clusterId) Restarts a given databricks cluster.
void	start(String clusterId) Starts a databricks cluster.
void	unpin(String clusterId) Unpin a cluster with a given id.
void	upsertCluster(NewClusterDTO clusterDTO) Given a cluster settings DTO object it will: - create the cluster if it doesn't exist - edit the cluster if it does exist.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ClusterServiceImpl

public ClusterServiceImpl(DatabricksRestClient client)

Method Detail

create

@Deprecated
public String create(CreateClusterRequest createClusterRequest)
                          throws IOException,
                                 DatabricksRestException

Deprecated.

Description copied from interface: ClusterService

Creates a databricks cluster.

Specified by:

create in interface ClusterService

Parameters:

createClusterRequest - the cluster request object

Returns:

the clusterId

Throws:

IOException - other connection errors

DatabricksRestException - any errors with the request

See Also:

https://docs.databricks.com/api/latest/clusters.html#create

create

public String create(NewClusterDTO clusterDTO)
              throws IOException,
                     DatabricksRestException

Description copied from interface: ClusterService

Creates a databricks cluster.

Specified by:

create in interface ClusterService

Parameters:

clusterDTO - cluster DTO

Returns:

the clusterId

Throws:

IOException - other connection errors

DatabricksRestException - any errors with the request

See Also:

https://docs.databricks.com/api/latest/clusters.html#create

edit

@Deprecated
public void edit(EditClusterRequest editClusterRequest)
                      throws IOException,
                             DatabricksRestException

Deprecated.

Description copied from interface: ClusterService

Edits the configurations of a databricks cluster.

Specified by:

edit in interface ClusterService

Parameters:

editClusterRequest - the edit cluster request object

Throws:

IOException - any other errors

DatabricksRestException - any errors with the request

See Also:

https://docs.databricks.com/api/latest/clusters.html#edit

edit

public void edit(UpsertClusterDTO clusterDTO)
          throws IOException,
                 DatabricksRestException

Description copied from interface: ClusterService

Edits the configurations of a databricks cluster.

Specified by:

edit in interface ClusterService

Parameters:

clusterDTO - cluster DTO

Throws:

IOException - any other errors

DatabricksRestException - any errors with the request

See Also:

https://docs.databricks.com/api/latest/clusters.html#edit

start

public void start(String clusterId)
           throws IOException,
                  DatabricksRestException

Description copied from interface: ClusterService

Starts a databricks cluster.

Specified by:

start in interface ClusterService

Parameters:

clusterId - the clusterId of the cluster you want to start.

Throws:

IOException - any other errors

DatabricksRestException - any errors with the request

See Also:

https://docs.databricks.com/api/latest/clusters.html#start

isClusterRunning

public boolean isClusterRunning(String clusterId)
                         throws IOException,
                                DatabricksRestException

Description copied from interface: ClusterService

Returns true if the cluster is running.

Specified by:

isClusterRunning in interface ClusterService

Parameters:

clusterId - the clusterId to check.

Returns:

true if the cluster is running.

Throws:

IOException - any other errors

DatabricksRestException - any errors with the request

restart

public void restart(String clusterId)
             throws IOException,
                    DatabricksRestException

Description copied from interface: ClusterService

Restarts a given databricks cluster.

Specified by:

restart in interface ClusterService

Parameters:

clusterId - the clusterid of the cluster you want to restart.

Throws:

IOException - any other errors

DatabricksRestException - any errors with the request

See Also:

https://docs.databricks.com/api/latest/clusters.html#restart

resize

public void resize(int numWorkers,
                   String clusterId)
            throws IOException,
                   DatabricksRestException

Description copied from interface: ClusterService

Resizes a cluster. Will give it a fixed number of workers.

Specified by:

resize in interface ClusterService

Parameters:

numWorkers - the number of workers you want to resize to

clusterId - the clusterId you want to operate on

Throws:

IOException - any other errors

DatabricksRestException - any errors with request

See Also:

https://docs.databricks.com/api/latest/clusters.html#resize

resize

public void resize(AutoScaleDTO autoscale,
                   String clusterId)
            throws IOException,
                   DatabricksRestException

Description copied from interface: ClusterService

Resizes a cluster. Will add/edit autoscale functionality.

Specified by:

resize in interface ClusterService

Parameters:

autoscale - the autoscale specs.

clusterId - the clusterId you want to work on

Throws:

IOException - any other errors

DatabricksRestException - any errors with request

See Also:

https://docs.databricks.com/api/latest/clusters.html#resize

delete

public void delete(String clusterId)
            throws IOException,
                   DatabricksRestException

Description copied from interface: ClusterService

Terminates a cluster.

Specified by:

delete in interface ClusterService

Parameters:

clusterId - the cluster you want to terminate

Throws:

IOException - any other errors

DatabricksRestException - errors with request

See Also:

https://docs.databricks.com/api/latest/clusters.html#delete-terminate

permanentDelete

public void permanentDelete(String clusterId)
                     throws IOException,
                            DatabricksRestException

Description copied from interface: ClusterService

Permanently terminates a cluster.

Specified by:

permanentDelete in interface ClusterService

Parameters:

clusterId - the cluster you want to permanently terminate

Throws:

IOException - any other errors

DatabricksRestException - errors with request

See Also:

https://docs.databricks.com/api/latest/clusters.html#permanent-delete

getInfo

public ClusterInfoDTO getInfo(String clusterId)
                       throws IOException,
                              DatabricksRestException

Description copied from interface: ClusterService

Gets information about a given cluster.

Specified by:

getInfo in interface ClusterService

Parameters:

clusterId - the cluster you want to get info about

Returns:

the information DTO object

Throws:

IOException - any other errors.

DatabricksRestException - any errors with the request

See Also:

https://docs.databricks.com/api/latest/clusters.html#get

list

public ClusterInfoDTO[] list()
                      throws IOException,
                             DatabricksRestException

Description copied from interface: ClusterService

Lists all of the clusters on a given databricks instance.

Specified by:

list in interface ClusterService

Returns:

an array of cluster information objects

Throws:

IOException - any other errors

DatabricksRestException - any errors with the request

See Also:

https://docs.databricks.com/api/latest/clusters.html#list

listEvents

public List<ClusterEventDTO> listEvents(String clusterId,
                                        ClusterEventTypeDTO[] eventsToFilter,
                                        int offset,
                                        int limit)
                                 throws IOException,
                                        DatabricksRestException

Description copied from interface: ClusterService

Lists events of a specific cluster on a given databricks instance. Allows you to filter which events you want to see too.

Specified by:

listEvents in interface ClusterService

Returns:

an array of cluster events objects

Throws:

IOException - any other errors

DatabricksRestException - any errors with the request

See Also:

https://docs.databricks.com/api/latest/clusters.html#events

upsertCluster

public void upsertCluster(NewClusterDTO clusterDTO)
                   throws IOException,
                          DatabricksRestException

Description copied from interface: ClusterService

Given a cluster settings DTO object it will: - create the cluster if it doesn't exist - edit the cluster if it does exist. Uses a combination of If cluster doesn't exist:

Specified by:

upsertCluster in interface ClusterService

Parameters:

clusterDTO - cluster DTO

Throws:

IOException

DatabricksRestException

See Also:

https://docs.databricks.com/api/latest/clusters.html#create If cluster exists:, https://docs.databricks.com/api/latest/clusters.html#edit

findByName

public List<ClusterInfoDTO> findByName(String clusterName)
                                throws IOException,
                                       DatabricksRestException

Description copied from interface: ClusterService

Look for clusters with a given name.

Specified by:

findByName in interface ClusterService

Parameters:

clusterName - name to look for

Returns:

a list of cluster information objects

Throws:

IOException - any other errors

DatabricksRestException - any errors with the request

See Also:

https://docs.databricks.com/api/latest/clusters.html#findByName

findUniqueByName

public Optional<ClusterInfoDTO> findUniqueByName(String clusterName)
                                          throws IOException,
                                                 DatabricksRestException

Description copied from interface: ClusterService

Same as findByName, but it will throw an exception if more then one cluster has the name.

Specified by:

findUniqueByName in interface ClusterService

Parameters:

clusterName - name to look for

Returns:

a cluster if only one was found

Throws:

IOException - any other errors

DatabricksRestException - any errors with the request, including if more then 1 cluster was found with the name.

See Also:

https://docs.databricks.com/api/latest/clusters.html#findByName

pin

public void pin(String clusterId)
         throws IOException,
                DatabricksRestException

Description copied from interface: ClusterService

Pin a cluster with a given id. Needs administrator rights.

Specified by:

pin in interface ClusterService

Parameters:

clusterId - the clusterId you want to pin

Throws:

IOException - any other errors

DatabricksRestException - any errors with the request

See Also:

https://docs.databricks.com/api/latest/clusters.html#pin

unpin

public void unpin(String clusterId)
           throws IOException,
                  DatabricksRestException

Description copied from interface: ClusterService

Unpin a cluster with a given id. Needs administrator rights.

Specified by:

unpin in interface ClusterService

Parameters:

clusterId - the clusterId you want to unpin

Throws:

IOException - any other errors

DatabricksRestException - any errors with the request

See Also:

https://docs.databricks.com/api/latest/clusters.html#unpin

listNodeTypes

public List<NodeTypeDTO> listNodeTypes()
                                throws IOException,
                                       DatabricksRestException

Description copied from interface: ClusterService

Return a list of supported Spark node types.

Specified by:

listNodeTypes in interface ClusterService

Returns:

an array of node types

Throws:

IOException - any other errors

DatabricksRestException - any errors with the request

See Also:

https://docs.databricks.com/api/latest/clusters.html#list-node-types

listZones

public List<String> listZones()
                       throws IOException,
                              DatabricksRestException

Description copied from interface: ClusterService

Return a list of availability zones where clusters can be created in. This method only works on AWS.

Specified by:

listZones in interface ClusterService

Returns:

the a list of availability zones

Throws:

IOException - any other errors

DatabricksRestException - any errors with the request

See Also:

https://docs.databricks.com/api/latest/clusters.html#list-zones

listSparkVersions

public List<SparkVersionDTO> listSparkVersions()
                                        throws IOException,
                                               DatabricksRestException

Description copied from interface: ClusterService

Return the list of available runtime versions.

Specified by:

listSparkVersions in interface ClusterService

Returns:

the a list of runtime versions

Throws:

IOException - any other errors

DatabricksRestException - any errors with the request

See Also:

https://docs.databricks.com/api/latest/clusters.html#spark-versions