com.edmunds.rest.databricks.service

Interface ClusterService

All Known Implementing Classes:

ClusterServiceImpl

public interface ClusterService

A Wrapper around the cluster part of the databricks rest api.

See Also:

https://docs.databricks.com/api/latest/clusters.html

Method Summary

Modifier and Type	Method and Description
String	create(CreateClusterRequest createClusterRequest) Deprecated. in version 2.3.2, please use create(NewClusterDTO)
String	create(NewClusterDTO clusterDTO) Creates a databricks cluster.
void	delete(String clusterId) Terminates a cluster.
void	edit(EditClusterRequest editClusterRequest) Deprecated. in version 2.3.2, please use edit(UpsertClusterDTO)
void	edit(UpsertClusterDTO clusterDTO) Edits the configurations of a databricks cluster.
List<ClusterInfoDTO>	findByName(String clusterName) Look for clusters with a given name.
Optional<ClusterInfoDTO>	findUniqueByName(String clusterName) Same as findByName, but it will throw an exception if more then one cluster has the name.
ClusterInfoDTO	getInfo(String clusterId) Gets information about a given cluster.
boolean	isClusterRunning(String clusterId) Returns true if the cluster is running.
ClusterInfoDTO[]	list() Lists all of the clusters on a given databricks instance.
List<ClusterEventDTO>	listEvents(String clusterId, ClusterEventTypeDTO[] eventsToFilter, int offset, int limit) Lists events of a specific cluster on a given databricks instance.
List<NodeTypeDTO>	listNodeTypes() Return a list of supported Spark node types.
List<SparkVersionDTO>	listSparkVersions() Return the list of available runtime versions.
List<String>	listZones() Return a list of availability zones where clusters can be created in.
void	permanentDelete(String clusterId) Permanently terminates a cluster.
void	pin(String clusterId) Pin a cluster with a given id.
void	resize(AutoScaleDTO autoscale, String clusterId) Resizes a cluster.
void	resize(int numWorkers, String clusterId) Resizes a cluster.
void	restart(String clusterId) Restarts a given databricks cluster.
void	start(String clusterId) Starts a databricks cluster.
void	unpin(String clusterId) Unpin a cluster with a given id.
void	upsertCluster(NewClusterDTO clusterDTO) Given a cluster settings DTO object it will: - create the cluster if it doesn't exist - edit the cluster if it does exist.

Method Detail

create

@Deprecated
String create(CreateClusterRequest createClusterRequest)
                   throws IOException,
                          DatabricksRestException

Deprecated. in version 2.3.2, please use create(NewClusterDTO)

Creates a databricks cluster.

Parameters:

createClusterRequest - the cluster request object

Returns:

the clusterId

Throws:

IOException - other connection errors

DatabricksRestException - any errors with the request

See Also:

https://docs.databricks.com/api/latest/clusters.html#create

create

String create(NewClusterDTO clusterDTO)
       throws IOException,
              DatabricksRestException

Creates a databricks cluster.

Parameters:

clusterDTO - cluster DTO

Returns:

the clusterId

Throws:

IOException - other connection errors

DatabricksRestException - any errors with the request

See Also:

https://docs.databricks.com/api/latest/clusters.html#create

edit

@Deprecated
void edit(EditClusterRequest editClusterRequest)
               throws IOException,
                      DatabricksRestException

Deprecated. in version 2.3.2, please use edit(UpsertClusterDTO)

Edits the configurations of a databricks cluster.

Parameters:

editClusterRequest - the edit cluster request object

Throws:

IOException - any other errors

DatabricksRestException - any errors with the request

See Also:

https://docs.databricks.com/api/latest/clusters.html#edit

edit

void edit(UpsertClusterDTO clusterDTO)
   throws IOException,
          DatabricksRestException

Edits the configurations of a databricks cluster.

Parameters:

clusterDTO - cluster DTO

Throws:

IOException - any other errors

DatabricksRestException - any errors with the request

See Also:

https://docs.databricks.com/api/latest/clusters.html#edit

start

void start(String clusterId)
    throws IOException,
           DatabricksRestException

Starts a databricks cluster.

Parameters:

clusterId - the clusterId of the cluster you want to start.

Throws:

IOException - any other errors

DatabricksRestException - any errors with the request

See Also:

https://docs.databricks.com/api/latest/clusters.html#start

isClusterRunning

boolean isClusterRunning(String clusterId)
                  throws IOException,
                         DatabricksRestException

Returns true if the cluster is running.

Parameters:

clusterId - the clusterId to check.

Returns:

true if the cluster is running.

Throws:

IOException - any other errors

DatabricksRestException - any errors with the request

restart

void restart(String clusterId)
      throws IOException,
             DatabricksRestException

Restarts a given databricks cluster.

Parameters:

clusterId - the clusterid of the cluster you want to restart.

Throws:

IOException - any other errors

DatabricksRestException - any errors with the request

See Also:

https://docs.databricks.com/api/latest/clusters.html#restart

resize

void resize(int numWorkers,
            String clusterId)
     throws IOException,
            DatabricksRestException

Resizes a cluster. Will give it a fixed number of workers.

Parameters:

numWorkers - the number of workers you want to resize to

clusterId - the clusterId you want to operate on

Throws:

IOException - any other errors

DatabricksRestException - any errors with request

See Also:

https://docs.databricks.com/api/latest/clusters.html#resize

resize

void resize(AutoScaleDTO autoscale,
            String clusterId)
     throws IOException,
            DatabricksRestException

Resizes a cluster. Will add/edit autoscale functionality.

Parameters:

autoscale - the autoscale specs.

clusterId - the clusterId you want to work on

Throws:

IOException - any other errors

DatabricksRestException - any errors with request

See Also:

https://docs.databricks.com/api/latest/clusters.html#resize

delete

void delete(String clusterId)
     throws IOException,
            DatabricksRestException

Terminates a cluster.

Parameters:

clusterId - the cluster you want to terminate

Throws:

IOException - any other errors

DatabricksRestException - errors with request

See Also:

https://docs.databricks.com/api/latest/clusters.html#delete-terminate

permanentDelete

void permanentDelete(String clusterId)
              throws IOException,
                     DatabricksRestException

Permanently terminates a cluster.

Parameters:

clusterId - the cluster you want to permanently terminate

Throws:

IOException - any other errors

DatabricksRestException - errors with request

See Also:

https://docs.databricks.com/api/latest/clusters.html#permanent-delete

getInfo

ClusterInfoDTO getInfo(String clusterId)
                throws IOException,
                       DatabricksRestException

Gets information about a given cluster.

Parameters:

clusterId - the cluster you want to get info about

Returns:

the information DTO object

Throws:

IOException - any other errors.

DatabricksRestException - any errors with the request

See Also:

https://docs.databricks.com/api/latest/clusters.html#get

list

ClusterInfoDTO[] list()
               throws IOException,
                      DatabricksRestException

Lists all of the clusters on a given databricks instance.

Returns:

an array of cluster information objects

Throws:

IOException - any other errors

DatabricksRestException - any errors with the request

See Also:

https://docs.databricks.com/api/latest/clusters.html#list

listEvents

List<ClusterEventDTO> listEvents(String clusterId,
                                 ClusterEventTypeDTO[] eventsToFilter,
                                 int offset,
                                 int limit)
                          throws IOException,
                                 DatabricksRestException

Lists events of a specific cluster on a given databricks instance. Allows you to filter which events you want to see too.

Returns:

an array of cluster events objects

Throws:

IOException - any other errors

DatabricksRestException - any errors with the request

See Also:

https://docs.databricks.com/api/latest/clusters.html#events

upsertCluster

void upsertCluster(NewClusterDTO clusterDTO)
            throws IOException,
                   DatabricksRestException

Given a cluster settings DTO object it will: - create the cluster if it doesn't exist - edit the cluster if it does exist. Uses a combination of If cluster doesn't exist:

Parameters:

clusterDTO - cluster DTO

Throws:

IOException

DatabricksRestException

See Also:

https://docs.databricks.com/api/latest/clusters.html#create If cluster exists:, https://docs.databricks.com/api/latest/clusters.html#edit

findByName

List<ClusterInfoDTO> findByName(String clusterName)
                         throws IOException,
                                DatabricksRestException

Look for clusters with a given name.

Parameters:

clusterName - name to look for

Returns:

a list of cluster information objects

Throws:

IOException - any other errors

DatabricksRestException - any errors with the request

See Also:

https://docs.databricks.com/api/latest/clusters.html#findByName

findUniqueByName

Optional<ClusterInfoDTO> findUniqueByName(String clusterName)
                                   throws IOException,
                                          DatabricksRestException

Same as findByName, but it will throw an exception if more then one cluster has the name.

Parameters:

clusterName - name to look for

Returns:

a cluster if only one was found

Throws:

IOException - any other errors

DatabricksRestException - any errors with the request, including if more then 1 cluster was found with the name.

See Also:

https://docs.databricks.com/api/latest/clusters.html#findByName

pin

void pin(String clusterId)
  throws IOException,
         DatabricksRestException

Pin a cluster with a given id. Needs administrator rights.

Parameters:

clusterId - the clusterId you want to pin

Throws:

IOException - any other errors

DatabricksRestException - any errors with the request

See Also:

https://docs.databricks.com/api/latest/clusters.html#pin

unpin

void unpin(String clusterId)
    throws IOException,
           DatabricksRestException

Unpin a cluster with a given id. Needs administrator rights.

Parameters:

clusterId - the clusterId you want to unpin

Throws:

IOException - any other errors

DatabricksRestException - any errors with the request

See Also:

https://docs.databricks.com/api/latest/clusters.html#unpin

listNodeTypes

List<NodeTypeDTO> listNodeTypes()
                         throws IOException,
                                DatabricksRestException

Return a list of supported Spark node types.

Returns:

an array of node types

Throws:

IOException - any other errors

DatabricksRestException - any errors with the request

See Also:

https://docs.databricks.com/api/latest/clusters.html#list-node-types

listZones

List<String> listZones()
                throws IOException,
                       DatabricksRestException

Return a list of availability zones where clusters can be created in. This method only works on AWS.

Returns:

the a list of availability zones

Throws:

IOException - any other errors

DatabricksRestException - any errors with the request

See Also:

https://docs.databricks.com/api/latest/clusters.html#list-zones

listSparkVersions

List<SparkVersionDTO> listSparkVersions()
                                 throws IOException,
                                        DatabricksRestException

Return the list of available runtime versions.

Returns:

the a list of runtime versions

Throws:

IOException - any other errors

DatabricksRestException - any errors with the request

See Also:

https://docs.databricks.com/api/latest/clusters.html#spark-versions