Package com.google.cloud.dataplex.v1

Class Task.SparkTaskConfig

java.lang.Object
com.google.protobuf.AbstractMessageLite
com.google.protobuf.AbstractMessage
com.google.protobuf.GeneratedMessage
com.google.cloud.dataplex.v1.Task.SparkTaskConfig
All Implemented Interfaces:
Task.SparkTaskConfigOrBuilder, com.google.protobuf.Message, com.google.protobuf.MessageLite, com.google.protobuf.MessageLiteOrBuilder, com.google.protobuf.MessageOrBuilder, Serializable
Enclosing class:
Task
public static final class Task.SparkTaskConfig extends com.google.protobuf.GeneratedMessage implements Task.SparkTaskConfigOrBuilder
 User-specified config for running a Spark task.
Protobuf type google.cloud.dataplex.v1.Task.SparkTaskConfig
See Also:

  • Field Details

    • MAIN_JAR_FILE_URI_FIELD_NUMBER

      public static final int MAIN_JAR_FILE_URI_FIELD_NUMBER
      See Also:

    • MAIN_CLASS_FIELD_NUMBER

      public static final int MAIN_CLASS_FIELD_NUMBER
      See Also:

    • PYTHON_SCRIPT_FILE_FIELD_NUMBER

      public static final int PYTHON_SCRIPT_FILE_FIELD_NUMBER
      See Also:

    • SQL_SCRIPT_FILE_FIELD_NUMBER

      public static final int SQL_SCRIPT_FILE_FIELD_NUMBER
      See Also:

    • SQL_SCRIPT_FIELD_NUMBER

      public static final int SQL_SCRIPT_FIELD_NUMBER
      See Also:

    • FILE_URIS_FIELD_NUMBER

      public static final int FILE_URIS_FIELD_NUMBER
      See Also:

    • ARCHIVE_URIS_FIELD_NUMBER

      public static final int ARCHIVE_URIS_FIELD_NUMBER
      See Also:

    • INFRASTRUCTURE_SPEC_FIELD_NUMBER

      public static final int INFRASTRUCTURE_SPEC_FIELD_NUMBER
      See Also:

  • Method Details

    • getDescriptor

      public static final com.google.protobuf.Descriptors.Descriptor getDescriptor()

    • internalGetFieldAccessorTable

      protected com.google.protobuf.GeneratedMessage.FieldAccessorTable internalGetFieldAccessorTable()
      Specified by:
      internalGetFieldAccessorTable in class com.google.protobuf.GeneratedMessage

    • getDriverCase

      public Task.SparkTaskConfig.DriverCase getDriverCase()
      Specified by:
      getDriverCase in interface Task.SparkTaskConfigOrBuilder

    • hasMainJarFileUri

      public boolean hasMainJarFileUri()
       The Cloud Storage URI of the jar file that contains the main class.
 The execution args are passed in as a sequence of named process
 arguments (`--key=value`).
      string main_jar_file_uri = 100;
      Specified by:
      hasMainJarFileUri in interface Task.SparkTaskConfigOrBuilder
      Returns:
      Whether the mainJarFileUri field is set.

    • getMainJarFileUri

      public String getMainJarFileUri()
       The Cloud Storage URI of the jar file that contains the main class.
 The execution args are passed in as a sequence of named process
 arguments (`--key=value`).
      string main_jar_file_uri = 100;
      Specified by:
      getMainJarFileUri in interface Task.SparkTaskConfigOrBuilder
      Returns:
      The mainJarFileUri.

    • getMainJarFileUriBytes

      public com.google.protobuf.ByteString getMainJarFileUriBytes()
       The Cloud Storage URI of the jar file that contains the main class.
 The execution args are passed in as a sequence of named process
 arguments (`--key=value`).
      string main_jar_file_uri = 100;
      Specified by:
      getMainJarFileUriBytes in interface Task.SparkTaskConfigOrBuilder
      Returns:
      The bytes for mainJarFileUri.

    • hasMainClass

      public boolean hasMainClass()
       The name of the driver's main class. The jar file that contains the
 class must be in the default CLASSPATH or specified in
 `jar_file_uris`.
 The execution args are passed in as a sequence of named process
 arguments (`--key=value`).
      string main_class = 101;
      Specified by:
      hasMainClass in interface Task.SparkTaskConfigOrBuilder
      Returns:
      Whether the mainClass field is set.

    • getMainClass

      public String getMainClass()
       The name of the driver's main class. The jar file that contains the
 class must be in the default CLASSPATH or specified in
 `jar_file_uris`.
 The execution args are passed in as a sequence of named process
 arguments (`--key=value`).
      string main_class = 101;
      Specified by:
      getMainClass in interface Task.SparkTaskConfigOrBuilder
      Returns:
      The mainClass.

    • getMainClassBytes

      public com.google.protobuf.ByteString getMainClassBytes()
       The name of the driver's main class. The jar file that contains the
 class must be in the default CLASSPATH or specified in
 `jar_file_uris`.
 The execution args are passed in as a sequence of named process
 arguments (`--key=value`).
      string main_class = 101;
      Specified by:
      getMainClassBytes in interface Task.SparkTaskConfigOrBuilder
      Returns:
      The bytes for mainClass.

    • hasPythonScriptFile

      public boolean hasPythonScriptFile()
       The Gcloud Storage URI of the main Python file to use as the driver.
 Must be a .py file. The execution args are passed in as a sequence of
 named process arguments (`--key=value`).
      string python_script_file = 102;
      Specified by:
      hasPythonScriptFile in interface Task.SparkTaskConfigOrBuilder
      Returns:
      Whether the pythonScriptFile field is set.

    • getPythonScriptFile

      public String getPythonScriptFile()
       The Gcloud Storage URI of the main Python file to use as the driver.
 Must be a .py file. The execution args are passed in as a sequence of
 named process arguments (`--key=value`).
      string python_script_file = 102;
      Specified by:
      getPythonScriptFile in interface Task.SparkTaskConfigOrBuilder
      Returns:
      The pythonScriptFile.

    • getPythonScriptFileBytes

      public com.google.protobuf.ByteString getPythonScriptFileBytes()
       The Gcloud Storage URI of the main Python file to use as the driver.
 Must be a .py file. The execution args are passed in as a sequence of
 named process arguments (`--key=value`).
      string python_script_file = 102;
      Specified by:
      getPythonScriptFileBytes in interface Task.SparkTaskConfigOrBuilder
      Returns:
      The bytes for pythonScriptFile.

    • hasSqlScriptFile

      public boolean hasSqlScriptFile()
       A reference to a query file. This should be the Cloud Storage URI of
 the query file. The execution args are used to declare a set of script
 variables (`set key="value";`).
      string sql_script_file = 104;
      Specified by:
      hasSqlScriptFile in interface Task.SparkTaskConfigOrBuilder
      Returns:
      Whether the sqlScriptFile field is set.

    • getSqlScriptFile

      public String getSqlScriptFile()
       A reference to a query file. This should be the Cloud Storage URI of
 the query file. The execution args are used to declare a set of script
 variables (`set key="value";`).
      string sql_script_file = 104;
      Specified by:
      getSqlScriptFile in interface Task.SparkTaskConfigOrBuilder
      Returns:
      The sqlScriptFile.

    • getSqlScriptFileBytes

      public com.google.protobuf.ByteString getSqlScriptFileBytes()
       A reference to a query file. This should be the Cloud Storage URI of
 the query file. The execution args are used to declare a set of script
 variables (`set key="value";`).
      string sql_script_file = 104;
      Specified by:
      getSqlScriptFileBytes in interface Task.SparkTaskConfigOrBuilder
      Returns:
      The bytes for sqlScriptFile.

    • hasSqlScript

      public boolean hasSqlScript()
       The query text.
 The execution args are used to declare a set of script variables
 (`set key="value";`).
      string sql_script = 105;
      Specified by:
      hasSqlScript in interface Task.SparkTaskConfigOrBuilder
      Returns:
      Whether the sqlScript field is set.

    • getSqlScript

      public String getSqlScript()
       The query text.
 The execution args are used to declare a set of script variables
 (`set key="value";`).
      string sql_script = 105;
      Specified by:
      getSqlScript in interface Task.SparkTaskConfigOrBuilder
      Returns:
      The sqlScript.

    • getSqlScriptBytes

      public com.google.protobuf.ByteString getSqlScriptBytes()
       The query text.
 The execution args are used to declare a set of script variables
 (`set key="value";`).
      string sql_script = 105;
      Specified by:
      getSqlScriptBytes in interface Task.SparkTaskConfigOrBuilder
      Returns:
      The bytes for sqlScript.

    • getFileUrisList

      public com.google.protobuf.ProtocolStringList getFileUrisList()
       Optional. Cloud Storage URIs of files to be placed in the working
 directory of each executor.
      repeated string file_uris = 3 [(.google.api.field_behavior) = OPTIONAL];
      Specified by:
      getFileUrisList in interface Task.SparkTaskConfigOrBuilder
      Returns:
      A list containing the fileUris.

    • getFileUrisCount

      public int getFileUrisCount()
       Optional. Cloud Storage URIs of files to be placed in the working
 directory of each executor.
      repeated string file_uris = 3 [(.google.api.field_behavior) = OPTIONAL];
      Specified by:
      getFileUrisCount in interface Task.SparkTaskConfigOrBuilder
      Returns:
      The count of fileUris.

    • getFileUris

      public String getFileUris(int index)
       Optional. Cloud Storage URIs of files to be placed in the working
 directory of each executor.
      repeated string file_uris = 3 [(.google.api.field_behavior) = OPTIONAL];
      Specified by:
      getFileUris in interface Task.SparkTaskConfigOrBuilder
      Parameters:
      index - The index of the element to return.
      Returns:
      The fileUris at the given index.

    • getFileUrisBytes

      public com.google.protobuf.ByteString getFileUrisBytes(int index)
       Optional. Cloud Storage URIs of files to be placed in the working
 directory of each executor.
      repeated string file_uris = 3 [(.google.api.field_behavior) = OPTIONAL];
      Specified by:
      getFileUrisBytes in interface Task.SparkTaskConfigOrBuilder
      Parameters:
      index - The index of the value to return.
      Returns:
      The bytes of the fileUris at the given index.

    • getArchiveUrisList

      public com.google.protobuf.ProtocolStringList getArchiveUrisList()
       Optional. Cloud Storage URIs of archives to be extracted into the working
 directory of each executor. Supported file types: .jar, .tar, .tar.gz,
 .tgz, and .zip.
      repeated string archive_uris = 4 [(.google.api.field_behavior) = OPTIONAL];
      Specified by:
      getArchiveUrisList in interface Task.SparkTaskConfigOrBuilder
      Returns:
      A list containing the archiveUris.

    • getArchiveUrisCount

      public int getArchiveUrisCount()
       Optional. Cloud Storage URIs of archives to be extracted into the working
 directory of each executor. Supported file types: .jar, .tar, .tar.gz,
 .tgz, and .zip.
      repeated string archive_uris = 4 [(.google.api.field_behavior) = OPTIONAL];
      Specified by:
      getArchiveUrisCount in interface Task.SparkTaskConfigOrBuilder
      Returns:
      The count of archiveUris.

    • getArchiveUris

      public String getArchiveUris(int index)
       Optional. Cloud Storage URIs of archives to be extracted into the working
 directory of each executor. Supported file types: .jar, .tar, .tar.gz,
 .tgz, and .zip.
      repeated string archive_uris = 4 [(.google.api.field_behavior) = OPTIONAL];
      Specified by:
      getArchiveUris in interface Task.SparkTaskConfigOrBuilder
      Parameters:
      index - The index of the element to return.
      Returns:
      The archiveUris at the given index.

    • getArchiveUrisBytes

      public com.google.protobuf.ByteString getArchiveUrisBytes(int index)
       Optional. Cloud Storage URIs of archives to be extracted into the working
 directory of each executor. Supported file types: .jar, .tar, .tar.gz,
 .tgz, and .zip.
      repeated string archive_uris = 4 [(.google.api.field_behavior) = OPTIONAL];
      Specified by:
      getArchiveUrisBytes in interface Task.SparkTaskConfigOrBuilder
      Parameters:
      index - The index of the value to return.
      Returns:
      The bytes of the archiveUris at the given index.

    • hasInfrastructureSpec

      public boolean hasInfrastructureSpec()
       Optional. Infrastructure specification for the execution.
      .google.cloud.dataplex.v1.Task.InfrastructureSpec infrastructure_spec = 6 [(.google.api.field_behavior) = OPTIONAL];
      Specified by:
      hasInfrastructureSpec in interface Task.SparkTaskConfigOrBuilder
      Returns:
      Whether the infrastructureSpec field is set.

    • getInfrastructureSpec

      public Task.InfrastructureSpec getInfrastructureSpec()
       Optional. Infrastructure specification for the execution.
      .google.cloud.dataplex.v1.Task.InfrastructureSpec infrastructure_spec = 6 [(.google.api.field_behavior) = OPTIONAL];
      Specified by:
      getInfrastructureSpec in interface Task.SparkTaskConfigOrBuilder
      Returns:
      The infrastructureSpec.

    • getInfrastructureSpecOrBuilder

      public Task.InfrastructureSpecOrBuilder getInfrastructureSpecOrBuilder()
       Optional. Infrastructure specification for the execution.
      .google.cloud.dataplex.v1.Task.InfrastructureSpec infrastructure_spec = 6 [(.google.api.field_behavior) = OPTIONAL];
      Specified by:
      getInfrastructureSpecOrBuilder in interface Task.SparkTaskConfigOrBuilder

    • isInitialized

      public final boolean isInitialized()
      Specified by:
      isInitialized in interface com.google.protobuf.MessageLiteOrBuilder
      Overrides:
      isInitialized in class com.google.protobuf.GeneratedMessage

    • writeTo

      public void writeTo(com.google.protobuf.CodedOutputStream output) throws IOException
      Specified by:
      writeTo in interface com.google.protobuf.MessageLite
      Overrides:
      writeTo in class com.google.protobuf.GeneratedMessage
      Throws:
      IOException

    • getSerializedSize

      public int getSerializedSize()
      Specified by:
      getSerializedSize in interface com.google.protobuf.MessageLite
      Overrides:
      getSerializedSize in class com.google.protobuf.GeneratedMessage

    • equals

      public boolean equals(Object obj)
      Specified by:
      equals in interface com.google.protobuf.Message
      Overrides:
      equals in class com.google.protobuf.AbstractMessage

    • hashCode

      public int hashCode()
      Specified by:
      hashCode in interface com.google.protobuf.Message
      Overrides:
      hashCode in class com.google.protobuf.AbstractMessage

    • parseFrom

      public static Task.SparkTaskConfig parseFrom(ByteBuffer data) throws com.google.protobuf.InvalidProtocolBufferException
      Throws:
      com.google.protobuf.InvalidProtocolBufferException

    • parseFrom

      public static Task.SparkTaskConfig parseFrom(ByteBuffer data, com.google.protobuf.ExtensionRegistryLite extensionRegistry) throws com.google.protobuf.InvalidProtocolBufferException
      Throws:
      com.google.protobuf.InvalidProtocolBufferException

    • parseFrom

      public static Task.SparkTaskConfig parseFrom(com.google.protobuf.ByteString data) throws com.google.protobuf.InvalidProtocolBufferException
      Throws:
      com.google.protobuf.InvalidProtocolBufferException

    • parseFrom

      public static Task.SparkTaskConfig parseFrom(com.google.protobuf.ByteString data, com.google.protobuf.ExtensionRegistryLite extensionRegistry) throws com.google.protobuf.InvalidProtocolBufferException
      Throws:
      com.google.protobuf.InvalidProtocolBufferException

    • parseFrom

      public static Task.SparkTaskConfig parseFrom(byte[] data) throws com.google.protobuf.InvalidProtocolBufferException
      Throws:
      com.google.protobuf.InvalidProtocolBufferException

    • parseFrom

      public static Task.SparkTaskConfig parseFrom(byte[] data, com.google.protobuf.ExtensionRegistryLite extensionRegistry) throws com.google.protobuf.InvalidProtocolBufferException
      Throws:
      com.google.protobuf.InvalidProtocolBufferException

    • parseFrom

      public static Task.SparkTaskConfig parseFrom(InputStream input) throws IOException
      Throws:
      IOException

    • parseFrom

      public static Task.SparkTaskConfig parseFrom(InputStream input, com.google.protobuf.ExtensionRegistryLite extensionRegistry) throws IOException
      Throws:
      IOException

    • parseDelimitedFrom

      public static Task.SparkTaskConfig parseDelimitedFrom(InputStream input) throws IOException
      Throws:
      IOException

    • parseDelimitedFrom

      public static Task.SparkTaskConfig parseDelimitedFrom(InputStream input, com.google.protobuf.ExtensionRegistryLite extensionRegistry) throws IOException
      Throws:
      IOException

    • parseFrom

      public static Task.SparkTaskConfig parseFrom(com.google.protobuf.CodedInputStream input) throws IOException
      Throws:
      IOException

    • parseFrom

      public static Task.SparkTaskConfig parseFrom(com.google.protobuf.CodedInputStream input, com.google.protobuf.ExtensionRegistryLite extensionRegistry) throws IOException
      Throws:
      IOException

    • newBuilderForType

      public Task.SparkTaskConfig.Builder newBuilderForType()
      Specified by:
      newBuilderForType in interface com.google.protobuf.Message
      Specified by:
      newBuilderForType in interface com.google.protobuf.MessageLite

    • newBuilder

      public static Task.SparkTaskConfig.Builder newBuilder()

    • newBuilder

      public static Task.SparkTaskConfig.Builder newBuilder(Task.SparkTaskConfig prototype)

    • toBuilder

      public Task.SparkTaskConfig.Builder toBuilder()
      Specified by:
      toBuilder in interface com.google.protobuf.Message
      Specified by:
      toBuilder in interface com.google.protobuf.MessageLite

    • newBuilderForType

      protected Task.SparkTaskConfig.Builder newBuilderForType(com.google.protobuf.AbstractMessage.BuilderParent parent)
      Overrides:
      newBuilderForType in class com.google.protobuf.AbstractMessage

    • getDefaultInstance

      public static Task.SparkTaskConfig getDefaultInstance()

    • parser

      public static com.google.protobuf.Parser<Task.SparkTaskConfig> parser()

    • getParserForType

      public com.google.protobuf.Parser<Task.SparkTaskConfig> getParserForType()
      Specified by:
      getParserForType in interface com.google.protobuf.Message
      Specified by:
      getParserForType in interface com.google.protobuf.MessageLite
      Overrides:
      getParserForType in class com.google.protobuf.GeneratedMessage

    • getDefaultInstanceForType

      public Task.SparkTaskConfig getDefaultInstanceForType()
      Specified by:
      getDefaultInstanceForType in interface com.google.protobuf.MessageLiteOrBuilder
      Specified by:
      getDefaultInstanceForType in interface com.google.protobuf.MessageOrBuilder