Package com.google.cloud.hadoop.util

Class HadoopCredentialsConfiguration

java.lang.Object
com.google.cloud.hadoop.util.HadoopCredentialsConfiguration
public class HadoopCredentialsConfiguration extends Object
The Hadoop credentials configuration.

When reading configuration this class makes use of a list of key prefixes that are each applied to key suffixes to create a complete configuration key. There is a base prefix of 'google.cloud.' that is included by the builder for each configuration key suffix. When constructing, other prefixes can be specified. Prefixes specified later can be used to override the values of previously set values. In this way a set of global credentials can be specified for most connectors with an override specified for any connectors that need different credentials.

  • Field Details

    • BASE_KEY_PREFIX

      public static final String BASE_KEY_PREFIX
      All instances constructed using the builder will use google.cloud as the first prefix checked. Other prefixes can be added and will override values in the google.cloud prefix.
      See Also:

    • CLOUD_PLATFORM_SCOPE

      public static final String CLOUD_PLATFORM_SCOPE
      See Also:

    • AUTHENTICATION_TYPE_SUFFIX

      public static final HadoopConfigurationProperty<HadoopCredentialsConfiguration.AuthenticationType> AUTHENTICATION_TYPE_SUFFIX
      Key suffix used to configure authentication type.

    • SERVICE_ACCOUNT_JSON_KEYFILE_SUFFIX

      public static final HadoopConfigurationProperty<String> SERVICE_ACCOUNT_JSON_KEYFILE_SUFFIX
      Key suffix used to configure the path to a JSON file containing a Service Account key and identifier (email). Technically, this could be a JSON containing a non-service account user, but this setting is only used in the service account flow and is namespaced as such.

    • WORKLOAD_IDENTITY_FEDERATION_CREDENTIAL_CONFIG_FILE_SUFFIX

      public static final HadoopConfigurationProperty<String> WORKLOAD_IDENTITY_FEDERATION_CREDENTIAL_CONFIG_FILE_SUFFIX
      Key suffix used to configure the path to a JSON file containing a workload identity federation, i.e. external account credential configuration. Technically, this could be a JSON containing an service account impersonation url and credential source. but this setting is only used in the workload identity federation flow and is namespaced as such.

    • ACCESS_TOKEN_PROVIDER_SUFFIX

      public static final HadoopConfigurationProperty<Class<? extends AccessTokenProvider>> ACCESS_TOKEN_PROVIDER_SUFFIX
      Key suffix used to configure AccessTokenProvider that will be used to generate AccessTokenProvider.AccessTokens.

    • IMPERSONATION_SERVICE_ACCOUNT_SUFFIX

      public static final HadoopConfigurationProperty<String> IMPERSONATION_SERVICE_ACCOUNT_SUFFIX
      Key suffix used to configure the impersonating service account with which to call GCS API to get access token.

    • USER_IMPERSONATION_SERVICE_ACCOUNT_SUFFIX

      public static final HadoopConfigurationProperty<Map<String,String>> USER_IMPERSONATION_SERVICE_ACCOUNT_SUFFIX
      Key prefix for the user identifier associated with the service account to impersonate when accessing GCS.

    • GROUP_IMPERSONATION_SERVICE_ACCOUNT_SUFFIX

      public static final HadoopConfigurationProperty<Map<String,String>> GROUP_IMPERSONATION_SERVICE_ACCOUNT_SUFFIX
      Key prefix for the group identifier associated with the service account to impersonate when accessing GCS.

    • TOKEN_SERVER_URL_SUFFIX

      public static final HadoopConfigurationProperty<String> TOKEN_SERVER_URL_SUFFIX
      Key suffix for setting a token server URL to use to refresh OAuth token.

    • PROXY_ADDRESS_SUFFIX

      public static final HadoopConfigurationProperty<String> PROXY_ADDRESS_SUFFIX
      Key suffix for setting a proxy for the connector to use to connect to GCS. The proxy must be an HTTP proxy of the form "host:port".

    • PROXY_USERNAME_SUFFIX

      public static final HadoopConfigurationProperty<RedactedString> PROXY_USERNAME_SUFFIX
      Key suffix for setting a proxy username for the connector to use to authenticate with proxy used to connect to GCS.

    • PROXY_PASSWORD_SUFFIX

      public static final HadoopConfigurationProperty<RedactedString> PROXY_PASSWORD_SUFFIX
      Key suffix for setting a proxy password for the connector to use to authenticate with proxy used to connect to GCS.

    • READ_TIMEOUT_SUFFIX

      public static final HadoopConfigurationProperty<Long> READ_TIMEOUT_SUFFIX
      Key suffix for setting the read timeout for HTTP request.

    • AUTH_CLIENT_ID_SUFFIX

      public static final HadoopConfigurationProperty<String> AUTH_CLIENT_ID_SUFFIX
      Configuration key for defining the OAUth2 client ID. Required when the authentication type is USER_CREDENTIALS

    • AUTH_CLIENT_SECRET_SUFFIX

      public static final HadoopConfigurationProperty<RedactedString> AUTH_CLIENT_SECRET_SUFFIX
      Configuration key for defining the OAUth2 client secret. Required when the authentication type is USER_CREDENTIALS

    • AUTH_REFRESH_TOKEN_SUFFIX

      public static final HadoopConfigurationProperty<RedactedString> AUTH_REFRESH_TOKEN_SUFFIX
      Configuration key for defining the OAuth2 refresh token. Required when the authentication type is USER_CREDENTIALS

  • Constructor Details

    • HadoopCredentialsConfiguration

      protected HadoopCredentialsConfiguration()

  • Method Details

    • getConfigKeyPrefixes

      public static List<String> getConfigKeyPrefixes(String... keyPrefixes)
      Returns full list of config prefixes that will be resolved based on the order in returned list.

    • getCredentials

      public static com.google.auth.oauth2.GoogleCredentials getCredentials(org.apache.hadoop.conf.Configuration config, String... keyPrefixesVararg) throws IOException
      Get the credentials for the configured HadoopCredentialsConfiguration.AuthenticationType
      Throws:
      IllegalStateException - if configured HadoopCredentialsConfiguration.AuthenticationType is not recognized
      IOException

    • getImpersonatedCredentials

      public static com.google.auth.oauth2.GoogleCredentials getImpersonatedCredentials(org.apache.hadoop.conf.Configuration config, com.google.auth.oauth2.GoogleCredentials sourceCredentials, String... keyPrefixesVararg) throws IOException
      Create a ImpersonatedCredentials based on service account to impersonate configuration
      Throws:
      IOException