Package org.apache.flink.table.factories

Class FactoryUtil

java.lang.Object

org.apache.flink.table.factories.FactoryUtil

@PublicEvolving
public final class FactoryUtil
extends Object

Utility for working with Factorys.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	FactoryUtil.CatalogFactoryHelper	Helper utility for validating all options for a CatalogFactory.
static class	FactoryUtil.CatalogStoreFactoryHelper	Helper utility for validating all options for a CatalogStoreFactory.
static class	FactoryUtil.DefaultCatalogContext	Default implementation of CatalogFactory.Context.
static class	FactoryUtil.DefaultCatalogStoreContext	Default implementation of CatalogStoreFactory.Context.
static class	FactoryUtil.DefaultDynamicTableContext	Default implementation of DynamicTableFactory.Context.
static class	FactoryUtil.DefaultModuleContext	Default implementation of ModuleFactory.Context.
static class	FactoryUtil.FactoryHelper<F extends Factory>	Base helper utility for validating all options for a Factory.
static class	FactoryUtil.ModuleFactoryHelper	Helper utility for validating all options for a ModuleFactory.
static class	FactoryUtil.TableFactoryHelper	Helper utility for discovering formats and validating all options for a DynamicTableFactory.

Field Summary

Fields

Modifier and Type	Field	Description
static org.apache.flink.configuration.ConfigOption<String>	CONNECTOR
static org.apache.flink.configuration.ConfigOption<String>	FORMAT
static String	FORMAT_SUFFIX	Suffix for keys of ConfigOption in case a connector requires multiple formats (e.g.
static String	PLACEHOLDER_SYMBOL	The placeholder symbol to be used for keys of options which can be templated.
static org.apache.flink.configuration.ConfigOption<Integer>	PROPERTY_VERSION	Describes the property version.
static org.apache.flink.configuration.ConfigOption<Integer>	SINK_PARALLELISM
static org.apache.flink.configuration.ConfigOption<Duration>	SOURCE_IDLE_TIMEOUT
static org.apache.flink.configuration.ConfigOption<Integer>	SOURCE_PARALLELISM
static org.apache.flink.configuration.ConfigOption<List<String>>	SQL_GATEWAY_ENDPOINT_TYPE
static org.apache.flink.configuration.ConfigOption<String>	WATERMARK_ALIGNMENT_GROUP
static org.apache.flink.configuration.ConfigOption<Duration>	WATERMARK_ALIGNMENT_MAX_DRIFT
static org.apache.flink.configuration.ConfigOption<Duration>	WATERMARK_ALIGNMENT_UPDATE_INTERVAL
static org.apache.flink.configuration.ConfigOption<WatermarkEmitStrategy>	WATERMARK_EMIT_STRATEGY
static org.apache.flink.configuration.ConfigOption<String>	WORKFLOW_SCHEDULER_TYPE

Method Summary

Modifier and Type	Method	Description
static Optional<String>	checkWatermarkOptions(org.apache.flink.configuration.ReadableConfig conf)	Check watermark-related options and return error messages.
static Catalog	createCatalog(String catalogName, Map<String,String> options, org.apache.flink.configuration.ReadableConfig configuration, ClassLoader classLoader)	Attempts to discover an appropriate catalog factory and creates an instance of the catalog.
static FactoryUtil.CatalogFactoryHelper	createCatalogFactoryHelper(CatalogFactory factory, CatalogFactory.Context context)	Creates a utility that helps validating options for a CatalogFactory.
static FactoryUtil.CatalogStoreFactoryHelper	createCatalogStoreFactoryHelper(CatalogStoreFactory factory, CatalogStoreFactory.Context context)	Creates a utility that helps validating options for a CatalogStoreFactory.
static DynamicTableSink	createDynamicTableSink(DynamicTableSinkFactory preferredFactory, ObjectIdentifier objectIdentifier, ResolvedCatalogTable catalogTable, Map<String,String> enrichmentOptions, org.apache.flink.configuration.ReadableConfig configuration, ClassLoader classLoader, boolean isTemporary)	Creates a DynamicTableSink from a CatalogTable.
static DynamicTableSink	createDynamicTableSink(DynamicTableSinkFactory preferredFactory, ObjectIdentifier objectIdentifier, ResolvedCatalogTable catalogTable, org.apache.flink.configuration.ReadableConfig configuration, ClassLoader classLoader, boolean isTemporary)	Deprecated. Use createDynamicTableSink(DynamicTableSinkFactory, ObjectIdentifier, ResolvedCatalogTable, Map, ReadableConfig, ClassLoader, boolean)
static DynamicTableSource	createDynamicTableSource(DynamicTableSourceFactory preferredFactory, ObjectIdentifier objectIdentifier, ResolvedCatalogTable catalogTable, Map<String,String> enrichmentOptions, org.apache.flink.configuration.ReadableConfig configuration, ClassLoader classLoader, boolean isTemporary)	Creates a DynamicTableSource from a CatalogTable.
static DynamicTableSource	createDynamicTableSource(DynamicTableSourceFactory preferredFactory, ObjectIdentifier objectIdentifier, ResolvedCatalogTable catalogTable, org.apache.flink.configuration.ReadableConfig configuration, ClassLoader classLoader, boolean isTemporary)	Deprecated. Use createDynamicTableSource(DynamicTableSourceFactory, ObjectIdentifier, ResolvedCatalogTable, Map, ReadableConfig, ClassLoader, boolean)
static Module	createModule(String moduleName, Map<String,String> options, org.apache.flink.configuration.ReadableConfig configuration, ClassLoader classLoader)	Discovers a matching module factory and creates an instance of it.
static FactoryUtil.ModuleFactoryHelper	createModuleFactoryHelper(ModuleFactory factory, ModuleFactory.Context context)	Creates a utility that helps validating options for a ModuleFactory.
static FactoryUtil.TableFactoryHelper	createTableFactoryHelper(DynamicTableFactory factory, DynamicTableFactory.Context context)	Creates a utility that helps in discovering formats, merging options with DynamicTableFactory.Context.getEnrichmentOptions() and validating them all for a DynamicTableFactory.
static DynamicTableSink	createTableSink(Catalog catalog, ObjectIdentifier objectIdentifier, ResolvedCatalogTable catalogTable, org.apache.flink.configuration.ReadableConfig configuration, ClassLoader classLoader, boolean isTemporary)	Deprecated. Use createDynamicTableSink(DynamicTableSinkFactory, ObjectIdentifier, ResolvedCatalogTable, Map, ReadableConfig, ClassLoader, boolean) instead.
static DynamicTableSource	createTableSource(Catalog catalog, ObjectIdentifier objectIdentifier, ResolvedCatalogTable catalogTable, org.apache.flink.configuration.ReadableConfig configuration, ClassLoader classLoader, boolean isTemporary)	Deprecated. Use createDynamicTableSource(DynamicTableSourceFactory, ObjectIdentifier, ResolvedCatalogTable, Map, ReadableConfig, ClassLoader, boolean) instead.
static <T extends Factory> T	discoverFactory(ClassLoader classLoader, Class<T> factoryClass, String factoryIdentifier)	Discovers a factory using the given factory base class and identifier.
static <T extends DynamicTableFactory> Optional<T>	getDynamicTableFactory(Class<T> factoryClass, Catalog catalog)	Returns the DynamicTableFactory via Catalog.
static String	getFormatPrefix(org.apache.flink.configuration.ConfigOption<String> formatOption, String formatIdentifier)	Returns the required option prefix for options of the given format.
static String	stringifyOption(String key, String value)
static void	validateFactoryOptions(Set<org.apache.flink.configuration.ConfigOption<?>> requiredOptions, Set<org.apache.flink.configuration.ConfigOption<?>> optionalOptions, org.apache.flink.configuration.ReadableConfig options)	Validates the required options and optional options.
static void	validateFactoryOptions(Factory factory, org.apache.flink.configuration.ReadableConfig options)	Validates the required and optional ConfigOptions of a factory.
static void	validateUnconsumedKeys(String factoryIdentifier, Set<String> allOptionKeys, Set<String> consumedOptionKeys)	Validates unconsumed option keys.
static void	validateUnconsumedKeys(String factoryIdentifier, Set<String> allOptionKeys, Set<String> consumedOptionKeys, Set<String> deprecatedOptionKeys)	Validates unconsumed option keys.
static void	validateWatermarkOptions(String factoryIdentifier, org.apache.flink.configuration.ReadableConfig conf)	Validate watermark options from table options.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

PROPERTY_VERSION

public static final org.apache.flink.configuration.ConfigOption<Integer> PROPERTY_VERSION

Describes the property version. This can be used for backwards compatibility in case the property format changes.

CONNECTOR

public static final org.apache.flink.configuration.ConfigOption<String> CONNECTOR

FORMAT

public static final org.apache.flink.configuration.ConfigOption<String> FORMAT

SINK_PARALLELISM

public static final org.apache.flink.configuration.ConfigOption<Integer> SINK_PARALLELISM

SQL_GATEWAY_ENDPOINT_TYPE

public static final org.apache.flink.configuration.ConfigOption<List<String>> SQL_GATEWAY_ENDPOINT_TYPE

SOURCE_PARALLELISM

public static final org.apache.flink.configuration.ConfigOption<Integer> SOURCE_PARALLELISM

WATERMARK_EMIT_STRATEGY

public static final org.apache.flink.configuration.ConfigOption<WatermarkEmitStrategy> WATERMARK_EMIT_STRATEGY

WATERMARK_ALIGNMENT_GROUP

public static final org.apache.flink.configuration.ConfigOption<String> WATERMARK_ALIGNMENT_GROUP

WATERMARK_ALIGNMENT_MAX_DRIFT

public static final org.apache.flink.configuration.ConfigOption<Duration> WATERMARK_ALIGNMENT_MAX_DRIFT

WATERMARK_ALIGNMENT_UPDATE_INTERVAL

public static final org.apache.flink.configuration.ConfigOption<Duration> WATERMARK_ALIGNMENT_UPDATE_INTERVAL

SOURCE_IDLE_TIMEOUT

public static final org.apache.flink.configuration.ConfigOption<Duration> SOURCE_IDLE_TIMEOUT

WORKFLOW_SCHEDULER_TYPE

public static final org.apache.flink.configuration.ConfigOption<String> WORKFLOW_SCHEDULER_TYPE

FORMAT_SUFFIX

public static final String FORMAT_SUFFIX

Suffix for keys of ConfigOption in case a connector requires multiple formats (e.g. for both key and value).

See createTableFactoryHelper(DynamicTableFactory, DynamicTableFactory.Context) for more information.

See Also:

Constant Field Values

PLACEHOLDER_SYMBOL

public static final String PLACEHOLDER_SYMBOL

The placeholder symbol to be used for keys of options which can be templated. See Factory for details.

See Also:

Constant Field Values

Method Detail

createDynamicTableSource

public static DynamicTableSource createDynamicTableSource(@Nullable
                                                          DynamicTableSourceFactory preferredFactory,
                                                          ObjectIdentifier objectIdentifier,
                                                          ResolvedCatalogTable catalogTable,
                                                          Map<String,String> enrichmentOptions,
                                                          org.apache.flink.configuration.ReadableConfig configuration,
                                                          ClassLoader classLoader,
                                                          boolean isTemporary)

Creates a DynamicTableSource from a CatalogTable.

If {@param preferredFactory} is passed, the table source is created from that factory. Otherwise, an attempt is made to discover a matching factory using Java SPI (see Factory for details).

createDynamicTableSource

@Deprecated
public static DynamicTableSource createDynamicTableSource(@Nullable
                                                          DynamicTableSourceFactory preferredFactory,
                                                          ObjectIdentifier objectIdentifier,
                                                          ResolvedCatalogTable catalogTable,
                                                          org.apache.flink.configuration.ReadableConfig configuration,
                                                          ClassLoader classLoader,
                                                          boolean isTemporary)

Deprecated.

Use createDynamicTableSource(DynamicTableSourceFactory, ObjectIdentifier, ResolvedCatalogTable, Map, ReadableConfig, ClassLoader, boolean)

createTableSource

@Deprecated
public static DynamicTableSource createTableSource(@Nullable
                                                   Catalog catalog,
                                                   ObjectIdentifier objectIdentifier,
                                                   ResolvedCatalogTable catalogTable,
                                                   org.apache.flink.configuration.ReadableConfig configuration,
                                                   ClassLoader classLoader,
                                                   boolean isTemporary)

Deprecated.

Use createDynamicTableSource(DynamicTableSourceFactory, ObjectIdentifier, ResolvedCatalogTable, Map, ReadableConfig, ClassLoader, boolean) instead.

Creates a DynamicTableSource from a CatalogTable.

It considers Catalog.getFactory() if provided.

createDynamicTableSink

public static DynamicTableSink createDynamicTableSink(@Nullable
                                                      DynamicTableSinkFactory preferredFactory,
                                                      ObjectIdentifier objectIdentifier,
                                                      ResolvedCatalogTable catalogTable,
                                                      Map<String,String> enrichmentOptions,
                                                      org.apache.flink.configuration.ReadableConfig configuration,
                                                      ClassLoader classLoader,
                                                      boolean isTemporary)

Creates a DynamicTableSink from a CatalogTable.

If {@param preferredFactory} is passed, the table sink is created from that factory. Otherwise, an attempt is made to discover a matching factory using Java SPI (see Factory for details).

createDynamicTableSink

@Deprecated
public static DynamicTableSink createDynamicTableSink(@Nullable
                                                      DynamicTableSinkFactory preferredFactory,
                                                      ObjectIdentifier objectIdentifier,
                                                      ResolvedCatalogTable catalogTable,
                                                      org.apache.flink.configuration.ReadableConfig configuration,
                                                      ClassLoader classLoader,
                                                      boolean isTemporary)

Deprecated.

Use createDynamicTableSink(DynamicTableSinkFactory, ObjectIdentifier, ResolvedCatalogTable, Map, ReadableConfig, ClassLoader, boolean)

createTableSink

@Deprecated
public static DynamicTableSink createTableSink(@Nullable
                                               Catalog catalog,
                                               ObjectIdentifier objectIdentifier,
                                               ResolvedCatalogTable catalogTable,
                                               org.apache.flink.configuration.ReadableConfig configuration,
                                               ClassLoader classLoader,
                                               boolean isTemporary)

Deprecated.

Use createDynamicTableSink(DynamicTableSinkFactory, ObjectIdentifier, ResolvedCatalogTable, Map, ReadableConfig, ClassLoader, boolean) instead.

Creates a DynamicTableSink from a CatalogTable.

It considers Catalog.getFactory() if provided.

createCatalogFactoryHelper

public static FactoryUtil.CatalogFactoryHelper createCatalogFactoryHelper(CatalogFactory factory,
                                                                          CatalogFactory.Context context)

Creates a utility that helps validating options for a CatalogFactory.

Note: This utility checks for left-over options in the final step.

createCatalogStoreFactoryHelper

public static FactoryUtil.CatalogStoreFactoryHelper createCatalogStoreFactoryHelper(CatalogStoreFactory factory,
                                                                                    CatalogStoreFactory.Context context)

Creates a utility that helps validating options for a CatalogStoreFactory.

Note: This utility checks for left-over options in the final step.

createModuleFactoryHelper

public static FactoryUtil.ModuleFactoryHelper createModuleFactoryHelper(ModuleFactory factory,
                                                                        ModuleFactory.Context context)

Creates a utility that helps validating options for a ModuleFactory.

Note: This utility checks for left-over options in the final step.

createTableFactoryHelper

public static FactoryUtil.TableFactoryHelper createTableFactoryHelper(DynamicTableFactory factory,
                                                                      DynamicTableFactory.Context context)

Creates a utility that helps in discovering formats, merging options with DynamicTableFactory.Context.getEnrichmentOptions() and validating them all for a DynamicTableFactory.

The following example sketches the usage:

 // in createDynamicTableSource()
 helper = FactoryUtil.createTableFactoryHelper(this, context);

 keyFormat = helper.discoverDecodingFormat(DeserializationFormatFactory.class, KEY_FORMAT);
 valueFormat = helper.discoverDecodingFormat(DeserializationFormatFactory.class, VALUE_FORMAT);

 helper.validate();

 ... // construct connector with discovered formats
 

Note: The format option parameter of FactoryUtil.TableFactoryHelper.discoverEncodingFormat(Class, ConfigOption) and FactoryUtil.TableFactoryHelper.discoverDecodingFormat(Class, ConfigOption) must be FORMAT or end with FORMAT_SUFFIX. The discovery logic will replace 'format' with the factory identifier value as the format prefix. For example, assuming the identifier is 'json', if the format option key is 'format', then the format prefix is 'json.'. If the format option key is 'value.format', then the format prefix is 'value.json'. The format prefix is used to project the options for the format factory.

Note: When created, this utility merges the options from DynamicTableFactory.Context.getEnrichmentOptions() using DynamicTableFactory.forwardOptions(). When invoking FactoryUtil.FactoryHelper.validate(), this utility checks for left-over options in the final step.

createCatalog

public static Catalog createCatalog(String catalogName,
                                    Map<String,String> options,
                                    org.apache.flink.configuration.ReadableConfig configuration,
                                    ClassLoader classLoader)

Attempts to discover an appropriate catalog factory and creates an instance of the catalog.

This first uses the legacy TableFactory stack to discover a matching CatalogFactory. If none is found, it falls back to the new stack using Factory instead.

createModule

public static Module createModule(String moduleName,
                                  Map<String,String> options,
                                  org.apache.flink.configuration.ReadableConfig configuration,
                                  ClassLoader classLoader)

Discovers a matching module factory and creates an instance of it.

This first uses the legacy TableFactory stack to discover a matching ModuleFactory. If none is found, it falls back to the new stack using Factory instead.

discoverFactory

public static <T extends Factory> T discoverFactory(ClassLoader classLoader,
                                                    Class<T> factoryClass,
                                                    String factoryIdentifier)

Discovers a factory using the given factory base class and identifier.

This method is meant for cases where createTableFactoryHelper(DynamicTableFactory, DynamicTableFactory.Context) createTableSource(Catalog, ObjectIdentifier, ResolvedCatalogTable, ReadableConfig, ClassLoader, boolean), and createTableSink(Catalog, ObjectIdentifier, ResolvedCatalogTable, ReadableConfig, ClassLoader, boolean) are not applicable.

validateFactoryOptions

public static void validateFactoryOptions(Factory factory,
                                          org.apache.flink.configuration.ReadableConfig options)

Validates the required and optional ConfigOptions of a factory.

Note: It does not check for left-over options.

validateFactoryOptions

public static void validateFactoryOptions(Set<org.apache.flink.configuration.ConfigOption<?>> requiredOptions,
                                          Set<org.apache.flink.configuration.ConfigOption<?>> optionalOptions,
                                          org.apache.flink.configuration.ReadableConfig options)

Validates the required options and optional options.

Note: It does not check for left-over options.

validateUnconsumedKeys

public static void validateUnconsumedKeys(String factoryIdentifier,
                                          Set<String> allOptionKeys,
                                          Set<String> consumedOptionKeys,
                                          Set<String> deprecatedOptionKeys)

Validates unconsumed option keys.

validateUnconsumedKeys

public static void validateUnconsumedKeys(String factoryIdentifier,
                                          Set<String> allOptionKeys,
                                          Set<String> consumedOptionKeys)

Validates unconsumed option keys.

getFormatPrefix

public static String getFormatPrefix(org.apache.flink.configuration.ConfigOption<String> formatOption,
                                     String formatIdentifier)

Returns the required option prefix for options of the given format.

getDynamicTableFactory

public static <T extends DynamicTableFactory> Optional<T> getDynamicTableFactory(Class<T> factoryClass,
                                                                                 @Nullable
                                                                                 Catalog catalog)

Returns the DynamicTableFactory via Catalog.

stringifyOption

public static String stringifyOption(String key,
                                     String value)

validateWatermarkOptions

public static void validateWatermarkOptions(String factoryIdentifier,
                                            org.apache.flink.configuration.ReadableConfig conf)

Validate watermark options from table options.

Parameters:

factoryIdentifier - identifier of table

conf - table options

checkWatermarkOptions

public static Optional<String> checkWatermarkOptions(org.apache.flink.configuration.ReadableConfig conf)

Check watermark-related options and return error messages.

Parameters:

conf - table options

Returns:

Optional of error messages