org.apache.flink.cdc.runtime.operators.schema.common

类 SchemaRegistry

java.lang.Object

org.apache.flink.cdc.runtime.operators.schema.common.SchemaRegistry

所有已实现的接口:

AutoCloseable, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.runtime.operators.coordination.CoordinationRequestHandler, org.apache.flink.runtime.operators.coordination.OperatorCoordinator

直接已知子类:

SchemaCoordinator, SchemaCoordinator

@Internal
public abstract class SchemaRegistry
extends Object
implements org.apache.flink.runtime.operators.coordination.OperatorCoordinator, org.apache.flink.runtime.operators.coordination.CoordinationRequestHandler

An abstract centralized schema registry that accepts requests from SchemaEvolutionClient. A legit schema registry should be able to:

• Handle schema retrieval requests from SchemaEvolutionClients
• Accept and trace sink writers' registering events
• Snapshot & Restore its state during checkpoints

These abilities are done by overriding abstract methods of SchemaRegistry. All methods will run in given ExecutorService asynchronously except SchemaRegistry#restore.

嵌套类概要

从接口继承的嵌套类/接口 org.apache.flink.runtime.operators.coordination.OperatorCoordinator

org.apache.flink.runtime.operators.coordination.OperatorCoordinator.Context, org.apache.flink.runtime.operators.coordination.OperatorCoordinator.Provider, org.apache.flink.runtime.operators.coordination.OperatorCoordinator.SubtaskGateway

字段概要

字段

限定符和类型	字段和说明
protected Set<Integer>	activeSinkWriters
protected org.apache.flink.cdc.common.pipeline.SchemaChangeBehavior	behavior
protected org.apache.flink.runtime.operators.coordination.OperatorCoordinator.Context	context
protected ExecutorService	coordinatorExecutor
protected int	currentParallelism
protected Map<Integer,Throwable>	failedReasons
protected org.apache.flink.cdc.common.sink.MetadataApplier	metadataApplier
protected String	operatorName
protected TableIdRouter	router
protected List<org.apache.flink.cdc.common.route.RouteRule>	routingRules
protected java.time.Duration	rpcTimeout
protected SchemaManager	schemaManager

从接口继承的字段 org.apache.flink.runtime.operators.coordination.OperatorCoordinator

NO_CHECKPOINT

构造器概要

构造器

限定符	构造器和说明
protected	SchemaRegistry(org.apache.flink.runtime.operators.coordination.OperatorCoordinator.Context context, String operatorName, ExecutorService coordinatorExecutor, org.apache.flink.cdc.common.sink.MetadataApplier metadataApplier, List<org.apache.flink.cdc.common.route.RouteRule> routingRules, org.apache.flink.cdc.common.pipeline.SchemaChangeBehavior schemaChangeBehavior, java.time.Duration rpcTimeout)

方法概要

限定符和类型	方法和说明
void	checkpointCoordinator(long checkpointId, CompletableFuture<byte[]> completableFuture)
void	close()
void	emplaceEvolvedSchema(org.apache.flink.cdc.common.event.TableId tableId, org.apache.flink.cdc.common.schema.Schema schema)
void	emplaceOriginalSchema(org.apache.flink.cdc.common.event.TableId tableId, org.apache.flink.cdc.common.schema.Schema schema)
void	executionAttemptFailed(int subTaskId, int attemptNumber, Throwable reason)
void	executionAttemptReady(int subTaskId, int attemptNumber, org.apache.flink.runtime.operators.coordination.OperatorCoordinator.SubtaskGateway gateway)
protected <T extends Throwable> void	failJob(String taskDescription, T t)
CompletableFuture<org.apache.flink.runtime.operators.coordination.CoordinationResponse>	handleCoordinationRequest(org.apache.flink.runtime.operators.coordination.CoordinationRequest request)
protected abstract void	handleCustomCoordinationRequest(org.apache.flink.runtime.operators.coordination.CoordinationRequest request, CompletableFuture<org.apache.flink.runtime.operators.coordination.CoordinationResponse> responseFuture) Coordination handler for customized CoordinationRequests.
void	handleEventFromOperator(int subTaskId, int attemptNumber, org.apache.flink.runtime.operators.coordination.OperatorEvent event)
protected abstract void	handleFlushSuccessEvent(FlushSuccessEvent event) Overridable handler for FlushSuccessEvents.
protected void	handleGetEvolvedSchemaRequest(GetEvolvedSchemaRequest request, CompletableFuture<org.apache.flink.runtime.operators.coordination.CoordinationResponse> responseFuture) Overridable handler for GetEvolvedSchemaRequests.
protected void	handleGetOriginalSchemaRequest(GetOriginalSchemaRequest request, CompletableFuture<org.apache.flink.runtime.operators.coordination.CoordinationResponse> responseFuture) Overridable handler for GetOriginalSchemaRequests.
protected void	handleSinkWriterRegisterEvent(SinkWriterRegisterEvent event) Overridable handler for SinkWriterRegisterEvents.
protected void	handleUnrecoverableError(String taskDescription, Throwable t) Last chance to execute codes before job fails globally.
protected void	loopUntil(java.util.function.BooleanSupplier conditionChecker, Runnable message, java.time.Duration timeout, java.time.Duration interval) Keeps checking if conditionChecker is satisfied.
protected void	loopWhen(java.util.function.BooleanSupplier conditionChecker, Runnable message, java.time.Duration timeout, java.time.Duration interval) Keeps checking if conditionChecker is satisfied.
void	notifyCheckpointComplete(long checkpointId)
void	resetToCheckpoint(long checkpointId, byte[] checkpointData)
protected abstract void	restore(byte[] checkpointData) Restore schema registry state from byte array.
protected void	runInEventLoop(org.apache.flink.util.function.ThrowingRunnable<Throwable> action, String actionName, Object... actionNameFormatParameters) Run a time-consuming task in given ExecutorService.
protected abstract void	snapshot(CompletableFuture<byte[]> resultFuture) Snapshot current schema registry state in byte array form.
void	start()
void	subtaskReset(int subTaskId, long checkpointId)

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

从接口继承的方法 org.apache.flink.runtime.operators.coordination.OperatorCoordinator

notifyCheckpointAborted

字段详细资料

context

protected final org.apache.flink.runtime.operators.coordination.OperatorCoordinator.Context context

operatorName

protected final String operatorName

coordinatorExecutor

protected final ExecutorService coordinatorExecutor

metadataApplier

protected final org.apache.flink.cdc.common.sink.MetadataApplier metadataApplier

rpcTimeout

protected final java.time.Duration rpcTimeout

routingRules

protected final List<org.apache.flink.cdc.common.route.RouteRule> routingRules

behavior

protected final org.apache.flink.cdc.common.pipeline.SchemaChangeBehavior behavior

currentParallelism

protected transient int currentParallelism

activeSinkWriters

protected transient Set<Integer> activeSinkWriters

failedReasons

protected transient Map<Integer,Throwable> failedReasons

schemaManager

protected transient SchemaManager schemaManager

router

protected transient TableIdRouter router

构造器详细资料

SchemaRegistry

protected SchemaRegistry(org.apache.flink.runtime.operators.coordination.OperatorCoordinator.Context context,
                         String operatorName,
                         ExecutorService coordinatorExecutor,
                         org.apache.flink.cdc.common.sink.MetadataApplier metadataApplier,
                         List<org.apache.flink.cdc.common.route.RouteRule> routingRules,
                         org.apache.flink.cdc.common.pipeline.SchemaChangeBehavior schemaChangeBehavior,
                         java.time.Duration rpcTimeout)

方法详细资料

start

public void start()
           throws Exception

指定者:

start 在接口中 org.apache.flink.runtime.operators.coordination.OperatorCoordinator

抛出:

Exception

close

public void close()
           throws Exception

指定者:

close 在接口中 AutoCloseable

指定者:

close 在接口中 org.apache.flink.runtime.operators.coordination.OperatorCoordinator

抛出:

Exception

snapshot

protected abstract void snapshot(CompletableFuture<byte[]> resultFuture)
                          throws Exception

Snapshot current schema registry state in byte array form.

抛出:

Exception

restore

protected abstract void restore(byte[] checkpointData)
                         throws Exception

Restore schema registry state from byte array.

抛出:

Exception

handleSinkWriterRegisterEvent

protected void handleSinkWriterRegisterEvent(SinkWriterRegisterEvent event)
                                      throws Exception

Overridable handler for SinkWriterRegisterEvents.

抛出:

Exception

handleFlushSuccessEvent

protected abstract void handleFlushSuccessEvent(FlushSuccessEvent event)
                                         throws Exception

Overridable handler for FlushSuccessEvents.

抛出:

Exception

handleGetEvolvedSchemaRequest

protected void handleGetEvolvedSchemaRequest(GetEvolvedSchemaRequest request,
                                             CompletableFuture<org.apache.flink.runtime.operators.coordination.CoordinationResponse> responseFuture)
                                      throws Exception

Overridable handler for GetEvolvedSchemaRequests.

抛出:

Exception

handleGetOriginalSchemaRequest

protected void handleGetOriginalSchemaRequest(GetOriginalSchemaRequest request,
                                              CompletableFuture<org.apache.flink.runtime.operators.coordination.CoordinationResponse> responseFuture)
                                       throws Exception

Overridable handler for GetOriginalSchemaRequests.

抛出:

Exception

handleCustomCoordinationRequest

protected abstract void handleCustomCoordinationRequest(org.apache.flink.runtime.operators.coordination.CoordinationRequest request,
                                                        CompletableFuture<org.apache.flink.runtime.operators.coordination.CoordinationResponse> responseFuture)
                                                 throws Exception

Coordination handler for customized CoordinationRequests.

抛出:

Exception

handleUnrecoverableError

protected void handleUnrecoverableError(String taskDescription,
                                        Throwable t)

Last chance to execute codes before job fails globally.

handleCoordinationRequest

public final CompletableFuture<org.apache.flink.runtime.operators.coordination.CoordinationResponse> handleCoordinationRequest(org.apache.flink.runtime.operators.coordination.CoordinationRequest request)

指定者:

handleCoordinationRequest 在接口中 org.apache.flink.runtime.operators.coordination.CoordinationRequestHandler

handleEventFromOperator

public final void handleEventFromOperator(int subTaskId,
                                          int attemptNumber,
                                          org.apache.flink.runtime.operators.coordination.OperatorEvent event)

指定者:

handleEventFromOperator 在接口中 org.apache.flink.runtime.operators.coordination.OperatorCoordinator

subtaskReset

public final void subtaskReset(int subTaskId,
                               long checkpointId)

指定者:

subtaskReset 在接口中 org.apache.flink.runtime.operators.coordination.OperatorCoordinator

executionAttemptFailed

public final void executionAttemptFailed(int subTaskId,
                                         int attemptNumber,
                                         @Nullable
                                         Throwable reason)

指定者:

executionAttemptFailed 在接口中 org.apache.flink.runtime.operators.coordination.OperatorCoordinator

executionAttemptReady

public final void executionAttemptReady(int subTaskId,
                                        int attemptNumber,
                                        org.apache.flink.runtime.operators.coordination.OperatorCoordinator.SubtaskGateway gateway)

指定者:

executionAttemptReady 在接口中 org.apache.flink.runtime.operators.coordination.OperatorCoordinator

checkpointCoordinator

public final void checkpointCoordinator(long checkpointId,
                                        CompletableFuture<byte[]> completableFuture)
                                 throws Exception

指定者:

checkpointCoordinator 在接口中 org.apache.flink.runtime.operators.coordination.OperatorCoordinator

抛出:

Exception

notifyCheckpointComplete

public final void notifyCheckpointComplete(long checkpointId)

指定者:

notifyCheckpointComplete 在接口中 org.apache.flink.api.common.state.CheckpointListener

指定者:

notifyCheckpointComplete 在接口中 org.apache.flink.runtime.operators.coordination.OperatorCoordinator

resetToCheckpoint

public final void resetToCheckpoint(long checkpointId,
                                    @Nullable
                                    byte[] checkpointData)
                             throws Exception

指定者:

resetToCheckpoint 在接口中 org.apache.flink.runtime.operators.coordination.OperatorCoordinator

抛出:

Exception

runInEventLoop

protected void runInEventLoop(org.apache.flink.util.function.ThrowingRunnable<Throwable> action,
                              String actionName,
                              Object... actionNameFormatParameters)

Run a time-consuming task in given ExecutorService. All overridable functions have been wrapped inside already, so there's no need to call this method again. However, if you're overriding methods from OperatorCoordinator or CoordinationRequestHandler directly, make sure you're running heavy logics inside, or the entire job might hang!

loopUntil

protected void loopUntil(java.util.function.BooleanSupplier conditionChecker,
                         Runnable message,
                         java.time.Duration timeout,
                         java.time.Duration interval)
                  throws TimeoutException

Keeps checking if conditionChecker is satisfied. If not, emit a message and retry.

抛出:

TimeoutException

loopWhen

protected void loopWhen(java.util.function.BooleanSupplier conditionChecker,
                        Runnable message,
                        java.time.Duration timeout,
                        java.time.Duration interval)
                 throws TimeoutException

Keeps checking if conditionChecker is satisfied. Otherwise, emit a message and retry.

抛出:

TimeoutException

failJob

protected <T extends Throwable> void failJob(String taskDescription,
                                             T t)

emplaceOriginalSchema

@VisibleForTesting
public void emplaceOriginalSchema(org.apache.flink.cdc.common.event.TableId tableId,
                                                     org.apache.flink.cdc.common.schema.Schema schema)

emplaceEvolvedSchema

@VisibleForTesting
public void emplaceEvolvedSchema(org.apache.flink.cdc.common.event.TableId tableId,
                                                    org.apache.flink.cdc.common.schema.Schema schema)