| Modifier and Type | Class and Description |
|---|---|
class |
AbstractFileSource<T,SplitT extends FileSourceSplit>
The base class for File Sources.
|
protected static class |
AbstractFileSource.AbstractFileSourceBuilder<T,SplitT extends FileSourceSplit,SELF extends AbstractFileSource.AbstractFileSourceBuilder<T,SplitT,SELF>>
The generic base builder.
|
class |
FileSourceSplitState<SplitT extends FileSourceSplit>
State of the reader, essentially a mutable version of the
FileSourceSplit. |
class |
PendingSplitsCheckpoint<SplitT extends FileSourceSplit>
A checkpoint of the current state of the containing the currently pending splits that are not yet
assigned.
|
class |
PendingSplitsCheckpointSerializer<T extends FileSourceSplit>
A serializer for the
PendingSplitsCheckpoint. |
| Modifier and Type | Method and Description |
|---|---|
static <T extends FileSourceSplit> |
PendingSplitsCheckpoint.fromCollectionSnapshot(Collection<T> splits) |
static <T extends FileSourceSplit> |
PendingSplitsCheckpoint.fromCollectionSnapshot(Collection<T> splits,
Collection<org.apache.flink.core.fs.Path> alreadyProcessedPaths) |
| Modifier and Type | Method and Description |
|---|---|
FileSourceSplit |
FileSourceSplitSerializer.deserialize(int version,
byte[] serialized) |
FileSourceSplit |
FileSourceSplit.updateWithCheckpointedPosition(CheckpointedPosition position)
Creates a copy of this split where the checkpointed position is replaced by the given new
position.
|
| Modifier and Type | Method and Description |
|---|---|
org.apache.flink.core.io.SimpleVersionedSerializer<FileSourceSplit> |
FileSource.getSplitSerializer() |
| Modifier and Type | Method and Description |
|---|---|
byte[] |
FileSourceSplitSerializer.serialize(FileSourceSplit split) |
| Modifier and Type | Method and Description |
|---|---|
static <T> FileSource.FileSourceBuilder<T> |
FileSource.forBulkFileFormat(BulkFormat<T,FileSourceSplit> bulkFormat,
org.apache.flink.core.fs.Path... paths)
Builds a new
FileSource using a BulkFormat to read batches of records from
files. |
| Modifier and Type | Method and Description |
|---|---|
Optional<FileSourceSplit> |
LocalityAwareSplitAssigner.getNext(String host) |
Optional<FileSourceSplit> |
SimpleSplitAssigner.getNext(String hostname) |
Optional<FileSourceSplit> |
FileSplitAssigner.getNext(String hostname)
Gets the next split.
|
Collection<FileSourceSplit> |
LocalityAwareSplitAssigner.remainingSplits() |
Collection<FileSourceSplit> |
SimpleSplitAssigner.remainingSplits() |
Collection<FileSourceSplit> |
FileSplitAssigner.remainingSplits()
Gets the remaining splits that this assigner has pending.
|
| Modifier and Type | Method and Description |
|---|---|
void |
LocalityAwareSplitAssigner.addSplits(Collection<FileSourceSplit> splits) |
void |
SimpleSplitAssigner.addSplits(Collection<FileSourceSplit> newSplits) |
void |
FileSplitAssigner.addSplits(Collection<FileSourceSplit> splits)
Adds a set of splits to this assigner.
|
FileSplitAssigner |
FileSplitAssigner.Provider.create(Collection<FileSourceSplit> initialSplits)
Creates a new
FileSplitAssigner that starts with the given set of initial splits. |
| Constructor and Description |
|---|
LocalityAwareSplitAssigner(Collection<FileSourceSplit> splits) |
SimpleSplitAssigner(Collection<FileSourceSplit> splits) |
| Modifier and Type | Method and Description |
|---|---|
Collection<FileSourceSplit> |
FileEnumerator.enumerateSplits(org.apache.flink.core.fs.Path[] paths,
int minDesiredSplits)
Generates all file splits for the relevant files under the given paths.
|
Collection<FileSourceSplit> |
NonSplittingRecursiveEnumerator.enumerateSplits(org.apache.flink.core.fs.Path[] paths,
int minDesiredSplits) |
| Modifier and Type | Method and Description |
|---|---|
protected void |
NonSplittingRecursiveAllDirEnumerator.addSplitsForPath(org.apache.flink.core.fs.FileStatus fileStatus,
org.apache.flink.core.fs.FileSystem fs,
ArrayList<FileSourceSplit> target) |
protected void |
BlockSplittingRecursiveAllDirEnumerator.addSplitsForPath(org.apache.flink.core.fs.FileStatus fileStatus,
org.apache.flink.core.fs.FileSystem fs,
ArrayList<FileSourceSplit> target) |
protected void |
NonSplittingRecursiveEnumerator.addSplitsForPath(org.apache.flink.core.fs.FileStatus fileStatus,
org.apache.flink.core.fs.FileSystem fs,
ArrayList<FileSourceSplit> target) |
protected void |
NonSplittingRecursiveEnumerator.convertToSourceSplits(org.apache.flink.core.fs.FileStatus file,
org.apache.flink.core.fs.FileSystem fs,
List<FileSourceSplit> target) |
protected void |
BlockSplittingRecursiveEnumerator.convertToSourceSplits(org.apache.flink.core.fs.FileStatus file,
org.apache.flink.core.fs.FileSystem fs,
List<FileSourceSplit> target) |
| Modifier and Type | Class and Description |
|---|---|
class |
DynamicFileSplitEnumerator<SplitT extends FileSourceSplit>
A SplitEnumerator implementation that supports dynamic filtering.
|
class |
FileSourceReader<T,SplitT extends FileSourceSplit>
A
SourceReader that read records from FileSourceSplit. |
| Modifier and Type | Method and Description |
|---|---|
PendingSplitsCheckpoint<FileSourceSplit> |
ContinuousFileSplitEnumerator.snapshotState(long checkpointId) |
PendingSplitsCheckpoint<FileSourceSplit> |
StaticFileSplitEnumerator.snapshotState(long checkpointId) |
| Modifier and Type | Method and Description |
|---|---|
BulkFormat.Reader<T> |
FileRecordFormatAdapter.createReader(org.apache.flink.configuration.Configuration config,
FileSourceSplit split)
Deprecated.
|
BulkFormat.Reader<T> |
StreamFormatAdapter.createReader(org.apache.flink.configuration.Configuration config,
FileSourceSplit split) |
BulkFormat.Reader<T> |
FileRecordFormatAdapter.restoreReader(org.apache.flink.configuration.Configuration config,
FileSourceSplit split)
Deprecated.
|
BulkFormat.Reader<T> |
StreamFormatAdapter.restoreReader(org.apache.flink.configuration.Configuration config,
FileSourceSplit split) |
| Modifier and Type | Method and Description |
|---|---|
void |
ContinuousFileSplitEnumerator.addSplitsBack(List<FileSourceSplit> splits,
int subtaskId) |
void |
StaticFileSplitEnumerator.addSplitsBack(List<FileSourceSplit> splits,
int subtaskId) |
| Constructor and Description |
|---|
ContinuousFileSplitEnumerator(org.apache.flink.api.connector.source.SplitEnumeratorContext<FileSourceSplit> context,
FileEnumerator enumerator,
FileSplitAssigner splitAssigner,
org.apache.flink.core.fs.Path[] paths,
Collection<org.apache.flink.core.fs.Path> alreadyDiscoveredPaths,
long discoveryInterval) |
StaticFileSplitEnumerator(org.apache.flink.api.connector.source.SplitEnumeratorContext<FileSourceSplit> context,
FileSplitAssigner splitAssigner) |
| Modifier and Type | Interface and Description |
|---|---|
interface |
BulkFormat<T,SplitT extends FileSourceSplit>
The
BulkFormat reads and decodes batches of records at a time. |
| Modifier and Type | Class and Description |
|---|---|
class |
LimitableBulkFormat<T,SplitT extends FileSourceSplit>
A
BulkFormat that can limit output record number. |
interface |
PartitionFieldExtractor<T extends FileSourceSplit>
Interface to extract partition field from split.
|
| Modifier and Type | Method and Description |
|---|---|
static <T,SplitT extends FileSourceSplit> |
LimitableBulkFormat.create(BulkFormat<T,SplitT> format,
Long limit) |
| Modifier and Type | Method and Description |
|---|---|
static PartitionFieldExtractor<FileSourceSplit> |
PartitionFieldExtractor.forFileSystem(String defaultPartValue) |
| Modifier and Type | Method and Description |
|---|---|
org.apache.flink.connector.file.table.DeserializationSchemaAdapter.Reader |
DeserializationSchemaAdapter.createReader(org.apache.flink.configuration.Configuration config,
FileSourceSplit split) |
org.apache.flink.connector.file.table.DeserializationSchemaAdapter.Reader |
DeserializationSchemaAdapter.restoreReader(org.apache.flink.configuration.Configuration config,
FileSourceSplit split) |
| Constructor and Description |
|---|
FileSystemTableSource(org.apache.flink.table.catalog.ObjectIdentifier tableIdentifier,
org.apache.flink.table.types.DataType physicalRowDataType,
List<String> partitionKeys,
org.apache.flink.configuration.ReadableConfig tableOptions,
org.apache.flink.table.connector.format.DecodingFormat<BulkFormat<org.apache.flink.table.data.RowData,FileSourceSplit>> bulkReaderFormat,
org.apache.flink.table.connector.format.DecodingFormat<org.apache.flink.api.common.serialization.DeserializationSchema<org.apache.flink.table.data.RowData>> deserializationFormat) |
| Modifier and Type | Method and Description |
|---|---|
static <T> CompactReader.Factory<T> |
CompactBulkReader.factory(BulkFormat<T,FileSourceSplit> format) |
Copyright © 2014–2025 The Apache Software Foundation. All rights reserved.