Package io.delta.kernel.internal.util

Class FileNames

Object
io.delta.kernel.internal.util.FileNames
public final class FileNames extends Object

  • Field Details

    • MULTI_PART_CHECKPOINT_FILE_PATTERN

      public static final Pattern MULTI_PART_CHECKPOINT_FILE_PATTERN
      Example: 00000000000000000001.checkpoint.0000000020.0000000060.parquet

    • STAGED_COMMIT_DIRECTORY

      public static final String STAGED_COMMIT_DIRECTORY
      See Also:

    • SIDECAR_DIRECTORY

      public static final String SIDECAR_DIRECTORY
      See Also:

  • Method Details

    • determineFileType

      public static FileNames.DeltaLogFileType determineFileType(FileStatus file)

    • getFileVersion

      public static long getFileVersion(Path path)
      Get the version of the checkpoint, checksum or delta file. Throws an error if an unexpected file type is seen. These unexpected files should be filtered out to ensure forward compatibility in cases where new file types are added, but without an explicit protocol upgrade.

    • deltaVersion

      public static long deltaVersion(Path path)
      Returns the version for the given delta path.

    • deltaVersion

      public static long deltaVersion(String path)

    • logCompactionVersions

      public static Tuple2<Long,Long> logCompactionVersions(Path path)
      Returns the start and end versions for the given compaction path.

    • logCompactionVersions

      public static Tuple2<Long,Long> logCompactionVersions(String path)

    • checkpointVersion

      public static long checkpointVersion(Path path)
      Returns the version for the given checkpoint path.

    • checkpointVersion

      public static long checkpointVersion(String path)

    • multiPartCheckpointPartAndNumParts

      public static Tuple2<Integer,Integer> multiPartCheckpointPartAndNumParts(Path path)

    • multiPartCheckpointPartAndNumParts

      public static Tuple2<Integer,Integer> multiPartCheckpointPartAndNumParts(String path)

    • stagedCommitDirectory

      public static String stagedCommitDirectory(Path logPath)

    • sidecarDirectory

      public static String sidecarDirectory(Path logPath)

    • deltaFile

      public static String deltaFile(Path path, long version)
      Returns the delta (json format) path for a given delta file.

    • deltaFile

      public static String deltaFile(String path, long version)
      Returns the delta (json format) path for a given delta file.

    • stagedCommitFile

      public static String stagedCommitFile(Path logPath, long version)

    • stagedCommitFile

      public static String stagedCommitFile(String logPath, long version)

    • sidecarFile

      public static String sidecarFile(Path path, String sidecar)
      Example: /a/_sidecars/3a0d65cd-4056-49b8-937b-95f9e3ee90e5.parquet

    • checksumFile

      public static Path checksumFile(Path path, long version)
      Returns the path to the checksum file for the given version.

    • checksumVersion

      public static long checksumVersion(Path path)

    • checksumVersion

      public static long checksumVersion(String path)

    • listingPrefix

      public static String listingPrefix(Path path, long version)
      Returns the prefix of all delta log files for the given version.

      Intended for use with listFrom to get all files from this version onwards. The returned Path will not exist as a file.

    • checkpointFileSingular

      public static Path checkpointFileSingular(Path path, long version)
      Returns the path for a singular checkpoint up to the given version.

      In a future protocol version this path will stop being written.

    • topLevelV2CheckpointFile

      public static Path topLevelV2CheckpointFile(Path path, long version, String uuid, String fileType)
      Returns the path for a top-level V2 checkpoint file up to the given version with a given UUID and filetype (JSON or Parquet).

    • v2CheckpointSidecarFile

      public static Path v2CheckpointSidecarFile(Path path, String uuid)
      Returns the path for a V2 sidecar file with a given UUID.

    • multiPartCheckpointFile

      public static Path multiPartCheckpointFile(Path path, long version, int part, int numParts)

    • checkpointFileWithParts

      public static List<Path> checkpointFileWithParts(Path path, long version, int numParts)
      Returns the paths for all parts of the checkpoint up to the given version.

      In a future protocol version we will write this path instead of checkpointFileSingular.

      Example of the format: 00000000000000004915.checkpoint.0000000020.0000000060.parquet is checkpoint part 20 out of 60 for the snapshot at version 4915. Zero padding is for lexicographic sorting.

    • logCompactionPath

      public static Path logCompactionPath(Path logPath, long startVersion, long endVersion)
      Return the path that should be used for a log compaction file.
      Parameters:
      logPath - path to the delta log location
      startVersion - the start version for the log compaction
      endVersion - the end version for the log compaction

    • isCheckpointFile

      public static boolean isCheckpointFile(String path)

    • isClassicCheckpointFile

      public static boolean isClassicCheckpointFile(String path)

    • isMultiPartCheckpointFile

      public static boolean isMultiPartCheckpointFile(String path)

    • isV2CheckpointFile

      public static boolean isV2CheckpointFile(String path)

    • isCommitFile

      public static boolean isCommitFile(String path)

    • isPublishedDeltaFile

      public static boolean isPublishedDeltaFile(String path)

    • isStagedDeltaFile

      public static boolean isStagedDeltaFile(String path)

    • isLogCompactionFile

      public static boolean isLogCompactionFile(String path)

    • isChecksumFile

      public static boolean isChecksumFile(String checksumFilePath)