Package org.apache.druid.java.util.common

Class FileUtils

java.lang.Object
org.apache.druid.java.util.common.FileUtils
public class FileUtils extends Object

  • Field Details

    • IS_EXCEPTION

      public static final com.google.common.base.Predicate<Throwable> IS_EXCEPTION
      Useful for retry functionality that doesn't want to stop Throwables, but does want to retry on Exceptions

  • Constructor Details

    • FileUtils

      public FileUtils()

  • Method Details

    • retryCopy

      public static FileUtils.FileCopyResult retryCopy(com.google.common.io.ByteSource byteSource, File outFile, com.google.common.base.Predicate<Throwable> shouldRetry, int maxAttempts)
      Copy input byte source to outFile. If outFile exists, it is attempted to be deleted.
      Parameters:
      byteSource - Supplier for an input stream that is to be copied. The resulting stream is closed each iteration
      outFile - Where the file should be written to.
      shouldRetry - Predicate indicating if an error is recoverable and should be retried.
      maxAttempts - The maximum number of assumed recoverable attempts to try before completely failing.
      Throws:
      RuntimeException - wrapping the inner exception on failure.

    • map

      public static MappedByteBufferHandler map(File file) throws IOException
      Fully maps a file read-only in to memory as per FileChannel.map(FileChannel.MapMode, long, long).

      Files are mapped from offset 0 to its length.

      This only works for files invalid input: '<'= Integer.MAX_VALUE bytes.

      Similar to Files.map(File), but returns MappedByteBufferHandler, that makes it easier to unmap the buffer within try-with-resources pattern: 

      
 try (MappedByteBufferHandler fileMappingHandler = FileUtils.map(file)) {
   ByteBuffer fileMapping = fileMappingHandler.get();
   // use mapped buffer
 }
      Parameters:
      file - the file to map
      Returns:
      a MappedByteBufferHandler, wrapping a read-only buffer reflecting file
      Throws:
      FileNotFoundException - if the file does not exist
      IOException - if an I/O error occurs
      IllegalArgumentException - if length is greater than Integer.MAX_VALUE
      See Also:

    • map

      public static MappedByteBufferHandler map(File file, long offset, long length) throws IOException
      Fully maps a file read-only in to memory as per FileChannel.map(FileChannel.MapMode, long, long).
      Parameters:
      file - the file to map
      offset - starting offset for the mmap
      length - length for the mmap
      Returns:
      a MappedByteBufferHandler, wrapping a read-only buffer reflecting file
      Throws:
      FileNotFoundException - if the file does not exist
      IOException - if an I/O error occurs
      IllegalArgumentException - if length is greater than Integer.MAX_VALUE
      See Also:

    • map

      public static MappedByteBufferHandler map(RandomAccessFile randomAccessFile, long offset, long length) throws IOException
      Fully maps a file read-only in to memory as per FileChannel.map(FileChannel.MapMode, long, long).
      Parameters:
      randomAccessFile - the file to map. The file will not be closed.
      offset - starting offset for the mmap
      length - length for the mmap
      Returns:
      a MappedByteBufferHandler, wrapping a read-only buffer reflecting randomAccessFile
      Throws:
      IOException - if an I/O error occurs
      IllegalArgumentException - if length is greater than Integer.MAX_VALUE
      See Also:

    • writeAtomically

      public static <T> T writeAtomically(File file, FileUtils.OutputStreamConsumer<T> f) throws IOException
      Write to a file atomically, by first writing to a temporary file in the same directory and then moving it to the target location. More docs at writeAtomically(File, File, OutputStreamConsumer) .
      Throws:
      IOException

    • writeAtomically

      public static <T> T writeAtomically(File file, File tmpDir, FileUtils.OutputStreamConsumer<T> f) throws IOException
      Write to a file atomically, by first writing to a temporary file in given tmpDir directory and then moving it to the target location. This function attempts to clean up its temporary files when possible, but they may stick around (for example, if the JVM crashes partway through executing the function). In any case, the target file should be unharmed. The OutputStream passed to the consumer is uncloseable; calling close on it will do nothing. This is to ensure that the stream stays open so we can fsync it here before closing. Hopefully, this doesn't cause any problems for callers. This method is not just thread-safe, but is also safe to use from multiple processes on the same machine.
      Throws:
      IOException

    • copyLarge

      public static <T> long copyLarge(T object, ObjectOpenFunction<T> objectOpenFunction, File outFile, byte[] fetchBuffer, com.google.common.base.Predicate<Throwable> retryCondition, int numTries, String messageOnRetry) throws IOException
      Copies data from the InputStream opened with objectOpenFunction to the given file. This method is supposed to be used for copying large files. The output file is deleted automatically if copy fails.
      Parameters:
      object - object to open
      objectOpenFunction - function to open the given object
      outFile - file to write data
      fetchBuffer - a buffer to copy data from the input stream to the file
      retryCondition - condition which should be satisfied for retry
      numTries - max number of retries
      messageOnRetry - log message on retry
      Returns:
      the number of bytes copied
      Throws:
      IOException

    • copyLarge

      public static long copyLarge(FileUtils.InputStreamSupplier inputSource, File outFile, byte[] fetchBuffer, com.google.common.base.Predicate<Throwable> retryCondition, int numTries, String messageOnRetry) throws IOException
      Copy a potentially large amount of data from an input source to a file.
      Throws:
      IOException

    • getFileSize

      public static long getFileSize(File file)
      Computes the size of the file. If it is a directory, computes the size up to a depth of 1.

    • createTempDir

      public static File createTempDir()
      Creates a temporary directory inside the configured temporary space (java.io.tmpdir). Similar to the method Files.createTempDir() from Guava, but has nicer error messages.
      Throws:
      IllegalStateException - if the directory could not be created

    • createTempDir

      public static File createTempDir(@Nullable String prefix)
      Creates a temporary directory inside the configured temporary space (java.io.tmpdir). Similar to the method Files.createTempDir() from Guava, but has nicer error messages.
      Parameters:
      prefix - base directory name; if null/empty then this method will use "druid"
      Throws:
      IllegalStateException - if the directory could not be created

    • getTempDir

      public static Path getTempDir()

    • createTempDirInLocation

      public static File createTempDirInLocation(Path parentDirectory, @Nullable String prefix)

    • mkdirp

      public static void mkdirp(File directory) throws IOException
      Create "directory" and all intermediate directories as needed. If the directory is successfully created, or already exists, returns quietly. Otherwise, throws an IOException. Simpler to use than File.mkdirs(), and more reliable since it is safe from races where two threads try to create the same directory at the same time. The name is inspired by UNIX mkdir -p, which has the same behavior.
      Throws:
      IOException

    • deleteDirectory

      public static void deleteDirectory(File directory) throws IOException
      Equivalent to FileUtils.deleteDirectory(File). Exists here mostly so callers can avoid dealing with our FileUtils and the Commons FileUtils having the same name.
      Throws:
      IOException

    • linkOrCopy

      public static FileUtils.LinkOrCopyResult linkOrCopy(File src, File dest) throws IOException
      Hard-link "src" as "dest", if possible. If not possible -- perhaps they are on separate filesystems -- then copy "src" to "dest".
      Returns:
      whether a link or copy was made. Can be safely ignored if you don't care.
      Throws:
      IOException - if something went wrong