Package org.apache.beam.sdk.io.csv

Class CsvIO.Write<T>

  • All Implemented Interfaces:
    java.io.Serializable, org.apache.beam.sdk.transforms.display.HasDisplayData
    Enclosing class:
    CsvIO
    public abstract static class CsvIO.Write<T>
extends org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<T>,​org.apache.beam.sdk.io.WriteFilesResult<java.lang.String>>
implements org.apache.beam.sdk.transforms.display.HasDisplayData
    PTransform for writing CSV files.
    See Also:
    Serialized Form

    • Field Summary

      • Fields inherited from class org.apache.beam.sdk.transforms.PTransform

        annotations, displayData, name, resourceHints

    • Constructor Summary

      Constructors 
      Constructor Description
      Write()  

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      org.apache.beam.sdk.io.WriteFilesResult<java.lang.String> expand​(org.apache.beam.sdk.values.PCollection<T> input)  
      void populateDisplayData​(org.apache.beam.sdk.transforms.display.DisplayData.Builder builder)  
      CsvIO.Write<T> withCompression​(org.apache.beam.sdk.io.Compression compression)
      Specifies the Compression of all generated shard files.
      CsvIO.Write<T> withMaxNumWritersPerBundle​(@Nullable java.lang.Integer maxNumWritersPerBundle)
      Set the maximum number of writers created in a bundle before spilling to shuffle.
      CsvIO.Write<T> withNoSpilling()
      Whether to skip the spilling of data.
      CsvIO.Write<T> withNumShards​(java.lang.Integer numShards)
      Specifies to use a given fixed number of shards per window.
      CsvIO.Write<T> withoutSharding()
      Forces a single file as output and empty shard name template.
      CsvIO.Write<T> withShardTemplate​(java.lang.String shardTemplate)
      Uses the given ShardNameTemplate for naming output files.
      CsvIO.Write<T> withSuffix​(java.lang.String suffix)
      Configures the filename suffix for written files.
      CsvIO.Write<T> withTempDirectory​(org.apache.beam.sdk.io.fs.ResourceId tempDirectory)
      Set the base directory used to generate temporary files.
      CsvIO.Write<T> withWindowedWrites()
      Preserves windowing of input elements and writes them to files based on the element's window.
      CsvIO.Write<T> withWritableByteChannelFactory​(org.apache.beam.sdk.io.FileBasedSink.WritableByteChannelFactory writableByteChannelFactory)
      Returns a transform for writing to text files like this one but that has the given FileBasedSink.WritableByteChannelFactory to be used by the FileBasedSink during output.

      • Methods inherited from class org.apache.beam.sdk.transforms.PTransform

        addAnnotation, compose, compose, getAdditionalInputs, getAnnotations, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, setDisplayData, setResourceHints, toString, validate, validate

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

    • Constructor Detail

      • Write

        public Write()

    • Method Detail

      • populateDisplayData

        public void populateDisplayData​(org.apache.beam.sdk.transforms.display.DisplayData.Builder builder)
        Specified by:
        populateDisplayData in interface org.apache.beam.sdk.transforms.display.HasDisplayData
        Overrides:
        populateDisplayData in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<T>,​org.apache.beam.sdk.io.WriteFilesResult<java.lang.String>>

      • withCompression

        public CsvIO.Write<T> withCompression​(org.apache.beam.sdk.io.Compression compression)
        Specifies the Compression of all generated shard files.

      • withNoSpilling

        public CsvIO.Write<T> withNoSpilling()
        Whether to skip the spilling of data. See WriteFiles.withNoSpilling().

      • withMaxNumWritersPerBundle

        public CsvIO.Write<T> withMaxNumWritersPerBundle​(@Nullable java.lang.Integer maxNumWritersPerBundle)
        Set the maximum number of writers created in a bundle before spilling to shuffle. See WriteFiles#withMaxNumWritersPerBundle().

      • withNumShards

        public CsvIO.Write<T> withNumShards​(java.lang.Integer numShards)
        Specifies to use a given fixed number of shards per window. See TextIO.Write.withNumShards(int).

      • withoutSharding

        public CsvIO.Write<T> withoutSharding()
        Forces a single file as output and empty shard name template. See TextIO.Write.withoutSharding().

      • withShardTemplate

        public CsvIO.Write<T> withShardTemplate​(java.lang.String shardTemplate)
        Uses the given ShardNameTemplate for naming output files. See TextIO.Write.withShardNameTemplate(java.lang.String).

      • withSuffix

        public CsvIO.Write<T> withSuffix​(java.lang.String suffix)
        Configures the filename suffix for written files. See TextIO.Write.withSuffix(java.lang.String).

      • withTempDirectory

        public CsvIO.Write<T> withTempDirectory​(org.apache.beam.sdk.io.fs.ResourceId tempDirectory)
        Set the base directory used to generate temporary files. See TextIO.Write.withTempDirectory(org.apache.beam.sdk.options.ValueProvider<org.apache.beam.sdk.io.fs.ResourceId>).

      • withWindowedWrites

        public CsvIO.Write<T> withWindowedWrites()
        Preserves windowing of input elements and writes them to files based on the element's window. See TextIO.Write.withWindowedWrites().

      • withWritableByteChannelFactory

        public CsvIO.Write<T> withWritableByteChannelFactory​(org.apache.beam.sdk.io.FileBasedSink.WritableByteChannelFactory writableByteChannelFactory)
        Returns a transform for writing to text files like this one but that has the given FileBasedSink.WritableByteChannelFactory to be used by the FileBasedSink during output. See TextIO.Write.withWritableByteChannelFactory(org.apache.beam.sdk.io.FileBasedSink.WritableByteChannelFactory).

      • expand

        public org.apache.beam.sdk.io.WriteFilesResult<java.lang.String> expand​(org.apache.beam.sdk.values.PCollection<T> input)
        Specified by:
        expand in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<T>,​org.apache.beam.sdk.io.WriteFilesResult<java.lang.String>>