Package org.apache.beam.runners.spark.structuredstreaming.examples

Class WordCount.CountWords

java.lang.Object
org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<String>,org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<String,Long>>>
org.apache.beam.runners.spark.structuredstreaming.examples.WordCount.CountWords
All Implemented Interfaces:
Serializable, org.apache.beam.sdk.transforms.display.HasDisplayData
Enclosing class:
WordCount
public static class WordCount.CountWords extends org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<String>,org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<String,Long>>>
A PTransform that converts a PCollection containing lines of text into a PCollection of formatted word counts.

Concept #3: This is a custom composite transform that bundles two transforms (ParDo and Count) as a reusable PTransform subclass. Using composite transforms allows for easy reuse, modular testing, and an improved monitoring experience.

See Also:

  • Field Summary

    Fields inherited from class org.apache.beam.sdk.transforms.PTransform

    annotations, displayData, name, resourceHints

  • Constructor Summary

    Constructors
    Constructor
    Description
    CountWords()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<String,Long>>
    expand(org.apache.beam.sdk.values.PCollection<String> lines)
     

    Methods inherited from class org.apache.beam.sdk.transforms.PTransform

    addAnnotation, compose, compose, getAdditionalInputs, getAnnotations, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, populateDisplayData, setDisplayData, setResourceHints, toString, validate, validate

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

  • Constructor Details

    • CountWords

      public CountWords()

  • Method Details

    • expand

      public org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<String,Long>> expand(org.apache.beam.sdk.values.PCollection<String> lines)
      Specified by:
      expand in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<String>,org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<String,Long>>>