Package org.apache.beam.runners.spark.util

Class SideInputReaderFactory

java.lang.Object
org.apache.beam.runners.spark.util.SideInputReaderFactory
public class SideInputReaderFactory extends Object
Utility class for creating and managing side input readers in the Spark runner.

This class provides factory methods to create appropriate SideInputReader implementations based on the execution mode (streaming or batch) to optimize side input access patterns.

  • Constructor Summary

    Constructors
    Constructor
    Description
    SideInputReaderFactory()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    static org.apache.beam.runners.core.SideInputReader
    create(boolean useStreamingSideInput, Map<org.apache.beam.sdk.values.TupleTag<?>,org.apache.beam.sdk.values.KV<org.apache.beam.sdk.values.WindowingStrategy<?,?>,SideInputBroadcast<?>>> sideInputs)
    Creates and returns a SideInputReader based on the configuration.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • SideInputReaderFactory

      public SideInputReaderFactory()

  • Method Details

    • create

      public static org.apache.beam.runners.core.SideInputReader create(boolean useStreamingSideInput, Map<org.apache.beam.sdk.values.TupleTag<?>,org.apache.beam.sdk.values.KV<org.apache.beam.sdk.values.WindowingStrategy<?,?>,SideInputBroadcast<?>>> sideInputs)
      Creates and returns a SideInputReader based on the configuration.

      If streaming side inputs are enabled, returns a direct SparkSideInputReader. Otherwise, returns a cached version of the side input reader using CachedSideInputReader for better performance in batch processing.

      Parameters:
      useStreamingSideInput - Whether to use streaming side inputs
      sideInputs - A map of side inputs with their windowing strategies and broadcasts
      Returns:
      A SideInputReader instance appropriate for the current configuration