Package org.apache.beam.sdk.io.splunk

Class SplunkIO

  • public class SplunkIO
extends java.lang.Object
    An unbounded sink for Splunk's Http Event Collector (HEC).

    For more information, see the online documentation at Splunk HEC.

    Writing to Splunk's HEC

    The SplunkIO class provides a PTransform that allows writing SplunkEvent messages into a Splunk HEC end point.

    It takes as an input a PCollection<SplunkEvent>, where each SplunkEvent represents an event to be published to HEC.

    To configure a SplunkIO, you must provide at a minimum:

    • url - HEC endpoint URL.
    • token - HEC endpoint token.

    The SplunkIO transform can be customized further by optionally specifying:

    • parallelism - Number of parallel requests to the HEC.
    • batchCount - Number of events in a single batch.
    • disableCertificateValidation - Whether to disable ssl validation (useful for self-signed certificates)
    • enableBatchLogs - Whether to enable batch logs.
    • enableGzipHttpCompression - Whether HTTP requests sent to Splunk HEC should be GZIP encoded.

    This transform will return any non-transient write failures via a PCollection<SplunkWriteError>, where each SplunkWriteError captures the error that occurred while attempting to write to HEC. These can be published to a dead-letter sink or reprocessed.

    For example: 

    
 PCollection<SplunkEvent> events = ...;

 PCollection<SplunkWriteError> errors =
         events.apply("WriteToSplunk",
              SplunkIO.write(url, token)
                  .withBatchCount(batchCount)
                  .withParallelism(parallelism)
                  .withDisableCertificateValidation(true));

    • Nested Class Summary

      Nested Classes 
      Modifier and Type Class Description
      static class  SplunkIO.Write
      Class SplunkIO.Write provides a PTransform that allows writing SplunkEvent records into a Splunk HTTP Event Collector end-point using HTTP POST requests.

    • Method Summary

      All Methods Static Methods Concrete Methods 
      Modifier and Type Method Description
      static SplunkIO.Write write​(java.lang.String url, java.lang.String token)
      Write to Splunk's Http Event Collector (HEC).
      static SplunkIO.Write write​(org.apache.beam.sdk.options.ValueProvider<java.lang.String> url, org.apache.beam.sdk.options.ValueProvider<java.lang.String> token)
      Same as write(String, String) but with ValueProvider.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Method Detail

      • write

        public static SplunkIO.Write write​(java.lang.String url,
                                   java.lang.String token)
        Write to Splunk's Http Event Collector (HEC).
        Parameters:
        url - splunk hec url
        token - splunk hec authentication token

      • write

        public static SplunkIO.Write write​(org.apache.beam.sdk.options.ValueProvider<java.lang.String> url,
                                   org.apache.beam.sdk.options.ValueProvider<java.lang.String> token)
        Same as write(String, String) but with ValueProvider.
        Parameters:
        url - splunk hec url
        token - splunk hec authentication token