Package org.apache.druid.server.compaction

Class ReindexingDeletionRule

java.lang.Object

org.apache.druid.server.compaction.AbstractReindexingRule

org.apache.druid.server.compaction.ReindexingDeletionRule

All Implemented Interfaces:

ReindexingRule

public class ReindexingDeletionRule
extends AbstractReindexingRule

A ReindexingRule that specifies patterns to match for removing rows during reindexing.

The deleteWhere clause defines rows to REMOVE from reindexed segments. For example, a `deleteWhere` DimFilter selector(isRobot=true) with AbstractReindexingRule.olderThan P90D will "remove rows where isRobot=true from data older than 90 days". The reindexing framework automatically wraps these delete where clauses in NOT logic during processing.

This is an additive rule. Multiple rules can apply to the same segment. When multiple rules apply, they are combined as NOT(A OR B OR C) where A, B, and C come from distinct ReindexingDeletionRules.

Example usage:

 {
   "id": "remove-robots-90d",
   "olderThan": "P90D",
   "deleteWhere": {
     "type": "selector",
     "dimension": "isRobot",
     "value": "true"
   },
   "description": "Remove robot traffic from data older than 90 days"
 }
 

Virtual column support for filtering on nested fields (MSQ engine only):

It is important to note that when using virtual columns in the filter, the virtual columns must be defined with unique names. Users will have to take care to ensure a rule always has the same unique virtual column names to not impact the fingerprinting of segments reindexed with the rule.

Example inline useage with virtual column:

 {
   "id": "remove-using-nested-field-filter",
   "olderThan": "P90D",
   "deleteWhere": {
     "type": "selector",
     "dimension": "extractedField",
     "value": "unwantedValue"
   },
   "virtualColumns": [
     {
       "type": "expression",
       "name": "extractedField",
       "expression": "json_value(metadata, '$.category')",
       "outputType": "STRING"
     }
   ],
   "description": "Remove rows where metadata.category = 'unwantedValue' from segments older than 90 days"
 }
 

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.druid.server.compaction.ReindexingRule

ReindexingRule.AppliesToMode

Constructor Summary

Constructors

Constructor	Description
ReindexingDeletionRule(String id, String description, org.joda.time.Period olderThan, DimFilter deleteWhere, VirtualColumns virtualColumns)

Method Summary

Modifier and Type	Method	Description
boolean	equals(Object o)
DimFilter	getDeleteWhere()
VirtualColumns	getVirtualColumns()
int	hashCode()
String	toString()

Methods inherited from class org.apache.druid.server.compaction.AbstractReindexingRule

appliesTo, getDescription, getId, getOlderThan

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Details

ReindexingDeletionRule

public ReindexingDeletionRule(@Nonnull
 String id,
 @Nullable
 String description,
 @Nonnull
 org.joda.time.Period olderThan,
 @Nonnull
 DimFilter deleteWhere,
 @Nullable
 VirtualColumns virtualColumns)

Method Details

getDeleteWhere

public DimFilter getDeleteWhere()

getVirtualColumns

@Nullable
public VirtualColumns getVirtualColumns()

equals

public boolean equals(Object o)

Overrides:

equals in class Object

hashCode

public int hashCode()

Overrides:

hashCode in class Object

toString

public String toString()

Overrides:

toString in class Object