Package org.apache.druid.segment.loading

Class StorageLocation

java.lang.Object

org.apache.druid.segment.loading.StorageLocation

@ThreadSafe
public class StorageLocation
extends Object

Logical representation of a local disk path to store CacheEntry, controlling that the total size of stored files doesn't exceed the maxSizeBytes and available space is always kept smaller than freeSpaceToKeep.

CacheEntry can be stored in two manners in a storage location. The first is to store the entry indefinitely with reserve(CacheEntry), where the space of the entry is accounted for and the storage space will not be recovered until release(CacheEntry) is called. These entries are stored in staticCacheEntries.

The second way is to store as a transient cache item with one of reserveWeak(CacheEntry), addWeakReservationHold(CacheEntryIdentifier, Supplier), or addWeakReservationHoldIfExists(CacheEntryIdentifier). CacheEntry stored in this manner will exist on disk in this location until the point that another new reservation needs more space than remains available in the location, at which point reclaim(long) will be called to try to call CacheEntry.unmount() on any eligible entries until enough space is available to store the new item.

Items are chosen for eviction using an algorithm based on SIEVE with additional mechanisms to place temporary holds on cache entries. The holds are required to support cases such as if a group of cache entries are taking part in a query and must all be loaded simultaneously before processing can continue.

Implementation-wise, CacheEntry are wrapped in StorageLocation.WeakCacheEntry to form a doubly linked list functioning as a queue which can be interacted with via 3 fields, head, tail, and hand. Head and tail expectedly mark the beginning and end of the queue, while the hand is the interesting bit and is used as the position of the current item to consider for eviction the next time reclaim(long) needs to be called. Entries are also stored in a map, weakCacheEntries, for fast retrieval. Using a weak cache entry sets StorageLocation.WeakCacheEntry.visited to true, and addWeakReservationHold(CacheEntryIdentifier, Supplier) and addWeakReservationHoldIfExists(CacheEntryIdentifier) additionally will call StorageLocation.WeakCacheEntry.hold() until StorageLocation.ReservationHold.close() is called which will call StorageLocation.WeakCacheEntry.release().

When it is time to reclaim space, first the hand is checked for holds - if any exist then the hand is moved to StorageLocation.WeakCacheEntry.prev immediately and we try again. If no holds are present, then it is checked if it has been marked as StorageLocation.WeakCacheEntry.visited - if so then it is unmarked as visited, and the hand moves to StorageLocation.WeakCacheEntry.prev (allowing this entry to be reclaimed the next time we pass if it has not been visited again). Lastly, if neither under a hold or marked, the entry will be unlinked from the queue AND unmounted from the storage location (deleting the files from disk) with unlinkWeakEntry(WeakCacheEntry). This process is repeated until either a sufficient amount of space has been reclaimed, or no additional space is able to be reclaimed, in which case the new reservation fails.

There is an auxilary mode for weak references when areWeakEntriesEphemeral is set to true. In this mode, weak entries are short-lived entries that only exist while one or more reservation hold are active, and are unmounted when the holds are released. This is useful in cases where entries are unlikely to be re-used such as in asynchronous tasks.

This class is thread-safe, so that multiple threads can update its state at the same time. One example usage is that a historical server can use multiple threads to load different segments in parallel from deep storage.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static final class	StorageLocation.ReclaimResult
static class	StorageLocation.ReservationHold<TEntry extends CacheEntry>	A Closeable CacheEntry wrapper representing both the entry and a 'hold' that is placed on it to prevent the entry from being dropped by reclaim(long) at least until the wrapper is closed.
static final class	StorageLocation.StaticStats
static final class	StorageLocation.WeakStats

Constructor Summary

Constructors

Constructor	Description
StorageLocation(File path, long maxSizeBytes, Double freeSpacePercent)

Method Summary

Modifier and Type	Method	Description
<T extends CacheEntry> StorageLocation.ReservationHold<T>	addWeakReservationHold(CacheEntryIdentifier entryId, Supplier<? extends CacheEntry> entrySupplier)	Returns a StorageLocation.ReservationHold of a CacheEntry with a 'hold' placed on it, preventing it from being automatically removed by reclaim(long) if the CacheEntry is one of weakCacheEntries until the hold is released by StorageLocation.ReservationHold.close().
<T extends CacheEntry> StorageLocation.ReservationHold<T>	addWeakReservationHoldIfExists(CacheEntryIdentifier entryId)	Returns a StorageLocation.ReservationHold of a CacheEntry with a 'hold' placed on it, preventing it from being automatically removed by reclaim(long) if the CacheEntry is one of weakCacheEntries until the hold is released by StorageLocation.ReservationHold.close().
long	availableSizeBytes()
long	currentSizeBytes()
long	currentWeakSizeBytes()
<T extends CacheEntry> T	getCacheEntry(CacheEntryIdentifier entryId)	If a CacheEntry for a CacheEntryIdentifier exists in either staticCacheEntries or weakCacheEntries, this method will return it.
ReadWriteLock	getLock()	Exposes the lock used by the StorageLocation to allow CacheEntry.mount(StorageLocation) and CacheEntry.unmount() to synchronize operations with this location.
File	getPath()	The place where the files are stored
<T extends CacheEntry> T	getStaticCacheEntry(CacheEntryIdentifier entryId)
int	getWeakEntryCount()
StorageLocation.WeakStats	getWeakStats()
boolean	isReserved(CacheEntryIdentifier identifier)	Returns true if a CacheEntry for a CacheEntryIdentifier exists in staticCacheEntries
boolean	isWeakReserved(CacheEntryIdentifier identifier)	Returns true if a CacheEntry for a CacheEntryIdentifier exists in weakCacheEntries
void	release(CacheEntry entry)	Removes an item from staticCacheEntries, reducing currSizeBytes by CacheEntry.getSize().
boolean	reserve(CacheEntry entry)	Reserves space to store the given CacheEntry, returning true if sucessful and false if already reserved, or unable to be reserved.
boolean	reserveWeak(CacheEntry entry)	Reserves space to store a 'weak' reservation for a given CacheEntry.
void	reset()	Unmounts all static and weakly held cache entries and resets stats and size tracking.
StorageLocation.StaticStats	resetStaticStats()
StorageLocation.WeakStats	resetWeakStats()
void	setAreWeakEntriesEphemeral(boolean areWeakEntriesEphemeral)	Sets whether weak cache entries should be retained after all holds are released.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

StorageLocation

public StorageLocation(File path,
 long maxSizeBytes,
 @Nullable
 Double freeSpacePercent)

Method Details

getLock

public ReadWriteLock getLock()

Exposes the lock used by the StorageLocation to allow CacheEntry.mount(StorageLocation) and CacheEntry.unmount() to synchronize operations with this location. Callers MUST pay attention to the type of lock they are using if planning to call any other methods of this class to ensure deadlocks are not possible.

getPath

public File getPath()

The place where the files are stored

setAreWeakEntriesEphemeral

public void setAreWeakEntriesEphemeral(boolean areWeakEntriesEphemeral)

Sets whether weak cache entries should be retained after all holds are released. If true, weak references are removed and unmounted immediately after all holds are released

getStaticCacheEntry

public <T extends CacheEntry> T getStaticCacheEntry(CacheEntryIdentifier entryId)

getCacheEntry

@Nullable
public <T extends CacheEntry> T getCacheEntry(CacheEntryIdentifier entryId)

If a CacheEntry for a CacheEntryIdentifier exists in either staticCacheEntries or weakCacheEntries, this method will return it. Additionally, StorageLocation.WeakCacheEntry will be marked as visited to reduce the chance that they are evicted during future calls to reclaim(long).

isReserved

public boolean isReserved(CacheEntryIdentifier identifier)

Returns true if a CacheEntry for a CacheEntryIdentifier exists in staticCacheEntries

isWeakReserved

public boolean isWeakReserved(CacheEntryIdentifier identifier)

Returns true if a CacheEntry for a CacheEntryIdentifier exists in weakCacheEntries

reserve

public boolean reserve(CacheEntry entry)

Reserves space to store the given CacheEntry, returning true if sucessful and false if already reserved, or unable to be reserved.

reserveWeak

public boolean reserveWeak(CacheEntry entry)

Reserves space to store a 'weak' reservation for a given CacheEntry. Returns true if already reserved or was able to be successfully reserved, or false if unable to be reserved. This method is intended for use during 'bootstrapping'. To use weak cache entries in a query engine use addWeakReservationHold(CacheEntryIdentifier, Supplier) or addWeakReservationHoldIfExists(CacheEntryIdentifier), which places a hold on cache entries to prevent eviction until the hold is released.

addWeakReservationHoldIfExists

@Nullable
public <T extends CacheEntry>
StorageLocation.ReservationHold<T> addWeakReservationHoldIfExists(CacheEntryIdentifier entryId)

Returns a StorageLocation.ReservationHold of a CacheEntry with a 'hold' placed on it, preventing it from being automatically removed by reclaim(long) if the CacheEntry is one of weakCacheEntries until the hold is released by StorageLocation.ReservationHold.close(). Callers must call close on the returned object.

This method only returns already existing entries, if callers want to insert a new entry if it doesn't already exist, use addWeakReservationHold(CacheEntryIdentifier, Supplier)

addWeakReservationHold

@Nullable
public <T extends CacheEntry>
StorageLocation.ReservationHold<T> addWeakReservationHold(CacheEntryIdentifier entryId,
 Supplier<? extends CacheEntry> entrySupplier)

Returns a StorageLocation.ReservationHold of a CacheEntry with a 'hold' placed on it, preventing it from being automatically removed by reclaim(long) if the CacheEntry is one of weakCacheEntries until the hold is released by StorageLocation.ReservationHold.close(). Callers must call close on the returned object.

If the entry already exists, this method will return it, else it will create a new entry if there is space available.

release

public void release(CacheEntry entry)

Removes an item from staticCacheEntries, reducing currSizeBytes by CacheEntry.getSize(). If the cache entry exists in weakCacheEntries, it is left in place to be removed by reclaim(long) instead.

getWeakEntryCount

public int getWeakEntryCount()

reset

public void reset()

Unmounts all static and weakly held cache entries and resets stats and size tracking. Currently only for testing.

getWeakStats

public StorageLocation.WeakStats getWeakStats()

resetStaticStats

public StorageLocation.StaticStats resetStaticStats()

resetWeakStats

public StorageLocation.WeakStats resetWeakStats()

availableSizeBytes

public long availableSizeBytes()

currentSizeBytes

public long currentSizeBytes()

currentWeakSizeBytes

public long currentWeakSizeBytes()