Package org.apache.sysds.runtime.instructions.ooc

Class OOCEvictionManager

java.lang.Object

org.apache.sysds.runtime.instructions.ooc.OOCEvictionManager

public class OOCEvictionManager
extends Object

Eviction Manager for the Out-Of-Core (OOC) stream cache.

This manager implements a high-performance, thread-safe buffer pool designed to handle intermediate results that exceed available heap memory. It employs a partitioned eviction strategy to maximize disk throughput and a lock-striped concurrency model to minimize thread contention.

1. Purpose

Provides a bounded cache for MatrixBlocks produced and consumed by OOC streaming operators (e.g., tsmm, ba+*). When memory pressure exceeds a configured limit, blocks are transparently evicted to disk and restored on demand, allowing execution of operations larger than RAM.

2. Lifecycle Management

Blocks transition atomically through three states to ensure data consistency:

• HOT: The block is pinned in the JVM heap (value != null).
• EVICTING: A transition state. The block is currently being written to disk. Concurrent readers must wait on the entry's condition variable.
• COLD: The block is persisted on disk. The heap reference is nulled out to free memory, but the container (metadata) remains in the cache map.

3. Eviction Strategy (Partitioned I/O)

To mitigate I/O thrashing caused by writing thousands of small blocks:

• Eviction is partition-based: Groups of "HOT" blocks are gathered into batches (e.g., 64MB) and written sequentially to a single partition file.
• This converts random I/O into high-throughput sequential I/O.
• A separate metadata map tracks the (partitionId, offset) for every evicted block, allowing random-access reloading.

4. Data Integrity (Re-hydration)

To prevent index corruption during serialization/deserialization cycles, this manager uses a "re-hydration" model. The IndexedMatrixValue container is never removed from the cache structure. Eviction only nulls the data payload. Loading restores the data into the existing container, preserving the original MatrixIndexes.

5. Concurrency Model (Fine-Grained Locking)

• Global Structure Lock: A coarse-grained lock (_cacheLock) guards the LinkedHashMap structure against concurrent insertions, deletions, and iteration during eviction selection.
• Per-Block Locks: Each BlockEntry owns an independent ReentrantLock. This decouples I/O operations, allowing a reader to load "Block A" from disk while the evictor writes "Block B" to disk simultaneously, maximizing throughput.
• Condition Queues: To handle read-write races, the system uses atomic state transitions. If a reader attempts to access a block in the EVICTING state, it waits on the entry's Condition variable until the writer signals that the block is safely COLD (persisted).

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static enum	OOCEvictionManager.RPolicy

Constructor Summary

Constructors

Constructor	Description
OOCEvictionManager()

Method Summary

Modifier and Type	Method	Description
static void	forget(long streamId, int blockId)	Removes a block from the cache without setting its data to null.
static IndexedMatrixValue	get(long streamId, int blockId)	Get a block from the OOC cache (deserialize on read)
static void	put(long streamId, int blockId, IndexedMatrixValue value)	Store a block in the OOC cache (serialize once)
static void	reset()

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

OOCEvictionManager

public OOCEvictionManager()

Method Details

reset

public static void reset()

forget

public static void forget(long streamId,
 int blockId)

Removes a block from the cache without setting its data to null.

put

public static void put(long streamId,
 int blockId,
 IndexedMatrixValue value)

Store a block in the OOC cache (serialize once)

get

public static IndexedMatrixValue get(long streamId,
 int blockId)

Get a block from the OOC cache (deserialize on read)