Package org.apache.sysds.runtime.io

Class FrameReaderDeltaParallel

java.lang.Object

org.apache.sysds.runtime.io.FrameReader

org.apache.sysds.runtime.io.FrameReaderDelta

org.apache.sysds.runtime.io.FrameReaderDeltaParallel

public class FrameReaderDeltaParallel
extends FrameReaderDelta

Parallel native Delta Lake frame reader. Delta tables are stored as one or more parquet data files; this reader decodes those files concurrently (one task per data file) and assembles them into a column-major FrameBlock in the original file order.

It mirrors ReaderDeltaParallel (the matrix variant) but produces typed column Arrays instead of a dense double[]. As with the matrix reader, the expensive part of a Delta read is the per-file parquet decode, so parallelizing across data files is the natural speedup. A table backed by a single data file cannot be split this way, so the reader transparently falls back to the sequential FrameReaderDelta.

Constructor Summary

Constructors

Constructor	Description
FrameReaderDeltaParallel()

Method Summary

Modifier and Type	Method	Description
FrameBlock	readFrameFromHDFS(String fname, Types.ValueType[] schema, String[] names, long rlen, long clen)

Methods inherited from class org.apache.sysds.runtime.io.FrameReaderDelta

readFrameFromInputStream

Methods inherited from class org.apache.sysds.runtime.io.FrameReader

getDefColNames, getDefSchema, readFrameFromHDFS, readFrameFromHDFS, readFrameFromInputStream, readFrameFromInputStream

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

FrameReaderDeltaParallel

public FrameReaderDeltaParallel()

Method Details

readFrameFromHDFS

public FrameBlock readFrameFromHDFS(String fname,
 Types.ValueType[] schema,
 String[] names,
 long rlen,
 long clen)
                             throws IOException,
DMLRuntimeException

Overrides:

readFrameFromHDFS in class FrameReaderDelta

Throws:

IOException

DMLRuntimeException