Package org.apache.sysds.runtime.compress.colgroup

Class ADictBasedColGroup

java.lang.Object

org.apache.sysds.runtime.compress.colgroup.AColGroup

org.apache.sysds.runtime.compress.colgroup.AColGroupCompressed

org.apache.sysds.runtime.compress.colgroup.ADictBasedColGroup

All Implemented Interfaces:

Serializable, IContainADictionary

Direct Known Subclasses:

AColGroupValue, ColGroupConst

public abstract class ADictBasedColGroup
extends AColGroupCompressed
implements IContainADictionary

See Also:

• Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.sysds.runtime.compress.colgroup.AColGroup

AColGroup.CompressionType

Method Summary

Modifier and Type	Method	Description
final AColGroup	copyAndSet(IDictionary newDictionary)	This method copies the column group and sets the dictionary to another dictionary.
final AColGroup	copyAndSet(IColIndex colIndexes)	Copy the content of the column group with pointers to the previous content but with new column given Note this method does not verify if the colIndexes specified are valid and correct dimensions for the underlying column groups.
final void	decompressToDenseBlock(DenseBlock db, int rl, int ru, int offR, int offC)	Decompress into the DenseBlock.
final void	decompressToDenseBlockTransposed(DenseBlock db, int rl, int ru)	Decompress a range of rows into a dense transposed block.
final void	decompressToSparseBlock(SparseBlock sb, int rl, int ru, int offR, int offC)	Decompress into the SparseBlock.
void	decompressToSparseBlockTransposed(SparseBlockMCSR sb, int nColOut)	Decompress the column group to the sparse transposed block.
long	estimateInMemorySize()	Get the upper bound estimate of in memory allocation for the column group.
IDictionary	getDictionary()
long	getExactSizeOnDisk()	Returns the exact serialized size of column group.
double	getSparsity()	Get an approximate sparsity of this column group
AColGroup	reduceCols()	Perform row sum on the internal dictionaries, and return the same index structure.
final AColGroup	rightMultByMatrix(MatrixBlock right, IColIndex allCols, int k)	Right matrix multiplication with this column group.
void	write(DataOutput out)

Methods inherited from class org.apache.sysds.runtime.compress.colgroup.AColGroupCompressed

getMax, getMin, getSum, isEmpty, preAggRows, sameIndexStructure, sameIndexStructure, tsmm, unaryAggregateOperations, unaryAggregateOperations

Methods inherited from class org.apache.sysds.runtime.compress.colgroup.AColGroup

addVector, append, appendN, binaryRowOpLeft, binaryRowOpRight, centralMoment, clear, colSum, combine, combineWithSameIndex, combineWithSameIndex, computeColSums, containsValue, decompressToDenseBlock, decompressToSparseBlock, get, getColIndices, getCompressionInfo, getCompressionScheme, getCompType, getCost, getEncoding, getIdx, getNumberNonZeros, getNumCols, getNumValues, leftMultByAColGroup, leftMultByMatrixNoPreAgg, morph, recompress, removeEmptyCols, removeEmptyRows, replace, rexpandCols, rightDecompressingMult, rightMultByMatrix, scalarOperation, selectionMultiply, shiftColIndices, sliceColumn, sliceColumns, sliceRows, sort, sortColumnIndexes, splitReshape, splitReshapePushDown, toString, tsmmAColGroup, unaryOperation

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Details

getDictionary

public IDictionary getDictionary()

Specified by:

getDictionary in interface IContainADictionary

decompressToDenseBlockTransposed

public final void decompressToDenseBlockTransposed(DenseBlock db,
 int rl,
 int ru)

Description copied from class: AColGroup

Decompress a range of rows into a dense transposed block.

Specified by:

decompressToDenseBlockTransposed in class AColGroup

Parameters:

db - Dense target block

rl - Row in this column group to start at.

ru - Row in this column group to end at.

decompressToSparseBlockTransposed

public void decompressToSparseBlockTransposed(SparseBlockMCSR sb,
 int nColOut)

Description copied from class: AColGroup

Decompress the column group to the sparse transposed block. Note that the column groups would only need to decompress into specific sub rows of the Sparse block

Specified by:

decompressToSparseBlockTransposed in class AColGroup

Parameters:

sb - Sparse target block

nColOut - The number of columns in the sb.

decompressToDenseBlock

public final void decompressToDenseBlock(DenseBlock db,
 int rl,
 int ru,
 int offR,
 int offC)

Description copied from class: AColGroup

Decompress into the DenseBlock. (no NNZ handling)

Specified by:

decompressToDenseBlock in class AColGroup

Parameters:

db - Target DenseBlock

rl - Row to start decompression from

ru - Row to end decompression at (not inclusive)

offR - Row offset into the target to decompress

offC - Column offset into the target to decompress

decompressToSparseBlock

public final void decompressToSparseBlock(SparseBlock sb,
 int rl,
 int ru,
 int offR,
 int offC)

Description copied from class: AColGroup

Decompress into the SparseBlock. (no NNZ handling) Note this method is allowing to calls to append since it is assumed that the sparse column indexes are sorted afterwards

Specified by:

decompressToSparseBlock in class AColGroup

Parameters:

sb - Target SparseBlock

rl - Row to start decompression from

ru - Row to end decompression at (not inclusive)

offR - Row offset into the target to decompress

offC - Column offset into the target to decompress

write

public void write(DataOutput out)
           throws IOException

Throws:

IOException

getExactSizeOnDisk

public long getExactSizeOnDisk()

Description copied from class: AColGroup

Returns the exact serialized size of column group. This can be used for example for buffer preallocation.

Overrides:

getExactSizeOnDisk in class AColGroup

Returns:

exact serialized size for column group

estimateInMemorySize

public long estimateInMemorySize()

Description copied from class: AColGroup

Get the upper bound estimate of in memory allocation for the column group.

Overrides:

estimateInMemorySize in class AColGroup

Returns:

an upper bound on the number of bytes used to store this ColGroup in memory.

rightMultByMatrix

public final AColGroup rightMultByMatrix(MatrixBlock right,
 IColIndex allCols,
 int k)

Description copied from class: AColGroup

Right matrix multiplication with this column group. This method can return null, meaning that the output overlapping group would have been empty.

Specified by:

rightMultByMatrix in class AColGroup

Parameters:

right - The MatrixBlock on the right of this matrix multiplication

allCols - A pre-materialized list of all col indexes, that can be shared across all column groups if use full, can be set to null.

k - The parallelization degree allowed internally in this operation.

Returns:

The new Column Group or null that is the result of the matrix multiplication.

copyAndSet

public final AColGroup copyAndSet(IColIndex colIndexes)

Description copied from class: AColGroup

Copy the content of the column group with pointers to the previous content but with new column given Note this method does not verify if the colIndexes specified are valid and correct dimensions for the underlying column groups.

Specified by:

copyAndSet in class AColGroup

Parameters:

colIndexes - the new indexes to use in the copy

Returns:

a new object with pointers to underlying data.

copyAndSet

public final AColGroup copyAndSet(IDictionary newDictionary)

This method copies the column group and sets the dictionary to another dictionary. The method shallow copies underlying data structures. NOTE: Be very carefull with this since it invalidate the contracts maintained via standard compression.

Parameters:

newDictionary - A new dictionary to point to.

Returns:

A new shallow copy of the column group

reduceCols

public AColGroup reduceCols()

Description copied from class: AColGroup

Perform row sum on the internal dictionaries, and return the same index structure. This method returns null on empty column groups. Note this method does not guarantee correct behavior if the given group is AMorphingGroup, instead it should be morphed to a valid columngroup via extractCommon first.

Specified by:

reduceCols in class AColGroup

Returns:

The reduced colgroup.

getSparsity

public double getSparsity()

Description copied from class: AColGroup

Get an approximate sparsity of this column group

Specified by:

getSparsity in class AColGroup

Returns:

the approximate sparsity of this columngroup