Class ColGroupDDC
- java.lang.Object
-
- org.apache.sysds.runtime.compress.colgroup.AColGroup
-
- org.apache.sysds.runtime.compress.colgroup.AColGroupCompressed
-
- org.apache.sysds.runtime.compress.colgroup.AColGroupValue
-
- org.apache.sysds.runtime.compress.colgroup.APreAgg
-
- org.apache.sysds.runtime.compress.colgroup.ColGroupDDC
-
- All Implemented Interfaces:
Serializable,Cloneable
- Direct Known Subclasses:
ColGroupDeltaDDC
public class ColGroupDDC extends APreAgg
Class to encapsulate information about a column group that is encoded with dense dictionary encoding (DDC).- See Also:
- Serialized Form
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from class org.apache.sysds.runtime.compress.colgroup.AColGroup
AColGroup.CompressionType
-
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description AColGroupbinaryRowOpLeft(BinaryOperator op, double[] v, boolean isRowSafe)Perform a binary row operation.AColGroupbinaryRowOpRight(BinaryOperator op, double[] v, boolean isRowSafe)Perform a binary row operation.longestimateInMemorySize()Get the upper bound estimate of in memory allocation for the column group.org.apache.sysds.runtime.compress.colgroup.AColGroup.ColGroupTypegetColGroupType()AColGroup.CompressionTypegetCompType()Obtain the compression type.doublegetCost(ComputationCostEstimator e, int nRows)Get the computation cost associated with this column group.int[]getCounts(int[] counts)longgetExactSizeOnDisk()Returns the exact serialized size of column group.doublegetIdx(int r, int colIdx)Get the value at a colGroup specific row/column index position.voidleftMultByMatrixNoPreAgg(MatrixBlock matrix, MatrixBlock result, int rl, int ru, int cl, int cu)Left multiply with this column group.voidpreAggregateDense(MatrixBlock m, double[] preAgg, int rl, int ru, int cl, int cu)Pre aggregate a dense matrix block into a pre aggregate target (first step of left matrix multiplication)voidpreAggregateSparse(SparseBlock sb, double[] preAgg, int rl, int ru)voidpreAggregateThatDDCStructure(ColGroupDDC that, Dictionary ret)voidpreAggregateThatSDCSingleZerosStructure(ColGroupSDCSingleZeros that, Dictionary ret)voidpreAggregateThatSDCZerosStructure(ColGroupSDCZeros that, Dictionary ret)voidreadFields(DataInput in)Deserialize column group from data input.booleansameIndexStructure(AColGroupCompressed that)AColGroupscalarOperation(ScalarOperator op)Perform the specified scalar operation directly on the compressed column group, without decompressing individual cells if possible.StringtoString()AColGroupunaryOperation(UnaryOperator op)voidwrite(DataOutput out)Serializes column group to data output.-
Methods inherited from class org.apache.sysds.runtime.compress.colgroup.APreAgg
getPreAggregateSize, leftMultByAColGroup, mmWithDictionary, preAggregate, preAggregateThatIndexStructure, tsmmAColGroup
-
Methods inherited from class org.apache.sysds.runtime.compress.colgroup.AColGroupValue
centralMoment, computeColSums, containsValue, copy, decompressToDenseBlock, decompressToSparseBlock, forceMatrixBlockDictionary, getCachedCounts, getCounts, getDictionary, getNumberNonZeros, getNumValues, replace, rexpandCols, rightMultByMatrix
-
Methods inherited from class org.apache.sysds.runtime.compress.colgroup.AColGroupCompressed
getMax, getMin, preAggRows, tsmm, unaryAggregateOperations, unaryAggregateOperations
-
Methods inherited from class org.apache.sysds.runtime.compress.colgroup.AColGroup
colSum, decompressToDenseBlock, decompressToSparseBlock, get, getColIndices, getNumCols, shiftColIndices, sliceColumn, sliceColumns
-
-
-
-
Method Detail
-
getCompType
public AColGroup.CompressionType getCompType()
Description copied from class:AColGroupObtain the compression type.- Specified by:
getCompTypein classAColGroup- Returns:
- How the elements of the column group are compressed.
-
getIdx
public double getIdx(int r, int colIdx)Description copied from class:AColGroupGet the value at a colGroup specific row/column index position.
-
getCounts
public int[] getCounts(int[] counts)
- Specified by:
getCountsin classAColGroupValue
-
leftMultByMatrixNoPreAgg
public void leftMultByMatrixNoPreAgg(MatrixBlock matrix, MatrixBlock result, int rl, int ru, int cl, int cu)
Description copied from class:AColGroupLeft multiply with this column group.- Specified by:
leftMultByMatrixNoPreAggin classAColGroup- Parameters:
matrix- The matrix to multiply with on the leftresult- The result to output the values into, always dense for the purpose of the column groups parallelizingrl- The row to begin the multiplication from on the lhs matrixru- The row to end the multiplication at on the lhs matrixcl- The column to begin the multiplication from on the lhs matrixcu- The column to end the multiplication at on the lhs matrix
-
preAggregateDense
public void preAggregateDense(MatrixBlock m, double[] preAgg, int rl, int ru, int cl, int cu)
Description copied from class:APreAggPre aggregate a dense matrix block into a pre aggregate target (first step of left matrix multiplication)- Specified by:
preAggregateDensein classAPreAgg- Parameters:
m- The matrix to preAggregatepreAgg- The preAggregate targetrl- Row lower on the left side matrixru- Row upper on the left side matrixcl- Column lower on the left side matrix (or row lower in the column group)cu- Column upper on the left side matrix (or row upper in the column group)
-
preAggregateSparse
public void preAggregateSparse(SparseBlock sb, double[] preAgg, int rl, int ru)
- Specified by:
preAggregateSparsein classAPreAgg
-
preAggregateThatDDCStructure
public void preAggregateThatDDCStructure(ColGroupDDC that, Dictionary ret)
-
preAggregateThatSDCZerosStructure
public void preAggregateThatSDCZerosStructure(ColGroupSDCZeros that, Dictionary ret)
-
preAggregateThatSDCSingleZerosStructure
public void preAggregateThatSDCSingleZerosStructure(ColGroupSDCSingleZeros that, Dictionary ret)
-
sameIndexStructure
public boolean sameIndexStructure(AColGroupCompressed that)
-
getColGroupType
public org.apache.sysds.runtime.compress.colgroup.AColGroup.ColGroupType getColGroupType()
-
estimateInMemorySize
public long estimateInMemorySize()
Description copied from class:AColGroupGet the upper bound estimate of in memory allocation for the column group.- Overrides:
estimateInMemorySizein classAColGroupValue- Returns:
- an upper bound on the number of bytes used to store this ColGroup in memory.
-
scalarOperation
public AColGroup scalarOperation(ScalarOperator op)
Description copied from class:AColGroupPerform the specified scalar operation directly on the compressed column group, without decompressing individual cells if possible.- Specified by:
scalarOperationin classAColGroup- Parameters:
op- operation to perform- Returns:
- version of this column group with the operation applied
-
unaryOperation
public AColGroup unaryOperation(UnaryOperator op)
- Specified by:
unaryOperationin classAColGroup
-
binaryRowOpLeft
public AColGroup binaryRowOpLeft(BinaryOperator op, double[] v, boolean isRowSafe)
Description copied from class:AColGroupPerform a binary row operation.- Specified by:
binaryRowOpLeftin classAColGroup- Parameters:
op- The operation to executev- The vector of values to apply, should be same length as dictionary length.isRowSafe- True if the binary op is applied to an entire zero row and all results are zero- Returns:
- A updated column group with the new values.
-
binaryRowOpRight
public AColGroup binaryRowOpRight(BinaryOperator op, double[] v, boolean isRowSafe)
Description copied from class:AColGroupPerform a binary row operation.- Specified by:
binaryRowOpRightin classAColGroup- Parameters:
op- The operation to executev- The vector of values to apply, should be same length as dictionary length.isRowSafe- True if the binary op is applied to an entire zero row and all results are zero- Returns:
- A updated column group with the new values.
-
write
public void write(DataOutput out) throws IOException
Description copied from class:AColGroupSerializes column group to data output.- Overrides:
writein classAColGroupValue- Parameters:
out- data output- Throws:
IOException- if IOException occurs
-
readFields
public void readFields(DataInput in) throws IOException
Description copied from class:AColGroupDeserialize column group from data input.- Overrides:
readFieldsin classAColGroupValue- Parameters:
in- data input- Throws:
IOException- if IOException occurs
-
getExactSizeOnDisk
public long getExactSizeOnDisk()
Description copied from class:AColGroupReturns the exact serialized size of column group. This can be used for example for buffer preallocation.- Overrides:
getExactSizeOnDiskin classAColGroupValue- Returns:
- exact serialized size for column group
-
getCost
public double getCost(ComputationCostEstimator e, int nRows)
Description copied from class:AColGroupGet the computation cost associated with this column group.
-
toString
public String toString()
- Overrides:
toStringin classAColGroupValue
-
-