Class MultiColumnEncoder
- java.lang.Object
-
- org.apache.sysds.runtime.transform.encode.MultiColumnEncoder
-
- All Implemented Interfaces:
Externalizable,Serializable,Encoder
public class MultiColumnEncoder extends Object implements Encoder
- See Also:
- Serialized Form
-
-
Field Summary
Fields Modifier and Type Field Description static booleanAPPLY_ENCODER_SEPARATE_STAGESstatic booleanMULTI_THREADED_STAGES
-
Constructor Summary
Constructors Constructor Description MultiColumnEncoder()MultiColumnEncoder(List<ColumnEncoderComposite> columnEncoders)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description <T extends LegacyEncoder>
voidaddReplaceLegacyEncoder(T encoder)voidallocateMetaData(FrameBlock meta)Pre-allocate a FrameBlock for metadata collection.MatrixBlockapply(CacheBlock in)MatrixBlockapply(CacheBlock in, int k)MatrixBlockapply(CacheBlock in, MatrixBlock out, int outputCol)Apply the generated metadata to the FrameBlock and saved the result in out.MatrixBlockapply(CacheBlock in, MatrixBlock out, int outputCol, int k)voidapplyColumnOffset()<T extends ColumnEncoder,E>
voidapplyToAll(Class<T> type, Consumer<? super T> function)<T extends ColumnEncoder,E>
voidapplyToAll(Consumer<? super ColumnEncoderComposite> function)voidbuild(CacheBlock in)Build the transform meta data for the given block input.voidbuild(CacheBlock in, int k)voidbuild(CacheBlock in, int k, Map<Integer,double[]> equiHeightBinMaxs)voidbuildPartial(FrameBlock in)Partial build of internal data structures (e.g., in distributed spark operations).<T extends ColumnEncoder>
booleancontainsEncoderForID(int colID, Class<T> type)MatrixBlockencode(CacheBlock in)MatrixBlockencode(CacheBlock in, int k)MatrixBlockgetColMapping(FrameBlock meta)Obtain the column mapping of encoded frames based on the passed meta data frame.<T extends ColumnEncoder>
TgetColumnEncoder(int colID, Class<T> type)List<ColumnEncoderComposite>getColumnEncoders()<T extends ColumnEncoder>
List<T>getColumnEncoders(Class<T> type)List<ColumnEncoderComposite>getCompositeEncodersForID(int colID)List<Class<? extends ColumnEncoder>>getEncoderTypes()List<Class<? extends ColumnEncoder>>getEncoderTypes(int colID)<T extends ColumnEncoder,E>
List<E>getFromAll(Class<T> type, Function<? super T,? extends E> mapper)<T extends ColumnEncoder>
double[]getFromAllDoubleArray(Class<T> type, Function<? super T,? extends Double> mapper)<T extends ColumnEncoder>
int[]getFromAllIntArray(Class<T> type, Function<? super T,? extends Integer> mapper)<T extends LegacyEncoder>
TgetLegacyEncoder(Class<T> type)FrameBlockgetMetaData(FrameBlock meta)Construct a frame block out of the transform meta data.FrameBlockgetMetaData(FrameBlock meta, int k)intgetNumExtraCols()intgetNumExtraCols(IndexRange ixRange)<T extends LegacyEncoder>
booleanhasLegacyEncoder()<T extends LegacyEncoder>
booleanhasLegacyEncoder(Class<T> type)voidinitMetaData(FrameBlock meta)Sets up the required meta data for a subsequent call to apply.voidlegacyBuild(FrameBlock in)voidmergeAt(Encoder other, int columnOffset, int row)voidmergeReplace(MultiColumnEncoder multiEncoder)voidprepareBuildPartial()Allocates internal data structures for partial build.voidreadExternal(ObjectInput in)MultiColumnEncodersubRangeEncoder(IndexRange ixRange)<T extends ColumnEncoder>
MultiColumnEncodersubRangeEncoder(IndexRange ixRange, Class<T> type)voidupdateIndexRanges(long[] beginDims, long[] endDims, int offset)Update index-ranges to after encoding.voidwriteExternal(ObjectOutput out)
-
-
-
Constructor Detail
-
MultiColumnEncoder
public MultiColumnEncoder(List<ColumnEncoderComposite> columnEncoders)
-
MultiColumnEncoder
public MultiColumnEncoder()
-
-
Method Detail
-
encode
public MatrixBlock encode(CacheBlock in)
-
encode
public MatrixBlock encode(CacheBlock in, int k)
-
build
public void build(CacheBlock in)
Description copied from interface:EncoderBuild the transform meta data for the given block input. This call modifies and keeps meta data as encoder state.
-
build
public void build(CacheBlock in, int k)
-
build
public void build(CacheBlock in, int k, Map<Integer,double[]> equiHeightBinMaxs)
-
legacyBuild
public void legacyBuild(FrameBlock in)
-
apply
public MatrixBlock apply(CacheBlock in)
-
apply
public MatrixBlock apply(CacheBlock in, int k)
-
apply
public MatrixBlock apply(CacheBlock in, MatrixBlock out, int outputCol)
Description copied from interface:EncoderApply the generated metadata to the FrameBlock and saved the result in out.
-
apply
public MatrixBlock apply(CacheBlock in, MatrixBlock out, int outputCol, int k)
-
allocateMetaData
public void allocateMetaData(FrameBlock meta)
Description copied from interface:EncoderPre-allocate a FrameBlock for metadata collection.- Specified by:
allocateMetaDatain interfaceEncoder- Parameters:
meta- frame block
-
getMetaData
public FrameBlock getMetaData(FrameBlock meta)
Description copied from interface:EncoderConstruct a frame block out of the transform meta data.- Specified by:
getMetaDatain interfaceEncoder- Parameters:
meta- output frame block- Returns:
- output frame block?
-
getMetaData
public FrameBlock getMetaData(FrameBlock meta, int k)
-
initMetaData
public void initMetaData(FrameBlock meta)
Description copied from interface:EncoderSets up the required meta data for a subsequent call to apply.- Specified by:
initMetaDatain interfaceEncoder- Parameters:
meta- frame block
-
prepareBuildPartial
public void prepareBuildPartial()
Description copied from interface:EncoderAllocates internal data structures for partial build.- Specified by:
prepareBuildPartialin interfaceEncoder
-
buildPartial
public void buildPartial(FrameBlock in)
Description copied from interface:EncoderPartial build of internal data structures (e.g., in distributed spark operations).- Specified by:
buildPartialin interfaceEncoder- Parameters:
in- input frame block
-
getColMapping
public MatrixBlock getColMapping(FrameBlock meta)
Obtain the column mapping of encoded frames based on the passed meta data frame.- Parameters:
meta- meta data frame block- Returns:
- matrix with column mapping (one row per attribute)
-
updateIndexRanges
public void updateIndexRanges(long[] beginDims, long[] endDims, int offset)Description copied from interface:EncoderUpdate index-ranges to after encoding. Note that only Dummycoding changes the ranges.- Specified by:
updateIndexRangesin interfaceEncoder- Parameters:
beginDims- begin dimensions of rangeendDims- end dimensions of rangeoffset- is applied to begin and endDims
-
writeExternal
public void writeExternal(ObjectOutput out) throws IOException
- Specified by:
writeExternalin interfaceExternalizable- Throws:
IOException
-
readExternal
public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException
- Specified by:
readExternalin interfaceExternalizable- Throws:
IOExceptionClassNotFoundException
-
getColumnEncoders
public <T extends ColumnEncoder> List<T> getColumnEncoders(Class<T> type)
-
getColumnEncoder
public <T extends ColumnEncoder> T getColumnEncoder(int colID, Class<T> type)
-
getFromAll
public <T extends ColumnEncoder,E> List<E> getFromAll(Class<T> type, Function<? super T,? extends E> mapper)
-
getFromAllIntArray
public <T extends ColumnEncoder> int[] getFromAllIntArray(Class<T> type, Function<? super T,? extends Integer> mapper)
-
getFromAllDoubleArray
public <T extends ColumnEncoder> double[] getFromAllDoubleArray(Class<T> type, Function<? super T,? extends Double> mapper)
-
getColumnEncoders
public List<ColumnEncoderComposite> getColumnEncoders()
-
getCompositeEncodersForID
public List<ColumnEncoderComposite> getCompositeEncodersForID(int colID)
-
getEncoderTypes
public List<Class<? extends ColumnEncoder>> getEncoderTypes(int colID)
-
getEncoderTypes
public List<Class<? extends ColumnEncoder>> getEncoderTypes()
-
getNumExtraCols
public int getNumExtraCols()
-
getNumExtraCols
public int getNumExtraCols(IndexRange ixRange)
-
containsEncoderForID
public <T extends ColumnEncoder> boolean containsEncoderForID(int colID, Class<T> type)
-
applyToAll
public <T extends ColumnEncoder,E> void applyToAll(Class<T> type, Consumer<? super T> function)
-
applyToAll
public <T extends ColumnEncoder,E> void applyToAll(Consumer<? super ColumnEncoderComposite> function)
-
subRangeEncoder
public MultiColumnEncoder subRangeEncoder(IndexRange ixRange)
-
subRangeEncoder
public <T extends ColumnEncoder> MultiColumnEncoder subRangeEncoder(IndexRange ixRange, Class<T> type)
-
mergeReplace
public void mergeReplace(MultiColumnEncoder multiEncoder)
-
mergeAt
public void mergeAt(Encoder other, int columnOffset, int row)
-
addReplaceLegacyEncoder
public <T extends LegacyEncoder> void addReplaceLegacyEncoder(T encoder)
-
hasLegacyEncoder
public <T extends LegacyEncoder> boolean hasLegacyEncoder()
-
hasLegacyEncoder
public <T extends LegacyEncoder> boolean hasLegacyEncoder(Class<T> type)
-
getLegacyEncoder
public <T extends LegacyEncoder> T getLegacyEncoder(Class<T> type)
-
applyColumnOffset
public void applyColumnOffset()
-
-