Package org.apache.sysds.runtime.transform.encode

Class LegacyEncoder

java.lang.Object
org.apache.sysds.runtime.transform.encode.LegacyEncoder
All Implemented Interfaces:
Externalizable, Serializable
Direct Known Subclasses:
EncoderMVImpute, EncoderOmit
public abstract class LegacyEncoder extends Object implements Externalizable
Base class for all transform encoders providing both a row and block interface for decoding frames to matrices.
See Also:

  • Method Details

    • getColList

      public int[] getColList()

    • setColList

      public void setColList(int[] colList)

    • initColList

      public int initColList(org.apache.wink.json4j.JSONArray attrs)

    • initColList

      public int initColList(int[] colList)

    • isApplicable

      public boolean isApplicable()
      Indicates if this encoder is applicable, i.e, if there is at least one column to encode.
      Returns:
      true if at least one column to encode

    • isApplicable

      public int isApplicable(int colID)
      Indicates if this encoder is applicable for the given column ID, i.e., if it is subject to this transformation.
      Parameters:
      colID - column ID
      Returns:
      true if encoder is applicable for given column

    • encode

      public abstract MatrixBlock encode(FrameBlock in, MatrixBlock out)
      Block encode: build and apply (transform encode).
      Parameters:
      in - input frame block
      out - output matrix block
      Returns:
      output matrix block

    • build

      public abstract void build(FrameBlock in)
      Build the transform meta data for the given block input. This call modifies and keeps meta data as encoder state.
      Parameters:
      in - input frame block

    • prepareBuildPartial

      public void prepareBuildPartial()
      Allocates internal data structures for partial build.

    • buildPartial

      public void buildPartial(FrameBlock in)
      Partial build of internal data structures (e.g., in distributed spark operations).
      Parameters:
      in - input frame block

    • apply

      public abstract MatrixBlock apply(FrameBlock in, MatrixBlock out)
      Encode input data blockwise according to existing transform meta data (transform apply).
      Parameters:
      in - input frame block
      out - output matrix block
      Returns:
      output matrix block

    • subRangeEncoder

      public LegacyEncoder subRangeEncoder(IndexRange ixRange)
      Returns a new Encoder that only handles a sub range of columns.
      Parameters:
      ixRange - the range (1-based, begin inclusive, end exclusive)
      Returns:
      an encoder of the same type, just for the sub-range

    • mergeAt

      public void mergeAt(LegacyEncoder other, int row, int col)
      Merges another encoder, of a compatible type, in after a certain position. Resizes as necessary. Encoders are compatible with themselves and EncoderComposite is compatible with every other Encoder.
      Parameters:
      other - the encoder that should be merged in
      row - the row where it should be placed (1-based)
      col - the col where it should be placed (1-based)

    • updateIndexRanges

      public void updateIndexRanges(long[] beginDims, long[] endDims)
      Update index-ranges to after encoding. Note that only Dummycoding changes the ranges.
      Parameters:
      beginDims - begin dimensions of range
      endDims - end dimensions of range

    • getMetaData

      public abstract FrameBlock getMetaData(FrameBlock out)
      Construct a frame block out of the transform meta data.
      Parameters:
      out - output frame block
      Returns:
      output frame block?

    • initMetaData

      public abstract void initMetaData(FrameBlock meta)
      Sets up the required meta data for a subsequent call to apply.
      Parameters:
      meta - frame block

    • getColMapping

      public MatrixBlock getColMapping(FrameBlock meta, MatrixBlock out)
      Obtain the column mapping of encoded frames based on the passed meta data frame.
      Parameters:
      meta - meta data frame block
      out - output matrix
      Returns:
      matrix with column mapping (one row per attribute)

    • writeExternal

      public void writeExternal(ObjectOutput os) throws IOException
      Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast/rdd serialization.
      Specified by:
      writeExternal in interface Externalizable
      Parameters:
      os - object output
      Throws:
      IOException - if IOException occurs

    • readExternal

      public void readExternal(ObjectInput in) throws IOException
      Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast/rdd deserialization.
      Specified by:
      readExternal in interface Externalizable
      Parameters:
      in - object input
      Throws:
      IOException - if IOException occur

    • shiftCols

      public void shiftCols(int offset)