Package org.apache.sysds.hops
Class DataOp
- java.lang.Object
-
- org.apache.sysds.hops.Hop
-
- org.apache.sysds.hops.DataOp
-
-
Field Summary
-
Fields inherited from class org.apache.sysds.hops.Hop
_beginColumn, _beginLine, _endColumn, _endLine, _filename, _text, CPThreshold
-
-
Constructor Summary
Constructors Constructor Description DataOp(String l, Types.DataType dt, Types.ValueType vt, Types.OpOpData dop, String fname, long dim1, long dim2, long nnz, int blen)READ operation for Matrix w/ dim1, dim2.DataOp(String l, Types.DataType dt, Types.ValueType vt, Types.OpOpData dop, String fname, long dim1, long dim2, long nnz, MatrixObject.UpdateType update, int blen)DataOp(String l, Types.DataType dt, Types.ValueType vt, Types.OpOpData dop, HashMap<String,Hop> params)READ operation for Matrix / SQL operation for tensor This constructor supports expressions in parametersDataOp(String l, Types.DataType dt, Types.ValueType vt, Types.OpOpData dop, Hop in, HashMap<String,Hop> inputParameters)WRITE operation for Matrix This constructor supports expression in parametersDataOp(String l, Types.DataType dt, Types.ValueType vt, Hop in, Types.OpOpData dop, String fname)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description booleanallowsAllExecTypes()voidcheckArity()Check for N (READ) or N+1 (WRITE) inputs.Objectclone()booleancompare(Hop that)LopconstructLops()voiddisableRecompileRead()Explicitly disables recompilation of transient reads, this additional information is required because requiresRecompile is set in a top-down manner, hence any value set from a consuming operating would be overwritten by opFindExecType.Types.FileFormatgetFileFormat()StringgetFileName()longgetInputBlocksize()Types.OpOpDatagetOp()StringgetOpString()intgetParameterIndex(String name)booleanhasOnlyRDD()booleanisFederatedDataOp()Checks if the hop is a DataOp with federated data.booleanisGPUEnabled()In memory-based optimizer mode (see OptimizerUtils.isMemoryBasedOptLevel()), the exectype is determined by checking this method as well as memory budget of this Hop.booleanisPersistentReadWrite()booleanisRead()booleanisWrite()voidrefreshSizeInformation()Update the output size information for this hop.voidremoveInput(String inputName)Remove an input from the list of inputs and from the parameter index map.voidsetDataOpType(Types.OpOpData type)voidsetFileFormat(Types.FileFormat ft)voidsetFileName(String fn)voidsetInputBlocksize(long blen)voidsetOnlyRDD(boolean flag)voidsetOutputParams(long dim1, long dim2, long nnz, MatrixObject.UpdateType update, int blen)-
Methods inherited from class org.apache.sysds.hops.Hop
activatePrefetch, addAllInputs, addInput, checkAndSetForcedPlatform, checkAndSetInvalidCPDimsAndSize, clearMemEstimate, colsKnown, compressedSize, computeBoundsInformation, computeBoundsInformation, computeBoundsInformation, computeMemEstimate, computeSizeInformation, computeSizeInformation, computeSizeInformation, constructAndSetLopsDataFlowProperties, createOffsetLop, deactivatePrefetch, dimsKnown, dimsKnown, dimsKnownAny, federatedCostInitialized, getBeginColumn, getBeginLine, getBlocksize, getCompressedSize, getDataCharacteristics, getDataType, getDim, getDim1, getDim2, getEndColumn, getEndLine, getExecType, getFederatedCost, getFederatedOutput, getFilename, getForcedExecType, getHopID, getInput, getInput, getInputMemEstimate, getInputMemEstimate, getInputOutputSize, getIntermediateMemEstimate, getLength, getLops, getMemEstimate, getName, getNnz, getOutputMemEstimate, getOutputMemEstimate, getParent, getPrivacy, getRepetitions, getSparsity, getSpBroadcastSize, getText, getUpdateType, getValueType, hasCompressedInput, hasFederatedOutput, hasLocalOutput, hasMatrixInputWithDifferentBlocksizes, hasValidCPDimsAndSize, isCompressedOutput, isFederated, isMatrix, isMemEstimated, isOutputEmptyBlocks, isRequiredDecompression, isScalar, isTransposeSafe, isVisited, prefetchActivated, printErrorLocation, refreshColsParameterInformation, refreshColsParameterInformation, refreshMemEstimates, refreshRowsParameterInformation, refreshRowsParameterInformation, requiresCheckpoint, requiresCompression, requiresLineageCaching, requiresReblock, requiresRecompile, resetExecType, resetFederatedCost, resetRecompilationFlag, resetRecompilationFlag, resetVisitStatus, resetVisitStatus, resetVisitStatus, resetVisitStatusForced, rowsKnown, setBeginColumn, setBeginLine, setBlocksize, setCompressedOutput, setCompressedSize, setDataType, setDim, setDim1, setDim2, setEndColumn, setEndLine, setExecType, setFederatedCost, setFederatedOutput, setFilename, setForcedExecType, setLops, setMemEstimate, setName, setNnz, setOutputEmptyBlocks, setParseInfo, setPrivacy, setRequiresCheckpoint, setRequiresCompression, setRequiresCompression, setRequiresDeCompression, setRequiresLineageCaching, setRequiresReblock, setRequiresRecompile, setText, setUpdateType, setValueType, setVisited, setVisited, someInputFederated, toString, updateLopFedOut, updateLopFedOut, updateRepetitionEstimates
-
-
-
-
Constructor Detail
-
DataOp
public DataOp(String l, Types.DataType dt, Types.ValueType vt, Types.OpOpData dop, String fname, long dim1, long dim2, long nnz, int blen)
READ operation for Matrix w/ dim1, dim2. This constructor does not support any expression in parameters- Parameters:
l- ?dt- data typevt- value typedop- data operator typefname- file namedim1- dimension 1dim2- dimension 2nnz- number of non-zerosblen- rows/cols per block
-
DataOp
public DataOp(String l, Types.DataType dt, Types.ValueType vt, Types.OpOpData dop, String fname, long dim1, long dim2, long nnz, MatrixObject.UpdateType update, int blen)
-
DataOp
public DataOp(String l, Types.DataType dt, Types.ValueType vt, Types.OpOpData dop, HashMap<String,Hop> params)
READ operation for Matrix / SQL operation for tensor This constructor supports expressions in parameters- Parameters:
l- ?dt- data typevt- value typedop- data operator typeparams- input parameters
-
DataOp
public DataOp(String l, Types.DataType dt, Types.ValueType vt, Hop in, Types.OpOpData dop, String fname)
-
DataOp
public DataOp(String l, Types.DataType dt, Types.ValueType vt, Types.OpOpData dop, Hop in, HashMap<String,Hop> inputParameters)
WRITE operation for Matrix This constructor supports expression in parameters- Parameters:
l- ?dt- data typevt- value typedop- data operator typein- high-level operatorinputParameters- input parameters
-
-
Method Detail
-
checkArity
public void checkArity()
Check for N (READ) or N+1 (WRITE) inputs.- Specified by:
checkArityin classHop
-
getOp
public Types.OpOpData getOp()
-
setDataOpType
public void setDataOpType(Types.OpOpData type)
-
setOutputParams
public void setOutputParams(long dim1, long dim2, long nnz, MatrixObject.UpdateType update, int blen)
-
setFileName
public void setFileName(String fn)
-
getFileName
public String getFileName()
-
getParameterIndex
public int getParameterIndex(String name)
-
setOnlyRDD
public void setOnlyRDD(boolean flag)
-
hasOnlyRDD
public boolean hasOnlyRDD()
-
isGPUEnabled
public boolean isGPUEnabled()
Description copied from class:HopIn memory-based optimizer mode (see OptimizerUtils.isMemoryBasedOptLevel()), the exectype is determined by checking this method as well as memory budget of this Hop. Please see findExecTypeByMemEstimate for more detail. This method is necessary because not all operator are supported efficiently on GPU (for example: operations on frames and scalar as well as operations such as table).- Specified by:
isGPUEnabledin classHop- Returns:
- true if the Hop is eligible for GPU Exectype.
-
constructLops
public Lop constructLops()
- Specified by:
constructLopsin classHop
-
setFileFormat
public void setFileFormat(Types.FileFormat ft)
-
getFileFormat
public Types.FileFormat getFileFormat()
-
setInputBlocksize
public void setInputBlocksize(long blen)
-
getInputBlocksize
public long getInputBlocksize()
-
isRead
public boolean isRead()
-
isWrite
public boolean isWrite()
-
isPersistentReadWrite
public boolean isPersistentReadWrite()
-
isFederatedDataOp
public boolean isFederatedDataOp()
Description copied from class:HopChecks if the hop is a DataOp with federated data.- Overrides:
isFederatedDataOpin classHop- Returns:
- true if hop is a federated DataOp
-
getOpString
public String getOpString()
- Specified by:
getOpStringin classHop
-
allowsAllExecTypes
public boolean allowsAllExecTypes()
- Specified by:
allowsAllExecTypesin classHop
-
refreshSizeInformation
public void refreshSizeInformation()
Description copied from class:HopUpdate the output size information for this hop.- Specified by:
refreshSizeInformationin classHop
-
disableRecompileRead
public void disableRecompileRead()
Explicitly disables recompilation of transient reads, this additional information is required because requiresRecompile is set in a top-down manner, hence any value set from a consuming operating would be overwritten by opFindExecType.
-
clone
public Object clone() throws CloneNotSupportedException
- Specified by:
clonein classHop- Throws:
CloneNotSupportedException
-
removeInput
public void removeInput(String inputName)
Remove an input from the list of inputs and from the parameter index map. Parameter index map values higher than the index of the removed input will be decremented by one.- Parameters:
inputName- The name of the input to remove
-
-