Class ParForProgramBlock
- java.lang.Object
-
- org.apache.sysds.runtime.controlprogram.ProgramBlock
-
- org.apache.sysds.runtime.controlprogram.ForProgramBlock
-
- org.apache.sysds.runtime.controlprogram.ParForProgramBlock
-
- All Implemented Interfaces:
ParseInfo
public class ParForProgramBlock extends ForProgramBlock
The ParForProgramBlock has the same execution semantics as a ForProgamBlock but executes the independent iterations in parallel. See ParForStatementBlock for the loop dependency analysis. At runtime level, iterations are guaranteed to be completely independent. NEW FUNCTIONALITIES TODO: reduction variables (operations: +=, -=, /=, *=, min, max) TODO: papply(A,1:2,FUN) language construct (compiled to ParFOR) via DML function repository => modules OK, but second-order functions required
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static classParForProgramBlock.PartitionFormatConvenience class to package PDataPartitionFormat and its parameters.static classParForProgramBlock.PDataPartitionerstatic classParForProgramBlock.PDataPartitionFormatstatic classParForProgramBlock.PExecModestatic classParForProgramBlock.POptModestatic classParForProgramBlock.PResultMergestatic classParForProgramBlock.PTaskPartitioner
-
Field Summary
Fields Modifier and Type Field Description static booleanALLOW_BROADCAST_INPUTSstatic booleanALLOW_NESTED_PARALLELISMstatic booleanALLOW_REUSE_PARTITION_VARSstatic booleanCONVERT_NESTED_REMOTE_PARFORstatic booleanCOPY_EVAL_FUNCTIONSstatic booleanCREATE_UNSCOPED_RESULTVARSstatic booleanFORCE_CP_ON_REMOTE_SPARKstatic booleanLIVEVAR_AWARE_EXPORTstatic intMAX_RETRYS_ON_ERRORstatic booleanOPTIMIZEstatic StringPARFOR_COUNTER_GROUP_NAMEstatic StringPARFOR_DATAPARTITIONS_FNAMEstatic StringPARFOR_FNAME_PREFIXstatic StringPARFOR_MR_RESULT_TMP_FNAMEstatic StringPARFOR_MR_RESULTMERGE_FNAMEstatic StringPARFOR_MR_TASKS_TMP_FNAMEstatic booleanRESET_RECOMPILATION_FLAGsstatic booleanUSE_PARALLEL_RESULT_MERGEstatic booleanUSE_PARALLEL_RESULT_MERGE_REMOTEstatic booleanUSE_PB_CACHEstatic booleanUSE_RANGE_TASKS_IF_USEFULstatic booleanUSE_STREAMING_TASK_CREATIONstatic intWRITE_REPLICATION_FACTOR-
Fields inherited from class org.apache.sysds.runtime.controlprogram.ProgramBlock
_beginColumn, _beginLine, _endColumn, _endLine, _filename, _text, PRED_VAR
-
-
Constructor Summary
Constructors Constructor Description ParForProgramBlock(int ID, Program prog, String iterPredVar, HashMap<String,String> params, ArrayList<ParForStatementBlock.ResultVar> resultVars)ParForProgramBlock constructor.ParForProgramBlock(Program prog, String iterPredVar, HashMap<String,String> params, ArrayList<ParForStatementBlock.ResultVar> resultVars)
-
Method Summary
-
Methods inherited from class org.apache.sysds.runtime.controlprogram.ForProgramBlock
addProgramBlock, getChildBlocks, getFromInstructions, getIncrementInstructions, getIterVar, getToInstructions, isNested, setChildBlocks, setFromInstructions, setIncrementInstructions, setToInstructions
-
Methods inherited from class org.apache.sysds.runtime.controlprogram.ProgramBlock
executePredicate, getBeginColumn, getBeginLine, getEndColumn, getEndLine, getExitInstruction, getFilename, getProgram, getStatementBlock, getText, getThreadID, hasThreadID, isThreadID, setBeginColumn, setBeginLine, setEndColumn, setEndLine, setExitInstruction, setFilename, setParseInfo, setProgram, setStatementBlock, setText, setThreadID
-
-
-
-
Field Detail
-
OPTIMIZE
public static final boolean OPTIMIZE
- See Also:
- Constant Field Values
-
USE_PB_CACHE
public static final boolean USE_PB_CACHE
- See Also:
- Constant Field Values
-
USE_RANGE_TASKS_IF_USEFUL
public static final boolean USE_RANGE_TASKS_IF_USEFUL
- See Also:
- Constant Field Values
-
USE_STREAMING_TASK_CREATION
public static final boolean USE_STREAMING_TASK_CREATION
- See Also:
- Constant Field Values
-
ALLOW_NESTED_PARALLELISM
public static final boolean ALLOW_NESTED_PARALLELISM
- See Also:
- Constant Field Values
-
CONVERT_NESTED_REMOTE_PARFOR
public static final boolean CONVERT_NESTED_REMOTE_PARFOR
- See Also:
- Constant Field Values
-
USE_PARALLEL_RESULT_MERGE
public static final boolean USE_PARALLEL_RESULT_MERGE
- See Also:
- Constant Field Values
-
USE_PARALLEL_RESULT_MERGE_REMOTE
public static final boolean USE_PARALLEL_RESULT_MERGE_REMOTE
- See Also:
- Constant Field Values
-
CREATE_UNSCOPED_RESULTVARS
public static final boolean CREATE_UNSCOPED_RESULTVARS
- See Also:
- Constant Field Values
-
ALLOW_REUSE_PARTITION_VARS
public static boolean ALLOW_REUSE_PARTITION_VARS
-
WRITE_REPLICATION_FACTOR
public static final int WRITE_REPLICATION_FACTOR
- See Also:
- Constant Field Values
-
MAX_RETRYS_ON_ERROR
public static int MAX_RETRYS_ON_ERROR
-
FORCE_CP_ON_REMOTE_SPARK
public static final boolean FORCE_CP_ON_REMOTE_SPARK
- See Also:
- Constant Field Values
-
LIVEVAR_AWARE_EXPORT
public static final boolean LIVEVAR_AWARE_EXPORT
- See Also:
- Constant Field Values
-
RESET_RECOMPILATION_FLAGs
public static final boolean RESET_RECOMPILATION_FLAGs
- See Also:
- Constant Field Values
-
ALLOW_BROADCAST_INPUTS
public static boolean ALLOW_BROADCAST_INPUTS
-
COPY_EVAL_FUNCTIONS
public static final boolean COPY_EVAL_FUNCTIONS
- See Also:
- Constant Field Values
-
PARFOR_FNAME_PREFIX
public static final String PARFOR_FNAME_PREFIX
- See Also:
- Constant Field Values
-
PARFOR_MR_TASKS_TMP_FNAME
public static final String PARFOR_MR_TASKS_TMP_FNAME
- See Also:
- Constant Field Values
-
PARFOR_MR_RESULT_TMP_FNAME
public static final String PARFOR_MR_RESULT_TMP_FNAME
- See Also:
- Constant Field Values
-
PARFOR_MR_RESULTMERGE_FNAME
public static final String PARFOR_MR_RESULTMERGE_FNAME
- See Also:
- Constant Field Values
-
PARFOR_DATAPARTITIONS_FNAME
public static final String PARFOR_DATAPARTITIONS_FNAME
- See Also:
- Constant Field Values
-
PARFOR_COUNTER_GROUP_NAME
public static final String PARFOR_COUNTER_GROUP_NAME
- See Also:
- Constant Field Values
-
-
Constructor Detail
-
ParForProgramBlock
public ParForProgramBlock(Program prog, String iterPredVar, HashMap<String,String> params, ArrayList<ParForStatementBlock.ResultVar> resultVars)
-
ParForProgramBlock
public ParForProgramBlock(int ID, Program prog, String iterPredVar, HashMap<String,String> params, ArrayList<ParForStatementBlock.ResultVar> resultVars)ParForProgramBlock constructor. It reads the specified parameter settings, where defaults for non-specified parameters have been set in ParForStatementBlock.validate(). Furthermore, it generates the IDs for the ParWorkers.- Parameters:
ID- parfor program block idprog- runtime programiterPredVar- ?params- map of parametersresultVars- list of result variable names
-
-
Method Detail
-
resetWorkerIDs
public static void resetWorkerIDs()
-
getID
public long getID()
-
getExecMode
public ParForProgramBlock.PExecMode getExecMode()
-
getResultVariables
public ArrayList<ParForStatementBlock.ResultVar> getResultVariables()
-
disableOptimization
public void disableOptimization()
-
getOptimizationMode
public ParForProgramBlock.POptMode getOptimizationMode()
-
setOptimizationMode
public void setOptimizationMode(ParForProgramBlock.POptMode mode)
-
getDegreeOfParallelism
public int getDegreeOfParallelism()
-
setDegreeOfParallelism
public void setDegreeOfParallelism(int k)
-
isDegreeOfParallelismFixed
public boolean isDegreeOfParallelismFixed()
-
setDegreeOfParallelismFixed
public void setDegreeOfParallelismFixed(boolean flag)
-
setCPCaching
public void setCPCaching(boolean flag)
-
setRuntimePiggybacking
public void setRuntimePiggybacking(boolean flag)
-
setExecMode
public void setExecMode(ParForProgramBlock.PExecMode mode)
-
setTaskPartitioner
public void setTaskPartitioner(ParForProgramBlock.PTaskPartitioner partitioner)
-
setTaskSize
public void setTaskSize(long tasksize)
-
setDataPartitioner
public void setDataPartitioner(ParForProgramBlock.PDataPartitioner partitioner)
-
enableColocatedPartitionedMatrix
public void enableColocatedPartitionedMatrix(String varname)
-
setTransposeSparseColumnVector
public void setTransposeSparseColumnVector(boolean flag)
-
setPartitionReplicationFactor
public void setPartitionReplicationFactor(int rep)
-
setExportReplicationFactor
public void setExportReplicationFactor(int rep)
-
disableJVMReuse
public void disableJVMReuse()
-
disableMonitorReport
public void disableMonitorReport()
-
setResultMerge
public void setResultMerge(ParForProgramBlock.PResultMerge merge)
-
setRecompileMemoryBudget
public void setRecompileMemoryBudget(double localMem)
-
setSparkRepartitionVariables
public void setSparkRepartitionVariables(Collection<String> vars)
-
getSparkRepartitionVariables
public Collection<String> getSparkRepartitionVariables()
-
setSparkEagerCacheVariables
public void setSparkEagerCacheVariables(Collection<String> vars)
-
getNumIterations
public long getNumIterations()
-
hasFunctions
public boolean hasFunctions()
-
execute
public void execute(ExecutionContext ec)
Description copied from class:ProgramBlockExecutes this program block (incl recompilation if required).- Overrides:
executein classForProgramBlock- Parameters:
ec- execution context
-
printBlockErrorLocation
public String printBlockErrorLocation()
- Overrides:
printBlockErrorLocationin classForProgramBlock
-
-