Class DataPartitionerSparkMapper
- java.lang.Object
-
- org.apache.sysds.runtime.controlprogram.paramserv.dp.DataPartitionerSparkMapper
-
- All Implemented Interfaces:
Serializable,org.apache.spark.api.java.function.PairFlatMapFunction<scala.Tuple2<Long,scala.Tuple2<MatrixBlock,MatrixBlock>>,Integer,scala.Tuple2<Long,scala.Tuple2<MatrixBlock,MatrixBlock>>>
public class DataPartitionerSparkMapper extends Object implements org.apache.spark.api.java.function.PairFlatMapFunction<scala.Tuple2<Long,scala.Tuple2<MatrixBlock,MatrixBlock>>,Integer,scala.Tuple2<Long,scala.Tuple2<MatrixBlock,MatrixBlock>>>
- See Also:
- Serialized Form
-
-
Constructor Summary
Constructors Constructor Description DataPartitionerSparkMapper(Statement.PSScheme scheme, int workersNum, SparkExecutionContext sec, int numEntries)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description Iterator<scala.Tuple2<Integer,scala.Tuple2<Long,scala.Tuple2<MatrixBlock,MatrixBlock>>>>call(scala.Tuple2<Long,scala.Tuple2<MatrixBlock,MatrixBlock>> input)Do data partitioning
-
-
-
Constructor Detail
-
DataPartitionerSparkMapper
public DataPartitionerSparkMapper(Statement.PSScheme scheme, int workersNum, SparkExecutionContext sec, int numEntries)
-
-
Method Detail
-
call
public Iterator<scala.Tuple2<Integer,scala.Tuple2<Long,scala.Tuple2<MatrixBlock,MatrixBlock>>>> call(scala.Tuple2<Long,scala.Tuple2<MatrixBlock,MatrixBlock>> input) throws Exception
Do data partitioning- Specified by:
callin interfaceorg.apache.spark.api.java.function.PairFlatMapFunction<scala.Tuple2<Long,scala.Tuple2<MatrixBlock,MatrixBlock>>,Integer,scala.Tuple2<Long,scala.Tuple2<MatrixBlock,MatrixBlock>>>- Parameters:
input- RowBlockID => (features, labels)- Returns:
- WorkerID => (rowBlockID, (single row features, single row labels))
- Throws:
Exception- Some exception
-
-