Uses of Class
org.apache.sysds.runtime.controlprogram.context.SparkExecutionContext
-
-
Uses of SparkExecutionContext in org.apache.sysds.api.mlcontext
Methods in org.apache.sysds.api.mlcontext with parameters of type SparkExecutionContext Modifier and Type Method Description static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock>MLContextConversionUtil. frameObjectToBinaryBlocks(FrameObject frameObject, SparkExecutionContext sparkExecutionContext)Convert aFrameObjectto aJavaPairRDD<Long, FrameBlock>.static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>MLContextConversionUtil. frameObjectToDataFrame(FrameObject frameObject, SparkExecutionContext sparkExecutionContext)Convert aFrameObjectto aDataFrame.static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock>MLContextConversionUtil. matrixObjectToBinaryBlocks(MatrixObject matrixObject, SparkExecutionContext sparkExecutionContext)Convert aMatrixObjectto aJavaPairRDD<MatrixIndexes, MatrixBlock>.static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>MLContextConversionUtil. matrixObjectToDataFrame(MatrixObject matrixObject, SparkExecutionContext sparkExecutionContext, boolean isVectorDF)Convert aMatrixObjectto aDataFrame.Constructors in org.apache.sysds.api.mlcontext with parameters of type SparkExecutionContext Constructor Description Frame(FrameObject frameObject, SparkExecutionContext sparkExecutionContext)Matrix(MatrixObject matrixObject, SparkExecutionContext sparkExecutionContext) -
Uses of SparkExecutionContext in org.apache.sysds.runtime.controlprogram.paramserv
Methods in org.apache.sysds.runtime.controlprogram.paramserv with parameters of type SparkExecutionContext Modifier and Type Method Description static org.apache.spark.api.java.JavaPairRDD<Integer,scala.Tuple2<MatrixBlock,MatrixBlock>>SparkParamservUtils. doPartitionOnSpark(SparkExecutionContext sec, MatrixObject features, MatrixObject labels, Statement.PSScheme scheme, int workerNum) -
Uses of SparkExecutionContext in org.apache.sysds.runtime.controlprogram.paramserv.dp
Constructors in org.apache.sysds.runtime.controlprogram.paramserv.dp with parameters of type SparkExecutionContext Constructor Description DataPartitionerSparkMapper(Statement.PSScheme scheme, int workersNum, SparkExecutionContext sec, int numEntries) -
Uses of SparkExecutionContext in org.apache.sysds.runtime.instructions.spark
Methods in org.apache.sysds.runtime.instructions.spark with parameters of type SparkExecutionContext Modifier and Type Method Description voidUnaryFrameSPInstruction. detectSchema(SparkExecutionContext sec) -
Uses of SparkExecutionContext in org.apache.sysds.runtime.instructions.spark.utils
Methods in org.apache.sysds.runtime.instructions.spark.utils with parameters of type SparkExecutionContext Modifier and Type Method Description static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock>RDDSortUtils. sortDataByValMemSort(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> val, org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> data, boolean asc, long rlen, long clen, int blen, SparkExecutionContext sec, ReorgOperator r_op)This function collects and sorts value column in memory and then broadcasts it.
-