public class Connection extends Object implements Closeable
Connection object. The JMLC API is designed after JDBC. A DML script is precompiled by calling
the prepareScript(String, String[], String[])
method or the prepareScript(String, Map, String[], String[])
method on the Connection object, which returns a
PreparedScript object. Note that this is similar to calling
a prepareStatement method on a JDBC Connection object.
Following this, input variable data is passed to the script by calling the
setFrame, setMatrix, and setScalar methods of the PreparedScript
object. The script is executed via PreparedScript's
executeScript method,
which returns a ResultVariables object, which is similar to a JDBC
ResultSet. Data can be read from a ResultVariables object by calling
its getFrame and
getMatrix methods.
| Constructor and Description |
|---|
Connection()
Connection constructor, the starting point for any other JMLC API calls.
|
Connection(CompilerConfig.ConfigType... cconfigs)
Connection constructor, the starting point for any other JMLC API calls.
|
Connection(DMLConfig dmlconfig)
Connection constructor, the starting point for any other JMLC API calls.
|
Connection(DMLConfig dmlconfig,
CompilerConfig.ConfigType... cconfigs)
Connection constructor, the starting point for any other JMLC API calls.
|
| Modifier and Type | Method and Description |
|---|---|
void |
close()
Close connection to SystemDS, which clears the
thread-local DML and compiler configurations.
|
double[][] |
convertToDoubleMatrix(InputStream input,
int rows,
int cols)
Converts an input stream of a string matrix in textcell format
into a dense double array.
|
double[][] |
convertToDoubleMatrix(InputStream input,
int rows,
int cols,
String format)
Converts an input stream of a string matrix in csv or textcell format
into a dense double array.
|
double[][] |
convertToDoubleMatrix(String input,
int rows,
int cols)
Converts an input string representation of a matrix in textcell format
into a dense double array.
|
double[][] |
convertToDoubleMatrix(String input,
String meta)
Converts an input string representation of a matrix in csv or textcell format
into a dense double array.
|
FrameBlock |
convertToFrame(InputStream input,
int rows,
int cols)
Converts an input stream of a string frame in textcell format
into a frame block.
|
FrameBlock |
convertToFrame(InputStream input,
int rows,
int cols,
String format)
Converts an input stream of a frame in csv or textcell format
into a frame block.
|
FrameBlock |
convertToFrame(InputStream input,
String meta)
Converts an input stream of a string frame in csv or textcell format
into a frame block.
|
FrameBlock |
convertToFrame(String input,
int rows,
int cols)
Converts an input string representation of a frame in textcell format
into a frame block.
|
FrameBlock |
convertToFrame(String input,
String meta)
Converts an input string representation of a frame in csv or textcell format
into a frame block.
|
MatrixBlock |
convertToMatrix(InputStream input,
int rows,
int cols)
Converts an input stream of a string matrix in text format
into a matrix block.
|
MatrixBlock |
convertToMatrix(InputStream input,
int rows,
int cols,
String format)
Converts an input stream of a string matrix in csv or text format
into a matrix block.
|
MatrixBlock |
convertToMatrix(InputStream input,
String meta)
Converts an input stream of a string matrix in csv or textcell format
into a matrix block.
|
MatrixBlock |
convertToMatrix(String input,
int rows,
int cols)
Converts an input string representation of a matrix in textcell format
into a matrix block.
|
MatrixBlock |
convertToMatrix(String input,
String meta)
Converts an input string representation of a matrix in csv or textcell format
into a matrix block.
|
String[][] |
convertToStringFrame(InputStream input,
int rows,
int cols)
Converts an input stream of a string frame in textcell format
into a dense string array.
|
String[][] |
convertToStringFrame(InputStream input,
int rows,
int cols,
String format)
Converts an input stream of a string frame in csv or text format
into a dense string array.
|
String[][] |
convertToStringFrame(String input,
int rows,
int cols)
Converts an input stream of a string frame in textcell format
into a dense string array.
|
String[][] |
convertToStringFrame(String input,
String meta)
Converts an input string representation of a frame in csv or textcell format
into a dense string array.
|
void |
gatherMemStats(boolean stats)
Sets a boolean flag indicating if memory profiling statistics should be
gathered.
|
PreparedScript |
prepareScript(String script,
Map<String,String> nsscripts,
Map<String,String> args,
String[] inputs,
String[] outputs)
Prepares (precompiles) a script, sets input parameter values, and registers input and output variables.
|
PreparedScript |
prepareScript(String script,
Map<String,String> args,
String[] inputs,
String[] outputs)
Prepares (precompiles) a script, sets input parameter values, and registers input and output variables.
|
PreparedScript |
prepareScript(String script,
String[] inputs,
String[] outputs)
Prepares (precompiles) a script and registers input and output variables.
|
double[][] |
readDoubleMatrix(String fname)
Reads an input matrix in arbitrary format from HDFS into a dense double array.
|
double[][] |
readDoubleMatrix(String fname,
Types.FileFormat fmt,
long rows,
long cols,
int blen,
long nnz)
Reads an input matrix in arbitrary format from HDFS into a dense double array.
|
String |
readScript(String fname)
Read a DML or PyDML file as a string.
|
String[][] |
readStringFrame(String fname)
Reads an input frame in arbitrary format from HDFS into a dense string array.
|
String[][] |
readStringFrame(String fname,
Types.FileFormat fmt,
long rows,
long cols)
Reads an input frame in arbitrary format from HDFS into a dense string array.
|
FrameBlock |
readTransformMetaDataFromFile(String metapath)
Reads transform meta data from an HDFS file path and converts it into an in-memory
FrameBlock object.
|
FrameBlock |
readTransformMetaDataFromFile(String spec,
String metapath)
Reads transform meta data from an HDFS file path and converts it into an in-memory
FrameBlock object.
|
FrameBlock |
readTransformMetaDataFromFile(String spec,
String metapath,
String colDelim)
Reads transform meta data from an HDFS file path and converts it into an in-memory
FrameBlock object.
|
FrameBlock |
readTransformMetaDataFromPath(String metapath)
Reads transform meta data from the class path and converts it into an in-memory
FrameBlock object.
|
FrameBlock |
readTransformMetaDataFromPath(String spec,
String metapath)
Reads transform meta data from the class path and converts it into an in-memory
FrameBlock object.
|
FrameBlock |
readTransformMetaDataFromPath(String spec,
String metapath,
String colDelim)
Reads transform meta data from the class path and converts it into an in-memory
FrameBlock object.
|
void |
setLineage(boolean lt)
Sets a boolean flag indicating if lineage trace should be captured
|
void |
setStatistics(boolean stats)
Sets a boolean flag indicating if runtime statistics should be gathered
Same behavior as in "MLContext.setStatistics()"
|
public Connection()
public Connection(CompilerConfig.ConfigType... cconfigs)
cconfigs - one or many boolean compiler configurations to enable.public Connection(DMLConfig dmlconfig, CompilerConfig.ConfigType... cconfigs)
dmlconfig - a dml configuration.cconfigs - one or many boolean compiler configurations to enable.public Connection(DMLConfig dmlconfig)
dmlconfig - a dml configuration.public void setStatistics(boolean stats)
stats - boolean value with true indicating statistics should be gatheredpublic void setLineage(boolean lt)
lt - boolean value with true indicating lineage should be capturedpublic void gatherMemStats(boolean stats)
stats - boolean value with true indicating memory statistics should be gatheredpublic PreparedScript prepareScript(String script, String[] inputs, String[] outputs)
script - string representing the DML or PyDML scriptinputs - string array of input variables to registeroutputs - string array of output variables to registerpublic PreparedScript prepareScript(String script, Map<String,String> args, String[] inputs, String[] outputs)
script - string representing the DML or PyDML scriptargs - map of input parameters ($) and their valuesinputs - string array of input variables to registeroutputs - string array of output variables to registerpublic PreparedScript prepareScript(String script, Map<String,String> nsscripts, Map<String,String> args, String[] inputs, String[] outputs)
script - string representing of the DML or PyDML scriptnsscripts - map (name, script) of the DML or PyDML namespace scriptsargs - map of input parameters ($) and their valuesinputs - string array of input variables to registeroutputs - string array of output variables to registerpublic void close()
close in interface Closeableclose in interface AutoCloseablepublic String readScript(String fname) throws IOException
fname - the filename of the scriptIOException - if IOException occurspublic double[][] readDoubleMatrix(String fname) throws IOException
fname - the filename of the input matrixIOException - if IOException occurspublic double[][] readDoubleMatrix(String fname, Types.FileFormat fmt, long rows, long cols, int blen, long nnz) throws IOException
fname - the filename of the input matrixfmt - file formatrows - number of rows in the matrixcols - number of columns in the matrixblen - block lengthnnz - number of non-zero values, -1 indicates unknownIOException - if IOException occurspublic double[][] convertToDoubleMatrix(String input, String meta) throws IOException
input - string matrix in csv or textcell formatmeta - string representing SystemDS matrix metadata in JSON formatIOException - if IOException occurspublic double[][] convertToDoubleMatrix(String input, int rows, int cols) throws IOException
input - string matrix in textcell formatrows - number of rows in the matrixcols - number of columns in the matrixIOException - if IOException occurspublic double[][] convertToDoubleMatrix(InputStream input, int rows, int cols) throws IOException
input - InputStream to a string matrix in textcell formatrows - number of rows in the matrixcols - number of columns in the matrixIOException - if IOException occurspublic double[][] convertToDoubleMatrix(InputStream input, int rows, int cols, String format) throws IOException
input - InputStream to a string matrix in csv or textcell formatrows - number of rows in the matrixcols - number of columns in the matrixformat - input format of the given streamIOException - if IOException occurspublic MatrixBlock convertToMatrix(String input, String meta) throws IOException
input - string matrix in csv or textcell formatmeta - string representing SystemDS matrix metadata in JSON formatIOException - if IOException occurspublic MatrixBlock convertToMatrix(InputStream input, String meta) throws IOException
input - InputStream to a string matrix in csv or textcell formatmeta - string representing SystemDS matrix metadata in JSON formatIOException - if IOException occurspublic MatrixBlock convertToMatrix(String input, int rows, int cols) throws IOException
input - string matrix in textcell formatrows - number of rows in the matrixcols - number of columns in the matrixIOException - if IOException occurspublic MatrixBlock convertToMatrix(InputStream input, int rows, int cols) throws IOException
input - InputStream to a string matrix in text formatrows - number of rows in the matrixcols - number of columns in the matrixIOException - if IOException occurspublic MatrixBlock convertToMatrix(InputStream input, int rows, int cols, String format) throws IOException
input - InputStream to a string matrix in csv or text formatrows - number of rows in the matrixcols - number of columns in the matrixformat - input format of the given streamIOException - if IOException occurspublic String[][] readStringFrame(String fname) throws IOException
fname - the filename of the input frameIOException - if IOException occurspublic String[][] readStringFrame(String fname, Types.FileFormat fmt, long rows, long cols) throws IOException
fname - the filename of the input framefmt - file format typerows - number of rows in the framecols - number of columns in the frameIOException - if IOException occurspublic String[][] convertToStringFrame(String input, String meta) throws IOException
input - string frame in csv or textcell formatmeta - string representing SystemDS frame metadata in JSON formatIOException - if IOException occurspublic String[][] convertToStringFrame(String input, int rows, int cols) throws IOException
input - string frame in textcell formatrows - number of rows in the framecols - number of columns in the frameIOException - if IOException occurspublic String[][] convertToStringFrame(InputStream input, int rows, int cols) throws IOException
input - InputStream to a string frame in text formatrows - number of rows in the framecols - number of columns in the frameIOException - if IOException occurspublic String[][] convertToStringFrame(InputStream input, int rows, int cols, String format) throws IOException
input - InputStream to a string frame in csv or text formatrows - number of rows in the framecols - number of columns in the frameformat - input format of the given streamIOException - if IOException occurspublic FrameBlock convertToFrame(String input, String meta) throws IOException
input - string frame in csv or textcell formatmeta - string representing SystemDS frame metadata in JSON formatIOException - if IOException occurspublic FrameBlock convertToFrame(InputStream input, String meta) throws IOException
input - InputStream to a string frame in csv or textcell formatmeta - string representing SystemDS frame metadata in JSON formatIOException - if IOException occurspublic FrameBlock convertToFrame(String input, int rows, int cols) throws IOException
input - string frame in textcell formatrows - number of rows in the framecols - number of columns in the frameIOException - if IOException occurspublic FrameBlock convertToFrame(InputStream input, int rows, int cols) throws IOException
input - InputStream to a string frame in textcell formatrows - number of rows in the framecols - number of columns in the frameIOException - if IOException occurspublic FrameBlock convertToFrame(InputStream input, int rows, int cols, String format) throws IOException
input - InputStream to a string frame in csv or textcell formatrows - number of rows in the framecols - number of columns in the frameformat - input format of the given streamIOException - if IOException occurspublic FrameBlock readTransformMetaDataFromFile(String metapath) throws IOException
metapath - hdfs file path to meta data directoryIOException - if IOException occurspublic FrameBlock readTransformMetaDataFromFile(String spec, String metapath) throws IOException
spec - transform specification as json stringmetapath - hdfs file path to meta data directoryIOException - if IOException occurspublic FrameBlock readTransformMetaDataFromFile(String spec, String metapath, String colDelim) throws IOException
spec - transform specification as json stringmetapath - hdfs file path to meta data directorycolDelim - separator for processing column names in the meta data file 'column.names'IOException - if IOException occurspublic FrameBlock readTransformMetaDataFromPath(String metapath) throws IOException
metapath - resource path to meta data directoryIOException - if IOException occurspublic FrameBlock readTransformMetaDataFromPath(String spec, String metapath) throws IOException
spec - transform specification as json stringmetapath - resource path to meta data directoryIOException - if IOException occurspublic FrameBlock readTransformMetaDataFromPath(String spec, String metapath, String colDelim) throws IOException
spec - transform specification as json stringmetapath - resource path to meta data directorycolDelim - separator for processing column names in the meta data file 'column.names'IOException - if IOException occursCopyright © 2020 The Apache Software Foundation. All rights reserved.