Interface ContinuousPartitionReaderFactory
- All Superinterfaces:
PartitionReaderFactory
,Serializable
A variation on
PartitionReaderFactory
that returns ContinuousPartitionReader
instead of PartitionReader
. It's used for continuous streaming processing.- Since:
- 3.0.0
-
Method Summary
Modifier and TypeMethodDescriptiondefault ContinuousPartitionReader<ColumnarBatch>
createColumnarReader
(InputPartition partition) Returns a columnar partition reader to read data from the givenInputPartition
.ContinuousPartitionReader<org.apache.spark.sql.catalyst.InternalRow>
createReader
(InputPartition partition) Returns a row-based partition reader to read data from the givenInputPartition
.Methods inherited from interface org.apache.spark.sql.connector.read.PartitionReaderFactory
supportColumnarReads
-
Method Details
-
createReader
ContinuousPartitionReader<org.apache.spark.sql.catalyst.InternalRow> createReader(InputPartition partition) Description copied from interface:PartitionReaderFactory
Returns a row-based partition reader to read data from the givenInputPartition
.Implementations probably need to cast the input partition to the concrete
InputPartition
class defined for the data source.- Specified by:
createReader
in interfacePartitionReaderFactory
-
createColumnarReader
Description copied from interface:PartitionReaderFactory
Returns a columnar partition reader to read data from the givenInputPartition
.Implementations probably need to cast the input partition to the concrete
InputPartition
class defined for the data source.- Specified by:
createColumnarReader
in interfacePartitionReaderFactory
-