Package opennlp.tools.tokenize
Class TokenizerStream
java.lang.Object
opennlp.tools.tokenize.TokenizerStream
- All Implemented Interfaces:
AutoCloseable,ObjectStream<TokenSample>
-
Constructor Summary
ConstructorsConstructorDescriptionTokenizerStream(Tokenizer tokenizer, ObjectStream<String> input) Initializes ainstance. -
Method Summary
Modifier and TypeMethodDescriptionvoidclose()Closes theObjectStreamand releases all allocated resources.read()Returns the nextObjectStreamobject.voidreset()Repositions the stream at the beginning and the previously seen object sequence will be repeated exactly.
-
Constructor Details
-
TokenizerStream
Initializes ainstance.- Parameters:
tokenizer- A workingTokenizerinstance.input- A plain textline stream.
-
-
Method Details
-
read
Description copied from interface:ObjectStreamReturns the nextObjectStreamobject. Calling this method repeatedly until it returnsnullwill return each object from the underlying source exactly once.- Specified by:
readin interfaceObjectStream<TokenSample>- Returns:
- The next object or
nullto signal that the stream is exhausted. - Throws:
IOException- Thrown if there is an error during reading.
-
close
Description copied from interface:ObjectStreamCloses theObjectStreamand releases all allocated resources. After close was called, it's not allowed to callObjectStream.read()orObjectStream.reset().- Specified by:
closein interfaceAutoCloseable- Specified by:
closein interfaceObjectStream<TokenSample>- Throws:
IOException- Thrown if there is an error during closing the stream.
-
reset
Description copied from interface:ObjectStreamRepositions the stream at the beginning and the previously seen object sequence will be repeated exactly. This method can be used to re-read the stream if multiple passes over the objects are required.The implementation of this method is optional.
- Specified by:
resetin interfaceObjectStream<TokenSample>- Throws:
IOException- Thrown if there is an error during resetting the stream.UnsupportedOperationException- Thrown if thereset()is not supported. By default, this is the case.
-