opennlp.tools.tokenize
Class TokenizerStream

java.lang.Object
  extended by opennlp.tools.tokenize.TokenizerStream
All Implemented Interfaces:
ObjectStream<TokenSample>

public class TokenizerStream
extends Object
implements ObjectStream<TokenSample>

The TokenizerStream uses a tokenizer to tokenize the input string and output TokenSamples.


Constructor Summary
TokenizerStream(Tokenizer tokenizer, ObjectStream<String> input)
           
 
Method Summary
 void close()
          Closes the ObjectStream and releases all allocated resources.
 TokenSample read()
          Returns the next object.
 void reset()
          Repositions the stream at the beginning and the previously seen object sequence will be repeated exactly.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

TokenizerStream

public TokenizerStream(Tokenizer tokenizer,
                       ObjectStream<String> input)
Method Detail

read

public TokenSample read()
                 throws IOException
Description copied from interface: ObjectStream
Returns the next object. Calling this method repeatedly until it returns null will return each object from the underlying source exactly once.

Specified by:
read in interface ObjectStream<TokenSample>
Returns:
the next object or null to signal that the stream is exhausted
Throws:
IOException

close

public void close()
           throws IOException
Description copied from interface: ObjectStream
Closes the ObjectStream and releases all allocated resources. After close was called its not allowed to call read or reset.

Specified by:
close in interface ObjectStream<TokenSample>
Throws:
IOException

reset

public void reset()
           throws IOException,
                  UnsupportedOperationException
Description copied from interface: ObjectStream
Repositions the stream at the beginning and the previously seen object sequence will be repeated exactly. This method can be used to re-read the stream if multiple passes over the objects are required. The implementation of this method is optional.

Specified by:
reset in interface ObjectStream<TokenSample>
Throws:
IOException
UnsupportedOperationException


Copyright © 2010. All Rights Reserved.