opennlp.tools.tokenize.lang.en
Class TokenSampleStream

java.lang.Object
  extended by opennlp.tools.tokenize.lang.en.TokenSampleStream
All Implemented Interfaces:
Iterator<TokenSample>

public class TokenSampleStream
extends Object
implements Iterator<TokenSample>

Class which produces an Iterator from a file of space delimited token. This class uses a number of English-specific hueristics to un-separate tokens which are typically found together in text.


Constructor Summary
TokenSampleStream(InputStream is)
           
 
Method Summary
 boolean hasNext()
           
static void main(String[] args)
           
 TokenSample next()
           
 void remove()
           
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

TokenSampleStream

public TokenSampleStream(InputStream is)
                  throws IOException
Throws:
IOException
Method Detail

hasNext

public boolean hasNext()
Specified by:
hasNext in interface Iterator<TokenSample>

next

public TokenSample next()
Specified by:
next in interface Iterator<TokenSample>

remove

public void remove()
Specified by:
remove in interface Iterator<TokenSample>

main

public static void main(String[] args)
                 throws IOException
Throws:
IOException


Copyright © 2010. All Rights Reserved.