opennlp.tools.tokenize
Class TokSpanEventStream

java.lang.Object
  opennlp.model.AbstractEventStream
      opennlp.tools.util.AbstractEventStream<TokenSample>
          opennlp.tools.tokenize.TokSpanEventStream

All Implemented Interfaces:

opennlp.model.EventStream

public class TokSpanEventStream
extends AbstractEventStream<TokenSample>

This class reads the TokenSamples from the given Iterator and converts the TokenSamples into Events which can be used by the maxent library for training.

Constructor Summary

TokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics)
Initializes the current instance.

TokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics, TokenContextGenerator cg)
Initializes the current instance.

Method Summary

protected Iterator<opennlp.model.Event>

createEvents(TokenSample tokenSample) Adds training events to the event stream for each of the specified tokens.

Methods inherited from class opennlp.tools.util.AbstractEventStream

hasNext, next

Methods inherited from class opennlp.model.AbstractEventStream

remove

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TokSpanEventStream

public TokSpanEventStream(ObjectStream<TokenSample> tokenSamples,
                          boolean skipAlphaNumerics,
                          TokenContextGenerator cg)

Initializes the current instance.

Parameters:

tokenSamples -

skipAlphaNumerics -

cg -

TokSpanEventStream

public TokSpanEventStream(ObjectStream<TokenSample> tokenSamples,
                          boolean skipAlphaNumerics)

Initializes the current instance.

Parameters:

tokenSamples -

skipAlphaNumerics -

Method Detail

createEvents

protected Iterator<opennlp.model.Event> createEvents(TokenSample tokenSample)

Adds training events to the event stream for each of the specified tokens.

Specified by:

createEvents in class AbstractEventStream<TokenSample>

Parameters:

tokens - character offsets into the specified text.

text - The text of the tokens.

Returns:

an Iterator of training events or an empty Iterator.