opennlp.tools.tokenize
Class DictionaryDetokenizer

java.lang.Object
  opennlp.tools.tokenize.DictionaryDetokenizer

All Implemented Interfaces:

Detokenizer

public class DictionaryDetokenizer
extends Object
implements Detokenizer

A rule based detokenizer. Simple rules which indicate in which direction a token should be moved are looked up in a DetokenizationDictionary object.

See Also:

Detokenizer, DetokenizationDictionary

Nested Class Summary

Nested classes/interfaces inherited from interface opennlp.tools.tokenize.Detokenizer

Detokenizer.DetokenizationOperation

Constructor Summary

DictionaryDetokenizer(DetokenizationDictionary dict)

Method Summary

Detokenizer.DetokenizationOperation[]

detokenize(String[] tokens) Detokenize the input tokens.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DictionaryDetokenizer

public DictionaryDetokenizer(DetokenizationDictionary dict)

Method Detail

detokenize

public Detokenizer.DetokenizationOperation[] detokenize(String[] tokens)

Description copied from interface: Detokenizer

Detokenize the input tokens.

Specified by:

detokenize in interface Detokenizer

Parameters:

tokens - the tokens to detokenize.

Returns:

the merge operations to detokenize the input tokens.