Uses of Interface
opennlp.tools.tokenize.Tokenizer

Packages that use Tokenizer

opennlp.tools.tokenize	Contains classes related to finding token or words in a string.
opennlp.tools.util.featuregen	This package contains classes for generating sequence features.

Uses of Tokenizer in opennlp.tools.tokenize

Classes in opennlp.tools.tokenize that implement Tokenizer

class	SimpleTokenizer Performs tokenization using character classes.
class	TokenizerME A Tokenizer for converting raw text into separated tokens.
class	WhitespaceTokenizer This tokenizer uses white spaces to tokenize the input text.

Constructors in opennlp.tools.tokenize with parameters of type Tokenizer

TokenizerEvaluator(Tokenizer tokenizer)
Initializes the current instance with the given Tokenizer.

TokenizerStream(Tokenizer tokenizer, ObjectStream<String> input)

Uses of Tokenizer in opennlp.tools.util.featuregen

Constructors in opennlp.tools.util.featuregen with parameters of type Tokenizer

TokenPatternFeatureGenerator(Tokenizer supportTokenizer)
Initializes a new instance.