tokenizer.h File Reference

Go to the source code of this file.

Namespaces
namespace	PhraseHunter
Classes
class	PhraseHunter::Input
class	PhraseHunter::StringInput
class	PhraseHunter::FileInput
class	PhraseHunter::Tokenizer
class	PhraseHunter::TextSaver
struct	PhraseHunter::TextSaver::TokenInformation
Defines
#define	TOKENIZER_H   TOKENIZER_H
#define	MAX_TOKEN_LEN   1024

---

Define Documentation

#define MAX_TOKEN_LEN   1024

Definition at line 39 of file tokenizer.h.

Referenced by PhraseHunter::TextSaver::nextToken().

#define TOKENIZER_H   TOKENIZER_H

Definition at line 23 of file tokenizer.h.

---