PhraseHunter::TextSaver Class Reference

#include <tokenizer.h>

Collaboration diagram for PhraseHunter::TextSaver:

[legend]List of all members.

Public Member Functions
	TextSaver (const std::string &infile, const std::string &outfile) throw (Exceptions::FileError)
bool	hasMoreTokens ()
TokenInformation	nextToken () throw (std::out_of_range)
Private Attributes
char	buf [1024]
Tokenizer	m_tokenizer
std::ofstream	m_normalized
Classes
struct	TokenInformation

---

Detailed Description

Definition at line 160 of file tokenizer.h.

---

Constructor & Destructor Documentation

PhraseHunter::TextSaver::TextSaver	(	const std::string &	infile,
		const std::string &	outfile
	)			throw (Exceptions::FileError) [inline]

Definition at line 182 of file tokenizer.h.

References m_normalized.

---

Member Function Documentation

bool PhraseHunter::TextSaver::hasMoreTokens

(

)

[inline]

Definition at line 191 of file tokenizer.h.

References PhraseHunter::Tokenizer::hasMoreTokens(), and m_tokenizer.

TextSaver::TokenInformation PhraseHunter::TextSaver::nextToken

(

)

throw (std::out_of_range)

Definition at line 66 of file tokenizer.cpp.

References buf, m_normalized, m_tokenizer, MAX_TOKEN_LEN, PhraseHunter::Tokenizer::nextToken(), and schma::UTF8Converter().

---

Member Data Documentation

char PhraseHunter::TextSaver::buf[1024] [private]

Definition at line 162 of file tokenizer.h.

Referenced by nextToken().

Tokenizer PhraseHunter::TextSaver::m_tokenizer [private]

Definition at line 163 of file tokenizer.h.

Referenced by hasMoreTokens(), and nextToken().

std::ofstream PhraseHunter::TextSaver::m_normalized [private]

Definition at line 164 of file tokenizer.h.

Referenced by nextToken(), and TextSaver().

---

The documentation for this class was generated from the following files:

• phrasehunter/include/phrasehunter/tokenizer.h
• phrasehunter/lib/tokenizer.cpp

---