AbstractScanner< TokenT > Class Template Reference

Base class for scanners. More...

#include <AbstractScanner.hpp>

Public Member Functions
ScannedTokens< TokenT >	scan ()
	Scan the input and return a ScannedTokens data structure. More...

Detailed Description

template<typename TokenT>
class Balau::Lang::AbstractScanner< TokenT >

Base class for scanners.

All scanning is made on unicode code points from the supplied UTF-8 encoded input stream supplied to the protected constructor.

////////// Line break handling //////////

There are four possible line break markers:

• newline;
• carriage return;
• carriage return then newline;
• newline then carriage return.

The marker characters are considered to be on the same line as the previous characters of that line. When there is a greedy match of a line break marker, the current line number is incremented. Thus the code span of a complete end of line marker has a column end number of 1 (the start of the next line).

Template Parameters

TokenT

The token type enum - must contain at least the Blank, LineBreak, and EndOfFile values

Member Function Documentation

scan()

ScannedTokens<TokenT> scan ( )

inline

Scan the input and return a ScannedTokens data structure.

---

The documentation for this class was generated from the following file:

• AbstractScanner.hpp (5)