Tokenizer class that breaks text into tokens. More...

#include <style_tokenizer.h>

Public Member Functions

 StyleTokenizer (const std::string &text)
 Construct tokenizer for a text string. More...
 
void read (StyleToken &out_token, bool eat_whitespace, bool eat_comments=true)
 Reads the next token. More...
 
std::vector< StyleTokenread_property_value (StyleToken &token, bool &out_important_flag)
 Reads all tokens belonging to a property value. More...
 

Static Public Member Functions

static std::vector< StyleTokentokenize (const std::string &text)
 Returns all tokens except start/end whitespace and comments. More...
 

Detailed Description

Tokenizer class that breaks text into tokens.

Constructor & Destructor Documentation

clan::StyleTokenizer::StyleTokenizer ( const std::string text)

Construct tokenizer for a text string.

Member Function Documentation

void clan::StyleTokenizer::read ( StyleToken out_token,
bool  eat_whitespace,
bool  eat_comments = true 
)

Reads the next token.

std::vector<StyleToken> clan::StyleTokenizer::read_property_value ( StyleToken token,
bool &  out_important_flag 
)

Reads all tokens belonging to a property value.

static std::vector<StyleToken> clan::StyleTokenizer::tokenize ( const std::string text)
static

Returns all tokens except start/end whitespace and comments.


The documentation for this class was generated from the following file: