Token Types

Constant Meaning
TOKEN_AND 0011 bit 1 always set for AND/OR.
TOKEN_CLOSEPAREN 0101 bit 2 always set for PAREN.
TOKEN_OPENPAREN 0100 bit 2 always set for PAREN.
TOKEN_OPERATOR Detects either AND or OR.
TOKEN_OR 0010 bit 1 always set for AND/OR.
TOKEN_PAREN Detects either open or close parenthesis.
TOKEN_USER 1000 bit 3 always set for user-token.