Defined in: packages/core/src/parsers/SqlTokenizer.ts:42
Class responsible for tokenizing SQL input.
Constructors ​
Constructor ​
new SqlTokenizer(
input):SqlTokenizer
Defined in: packages/core/src/parsers/SqlTokenizer.ts:66
Initializes a new instance of the SqlTokenizer.
Parameters ​
input ​
string
Returns ​
SqlTokenizer
Methods ​
tokenize() ​
Call Signature ​
tokenize():
Lexeme[]
Defined in: packages/core/src/parsers/SqlTokenizer.ts:134
Tokenizes the input SQL with optional formatting preservation.
Returns ​
Lexeme[]
Call Signature ​
tokenize(
options):FormattingLexeme[]
Defined in: packages/core/src/parsers/SqlTokenizer.ts:135
Tokenizes the input SQL with optional formatting preservation.
Parameters ​
options ​
preserveFormatting ​
true
Returns ​
Call Signature ​
tokenize(
options?):FormattingLexeme[] |Lexeme[]
Defined in: packages/core/src/parsers/SqlTokenizer.ts:136
Tokenizes the input SQL with optional formatting preservation.
Parameters ​
options? ​
Returns ​
FormattingLexeme[] | Lexeme[]
readLexmes() ​
readLexmes():
Lexeme[]
Defined in: packages/core/src/parsers/SqlTokenizer.ts:151
Returns ​
Lexeme[]
Deprecated ​
Use readLexemes (correct spelling) instead. This legacy alias remains for backwards compatibility and delegates to the new method.
readLexemes() ​
readLexemes():
Lexeme[]
Defined in: packages/core/src/parsers/SqlTokenizer.ts:161
Reads the lexemes from the input string.
Returns ​
Lexeme[]
An array of lexemes extracted from the input string.
Throws ​
Error if an unexpected character is encountered.
readNextStatement() ​
readNextStatement(
startPosition,carryComments):StatementLexemeResult|null
Defined in: packages/core/src/parsers/SqlTokenizer.ts:174
Parameters ​
startPosition ​
number = 0
carryComments ​
string[] | null
Returns ​
StatementLexemeResult | null