A lexeme is simply the actual text written by the programmer. (krypticmouse)
Example:
int age = 25;
The lexemes are:
int
age
=
25
;
Think of lexeme as:
"The raw word or symbol found in the source code."
A token is the category of a lexeme. (DEV Community)
Example:
int age = 25;
| Lexeme | Token |
|---|---|
| int | KEYWORD |
| age | IDENTIFIER |
| = | ASSIGN |
| 25 | INTEGER_LITERAL |
| ; | SEMICOLON |
So:
Lexeme = actual text
Token = meaning/category of that text
Imagine parser gets this: