How does a lexical analyzer treat whitespace and comments?
2025
How does a lexical analyzer treat whitespace and comments?
- A.
Passes them to semantic analysis
- B.
Includes them in the parse tree
- C.
Converts them into tokens
- D.
Discards them during scanning
Attempted by 46 students.
Show answer & explanation
Correct answer: D
A lexical analyzer processes source code sequentially as a long string of individual characters. Because whitespace (spaces, tabs, newlines) and comments are put into code strictly for human readability, they have absolutely zero impact on the programmatic logic or structural syntax of a compiler.
When the lexer encounters them during scanning:
It reads them to determine where one distinct word or token ends and another begins.
It immediately discards (ignores) them instead of bundling them into a token package.
By filtering out this structural "noise" right at the very first stage, the lexer hands a completely clean, streamlined stream of pure tokens (like keywords, identifiers, and operators) to the parser, keeping the downstream compilation process highly efficient.