What is the role of the character set in the formation of tokens during the…
2025
What is the role of the character set in the formation of tokens during the lexical analysis phase of program compilation?
- A.
It governs how functions are linked during the compilation process.
- B.
It ensures that runtime memory is properly allocated for variables.
- C.
It specifies the logical grouping of code into syntactic units.
- D.
It defines the range of permissible symbols used to construct valid tokens.
Attempted by 37 students.
Show answer & explanation
Correct answer: D
A computer programming language is a formal language. Every formal language requires a predefined alphabet (Σ). In computer science, this alphabet is represented by a Character Set (traditionally ASCII, and in modern languages like Java or Python, UTF-8/Unicode).The character set acts as the raw boundary. If a programmer types a symbol that is not present in the language's allowed character set (for example, placing an unauthorized control character or an unescaped special symbol inside raw code), the lexical analyzer will immediately throw a Lexical Error because it cannot process the symbol.