Lexical Analysis Mistakes for GATE: 7 Mark-Losing Errors and Exact Fixes
These seven repairs show exactly where lexical analysis answers go wrong. Trace one shared source, separate token counts from lexemes and categories, and keep each compiler phase in its lane.
KnowledgeGate Team
Exam prep & CS education

Lexical-analysis questions look simple, but marks disappear when you count categories instead of occurrences, stop before the longest match, absorb a sign without checking the rule, or ask the lexer to judge syntax. The teaching trace is if (count<=10) total=count--+3.5;. Its correct result is 13 token occurrences, 12 distinct lexemes and 11 distinct token categories. For each error, identify why it happens, what goes wrong, and the repair.
All traces use a simplified C-like teaching specification. Keywords outrank identifiers on equal-length ties; <= and -- are operators; identifiers match [A-Za-z_][A-Za-z0-9_]*; integers match [0-9]+; floats match [0-9]+\.[0-9]+([eE][+-]?[0-9]+)?; whitespace and /* ... */ comments are skipped; and - is separate. Other languages may differ.
1. Mistake: Treating Token, Lexeme, Pattern and Token Count as the Same Thing
Why it happens: The terms are learnt together, so a student counts unique labels such as ID, or unique spellings such as count, when the question asks for occurrences.
What goes wrong: The source emits IF("if") | LPAREN("(") | ID("count") | LE("<=") | INT("10") | RPAREN(")") | ID("total") | ASSIGN("=") | ID("count") | DEC("--") | PLUS("+") | FLOAT("3.5") | SEMI(";"). That gives 13 occurrences, 12 lexemes because count repeats, and 11 categories because ID occurs three times. The pattern recognises text, not an extra token.
What to do instead: Draw source spelling, category and occurrence columns. Answer the noun requested. The earlier Lexical Analysis and Tokens in Compiler Design: Worked Examples for GATE CS covers token definitions, attributes and a full worked scan; the seven fixes target exam mistakes with a separate trace.
2. Mistake: Stopping at the First Valid Prefix

Why it happens: A left-to-right scan is confused with accepting the first prefix that becomes valid. Keyword priority is also wrongly mixed into longest match.
What goes wrong: Splitting < and = creates two tokens instead of LE("<="). Splitting the hyphens creates two MINUS tokens instead of DEC("--"). The two extra splits raise the total from 13 to 15. At if, both the keyword and identifier rules match two characters, so length cannot settle the tie. The stated priority makes it IF.
What to do instead: At each position, list valid candidates, consume the longest lexeme, then apply the tie rule. Choose <= at indices 9 to 10, -- at 26 to 27, and keyword priority for if at 0 to 1.

3. Mistake: Assuming a Leading Sign Belongs to the Number
Why it happens: In arithmetic, -1250 looks like one number, so the sign gets absorbed before the token patterns are checked.
What goes wrong: Here, x=-12.5e+2; emits ID("x") | ASSIGN("=") | MINUS("-") | FLOAT("12.5e+2") | SEMI(";"), exactly 5 tokens. The float represents 12.5 × 10^2 = 1250; unary minus later gives -1250. Similarly, a-5 becomes ID("a") | MINUS("-") | INT("5"), not a signed integer token.
What to do instead: Check whether the numeric pattern includes a sign. If not, emit MINUS separately and let the grammar decide whether it is unary or binary. Another question may define signed literals differently.
4. Mistake: Counting Whitespace and Comment Contents
Why it happens: Students count every recognisable word or assume skipped text is never examined.
What goes wrong: For sum = a /* b + 7 */ + 2;, the stream is ID("sum") | ASSIGN("=") | ID("a") | PLUS("+") | INT("2") | SEMI(";"), exactly 6 tokens. The b, +, 7 and spaces emit nothing. However, sum = a /* b + 7; reaches the end before */, so it is a lexical error under these rules.
What to do instead: Mark skipped spans first, but verify that each comment closes. Count only emitted tokens unless the question explicitly makes whitespace, newlines or comments significant.
5. Mistake: Sending Syntax and Semantic Errors to the Lexer
Why it happens: Since lexing comes first, every bad-looking program gets blamed on the lexer. Successful tokenisation is also mistaken for proof that the program is valid.
What goes wrong: In a = b @ 2;, @ matches no rule, so scanning fails lexically. In a = b + ;, every character forms a token, but the parser finds a missing operand, a syntax error. In a = missing + 2;, the token sequence and grammar are valid, but an undeclared missing is a semantic issue. Even 123abc can scan as INT("123") | ID("abc"); its adjacency may fail later.
What to do instead: Find the earliest broken contract: character-to-token recognition, grammatical structure, or declared and type-consistent meaning.
6. Mistake: Writing an Identifier Regex That Accepts Empty or Digit-First Strings
Why it happens: A * is placed on both character classes because an identifier can contain many characters. But * also permits zero characters.
What goes wrong: [A-Za-z_]*[A-Za-z0-9_]* accepts the empty string and can accept 2rate by letting the first part match nothing. [A-Za-z_][A-Za-z0-9_]* accepts _x7 and rate2, while rejecting both the empty string and 2rate as one identifier. In rate-2, the identifier ends after rate; the remainder emits MINUS("-") | INT("2").
What to do instead: Test every proposed regex against two valid examples, a digit-first case, an illegal-character case and the empty string. Ensure the first character is mandatory, with no * or ? that can erase it.
7. Mistake: Asking a Finite-State Lexer to Validate Nesting or Declarations
Why it happens: Regexes and finite automata describe tokens, so one larger regex can seem able to check a program.
What goes wrong: f((a+b), c) emits 10 tokens: ID | LPAREN | LPAREN | ID | PLUS | ID | RPAREN | COMMA | ID | RPAREN. The incomplete f((a+b), c still emits 9 tokens. The missing RPAREN requires parser-tracked nesting. A finite-state lexer cannot track arbitrary depth; declaration and type checks belong to semantic analysis.
What to do instead: Give spelling to the lexer, recursive structure to the parser, and declarations or types to semantics. Fixed depth is finite; arbitrary nesting is not.
8. The 45-Second Lexical-Analysis Error Audit
Before you submit an answer, distinguish occurrences, lexemes and categories. Apply longest match before priority. Check whether the sign belongs to the number. Remove only closed skipped spans. Find the earliest broken compiler contract. Test regexes against empty and digit-first strings. Leave nesting and declarations to later phases.
Reverse-check the source: 13 occurrences, 12 distinct lexemes and 11 categories.
Study Compiler Design, practise with GATE Test Series, or browse CS Fundamentals. Apply the audit to 12 solved lexical-analysis MCQs.
Keep learning
Intermediate Code Generation Mistakes: 7 Mark-Losing Errors and Exact Fixes
Seven tempting shortcuts can corrupt an otherwise neat intermediate-code answer. Trace each wrong result, then apply the exact repair for TAC, arrays, records, jumps, loops and expression reuse.
Run-Time Environment Mistakes: 7 Mark-Losing Errors and Exact Fixes
Definitions are not enough when a trace depends on storage duration, calling order, lexical nesting and frame-address rules. Repair seven common errors through small, checkable examples.
Control Flow in C: 7 Mark-Losing Mistakes and Precise Fixes
Trace seven common C control-flow mistakes step by step, see exactly where execution changes course, and learn a short routine for checking every branch and loop.
Semantic Analysis and SDT Mistakes That Cost Marks: Exact Fixes
Knowing a grammar is not enough if attributes flow backwards or actions fire at the wrong point. Correct seven common errors with worked semantic and TAC traces.





