Lexical Analysis Mistakes for GATE: 7 Mark-Losing Errors and Exact Fixes

These seven repairs show exactly where lexical analysis answers go wrong. Trace one shared source, separate token counts from lexemes and categories, and keep each compiler phase in its lane.

KnowledgeGate Team

Exam prep & CS education

Updated 30 Oct 20265 min read
Lexical Analysis Mistakes for GATE: 7 Mark-Losing Errors and Exact Fixes

Lexical-analysis questions look simple, but marks disappear when you count categories instead of occurrences, stop before the longest match, absorb a sign without checking the rule, or ask the lexer to judge syntax. The teaching trace is if (count<=10) total=count--+3.5;. Its correct result is 13 token occurrences, 12 distinct lexemes and 11 distinct token categories. For each error, identify why it happens, what goes wrong, and the repair.

All traces use a simplified C-like teaching specification. Keywords outrank identifiers on equal-length ties; <= and -- are operators; identifiers match [A-Za-z_][A-Za-z0-9_]*; integers match [0-9]+; floats match [0-9]+\.[0-9]+([eE][+-]?[0-9]+)?; whitespace and /* ... */ comments are skipped; and - is separate. Other languages may differ.

1. Mistake: Treating Token, Lexeme, Pattern and Token Count as the Same Thing

Why it happens: The terms are learnt together, so a student counts unique labels such as ID, or unique spellings such as count, when the question asks for occurrences.

What goes wrong: The source emits IF("if") | LPAREN("(") | ID("count") | LE("<=") | INT("10") | RPAREN(")") | ID("total") | ASSIGN("=") | ID("count") | DEC("--") | PLUS("+") | FLOAT("3.5") | SEMI(";"). That gives 13 occurrences, 12 lexemes because count repeats, and 11 categories because ID occurs three times. The pattern recognises text, not an extra token.

What to do instead: Draw source spelling, category and occurrence columns. Answer the noun requested. The earlier Lexical Analysis and Tokens in Compiler Design: Worked Examples for GATE CS covers token definitions, attributes and a full worked scan; the seven fixes target exam mistakes with a separate trace.

2. Mistake: Stopping at the First Valid Prefix

Lexer scan groups a source string into 13 tokens, highlighting longest matches for <= and --.

Why it happens: A left-to-right scan is confused with accepting the first prefix that becomes valid. Keyword priority is also wrongly mixed into longest match.

What goes wrong: Splitting < and = creates two tokens instead of LE("<="). Splitting the hyphens creates two MINUS tokens instead of DEC("--"). The two extra splits raise the total from 13 to 15. At if, both the keyword and identifier rules match two characters, so length cannot settle the tie. The stated priority makes it IF.

What to do instead: At each position, list valid candidates, consume the longest lexeme, then apply the tie rule. Choose <= at indices 9 to 10, -- at 26 to 27, and keyword priority for if at 0 to 1.

Longest-match lexical scan of a C-like line, with indexed characters, token categories, and callouts showing <= and -- remain single tokens.

3. Mistake: Assuming a Leading Sign Belongs to the Number

Why it happens: In arithmetic, -1250 looks like one number, so the sign gets absorbed before the token patterns are checked.

What goes wrong: Here, x=-12.5e+2; emits ID("x") | ASSIGN("=") | MINUS("-") | FLOAT("12.5e+2") | SEMI(";"), exactly 5 tokens. The float represents 12.5 × 10^2 = 1250; unary minus later gives -1250. Similarly, a-5 becomes ID("a") | MINUS("-") | INT("5"), not a signed integer token.

What to do instead: Check whether the numeric pattern includes a sign. If not, emit MINUS separately and let the grammar decide whether it is unary or binary. Another question may define signed literals differently.

4. Mistake: Counting Whitespace and Comment Contents

Why it happens: Students count every recognisable word or assume skipped text is never examined.

What goes wrong: For sum = a /* b + 7 */ + 2;, the stream is ID("sum") | ASSIGN("=") | ID("a") | PLUS("+") | INT("2") | SEMI(";"), exactly 6 tokens. The b, +, 7 and spaces emit nothing. However, sum = a /* b + 7; reaches the end before */, so it is a lexical error under these rules.

What to do instead: Mark skipped spans first, but verify that each comment closes. Count only emitted tokens unless the question explicitly makes whitespace, newlines or comments significant.

5. Mistake: Sending Syntax and Semantic Errors to the Lexer

Why it happens: Since lexing comes first, every bad-looking program gets blamed on the lexer. Successful tokenisation is also mistaken for proof that the program is valid.

What goes wrong: In a = b @ 2;, @ matches no rule, so scanning fails lexically. In a = b + ;, every character forms a token, but the parser finds a missing operand, a syntax error. In a = missing + 2;, the token sequence and grammar are valid, but an undeclared missing is a semantic issue. Even 123abc can scan as INT("123") | ID("abc"); its adjacency may fail later.

What to do instead: Find the earliest broken contract: character-to-token recognition, grammatical structure, or declared and type-consistent meaning.

6. Mistake: Writing an Identifier Regex That Accepts Empty or Digit-First Strings

Why it happens: A * is placed on both character classes because an identifier can contain many characters. But * also permits zero characters.

What goes wrong: [A-Za-z_]*[A-Za-z0-9_]* accepts the empty string and can accept 2rate by letting the first part match nothing. [A-Za-z_][A-Za-z0-9_]* accepts _x7 and rate2, while rejecting both the empty string and 2rate as one identifier. In rate-2, the identifier ends after rate; the remainder emits MINUS("-") | INT("2").

What to do instead: Test every proposed regex against two valid examples, a digit-first case, an illegal-character case and the empty string. Ensure the first character is mandatory, with no * or ? that can erase it.

7. Mistake: Asking a Finite-State Lexer to Validate Nesting or Declarations

Why it happens: Regexes and finite automata describe tokens, so one larger regex can seem able to check a program.

What goes wrong: f((a+b), c) emits 10 tokens: ID | LPAREN | LPAREN | ID | PLUS | ID | RPAREN | COMMA | ID | RPAREN. The incomplete f((a+b), c still emits 9 tokens. The missing RPAREN requires parser-tracked nesting. A finite-state lexer cannot track arbitrary depth; declaration and type checks belong to semantic analysis.

What to do instead: Give spelling to the lexer, recursive structure to the parser, and declarations or types to semantics. Fixed depth is finite; arbitrary nesting is not.

8. The 45-Second Lexical-Analysis Error Audit

Before you submit an answer, distinguish occurrences, lexemes and categories. Apply longest match before priority. Check whether the sign belongs to the number. Remove only closed skipped spans. Find the earliest broken compiler contract. Test regexes against empty and digit-first strings. Leave nesting and declarations to later phases.

Reverse-check the source: 13 occurrences, 12 distinct lexemes and 11 categories.

Study Compiler Design, practise with GATE Test Series, or browse CS Fundamentals. Apply the audit to 12 solved lexical-analysis MCQs.