Memory Hierarchy and Locality: Worked AMAT Example and Exam Traps
Learn why memory levels exist, classify temporal and spatial locality from addresses, and calculate two-level AMAT. Includes two cross-checks and exam traps.
KnowledgeGate Team
Exam prep & CS education

Registers, cache, RAM and storage form a hierarchy because no single technology combines register-like latency, RAM-scale capacity and storage cost. A repeated address demonstrates temporal locality; nearby addresses within one fetched block demonstrate spatial locality. In a two-level cache, average memory access time weights the L1-hit, L2-hit and memory paths by their probabilities, while the L2 local miss rate remains conditional on an L1 miss.
1. Why a Computer Uses a Memory Hierarchy
A memory hierarchy is a set of storage levels that trade off access time, capacity and cost per bit. The conceptual order is registers, L1 cache, L2 cache, main memory and SSD. Upper levels are generally smaller and faster. Lower levels are generally larger and slower.
The processor checks a nearby level first. Data found there is a hit; absent data is a miss and sends the request lower. Hit time is the time to check and obtain data. Miss penalty is the extra time spent going farther down.
Memory hierarchy sits in the computer organisation part of CS Fundamentals for Exams & Placements.

2. Temporal and Spatial Locality in an Address Trace
Temporal locality means reusing the same item soon. Spatial locality means accessing nearby items soon. Caches exploit both by retaining recently used blocks and neighbouring bytes.
Consider this six-access trace of 4-byte words:
0x1000, 0x1004, 0x1008, 0x1000, 0x100C, 0x1010
Assume a cold cache, a 16-byte aligned block and no eviction. The first 0x1000 access fetches bytes 0x1000 through 0x100F.
0x1004,0x1008and0x100Cshow spatial reuse because they lie inside that fetched block.The second access to
0x1000shows temporal reuse because the same word is requested again.0x1010is the first word of the next aligned block, so it is not covered by the earlier fetch.
Sequential addresses often show spatial locality. Temporal locality requires reuse of the same item.
3. From Hits and Misses to Average Memory Access Time
At one cache level:
hit rate + miss rate = 1
The L2 local miss rate is measured only among requests that have missed L1 and reached L2. It is not the fraction of all references that reach memory.
For serial lookup, the form needed here is:
AMAT = tL1 + mL1 x (tL2 + mL2 x tMem)
Every request pays for L1. Only L1 misses pay for L2, and only double misses pay for memory.
For direct, associative and set-associative placement plus a single-level access-time example, use Cache Memory Mapping and Hit Ratio: Solved Examples.
Cache Memory Organization in COA: Mapping, Hit Ratio and Worked Examples carries one cache configuration through field splits, all three mapping schemes, write policies and directory overhead. Keep placement fixed here: the six-address trace separates temporal and spatial reuse, while the L1-L2 tree separates local from global miss probability.
4. Worked Two-Level AMAT Example
Use this complete illustrative system:
L1 hit time =
1 nsL1 miss rate =
10% = 0.10L2 lookup time after an L1 miss =
5 nsL2 local miss rate =
20% = 0.20Main-memory service after an L2 miss =
80 nsLookup is serial, with no extra transfer time beyond the stated values
L1 hit:
1 - 0.10 = 0.90.L2 hit as a fraction of all references:
0.10 x (1 - 0.20) = 0.10 x 0.80 = 0.08.Main-memory access:
0.10 x 0.20 = 0.02.
Check the outcomes: 0.90 + 0.08 + 0.02 = 1.00.
AMAT = 1 + 0.10 x (5 + 0.20 x 80)
= 1 + 0.10 x (5 + 16)
= 1 + 0.10 x 21
= 3.1 ns
Cross-check the path times: L1 hit 1 ns, L2 hit 1 + 5 = 6 ns, and memory 1 + 5 + 80 = 86 ns.
AMAT = 0.90 x 1 + 0.08 x 6 + 0.02 x 86
= 0.90 + 0.48 + 1.72
= 3.10 ns
Both methods agree. Without L2, with the same 80 ns service, AMAT = 1 + 0.10 x 80 = 9 ns. Here, L2 lowers AMAT by 9 - 3.1 = 5.9 ns. This saving is specific to the example.

5. Turning the Locality Trace into a Hit Ratio
Use the six-access trace to calculate the outcomes. With a cold cache and a 16-byte block, the outcomes are:
miss, hit, hit, hit, hit, miss
Four hits in six accesses give 4/6 = 2/3, or about 66.7%. The second 0x1000 is temporal reuse; the nearby words are spatial reuse. The outcome is the same, but the reason differs.
Now change only the block size to 4 bytes. Keep the cache cold and assume no eviction. Each distinct word occupies its own block, giving:
miss, miss, miss, hit, miss, miss
Only the repeated 0x1000 hits, giving 1/6, or about 16.7%. Here, the 16-byte block captures spatial locality that the 4-byte block cannot.
Larger blocks are not always better. Unused fetched bytes, fewer resident blocks and cache pollution can offset the benefit.
6. Traps That Make Correct Formulas Produce Wrong Answers
Trap 1: treating a local rate as a global probability. The wrong expression 1 + 0.10 x 5 + 0.20 x 80 = 17.5 ns charges memory to 20% of all references. Memory is reached after both misses, with probability 0.10 x 0.20 = 0.02.
Trap 2: inserting a hit rate where a miss rate is required. Also, 10% becomes 0.10, not 10. Convert probabilities and check they lie between zero and one.
Trap 3: mislabelling locality or treating example timings as universal. A repeated address is temporal reuse. A different address in the fetched block is spatial reuse. The 1 ns, 5 ns and 80 ns values describe only this worked system.
7. How Exams Turn These Ideas into Questions
Questions may ask you to order levels, classify a loop or address trace, count hits for a block size, calculate AMAT, or find an unknown miss rate. Keep cache locality separate from address translation, paging and virtual memory.
Use this solving order:
Write every given time and rate with its unit.
Mark each rate as conditional or global.
Draw the possible access paths.
Multiply conditional rates to obtain path probabilities.
Calculate AMAT and perform a bounds check.
Here, 3.1 ns is at least the 1 ns L1 time and far below the 86 ns full-miss path. An answer outside those bounds would expose an error.
8. The Short Version and Your Next Step
A hierarchy balances access speed, capacity and cost per bit.
Temporal locality reuses the same item.
Spatial locality accesses nearby items.
AMAT weights each path by the probability of reaching it.
For a structured pass across core CS subjects, continue with Zero to Hero, Complete CS Course. If your immediate goal is placement-focused fundamentals, use CS Fundamentals for Placements by Sanchit Sir.
Now redraw the probability tree and change only the L1 miss rate from 10% to 5%. Your check value is 1 + 0.05 x (5 + 0.20 x 80) = 1 + 0.05 x 21 = 2.05 ns.
Keep learning
Hierarchy & Locality MCQs: 12 Solved Questions with Explanations
Test memory hierarchy, locality, cache hits, AMAT, cache levels, and block-size trade-offs through 12 solved exam MCQs with worked explanations.
Memory Organisation and Performance: Cache Mapping, AMAT and Worked GATE Examples
Connect cache capacity, address fields and access time through one 32-bit worked system. Then compare mapping choices and solve a past GATE-style field relation.
Memory Management Basics and Hierarchy MCQs: 12 Solved Questions with Explanations
Solve 12 published memory-management MCQs with step-by-step explanations covering hierarchy, volatility, locality, MMU translation, address bits, and protection.
Memory Hierarchy and Virtual Memory: paging, the TLB, and address translation
Memory hierarchy and virtual memory explained: registers to disk, locality, paging, page tables, the TLB, a worked address translation, and access time.





