Memory Layout — Stack vs Heap vs Data

Every program's memory is divided into distinct segments:

┌─────────────────────────────────┐  High Address (0xFFFF...)
│           Stack                  │  ← grows downward ↓
│  (local variables, function     │
│   calls, return addresses)      │
├─────────────────────────────────┤
│              ↓                   │
│         (free space)             │
│              ↑                   │
├─────────────────────────────────┤
│           Heap                   │  ← grows upward ↑
│  (dynamically allocated memory) │
│  (malloc, new, objects)         │
├─────────────────────────────────┤
│      BSS (uninitialized data)   │
│  (global vars set to 0)         │
├─────────────────────────────────┤
│      Data (initialized data)    │
│  (global/static vars with       │
│   assigned values)              │
├─────────────────────────────────┤
│         Text (Code)             │
│  (compiled machine instructions)│
└─────────────────────────────────┘  Low Address (0x0000...)

Comparison Table

StackHeapData (Static)Text (Code)What's storedLocal variables, function args, return addressesDynamically allocated objectsGlobal/static variablesCompiled instructionsAllocationAutomatic (push/pop)Manual or GCCompile timeCompile timeSpeedVery fast (move pointer)Slower (find free block)Fast (fixed address)N/ASizeSmall (~1-8 MB)Large (limited by RAM)FixedFixedLifetimeUntil function returnsUntil freed/GCEntire programEntire programThread safetyEach thread owns its stackShared (needs locks!)Shared (needs sync)Read-only (safe)GrowsDownward ↓Upward ↑Doesn't growDoesn't growFragmentationNeverYesNeverNever

Virtual Memory — The Illusion

Every process thinks it has the entire memory space to itself. The OS and CPU cooperate to create this illusion via page tables.

Process A sees:              Process B sees:
┌──────────────────┐         ┌──────────────────┐
│ 0xFFFF: Stack    │         │ 0xFFFF: Stack    │
│ 0x1000: Heap     │         │ 0x1000: Heap     │
└──────────────────┘         └──────────────────┘
     Both think they "own" address 0x1000!

        Page Table (per process):
        Process A: 0x1000 → Physical 0x8000
        Process B: 0x1000 → Physical 0xF000

TLB — Translation Lookaside Buffer

CPU Memory Access:

  CPU → TLB (cache) → Page Table (RAM) → Physical RAM
          │                    │
      TLB Hit: 1ns        TLB Miss: 100ns
      (99% of time)       (must walk page table)

CPU Cache Hierarchy

┌──────────┐
│CPU Core  │
│ L1 Cache │  ~1 ns      32-64 KB
│ L2 Cache │  ~3-5 ns    256 KB-1 MB
└──────────┘
 L3 Cache     ~10-20 ns   8-64 MB (shared)
 RAM          ~50-100 ns  16-512 GB
 SSD/Disk     ~100,000 ns TB+

Access time:
  L1:   ████                                    1 ns
  L2:   ████████████                            5 ns
  L3:   ████████████████████████████            20 ns
  RAM:  ████████████████████████████████████... 100 ns
  SSD:  (1000× off screen)                      100,000 ns

Cache Lines — Why Sequential Access is Fast

CPU loads 64 bytes at a time (one cache line):

Sequential access (array):
  array[0] → MISS, load 64 bytes into cache
  array[1] → HIT (free!)
  array[2] → HIT (free!)
  ...
  array[15] → HIT (free!)
  Hit rate: 94% ← FAST

Random access (linked list):
  node at 0x1000 → MISS
  node at 0xF000 → MISS
  node at 0x3000 → MISS
  Hit rate: ~0% ← SLOW (30× slower in practice!)

Page Faults

Types:
  Minor: Page exists but not mapped yet → ~1 μs
  Major: Page on disk (swap) → ~10 ms (100,000× slower!)
  Invalid: Illegal access → SEGFAULT

char *buf = malloc(1GB);  // no page fault (just virtual)
buf[0] = 'x';            // MINOR fault → maps one 4KB page
// Only 4KB physical RAM used, not 1GB!

Complete Memory Access Path

Your code: x = array[i]
      │
      ▼
1. Virtual address generated
      │
      ▼
2. TLB lookup (virtual → physical)
   ├─ HIT: 1ns
   └─ MISS: walk page table (~100ns)
            └─ Page not in RAM? PAGE FAULT (~10ms!)
      │
      ▼
3. Check L1 → L2 → L3 → RAM
   ├─ L1 HIT: 1ns
   ├─ L2 HIT: 5ns
   ├─ L3 HIT: 20ns
   └─ RAM: 100ns
      │
      ▼
4. Data returned to CPU

Best case: 1ns  |  Worst case: 10,000,000ns
Ratio: 10,000,000× difference!

Study Path

1. Pointers & References — how stack/heap are accessed
2. Memory Alignment & Padding — why structs aren't the size you expect
3. Virtual Memory & Paging — how OS maps logical → physical
4. CPU Cache (L1/L2/L3) — why layout affects performance 10-100×
5. Garbage Collection — mark-sweep, generational, concurrent
6. Escape Analysis — how Go/Java decide stack vs heap
7. Ownership (Rust) — no GC, no manual free
8. NUMA — multi-socket memory locality