Backend Concurrency Keywords Explained

A deep-dive into the key terms you'll encounter when comparing backend architectures. Understanding these unlocks the "why" behind performance differences.

Threading & Locking

KeywordFull NameUsed InWhat It DoesGVLGlobal VM LockRuby (CRuby/MRI)Only 1 thread executes Ruby code at a time per process. I/O operations release it. This is why Rails needs multiple processes for true parallelism.GILGlobal Interpreter LockPython (CPython)Same concept as GVL. Python 3.13+ is experimenting with removing it (free-threaded mode).MutexMutual ExclusionAll languagesA lock that ensures only one thread accesses a shared resource at a time. Unlike GVL, you choose where to apply it.Deadlock—All languagesTwo threads each hold a lock the other needs. Both wait forever. Common bug in multi-threaded code.Race Condition—All languagesTwo threads read/write shared data simultaneously, causing unpredictable results.

Concurrency Models

KeywordUsed InWhat It MeansEvent LoopNode.js, Nginx, RedisSingle thread that continuously checks for I/O events and dispatches callbacks. Never blocks — if something takes time, it registers a callback and moves on.GoroutineGoLightweight user-space thread (~2KB stack). Go runtime schedules millions of these onto a few OS threads. Blocking a goroutine is cheap.Virtual ThreadJava 21+ (Project Loom)Java's answer to goroutines. Lightweight thread managed by JVM, not OS. Millions possible. Blocking is cheap.CoroutineKotlin, Python (asyncio)A function that can suspend and resume. Cooperative — must explicitly yield control. Used in async/await patterns.FiberRuby 3+Ruby's coroutine. Cooperative concurrency within a thread. Used by Async gem for non-blocking I/O.Actor ModelErlang, Akka (Java/Scala)Each "actor" has its own mailbox and state. Actors communicate by sending messages. No shared memory = no locks needed.

Scheduling

KeywordWhat It MeansExamplePreemptiveScheduler can interrupt a running task at any timeOS threads, Go goroutines (since Go 1.14)CooperativeTask runs until it voluntarily yields controlNode.js event loop, Python asyncio, Ruby FibersM:N SchedulingM user-space threads mapped to N OS threadsGo (goroutines), Java (virtual threads)1:1 SchedulingEach user thread = one OS threadJava (classic), Ruby threads, C pthreadsWork StealingIdle processor takes work from busy processor's queueGo scheduler, Java ForkJoinPool, Tokio (Rust)

I/O Models

KeywordWhat It MeansImpactBlocking I/OThread waits until I/O completesSimple code but wastes thread while waitingNon-blocking I/OI/O returns immediately, notifies laterEfficient but complex callback/async codeepollLinux kernel I/O event notificationUsed by Node.js, Go, Nginx under the hoodkqueuemacOS/BSD kernel I/O event notificationSame purpose as epoll for macOSio_uringLinux async I/O interface (newer)Faster than epoll for high-throughput I/OlibuvCross-platform async I/O libraryPowers Node.js event loop. Wraps epoll/kqueue/IOCP.

Runtime & Compilation

KeywordFull NameUsed InWhat It MeansV8—Node.js, ChromeGoogle's JavaScript engine. Compiles JS to native machine code via JIT.JVMJava Virtual MachineJava, Kotlin, ScalaRuns bytecode, manages memory (GC), JIT compiles hot paths.JITJust-In-Time CompilationJava, Node.js, .NETCompiles code to native during execution. Slow start, fast steady-state.AOTAhead-Of-Time CompilationGo, Rust, C/C++Compiles to native binary before running. Fast cold start.GCGarbage CollectionGo, Java, Node.js, RubyAutomatic memory management. Can cause brief pauses (stop-the-world).Zero-cost abstractions—Rust, C++High-level code compiles to same performance as hand-written low-level code.

Web Server Patterns

KeywordUsed InPatternThread-per-requestTomcat, classic SpringEach request gets a dedicated OS thread. Simple but expensive (1MB each).Event-drivenNode.js, Nginx, NettySingle/few threads handle all requests via event callbacks. Memory efficient.Multi-processPuma (Rails), Gunicorn (Python)Fork multiple worker processes. Each is independent. Simple but heavy on RAM.Reactor patternSpring WebFlux, Netty, TokioEvent loop dispatches I/O events to handlers. Non-blocking throughout.Proactor patternWindows IOCP, io_uringOS handles I/O completion and notifies app. Even more efficient than reactor.

Connection Handling

What happens when 10,000 users connect simultaneously:

Node.js (Event Loop):
  1 thread, 10K events in queue
  Memory: ~few KB × 10K = ~50 MB
  Status: ✓ handles fine

Go (Goroutines):
  10K goroutines on ~8 OS threads
  Memory: 2KB × 10K = ~20 MB
  Status: ✓ handles easily

Spring Classic (Thread Pool):
  200 thread max → 9,800 requests queued/rejected
  Memory: 1MB × 200 = ~200 MB
  Status: ✗ needs Virtual Threads or WebFlux

Spring + Virtual Threads:
  10K virtual threads on ~8 OS threads
  Memory: ~few KB × 10K = ~50 MB
  Status: ✓ handles fine

Rails (Puma, 4 workers × 5 threads):
  20 concurrent max → 9,980 requests queued
  Memory: 4 × 200MB = ~800 MB
  Status: ✗ needs many more workers or caching layer

Performance Keywords

KeywordWhat It MeansWhy It MattersThroughputRequests per second (RPS)How many users you can serve simultaneouslyLatency (p50/p99)Response time at 50th/99th percentilep99 = worst case for 99% of usersBackpressureMechanism to slow down producers when consumers are overwhelmedPrevents OOM crashes under load spikesConnection poolingReuse existing DB/HTTP connectionsAvoids expensive connection setup per requestC10K problemHandling 10,000 concurrent connectionsClassic benchmark. Solved by event-driven architectures.C10M problemHandling 10 million concurrent connectionsModern goal. Requires kernel bypass (DPDK, io_uring).Tail latencyLatency at high percentiles (p99, p99.9)GC pauses and thread contention show up here

Visual: How Each Handles "Waiting"

Request arrives → needs DB query (50ms wait)

┌─────────────────────────────────────────────────────────┐
│ Node.js:                                                │
│   [receive] → [send DB query] → [move to next request]  │
│                                   ↓                     │
│                            [DB responds] → [send reply] │
│   Thread: always busy (never waits)                     │
└─────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────┐
│ Go:                                                     │
│   [goroutine: receive → send DB query → PARK]           │
│   [scheduler runs other goroutines during 50ms]         │
│   [DB responds → UNPARK → send reply]                   │
│   Thread: always busy (parked goroutines cost nothing)  │
└─────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────┐
│ Java Classic:                                           │
│   [thread: receive → send DB query → BLOCK 50ms]        │
│   [thread does NOTHING for 50ms] ← wasted!              │
│   [DB responds → send reply]                            │
│   Thread: idle during wait (1MB wasted)                 │
└─────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────┐
│ Rails:                                                  │
│   [thread: receive → send DB query → BLOCK 50ms]        │
│   [GVL released during I/O → other thread can run]      │
│   [DB responds → reacquire GVL → send reply]            │
│   Thread: idle during wait + GVL contention on CPU work │
└─────────────────────────────────────────────────────────┘

Key Insight Summary

1. GVL/GIL is the #1 reason Ruby/Python can't match Go/Java in CPU-bound concurrency
2. Event Loop (Node.js) is brilliant for I/O but terrible for CPU work
3. Goroutines and Virtual Threads are the modern sweet spot — simple blocking code with async performance
4. M:N scheduling is the key architecture that enables millions of concurrent tasks
5. epoll/kqueue are the kernel primitives that make all non-blocking I/O possible
6. The C10K problem is solved — the industry is now working on C10M