Skip to content

multithreading

on 1 CPU core -> concurrency only (swapping between threads/tasks) on N CPU cores -> concurrency and parallelism

models

thread per task

  • lots of threads may sit blocked waiting on I/O, consuming unnecessary memory
  • still high concurrency and cpu utilization (cpu does not get blocked by blocked thread), but when threads are blocked, new threads are spawned (keeping the blocked threads in memory)

async / event loop

  • a small number of threads stop working on the blocked task while it waits and pick up other work
  • re-allocate blocked threads to work on another task
  • ex: js event loop

benefits:

  • less memory per concurrent task
  • less thread/context-switch overhead
  • scales much better for I/O-heavy workloads

problem it solves:

  • waiting threads (even if CPU is not processing them) take up memory
  • scaling many waiting threads uses unnecessary memory due to thread memory overhead
  • synchronous I/O by definition makes the thread wait for I/O response
  • to solve that, we can make asynchronous I/O calls
  • issue: asynchronous by itself means we fire and forget, lose track of the response
  • what if we need/want to response tracked
  • that's where event loops are helpful

track the asynchronous threads to get response information without holding the thread blocked, causing many new threads to spawn to take on active work (even if it's not a CPU burden, it's a memory burden as new threads take memory)