Notes ✏️

06 - Goroutines

6 min read

golang

Go routines

concurrency: Execution out of order.

schedule period: The amount of time in which all threads have to run.

Context switch usually takes 1ms-2ms. 1ns = 12 instructions, we are wasting 12k/24k instructions every time we context switch and it has diminishing returns since we are doing more context switching than actual work.

Another parameter called MTS (minimum time slice) which states the minimum time a thread needs to run before context switching. But this will increase the scheduler period, as scheduler period = number of threads x MTS. if we have 1000 thread and MTS = 10ms, scheduler period = 10second (which is HUGE).

Usually this doesn’t happen, as most work is io bound/blocking, the thread never reaches its MTS.

Understanding our workload is required. If our workload is CPU bound, we never want threads > hardware threads, as the context switching results in diminishing returns or large scheduler period.

If our workload is mostly IO bound, we can have threads > hardware threads.

Go Scheduler

cooperative scheduler: similar to python async/await event loop scheduling. Context switch on events.

Context switch between go routines is ~200ns.

The following events are:

  • keyword go
  • Garbage collection
  • System calls (logging, output, io, etc)
    • Async syscalls
    • Sync syscalls
  • Blocking calls (atomic instructions, mutex locks, calling C libs, etc)

Everytime go program runs, logical processors are created = number of hardware threads. Go routines are scheduled on these logical processors. Each logical processor has a queue called Local Run Queue (LRQ) which holds all the go routines scheduled to run on that logical processor. Each logical processor has 1 OS thread to execute tasks which is then executed on the hardware thread.

Network Pooler

Its a thread pooling tech. Handle async syscalls. All these are network calls.

File IO is sync (OS yet doesn’t support it yet except windows).

If a go routine does a network call, it put on the pool of threads on the network pooler and other go routines can be scheduled on the logical processors.

If a go routine has to do FILE IO, the entire OS thread along with the go routine is removed from the logical processor and executed separately. This frees up the logical processor to pickup additional go routines using a different OS thread.

If we perform a blocking call (to a C lib), there is another process called System monitor, it monitors go routines and if a go routine has been idle for ~20ns, it will perform the same process as above as for File IO.

Go scheduler is a work stealing algorithm. If a logical processor is done executing all go routines, it will steal some go routines from other logical processors or even the Global Run Queue (a queue where go routines which haven’t been assigned a logical processor yet).

Go routines sort of convert IO bound to CPU bound since CPU work is the most efficient since context switching is not required. There is context switching happening at go routine level which is 200ns, this keeps the OS thread always in running state avoiding the 1000-2000ns context switching at hardware.

Synchronisation: Go routines waiting in line (timer constructions, mutexes are for synchronization) Orchestration: Go routines talking to each other (channels are for orchestration, etc)

Channels

Channel allows 1 go routine to signal another go routine about an event. The signal can be with data.

The sender go routine is called signalling go routine. There can be 2 dynamics:

  1. Do we want an ack that the signal has been received.
  2. Or we don’t care if the signal has been received or not.

basically if we need a guarantee or not. Guarantee comes at a cost, i.e, latency. But not have guarantee introduces risk, the signal might never be received.

Channel can be in the 3 states:

  • Open (can send and recv)
  • Zero value state or nil state (send and recv blocked, used for rate limits, etc)
  • Closed (used for cancellation and shutdowns).

If a channel is full, additional send code lines will be blocked. Similarly, if the channel is empty, recv code lines will be blocked.

Patterns

  • Wait For result
    • Spawn go routines for some work and wait for the result.
    • Base pattern for fan-out pattern.
  • Fan-out
    • A pattern where we spawn multiple other go routines to perform multiple pieces of work concurrently and then we wait for all of them to return.
    • This can be extremely dangerous in service based software as it can induce a lot of load on the system quickly.
  • Wait for task
    • Launch go routine to do some work but it doesn’t know what to do. Its used in pooling patterns.
  • Pooling
    • If multiple go routines are waiting on a channel, only one go routine will get the signal.
    • If all go routines needs to get the signal, like a broadcast, can only happen when the channel is closed.

Complex patterns

  • Fan-out semaphore
    • Batching go routines instead of running all go routines at once
    • Have a buffered channel of fixed size, when the channel is full, future go routines will wait until there is space. This way we can limit the number of go routines running at a given instance. But this doesn’t limit the creation of go routines. Go routines will be created but their execution will be blocked.
  • Fan-out bounded
    • Limit the creation of go routines to a specific number.
    • These limited go routines will execute all the work.
    • A good number of go routines is = num CPU
  • Drop pattern
    • By knowing the capacity of your system, accept the capacity of requests and drop the rest.
    • select case statement is used to perform multiple channel operations on the same go routine. If a case is blocked, then it goes to the default case
    • example
    select {
case ch <- "work":
	fmt.Println("Sent signal")
default:
	fmt.Println("Channel is full, above case statement is blocked. Dropping work")
}

- Cancellation Pattern
	- Most important pattern for long running tasks, example: http servers
	- We need to cancel long running tasks, settings timeouts for external calls such as DB calls, etc.
	- Design software with cancellation pattern and then drop the timeout numbers later as we might not know them during development.
	- Can use `context` package.
	- Use buffered channel always, if we have un-buffered channel, if the main go routine moves on, the spawned go routine will get stuck on the signaling as the channel is unbuffered and we will endup with a go routine leak.
	-  Buffer the context.cancel function
	- context.Done() function is when the clock starts ticking and use it with a select case statement with the blocked channel recv and context.Done(). If the channel case statement exceeds the timeout, the context.Done() case statement executes and the go routine can move on.