Harnessing Counting Semaphores for Optimal Resource Management.

Imagine we are hosting a website that changes images from one format to another. The Golang backend application responsible for managing the user requests internally calls the image converter service for conversion. The image converter service which is deployed on multi-core machine, can convert up to four images at once. However, converting images takes a lot of time, so we need to make sure we don’t overload the system with too many requests at the same time.

To manage this, we need to throttle the number of requests we allow at once. If we treat each call to image conversion as a critical section can we use a simple lock (mutex) to manage access? Probably not. A standard mutex would only allow one request to be served at a time, which would slow things down a lot because our system can handle four image conversion requests at once.

Can we do better? As it turns out, yes. We can make use of a counting semaphore in this particular scenario. The counting semaphore will make sure that only a certain number of concurrent calls to image conversion service is allowed. Let’s first understand the concept of a counting semaphore and then I will show you how it can used for throttling in the current scenario.

What is a counting semaphore?

Working of a Semaphore:

1. Initialization — A counting semaphore is Initialized with a given permit size. The permit size defines the maximum number of threads that can access the critical section concurrently.
2. Acquire Permit — Once a thread wants to enter the critical section, it needs to first acquire the permission from semaphore. This step reduces the permit by one. If the existing permit is already zero, thread waits for its turn.
3. Release Permit- After a thread is done with the critical section, it increments the permit by one. Internally, a singal is sent to all the sleeping threads and one of them is chosen randomly for them to proceed.

If you think about it, a counting semaphore is essentially a counter. Whenever a thread wants to execute in critical section, it first decrements the permit. If the permit is already zero, thread waits until the permit becomes non zero again. After the critical section work is complete, thread increments the permit. If other threads are waiting, one of them is allowed to proceed.

Implementing Counting Semaphore using Condition Variable

In Golang, the standard library does not include a counting semaphore, but it’s quite straightforward to implement one using a condition variable.

We have discussed the three main operations of a counting semaphore: Initialization, Acquire, and Release. Before we proceed with implementing these steps, let’s first define the structure that represents a counting semaphore.

type CountingSemaphore struct {
	permits int       // defines the maximum number of permits
	cond *sync.Cond.  // Condition Variable
}

Now that we’ve defined the CountingSemaphore structure, let’s go ahead and implement the three key steps.

Now that we’ve defined the CountingSemaphore structure, let’s go ahead and implement the three key steps.

Step 1: Semaphore creation

This step sets up the semaphore with a specified number of permits.

func NewCountingSemaphore(permits int) *CountingSemaphore {
	return &CountingSemaphore{permits: permits, cond: sync.NewCond(&sync.Mutex{})}
}

Step 2: Acquiring the semaphore

When Acquire is called, it reduces the permit count by one. If the permit count is already at zero, the goroutine that called Acquire will block.

func (sema *CountingSemaphore) Acquire() {
	sema.cond.L.Lock()
	if sema.permits == 0 {
		sema.cond.Wait()
	}
	sema.permits--
	sema.cond.L.Unlock()
}

Step 3: Releasing the semaphore

When Release is called, it increases the permit count by one. It also signals all the blocked Goroutines, allowing one of them, chosen at random, to continue execution.

func (sema *CountingSemaphore) Release() {
	sema.cond.L.Lock()
	sema.permits += 1
	sema.cond.Signal()
	sema.cond.L.Unlock()
}

That’s all there is to it. We’ve successfully created our own version of a counting semaphore using condition variable. Now, let’s revisit our original example to explore how counting semaphores can be used to throttle image conversion requests.

Throttling the request to image conversion service

We will use two mock method to mimic user Request handler and external image conversion service. The external service will mimic intensive processing by pausing for 3 seconds.

The main method will simulate user load by launching multiple user request handler goroutines simultaneously. Despite the presence of multiple concurrent user requests, the use of a semaphore will regulate access, ensuring that only a limited number of requests can invoke the image conversion service at any given time. In this example, we will limit this to four requests.

package main

import (
	"fmt"
	"sync"
	"time"
)

type CountingSemaphore struct {
	size int
	cond *sync.Cond
}

func NewCountingSemaphore(size int) *CountingSemaphore {
	return &CountingSemaphore{size: size, cond: sync.NewCond(&sync.Mutex{})}
}

func (sema *CountingSemaphore) Acquire() {
	sema.cond.L.Lock()
	if sema.size == 0 {
		sema.cond.Wait()
	}
	sema.size--
	sema.cond.L.Unlock()
}

func (sema *CountingSemaphore) Release() {
	sema.cond.L.Lock()
	sema.size += 1
	sema.cond.Signal()
	sema.cond.L.Unlock()
}

func externalImageConversionService(userID int) {
	fmt.Println("Timestamp:", time.Now().Second(), "", "Converting image for user id ", userID)
	time.Sleep(3 * time.Second)
}

func userRequestHandler(userID int, sema *CountingSemaphore, wg *sync.WaitGroup) {
	sema.Acquire()
	externalImageConversionService(userID)
	sema.Release()
	wg.Done()
}

func main() {
	sema := NewCountingSemaphore(4)
	wg := sync.WaitGroup{}
	for userId := range 10 {
		wg.Add(1)
		go userRequestHandler(userId, sema, &wg)
	}

	wg.Wait()
}

When we run the above code, it gives the following output.

Timestamp: 52  Converting image for user id  1
Timestamp: 52  Converting image for user id  9
Timestamp: 52  Converting image for user id  0
Timestamp: 52  Converting image for user id  4

Timestamp: 55  Converting image for user id  5
Timestamp: 55  Converting image for user id  7
Timestamp: 55  Converting image for user id  8
Timestamp: 55  Converting image for user id  2

Timestamp: 59 Converting image for user id  3
Timestamp: 59 Converting image for user id  6

As you can see, at any given moment, no more than four requests are made to the image conversion service. This occurs because the semaphore we implemented restricts access to the critical section to only four Goroutines at any time.