In modern multithreaded applications, processing multiple concurrent requests is essential. However, creating a new thread for every single task is highly inefficient and resource-heavy. To solve this, developers use Thread Pools. By keeping worker threads alive and reusing them, we avoid the overhead of thread creation and destruction.

But how does a thread pool actually coordinate tasks under the hood? Let's translate this concurrency concept into simple terms using a cozy coffee shop kitchen analogy.

Illustration of a cozy coffee shop kitchen with baristas and order tickets
Real-World Analogy: The Coffee Shop Kitchen

Imagine you own a busy coffee shop. Making a single cup of coffee is a Task.

In a naive, non-pooled system, every time a customer walks in and orders a coffee, you would hire a brand-new barista from the street, train them, buy them a uniform, have them make that single coffee, and then immediately fire them once the coffee is served. Doing this is incredibly slow and wastes massive resources. In programming, creating a new `Thread` for every single task is just like that—it's highly "expensive" and sluggish.

Instead, you use a **Thread Pool**. You hire a **fixed team of permanent baristas** (worker threads) who stay in the kitchen all day. They hang around the counter, wait for order tickets (tasks) to land on the orders board (blocking queue), work on them one by one, and immediately pick up the next task when they finish. No hiring, no firing—just steady, reused execution.

Walkthrough of a Real Scenario

Let's trace exactly how our custom thread pool executes starting from the sample main method code:

Step 1: Hiring the Staff (Core Pool Initialization)

We start the shop by setting up our pool with 10 permanent baristas:

CustomThreadPool customThreadPool = CustomThreadPool.getInstance(false);

This creates a pool with a CAPACITY of 10. The constructor starts 10 Worker threads that enter a loop, waiting on the shared BlockingQueue. Since no orders have been submitted yet, all 10 baristas stand idle at the counter.

Step 2: Submitting 100 Orders (Task Queueing)

Suddenly, a large group of customers orders 100 coffees:

for (int index = 0; index < 100; ++index) {
    customThreadPool.submitTask(new Task("Task -- " + index));
}

Here's how the threads handle this surge:

  • The 10 worker threads immediately take the first 10 orders and start processing them.
  • Each task sleeps for 5 seconds (Thread.sleep(5000)) to simulate making a coffee.
  • The remaining 90 tasks wait in the LinkedBlockingQueue.
  • As soon as a worker thread completes a task, it automatically grabs the next one from the front of the queue using `taskQueue.take()` and continues working.
Step 3: Closing the Shop (Shutdown Initiation) 
customThreadPool.shutdown();

The manager sets isShutDownInitiated to true. This stops any new orders from being accepted. However, because our worker loops check while (!isShutDownInitiated || !taskQueue.isEmpty()), the baristas continue making all the remaining coffees that are already queued up. Once the queue is empty, they exit the loop and the threads shut down cleanly.

Custom Thread Pool Java Implementation

Here is the full Java source code showing how to build this custom thread pool, worker threads, and queue management from scratch:

package io.practise.accolite;

import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.atomic.AtomicBoolean;

public class CustomThreadPool {

    private static final int CAPACITY = 10;
    private BlockingQueue<Runnable> allThreads;
    private final AtomicBoolean isShutDownInitiated;
    private static CustomThreadPool customThreadPoolInstance;
    private final Thread[] workers;

    private CustomThreadPool(boolean isShutdownInitiated) {
        this.isShutDownInitiated = new AtomicBoolean(isShutdownInitiated);
        this.allThreads = new LinkedBlockingQueue<>(CAPACITY);
        this.workers = new Thread[CAPACITY];

        for (int index = 0; index < CAPACITY; ++index) {
            workers[index] = new Worker(allThreads, isShutDownInitiated);
            workers[index].start();
        }
    }

    public static CustomThreadPool getInstance(boolean isShutdownInitiated) {
        if (customThreadPoolInstance == null) {
            customThreadPoolInstance = new CustomThreadPool(isShutdownInitiated);
        }
        return customThreadPoolInstance;
    }

    public void submitTask(Runnable task) {
        if (!isShutDownInitiated.get()) {
            allThreads.add(task);
        }
    }

    public void shutdown() {
        isShutDownInitiated.set(true);
        for (Thread worker : workers) {
            worker.interrupt();
        }
    }

    public static void main(String[] args) {
        CustomThreadPool customThreadPool = CustomThreadPool.getInstance(false);

        for (int index = 0; index < 100; ++index) {
            Task task = new Task("Task -- " + index);
            customThreadPool.submitTask(task);
        }

        customThreadPool.shutdown();
    }
}

class Worker extends Thread {
    private final BlockingQueue<Runnable> taskQueue;
    private final AtomicBoolean isShutDownInitiated;

    public Worker(BlockingQueue<Runnable> taskQueue, AtomicBoolean isShutDownInitiated) {
        this.taskQueue = taskQueue;
        this.isShutDownInitiated = isShutDownInitiated;
    }

    @Override
    public void run() {
        while (isShutDownInitiated.get() != true || !taskQueue.isEmpty()) {
            try {
                Runnable task = taskQueue.take();
                task.run();
            } catch (InterruptedException e) {
                if (isShutDownInitiated.get()) {
                    Thread.currentThread().interrupt();
                    break;
                }
            }
        }
    }
}

class Task implements Runnable {
    private String taskName;

    public Task(String task) {
        this.taskName = task;
    }

    @Override
    public void run() {
        System.out.println("Task started !! " + taskName);
        try {
            Thread.sleep(5000);
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
        }
        System.out.println("Task Ended -- " + taskName);
    }
}

Conclusion

Creating custom thread pools allows you to grasp the core architecture that makes Java's ExecutorService so powerful under the hood. By utilizing blocking queues and permanent worker loops, thread pools balance high workload demands without blowing up memory footprints or wasting CPU cycles on thread allocation lifecycle costs!