The Executor Framework and Thread Pools in Java

Why pools exist

// One thread per request: creation cost per task, and no upper bound
for (Request request : requests) {
    new Thread(() -> handle(request)).start();
}

Creating a platform thread costs time and around a megabyte of stack. Ten thousand requests would create ten thousand threads and exhaust memory. A pool creates a fixed set of threads once and feeds them tasks from a queue.

ExecutorService

ExecutorService executor = Executors.newFixedThreadPool(4);

executor.execute(() -> handle(request));            // Runnable, no result
Future<String> future = executor.submit(this::load); // Callable, a result

executor.shutdown();                                 // no new tasks accepted
if (!executor.awaitTermination(30, TimeUnit.SECONDS)) {
    executor.shutdownNow();                          // interrupt what is running
}

// Java 19 onwards: ExecutorService is AutoCloseable, close() shuts down and waits
try (ExecutorService executor = Executors.newFixedThreadPool(4)) {
    executor.submit(task);
}

An executor must be shut down. Its threads are non daemon by default, so a program that forgets never exits.

The factory methods

// A cached pool is unbounded: a burst of slow tasks can create thousands of threads
ExecutorService risky = Executors.newCachedThreadPool();

Sizing a pool

int processors = Runtime.getRuntime().availableProcessors();

ExecutorService cpuBound = Executors.newFixedThreadPool(processors);
ExecutorService ioBound = Executors.newFixedThreadPool(processors * 4);   // a rough start

CPU bound work gains nothing from more threads than processors. I/O bound work benefits from more, because threads spend most of their time waiting. The multiplier is a starting point to be replaced by measurement.

Configuring the pool directly

ThreadPoolExecutor pool = new ThreadPoolExecutor(
        4,                                   // core threads
        16,                                  // maximum threads
        60, TimeUnit.SECONDS,                // idle timeout above the core size
        new ArrayBlockingQueue<>(1000),      // a bounded queue
        new ThreadPoolExecutor.CallerRunsPolicy());   // what to do when full

Use a bounded queue. An unbounded one turns a load spike into an out of memory error, and by then there is no way to shed load gracefully.

Scheduled tasks

ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(2);

scheduler.schedule(this::sendReminder, 10, TimeUnit.SECONDS);

scheduler.scheduleAtFixedRate(this::collectMetrics, 0, 1, TimeUnit.MINUTES);
scheduler.scheduleWithFixedDelay(this::cleanUp, 0, 1, TimeUnit.MINUTES);

scheduleAtFixedRate measures from each start, so a slow run causes the next to begin immediately. scheduleWithFixedDelay measures from each finish and leaves a real gap. If a repeating task throws, it is silently cancelled, so wrap the body in a try catch.

Submitting many tasks

List<Callable<String>> tasks = urls.stream()
        .map(url -> (Callable<String>) () -> fetch(url))
        .toList();

List<Future<String>> results = executor.invokeAll(tasks);   // waits for all
String firstDone = executor.invokeAny(tasks);                // waits for one

Naming threads

ThreadFactory factory = Thread.ofPlatform().name("import-", 0).factory();
ExecutorService executor = Executors.newFixedThreadPool(4, factory);

Default names such as pool-1-thread-3 tell you nothing in a stack trace. Naming pools is a small change that repays itself the first time something goes wrong.

Virtual threads

try (ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor()) {
    for (Request request : requests) {
        executor.submit(() -> handle(request));    // one virtual thread each
    }
}

Virtual threads are scheduled by the JVM onto a small set of carrier threads. When one blocks on I/O, its carrier is released to run another. Millions can exist at once, so pooling becomes unnecessary for I/O bound work: create one per task.

A worked example

public Map<String, Integer> countWordsInFiles(List<Path> files)
        throws InterruptedException {

    Map<String, Integer> totals = new ConcurrentHashMap<>();

    try (ExecutorService executor = Executors.newFixedThreadPool(
            Runtime.getRuntime().availableProcessors())) {

        for (Path file : files) {
            executor.submit(() -> {
                try (Stream<String> lines = Files.lines(file)) {
                    lines.flatMap(line -> Arrays.stream(line.split("[^a-zA-Z]+")))
                         .filter(word -> !word.isBlank())
                         .forEach(word -> totals.merge(word, 1, Integer::sum));
                } catch (IOException e) {
                    logger.warning("Skipped " + file);
                }
            });
        }
    }
    return totals;
}

Common mistakes

• Forgetting to shut down, so the JVM never exits.
• Using an unbounded queue and turning a load spike into an out of memory error.
• Using newCachedThreadPool for slow tasks and creating thousands of threads.
• Ignoring the Future from submit, which silently swallows every exception.
• Letting a scheduled task throw, which cancels all future runs.
• Pooling virtual threads, which defeats their purpose.

Best practices

• Prefer an executor to creating threads by hand.
• Use a bounded queue with a deliberate rejection policy.
• Size the pool from the workload type, then measure.
• Name your pool threads.
• Always shut down, ideally with try with resources.
• Check the Future, or exceptions vanish.
• Use virtual threads for I/O bound work on Java 21 and later.

Practice

1. Submit ten tasks to a pool of two threads and observe how they queue.
2. Why does a task that throws inside submit appear to succeed?
3. Explain the difference between scheduleAtFixedRate and scheduleWithFixedDelay.
4. Choose a pool size for a CPU heavy job and for a job that mostly waits on a network, and justify each.
5. Rewrite a fixed pool solution using a virtual thread per task executor.

Conclusion

An executor decouples what runs from where it runs. Choose a bounded queue and a deliberate rejection policy, size the pool for the workload, always shut down, and reach for virtual threads when the work is dominated by waiting.