Java Virtual Threads: The Good, The Bad and The Ugly

All that glitters is not gold

The introduction of virtual threads in the LTS version of Java is a big milestone — they are no longer a "future" feature. There are countless articles which praise this feature, demonstrating that one can run millions of them without any issues, speculating how this feature will pave the way to the bright future of Java applications. I’m not convinced in this, and below I’ll try to explain why.

The Good

The main benefit of the addition of virtual threads is not related to threads itself. Their implementation required JVM maintainers to review and cleanup some Augean stables residing in JVM since V1.0 (for example, Socket API). Needless to say, this improves the quality and performance of the whole platform and benefits every language built on top of the JVM.

The Bad

The vast majority of examples showing usage of virtual threads, generate millions of virtual threads, where each thread just calls Thread.sleep() inside. Does that prove or show anything? Well, anyone with basic knowledge about ExecutorService easily can submit millions of instances of Runnable to ExecutorService (that’s what basically happens under the hood of the virtual threads). So, those examples show nothing except Java's ability to properly instantiate and then dispose of millions of objects. Is that so impressive these days?

In real scenarios, use of blocking calls doing something more useful than plain delay will require platform threads, and they still be a bottleneck. Fortunately, this does not include network I/O, which can be done asynchronously (just like we did long before virtual threads). But for other types of I/O it might result in hard to diagnose and fix bottlenecks. Which, in turn, requires in-depth understanding of how things are done internally.

Virtual threads are considered so lightweight, that the general recommendation is to not use pooling at all and create a new instance every time one needs to run a new task. This means that if you need to create many instances of virtual threads frequently, GC might sensibly affect performance. Of course, with the traditional approach — submitting tasks to ExecutorService, this issue also exists. But often, pooling allows significantly reduce memory allocation, often down to zero. With virtual threads, at least one allocation will remain.

Nevertheless, all of the above are rather minor issues, the most problematic thing lies in an entirely different area.

The Ugly

Virtual threads are positioned as a convenient way to continue using "thread-per-request" processing model. For some reason, it is considered "simple" and "convenient to use" in contrast to "complex" asynchronous processing. I'll return to the "simplicity" of this processing model soon. Before that, it is worth noting, that there are several different asynchronous processing models. Each has its own ergonomics and complexity, and this complexity is not necessarily greater than the complexity of the "thread-per-request" model.

Let's take a look at the "thread-per-request" model. Despite claimed "simplicity", it has several inherent issues, which make it not so simple to use in practice. Few most obvious ones:

• Access to shared mutable data requires synchronization and is prone to race conditions and deadlocks
• Thread-local variables, which are prone to memory leaks and other issues
• Parallelizing processing of the parts of the request requires manual thread management and often results in mixing of business logic and low level technical details of parallel execution. Handling errors in these scenarios is tedious and error-prone.

Project Loom authors perfectly aware about these issues. Introduction of scoped values and structured concurrency explicitly addresses exactly two last issues. But if the processing model is "simple" why are these tools necessary?

As an advantage of the "thread-per-request" often mentioned the fact that stack trace matches call stack, making finding issues simpler. That's definitely true. But this is relevant only for the “thread-per-request” code itself. Moreover, it’s relevant only to the code which uses exceptions to report errors. If one decides to use functional style error handling, it loses stack traces. Of course, it does not mean that functional style error handling is worse in this regard, or it makes nailing down issues harder. It does mean that the value of the stack traces exaggerated. Yes, they show invocation chain, but that information usually not so much important. Much more important information — context, i.e. values of parameters and local variables, is lost. At best, some details are provided from the point where the exception is thrown.

Conclusion

From my point of view, the evolution of the Java and JVM moves into the right direction — it gets more modern, functional and convenient to use, while maintaining a relative simplicity of the language. Virtual threads do not fit into that trend — they are conserving bad approaches which suffer from inherent issues and were proven bad decades ago.