Hi there 👋

In the previous post I wrote about releasing ComposableThreads for Rust (also published on crates.io). I’ve kept improving it since, so let me lay out where that’s gone. As I mentioned last time, while benchmarking I found that rayon’s join() was faster than I expected. At first I figured that if I just tuned the stackless (async) implementation properly, I could catch up to rayon’s performance, but it became clear there’s a limit to how far tuning async alone can get you. Thinking it through: as long as you’re building a mechanism to suspend a task synchronously, you can’t avoid the bookkeeping needed to track task state, or the branching cost of returning to the scheduler once. Async is marketed as “zero overhead,” but the cost of pulling out a continuation is unavoidable — I realized there was no catching up to rayon’s performance without changing the API itself. So I reworked the implementation to bring in rayon’s API as another model. ...

I released ComposableThreads (Rust edition) — a library for working with what are commonly called user-level threads in Rust. https://github.com/endowataru/cmpth-rs This was also my first time publishing a package to crates.io. https://crates.io/crates/cmpth ComposableThreads originally started as a C++ library I wrote during my PhD. I built it on the side while working on Distributed Shared Memory (DSM) for HPC — the main theme of my doctoral dissertation. When you try to build a DSM system, multithreading becomes deeply involved on both the application and system sides. ...