Native OS thread pool primitives

[ishitatsuyuki]

Currently, Rayon's thread pool is based on manually spawning and managing threads, which gives better control of the actual behavior.

By contrast, Windows and macOS have thread pool primitives that specializes on managing threads and running tasks efficiently, e.g. based on resource usages. These APIs exposes a FIFO queue, and doesn't offer control over the underlying threads.

While these APIs works worse in cases like benchmark, it's often better for general purpose applications, especially desktop apps for end-user. I request this to be implemented in addition to the traditional API, and also made the default thread pool.

Links to documentation:

• Windows: Thread Pool API
• macOS: Grand Central Dispatch
• (This functionality does not exist on Linux, and most other OSes.)

[daira]

This functionality does not exist on Linux, and most other OSes.

While it doesn't exist at the OS level, it can be provided by libraries, and for code running in a platform-like context such as a web browser, the use of a particular thread pool abstraction might be expected:

• Chromium-based browsers
• Firefox / NSPR (on the other hand, there's no problem using rayon in Firefox because it already uses it internally for Stylo)

A sufficiently flexible API in rayon could potentially support both OS-based thread pools and user-level thread pools (#93). The current ThreadPoolBuilder API doesn't have the required flexibility: it allows some control over how threads are constructed in a rayon thread pool, but once rayon has control of a thread, it cannot relinquish it.

(You could run a rayon task that reuses its thread for something else and doesn't yield, relying on its thread's remaining work items to be stolen by other rayon threads. I don't know how well that would perform or work in practice; it's kind-of abusing the rayon API and breaking some of its assumptions. In any case it's different to using an external thread pool: all of the OS-level and library-level thread pools mentioned above tend to assume that an application submits relatively short-lived work items.)

This comment is motivated by wanting to use code written for rayon in a Chromium-based browser, Brave.

[cuviper]

For designing this integration, I think the first step should be to consider the join primitive. If we can sketch out how that would work, that should get us most of the way there.