llm-all-models-async 0.1

Simon Willison's Blog / 4/1/2026

📰 NewsDeveloper Stack & InfrastructureTools & Practical UsageModels & Research

Read original →

共有:

Key Points

The release llm-all-models-async 0.1 adds “async” versions of LLM plugin models that originally only provide a synchronous implementation.
The author describes using a sync-only plugin (llm-mrchatterbox) with Datasette’s LLM features, which require async models to function.
Making this work required an additional async hook mechanism inside the LLM framework itself, delivered in LLM v0.30.
The approach uses a thread pool to run sync model calls in an async-compatible way, particularly for scenarios that need API-like async behavior.
The change expands compatibility between plugin ecosystems by allowing sync-only model plugins to be used where async-only consumers are expected.

Simon Willison’s Weblog

Sponsored by: WorkOS — Ready to sell to Enterprise clients? Build and ship securely with WorkOS.

31st March 2026

Release llm-all-models-async 0.1 — Register async versions of models from LLM plugins that only provide a sync version

LLM plugins can define new models in both sync and async varieties. The async variants are most common for API-backed models - sync variants tend to be things that run the model directly within the plugin.

My llm-mrchatterbox plugin is sync only. I wanted to try it out with various Datasette LLM features (specifically datasette-enrichments-llm) but Datasette can only use async models.

So... I had Claude spin up this plugin that turns sync models into async models using a thread pool. This ended up needing an extra plugin hook mechanism in LLM itself, which I shipped just now in LLM 0.30.

Posted 31st March 2026 at 8:52 pm