The first release of OhMyGPU Runtime is out: one headless daemon that gives applications local open-source models behind the OpenAI APIs they already speak. No chat app, no desktop shell, no agent framework — the deliberately boring layer underneath those things.
What ships in v0.1:
- Two inference APIs, one pipeline.
/v1/responsesas the canonical surface and/v1/chat/completionsfor compatibility — same model, same internal pipeline, both streaming, verified against the official OpenAI SDK. - A Management API (
/ohmygpu/v1/*) for the whole lifecycle: pull with resumable downloads, start, stop, delete, hardware detection, health, shutdown. Every state and failure reason is visible over HTTP. - Supervised llama.cpp — auto-installed per platform, one subprocess per running model, crashes surfaced with the log tail instead of swallowed.
- A verified catalog of GGUF instruct models, plus any GGUF by
hf:reference or URL. - A thin CLI —
omgis an administrative client of the API, nothing more.
The definition of done we held ourselves to: an Electron developer can bundle the runtime, tell it ensure model exists → start model over HTTP, and use either endpoint from existing code — simple, predictable, reliable and boring.
Binaries for macOS (Apple Silicon and Intel), Linux (x86_64, arm64) and Windows are on the releases page, with SHA-256 sums. What comes next is a bigger story.