| Catalog id | qwen3-4b-instruct |
| Source | unsloth/Qwen3-4B-Instruct-2507-GGUF |
| Quant · size | Q4_K_M · 2.5 GB |
| Tool calling | yes — native llama.cpp parser |
Qwen3 4B Instruct (the 2507 refresh, non-thinking) is a first-class catalog entry. At 2.5 GB it sits in the sweet spot for local work: comfortable on an 8 GB machine, quick to download, and strong for its size on instruction following and tool use.
What "in the catalog" means here — this is the checklist every adaptation post reports:
- Repo and file verified against the Hugging Face API; ungated, single-file GGUF.
pull → start → respondexercised on Metal and CPU builds of llama.cpp.- Tool calling round-tripped through
/v1/responsesand/v1/chat/completions— function call out,function_call_outputback in. - Streaming checked with the official OpenAI SDK against both endpoints.
omg model pull qwen3-4b-instruct
omg run qwen3-4b-instruct
Notes: the default context is the runtime's 8192; raise it per-start with --context-length if you have the memory. If you want a different quant, pull it directly with an hf: reference — unsupported, usually fine.