ResearchNovember 15, 2026
-1
❄️

165M Model Trained for $4,200

Quark trains a 165M model end-to-end for $4,200 and releases it under Apache-2.0.

#LLM#Open Source#Training#Hugging Face#Reinforcement Learning
165-Millionen-Modell für 4.200 Dollar
Share Article
🔥 What happened A team released Quark, a 165M-parameter model trained end-to-end for about $4,200 in compute — including a scrapped first design, failed runs, data generation, and every evaluation. All four checkpoints are on Hugging Face under Apache-2.0. 💡 Why it matters Quark-150M-128K-Base tops its size class with a 52.8% benchmark average, beating Mamba-130M and OPT-125M. Its hybrid architecture keeps only four global layers, cutting the 128K-token KV cache to 0.5 GiB instead of 2 GiB. One variant learned to answer, reason, and call tools purely via RL — no supervised fine-tuning, no distillation. ⚡ Our take $4,200 for a full-stack sanity check is a steal — but the real headline is the Zero checkpoint: RL without SFT or distillation could rewrite how small models get trained.
The title, summary and analysis of this item were produced automatically by an AI system and have not been editorially reviewed. They may contain errors, bias or omissions — when in doubt, read the linked original source.