Bonsai 27B is PrismML's ternary-quantized build of Qwen3.6 27B, and its headline claim is real: it packs a 27-billion-parameter model into 3.9GB and runs on an iPhone 17 Pro at about 11 tokens per second. The weights are free under Apache 2.0, and the ternary version keeps more than 95% of the full-precision benchmark score. The catch is that the quality it keeps is uneven: math and coding stay near-parity, while tool-calling and vision drop off. So Bonsai 27B is a useful on-device model for reasoning and code, and a shaky one for agentic, tool-heavy work. This guide covers how the compression works, what the numbers mean, the four ways to run it, and who should bother.
How PrismML shrank a 27B model to fit a phone
A standard 27B model in FP16 needs roughly 54GB of memory, far more than any phone has. Bonsai 27B gets under 6GB by replacing full-precision weights with tiny discrete values.
The ternary variant constrains every weight to one of three values: -1, 0, or +1. That is about 1.58 bits of information per weight in theory. PrismML adds FP16 group-wise scaling (storing a full-precision scaling factor per group of weights so the collapsed values still land at the right magnitude), which brings the real cost to roughly 1.71 effective bits per weight. The more aggressive 1-bit binary variant drops the zero and keeps only -1 and +1, landing near 1.125 effective bits.
Two details matter. First, the compression is end-to-end: embeddings, attention, the MLP blocks, and the language-model head are all quantized, with no full-precision components left as a crutch. Second, the base is Qwen3.6 27B (27.8B parameters, a hybrid-attention causal model), so Bonsai inherits that model's 262K-token context window and multimodal input.
Ternary or 1-bit: which build to download
PrismML ships two builds, and picking the wrong one wastes either memory or quality. The trade is simple: smaller file, lower accuracy.
| Build | Effective bits/weight | Size | Quality retained | Best for |
|---|---|---|---|---|
| Ternary (-1, 0, +1) | ~1.71 | 5.9GB | >95% of FP16 | Desktop, quality-sensitive work |
| 1-bit binary (-1, +1) | ~1.125 | 3.9GB | >90% of FP16 | Phones, tight memory budgets |
If you are running on a phone, the 1-bit build is the one that fits inside an iPhone 17 Pro's memory headroom. If you have a laptop or desktop with room to spare, the ternary build buys back several points of accuracy for two extra gigabytes, worth it whenever the output quality matters more than the footprint.
What the benchmarks actually say (and where quality slips)
The official numbers are for the ternary build, which averages 80.49 across 15 thinking-mode benchmarks. The standouts are strong:
| Benchmark | Score | What it measures |
|---|---|---|
| MATH-500 | 99.20 | Grade-to-competition math |
| AIME 2025 | 90.84 | Hard competition math |
| HumanEval+ | 93.90 | Code generation |
| BFCL v3 | 74.41 | Function/tool calling |
Read those side by side and the shape of Bonsai 27B becomes clear. Math and code sit in the 90s. Tool calling sits in the 70s. That gap is the single most important thing to understand before you rely on it: the quantization preserves reasoning and code far better than it preserves the structured, multi-step behavior that agentic workflows depend on.
Those figures are PrismML's own, so treat them as a starting point rather than the final word until independent benchmarks accumulate. The signals worth watching are the ones a headline average hides: check the per-task scores rather than the 80.49 mean, and note the failure modes people actually hit. Early testers have reported the ternary build occasionally getting stuck in reasoning loops, and extreme quantization tends to erode long-context stability more than a single-task score suggests. Both are worth a quick test on your own prompts before you rely on it.
How fast it runs, and on what hardware
Compression this aggressive only matters if inference is fast enough to use. It is, on capable hardware. These are PrismML's own reported figures:
| Device | 1-bit | Ternary |
|---|---|---|
| iPhone 17 Pro | 11 tok/s | — |
| Apple M5 Max | 87 tok/s | 58 tok/s |
| NVIDIA RTX 5090 | 163 tok/s | 134 tok/s |
The 11 tokens per second on a phone is usable for chat and short reasoning, if not blazing. On an M5 Max, 87 tokens per second is fast enough that, once you count network round-trips, local inference can beat a cloud API call for short prompts. One way PrismML frames the achievement is intelligence density (benchmark score per gigabyte), where Bonsai lands around 0.53, roughly ten times a full-precision baseline.
Four ways to run Bonsai 27B right now
Because the weights are open, there is no single "install" path. Here are the four that work today, from zero-setup to full control.
On an iPhone
The Locally AI iOS app runs the 1-bit 3.9GB build directly on-device, no account or server required. This is the path that delivers on the "27B on a phone" promise, and it works fully offline once the weights are downloaded.
In a browser
PrismML publishes WebGPU kernels that run the 1-bit model inside a browser tab, with no install at all. It is the fastest way to try Bonsai 27B before committing disk space, though you need a machine with a WebGPU-capable GPU.
On your own machine
The GGUF, MLX, and ONNX weights are all on Hugging Face and GitHub under Apache 2.0. The main repositories are prism-ml/Bonsai-27B-gguf (1-bit) and prism-ml/Ternary-Bonsai-27B-gguf (ternary), with MLX 2-bit and ONNX builds alongside. Budget roughly 4GB of free storage and RAM for the 1-bit build and 6GB for ternary. One caveat: tooling maturity lags the release. The weights load in several runtimes, but full support in popular local apps like LM Studio was not yet in place at launch, so expect some manual setup.
Via a hosted API
If you would rather not manage weights, Together.ai serves the ternary build behind a standard API with the full 262K context window, vision input, and JSON mode. Input pricing was listed at $0.00 per million tokens during the launch promotion, so verify the current rate before you budget around it, since promotional pricing changes.
Bonsai 27B vs Gemma 4 12B QAT
The comparison that keeps coming up is Gemma 4 12B QAT, Google's quantization-aware-trained model, which lands around 7GB, only slightly larger than Bonsai's ternary build.
They win on different axes. Bonsai 27B clearly beats Gemma on math and coding benchmarks, thanks to that 27B base. Gemma tends to hold up better on vision and tool calling, and its slightly larger, less aggressively compressed weights make it a steadier general-purpose choice on a phone. If your on-device workload is reasoning and code, Bonsai is the stronger pick; if it leans on images or function calling, Gemma 4 12B QAT is the safer one.
Who should actually use Bonsai 27B
Use Bonsai 27B if you want offline, private, on-device reasoning or coding assistance and you have an iPhone 17 Pro-class device or a capable laptop. It is also a compelling object of study for anyone interested in extreme quantization: the fact that a 27B model runs in a browser tab is a real milestone, and the open Apache 2.0 license makes it easy to experiment with. It slots naturally alongside other open and free models worth running for programming when you want a local option.
Do not use it, yet, for production agentic systems that depend on reliable tool calling, for vision-critical tasks, or for anything where the reasoning-loop failure mode would be costly. For those, a less compressed model, or a full-precision one behind an API, remains the safer bet.
Frequently asked questions
Is Bonsai 27B free to use?
The weights are free under the Apache 2.0 license, so you can download and run it at no cost. Hosted access through Together.ai carries its own API pricing, which was listed at $0.00 per million input tokens during launch, so confirm the current rate.
Can Bonsai 27B really run on a phone?
Yes. The 1-bit build is 3.9GB and runs on an iPhone 17 Pro at about 11 tokens per second via the Locally AI app, fully offline once downloaded.
Is Bonsai 27B as good as the full Qwen3.6 27B?
Close, but not identical. The ternary build retains more than 95% of full-precision benchmark performance and the 1-bit build more than 90%, with the retention strongest on math and code and weakest on tool calling.
Which Bonsai 27B file should I download?
On a phone or a memory-tight machine, take the 1-bit build (prism-ml/Bonsai-27B-gguf, about 3.9GB). On a desktop or GPU where you have room to spare, take the ternary build (prism-ml/Ternary-Bonsai-27B-gguf, about 5.9GB) for the extra few points of accuracy. MLX 2-bit and ONNX variants exist for Apple Silicon and cross-platform runtimes.
What is ternary quantization?
It stores each model weight as one of three values (-1, 0, or +1) instead of a full-precision number, which is why the model shrinks so dramatically. FP16 scaling factors keep the collapsed weights at roughly the right magnitude.
Does Bonsai 27B support images?
Yes, it accepts image input alongside text and returns text output, inheriting the multimodal capability of its Qwen3.6 27B base. Vision quality holds up less well than math and code under the compression.