Bonsai 2: Can Smaller Local AI Keep My Existing Devices Useful?
![]() |
| AI-generated illustration for Routes, Bytes & Budgets. Generic devices and a concept interface; this is not a Bonsai screenshot or a photograph of a tested setup. |
My ThinkPad E15 has been working for me for four years, and it still handles my everyday work on Ubuntu well. A battery replacement and some thermal maintenance have kept it useful. That experience shapes how I look at new technology: I’m interested in what my existing equipment can do next.
That is why PrismML’s October 15 announcement of Bonsai 2 models in 2B, 4B and 9B sizes caught my attention. Smaller local AI could give devices we already own another useful job, without making a new purchase the starting point.
The promise is appealing. The practical questions are less glamorous: which software runs it, how much memory does the complete setup need, and is the result useful enough to justify the effort?
What makes Bonsai 2 smaller?
The three new models are based on Qwen3.5. PrismML uses a ternary representation, storing language-model weights around three values: −1, 0 and +1. This reduces the space needed for the model’s learned parameters compared with storing them at higher precision.
The official documentation describes text and image input, text responses and tool calling. Think of the model as a component for building an assistant. Downloading its weights does not automatically produce a finished application that understands your files, manages your calendar or safely controls your computer.
0.43 GB is the headline, not the whole installation
The most useful table in the documentation separates the compact language-model files from the Apple MLX packages that include vision:
| Model | PTQ1_0 GGUF: language-model weights | MLX package, including vision |
|---|---|---|
| Bonsai 2 2B | 0.43 GB | 1.34 GB |
| Bonsai 2 4B | 0.95 GB | 2.05 GB |
| Bonsai 2 9B | 2.00 GB | 3.46 GB |
These are published file sizes, not total memory requirements. The GGUF language-model files need a separate vision component for image input. For example, the 2B model page lists a recommended 0.36 GB vision file alongside its language-model downloads.
Running the model also requires working memory for processing, conversation state and other runtime allocations. The operating system and your open applications need room too. A model that fits on the SSD can still be uncomfortable to use when the computer runs short of RAM.
There is also installation overhead. PrismML’s quickstart explains that its optional web interface and code-interpreter environment add several gigabytes. On Apple Silicon, the setup can download both GGUF and MLX packages. Checking the complete download plan matters more than admiring one small number.
Compatibility comes before speed
Bonsai offers GGUF files for llama.cpp-based software and MLX packages for Apple Silicon. PrismML documents CPU and several GPU backends, but support depends on the particular file, runtime and device.
The format and runtime guide contains an especially useful warning: the smallest PTQ1_0 files currently need PrismML’s binaries. The more widely compatible Q2_0 version needs upstream llama.cpp build b11555 or newer. For the 2B model, that Q2_0 file is 0.55 GB.
In other words, “my app opens GGUF files” is not enough. An application may bundle an older engine. Likewise, an MLX file needs a loader that understands Bonsai 2, rather than any application with MLX somewhere in its description.
For an older laptop, I’d begin with the smallest suitable model and a short task. That gives a much clearer answer than assuming the largest downloadable version must be the best choice. My existing travel setup works because it remains dependable; an assistant should earn its place in that routine.
What the performance claims do—and don’t—tell us
PrismML reports up to five times the throughput and three times the energy efficiency of the corresponding full-precision models on specified hardware. Its mobile examples include an iPhone 18 Pro Max, with claimed speeds of up to 120 tokens per second for the 2B model.
Those are the developer’s results on particular configurations. They do not establish that an older Android phone, a budget tablet or my work laptop will deliver the same experience.
There are capability trade-offs too. PrismML’s own benchmark table shows that retaining performance varies by task; compactness does not guarantee an identical answer to the larger, higher-precision baseline.
For everyday use, the questions I care about are more concrete: does it summarize a short document accurately, follow a simple instruction, and respond without making the rest of the machine sluggish? On a phone, heat and battery drain belong in that assessment as well.
Local processing gives privacy options, not automatic privacy
With the model and inference software running locally, generating a response can happen on your own device. That opens up useful possibilities for working with personal notes or documents without sending their contents to a hosted model.
But a local model can be part of a connected application. Web search sends a query somewhere. Cloud storage synchronization moves files. An email or calendar integration communicates with its provider. The model being local does not make those other steps disappear.
PrismML’s local server documentation describes the endpoint that applications can connect to. The surrounding application, its tools and its permissions still determine what happens to the information you provide.
This is the same distinction I discussed in my post about personal AI agents and control over our information. A sensible first trial uses a non-sensitive sample document and no connected tools. Extra access can come later, when there is a clear reason for it.
A small experiment before a hardware upgrade
My starting plan is straightforward:
- Choose one useful task. A short summary or turning a paragraph into a checklist is easier to evaluate than a vaguely defined “personal agent.”
- Check the exact runtime and model package. Include the vision component only if the task needs images, and leave memory available for normal work.
- Use material whose answer I can check. Fluency matters less than whether the result is correct and complete.
- Watch the whole experience. Response time, memory pressure, fan noise and battery use all affect whether the tool is worth keeping.
- Keep spending at zero initially. If the setup is unsuitable, that is useful information about this task and this device—not an automatic reason to replace the computer.
The model weights ship under Apache 2.0, according to PrismML’s documentation. That is helpful for experimentation, although the applications and connected services used alongside them can have separate terms and costs.
What interests me about Bonsai 2 is the possibility of fitting a useful tool into the equipment I already trust. The best outcome is quite ordinary: the same laptop, the same familiar workflow, and one more task it can handle well.
.png)
Comments
Post a Comment