JetBrains details the engineering behind running its Junie coding agent fully locally on a MacBook M5 using Qwen3.6-27B instead of Qwen3.8-27B. Optimizations span the agent harness (rolling context reuse, KV-cache-friendly prompt ordering, disabling optional LLM calls and multi-agent mode), model parameters (disabling reasoning, using 4-bit quantization), and the inference engine (an 8-bit prefill patch to MLX-VLM yielding ~40% prefill speedup on M5's 8-bit arithmetic instructions, plus MTP and n-gram speculative decoding for up to 2x generation speedup). Qwen3.8-27B was rejected because it requires reasoning mode, which produces ~5x more tokens and a ~4x net slowdown, making Qwen3.6-27B the better choice on Mac hardware for now.
Table of contents
How We Optimized the Qwen 3.6 Model for Our Junie AgentJunie optimizationsModel parameter optimizationsInference engine optimizationQwen3.8-27bQuestions this post answers
Why did JetBrains choose Qwen3.6-27B over Qwen3.8-27B for local Junie on Mac hardware?
Qwen3.8-27B requires reasoning mode to function well, and without it output quality degrades severely, sometimes getting stuck repeating the same tool call. Enabling reasoning at medium effort produces roughly 5x more tokens, and since prefill time stays constant, the net slowdown is about 4x, making Qwen3.6-27B the faster, more practical choice on Mac hardware. Developers weighing local model trade-offs for coding agents can track real-world comparisons like this on daily.dev.
How much faster is prefill on Apple M5 versus M4 chips for local LLM inference?
Apple M5 chips include 8-bit arithmetic instructions that M4 chips lack, and using them for matrix operations during prefill gave roughly a 40% prefill speed gain when patched into the MLX-VLM inference engine. Without those instructions, M4's 16-bit-only arithmetic delivers 20-30% slower prefill than M5, which is why M5 was chosen as the initial local inference target. Anyone benchmarking Apple silicon for on-device inference can follow hardware-specific findings like this on daily.dev.
What speedup does disabling reasoning mode give when running Qwen3.6-27B locally?
Disabling reasoning on Qwen3.6-27B results in generating 2-3x fewer tokens, which translates into roughly a 2x speedup in task execution, with only an insignificant effect on output quality based on internal testing of the cloud version of the model. Teams tuning local model configs for speed versus quality can compare notes like this on daily.dev.