Parameter count is a poor predictor of how well a local LLM performs as an AI agent. Docker's evaluation of 21 models across 3,570 tests showed that a Qwen3 14B scored 0.971 — nearly matching GPT-4's 0.974 — while Llama 3.3 70B scored only 0.607. Tool-calling reliability, not model size, is what determines agent usefulness. Models explicitly trained for tool calling outperform larger general-purpose models. A capability threshold exists around 7–9B parameters for general models, below which tool-calling degrades significantly. Fine-tuning can dramatically improve smaller models' tool-calling ability. Standard GGUF quantization generally preserves tool-calling performance, contrary to common fears.
Table of contents
Model size doesn't predict tool-calling abilityTool calling is what makes an agent an agentThe models that work, at every scaleQuantization isn't the problem you think it is84 Impressions