Kent Beck reflects on his fascination with how AI models are built, using a baking analogy to explain pre-training and post-training. Pre-training is described as a massive, expensive, one-shot batch process (akin to cold proofing dough) that produces raw, unusable model weights, while post-training is an iterative, collaborative series of small tweaks (the shaping and baking) that turns the raw model into something useful for humans. The piece is exploratory and speculative, framed as the author's attempt to understand the process, with a promise of a followup on the teams and roles involved. Ends with a promotional pitch for the author's advisory and speaking services.
Questions this post answers
What is the difference between pre-training and post-training in building an AI model?
Pre-training is a single massive batch process where a team sets initial conditions—data and a blank model—and runs the data through it repeatedly, costing hundreds of millions of dollars and months of time, producing a raw but unusable model. Post-training consists of many small, iterative experiments that researchers apply as targeted tweaks to fix specific weaknesses, turning the raw model into something usable for tasks like conversation. Developers tracking how LLMs actually get built can follow ongoing explainers like this on daily.dev.