Essay

Why AI plans fail: good at advice, bad at arithmetic

I asked a very capable model to build a 45-week HYROX training plan. It came back with 92% of the runway in the base phase. Not wrong in a way you could argue with — wrong in a way that would have wasted ten months of somebody's training.

The instinctive fix is a better prompt. It doesn't work, and the reason it doesn't work is the interesting part.

What actually went wrong

The model wasn't reasoning badly about training. Ask it in prose how to periodise a 45-week block and it gives a genuinely good answer: a long aerobic base, a build phase with station work, a peak, then a taper into race day. It knows the theory. It has read more coaching literature than any coach.

The failure was that I asked it to do the arithmetic in prose, inside a 300-line JSON document. Somewhere around week 30 of emitting dated objects, the structure drifted. There was no moment of bad judgement to point at. There was a long document and a slow loss of the plot.

This is the thing worth internalising: dividing 45 weeks into phases at roughly 35/40/25 plus a taper is not reasoning. It is division. And a language model doing division across a long generation is doing the one thing it is worst at, in the place you are least likely to check.

Why better prompting doesn't fix it

I tried. You can state the phase proportions, restate them, give worked examples, and add a critic pass that re-reads the output. All of it helps a little and none of it holds, because you are still asking for a property of the whole document to be maintained token by token, with no mechanism that can fail loudly when it isn't.

A rule that lives in a prompt is a request. A rule that lives in code is a guarantee. Once you say that out loud, the fix is obvious.

The fix: ordering is the algorithm

Phase boundaries are now computed by a pure function whose insides the model never sees. What makes it correct is not cleverness — it is the order the decisions happen in:

1. The taper is carved off the end first. It is never what gets squeezed. Under time pressure everyone shortens the taper, and the taper has some of the strongest evidence behind it of anything in the plan.
2. The peak lands immediately before the taper — never later. Peaking into race week is a mistake that reads as reasonable right up until it ruins the race.
3. Whatever runway remains is split proportionally between the earlier phases, using largest-remainder allocation so the weeks stay whole numbers that still add up.
4. Every Nth week is a cutback, roughly 30% down, so the training gets absorbed rather than accumulated.
5. Week one is deliberately light. Early wins compound; early failures get abandoned.

The same 45-week plan now comes out 34/40/24 plus taper. It cannot come out any other way, because there is no path through the code that produces a different answer.

The second bug, which the first fix exposed

With the phases correct, an eight-week marathon plan cheerfully interpolated its long run up to 32km. The structure was right and the content was dangerous.

Interpolating between "where you are" and "where you need to be" is arithmetic too, and it has a constraint the model never applied: you cannot safely add much more than about 10% of volume per week. The ladder is now capped at that, every week. When the runway is genuinely too short, the honest answer is returned instead of a plan:

A tool that won't tell you that isn't being helpful. It's being agreeable.

Three failures that made no noise at all

These cost me more time than the periodisation bug, because a wrong plan at least announces itself.

The fallback that hid the outage. There is a secondary model configured for when the primary is unavailable — sensible, and it kept the product up. Then plan quality quietly degraded. The primary had started returning HTTP 400 because temperature is deprecated on that model, every request was silently falling back, and nothing surfaced it. I only noticed because responses carry an engine_used field and one of them read "fell back". A fallback that protects your users also hides your failures. Report which path served each request, or you are flying blind by design.

Truncation that looked exactly like a format error. Responses came back as unparseable JSON, so the retry loop dutifully retried and burned the budget. The output was not malformed — it was cut off. Extended thinking was consuming the token allowance before the answer began, so generation stopped at the cap having emitted about 1,900 characters of a much longer document. The tell was stop=max_tokens, which I was not logging. I now log the stop reason, the provider and the character count on every failed parse, because "invalid JSON" is a symptom shared by at least three unrelated diseases.

The sixty-second wall. Generating a whole plan in one request exceeded the platform's request timeout. No amount of optimisation was going to fit it, so the work is split across three requests that each do less. Some limits are not performance problems.

What the model is actually for

None of this is an argument against using language models here. It is an argument about where the seam goes.

The model does the part that is genuinely hard for code: understanding that you have a toddler and can only train at 5am, that your longest run so far was 12km and you described it as nearly killing you, that this is the third attempt at the exam and the confidence matters as much as the syllabus. It writes the session that says "6 × 400m at 5k effort, 90s jog" rather than "intervals". It knows what a sled push does to your legs and what not to schedule the day after.

What it never does is emit the calendar. It proposes; the compiler disposes. Every plan is validated before it reaches a person, and the validator rejects things the model has no business getting wrong: a plan with no sessions, a plan with nothing to tick, a metric moving faster than is safe.

What it cost, in case it saves you the same

• A working engine already existed in the browser, and the AI path ignored it and re-derived the same logic, badly, in prose. Shared code beats shared instructions.
• Retries are ruinous when a stage takes thirty seconds. The retry budget is now per-stage, and small.
• A validator that passes a calendar with zero sessions is not a validator. Test what "valid but useless" looks like, not just what invalid looks like.
• Log the stop reason. Log which provider served it. You cannot debug what you cannot distinguish.

More: the research behind the product · why the taper is non-negotiable · 49 personal development books, honestly rated