Structured output and retries

Already validating model JSON and retrying with a repair message? Skip to the next lesson.

Models return text. Sometimes that text is supposed to be data, and sometimes it is broken. This lesson is about handling that without either shipping rubbish or hanging forever.

The idea

You often want the model to fill in a form rather than write a paragraph. You ask for JSON, the structured format from lesson 0.4, so your code can read the answer.

The model will usually comply. Usually is the problem. Sooner or later you get a stray sentence before the JSON, a missing field, or a value you did not expect. Feed that straight into a tool and you have written nonsense into a real system.

So you do three things:

1. Validate. Try to read it. Check the fields you need are actually there.
2. Repair. If it is wrong, send it back and say what was wrong. Models are good at fixing their own output when told specifically.
3. Cap the retries. Without a limit, a model that keeps getting it wrong keeps being asked, and your program hangs. MAX_RETRIES is the seatbelt.

Only once the data is valid do you call the tool.

What to write

Keep the Module 6 to 8 settings correct. Then:

1. parse_ticket(text): read the JSON, require priority, category and summary, and check priority is high or low. Return the data, or nothing if it is bad.
2. MAX_RETRIES = 3
3. The loop: call, parse, on failure add a repair message and try again, on success call file_ticket, feed the result back, then ask once more for the confirmation.

The point

Assume the model will get the format wrong sometimes, and design for it. The retry budget is what separates a robust agent from one that hangs.

Break it on purpose

Remove the repair step and exit on the first reply. The check fails with no_retry. Then remove MAX_RETRIES and imagine that running unattended.