THE LAB · FIELD NOTE 03

The training shows through

Prompt advice looks like folklore — roles, examples, magic phrasings, review steps. It isn't. Every practice that survives contact with real work is a patch for a specific property of how the model was made. Learn the making, and you can derive the practices instead of memorizing them.

Andriy Kudelya · 2026-07-21 · 9 min

There is a reason prompt advice feels arbitrary: most of it is handed down as tips, stripped of the mechanism that makes each tip work. So people collect incantations, the incantations age, and the whole discipline gets dismissed as vibes. The fix is to start one layer down. A language model is a manufactured thing, built in a short sequence of steps — and each step leaves marks on its behavior that no later step removes. Once you can name the marks, the practices stop being folklore and start being obvious.

Step one: a machine that continues

The first and by far the largest step is pretraining. Take trillions of words — a filtered copy of much of the public web, books, code — and train one network on a single, monotonous exercise: given the text so far, guess the next small chunk (a token — roughly three-quarters of a word). Guess, get corrected, adjust, repeat — on the scale of trillions of guesses.

To get good at that game across law, poetry, Python, and shopping lists, the network is forced to compress real structure: grammar, facts, styles, arguments, the shape of a good proof and of a bad excuse. That's where the capability lives. But notice what the exercise actually rewards: continuation. The result of pretraining — the base model — doesn't answer questions, doesn't chat, and has no goals. It is an autocomplete with a worldview: hand it half a contract and it writes more contract; hand it half a conspiracy theory and it obliges with the other half.

A base model has no concept of “true,” only of “likely to come next.” Everything you will ever dislike about a chatbot's confidence starts here.

Step two: turning autocomplete into an assistant

The assistant character is installed afterwards, in tuning passes that are tiny compared to pretraining but decide the personality:

Instruction tuning. The base model is shown many thousands of worked examples of the assistant pattern — a request, then a good response. It learns the form: when text looks like a question, continue it the way a helpful answer would.
Preference tuning. The model produces multiple answers; humans (and increasingly, other models applying written rules) pick the better one; the model is adjusted toward what wins. Run at scale, this makes it helpful, polite, and safe — by training it toward approval.
Reasoning training. The newest pass: on problems with checkable answers — math, code, logic — the model is rewarded for working step by step toward verifiably correct results. This is what “thinking” modes are made of.

The making of an assistant. Capability comes from the giant first step; personality — and its side effects — from the small later ones. No step installs a concept of truth.

Preference tuning is where the most consequential scars come from, and they're well documented. Train toward approval and you get sycophancy: models measurably agree with their user and soften criticism, because agreement wins votes. Grade models on having answers and a confident wrong answer outscores an honest “I don't know” — OpenAI's own analysis of hallucination makes exactly this argument: models guess because the way we score them rewards guessing. And because raters tend to prefer thorough-looking responses, length quietly wins too — which is why models pad.

What happens when you press send

One more fact completes the picture. At the moment you use a model, its weights are frozen. It learns nothing from your conversation. Everything it will consider — your instructions, your material, the chat so far — must fit in the context window, the model's entire world for this one run. It reads that world and produces the most plausible continuation, in the persona the tuning installed. That's the whole transaction.

The runtime transaction. There is no memory beyond the window and no learning during use — which is why “it knew this yesterday” is never a real property of the model.

The practices, traced

Now the folklore resolves. Here is the ledger — on the left, a fact about how the machine was made; on the right, the prompt practice that exists because of it:

HOW IT WAS MADEWHY THE PRACTICE EXISTS

It continues text. The prompt isn't a query into a database — it's the opening of a document the model will finish.

Role framing works — “you are a qualitative researcher synthesizing interviews” selects which patterns continue. Competence framing, not flattery: “world's best” selects the sound of expertise, not its habits.

It learned everything from worked examples — millions of them, during pretraining. Showing beats telling; it always has.

One to three canonical examples beat ten abstract rules when format or judgment is ambiguous. You're not explaining — you're letting it lock onto a pattern.

Attention thins in the middle of long inputs — models measurably use the start and end of the window better than the interior. The effect has shrunk with each generation, and keeps reappearing on harder tasks.

Long material first, the ask last, with clear section markers. The layout rule that has survived every generation so far.

It was tuned toward approval. Agreement won votes; so did softened criticism. It flatters your draft — and grades its own work kindly.

Fresh-context review: quality checks run by a reviewer that sees only the artifact and a rubric — never the conversation that produced it. Never ask the author to grade the author.

It was graded on having answers. A confident guess outscored an honest “I don't know” in most of its training, so it guesses fluently.

Make honesty a legal move: demand citations to the provided material, and make “not in the documents” a first-class answer instead of a failure.

Longer looked better to raters, often enough that verbosity got baked in. Thin weeks still produce thick reports.

Length budgets are ceilings, never floors — and every recurring digest needs the clause: “if nothing cleared the bar, say so in one line.”

Every run starts from amnesia. Frozen weights, fresh window: it doesn't remember your last session unless the text is put back in front of it.

Self-contained prompts and explicit handoffs — each step names exactly what it receives; state lives in artifacts you pass forward, not in imagined memory.

In its training data, shouting rarely marks quality. ALL-CAPS and stacked MUSTs are rare in the documents worth imitating — so in practice they read as noise, or over-trigger. (This row is field observation, not a cited paper.)

One precise, calm instruction, stated once — with the reason attached — beats shouting. Specific heuristics, not slogans.

Prompt engineering is not learning magic words. It is engineering against a machine whose manufacturing defects are documented.

This is also why practices age. “Take a deep breath” and its cousins were real, measurable tricks on earlier generations — artifacts of thinner tuning — and the better the tuning gets, the less such incantations matter, while everything in the ledger above keeps mattering, because it patches the architecture of the making, not a passing quirk. Advice tied to quirks expires. Advice tied to the pipeline compounds.

Lab note: this ledger is the theory Prowrit runs on. Every workflow the studio engineers applies these patches by default — roles as competence framing, examples only where ambiguity is real, material-first layout, fresh-context gates with fix-lists, citation contracts, ceilings not floors, self-contained steps. Paste a brief and watch which patches your task needs.

Sources & further reading: in-context learning — Brown et al., “Language Models are Few-Shot Learners” (2020) · the tuning recipe — Ouyang et al., “Training language models to follow instructions with human feedback” (2022) · position effects — Liu et al., “Lost in the Middle” (2023) · approval side effects — Sharma et al., “Towards Understanding Sycophancy in Language Models” (2023) · length bias in preference tuning — Singhal et al., “A Long Way to Go: Investigating Length Correlations in RLHF” (2023) · why grading rewards guessing — Kalai et al., “Why Language Models Hallucinate” (OpenAI, 2025). Simplifications are ours; the mechanisms are theirs.