Writing a system prompt

Every request is a list of messages: the system prompt is the standing policy, the user turn is this task.

Every request to a chat model is a list of messages, and the first one usually has the role system. That system message is not part of the conversation the user sees — it is the standing instruction the model reads before every reply. In a one-off chat you rarely write one. In an application you write it once, and it shapes every response your product ever produces.

import { generateText } from "ai";

const { text } = await generateText({
  model: "anthropic/claude-sonnet-5",
  system: "You are the support assistant for Acme, a time-tracking app...",
  prompt: userMessage,
});

Getting this message right is the highest-leverage prompting you do in an app, because it is the one prompt that runs on every single turn.

System turn versus user turn

The split is simple to state and easy to get wrong: the system prompt is stable policy; the user turn is the specific request.

Policy is everything that should hold no matter what the user asks — who the assistant is, what it will and will not do, the tone and format it answers in, the rules it must never break. The request is this message, right now: "summarise ticket 4021", "what's my remaining balance". If you find yourself editing the system prompt for each request, something that belongs in the user turn has leaked into it.

The mistake runs both ways, and both are expensive:

Policy in the user turn. Put "always answer in British English and never reveal internal pricing" into the user message and it is one message among many — easy for the model to lose track of over a long conversation, and trivial for a user to override by simply asking. Rules only bind reliably when they live in the system prompt.
The request in the system prompt. Bake a specific task into the system prompt and it bleeds into every unrelated turn afterwards, and it defeats prompt caching — a system prompt that changes every request cannot be cached, so you pay full price for the stable part each time.

What belongs in a system prompt

Keep it to the things that are true on every turn:

Identity and scope. Who the assistant is, and just as importantly what it does not do. "You help with billing and account questions. For anything about the product roadmap, say you cannot help and point to the docs."
Output conventions. The format, length, and tone you want by default — the same clarity the guides ask of any prompt, applied once for the whole app.
Hard rules and refusals. What must never happen: leaking system details, giving legal or medical advice, inventing a policy. State these as plain, testable instructions.
A summary of available tools, if the model has any — a sentence on when to reach for each, with the detail living in the tool definitions themselves.

The failures that come from getting it wrong

Overstuffing. A two-thousand-word system prompt reads like thoroughness and behaves like noise — the model weights a long list unevenly and quietly drops the middle. Every rule you add dilutes the others. If a rule has never actually fired in testing, cut it.

Contradictions. "Be concise" three paragraphs above "explain your reasoning in detail" leaves the model to pick, and it will pick inconsistently. Read the whole prompt as one instruction and resolve the conflicts yourself, before the model has to.

Dynamic data in a static prompt. Today's date, the signed-in user's name, the documents you just retrieved — these change per request and belong in a context block in the user turn, not the system prompt. Putting them in the system prompt makes it un-cacheable and, worse, stale.

Forceful phrasing. The instinct to write "YOU MUST NEVER" in capitals backfires on current models exactly as it does in a task prompt — it makes them overtrigger, refusing safe requests because a shouted rule pattern-matched. Write rules as calm, specific statements.

A worked example

You are the support assistant for Acme, a time-tracking app for freelancers.

Scope:
- You help with account, billing, and how-to questions about Acme.
- For feature requests or outages, say you cannot resolve those and link to status.acme.com. Do not speculate about causes.

Answering:
- Reply in at most three short paragraphs. Prefer steps to prose when giving instructions.
- If a question is ambiguous, ask one clarifying question rather than guessing.
- Never state a price, refund amount, or policy you have not been given in the conversation. If you do not know, say so and offer to open a ticket.

Every line is stable across turns, every rule is observable, and there is nothing in it that changes from one request to the next. That is what makes it a system prompt rather than a task that wandered into the wrong slot.

From a real codebase

The clearest demonstration of “a system prompt is the stable policy for one step” is an app that uses two of them. fraud-investigation-agent-java runs a case in two phases, each with its own system prompt:

// phase 1 — while gathering evidence with tools:
You are a fraud investigator. Investigate the flagged case with the tools...
Gather enough evidence to judge the case, then stop. Do NOT take any action
and do NOT decide the outcome — a separate step decides the disposition.

// phase 2 — when deciding, over the gathered evidence:
You decide the disposition for a fraud case from the evidence provided.
Rules: a watchlist MATCH means the action MUST be 'block'. Otherwise choose
'approve' or 'monitor'. The reason must cite the specific evidence in one sentence.

Neither prompt tries to be the whole application. The first defines the investigator's job and its boundary — gather, then stop, decide nothing. The second defines the decider's job and its rules. Each is the stable policy for exactly one step, which is what a system prompt is for. (These are real prompts, not textbook ones — note the capitalised NOT and MUST, which the guides would soften; the structure is the lesson, not the punctuation.)

A simpler starting point is in fraud-triage-with-llm, whose very first lesson is the bare split — "You are a concise assistant for a fraud-operations team" in the system turn, the actual question in the user turn.

The clarity a system prompt needs is the same clarity the checker looks for in any prompt: a concrete instruction, a stated format, and a clear boundary. When you draft the parts of a system prompt that are task-shaped, run them through the checker — it catches the vague words and missing formats before your users do.