How to Optimize Your Prompts for Better Results, Lower Cost, and Faster Responses

Step 1: Measure Your Baseline

Before you touch a single word of your prompt, establish a quantitative baseline. You need three numbers: a quality score, a cost metric, and a latency metric. Without these, you cannot tell whether your optimizations are helping or hurting.

For quality, run your prompt against a test suite of at least 30 representative inputs and score the outputs. The scoring method depends on the task: accuracy percentage for classification, schema compliance rate for structured output, rubric scores for open-ended generation, or a combination. Record the average score and the distribution, because a prompt that averages 85 percent but drops to 40 percent on certain input types has a different problem than one that averages 80 percent consistently.

For cost, count the total tokens in and out for each test case. Input tokens include the system prompt, any few-shot examples, the user message, and any retrieved context. Output tokens are the model's response. Calculate the cost per call using your provider's pricing. For Anthropic Claude, input tokens cost $3 per million for Sonnet and $15 per million for Opus (as of mid-2026). For OpenAI, GPT-4o is $2.50 per million input tokens. These numbers define the unit economics of your application, and small per-call savings compound into significant monthly savings at scale.

For latency, measure time-to-first-token (TTFT) and total response time. TTFT determines how quickly the user sees something happening. Total time determines how long they wait for the complete answer. Log both for each test case, because optimizations that reduce total time but increase TTFT (or vice versa) have different UX implications depending on whether you stream responses.

Step 2: Remove Redundant Instructions

Most prompts accumulate instructions over time as practitioners add rules to fix specific failures. After several rounds of iteration, the prompt contains instructions that duplicate each other, rules that address scenarios that never occur, and guidelines that describe the model's default behavior. Removing these redundancies is the easiest, highest-impact optimization.

Start by listing every instruction in your prompt as a separate line item. For each one, ask three questions. First, does this instruction change the model's behavior, or would the model do this anyway? Instructions like "be accurate" or "respond in English" (when the input is already English) are wasted tokens. Remove them and re-run your test suite. If scores are unchanged, the instruction was redundant. Second, does another instruction already cover this case? If you have both "always respond in JSON" and "format all output as valid JSON objects," one of them is redundant. Keep the more specific one. Third, does this instruction address a failure that actually occurs? If you added a rule six months ago to fix a specific edge case and that edge case has not appeared since (or the model has improved and no longer makes that error), the instruction may no longer be necessary.

A systematic redundancy audit typically removes 15 to 25 percent of a prompt's tokens without affecting output quality. On a 2,000-token system prompt running 50,000 calls per day, removing 400 tokens saves roughly $60 per day on Claude Sonnet, which is $1,800 per month. The savings scale linearly with call volume.

Step 3: Compress Without Losing Meaning

After removing redundancies, the remaining instructions are all necessary, but they may not be expressed efficiently. Compression means rewriting instructions to communicate the same rule in fewer tokens while preserving the model's ability to follow them.

Replace verbose explanations with concise directives. "When the user provides a question that seems ambiguous and could be interpreted in multiple ways, you should ask a clarifying question to determine which interpretation the user intended before proceeding with your response" compresses to "If the query is ambiguous, ask which interpretation the user means before answering." Both convey the same rule; the second uses 60 percent fewer tokens.

Use formatting to replace prose. A rule expressed as a paragraph can often be expressed as a short list or a simple if/then. "You handle three types of requests: billing questions where you look up the account, technical issues where you check the knowledge base, and feature requests where you log them in the tracker" compresses to a three-item list that the model parses just as well. Lists are often both shorter and more reliably followed than paragraphs, because the model can see each rule as a distinct item rather than parsing it out of flowing text.

Abbreviations and shorthand work when the meaning is unambiguous in context. "resp" for response, "req" for request, "max" for maximum are all understood by the model. However, do not compress to the point of ambiguity. If a compressed instruction could be interpreted in two ways, the tokens you saved will cost you far more in debugging and error correction. The goal is density, not obscurity.

Test compression aggressively. After each round of compression, re-run the full test suite. If scores drop, the compression went too far on some instruction and you need to restore clarity on that specific point. Compression should never be a net negative on quality; if it is, you have compressed the wrong things or compressed too aggressively.

Step 4: Optimize Your Examples

Few-shot examples are the most expensive component of many prompts, often consuming 500 to 1,500 tokens. Optimizing examples means achieving the same or better model behavior with fewer, better-chosen examples.

First, determine whether you need examples at all. Run your prompt with zero examples and measure the quality score. For tasks where the model has strong prior knowledge (summarization, translation, simple classification), zero-shot performance may be adequate, saving you all the example tokens. If quality drops significantly without examples, you need them, but probably fewer than you think.

Second, test whether one carefully chosen example performs as well as three mediocre ones. A single example that demonstrates the exact input/output pattern you want, including any tricky formatting rules, often anchors the model's behavior as effectively as three examples that partially overlap. Try one example, measure, add a second only if the first is insufficient, and add a third only if two are insufficient. Most teams run three to five examples by default without ever testing whether one or two would suffice.

Third, optimize the examples themselves. Trim unnecessary detail from the example inputs and outputs. If the example exists to demonstrate a JSON output format, the input does not need to be a realistic 200-word customer email; a 30-word synthetic input that triggers the same response pattern works just as well and costs 170 fewer tokens per call. Similarly, if the example output demonstrates a structure, use short placeholder values rather than realistic paragraphs of text.

Fourth, consider dynamic example selection instead of static examples. Rather than including the same examples in every call, retrieve the example most similar to the current input from a pool. This gives the model a more relevant demonstration while potentially using fewer examples per call. Dynamic selection adds complexity to the system but provides better results on diverse inputs.

Step 5: Apply Caching and Batching

Prompt caching is the single largest cost reduction available for applications with long, stable system prompts. Both Anthropic and OpenAI offer caching mechanisms that store the processed representation of your system prompt between calls, so you pay the full input processing cost only once and a reduced cost on subsequent calls that reuse the same prefix.

Anthropic's prompt caching reduces the cost of cached tokens to 10 percent of the normal input price. For a 3,000-token system prompt at $3 per million tokens, that is $9 per day for 100,000 calls without caching versus $0.90 per day with caching, an order-of-magnitude savings. The requirement is that the cached prefix must be identical across calls, character for character. This means your system prompt, tool descriptions, and any static few-shot examples must be exactly the same on every call. Dynamic content (the user message, retrieved context) goes after the cached prefix.

Structure your prompt to maximize the cacheable prefix. Put all fixed content at the beginning: system prompt, tool descriptions, static examples, output format rules. Put all variable content at the end: the user's input, retrieved documents, conversation history. The more tokens you can make static and cacheable, the larger the savings. Some teams redesign their prompt architecture specifically to maximize the cacheable portion, and the effort pays for itself within days at high call volumes.

Batching aggregates multiple requests into a single API call for workloads that are not latency-sensitive. Anthropic's Batches API processes requests at 50 percent of the standard price in exchange for a 24-hour processing window. If you are processing documents, generating reports, or running evaluations in bulk, batching cuts your cost in half with no impact on quality. The tradeoff is latency: batch results are not real-time, so batching is for background processing, not interactive applications.

Step 6: Match Model to Task Difficulty

The most over-looked optimization is model selection. Many teams default to the largest, most expensive model for every task, even when a smaller model would produce identical results. The price difference between model tiers is substantial: Claude Opus costs 5x more than Claude Sonnet per token, and Sonnet costs roughly 20x more than Haiku. Routing easy tasks to cheap models and hard tasks to expensive models can reduce overall costs by 40 to 60 percent without any quality loss on the tasks that matter.

Build a routing classifier that evaluates each incoming request and assigns it to the appropriate model tier. The classifier can be rule-based (short queries go to Haiku, complex multi-step queries go to Sonnet, ambiguous edge cases go to Opus) or model-based (a lightweight model evaluates request complexity and routes accordingly). The classifier itself runs on the cheapest model available, so its cost is negligible compared to the savings it produces.

Profile your traffic to understand the distribution. Most applications follow a power law: 70 percent of requests are straightforward and can be handled by the smallest model, 25 percent are moderate and need a mid-tier model, and 5 percent are complex and genuinely need the top-tier model. If you are currently running everything on the top tier, routing just the easy 70 percent to a cheap model cuts your bill by more than half.

Validate routing decisions with quality checks. Periodically sample outputs from each model tier and compare them to what the top-tier model would have produced. If the cheap model's outputs are indistinguishable from the expensive model's on routed tasks, the routing is working. If you find quality gaps, adjust the routing thresholds to send fewer requests to the cheap model. The goal is not to minimize cost unconditionally; it is to find the cheapest model that meets your quality bar for each request type.