5 Effective Techniques for Optimizing Prompts to Enhance LLM Output

If you already have a prompt that produces okay results from an LLM but you want better ones, this article is for you. Rather than starting from a blank page, you'll learn five specific, evidence-backed changes you can make to an existing prompt — each demonstrated on a real, messy meeting transcript that needs to be turned into a clean list of action items.

Engineering vs. Optimization: Why the Difference Matters

People often use "prompt engineering" and "prompt optimization" as if they mean the same thing, but they don't. Prompt engineering is designing a prompt from scratch. Prompt optimization is refining a prompt you already have — through specificity, structure, and iteration — without changing the model itself.

That distinction matters because most people who ask "how do I get better output from this LLM?" already have a working prompt. What they need isn't a blank-page framework; it's knowledge of which concrete edits actually move the needle and which ones just feel like they should.

Here's the running example: a meeting transcript with three participants, some mid-conversation confusion, and a known-correct answer we can measure every strategy against.

Priya: Okay so first thing, the checkout redesign. Where are we.

Tom: Mostly done, I just need someone to review the mobile layout before Friday.

Priya: I can do that. Actually wait, Jake said he'd look at it, let's leave it with him.

Jake: Yeah I can take the mobile review, I'll get to it by Thursday.

Tom: Cool. Second thing, we said last week we'd migrate the billing service to the new queue, but honestly I think we should hold off, the queue library had a security patch yesterday and I haven't read the changelog yet.

Priya: Agreed, let's not touch billing until that's reviewed. Tom, can you read through the changelog and flag anything concerning?

Tom: Sure, I'll do that tomorrow morning.

Jake: Also, sorry to jump in, but the support queue is getting bad again, we're at like 40 open tickets. Someone needs to triage that this week or it's going to snowball.

Priya: Yeah that's fair. I don't think it should be Tom or Jake given what's already on their plate. I'll pull someone from the support rotation, I just need to check who's free.

Tom: One more thing actually, going back to the mobile review, Jake, can you also check the tablet breakpoint while you're in there? We got a complaint about it last week.

Jake: Sure, I'll fold that into the same review.

Three details make this genuinely tricky, not just long. First, the mobile review gets reassigned mid-conversation from Priya to Jake. Second, the tablet-breakpoint check gets folded into that same review instead of becoming a separate item. Third, the support-queue triage owner is explicitly left unresolved — it shouldn't be dropped or guessed at. A prompt that nails the easy parts but stumbles on these three details isn't actually working, even if the output looks fine at a glance.

1. Specify a Structured Output Format

This is the most measurable lever available, and the easiest to prove isn't cosmetic. Asking a model to "list the action items" produces fluent, readable text — but not something a downstream system can reliably parse. In production, unparseable output isn't a minor annoyance; it's a hard failure.

When tested against the transcript, a plain-prose response ("Here's what I found from the meeting: 1. Jake will review the mobile layout by Thursday...") failed to parse entirely — a `parse_structured_output` function returned `None` with a validation error, because prose isn't JSON no matter how well organized it looks. The same information, requested with an explicit schema, parsed cleanly into three validated `ActionItem` objects.

That's the real payoff of structured-output prompting: not prettier text, but output your code can actually use instead of output that requires a human to re-read and transcribe by hand.

2. Assign a Role or Persona

Giving the model a specific role activates a different part of its training, producing more structured, context-aware output than a generic instruction alone. It's a small change with a real effect, and it costs nothing to try.

Before:

Extract the action items from this meeting transcript.

After:

You are a meticulous executive assistant who has sat through hundreds of these meetings. You know that people change their minds mid-sentence, that assignments get reassigned, and that a good notes-taker never guesses at an owner who wasn't actually confirmed. Extract the action items from this meeting transcript.

With the generic version, the model has no reason to watch specifically for a mid-conversation reassignment or an unresolved owner — nothing flagged those as things to look for. The role-based version primes it to expect that kind of ambiguity before it starts reading. That matters most on transcripts messy enough that a careless first pass would miss it — exactly the kind we're using here.

3. Choose Few-Shot Examples Carefully

A well-known synthesis of prompt-optimization research found something worth taking seriously: how you select demonstrations can matter more for output quality than how you word the instructions, and combining the two deliberately beats either one alone. The detail most people miss is that it's not "add a few examples" — it's which examples. A set that accidentally contains three variations of the same pattern teaches the model almost nothing new.

In one test, a candidate pool deliberately included a near-duplicate pair — two examples both following the identical "owner confirms a deadline, high priority" pattern. Naively grabbing the first three candidates pulled in both duplicates, wasting two of three demonstration slots on the same lesson. Diversity-aware selection caught the pair (the two most similar examples in the whole set) and swapped one for a genuinely different pattern.

Applied to our task, a worthwhile few-shot set should include: one example with a confirmed owner, one with an explicitly unresolved owner, and one where an item gets merged into an earlier one. Three different real patterns — not three restatements of the easy case.

4. Prompt for Chain-of-Thought Reasoning

Chain-of-thought prompting — asking a model to reason step by step before answering — is still genuinely useful, but its role has shifted. Frontier models now reason natively, so explicitly requesting step-by-step reasoning matters less for them than it did in 2022 and 2023, when the original research showed dramatic gains on models that couldn't. Where it still earns its keep is on genuinely ambiguous cases — and this transcript has one: the mobile-review reassignment.

Without reasoning prompted, a model can latch onto the first mention — Priya's "I can do that" — and miss the correction two lines later. With reasoning prompted:

Before extracting each action item, first trace who was assigned across the whole conversation, since assignments sometimes change mid-discussion. Only report the final, confirmed owner.

This forces the model to hold the entire exchange in view rather than pattern-matching on the first plausible assignment — precisely the kind of ambiguity where reasoning-before-answering visibly changes the result rather than just adding latency.

For cost-conscious users: a newer variant called Chain of Draft asks the model to draft each reasoning step in roughly five words instead of full sentences. Research shows it can match chain-of-thought accuracy while using as little as 7.6% of the reasoning tokens — a useful option once you've confirmed reasoning helps and want to optimize cost on top of that.

5. Run Automated, Iterative Prompt Optimization

The most advanced strategy on this list, and the one that turns "which fix do I need?" from a guess into something you can search for and measure. Instead of hand-tuning by feel, you score candidate prompts against real test cases and let a search process find the fixes that matter.

This is the same underlying mechanism behind production automated prompt-optimization tools: generate variations, score each against real cases, keep what works, repeat. The scoring here uses fuzzy task-matching against the transcript's known-correct answer — checking recall (did it find the real items), owner accuracy (did it attribute them correctly), and penalizing fabricated items that correspond to nothing real. Not just "did it return valid JSON."

Starting from a bare "extract action items as JSON" instruction with none of the five candidate fragments, the search began at a 51.6% composite score. Three iterations later, it had discovered and added exactly the three fragments that mattered for this transcript's real failure modes — final-owner tracking, no guessing at unassigned items, and excluding general discussion — reaching a perfect 1.000 score, without needing the other two available fragments at all.

That result is worth sitting with: the search found the minimum effective fix rather than throwing every available instruction at the problem. That's exactly the advantage of measuring against real cases instead of guessing which fragments sound helpful.

Bringing It Together

Layering all five strategies produces a prompt built from real, individually verified pieces rather than accumulated guesses: a defined role that primes the model to expect ambiguity, a JSON schema it must return, three deliberately diverse few-shot examples, a reasoning instruction pointed at the ownership-tracking failure mode, and the three corrective fragments the automated search actually proved necessary.

Compare that with the naive "list the action items" prompt from the start of this article. That version would plausibly report Priya as the mobile-review owner, miss the tablet-breakpoint merge entirely, and either drop the support-queue triage item or invent an owner for it instead of correctly leaving it unresolved. Every one of those failures is invisible on a quick read — and every one is a real error a team would eventually catch the hard way, in a missed deadline or a dropped ticket rather than a code review.

Wrapping Up

Five strategies, but really one underlying discipline: stop guessing at what might improve a prompt and start testing specific, individually verifiable changes against real cases. If your output looks plausible but keeps failing to parse, that's a structured-output problem — fix that first. If the same task drifts depending on how the input is phrased, that's a demonstration-selection problem, not an instruction-wording one. If the model misses something a careful human would catch on ambiguous input, that's what reasoning prompts are for. And once you've hand-tuned as far as intuition takes you, that's the point where an automated, scored search starts finding fixes a manual pass would miss — just as it found the minimum three-fragment fix on this transcript instead of the five anyone might have guessed at.