The Real Python Podcast Episode #306: Building LLM Prompts Programmatically Using DSPy

Meta description: Learn how Brett Kennedy's new book on DSPy helps LLM app builders move from manual prompt writing to programmatic prompt definitions for teams.

Tags: DSPy, LLM applications, prompt engineering, Brett Kennedy

Featured image: abstract, friendly illustration style with no real people, no logos, and no text.

This article is for developers and product teams building large language model applications that depend on prompts. It summarizes a shift described in a recent show segment: moving from manually writing prompts for an LLM application toward programmatic prompt definitions. This week on the show, Brett Kennedy returns to discuss his new book, "Building LLM Applications with DSPy." The key takeaway is not a new benchmark score or a product announcement with hard numbers, but a workflow change: prompt behavior becomes something that can be maintained more like software than loose text.

Why manual prompts become a bottleneck

Most early LLM applications start with hand-written prompt strings. A developer writes a sentence, runs it, edits it, and repeats the process until the model produces a useful answer. This is fast for a prototype, but it can create maintenance problems once the application grows.

Prompts often contain several responsibilities at once: instructions for tone, formatting rules, examples, safety guardrails, context boundaries, and output expectations. When these are stored as plain text, small changes can affect multiple parts of the answer. A prompt that works for one user request may fail for another because the task, context length, data format, or expected output changed.

The bottleneck is not simply that prompt engineering is hard. It is that prompt strings are difficult to test, version, reuse, and improve systematically. A team may need to know which prompt version produced a given result, whether a change improved accuracy, or how to adapt the same behavior across different models and use cases. Manual prompt writing can also make handoffs difficult. If the knowledge lives in one developer's editing history, the application becomes harder to review, document, and operate.

What programmatic prompt definitions mean

Programmatic prompt definitions are the alternative approach. Instead of treating each prompt as a one-off text block, the application represents prompt behavior through code, templates, configuration objects, or reusable components. That does not mean prompts disappear. It means they are managed through a structure that can be composed, parameterized, tested, and updated.

For example, a team might define an input type, an output type, and a reusable instruction pattern as part of the application code. The prompt can then be assembled from those parts rather than copied into a string. In practice, this can make it easier to change one rule and see the effect across many tasks, add tests that check whether outputs follow a format, or compare different prompt configurations in a more controlled way.

DSPy is the framework named in Brett Kennedy's new book title. The source does not provide a feature list, benchmark numbers, or a detailed chapter outline, so it is best to describe the shift at a general level. The important idea is that DSPy is being discussed as a way to move LLM application development away from hand-tuned prompt strings and toward programmatic definitions. For readers, that suggests a focus on repeatability and maintainability rather than isolated prompt tweaks.

What the episode and book signal for LLM teams

The value of this topic is mostly practical. Teams that ship LLM features often face a tension: language models are flexible, but production software needs reliability. Manual prompts can feel flexible, but they can also create hidden complexity. If every answer depends on subtle wording, small model upgrades, context changes, or formatting rules can become hard to manage.

A programmatic approach can help in several ways. First, it can make prompt behavior more explicit. When instructions are defined in code or structured objects, developers can see how inputs, examples, constraints, and outputs fit together. Second, it can improve iteration. A team can change a component and evaluate its effect rather than editing an entire prompt by hand. Third, it can support collaboration. Product managers, engineers, and quality reviewers can discuss defined behaviors instead of guessing which wording caused a failure.

This is not a claim that programmatic prompts automatically solve reliability, cost, or safety. Prompting still interacts with model behavior, retrieval data, tool calls, guardrails, and evaluation design. The shift is about control: making prompt logic easier to manage inside a software workflow. The source does not state that this approach outperforms other methods, and no performance numbers are provided, so readers should treat it as a workflow pattern rather than a proven performance upgrade.

What readers can take away

The most useful takeaway is a checklist for evaluating your current LLM application. Ask whether your prompts are stored as isolated strings or as defined components. Ask whether you can test them, version them, and reuse them across similar tasks. Ask whether a model upgrade would require rewriting many hand-tuned prompts, or whether your application can adjust through structured definitions. If the answers point toward manual strings, the DSPy book and the show segment may be worth exploring as an introduction to a more programmatic workflow.

The segment also highlights a broader trend in LLM application development: the field is moving from demo-stage prompt crafting toward production-stage software engineering. Prototypes can survive on clever wording. Products need reproducibility, reviewability, and maintainability. Defining prompts programmatically is one way to bring language model behavior closer to those expectations.

Who should use it / who should skip it

Who should use it: developers building LLM applications that are likely to grow beyond a simple demo, teams maintaining several prompt-driven features, and engineers who want a framework for treating prompts as software artifacts. If you are already struggling with prompt sprawl, version confusion, or inconsistent output formatting across tasks, the book's topic is directly relevant.

Who should skip it: readers looking for benchmark numbers, price information, or a detailed technical comparison of frameworks. The source provides only the show segment and the book title, not those specifics. Teams with a single, short-lived prompt experiment may also not need the overhead of a programmatic prompt system right away. For them, manual prompt writing can remain a reasonable starting point until the application needs repeatable maintenance.