Building AI Into Your Software — from your first API call to a feature you ship

The AI in Software Dev path taught you to *use* AI to write code. This one teaches you to *build* AI into the software you ship — to call a model from your own program and turn it into a feature. Start with what changes when a language model becomes a component in your architecture, make your first API call, and learn to treat the model as a powerful but unreliable dependency. From there you'll write prompts as code, get reliable structured output, give the model tools, ground it in your own data with retrieval (RAG), let it act as an agent, and — crucially — evaluate, secure, and operate all of it in production. Examples lean on real software, including AI features you might add to a project like GopherTrunk. Pairs with the AI in Software Dev path; no AI background beyond it required.

Start lesson 1 →Jump to glossary

You’ve used AI to help write code. This path is about the next step: building AI into the software you ship — calling a model from your own program and turning it into a real feature that users touch. That’s a different skill from prompting a chat window, and it has its own toolkit: model APIs, prompts written as code, structured output, tools, retrieval, agents, evaluation, and the operational work of running it all in production.

Who this is for. Developers who can already write a bit of software and want to add AI features to it. You don’t need a machine-learning background — this is about using models through their APIs, not training them. If you’ve worked through the AI in Software Dev path, you’re perfectly set up; if not, skim it first for how models actually work.

How the path works. Six modules take you from your first API call to a shipped feature. The early modules build the core mechanics — calling a model, prompting it reliably, and getting structured output and tool use. The middle modules cover the two techniques most real features are built on: grounding a model in your own data with retrieval (RAG), and letting it act as an agent. The last modules are the parts that separate a demo from a product — evaluating quality, defending against prompt injection, and controlling cost, privacy, and reliability in production. Examples lean on real software, including AI features you might add to GopherTrunk itself. Mark lessons complete as you go — your progress is saved in your browser. New here? Start with lesson 1: What it means to build AI into software

Module 1 — From Using AI to Building With It

The shift from chatting with AI to calling it from your own code: what an AI feature is, the model as a component, how the API works, your first call, and the cost it carries.

  1. What it means to build AI into software The jump from using an AI tool to shipping an AI feature — what changes when the model runs inside your program instead of a chat window. beginner 8 min
  2. The model as a probabilistic component Designing around a dependency that is powerful, non-deterministic, and occasionally wrong — the mindset shift everything else builds on. beginner 9 min
  3. How a model API call works Messages, roles, tokens, and the request/response cycle — what actually goes over the wire when your code talks to a model. beginner 10 min
  4. Your first API call API keys, SDKs, and a first request in code — the smallest end-to-end example of software calling a model. beginner 9 min
  5. Cost, latency & rate limits by design Every call costs tokens and time — why cost, latency, and rate limits are design constraints you plan for, not afterthoughts. intermediate 9 min

Module 2 — Prompting in Code

Turning prompts from throwaway chat messages into reliable, versioned parts of your program that produce output consistent enough to build on.

  1. Prompts as code — system prompts & templates Treating prompts as first-class, version-controlled artifacts — system vs. user roles, templating, and separating instructions from data. intermediate 9 min
  2. Feeding the model context & user input Assembling the right context safely — injecting your data and user input into a prompt without letting it hijack your instructions. intermediate 9 min
  3. Few-shot examples & steering output Showing the model what "good" looks like — examples, constraints, and the levers that make output consistent enough to rely on. intermediate 9 min
  4. Getting usable output back The model returns text — parsing, validating, and retrying it so your program gets data it can actually trust. intermediate 9 min

Module 3 — Structured Output, Tools & Streaming

The features that turn a text generator into a programmable component: JSON output, tool and function calling, streaming, and multimodal inputs.

  1. Structured output & JSON Getting the model to return machine-readable JSON that matches a schema — the key to wiring a model into the rest of your code. intermediate 10 min
  2. Tool use & function calling Letting the model call your functions — how tool use works, and how a model that can't do math or fetch live data suddenly can. intermediate 11 min
  3. Streaming responses Showing output token-by-token as it's generated — why streaming matters for user experience and how it changes your code. intermediate 8 min
  4. Multimodal — images, audio & documents Models that read more than text — sending images, audio, and documents, and the feature ideas that opens up. intermediate 9 min

Module 4 — Retrieval-Augmented Generation (RAG)

Grounding the model in your own data so it answers from your documents and facts instead of only its training — the most common way to build a genuinely useful AI feature.

  1. Why retrieval — grounding the model in your data The model only knows its training data — retrieval feeds it your facts at query time, cutting hallucination and keeping answers current. intermediate 9 min
  2. Embeddings & vector search Turning text into vectors so you can search by meaning, not keywords — the engine underneath retrieval. intermediate 10 min
  3. Building a RAG pipeline The end-to-end shape — ingest, chunk, embed, store, retrieve, and augment the prompt — assembled into a working feature. advanced 11 min
  4. Chunking & retrieval quality RAG lives or dies on retrieval — chunking, metadata, re-ranking, and how to tell when retrieval is the thing that's broken. advanced 10 min
  5. RAG vs. long context vs. fine-tuning Three ways to give a model knowledge it didn't ship with — what each is for, what each costs, and how to choose. advanced 10 min

Module 5 — Agents, Evaluation & Safety

Letting the model act, then making sure it acts well: agent loops and tools, evaluating a probabilistic feature, handling failure, and defending against prompt injection.

  1. Agents — letting the model act in a loop From one-shot calls to a plan-act-observe loop — what an agent is, what it can do, and where the risk goes up. advanced 10 min
  2. Giving agents tools (and MCP) Tools are how an agent affects the world — designing safe tools and the Model Context Protocol (MCP) that standardizes them. advanced 10 min
  3. Evaluating AI features You can't unit-test a probabilistic feature the old way — evals, test sets, LLM-as-judge, and measuring quality before and after a change. advanced 11 min
  4. Handling hallucinations & failure The model will be confidently wrong, slow, or unavailable — grounding, guardrails, fallbacks, and failing safe when it does. intermediate 10 min
  5. Prompt injection & data safety Untrusted text can hijack your prompt — prompt injection, data exfiltration, the "lethal trifecta", and how to design against them. advanced 11 min

Module 6 — Shipping & Operating AI Features

Everything applied: choose a model, control cost and latency, protect user data, watch it in production, and ship your first AI feature end to end.

  1. Choosing a model for a feature Matching model to job — capability, context, speed, cost, and privacy — and why most products end up using more than one. intermediate 9 min
  2. Optimizing cost & latency Prompt caching, smaller models, batching, and streaming — the practical levers that make an AI feature affordable and fast. advanced 10 min
  3. Privacy, data & compliance Where prompts and user data go, what providers do with them, and the privacy, retention, and compliance questions to settle before you ship. intermediate 9 min
  4. Observability & monitoring You can't fix what you can't see — logging prompts and outputs, tracing, cost and latency metrics, and catching quality drift in production. advanced 9 min
  5. Shipping your first AI feature Putting it all together — an end-to-end worked example that adds a real AI feature to a project like GopherTrunk, and how to keep improving it. intermediate 10 min
  1. Glossary of AI-engineering terms Plain-language definitions for every term in the path — API, token, system prompt, structured output, tool use, embedding, RAG, agent, MCP, eval, prompt injection, and more — cross-linked to the lessons.