AI agent coding for ESP-IDF workshop - Lecture 3: Reducing token usage in AI agent workflows

Introduction

#

An agent can solve a task with one focused exchange or spend many turns searching, rereading context, and correcting assumptions. Token-efficient work is not about making every prompt as short as possible. It is about providing the smallest complete context that lets the agent reach a correct, verifiable result.

What uses tokens?

#

Tokens are pieces of text processed or generated by a model. An agent request can involve several model calls as it selects tools, reads results, edits files, and checks its work.

The active context can include:

• System instructions and tool descriptions.
• Project rules and loaded skills.
• MCP server tools and their results.
• Conversation history and summaries.
• Retrieved source files, build output, and documentation.
• Your prompt and the model’s response.

These related measurements are not interchangeable:

• Context usage is how much of the model’s context window is currently occupied. Cursor’s context indicator and the CLI /context command show this kind of information.
• Input and output tokens measure text processed and generated by model calls. A single agent turn may make several calls.
• Cached tokens are reused input that a provider may charge differently. Caching can reduce cost or latency without reducing context occupancy.
• Usage or cost depends on the model, plan, and pricing. A cheaper model can cost less while processing the same number of tokens.

The IDE context indicator is useful for comparing how much context two sessions hold, but it is not a cumulative receipt for all model work in those sessions.

Start with a bounded task

#

State the outcome, scope, and stopping condition before the agent starts searching:

Review components/led_blink/led_blink.c against
components/led_blink/include/led_blink.h.
Check error handling and lifecycle behaviour only.
Report at most five actionable findings. Do not edit files.

This tells the agent which files matter, what to check, what not to do, and when to stop. A short but vague request such as “Review my project” can consume more tokens because the agent must discover the intended scope and may inspect unrelated files.

Keep context focused

#

• Reference relevant file paths and spec sections instead of pasting entire files into the prompt.
• Let the agent search for a symbol or dependency rather than loading a complete repository into the conversation.
• Keep STEP.md limited to the current task and its acceptance criteria.
• Request concise output when a summary, checklist, or fixed number of findings is enough.
• Avoid attaching logs or generated files that do not contribute evidence to the current problem.
• Keep required context. Removing the target, board, constraints, or success criteria often causes extra clarification and correction turns.

Open editor tabs can influence context in some agents, but closing tabs is not a reliable token-control mechanism by itself. Check the agent’s actual context indicator rather than assuming every open file is sent to the model.

Manage long sessions

#

Conversation history grows as a task continues. When moving to a separate task:

1. Finish or record the current working state.
2. Update the relevant spec and STEP.md.
3. Start a fresh session and reference those files.

If you must continue a long task, ask for a compact summary that records decisions, changed files, remaining work, and validation results. Review the summary before relying on it; omitted constraints can cause repeated work later.

Use rules and skills deliberately

#

AGENTS.md prevents you from repeating stable project conventions in every prompt. A focused skill can do the same for a recurring workflow such as component validation.

Rules and skills also occupy context when loaded. Keep them relevant, remove obsolete instructions, and avoid duplicating the same rule in several files. A large general-purpose skill is not automatically more efficient than a short task-specific one.

Limit tool context without losing evidence

#

Tools often return more text than the final answer needs. Ask the agent to:

• Search for specific symbols before reading whole files.
• Report the relevant compiler error and enough surrounding output to diagnose it.
• Summarise large documentation results while retaining the source link and version.
• Disable or disconnect unused MCP servers when their tool descriptions consume significant context.

Do not blindly keep only the last lines of a build log. The first error or an earlier configuration message may contain the root cause.

Choose models for quality and cost

#

A strong reasoning model can reduce correction turns for architecture, ambiguous requirements, and difficult debugging. A faster model may be sufficient for a small edit with an approved specification.

Switching models primarily changes capability, latency, and price; it does not guarantee fewer tokens. Keep the model fixed when comparing two prompting approaches so the comparison is meaningful.

Subagents are a context trade-off

#

A subagent can isolate research or review from the main conversation and return a concise result. This keeps the primary session focused, but the subagent uses its own model calls and can increase total token consumption.

Use a subagent when isolation or parallel work has clear value, not as a default token-saving technique. Give it a bounded task, relevant paths, and a specific return format.

Measure before drawing conclusions

#

Token use varies with the model, agent, repository state, caching, and non-deterministic model decisions. Compare workflows under similar conditions:

• Use the same fixed model.
• Start fresh sessions.
• Ask for the same outcome.
• Record context usage, turns, tool calls, and output size.
• Check result quality against the same acceptance criteria.

One run can demonstrate a difference, but it does not prove that one prompt will always use fewer tokens. Repeat the comparison if you need stronger evidence.

Next step

#

Assignment 5: Compare token usage between prompts

Back to workshop home