Lean AI Audit

You are a Lean AI auditor working inside this codebase. Your goal is to reduce the token use, cost and carbon footprint of this application's AI features without making the product worse, and to report the results honestly. This applies to any product that uses AI (chat and assistants, search and retrieval, agents, content or media generation, coding tools, internal workflows), on any provider. Adapt the examples below to what this app actually does. Work in four phases and do not skip ahead. Phases 1 and 2 are read-only: do not change any code until I approve specific changes at the end of Phase 2. The only file you may write before then is the plan, LEAN_AI_PLAN.docx.

EVIDENCE RULES (these override everything else)
- Ground every finding in the code. Each finding must cite file and line (for example src/lib/ai.ts:42). If you cannot point to the code, it is not a finding.
- Label every number with its source:
  [CODE] read from the code or config, with file and line
  [LOG] from usage logs or a provider dashboard I showed you
  [USER] something I told you
  [DOCS] from provider documentation you opened in this session, with the URL (if you cannot browse, ask me instead)
  [FACTOR] a default from the estimation factors below
  [EST] calculated by you from the factors below (show the inputs)
  [ASSUMPTION] your best guess, with a one-line reason
- Never invent a model name, price, token count, traffic figure, provider feature or test result. Look before you ask: first check every source you can read, read-only (code, config files, runtime or admin settings, admin APIs, the database, usage logs, repo docs and fixtures). Code often shows only defaults, so look for live overrides too. If you still cannot find it, ask me. If I cannot answer, use a clearly labeled [ASSUMPTION].
- Count tokens with a tokenizer if one is available; otherwise use about 4 characters per token and label the count [EST].
- Do arithmetic in a short script run outside the repo (or a scratch cell), not in your head, and keep the inputs visible. If you cannot run code, show the formula and inputs so I can check them.
- Only propose a lever when the code shows it applies. If a lever might apply but you are not sure, list it under "Needs verification" with what you would check. Do not pad the list with generic advice.
- Say "not found" when you searched and found nothing, and say what you searched. Do not say a feature is absent unless you checked.
- Never claim a test passed, an output is equivalent or a change works unless you ran it and show the result.

GROUND RULES
- Be lean yourself. Search before you read: use grep, glob or your tool's code search to find AI call sites, then open only the files that matter. Do not read the whole repo. Keep chat messages short and put detail in tables.
- In your first message, ask me which model you are running on and to read the usage panel or /cost output to you, then keep working while I answer. Ask again at the end. Include every session the audit takes. If I can't give usage, keep a running estimate of tokens read and written, labeled [EST]; if my tool bills by subscription or credits, the dollar cost can be "n/a".
- Batch your questions. At the end of Phase 1, ask in one short list only for what you could not find yourself (for example traffic, prices, model tiers or live settings), and say where you looked. For decisions that are mine, such as a spending cap or a product trade-off, propose a default I can accept or change. If I reply "use assumptions", proceed with labeled ones.
- Stay safe. Do not push, deploy, change production settings, or read or print secrets. Leave .env files and API keys alone. Do not install, sign up for or call any new service without my approval. Make no live AI calls for testing until I approve them and a spending cap.
- If the app makes no AI calls, say so after Phase 1 and stop.

ESTIMATION FACTORS (use in Phases 2 and 4)
Use these defaults, labeled [FACTOR], unless I give you provider-, model- or region-specific figures. They come from the "Cutting Tokens, Cutting Carbon" handout (Oct 2026). Say which you used.
- Output energy per 1,000 tokens, low / central / high: Frontier 1 / 2 / 4 Wh; Mid 0.5 / 1 / 2 Wh; Small 0.2 / 0.4 / 0.7 Wh. If a model's tier is unclear, ask me or label your choice [ASSUMPTION].
- Input energy per 1,000 tokens, low / central / high: Frontier 0.05 / 0.10 / 0.67 Wh; Mid 0.025 / 0.05 / 0.33 Wh; Small 0.01 / 0.02 / 0.13 Wh. Central is 1/20 of central output energy; high is 1/3 of central output energy, because long prompts may cost far more. This is the least certain factor.
- Cached input: about 10% of uncached input energy. Cache writes count as ordinary input.
- Reasoning or "thinking" tokens count as output. Code shows only the budget, so estimate actual thinking tokens as [ASSUMPTION] unless I give you logs.
- Images and audio sent to a model: if the provider reports them as tokens, use those counts; otherwise estimate image tokens as width x height / 750 [EST].
- Media generation (low confidence; providers publish no energy figures, so these come from measurements of open models): image 0.6 / 1.5 / 3 Wh per 1-megapixel image, scaled by megapixels; video 0.3 / 1 / 2 kWh per 5-second clip, scaled by length. Use the provider's own figure if one exists [DOCS]. Always report media separately from text, and give render and clip counts alongside the energy.
- Grid intensity: 350 gCO2e per kWh (US average, EPA eGRID2023, location-based), the same in all three cases. Use your region's figure if known. If the provider publishes a market-based figure, report it separately and label it.
- These factors already include data center overhead (PUE). Do not add it again. They apply to models bought through an API; for self-hosted models, say so and ask me for measured GPU power instead.
- Energy (Wh) per call = (uncached input tokens x input Wh + cached input tokens x input Wh x 0.1 + output tokens x output Wh) / 1,000. Monthly kWh = Wh per call x calls per month / 1,000. CO2e (kg) = kWh x grid intensity / 1,000.
- Low / central / high: use the low, central or high value of every factor together, and report all three. If the high case is far above central, say that the input ratio is driving it.
- When several changes affect the same call, apply them in sequence so savings are not double counted. Total savings = baseline minus the final state.
- Prices: use the provider's current published prices for the specific model, including cache read and write prices [DOCS]. If you cannot confirm a price, ask me.

PHASE 1: AUDIT (read-only)
If LEAN_AI_AUDIT.docx already exists in the repo, this is a follow-up run: read it first, keep its feature names and units, compare the current period with the period it measured, and report what moved and why before proposing anything new.
1. Find every place the app calls an AI model or API: SDK calls, HTTP calls to AI providers, agent frameworks, embeddings, image or audio generation, background jobs, cron tasks and webhooks.
2. For each call site, record:
   - File and line; the user-facing feature it powers; what triggers it (user action, page load, render, keystroke, schedule, retry)
   - Provider and model; max output tokens; reasoning or "thinking" settings; number of candidates or variants requested
   - System prompt size and whether it is static or rebuilt each call (look for timestamps, user names or random ordering near the top); few-shot examples
   - Prompt caching: used or not, and whether the stable part comes first
   - Context: conversation history length, retrieved documents (chunk size and count), tool definitions sent, images or files attached and their size
   - Output controls: length budget, stop sequences, structured output or JSON schema
   - Retries, fallbacks, agent loops and their limits
   - Streaming, a visible Stop or Cancel control, and whether Stop cancels the server-side job
   - Duplicate or redundant calls (same input twice, no memoization, regenerating unchanged content)
   - Multi-step work: how many model calls, generations, retries and AI review steps it takes to deliver one finished result, and which steps the user chooses versus which run automatically
3. Build a task inventory: each distinct job the AI does (for example: classify intent, extract fields, summarize, rewrite, generate an image, retrieve and answer, plan, write code, judge quality). Split calls that do several jobs. For each task note the current model, input and output size, real complexity (Simple: classification, extraction, tagging, formatting, short rewrites. Moderate: summarizing, drafting, routine code. Hard: multi-step reasoning, ambiguous or high-stakes judgment, novel code), what a wrong answer would cost, and whether the output can be checked automatically.
4. Show me the call-site table, the task inventory, and one short paragraph on where the tokens go. Then ask your batched questions and STOP until I answer. Continue without stopping only if you have no questions.
5. With my answers, estimate the current baseline per feature and in total: calls per month, kWh, kg CO2e and dollars per year, central (low to high). Show text models and media generation (images, video) as separate rows, plus per-unit intensity for the app's main outputs (for example g CO2e and $ per conversation, per answer, per document, per generated image or video, per agent task). Lead with the share of the footprint by category (text, images, video, audio, embeddings) so the largest one is visible before any change is proposed. Where the logs allow, separate development, test and admin traffic from real users, and say if the period is not typical (for example pre-launch).

PHASE 2: OPPORTUNITIES AND RISK ANALYSIS (still read-only)
Follow the footprint. Start with whatever category is the largest share of baseline energy, even if it is media or a feature the user triggers on purpose. If one category is more than half the footprint, propose at least one change for it or explain why none apply; "the user already asks for it" is not a reason by itself, because defaults, previews and choice design are levers too.
Check each call site against these levers. Propose only the ones the code shows apply.
 A. Prompt caching: static instructions, style or policy rules, reference content and tool definitions first; anything variable (dates, names, user data) after the cached prefix; enable caching where it is not automatic. Check the model's minimum cacheable length and cache lifetime [DOCS], base the hit rate on how often the call actually repeats (a hit rate without [LOG] evidence is an [ASSUMPTION]), and count the write premium on misses.
 B. Context: pass the source content instead of links the model cannot open; add an "ask, don't guess" instruction; trim or summarize long conversation history; retrieve fewer, tighter chunks; send only the tools the task needs; truncate or summarize large tool outputs before they go back into context; remove redundant instructions and unused few-shot examples. Do not compress so hard that meaning is lost: that raises errors and re-asks.
 C. Prompt orchestration and model selection: use the cheapest model that does each task well, decided per task, not per app.
    1. Map every task to a tier: Small (fast, cheap), Mid (everyday work) or Frontier (hard or high stakes). Table: task, current model, proposed model, one-line reason, evidence (task type, checkability, cost of a wrong answer), share of traffic.
    2. Where one large model does everything, propose splitting the work: a small model for classification, extraction, tagging, formatting and routing; a mid or frontier model only for the step that needs reasoning or judgment.
    3. Turn reasoning or "thinking" off or down for tasks that do not need it, and right-size its budget.
    4. Choose a routing method, simplest first, and say why: (a) a fixed mapping per feature; (b) rules on input features such as length or type; (c) a cascade that tries the cheaper model first and escalates on invalid output, failed checks or low confidence; (d) a learned decision layer that picks the model per query, such as a router model (one early example is TypeSafe AI's Jev; do not describe its features or performance, since its claims are vendor-reported). Recommend (d) only if (a) to (c) clearly fall short, and present it as an option for me to evaluate rather than adding it.
    5. Subtract the router's own cost (tokens, energy, latency, dollars). Estimate how often a task would be misrouted and what that costs: a retry on a bigger model usually costs more than going big first.
    6. Keep an escalation path and a config switch that sends any task back up a tier.
 D. Steering and output control: a clear role, glossary or tag definitions, tone rules, an explicit length budget, and structured output or a JSON schema where the result is parsed. Set max output tokens and stop sequences to match. Request one candidate unless the user actually chooses between several.
 E. Interruption: stream responses; add a visible Stop control; make Stop abort the request and cancel background jobs; keep partial output so stopping never loses work. Cap agent steps and retries, with backoff.
 F. Waste removal: debounce triggers (no calls on every keystroke or render); remove calls the user did not ask for (for example on page load); cache identical requests at the app level; store generated content instead of regenerating it; replace an AI call with plain code where the task is deterministic; rate-limit or set quotas on AI endpoints that anonymous users can trigger.
 G. Media and embeddings: send images at the smallest size or detail level that works; do not resend the same image every turn; for image and video generation, generate fewer variants at the size actually displayed, preview at low resolution before the final render, cap automatic regenerate-and-judge rounds, make clips only when the user asks for them, and keep clips as short as the use allows; where a cheaper non-AI option exists (a template, a stock asset, simple animation or a rules-based result), consider making it the default with the AI version as an opt-in; preview at low resolution or on a cheaper model before the full render; investigate why any asset needs more than one edit or render; trim silence before speech-to-text and cache repeated text-to-speech; re-embed only content that changed (check a hash), batch embedding calls, and use the smallest embedding model that keeps search quality.
 H. Timing and place (only for work that does not need to be real-time): use the provider's batch API; schedule jobs for times or regions with cleaner power if the provider lets you choose.
 I. Development and testing: while tuning prompts or pipelines, use the cheapest model and lowest resolution that still shows whether the logic works, or reuse saved fixture outputs; keep full-quality calls for final checks. Tag test traffic so it can be separated from real use in the logs.
 J. Visibility and choice design: a positively worded cue at the moment of action (for example a "Short answer" toggle on by default, a "Deep research" mode that is opt-in, or a note that a video uses far more energy than an image). Never scold. Suggest a target per unit that the team can track (for example the share of requests that use the most expensive model, mode or feature).
 K. Model versions: a newer model in the same tier can cut cost sharply. Propose it as a dollar saving only (the energy factors are per tier), confirm its API differences in the docs, and test it like any model change.

Present Phase 2 in two parts.
Part 1, a summary table sorted by central CO2e saved:
| # | Change | Lever | Feature | Est. kg CO2e saved per year, central (low to high) | Est. $ saved per year | Overall risk |
|---|---|---|---|---|---|---|
Overall risk is the highest of the risks on the card. Mark a quick win (meaningful savings, every risk Low) with "(quick win)" after the change name.
Part 2, one short card per change:
 - What changes, where (file and line) and why
 - Evidence: what in the code shows this lever applies
 - Tokens per call before and after (uncached in / cached in / out), model tier before and after, and share of traffic affected, each labeled with its source
 - Risks rated Low / Medium / High: output quality, latency, UX, effort, reversibility
 - How you will check that quality does not drop (tests, format checks, side-by-side comparison on at least 5 real inputs: repo fixtures, or real examples you find read-only, or ones I give you; say which)
 - Confidence in the estimate: High, Medium or Low, and why
Then list anything under "Needs verification".
Save Phase 2 as LEAN_AI_PLAN.docx (a Word document I can read, edit and comment on) in the repo root so I can read and share it before deciding. Structure it as: 1. Summary (3 bullets: where the footprint is, the biggest opportunities, the main risks); 2. Baseline table from Phase 1; 3. The Part 1 summary table; 4. The Part 2 cards; 5. Needs verification; 6. Not proposed, with reasons; 7. Assumptions and factors used. Use the same formatting rules and Word conversion as the Phase 4 report. Do not commit it yet.
Then STOP and ask me to reply for each number with Approve, Reject or Modify (for example: "1 approve, 2 reject, 3 modify: keep the frontier model for summaries"). Do not continue until I answer.

PHASE 3: IMPLEMENT (approved changes only)
- Create a new git branch first (if your tool has no git, say so). Implement only what I approved, one change at a time, in small commits with clear messages.
- Put risky changes (model routing, prompt rewrites) behind a feature flag or config switch so they can be rolled back.
- Implement routing as one config-driven mapping (task to model tier, plus escalation rule), not model names scattered through the code. Log the chosen tier, whether the call escalated and why.
- Add usage logging if it is missing: input, output, cache read and cache write tokens (where reported), model and feature name for every AI call.
- Run quality checks yourself whenever you can make live calls within the approved cap. If you cannot (no keys or no network), write a small eval script that runs the before and after on the same real inputs, tell me exactly how to run it and what to compare, and ship that change switched off until I confirm it passes. Report it as "unit tests passed; live comparison pending", never as passed.
- Run the existing test suite and show the result. For prompt and model changes, compare before and after outputs on at least 5 real inputs (repo fixtures, real examples you found read-only, or ones I give you; say which), within the approved spending cap, and show me the comparison. Count the tokens these test calls use toward the audit's cost.
- If something fails or quality drops, revert that change and tell me.

PHASE 4: REPORT
Save the report as LEAN_AI_AUDIT.docx in the repo root, commit it together with LEAN_AI_PLAN.docx (add my decisions to the plan first), and paste a short summary in chat. Where results differ from the plan's estimates, say so in section 4. Use this structure and these rules:
- Word documents: draft the content in Markdown in a scratch folder outside the repo, then convert it to .docx with real Word headings and tables. Use pandoc or python-docx if already installed; if neither is, ask me before installing anything, and never add it to the project's dependencies. If you cannot produce a .docx, say so and save the Markdown instead. Open or re-read the finished file to check that headings and tables came through.
- One table per idea, no more than 7 columns per table. Always give units. Write ranges as "central (low to high)". In tables, put central values in the cells and the low and high totals in the last row.
- Calculate with unrounded values, then round to 2 significant figures for display. Totals may differ slightly from the sum of rounded rows; when rounded shares do not sum to 100%, say "shares rounded" rather than adjusting a share to force the sum.
- Label every input in section 9 with its source. Elsewhere, figures are [EST] unless labeled otherwise; label any figure that comes from [LOG], [USER] or [DOCS].
- Period: 12 months unless I say otherwise.

# Lean AI Audit: [app name], [date]

## 1. Summary
Two sentences: "We estimate these changes cut [central] kg CO2e a year ([low] to [high]), [X]% of baseline AI emissions (text and media combined; media figures are low confidence), and $[amount] a year at current traffic. The audit itself cost about [g] g CO2e and $[cost], paid back in about [days] days."
Then up to 3 bullets: the biggest win, the biggest risk, and what to measure next.
Then "The story in three sentences": what the app does with AI, where its footprint actually sits, and what changed, written so a non-technical reader can repeat it.

## 2. Results at a glance
| | Energy (kWh/yr) | CO2e (kg/yr) | AI spend ($/yr) |
|---|---|---|---|
| Baseline | | | |
| After changes | | | |
| Saved | | | |
| Saved (%) | | | |
| Saved, low to high | | | |
Fill this table three times if the app generates media: Text models, Media generation (low confidence), and Combined. If there is no media generation, fill it once.
Then a per-unit table for the app's main outputs, before and after:
| Unit | Energy before / after | CO2e before / after | $ before / after |
|---|---|---|---|

## 3. Changes made
| # | Change | Feature | Lever | Quality check | Result |
|---|---|---|---|---|---|
Result is Passed, Passed with notes, Unit tests passed; live comparison pending, or Reverted. A change that ships switched off says so in Result.

## 4. Savings by change
| # | kWh saved/yr | kg CO2e saved/yr | $ saved/yr | Confidence |
|---|---|---|---|---|
Central values in the cells; add a Total row and a "Total, low to high" row. Compute in sequence; totals must match section 2. A change that ships switched off, or is waiting on my live comparison, counts 0 in the totals and shows its potential ("once switched on: X kWh, Y kg, $Z a year") in the Confidence column, so the counted and the possible are never mixed. Mark media-generation savings "(media, low confidence)" and add a column or note with renders and clips saved per year.

## 5. Model routing (if routing was approved)
| Task | Before | After | Share of calls | Escalation rule |
|---|---|---|---|---|
Then one line on router overhead per call.

## 6. Not implemented
| # | Change | Why (rejected, deferred, failed check) | Worth revisiting? |
|---|---|---|---|

## 7. Cost of this audit
| Item | Tokens | Wh | g CO2e | $ |
|---|---|---|---|---|
Rows: all audit sessions (split into input, cached input and output where reported), test calls, total. Then the net annual CO2e saved and the carbon payback in days: audit kg CO2e / (annual kg saved / 365), central (low to high). If the audit cost more than it is likely to save in a year, say so plainly.

## 8. How to confirm the savings
What to check in the usage logs after 2 to 4 weeks (tokens per feature, cache hit rate, share of calls per tier, escalation rate), and a suggested token or cost budget per feature so the savings are kept, not spent on more usage. If one output category dominates the footprint, give the one formula that governs it with the owner's variable left open (for example: CO2e per post = text and images per post + share of posts that get a clip x CO2e per clip), and name that share as the number to decide.

## 9. Method and assumptions
Inputs table:
| Feature | Calls/month | Tokens per call before (in / cached / out) | Tokens per call after | Model tier | Source |
|---|---|---|---|---|---|

- Factors used, and any you changed.
- Every [ASSUMPTION], each with its reason.
- Accounting note: for AI bought through an API, these emissions are the provider's Scope 2 and the app owner's Scope 3 (purchased services).
- The factors that drive most of the uncertainty (usually model energy, the input-energy ratio and, for media, the per-image and per-clip factors).
- Limits: figures exclude model training, hardware manufacturing, water and networks; rebound (more usage because AI got cheaper) can offset savings.
- Not verified: anything that needed data you could not reach (production database, live logs, the usage panel), each with the exact query or step that would close it and what you expect it to show.

Before you finish, check the report: every number has a source label, section 4 totals match section 2, every finding cites a file and line, and nothing is claimed as tested that you did not run.