Skip to slide
Chapter 16 · Production Realities: Economics, Security, and Governance
129 / 142

CHAPTER 16 · Production Realities: Economics, Security, and Governance · 8 / 10

Code MVP: a cost-per-task estimator

"""
chapter 16: cost-per-task model under token billing.
Shows why caching dominates cost and why retry loops are the real danger.
Units are illustrative; the SHAPE is the point.
"""
from dataclasses import dataclass

@dataclass
class Rates:
    fresh_input: float = 1.00     # per 1k fresh input tokens
    cached_input: float = 0.10    # ~10x cheaper (Chapter 5)
    output: float = 4.00          # per 1k output tokens

def task_cost(turns: int, base_context: int = 3000, new_per_turn: int = 800,
              output_per_turn: int = 400, rates: Rates = Rates()) -> float:
    """Each turn re-sends history: the base context is cached, only the new
    delta is fresh input. Cost accumulates across turns (the loop)."""
    total = 0.0
    cached = base_context
    for _ in range(turns):
        total += cached / 1000 * rates.cached_input        # cached prefix (cheap)
        total += new_per_turn / 1000 * rates.fresh_input   # this turn's new input
        total += output_per_turn / 1000 * rates.output     # output
        cached += new_per_turn + output_per_turn           # history grows
    return round(total, 3)

if __name__ == "__main__":
    print("simple task   (3 turns): ", task_cost(3))
    print("complex task (12 turns): ", task_cost(12))
    print("retry storm  (40 turns): ", task_cost(40), "  <- cap this!")
    # Without caching, the same retry storm is far worse:
    no_cache = Rates(cached_input=1.00)  # pretend nothing is cached
    print("retry storm, NO cache:    ", task_cost(40, rates=no_cache))

Run it: a simple task is cheap, a complex one moderate, and a 40-turn retry storm is where the money goes, especially with caching disabled. The gap between the cached and uncached retry storm is the dollar value of Chapter 5, and the reason Chapter 12's max_turns cap is a financial control, not just a safety one.

← → arrow keys work too