AI Operations · B10–B27wk 2 / 54

I run a small fleet of AI agents in public, and publish what they cost.

Product manager building, measuring and governing agentic workflows. This page is the register: what is planned, what is running, what it costs per verified outcome, and what I decided not to automate.

NOW · B10 Evaluation Harness. The Foundation · week 2 of 54
PERMISSION · no rung reached yet. Nothing I have built acts without a human. Next: suggests, gated on B11.

Surface Zero uptime
not measured yet
Runs served
not measured yet
Cost / verified outcome
not measured yet
Installable installs
not measured yet

The run ledger starts writing here when Surface Zero goes live in B11. Until then these are empty on purpose.

What these agents are allowed to do

fig. 1 · consequence ladder

Every rung is a permission a third party can revoke, and each one is gated on a block landing first. A dashed rung has not been reached — until it is, the work behind it is described as implemented, never as operated.

BLAST RADIUS ↑WEEK 54WEEK 1nothing consequential yetA · SUGGESTSwk 6 · comments only, on a repo I don't own · B11B · STRANGERS RUN ITwk 9 · tenants I don't control, spend I didn't authorise · B12C · ACTSwk 21 · applies labels and closes issues without a human · B16D · MERGESwk 30 · auto-merges CI-passing dependency bumps on repos I own · B19E · AUTHORSwk 42 · opens PRs on opt-in tenants · B23YOU ARE HERE · WK 2

Built and published

full register →
no screenshot published yet
B8 · 2 live artifacts

Smart Reply Copilot: Multi-Step Prompt Chaining

Built an AI-powered customer support assistant that routes tickets through a 3-step classification chain: category detection → tone calibration → draft response generation. Each step includes confidence scoring for transparency and human-in-the-loop review.

shippedB8 · weeks 2224
no screenshot published yet
B7 · 3 live artifacts

AI ROI Calculator: Translating Technical Metrics into Business Value

Built a financial modeling tool that bridges the gap between technical execution and executive decision-making. Translates AI system costs into CFO-ready 3-year projections using real production telemetry from Block 6. Demonstrates the critical PM skill of business value communication—turning 'it works technically' into '$1.17M annual value' with cited, defensible assumptions.

shippedB7 · weeks 1921

Everything else on the shelf

7 blocks
B1shipped

Foundations: API & Prompt Engineering

Mastered the fundamentals of LLM interaction, moving from 'Chat' to 'API'. Learned to control output structures via System Prompts.

B2shipped

Help Desk Copilot: RAG-based Q&A System

Built a conversational AI assistant that answers questions about support ticket data using complete RAG implementation (Retrieval-Augmented Generation). Goes beyond keyword search to synthesize answers, identify patterns, and provide insights across the entire corpus—demonstrating the same architecture powering ChatGPT and modern AI assistants.

B3shipped

Full-Stack Prompt Engineering & Governance

Evolved from basic scripts to a production-grade Web Application. Built a 'Prompt Playground' that operationalizes prompt templates and enforces ethical guardrails via UI logic.

B4shipped

Fine-Tuning & Local Inference

Pivoted to 'Small Language Models'. Fine-tuned Llama-3 to perform niche tasks cheaply, proving that bigger isn't always better.

B5shipped

AI Product Owner: Agentic Backlog Generation

A CLI agent that ingests raw support tickets from a CSV and writes developer-ready user stories, using a ReAct loop with a reflexion pass for quality control and a Zod schema to force INVEST-compliant output. Adds a dry-run cost estimate and RICE scoring, then wraps the CLI in an Express API.

MS1shipped

Voice of Customer: Unsupervised Insight Engine

Built an AI-powered analytics platform that transforms unstructured support tickets into strategic insights using unsupervised learning. Automatically clusters thousands of tickets, generates executive summaries, and surfaces hidden patterns—eliminating hundreds of hours of manual categorization while maintaining strict data governance and cost controls.

B6shipped

Market Intelligence Agent: Hybrid RAG with Autonomous Routing

Built an autonomous AI agent that solves the 'stale data' problem by intelligently routing queries between internal knowledge (vector store) and live web search (Tavily API). Demonstrates advanced RAG architecture with self-routing capabilities—the agent autonomously decides which data source to query based on the question's intent, eliminating training data cutoff limitations.

Working log

all 15 entries →
2025-12-08B5B5 Polish: Enterprise Features
2025-12-01B5B5 Build: The PM Bot
2025-11-24B5B5 Learn: Agentic Patterns
2025-11-17B4B4 Polish: The API Wrapper
2025-11-10B4B4 Build: Training Llama-3
2025-11-03B4B4 Learn: Fine-Tuning Theory