python-sniper — Context Cost Reduction for AI Coding Agents

The problem

Bulk-reading source is the single biggest avoidable cost in an agent loop, and it is invisible. The agent asks to read models.py; the harness hands over 40 KB; the agent needed six function signatures. Nothing in the loop notices, because nothing is measuring. The cost shows up later as a truncated context, a lost thread, or a bill.

Why it mattered

Context is the agent's working memory, and it is the constraint that actually binds. An agent that burns its window on function bodies it never reads has less room for the problem it was asked to solve — so it forgets earlier decisions, re-reads files it already saw, and produces worse work. Left alone, the failure mode is silent.

What I built

I built a pi extension that sits in front of every channel an agent can read Python through — the read tool, bash invocations of cat/sed/head/tail/grep/rg, sandboxed ctx_execute calls to open(), and @file.py:symbol at-refs. Each read is classified by intent and routed one of three ways: to a Language Server outline when the LSP is alive, to a hand-written indentation-aware AST outline when it isn't, or straight through when the agent explicitly asked for a byte range. The agent gets the structure plus a hint naming the precise tool to call if it genuinely needs a body. 1,216 lines across six modules, and zero runtime dependencies — the cache key is a hand-rolled FNV-1a hash rather than a dependency on crypto.

Cutting context cost is only half of it. Every routed read also appends a row to a JSONL log, because the interception point is the one place in the loop that sees both what the agent asked for and what it should have asked for instead. That log is intended as the training corpus for fine-tuning a model to reach for read_symbol and ast_grep_search on its own — at which point the interceptor has taught itself out of a job.

1. Step 1
   Intercept the read

   Four channels are watched — the read tool, bash cat/sed/head/tail/grep/rg, ctx_execute open(), and @file.py:symbol at-refs. Anything targeting Python source is caught before it reaches the filesystem.

2. Step 2
   Classify the intent, pick a route

   An explicit offset+limit means the agent wants a specific region, so it passes through untouched. Otherwise a latency probe decides — LSP outline if the language server answers inside the adaptive P95 threshold, hand-written AST outline if it doesn't.

3. Step 3
   Return structure, name the next tool

   The agent receives classes, signatures, imports and a routing header telling it which precise tool (read_symbol, ast_grep_search) to call if it truly needs a function body — plus a cache entry keyed on file hash, so the second read of an unchanged file costs nothing.

4. Step 4
   Log the lesson

   Every routed attempt appends a row to a git-ignored JSONL log. The interception point is the only place that sees both the read the agent chose and the read it should have made — which makes it the place to harvest data for fine-tuning the model to reach for those tools unprompted.

Key decisions

Mirror the LSP payload shape rather than depend on the LSP. A pi extension cannot invoke the pi-lens MCP tools (lsp_navigation, read_symbol) from extension code — a hard platform constraint. I could have made the extension require pi-lens and fail without it. Instead the AST fallback emits the same documentSymbol payload shape the LSP route emits, so the agent sees one consistent contract whether or not a language server is running. The extension degrades; it doesn't break.

Adaptive P95 latency probe over a fixed timeout. A fixed LSP timeout is wrong twice: too short on a cold server and it never uses the good path, too long on a dead one and every read stalls. The LatencyTracker accumulates observed probe latencies and computes a rolling P95 threshold, falling back to a default during warmup. The route decision adapts to the machine it's actually running on.

Teach the model, don't just tax it. The cheap version of this extension silently swaps the payload and moves on. Instead every reroute carries a hint naming the canonical tool — "use ast_grep_search for a structural pattern match instead of grepping raw source" — and logs the attempt. The interception point is the only place in the loop that observes both the read the agent chose and the read it should have made, which makes it the natural place to harvest training data. The header is not overhead; it is tuition. The end state is a fine-tuned model that reaches for read_symbol unprompted and makes the interceptor redundant.

Outcome

What a production version needs

The log cannot yet train the model it exists to train. This is the real gap, and I found it by reading my own rows. A preference-tuning run needs to know what the model was told, what it then did, and whether that was better. Each row records ts, channel, route, why, path, intent, cache, symbol, hash, latency — and so:

• The hint isn't logged. The instruction the agent received is absent, so the input side of the training pair can't be reconstructed.
• The agent's next action isn't logged. Whether it obeyed the hint and called read_symbol, or re-read the file raw, is the label — and nothing captures it.
• The byte counts aren't logged, though TEST_PLAN.md specifies them and the parser already computes both. The one reward signal in reach isn't written down.

It is an observability log, not a preference dataset. The fix is close, because the data half-labels itself: a cache: "hit" means the agent re-read a file it had already been given an outline for — a revealed refusal of the hint, free of human annotation. Log hint, next_tool, and bytes_in/bytes_out, and rows become DPO pairs mechanically.

Two-thirds of the corpus carries no teaching signal. Of 90 logged attempts, 32 were ctx_execute_file passthroughs and 28 were explicit raw-region requests — logged, never rerouted. Only 30 (33%) produced an outline with a hint attached.

The routing header is unconditional, so on small files the tuition exceeds the tuition's worth. Header and footer add ~750 bytes to every intercepted read, regardless of size. On the 1,016-file library corpus 691 files (68%) come back larger than the raw source, and the median library file inflates to 144% of raw; break-even sits at roughly 836 bytes of source. The aggregate still nets 64.5% because large files dominate the byte count. Teaching is worth paying for, but not on a seven-line config module — a guard that falls through to the raw read when the outline can't beat it would keep the lesson where it's worth teaching.

There are no tests. TEST_PLAN.md is a well-specified plan — Vitest, fake timers, a ≥90% line-coverage target on the pure logic — and every case in it is unimplemented. CI runs npm install and npx tsc --noEmit. Type-checking is not testing. The plan and the code have also drifted: the plan promises bytes_in/out in each row, the logger.ts docstring lists nine fields and omits latency, and the interface ships ten fields and no bytes.

It isn't published. The README instructs you to install npm:python-sniper; no such package exists on the registry yet. The auto-discovery path (dropping it in ~/.pi/agent/extensions/) is the one that works.