The Engine Is Allowed to Forget
· 6 min read
An inference server stays fast by treating every request as its first. vLLM's agentic layer puts the memory beside the engine, and most of its design follows from one rule about who owns the history.
Writing
Notes on agents, inference systems, and measurement.
A Loop Is Only as Good as Its Sensor
· 5 min read
An agent that grades its own work is a control loop whose sensor sits inside the plant. Every time I fixed my evaluation, it started rewarding a different degenerate agent.
The 31% Page-Size Slowdown That Wasn't
· 12 min read
An SGLang result survived every rerun and vanished only when I timed the right interval. Nsight and source tracing revealed when KV-cache pages actually matter.
The Honest Machine
· 4 min read
A generated paper can look finished and be wrong in the two places nobody checks. The useful engineering is the part that fails the build.
See all writing· 2 more
The Detector Was the Easy Half
· 4 min read
A focus tracker that reads your pulse through the webcam and decides when to interrupt you. The signal took a month. The interruption is a different kind of problem, and I turned it off within a day.
The Ghost Stories Had the Best Data
· 4 min read
Song-dynasty supernatural tales were classed as trivial anecdotes. They are full of geographic and social detail, because a haunting needs an ordinary world to interrupt.