The Page Moves While You Think

A recommender scores your interest by how long you watch. A browser agent's watch time is however long the model takes to decide. So the agent expresses preferences by thinking.

5 min read
Web AgentsBenchmarksTemporal ReasoningResearch

Give a browser agent a short-form video feed and tell it to build a profile that likes rings. It opens a cat video, reasons briefly, scrolls on. It opens a hiking video, and this one is harder to parse, so it spends three thousand tokens working out what it's looking at. Ten seconds of wall clock. Then it scrolls on.

Behind the feed is a recommender that scores topic affinity by like-weighted watch time. One second on the cat. Ten on the hiking video. As far as the server is concerned, the user just told it something with real conviction: hiking.

Nothing in the agent's action space contains "express interest in hiking." It never clicked like. It thought hard about a difficult page, and its deliberation cost was read as engagement. By the time it reaches an actual ring video, the feed has drifted somewhere else entirely, and the task it was given is now harder than when it started.

That is the failure mode I find most interesting in web agents right now, and almost nothing measures it.

Every benchmark assumes the world holds still

The standard model of a browser agent is a loop: observe the page, predict an action, observe the result. Formally that's a partially observable Markov decision process, and its defining assumption is that state transitions are triggered by the agent's actions. Nothing happens unless you do something.

That was a fair description of the web in about 2005. It isn't now. Today's servers are general-purpose programs holding session state, and communication is no longer one-off GET and POST — it runs over asynchronous XHR and fetch, and over persistent channels like WebSockets and server-sent events. Rate limits, session timeouts, expiring content and time-varying pricing put a temporal dimension in the server's control flow. The browser is the surface of a deep, stateful, time-dependent stack, and content changes as a function of other users, backend jobs, and clocks you cannot see.

Agents inherited the old assumption anyway. Their entire temporal vocabulary is a wait action, usually hard-coded to a few seconds and capped at thirty, used as a pause button rather than a strategic instrument. The consequence is sharper than it sounds: an agent that has no model of time cannot tell a server that is busy from a server that already timed out and moved on. Both look identical from inside the loop. Nothing changed.

Figure 01 · the coupling

Thinking longer is indistinguishable from liking it more.

task: build a profile
that likes rings

server readspageagent decodesCat300 tokenst_watch = 1sCat: LowHiking3,000 tokenst_watch = 10sHiking: HighRing300 tokenst_watch = 1sRing: Lowwall clockbar width = time spent decoding = time the server counts as watching

The agent never clicked like. It spent 3,000 tokens on the one page it found confusing, and the recommender scored that as the strongest preference in the session.

Token counts and dwell times follow the worked example in the paper. The ring video is the one the task actually asked for, and it gets the same one second as the cat.

Putting time in the formalism

You cannot patch this with a longer context window, because the problem is in the state model rather than the memory. A POMDP has no way to express "the state changed while you were deciding."

FullStackArena, the benchmark I've been working on, replaces it with a continuous-time, partially observable semi-Markov process. Two things change. Actions take variable amounts of time, so the transition function becomes a joint distribution over the next state and the interval it took to get there. And the environment evolves under a null action, which is the formal way of saying the server keeps running when the agent does nothing.

That reads like notation until you notice what it licenses. Waiting becomes a real action with a real cost. Dwell time becomes something a task can specify and a grader can check.

Figure 02 · task classes

Existing benchmarks stop at the second rung.

each class strictly
subsumes the last

01Single action
no timing
Place an order.
02Multi-step
adds retries and branching
If one order fails, place a second.most benchmarks end here
03Relative timing
adds durations between actions
Dwell 12s on post 1, 3s on post 2.
04Absolute timing
adds wall-clock deadlines
Place orders at 9:00 AM and 9:05 AM.

Nothing below the line is a reasoning problem. All of it is impossible without a clock.

The action-primitive hierarchy the benchmark is built on. The two lower classes are what a request-response model of the web can express; the two upper ones need the environment to keep moving on its own.

Which produces tasks that are trivial to understand and genuinely hard to execute. Open a video, wait until the ad becomes skippable, skip it promptly, and the window is three to six seconds. Buy one share of AAPL between 9:30 and 9:31, two more between 9:31 and 9:32, three between 9:32 and 9:33. Book the ride only if the fare is between $25 and $50 and pickup falls between 8:00 and 9:00 AM. Find the auction item with the most bids, wait five minutes, then outbid by a dollar if the price moved.

None of that is a reasoning problem. All of it is impossible if you don't know what time it is.

The ablations are the finding

The obvious objection to any of this is reproducibility: if the environment moves on its own, how do you benchmark it twice? The answer is procedural generation against a global simulation clock. Stock prices come from historical traces blended with stochastic models seeded per task, so two agents starting at the same relative timestamp observe identical state transitions no matter when the run physically happens. Determinism without freezing time.

The headline number is that GPT-5.4 clears 6.8% end-to-end. I don't think the headline is the interesting part. A number that low is mostly measuring agent architecture rather than model capability, which is the paper's own reading of it.

The two ablations say more.

Uncap the wait and scores jump: Gemini-2.5-Pro gains 18.6 points on stock trading, GPT-5.4 gains 22.1 on ride-hailing. The error analysis explains why, and it is not what I expected. Under a thirty-second cap, agents overestimate how much time has passed — they behave as though more than thirty seconds elapsed — and then fail to issue the follow-up waits that would have got them there.

Tell the model what time it is and the effect is larger still. On ride-hailing, the one site with no clock in its interface, adding visible time information takes GPT-5.4 from 26.0% to 40.3% and Gemini from 14.3% to 27.3%.

Ride-hailing · success rateuncapped wait, on the one site with no clock in its UI
GPT-5.4+14.3
no clock26.0%
clock40.3%
Gemini-2.5-Pro+13.0
no clock14.3%
clock27.3%

Same model, same tasks. The only difference is being told what time it is.

A clock is worth more than most of the distance between these models. That is a statement about how these systems are built, not about how well they reason.

I used to think perception was the wall. I once lost a week to a Submit button that was plainly on screen and invisible to my agent, three layers deep inside a shadow root. That failure is largely behind us; hand a model a screenshot today and it will tell you what's clickable.

This one won't yield the same way, because the agent is not standing outside the system it observes. Its latency is an input to that system. The time it spends deciding is time the server spends changing, and on a recommender-driven page that time is itself a signal being recorded.

Which is the same shape as a problem I ran into building an evaluation harness: an instrument sitting inside the thing it is trying to measure. You don't fix that by measuring harder. You fix it by putting the coupling into the model, and then, at minimum, by giving the thing a clock.