tboulet

ModAct

Modelling and Acting

An agent that writes a program of its world, and acts on it.

ModAct is a way of running an LLM agent on an interactive task it knows nothing about, such as an ARC-AGI-3 game. Rather than only acting and exploring, the agent alternates two phases: it models the game by writing a program that explains everything it has seen, then acts in the game while that program is checked against reality at every step.

The ModAct loop The dataset of observed transitions feeds the modelling phase, where the agent writes a Code World Model with a State, get_initial_state, step and render, and validation replays every recorded episode; a mismatch returns a counterexample and a diff. Once accepted, the acting phase runs the agent's controller: for each action the CWM predicts the next observation, the real game plays the action, and the two are compared. A match continues; the first mismatch adds the new transition to the dataset and returns to modelling. DATASET every transition observed so far (obs, action, next obs) replayed MODELLING The agent writes a Code World Model: State get_initial_state(obs) step(state, action) render(state) VALIDATION replays every recorded episode, step by step render(step(state, a)) == observation mismatch: counterexample and diff, fix the model and resubmit accepted: reproduces all the data ACTING match: keep acting controller act(state) CWM predicts the next observation real game plays the action =? compare first mismatch: the new transition joins the dataset, and the agent goes back to modelling The ModAct loop The dataset of observed transitions feeds the modelling phase, where the agent writes a Code World Model with a State, get_initial_state, step and render, and validation replays every recorded episode; a mismatch returns a counterexample and a diff. Once accepted, the acting phase runs the agent's controller: for each action the CWM predicts the next observation, the real game plays the action, and the two are compared. A match continues; the first mismatch adds the new transition to the dataset and returns to modelling. DATASET every transition observed so far replayed MODELLING The agent writes a Code World Model: State get_initial_state(obs) step(state, action) render(state) VALIDATION replays every recorded episode render(step(state, a)) == observation mismatch: counterexample and diff, fix the model and resubmit accepted ACTING controller act(state) CWM predicts the next observation real game plays the action =? match: keep acting first mismatch: the new transition joins the dataset, back to modelling
The ModAct loop. Modelling ends when the Code World Model reproduces every recorded transition; acting ends at the first observation it fails to predict, which becomes new data for the next round of modelling.

Two phases

model, then act, then model again

The agent starts from the game’s first observation. In the modelling phase it writes a Code World Model (CWM): Python code that represents the game’s state and predicts the effect of each action. The framework accepts the CWM only once it reproduces every transition recorded so far.

In the acting phase the agent writes a controller that plays the real game, and the CWM must predict each observation before it arrives. The first prediction that fails ends the phase: the game has done something the model does not explain, and the agent goes back to modelling with that new transition in its data.

The plain agent, called vanilla in Regact, also writes controllers, but plays and explores the game directly, with no model it has to keep consistent. It remains free to write predictive code; what ModAct adds is a world model that is required, explicit and checked.

Modelling

a program that fits all the data

A CWM is four definitions in the agent’s own files:

State
A compact representation of the game, usually a dataclass.
get_initial_state(obs)
Builds the State from the first observation of an episode.
step(state, action)
Predicts the State after one action, including what the screen does not show.
render(state)
Reconstructs the complete observation from the State.

To validate it, the framework replays every recorded episode in order: it builds the State from the first observation, applies step for each recorded action, and checks that render gives back exactly the observation that was recorded. Exactly means the whole observation: the frame, but also the reward, the end-of-episode flag, the available actions and the game’s metadata. The model must also be repeatable, and its States must stay small next to the observations they render, which pushes the agent towards rules rather than copies of the screen.

When validation fails, the agent gets counterexamples: where the prediction diverged (which episode, which step) and what differs between the predicted and the recorded observation, with differing grid cells summarised as regions. It repairs the model and resubmits until it fits. A refused model never replaces the last accepted one.

Because step carries the State from one action to the next, a CWM can track hidden state, such as a budget the screen only shows at a coarser resolution than it counts.

Acting

every action is a prediction

With an accepted model, the agent writes a controller: an object whose act(state) returns the next action, given the State maintained by the CWM. Each controller is an experiment, and its docstring says what it tests. The framework runs it in an isolated process, from a frozen copy of the code, and for each action:

  1. computes the CWM’s prediction of the next observation;
  2. plays the action in the real game and records the real transition;
  3. compares the two. On a match, the predicted State becomes the current one and the controller continues.

The first contradiction stops the controller. The action has really been played, so its consequence stays in the game and the transition stays in the data. The agent is back in modelling, with one more transition its model has to explain.

The model as a simulator

try a plan before playing it

Because the CWM is a program, it is also a local simulator. The agent can start it from the State the next controller will start from and step through a plan without spending a single real action. An optional breadth-first planner can search the CWM for an action sequence that reaches a goal the agent writes as a function of the State.

A plan found in the model is still only a prediction: it is played in the real game through a controller, under the same per-action checks.

What acceptance means

assumptions and limits

The design assumes a deterministic game whose first observation determines its whole state, hidden part included. If the same history ever leads to two different observations, the run stops: no model of this form could explain it.

An accepted CWM is consistent with everything observed so far. That does not prove its rules generalise, and a model can memorise cases instead of rules. Acting is where the model meets situations it has not seen, and where it gets corrected.

ModAct is implemented as the cwm protocol of Regact, next to the plain vanilla protocol, so the two can be compared with the same agents, games and budgets.