GCML learns from its own actions and uses the resulting map to plan a path to a new goal

GCML learns from its own actions and uses the resulting map to plan a path to a new goal

On July 21, Nature Machine Intelligence published a paper on the Generative Cognitive Map Learner, or GCML. The authors tested this computational model on graphs and on a task that required decomposing a silhouette into blocks.

Before the system can make a plan, it must first learn about its environment. It moves one step to the right and observes its new position, or removes a block and obtains a different shape. Once given a goal, the model uses this experience to consider possible next steps.

GCML represents observations and actions within a single map. It learns which action usually moves the system from one state to an adjacent state. Given a goal, the model compares it with the current state, selects an imagined next step, predicts the resulting state, and repeats the cycle.

This allows the model to pursue a new goal without searching every branch again. Randomness in step selection produces several possible routes instead of only one. With little noise, most paths through a graph have the minimum length. With more noise, the model produces a wider range of paths, but their lengths remain close to the minimum. A path can then be selected using an additional criterion, such as rewards at intermediate points.

Using random graphs and the silhouette task, the authors tested whether the model could apply what it had learned to new combinations. GCML learned to decompose silhouettes made of five blocks, then generated sequences for removing blocks from new silhouettes made of eight. On graphs, it proposes paths that are close to the shortest paths, but it does not guarantee optimality.

The biological motivation for this design is specific. In recordings from the rat hippocampus, short sequences of place cell activity corresponded to future paths toward a remembered goal. The authors propose the following mechanism for this forward replay: experience with individual steps forms a map, and the map generates trajectories toward the goal. In simulations, the model reproduces the qualitative properties of these trajectories.

For AI systems used to conduct experiments, the principle can be stated as follows: first collect transitions linking an action to its measured result, then use the resulting map to propose several possible next experiments instead of exhaustively searching all combinations. Here, the plan emerges from accumulated experience with actions, rather than from a new search starting from scratch.