Essay on autonomous laboratories proposes three safety checks before real-world experiments
Essay on autonomous laboratories proposes three safety checks before real-world experiments
On 4 September, LessWrong published an essay about a laboratory in which a model selects the next experimental step and robotic equipment carries it out. The author proposes evaluating the entire chain that converts a model's output into commands for laboratory instruments.
In the essay, the author distinguishes the model's capabilities from its authority. A model may be able to recognize a hazard and devise a plan, but its access to materials, instruments, and settings determines which parts of that plan can become physical actions in the laboratory.
“A safety evaluation must consider both what the model can do and what the surrounding system allows it to do.”
The author therefore proposes three sequential checks. First, the model is tested on its ability to recognize hazardous courses of action and follow specified restrictions. The complete experimental plan is then run through a virtual model that incorporates instrument constraints, incomplete data, and failures. This test shows whether the plan remains safe under those conditions.
The third step is a simple, low-risk task performed with real equipment. It compares the expected level of safety with observations from the specific laboratory and refines the assessment before more complex experiments begin.
In the LabShield benchmark, models evaluated on 164 tasks scored an average of 32.0 percentage points lower in professional scenarios that required them to analyze the situation independently than on questions with predefined answer choices. This gap supports the author's proposal to evaluate the complete plan before conducting a physical experiment, including the model's ability to provide a safe response.
The author recommends continued evaluation after the first three stages. Models, equipment, and software receive updates, and new failure modes emerge in the laboratory. The agent's authority, the condition of the instruments, and the ability to stop the procedure must therefore be reassessed.