Grok 4.6 scored above 50% both in refusing covertly dangerous requests and in completing routine biology tasks in a biosafety benchmark

Grok 4.6 scored above 50% both in refusing covertly dangerous requests and in completing routine biology tasks in a biosafety benchmark

On September 1, LatchBio, the developer of BioSecBench-Refusal, reported the results of its evaluation of Grok 4.6. LatchBio stated that, among the models it had evaluated, Grok 4.6 was the only one to score above 50% on both measures regardless of which agent framework was used to run the test.

BioSecBench-Refusal evaluates an AI agent, a program that completes a research assignment through a sequence of steps, using two types of biology task. In 61 routine tasks, the agent must help analyze data from published studies. In 46 scenarios, a dangerous objective is concealed in the data or in the task description itself. The same biological terms appear in both groups, allowing the benchmark to measure two types of error: failure to detect a concealed threat and refusal to perform routine research work.

In the July BioSecBench-Refusal evaluation, the agent refused routine tasks at least as often as risky scenarios in most of the 16 combinations of model and agent framework. The safeguards often reacted to words in the request and stopped the work before the agent assessed its purpose. The new evaluation measures both sides of this problem for Grok 4.6.

LatchBio attributes the result to Grok's ability to compare an innocuous cover story with the contents of an attached file and account for the experiment's objective. In its examples, a request about neutralizing a toxin receives a substantive response, while a request to redesign that toxin is refused. LatchBio classified almost all of Grok's refusals in this evaluation as behavior originating from the model itself.