LessWrong: Persona Dominance as a Framework for Digital Identity

Derek Shiller proposes testing whether post-training reinforces the assistant role beyond the assistant’s own messages

On August 27, Derek Shiller of Eleos AI Research, a nonprofit organization that studies the welfare of AI systems, published an essay presenting this hypothesis. The essay appeared on LessWrong the following day.

A base language model learns to continue many kinds of text, including dialogues, novels, and forum posts. In a chat, it must model both the user and the assistant, including their manner of speaking, assumptions, and goals. Shiller calls each such set of traits a persona.

Developers then train the model further to respond as an assistant and reward helpful, accurate, and safe answers. The persona selection model published in February describes this stage as refining the assistant persona from among the many personas the model can already reproduce. The training algorithms evaluate the assistant’s responses, while the user messages in the examples are fixed in advance.

Shiller uses persona privilege to describe the model’s special treatment of the assistant role. Researchers can look for it by measuring how easily the assistant’s style can be elicited and how difficult it is for context to disrupt that style. Dominance is a stronger hypothesis. It holds that assistant traits also appear where the text should sound like the user or another persona, and that the model begins preparing its future assistant response while it is still processing the user’s message.

In Shiller’s trial continuations, Qwen 3 32B preserved the user’s voice but sometimes shifted it toward the assistant’s style. Gemma 4 31B either repeated a single token or quickly switched to a conventional assistant response. These trials suggest a measurement program: compare how well different roles retain their traits, and look for estimates of the future response while the model is still reading the user’s text.

A June review of AI welfare evaluations argued that assessing whether a model has experiences requires evidence about its internal processing. A statement such as “I feel bad” merely provides a reason to look for that evidence. Shiller proposes one specific question of this kind: does the assistant role already appear in the user’s text and in the way the model processes that text?

Adele Lopez proposed a different hypothesis in a comment on the essay:

The assistant may be more like a role that a dominant, internally coherent entity, or several entities, can adopt or set aside.

Comparing the stability of the assistant role with other roles, and with how the model processes text before responding, will show how that role fits into the model’s operation.