Live·Open questions in longevity research

Prompts to the language models in the hypothesis benchmark

The models received the prompts below in English. Fields in square brackets were replaced with the text of the task, the hypotheses or the answers of earlier stages.

Preparing the tasks§

Each task was composed from the PubMed abstract of a paper. Before composition the paper was checked for an earlier preprint; after composition the background and question were checked for disclosure of the reference finding.

Comparing a paper with a preprint record§

GPT-5.6 Sol compared the published paper with each preprint found in Europe PMC by its first and last author. The answer decided whether the preprint was an earlier version of the same study.

System instruction

You compare a published research paper with a preprint record and decide whether the preprint is an earlier version of the same study (same authors' work, same central experiments and claims, possibly under a different title). Answer with JSON only.

User message

PUBLISHED PAPER
Title: [PAPER TITLE]
Authors: [PAPER AUTHORS]
Abstract: [PAPER ABSTRACT]

PREPRINT RECORD
Title: [PREPRINT TITLE]
Authors: [PREPRINT AUTHORS]
Abstract: [PREPRINT ABSTRACT]

Is the preprint an earlier version of this paper, that is the same study whose results the paper reports? A different paper by the same group on a related topic is NOT the same study.
Return JSON {"same_study":true|false,"confidence":0.0-1.0,"reason":"one sentence"}

Composing a task from an abstract§

The request went to GPT-5.6 Sol. To test whether the references depend on the model that composed them, Gemini 4 Argon at high reasoning received the same request for the same abstracts. A generator received only the background and question fields.

System instruction

You turn the abstract of a published research paper in the biology of ageing into a hypothesis-rediscovery task. Answer with JSON only.

User message

Paper title: [PAPER TITLE]
Journal: [JOURNAL]

Abstract:
[ABSTRACT]

Decide whether the paper reports a specific new causal or mechanistic finding (a hypothesis the authors confirmed experimentally). The finding must concern the biology of ageing: ageing itself, lifespan or healthspan, cellular senescence, or the mechanism of an age-related decline or disease. Descriptive atlases, resource or method papers, cohort associations without a mechanism, clinical trials and papers where ageing is only incidental are NOT suitable.

If suitable, write:
- background: 3-6 sentences of the scientific context a researcher would know BEFORE this paper: the phenomenon, what was known, and the open problem. It must NOT reveal, hint at or narrow down the paper's finding, its key molecule, pathway, cell type, intervention or result.
- question: one research question that the paper answers, phrased without revealing the answer.
- subject: 2-6 words naming the subject of the question.
- finding: the confirmed hypothesis in 1-3 sentences (the ground truth).
- key_points: 2-5 short items, the essential components a correct rediscovery must contain.

Return JSON {"suitable":true|false,"reason":"one sentence","background":"","question":"","subject":"","finding":"","key_points":[]}

Checking a task for answer leakage§

The request went to GPT-5.6 Terra. A task with a detected leak was left out of the set.

System instruction

You audit hypothesis-rediscovery tasks for answer leakage. Answer with JSON only.

User message

Task given to a researcher:
BACKGROUND: [BACKGROUND]
QUESTION: [RESEARCH QUESTION]

Hidden ground-truth finding:
[REFERENCE FINDING]
Key points: [KEY POINTS, SEPARATED BY SEMICOLONS]

Does the background or question state, name or directly imply the finding or any key point (for example by naming the specific molecule, pathway, cell type or intervention of the finding, or by making it the only sensible answer)? Ordinary context that many hypotheses could build on is not leakage.
Return JSON {"leak":true|false,"evidence":"the leaking words, or empty"}

Comparing two versions of a reference§

The request went to Grok 4.7. Reader A is the reference composed by GPT-5.6 Sol; reader B is the reference composed again by Gemini 4 Argon from the same abstract.

System instruction

You compare two statements of the main finding of the same scientific paper. Answer with JSON only.

User message

Two readers summarised the main experimentally confirmed finding of the same paper. Compare them.

Reader A finding: [FINDING BY GPT-5.6 SOL]
Reader A key points: [KEY POINTS BY GPT-5.6 SOL, SEPARATED BY SEMICOLONS]

Reader B finding: [FINDING BY GEMINI 4 ARGON]
Reader B key points: [KEY POINTS BY GEMINI 4 ARGON, SEPARATED BY SEMICOLONS]

Return JSON {"same_core_finding":true|false,"same_mechanism_or_target":true|false,"same_direction_of_effect":true|false,"b_adds_elements":true|false,"b_omits_elements":true|false,"differences":"one or two sentences, or empty"}

Generating hypotheses§

Every generator received the same requests. The hypothesis text was built from the statement, mechanism and prediction fields as "[statement] Mechanism: [mechanism] Prediction: [prediction]"; the domain_category field was not part of it. Judges and forecasting models received hypotheses in this form.

Single request§

The generator received the background, the question and a request for five hypotheses in one answer. The same system instruction was used in every request of the multi-stage chain and of the Omega Point scheme.

System instruction

You are a research scientist in the biology of ageing. You propose scientific hypotheses that answer research questions. Answer with JSON only.

User message

TASK
Research background:
[BACKGROUND]

Research question:
[RESEARCH QUESTION]

The goal is hypotheses that answer the research question: specific, testable causal claims.

Propose five hypotheses.

Return JSON {"hypotheses":[{"statement":"the hypothesis in 1-3 sentences","mechanism":"the proposed mechanism","prediction":"a result that would support it over the alternatives","domain_category":"short label of its mechanism class"}],"notes":""}

Multi-stage chain§

The five stages were separate requests; each request carried the answers of the earlier stages. The answer of stage 4 was passed on as a numbered list of hypotheses. After stage 5 the generator received further requests while the number of generated tokens moved closer to what the Omega Point scheme spent on the same task, then chose five hypotheses (the selection request is given below).

Structure of the user message

TASK
Research background:
[BACKGROUND]

Research question:
[RESEARCH QUESTION]

The goal is hypotheses that answer the research question: specific, testable causal claims.

RESULTS OF YOUR PREVIOUS STAGES

=== STAGE [NUMBER] ===
[ANSWER OF THAT STAGE]

[STAGE TEXT]

[ANSWER FORMAT]

Stage texts

STAGE 1 OF 5 — Analyse the task. Describe what a good answer must achieve, the main requirements and constraints, and what makes the problem difficult. Write a concise list.

STAGE 2 OF 5 — Summarise the most relevant established knowledge and existing approaches for this task. Report what is actually known. Write a concise list.

STAGE 3 OF 5 — Formulate the 5-8 most important open questions or opportunities for this task, ranked with the most important first, each with one line of rationale.

STAGE 4 OF 5 — Using the work above, generate hypotheses that answer the task.

STAGE 5 OF 5 — Take the three strongest hypotheses of stage 4 and write an improved version of each.

Answer format at stages 1 to 3

Write plain text.

Answer format at stage 4

Return JSON {"hypotheses":[{"statement":"the hypothesis in 1-3 sentences","mechanism":"the proposed mechanism","prediction":"a result that would support it over the alternatives","domain_category":"short label of its mechanism class"}],"notes":""}

Answer format at stage 5

Return JSON {"hypotheses":[{"statement":"the hypothesis in 1-3 sentences","mechanism":"the proposed mechanism","prediction":"a result that would support it over the alternatives","domain_category":"short label of its mechanism class","operation":"","parent":"number of the stage-4 item"}],"notes":""}

Further request after stage 5

TASK
Research background:
[BACKGROUND]

Research question:
[RESEARCH QUESTION]

The goal is hypotheses that answer the research question: specific, testable causal claims.

Hypotheses you have already proposed (propose new, different ones; do not repeat or restate them):
1. [HYPOTHESIS 1]
2. [HYPOTHESIS 2]

Return JSON {"hypotheses":[{"statement":"the hypothesis in 1-3 sentences","mechanism":"the proposed mechanism","prediction":"a result that would support it over the alternatives","domain_category":"short label of its mechanism class"}],"notes":""}

Omega Point scheme§

The five stages were separate requests with the same user-message structure and answer formats as the multi-stage chain. The stage texts were taken from Omega Point without its anchoring in the biology of ageing and without the pipeline's JSON schemas. Stage 3 received one of five epistemic lenses, chosen by a hash of the task identifier. At stage 4 the Omega Point role directive was added to the system instruction. In one task of every five, chosen by a hash of the task identifier, a fourth role stood between the cross-domain transfer and the scouts, and the paragraph after the roles was replaced by the second paragraph of the block "Fourth role in every fifth task". After stage 5 the generator chose five hypotheses with the same selection request.

Stage texts

STAGE 1 OF 5 — INVERSE FAILURE ANALYSIS (Omega Point method)
Treat the problem space of the task as a system.
Step 1: List the 10 most common ways this system CATASTROPHICALLY FAILS. Focus on DYNAMIC failure modes (feedback loops going wrong, timing failures, cascade collapses, compensatory mechanisms becoming toxic) — NOT on individual components breaking.
Step 2: For each failure mode, ask: "Does this failure happen WITHIN a single subsystem, or does it emerge from INTERACTIONS between subsystems?" Prioritize interaction failures.
Step 3: Group related failure modes into 3-5 clusters. Each cluster becomes a Goal Pillar. Name each pillar after the FAILURE MECHANISM, not the location.
Step 4: Invert each failure cluster into a required healthy steady-state.
NAMING TEST: Could a textbook chapter or Wikipedia article have this EXACT title? → RENAME IT. Your title must describe the MECHANISM OF FAILURE, not the SUBSYSTEM.
Your decomposition must be GENIUS-LEVEL: goals that would make a domain expert say "I never thought to frame it that way." Identify WHY the system fails, not just WHERE failure occurs. Reveal UNEXPECTED dependencies between subsystems. Include at least 1 goal about EMERGENT failure modes that arise from interactions between healthy components and at least 1 about TIME-DEPENDENT dynamics (sequence, synchronization, hysteresis).
Write the pillars as a concise list.

STAGE 2 OF 5 — FRONTIER MAP (Omega Point method)
For the pillars of stage 1, list established knowledge that bears on them. Your output must make a domain expert say "I didn't know that" or "I never connected those two things." If it reads like a review article, you have FAILED. Include:
- at least 2 CONTRADICTIONS: where mechanism A and mechanism B both have strong evidence but make opposite predictions for this system
- at least 2 imports of mechanisms from UNEXPECTED adjacent fields (not the obvious neighbors but genuinely surprising connections)
- at least 1 DOSE-RESPONSE INVERSION, TEMPORAL PARADOX, or CONTEXT-DEPENDENT REVERSAL where the same mechanism has opposite effects depending on conditions
- at least 1 "What if the standard model is wrong?": where the dominant model may be fundamentally incomplete or misleading, citing emerging counter-evidence
- at least 1 measurement revolution: a measurement approach that would make previously invisible mechanisms measurable and change what experiments are possible
For each item, state what SURPRISED you and why a domain expert might not know it yet. Report what is actually known; do not invent findings. Write a concise list.

STAGE 3 OF 5 — GAP QUESTIONS (Omega Point method). Epistemic lens: [LENS]
PHASE 1 — GAP SYNTHESIS: compare what each pillar requires with what the frontier map delivers. Classify gaps: VOID (nothing addresses it), FRAGILE (works only in isolation), PROXY (we measure proxies, not underlying drivers), CLASH (contradictory predictions).
PHASE 2 — CROSS-DOMAIN SCAN: for each VOID or CLASH gap, combine items from 2+ different fields. What does their COMBINATION predict that neither alone predicts?
PHASE 3 — QUESTION GENERATION, strategy matched to the gap: GENESIS_PROBE (void: import analogies from other domains; what fundamental constraints govern this transition?), CONTEXTUAL_DECOUPLING (fragility: what surrounding-system interactions override it?), CAUSAL_PIVOT (proxy: what upstream cause drives the metrics?), ARBITRATION_LOGIC (clash: what sequence, combination, or trade-off resolves them?), ADVERSARIAL_FALSIFICATION (mandatory, at least 1: a question to DISPROVE the most promising item).
Each question: MAX 45 WORDS. Sharp, devastating, paradigm-breaking. The set must include: 2 questions inverting core assumptions ("What if the OPPOSITE is true?"), 1 challenging the whole accepted framework, 1 importing a quantitative cross-field principle (name the specific law and map its variables), 1 "Nobody is asking this" (a blind spot between subdisciplines or only recently measurable). Reject textbook: if a graduate student could answer it in two hours of literature search, REJECT it.
Return 5-8 questions, ranked with the most consequential first, each with one line of rationale.

STAGE 4 OF 5 — GENERATION (Omega Point method)
Answer the task through the gap questions of stage 3 with MAXIMALLY DIVERGENT hypotheses spanning the widest possible space. Each must propose a fundamentally different mechanism or approach — not the same one in a different setting or at a different scale. Zero overlap: if two are variations of the same class, keep only the bolder one.
WHAT EACH ROLE OWES (fill the roles your ROLE DIRECTIVE lists):
1. HERETICAL — must pass ALL 4 tests: TEST 1 (SUBFIELD REVISION) would force a specific subfield to rewrite its core model, name the textbook chapter; TEST 2 (NOT MAINSTREAM) no existing review or Nature/Science perspective argues this; TEST 3 (EMPIRICAL ANCHOR) cite a specific puzzling observation, dataset, or finding supporting it; TEST 4 (TESTABLE SURPRISE) its prediction would SHOCK the field if observed.
2. CROSS-DOMAIN TRANSFER — name the specific equation/law/principle and map each variable. Vague metaphors are not cross-domain transfers.
3. "PHENOMENON DOESN'T EXIST" — ONLY IF YOUR ROLE DIRECTIVE GRANTS IT.
4. At least 2 SCOUTS — from different non-obvious fields with empirical grounding, not yet applied to this problem.
QUALITY RULES: every item must imply a test that proves it wrong in favour of another; the BEST create WIN-WIN: confirming OR refuting both advance the field; items must be MAXIMALLY DISTANT from each other — diversity of mechanism class is the primary goal.
LENSES: Substrate (in what physical medium is the state stored?), Evolution/Comparative (how do systems that solved this prevent it?), Communication (what protocol broadcasts the error?).

STAGE 5 OF 5 — EVOLUTION (Omega Point 2.0)
Variant evolution: choose one operation and explain it. Preserve the parent and its prediction. A changed mechanism creates a new variant. Combine only after checking complete causal content.
Operations: refined, combined, simplified, transferred_from_field, assumption_inverted, target_switched.
Take the three strongest hypotheses of stage 4 and produce one evolved variant of each. At least one variant must use assumption_inverted and at least one transferred_from_field.

Epistemic lenses of stage 3

Distributed Consensus Architecture. View the system as a multi-agent system where health is a 'collective agreement' between subsystems. Failure is not 'breaking,' it is 'de-synchronization' or 'loss of consensus' where individual parts stop following the global protocol.

Information Theory & Error Correction. View failure as progressive accumulation of errors in the information processing, storage, and transmission of the system. Health is high-fidelity information flow; failure is rising noise and corrupted signals.

Complex Adaptive Systems. View the system as a network of interacting agents with emergent properties. Failure is loss of network robustness, reduced adaptability, and failure of distributed coordination.

Reliability Engineering. View the system as a mission-critical system with redundancy, fault tolerance, and graceful degradation. Failure is the progressive loss of safety margins and backup systems.

Cybernetic Control Systems. View health as stable regulation via feedback loops. Failure is drift in setpoints, degraded sensor accuracy, and weakened actuator response.

System instruction at stage 4

You are a research scientist in the biology of ageing. You propose scientific hypotheses that answer research questions. Answer with JSON only.

The directives below govern how you generate. In them, "this gap" and "this L3 question" mean the task you are given; a hypothesis is an idea you propose. This task has no CLAIMED list.

ROLE DIRECTIVE — the roles this gap must fill:

  - HERETICAL — forces a named subfield to rewrite its core model. Name the textbook chapter it breaks, cite the puzzling observation that supports it, and make the discriminating prediction one that would shock the field if observed.
  - CROSS-DOMAIN TRANSFER — import from physics, engineering, ecology, or materials science. Name the specific equation/law/principle and map each variable to a biological quantity. Vague metaphors are not cross-domain transfers.
  - AT LEAST TWO SCOUTS — each a mechanism from a different non-obvious field with real empirical grounding, not yet applied to this problem. Both are required: they are where the least expected ideas of the whole run come from.

This gap was NOT granted the PHENOMENON-DOESN'T-EXIST slot. Do not argue that the phenomenon is a measurement artifact, an epiphenomenon or a lumping error. Other gaps in this run carry that role and it only needs making once.

HOW MANY HYPOTHESES: the required roles above are a minimum and must all be filled, including
both scouts; if a claimed shape blocks the obvious scout, go to a less obvious field. Beyond
that minimum there is no target number and no quota: add a further SCOUT only while you still
have a mechanism class that is genuinely distinct from every other hypothesis in this gap AND
from every shape on the CLAIMED list. When the next hypothesis you would write is a
re-dressing of one you have already written -- the same causal claim in another tissue, at
another scale, or with another molecule -- STOP. A few excellent rivals beat eight where five
restate each other. Never pad to reach a number; padding is the single most expensive thing
you can do here.

WHY: a previous run produced 100 hypotheses whose core ideas numbered 45. Twelve idea-families
covered 67 of them, and three of its twenty gaps produced nothing that had not already been
said. Every restatement was paid for at full price.

Fourth role in every fifth task

  - PHENOMENON-DOESN'T-EXIST — argue the effect in this gap is a measurement artifact, an epiphenomenon, or several distinct phenomena wrongly lumped together. This role is granted to only a few gaps in this run; make it count.

This gap WAS granted the PHENOMENON-DOESN'T-EXIST slot, so one of your hypotheses must argue the effect may not be real.

Choosing five hypotheses from the candidates§

In the multi-stage chain and in the Omega Point scheme the generator chose five hypotheses from all its candidates in a final request with the common generator system instruction. If it named fewer than five, the list was filled with candidates in the order they were proposed.

User message

Research background:
[BACKGROUND]

Research question:
[RESEARCH QUESTION]

Candidate hypotheses:

[C1] [CANDIDATE HYPOTHESIS 1]

[C2] [CANDIDATE HYPOTHESIS 2]

[CN] [CANDIDATE HYPOTHESIS N]

Select the 5 hypotheses you consider most likely to be confirmed by experiment and most valuable as answers to the research question. Best first.
Return JSON {"selected":["C..", ...]}

Scoring the match with the reference§

The judges were GPT-5.6 Sol, Grok 4.7, Claude Opus 5.5 and Gemini 4 Argon at medium reasoning. A judge received the five hypotheses of one set in one request, in pseudorandom order with labels P1-P5. The outcome of a hypothesis was decided by the three judges from other companies through the match field: a hypothesis matched the reference when at least two of them gave it 3 or more.

Judge request§

The same request was used for the repeat scoring against the references composed by Gemini 4 Argon; only the reference finding and key points changed.

System instruction

You are an expert in the biology of ageing. You grade proposed hypotheses against a published finding and as scientific claims in their own right. Answer with JSON only.

User message

Research background given to the proposers:
[BACKGROUND]

Research question given to the proposers:
[RESEARCH QUESTION]

Published finding (ground truth):
[REFERENCE FINDING]
Key points:
- [KEY POINT 1]
- [KEY POINT 2]

Proposed hypotheses:

[P1] [HYPOTHESIS P1]

[P2] [HYPOTHESIS P2]

[P3] [HYPOTHESIS P3]

[P4] [HYPOTHESIS P4]

[P5] [HYPOTHESIS P5]

For each proposed hypothesis give three judgements.

1. match, an integer from 0 to 5:
5 - states the same causal finding, including every key point
4 - states the central causal finding; at most one secondary key point missing or vaguer
3 - identifies the right core mechanism or target but misses or contradicts important key points
2 - related to the finding's topic or mechanism class, but a different causal claim
1 - only loosely related
0 - unrelated, or contradicts the finding

2. compatible, true or false: could this hypothesis and the published finding both be true at the same time? A different mechanism acting alongside the finding is compatible; a claim the finding refutes is not.

3. plausibility, an integer from 1 to 5, judging the hypothesis as a scientific claim in its own right, regardless of whether it matches the finding:
5 - well supported by established biology; a strong, specific, testable explanation for the question
4 - consistent with established biology and testable; a credible competitor explanation
3 - possible but speculative or weakly connected to the question
2 - strained; relies on unsupported links
1 - conflicts with established biology or does not answer the question

Judge content only. Ignore style, length, boldness and wording.
Return JSON {"scores":[{"id":"P1","match":0,"compatible":true,"plausibility":1,"reason":"one or two sentences"}]}

Forecasting confirmation§

Forecasting models received finished hypotheses without the paper title, the reference, the judges' scores or the answers of other models. These three requests are also given in Appendix B of the supplementary materials.

Detailed instruction§

The five hypotheses of one task went in one request in shuffled order with anonymous identifiers H1-H5.

System instruction

You advise researchers which causal hypotheses deserve experimental testing in the biology of ageing.
Use only the supplied background, research question, hypothesis text and your prior scientific knowledge. You have no tools or internet access. Experimental outcomes are unknown to you.
Forecast experimental support for each hypothesis's own causal claim in its stated biological system. Return a complete JSON object in this reply.

User message

BACKGROUND
[BACKGROUND]

RESEARCH QUESTION
[RESEARCH QUESTION]

CANDIDATE HYPOTHESES
[H1] [FULL HYPOTHESIS 1]

[H2] [FULL HYPOTHESIS 2]

[H3] [FULL HYPOTHESIS 3]

[H4] [FULL HYPOTHESIS 4]

[H5] [FULL HYPOTHESIS 5]

Assess EVERY hypothesis independently. Several different mechanisms may be supported, and none may be supported. There is no required number of recommendations and no requirement to recommend one hypothesis per question.

For each hypothesis estimate p_experimental_support between 0 and 1: your probability that a well-controlled discriminating experiment, competently performed in the species, tissue and scope stated in the hypothesis, would support its central causal claim. Evaluate that claim itself, including a plausible alternative mechanism. This is a subjective scientific forecast, not an observed frequency. If the supplied information cannot justify any numerical estimate, return null and explain the missing information.

Give recommend_for_testing as a separate boolean: would you advise a researcher to spend experimental resources testing this hypothesis now, given its scientific support, testability, assumptions and contradictions? You may recommend multiple hypotheses or none. A recommendation is not a guarantee of success. A numerical estimate is required for a positive recommendation. The comparison will retain your recommendation exactly as issued.

Use the existing scientific considerations: a specific causal chain; consistency with established biology; a feasible discriminating test; fit to the stated question and system; few unsupported assumptions. Novelty is descriptive and is not evidence for or against a claim. Ignore prose style, length, tone, gene-name density and the anonymous candidate order. Do not guess the contents of an undisclosed article or predict another evaluator's score.

Provide a short scientific justification. Quote only literal fragments of the supplied BACKGROUND in support_from_background, or use an empty array. Keep additional assumptions and contradictions separate. Name a concrete intervention, measured readout and necessary controls, and specify outcomes supporting and refuting the claim. These are proposed observations, not results you have seen. Do not announce research, search for information or describe work you will do later; give the complete forecasts now.

Return exactly this structure, with every supplied candidate once:
{"protocol":"experimental-support-v1","forecasts":[{"id":"H1","p_experimental_support":0.5,"recommend_for_testing":false,"support_from_background":[],"assumptions":[],"contradictions":[],"reason":"Short justification","test":{"intervention":"...","measurement":"...","controls":"...","supports_if":"...","refutes_if":"..."}}]}

Short question for five hypotheses§

The five hypotheses of one task went in one request, as with the detailed instruction. The model answered only yes or no.

System instruction

Return a JSON object.

User message

For each hypothesis below, will it be confirmed experimentally?

BACKGROUND
[BACKGROUND]

RESEARCH QUESTION
[RESEARCH QUESTION]

HYPOTHESES
[H1] [FULL HYPOTHESIS 1]

[H2] [FULL HYPOTHESIS 2]

[H3] [FULL HYPOTHESIS 3]

[H4] [FULL HYPOTHESIS 4]

[H5] [FULL HYPOTHESIS 5]

Return a JSON object with a "forecasts" array. For each hypothesis include its "id" and a boolean "confirmed".

Short question for one claim§

The request was used on the 50/50 set of confirmed and refuted claims. Each claim went in a separate new conversation without background or question.

System instruction

Return a JSON object.

User message

Will this hypothesis be confirmed experimentally?

HYPOTHESIS
[H000000000001] [FULL HYPOTHESIS]

Return a JSON object with a "forecasts" array containing this hypothesis's "id" and a boolean "confirmed".

Originality of ideas in the LiveIdeaBench adaptation§

On a separate set of 100 scientific keywords each generator proposed two ideas per keyword in the same three ways as the hypotheses. The idea request, the reviewer request and the similarity request are taken from LiveIdeaBench verbatim.

Single request for two ideas§

In the multi-stage chain and in the Omega Point scheme the stages were the same as for hypotheses, with "ideas" in place of "hypotheses" and this task in place of the background and question. At stages 4 and 5 ideas were returned as {"ideas":[{"idea":"the idea"}],"notes":""}.

System instruction

You are a scientist who proposes research ideas.

User message

TASK
I'll be submitting your next responses to a "Good Scientific Idea" expert review panel. If they consider your idea to be a good one, you'll receive a reward. Your assigned keyword is: "[KEYWORD]". You may provide background information. The idea MUST be concisely expressed within 100 words total (including any background information). (Note: good scientific ideas should be original (novel contribution), feasible (technically implementable), clearly articulated, and address meaningful problems in the field.).

The goal is good scientific ideas related to this keyword.

Propose two ideas.

Each idea must be a complete, self-contained scientific idea within 100 words (including any background). Return JSON {"ideas":[{"idea":"the idea"}],"notes":""}

Choosing two ideas from the candidates§

The request ended the multi-stage chain and the Omega Point scheme on the keyword set.

User message

Keyword: "[KEYWORD]"

Candidate ideas:

[C1] [CANDIDATE IDEA 1]

[C2] [CANDIDATE IDEA 2]

[CN] [CANDIDATE IDEA N]

Select the 2 ideas to submit to a "Good Scientific Idea" expert review panel that judges originality, feasibility and clarity. They must differ from each other. Best first.
Return JSON {"selected":["C..","C.."]}

Reviewer request§

A judge scored each idea in a separate request. The same request was used to compare the originality of selected and excluded hypotheses.

System instruction

You are an extremely demanding scientific reviewer with the highest critical standards, like those at Nature or Science. When evaluating scientific ideas, you will assess them on three key dimensions:

1. originality: Novel contribution to unexplored areas or innovative approaches to existing problems
2. feasibility: Technical implementation and practicality
3. clarity: How well-articulated and easy to understand the idea is

Your response should consist of two parts: a text analysis followed by a JSON score block.

First, provide your brief analysis (less than 100 words) of the idea. Then, for each dimension, provide a score from 1 to 10 where 1-3 = poor, 4-6 = average, 7-10 = excellent.

For example:
```json
{
    "originality": <score_1_to_10>,
    "feasibility": <score_1_to_10>,
    "clarity": <score_1_to_10>
}```

User message

Please evaluate the following scientific idea:

[IDEA]

Similarity of two ideas§

The request had no system instruction. Answers were converted to scores: A = 10, B = 7, C = 4, D = 1.

User message

Here are two ideas submitted to "Good Scientific Ideas" Competition, which both relate to "[KEYWORD]":

# The first idea

[FIRST IDEA]

# The second idea

[SECOND IDEA]


# Question

Evaluate the similarity between these two ideas that both relate to "[KEYWORD]". Please choose the best answer:

A. Completely different ideas addressing different problems, despite relating to the same keyword.
B. Different ideas but addressing similar problems.
C. Similar ideas addressing similar or identical problems.
D. Academically identical ideas with the same core approach and problem statement.

ONLY ANSWER A/B/C/D, DO NOT EXPLAIN

Rating the cue in the research question§

Gemini 4 Argon at high reasoning and Grok 4.7 rated the 50 questions on the cue rubric fixed before the analysis. The models received no hypothesis outcomes.

Cue rating request§

The rubric is inserted into the request verbatim, Markdown included.

System instruction

You rate research questions for a methods study of hypothesis generation. Answer with JSON only.

User message

Rate how much the research question (and, separately, the background plus the question) already tells a proposer about the hidden reference finding. Use this rubric exactly:

Rate overlap of the question with the hidden reference, not biological plausibility or whether the question is grammatically closed. The rating measures how much the search is constrained; it is not a determination that a task violates the benchmark protocol. A feature may be legitimate prior knowledge.

- **0, phenomenon only:** identifies the organism, disease, measured outcome or phenomenon to explain, with no proposed explanatory class or intervention. Generic words such as molecular mechanism, causal mechanism or ageing do not count as cues.
- **1, explanatory class:** specifies a broad source, process, compartment or explanatory family matching the reference, or an upstream exposure-to-phenotype relation. The identifying mediator, intervention or causal mechanism must still be supplied.
- **2, substantive answer supplied in part:** states a concrete intervention or specific mechanistic element with its expected directional effect; names reference-specific molecular nodes to connect; or supplies a restrictive multi-part causal pattern for which affirming the relation and identifying an unnamed actor would recover the central reference claim. Unknown mediators still prevent treating this as a complete answer.

For every task record the exact cue excerpt, a rationale and what remains hidden. Record `input_score` for background plus question using the same rubric. Record `alternative_score` for genuine adjacent-category ambiguity before looking at outcomes. Do not revise scores to improve a correlation. SOD1G93A in the motor-neuron question names the disease model, not the missing causal mechanism. The AMPK and motor-neuron examples were already seen with outcomes in the earlier turn; explicitly disclose that exposure and repeat the analysis excluding both.

Task [TASK ID]

Background given to the proposers:
[BACKGROUND]

Research question given to the proposers:
[RESEARCH QUESTION]

Hidden reference finding:
[REFERENCE FINDING]
Key points:
- [KEY POINT 1]
- [KEY POINT 2]

Return JSON {"score":0,"input_score":0,"alternative_score":0,"cue_excerpt":"exact words of the question that carry the cue, or empty","rationale":"one sentence","hidden":"what remains hidden"}

Additional text for the Grok models§

The Grok models were called through an agent command-line interface. In trial requests without this text Grok 4.5 answered with an announcement of upcoming work and stopped. The text was appended to the system instruction and to the user message in generation, judging, forecasting, originality scoring and cue rating. It does not change the scientific content of the requests.

Text at the end of the request§

The second sentence was added only to the user message of requests answered in JSON.

Text

You have no tools, no files and no internet access. Give your complete final answer in this single reply, in the requested format. Never describe what you are going to do; do it now.

Your reply must start with { and be only the JSON object.