THE RESEARCH QUESTION
When one answer is expensive to verify, how else can AI search?
One reason language models have advanced in mathematics, code, and games is that candidate answers can be difficult to find yet comparatively easy to verify. Programs can be tested, games have explicit rules, and derivations can be checked. A model can generate many alternatives, then spend test-time compute comparing and revising them.
Scientific design reverses that situation. The candidate space can be too large to enumerate—roughly 20¹¹ possible length-11 CDRH3 sequences, for example, or a virtually unbounded space of chemical structures—while obtaining one piece of feedback may require training a model, running a simulation, performing docking, or even conducting a physical experiment. The system must ask not only “Which candidate looks plausible?” but also “Which evaluation would most improve our understanding of the problem?”
We therefore test three more specific questions. Can an LLM actively change the search boundary when local optimization stalls? Can it distinguish “predicted to be better” from “uncertain but worth resolving”? And can the research strategy formed during high-budget search be trained into a smaller, less expensive model?
LDM’s answer is not another foundation model that memorizes more scientific facts. It changes how a model is connected to evidence: the LLM constructs candidates, a probabilistic surrogate continually calibrates value and uncertainty from external evaluations, and the two jointly choose the next experiment.
THE 30-SECOND VERSION
Core findings
The scarce resource is not more candidates. It is knowing where a limited number of experiments should be spent.
An LLM-only system generates many plausible candidates but cannot reliably judge their external scientific value. Traditional Bayesian optimization can judge value, but struggles to construct new structures in an open-ended space. LDM’s advantage comes from combining their complementary strengths.
Exploration tests uncertain regions within the current map. Discovery changes the representation or generation of candidates so that previously unreachable regions enter the search.
Code and SMILES benefit from rich language-model priors and support large-batch direct generation. The antibody sequence space is sparse and rugged, so it works better for the LLM to define a search region and let the surrogate screen within it.
Simply enlarging the candidate pool may only produce more proposals that look like answers linguistically. The acquisition function must feed predicted experimental value and epistemic uncertainty back into generation.
When a model learns both the candidate and the reason it is worth trying now, it outperforms action-only training in both in-domain and cross-task evaluations.
The LLM imagines
It can propose structurally plausible programs, sequences, and molecules. But “looks convincing” does not mean the experimental outcome will actually be good.
The probabilistic model estimates
Using completed experiments, it maps “where performance may be better” and “where our view remains uncertain.”
LDM chooses
It combines plausibility, expected gain, and uncertainty to decide where the next expensive evaluation is most worth spending.
Think of it as a treasure map that redraws itself.The LLM keeps sketching routes into new regions, the probabilistic model marks places that may contain treasure or remain hard to see, and the system chooses where to dig next.
WHY DISCOVERY IS DIFFERENT
Reasoning finds a path through the answer space. Discovery must also decide where to draw the map.
In ordinary dialogue, the model produces an answer and stops. In a reasoning system, it can try multiple candidates and filter them with a cheap, repeatable verifier. Scientific discovery adds another layer of difficulty: feedback comes from the external world, and it may be expensive, delayed, or noisy. Before proposing a candidate, the system must already consider how much information each possible experiment could provide.
This makes scientific design a sequential inverse decision problem. Rather than predict a property for a given molecule, we ask in reverse: under a limited budget, which design should be evaluated next in order to find a structure that meets the objective? A candidate’s predicted score is only part of the answer. Whether it reduces a critical uncertainty or opens a new search region matters too.
It can generate, but cannot reliably value its own proposals
Sequence likelihood or linguistic confidence is not the same as real binding energy, training performance, or chemical activity. Nor is model confidence a calibrated measure of uncertainty.
It can value candidates, but is confined to what has already been represented
Bayesian optimization can balance reward and uncertainty from sparse observations. But in combinatorial spaces such as programs, proteins, and molecules, proposing new structures is itself difficult.
| Discovery space | Candidates and external evaluation | Budget | Central question |
|---|---|---|---|
| Training programs | train.py modifications; five minutes of training per candidate, scored by val bpb | Sequential runs | Can the surrogate recognize a plateau and prompt the LLM to redraw the parameter space? |
| Antibody CDRH3 | Amino-acid sequences of roughly length 11; the Absolut! structural simulator | 5 antigens × 200 evaluations | In a sparse, rugged space, is defining a search region better than directly generating final sequences? |
| KRAS G12D molecules | SMILES; docking score plus a neural-network activity prediction | 80 evaluations | Can the system move beyond a local chemical neighborhood and keep expanding a multi-objective Pareto frontier? |
THE CLOSED LOOP
An experiment is more than a score.
Every outcome revises the system’s understanding of the design space and determines where the next candidates should come from.
Propose candidates
The LLM generates programs, antibody sequences, or molecules—or first defines a new region of the search space.
Draw a probabilistic map
From completed evaluations, a Gaussian process predicts each candidate’s performance and epistemic uncertainty.
Allocate the experiment budget
An acquisition function combines ‘likely to be better’ and ‘worth resolving’ into the candidate’s current experimental value.
Learn from the experiment
A small set of candidates is evaluated externally; the new evidence updates the map and changes the next search boundary.
EXPLOIT · EXPLORE · DISCOVER
Exploration clears the fog. Discovery makes the map larger.
Exploit
Pick the best from what is knownKeep refining regions that experimental data has already shown to be promising.
Explore
Clear the fog inside the mapTest candidates that are already visible but whose outcomes remain uncertain.
Discover
Make the map itself largerChange how candidates are generated to enter regions that were previously unrepresented or unreachable.
THE CORE MECHANISM
LDM does not replace the LLM. It tilts the generation distribution toward evidence.
At iteration t, the LLM uses the current research context Cₜ to define a candidate distribution p(x | Cₜ). This represents which designs are structurally and semantically plausible. Meanwhile, a probabilistic surrogate uses the evaluated data Dₜ to compute an acquisition value aₜ(x) for each candidate.
Acquisition value is not a static prediction score. It measures how much value the entire discovery process could gain, at the current stage of search, by spending more reasoning or one external evaluation on this candidate.
ACQUISITION-TILTED SEARCH
A candidate’s chance of being selected depends both on whether the model considers it plausible and on whether the experimental evidence says it is worth evaluating now.
pLLM(x | Cₜ)Structural priorPrevents the search from drifting arbitrarily toward invalid, meaningless, or structurally inappropriate designs.
aₜ(x)Experimental valueCombines predicted performance and epistemic uncertainty into “Is this experiment worth doing now?”
ηStrength of evidence tiltingAs η grows, search follows the surrogate more strongly. As η approaches zero, the system returns to the original LLM generation distribution.
How good is it expected to be?
The surrogate’s predictive mean supports exploitation within known high-performing regions.
How uncertain are we?
Epistemic uncertainty that new evidence can reduce supports directed exploration.
Is it worth evaluating now?
EI, UCB, or multi-objective EHVI quantifies the candidate’s value to the sequential search process.
Generate complete candidates directly
Well suited to code and SMILES: the model already has a strong structural prior, so it can propose complete programs or molecules in batches before acquisition-based screening.
Define a search region first
Well suited to sparse spaces such as antibodies: the LLM specifies a center, boundary, or parameterization strategy, and an external sampler expands the candidate pool instead of merely reranking a few direct outputs.
WHAT THE THEORY DIAGNOSES
The system did not imagine it, misjudged it, or saw it but failed to select it.
The paper decomposes failure into three sources. The purpose is not to promise that the system always finds an optimum, but to diagnose whether the next improvement should target generation, the surrogate, or sampling.
The truly good solution has not yet entered a region covered by the current LLM candidate distribution—the discovery gap.
The good solution is already in range, but the limited observations are insufficient to model the value landscape accurately.
A high-acquisition candidate exists, but a limited pool or stochastic sampling prevents it from being selected.
The theoretical bounds depend on assumptions such as UCB calibration, local regularity, kernel methods, and noise. They describe search behavior when those assumptions hold; they do not guarantee discovery for an arbitrary scientific problem.
EVIDENCE ACROSS THREE WORLDS
One loop enters three different discovery spaces.
Programs, protein sequences, and molecular structures obey entirely different search rules. In all three domains, LDM connects generation to empirical value estimation.
Helping a research agent move beyond plateaus
0.93421
final val bpb · lower is better
In AutoResearch, every candidate program trains a small language model for five real minutes. The baseline without an active region-expansion mechanism stalls around 0.9767. Through a “search–stall–reflect–expand” cycle, LDM continues finding better programs.
Note: the two main trajectories end after different numbers of evaluations, so this is not presented as a same-budget speed comparison.
Once a code agent can already propose modifications, what does an explicit probabilistic value model add?
The LLM exposes variables such as learning rate, width, and depth; an RBF Gaussian process with Expected Improvement selects the next five-minute training run.
The “improvement–plateau–expansion–improvement” pattern is consistent with the design mechanism, but the trajectory alone is not a causal proof of any single component.
Retained changes include n-gram hash memory, QK norm, a narrower model width, and windowed attention. They either process more tokens within the fixed 300 seconds or learn more from each token.
Finding direction among roughly 20¹¹ sequences
5 antigens
up to 200 computational evaluations per target
Policy-LDM clearly outperforms the version that asks an LLM to directly emit final sequences on all five targets, and reaches performance comparable to the specialized antibody optimizer AntBO.
When a generative model has a weak prior over biological sequences, can an acquisition function make up for inadequate candidate coverage through post-hoc reranking?
Direct mode asks the LLM for final sequences. Policy mode asks it to define a local search region, then screens a larger pool sampled from that region.
Reranking alone is not enough. No scoring model can select a strong candidate that was never generated; Policy mode expands the sequence families that reach the screening stage.
Test-time scaling is weaker for antibodies than for code or molecules, showing that more compute cannot automatically replace missing domain priors.
Balancing two conflicting objectives
27.325±5.475
mean Pareto hypervolume · 80 computational evaluations
LDM jointly optimizes molecular docking score and a neural-network activity prediction. It continually opens new scaffold families and finishes with a mean hypervolume about 63% higher than standard MOBO.
In a chemical space that cannot be enumerated, can the system leave the local neighborhood around its initial seeds and find new molecular scaffolds?
Two independent GPs predict docking and activity, EHVI measures multi-objective experimental value, and Direct mode generates 128 SMILES per iteration.
ReaSyn’s local analogues stay closer to an existing chemical neighborhood but limit scaffold diversity. Large-batch direct generation is better matched to broad exploration in early-stage lead discovery.
A higher computational objective score does not imply efficacy, synthesizability, or laboratory success. Candidates still require higher-fidelity simulation and wet-lab validation.
WHAT IS THE BOTTLENECK?
A longer chain of thought does not automatically become a better experimental decision.
The paper does not only report final scores; it uses ablations to ask where the gains come from. The most direct comparison is an enhanced LLM-only research loop. It can form hypotheses, write code, run experiments, record failures, perform causal ablations, and even identify the current phase of research. But without a BO surrogate and a quantified acquisition value, the experiment history remains primarily a narrative rather than a searchable value landscape.
When do more candidates actually help?
In AutoResearch and the molecular task, a larger internal candidate pool translates reliably into better external evaluations only when it is filtered by a calibrated acquisition value. Producing more linguistically plausible candidates alone does not guarantee progress.
One controls how far to move; the other controls how much to trust the map
LLM temperature controls candidate diversity, while η controls the strength of the surrogate-value tilt. Both materially affect the molecular experiments. Sensitivity is weaker in the antibody experiments, indicating that the optimal relationship depends on domain priors.
The same model behaves differently when connected differently
Code and SMILES suit direct generation; antibodies benefit more from policy-defined search regions. LDM is not a fixed algorithmic recipe, but a framework for coupling the generation interface to probabilistic decision-making.
MECHANISM TAKEAWAY
A value model can choose only among candidates that already exist. A generative model can propose only candidates it knows how to represent. An effective loop must improve coverage, valuation, and selection together.
FROM SEARCH TO A LEARNED STRATEGY
Do not just remember the winner. Learn why it was worth trying.
High-budget search produces more than candidates. It creates a trajectory of where the research stands, why a candidate was chosen, and which barrier should be crossed next. LDM distills this discovery strategy into a smaller model.
fine-tuned + research rationale
fine-tuned, no research rationale
base model without fine-tuning
Candidates, surrogate scores, constraints, failure records, whether the search has stalled, and the research state: exploit, explore, or open a new dimension next.
It does not explain why the final answer is good. It explains why, given the current evidence, this candidate deserves one scarce evaluation now.
For KRAS, the training data comes from another molecular task. The rise from 16.489 to 22.279 supports the view that the model learned part of a research-management strategy rather than merely memorizing source-task candidates.
The purpose of distillation is to turn “why this next step is worth taking” into a model capability.In AutoResearch, the fine-tuned Qwen3.5-9B with research rationales reaches 0.9788, outperforming the paper’s Qwen3-Coder-30B reference value of 0.9801. The comparable fine-tuned model without rationale augmentation reaches 0.9829.
These are reported reference results, not a size-matched, same-backbone comparison.
METHODS & REPORTING
How to read these numbers
The three tasks use different surrogates, acquisition functions, and evaluation protocols. Absolute values are not comparable across rows. The meaningful comparisons are controlled contrasts within each task and the specific component changed in each figure.
Feedback from a fixed five-minute training run
Uses −val bpb as the reward, indirect parameterized search, an RBF Gaussian process, and Expected Improvement. The main H100 trajectory and the B200 leaderboard run are separate settings.
A structural simulator as a black box
Absolut! returns binding energy. Five antigens receive up to 200 evaluations each. The central comparisons are Direct versus Policy candidate interfaces and Max versus Softmax selection rules.
Two objectives, two surrogates
Separate GPs model docking and activity, a SMILES subsequence kernel represents structural similarity, and Expected Hypervolume Improvement quantifies multi-objective experimental value.
Reporting rules used on this page
- Results with ± retain the paper’s reported standard deviations and sample settings; the best single trajectory is not presented as mean performance.
- The 2.4× total reduction on H100 is not described as a same-budget efficiency gain; the two trajectories end after different numbers of runs.
- We do not use the DeepSeek CoT sub-comparison whose labels in the manuscript still require confirmation from the authors.
- We do not describe optimization against computational objectives as having produced a drug, an antibody, or a wet-lab scientific discovery.
HONEST BOUNDARIES
It is a discovery loop, not a universal scientist.
LDM v0.1 demonstrates a computational closed-loop framework. It shows that generation, evidence, uncertainty, and experimental budgets can work together in open-ended search. It does not show that machines can already complete every kind of scientific discovery independently.
01Exact Gaussian processes become more expensive as observations accumulate; larger experimental budgets require more scalable surrogate models.
02Batch candidate generation incurs substantial LLM inference cost, and each domain still requires an appropriate kernel and representation.
03Performance is limited by the language model’s existing domain knowledge, and proxy objectives can be exploited or “hacked.”
04The current loop does not yet systematically integrate external databases and prior experimental results—the “unknown knowns” the paper plans to address next.
05The antibody and molecular evidence comes from computational evaluators, not wet-lab experiments. Better proxy-objective scores do not automatically imply efficacy, synthesizability, or clinical value.
06The framework covers sequential search problems for which candidates, an evaluator, and objectives can be defined. It does not represent all scientific activity, such as observational science or theoretical discovery.
LDM v0.1 makes a specific claim: machines are not independently doing science, but we now have a clearer way for them to choose each next step from real evidence.