Hypothesis generation that survives scrutiny
Models can produce plausible ideas endlessly. The hard part is proposing experiments that are worth their compute and that fail informatively when they fail.
Research
Open problems worth automating
Building an automated lab and earning trust in one turn out to be the same job. These are the problems we are chewing on right now, written down before they are solved — which is the only honest time to write them down.
01 · Themes
01
02
03
04
05
06
Six problems that decide whether the loop works
Each of these is a place where today's systems are weaker than they look from a demo. Progress on any of them raises the ceiling for every domain the loop touches.
Experimental design under budget
Choosing what to run next when trials cost money and time: sequential decisions, early stopping, and knowing when a search has stopped being worth continuing.
Search efficiency
Getting more information per unit of compute. A loop that needs a thousand trials to find what a hundred could have found is a loop that will run out of budget.
Evaluating automated research
The metascience problem: how do you tell that an automated system is doing science, rather than generating activity that looks like it? Benchmarks here are immature, and we treat that as an opportunity.
Reliability over long horizons
Errors compound. A loop that runs for weeks needs to detect its own drift, recover from bad rounds, and not silently redefine success midway through a campaign.
Reproducibility at machine scale
Thousands of runs make provenance a systems problem: pinning environments, versioning data, and recording enough that any promoted result can be replayed later.
02 · Method
How we work, written down so it can be argued with
Automation makes it easy to generate results faster than anyone can evaluate their quality. These rules exist to slow us down in exactly the places where speed is dangerous.
- Objective first
- The metric, budget and failure condition are recorded before the search starts, and are not edited mid-cycle to flatter a result.
- Falsifiable claims
- If we cannot state what observation would prove us wrong, we are not making a claim yet.
- Ablation discipline
- Improvements are attributed to a cause we can remove and show the loss of — otherwise it is correlation with extra compute.
- Failures are first-class
- Negative results are stored, classified and reported with the same care as wins.
- Staged capability
- New autonomy is earned on a measurable task before it is allowed near a broader one.
- Internal replication
- Any result we intend to report is reproduced independently inside the team first.
Cycle log · example
IllustrativeObjective recordedpre-run
Baseline frozenyes
Ablation reproducedrequired
Independent replicationopen
03 · Publication
What we will publish, and what we will not
We intend to publish
- Method: how the loop is constructed and evaluated, including its failure modes.
- Measurement: harness design, baselines and the way we decide that something worked.
- Results that clear our internal bar of replication, with artefacts attached.
- Negative findings, because they save other teams the same compute.
We will not publish
- Numbers we cannot reproduce from a run record.
- Curated successes with the losses removed.
- Safety-relevant capability details that would be more useful to a misuser than to a defender.
- Anything framed as a breakthrough before independent eyes have seen it.
Research notes
This is where our notes, preprints and technical reports will appear. It is intentionally empty: AutoScientia was founded in 2025 and is in its first phase, building the loop before publishing about it. Rather than fill this space with marketing, we will leave it until there is work that meets the bar described above.
No publications yet
Founded 2025
04 · Collaboration
Research with people who can measure
The most useful collaborations for us are with groups that already own a slow experiment cycle and the instrumentation to score it. We bring the loop; the domain stays yours.
Academic groups
Joint work on a defined cycle, with clear authorship and no restrictions on publishing method. We are particularly interested in groups with strong measurement discipline.
Industry labs
Teams with an expensive internal experiment loop and a north-star metric. We start with a bounded campaign and a pre-agreed definition of success.
Infrastructure partners
Hardware, simulation and data partners who can widen the number of trials we can run or the fidelity with which we can score them.
Bring us a problem you can measure
If you can define the objective and the cost of one iteration, we can talk about turning it into a loop.