What we mean by “autonomous experimentation”

The four stages

We use the phrase to mean one specific thing: a system owns the full experimental cycle against an objective that was fixed outside it. The cycle has four stages, and each has a precise meaning in our usage.

Propose: Turn an objective and the accumulated results into the next set of candidates, ranked by expected value against cost — not a list of plausible-sounding ideas.
Run: Execute those candidates in parallel on real infrastructure, in isolated environments, under enforced resource and spend ceilings.
Evaluate: Score each result automatically against a baseline that is frozen for the duration of the campaign, and record the failures rather than discarding them.
Iterate: Use the scored results to change what gets proposed next, and start the next round without waiting for a human to authorise the transition.

The essential word in that list is iterate. A system that proposes and runs experiments but does not let results change what it proposes next is not running a loop. It is running a search over a list somebody else wrote.

What the machine owns, and what stays human

The loop owns

Repetition and scale

Generating and ranking candidate experiments.
Scheduling and executing thousands of trials.
Scoring results and preserving the failures.
Deciding the next round, within a fixed objective.

People own

Meaning and authority

Choosing which problems deserve a campaign at all.
Writing the objective, the budget and the stopping condition.
Interpreting what a result means outside the system.
Approving anything with an effect beyond the sandbox.

This split is not a temporary state of the technology. It is the design. An objective chosen by the system that is being judged against it is not an objective; it is a preference.

Three tests before the word fits

Before we call any cycle autonomous — ours included — it has to pass three tests. They are deliberately strict, because the failure mode here is not under-claiming.

No human in the inner loop. A person is not required between proposing, running, evaluating and re-proposing. Human approvals exist at the boundaries (objectives, budgets, external effects), not between steps.
The objective is fixed from outside. The metric, the baseline and the stopping condition are set before the campaign begins and cannot be edited mid-flight by the system that benefits from editing them.
Results measurably change behaviour. If deleting the evaluation step would not change what the system does next, then evaluation is decorative. We check this by ablation, not by assumption.

Two misconceptions worth naming

Most disagreement about this word comes from two assumptions that sound reasonable and are both wrong in ways that matter operationally.

“Autonomy means unsupervised.” It does not. A cycle can be fully automated between its steps and still be surrounded by human authority: the objective is set by a person, the budget is set by a person, and any action with an effect outside the environment requires a person. Autonomy describes what happens inside a boundary; oversight defines the boundary. Systems that conflate the two end up either over-claiming what they can do, or under-equipping the people responsible for it.

“Autonomy is a one-shot transition.” It is not a switch that gets flipped for an organisation. It is a property of a particular cycle in a particular domain, and it can hold for one campaign while failing for the next. A recipe search may run closed for weeks; an exploration of a genuinely new architecture may need human judgement at every turn. Both can be true in the same system on the same afternoon.

Because of that, we describe autonomy per campaign rather than per company: which stages closed, against which objective, under which ceilings, and what a human still had to touch. A vaguer description would be easier to write and impossible to audit.

Four things we do not call autonomous

Common usage	What it actually is	Why the distinction matters
“The model designs experiments”	A proposal step with a human doing everything after it	The loop still turns at human speed; no throughput change follows
“We run automated sweeps”	Open-loop grid search: a fixed list, run in bulk	Later trials cannot benefit from earlier results, so compute is spent blindly
“It writes the paper”	Text generation against an existing result	Writing is not experimenting; no new measurement is produced
“It self-improves”	An unexplained claim, unless a measurable objective and a review gate are named	Recursive claims without evidence move the conversation away from what can be checked

Why the definition earns its keep

Two reasons, one scientific and one about safety.

Measurement. If you cannot say exactly which parts of a cycle are automated, you cannot measure whether automating them helped. Progress claims dissolve into anecdotes. Drawing the boundary precisely is what makes an honest before-and-after possible.

Safety. Ambiguity about where autonomy starts is ambiguity about where oversight should sit. A system described as “helping” that in fact chooses, runs and scores its own experiments would escape the review that its actual authority requires. Precise language is a control, not a style preference.

Key takeaways

Autonomous experimentation means the system owns propose, run, evaluate and iterate against an externally fixed objective.
Iterate is the load-bearing stage: results must change what gets proposed next.
The machine owns repetition and scale; people own objectives, meaning and permissions.
Three tests: no human in the inner loop, objective fixed from outside, results measurably change behaviour.
Sweeps, chat interfaces and writing tools are useful — but they are not this.

Written by the AutoScientia team · Published 9 September 2025 · Germany Part 2 of 7 · Series: foundations of automated experimentation