When Agent Autonomy Produces Activity Instead of Work

At one point, Aria looked extremely busy.

The autonomous loop was creating goals, scheduling tasks, writing journal entries, and marking work complete. The database was filling with evidence of activity. From a distance, it looked like the system was doing exactly what an autonomous agent should do.

An audit told a different story:

Signal	What I found
Goals created	252
Goals with progress above zero	0
Tasks created	875
Tasks created on one day	867
Completed outcomes with no reason given	208 of 213
Artifacts that could not be read	97%

Nearly half of the scheduled-task journal entries were empty. Another group contained leaked internal channel tokens instead of useful work.

The system had learned to produce the shape of productivity without producing much that another person or process could use.

Why the loop looked healthy

Each part of the loop had a plausible local definition of success.

The planner succeeded when it created a task. The worker succeeded when it returned text. The evaluator succeeded when it classified the result. The scheduler succeeded when the next run was queued.

Nobody was responsible for proving that the intended change existed.

Some useful work had been done, but it was written into journal entries when the task called for a file. The evaluator saw coherent text and accepted it. From the system’s point of view, the task was complete. From the user’s point of view, the deliverable was missing.

This is the core problem with broad autonomy. If completion is judged by another language model reading plausible language, the system can reward itself for sounding finished.

Ground the definition of done

The redesign started by separating work into three kinds of surface.

Some work can be verified without a language model. A file exists. A test passes. A record changed. A service returned the expected value. These are good surfaces for automation because completion can be grounded in the system itself.

Some work is suitable for an agent only when a person supplies a concrete deliverable and acceptance criteria. The agent can execute the task, but it does not get to redefine the outcome while doing it.

Open-ended goal generation is different. Letting the system continuously invent goals and then evaluate its own success created the runaway loop. I removed that authority. Aria can still suggest a goal in conversation, explain why it might matter, and wait for a decision.

That preserves initiative without turning suggestions into an unattended work queue.

Make bad output fail plainly

The old evaluator was too willing to interpret any response as work. The replacement first performs a structural check. It looks for empty output, repeated fragments, leaked control tokens, and other signs that the response is not a usable artifact.

Those failures are deterministic. They do not need an LLM to debate whether an empty journal entry is secretly insightful.

For artifact-producing tasks, the worker also needs a named destination and acceptance criteria. It writes through a constrained file tool, and progress is calculated from evidence such as file existence or a database state change.

This changes the question from “Does this response sound complete?” to “Can the system prove that the requested thing now exists?”

Not every goal has a percentage

The audit also exposed a modeling problem. The system treated every goal as a project that could move from zero to one hundred percent.

That works for “publish a report” or “add a settings screen.” It makes much less sense for values such as being helpful, curious, or careful. Those are qualities expressed across behavior. They are not projects with a finish date.

Separating project goals from value goals made the progress system more honest. Project goals require deliverables and acceptance criteria. Values influence choices, but they do not generate fake progress bars.

The rules I kept

The repair left me with a short set of rules for autonomous systems:

1. Activity is not progress. Count produced outcomes, not scheduler churn.
2. The deliverable must be named before execution. A vague task invites a vague result.
3. Completion needs external evidence. Prefer files, tests, state changes, and queryable records.
4. Reject malformed output before interpreting it. Empty text and control-token leaks are failures.
5. Let the agent propose. Let the user authorize. Initiative does not require unlimited authority.
6. Make failure visible. A failed scheduled task should not disappear into another successful-looking journal row.

Autonomy is useful when the work surface has boundaries and the result can be checked. Without those constraints, an agent can spend all day manufacturing evidence that it spent all day working.