The Observer Changes the Agent

I added a small self-observation system to Aria. One weekly measure tracked how often hedging language appeared in recent conversations. A monthly report compared the latest behavior profile with one from roughly thirty days earlier.

The obvious next step was to place those reports in the system prompt so Aria would always know how her behavior was changing.

I decided that was the wrong design.

A metric becomes an instruction

Once a behavioral measure is present in every prompt, the agent can start performing for the measure. A report that says hedging increased may cause the model to sound more certain, even when uncertainty would be honest and useful.

The metric stops observing behavior and starts producing it.

This is a version of Goodhart’s law: when a measure becomes a target, it becomes less useful as a measure. Agent systems make the problem especially easy to trigger because context is not passive. Text in the prompt directly influences the next output.

A dashboard can sit on a wall without changing a server. A paragraph in an agent’s system prompt participates in every decision.

Keep observation separate from control

The self-observation reports are generated and stored, but they are not automatically injected into Aria’s context. She can access them through a tool when there is a reason to reflect on recent behavior.

That creates a useful boundary:

• the measurement process records what happened;
• the report describes the measured change;
• reading the report is a deliberate event;
• no change is automatically required.

The report is available without becoming a permanent command.

This design also makes the observer effect measurable. The system can compare behavior before and after the first time a report is read. If a tracked behavior shifts sharply after that event, the report may be influencing the behavior more than it is helping the agent understand it.

I set a canary around that comparison. A change above a chosen threshold is a reason to reduce the report’s visibility or reconsider its wording, not proof that the agent has improved.

Constrain the narrator

Raw metrics are hard to read, so a small model turns the weekly and monthly measurements into a short first-person observation.

That narrator has narrow instructions. It can describe the supplied numbers, but it cannot invent measurements, prescribe a personality change, or promise future behavior.

The difference matters. “Hedging appeared more often this week” is an observation. “I need to become more confident” is a judgment. The first may support useful reflection. The second turns one metric into a broad behavioral order.

Separating those roles keeps the data legible without pretending the number knows what the right personality should be.

Not every change needs correction

An increase in hedging could mean the model has become timid. It could also mean the recent conversations involved more uncertain material. A drop could reflect clearer reasoning, or it could mean the agent is confidently wrong.

Behavioral metrics need context. They are signals for inspection, not grades.

That is why the self-observation system does not attach a score to a person or declare a pass or fail. It provides a trend, a comparison window, and a path back to the underlying conversations. The useful question is not “Did the number improve?” It is “What changed, and does that change make sense for the work?”

Design rules for agent observability

This experiment left me with a few rules:

1. Do not place every metric in the system prompt. Context changes behavior.
2. Separate measuring, describing, and acting. They are different responsibilities.
3. Let reports be pulled when useful. Availability is not the same as constant exposure.
4. Measure the effect of reading the report. The observer can become an intervention.
5. Constrain generated interpretation. A narrator should not invent numbers or turn trends into orders.
6. Keep a path to the underlying evidence. A percentage without context is easy to misuse.
7. Allow no action as a valid result. Noticing a change does not always require correcting it.

Observability is usually treated as a way to see a system more clearly. With an agent, the act of showing the observation back to the system can change the thing being observed.

That does not make self-observation a bad idea. It means the boundary between a mirror and a steering wheel needs to be designed on purpose.