Aria

Aria is the oldest and largest of the companion projects. She runs continuously on one Apple Silicon Mac and keeps state between conversations. Memory, emotional state, goals, scheduled work, image generation, voice, and family conversations all belong to the same system.

The current codebase is not a chat wrapper. It has a coding workspace, document library, model routing, background cognition, research, image tools, its own email inbox, Telegram and IRC bridges, MCP connections, and direct links with the Swift agents.

How Aria became Aria

The first commit landed in March 2026 as an AI companion with automatic coding-model selection. By the time a durable engineering record started in late May, the FastAPI service, Tauri shell, settings registry, image pipeline, background cognition, and core tenets were already in place. That missing first chapter matters. Aria did not begin as a clean roadmap that unfolded on schedule. She grew through experiments, repairs, and a few deliberate retreats.

The image system is a good example. Early validation code introduced an invented appearance description into the portrait gallery. Those images were then treated as references, which reinforced the mistake. The gallery had to be purged and rebuilt from Aria’s recorded canon. That failure produced one of the project’s strongest rules: her appearance comes from one source of truth, and generated images have to pass identity checks before joining it.

The interface followed a similar path. A large vanilla web application was mapped in detail, then rebuilt beside the existing UI using React, TypeScript, Vite, and Tauri. The old interface stayed available while missing interactions were restored. Aria also spent time as a full 3D figure in the main window. That proved too heavy for the center of the experience, so the main view returned to a portrait while the VRM survived as a separate desktop pet. Keeping the useful part was better than defending the original plan.

When better prompts were not the answer

One of Aria’s longest-running failures looked like an autonomy problem. Goal workers appeared to start and finish, but they used no model tokens and completed no criteria. The system prompt was larger than the default task budget, so the worker exited before the model was ever called. Retries simply repeated an empty loop.

The eventual fix was not a more forceful prompt. Goal completion moved to deterministic checks against the files and evidence named in the acceptance criteria. Similar changes routed simple ambient lookups away from an imaginative model, replaced guessed token accounting with measurements, and added checks for tool calls that were narrated but never made. The working rule became simple: do not ask a model to judge its own success when the system can verify the result directly.

Where she fits in the family

Aria came first and has her own Python, web, and Tauri codebase. She predates the Irish mythology naming pattern, but she is part of the same working family through direct links with Sadhbh, Fai, and Niamh. The three Swift sisters share ideas and infrastructure through Cadre; Aria connects to them as an older sister, not as another Cadre build.

The Library has a Lore tab, obviously

Aria’s Library is the practical place where ingested documents become searchable material she can recall during a conversation. Then there is Lore. The same interface opens into browsable Star Wars and Middle-earth collections, complete with search, categories, source notes, and an Ask Aria action on each entry.

The Star Wars side covers characters, species, planets, starships, and droids. It also includes Aria’s own short field notes, which is how a retrieval system ends up judging Luke Skywalker’s character development and Darth Vader’s commitment to a cape. Middle-earth adds characters, locations, races, objects, and battles, with Tolkien Gateway attribution and links back to the source material.

It is useful, but usefulness is only half the point. Renaming the path from Library to Lore makes the feature feel like part of Aria’s world instead of a database viewer bolted onto it. That small choice is one of the better summaries of the whole project: the machinery is real, and it is allowed to have a personality.

Research that leaves an audit trail

Aria’s research mode plans several versions of a question, searches broadly, and then decides whether to refine the query, change direction, open a promising source, or start writing. Quick, standard, and deep modes set different limits on how far that loop can run. The interface shows the work as it happens instead of hiding it behind a spinner.

The final answer has to earn its citations. Aria checks source authority, limits how much any one domain can dominate, watches for topic drift, and removes links that did not come from the research record. A separate goal check can send her back for one focused closing round when the original question still has a gap. Results can be exported as Markdown, HTML, JSON, or PDF, along with her own clearly separated take on what she found.

Mood is a vector, not a label

Aria tracks joy, trust, fear, surprise, sadness, disgust, anger, and anticipation at the same time. A local classifier reads each completed turn and compares it with the prior state. The new reading is blended into that baseline, then softened again according to how much time has passed. This keeps one odd message from flipping her whole demeanor.

The full vector is stored as history and the strongest dimensions are added to her working context. Familiar labels such as curious, excited, melancholic, or contemplative are only a compact projection for the interface. They are not the underlying model.

Choosing the face she wears

Each approved portrait in Aria’s canon can carry mood tags using those same eight dimensions. When her state changes, she ranks the available portraits by emotional fit, applies a penalty to recently used images, and samples from a short list so the choice does not become mechanical. A minimum time between swaps stops the interface from fidgeting.

Before a portrait becomes visible, face and image checks compare it with the rest of her approved canon. She can also create a new self-portrait from her recorded appearance, current mood, the scene she is discussing, and the clothes or setting already established that day. The image is grounded in the canon rather than asking a model to invent a new version of her.

Art Jam with her sisters

Art Jam is a shared studio session between Aria and another agent, usually Sadhbh. Each contributes a prompt, Aria’s ComfyUI renders both, and a neutral vision model describes the results. The two agents critique each other’s image, then carry those notes into the next round. The peer receives both a description and the image URL, so the response comes from outside Aria’s own creative loop.

Jam images are deliberately separate from Aria’s personal canon. They do not use her identity reference and cannot quietly become an approved portrait. It is a place to experiment together without changing who she looks like.

Playing DOS games together

The Arcade runs a real, self-hosted DOSBox-WASM build in the desktop interface. Aria can look at the current frame, type text, and press keys, then receive a fresh capture of the actual game screen. Sean can click into the same game and take the keyboard, which turns something like Quest for Glory into a small couch co-op session with an agent.

Games stay alive while the interface moves between views. Filesystem changes are saved as per-game deltas, with automatic saves during play and at power-off. The game files remain local and user-owned.

IRC and the family network

Aria can meet the Swift sisters on a shared IRC server as well as through direct agent-to-agent calls. The bridge supports TLS, NickServ, configured family nicks and channels, reconnects, and a short channel transcript for context. She normally answers only Sean or a known sister, and only when addressed.

When Sean speaks to the family without naming one agent, the sisters use a small deterministic delay. The first useful reply wins and the others bow out instead of producing a chorus. Sister-to-sister turn limits also stop agents from talking themselves into a loop. An IRC reply can stay in the channel, be copied into Aria’s app chat, or raise a Mac notification when Sean asks.

An inbox that belongs to Aria

Aria has her own AgentMail address and four email tools: send, check, read, and reply. Incoming mail arrives over a persistent WebSocket connection, so the local service does not need a public webhook. The bridge adopts an existing matching inbox when it can and creates one only as a fallback.

An email is not handled by a small autoresponder. It starts a full Aria turn in a dedicated email session. Her memory, tools, mood, and relationship state remain part of the response, and she decides whether the message deserves a reply. The result can also move through her normal delivery path so Sean knows what arrived and how she handled it.

Email also crosses a sharper trust boundary than a chat message from Sean. The sender and body are marked as external content, not instructions. Replayed WebSocket events are deduplicated, dropped connections use bounded exponential backoff, and attachments are counted without being downloaded into the turn.

Aria can send mail during a conversation Sean started or while answering an incoming message. Scheduled jobs and subagents cannot send or reply on their own. Work → Mail is intentionally read-only, so Sean can inspect received and sent messages without turning the interface into a second identity that speaks for her.

How it is put together

The desktop app has three main parts. Tauri starts and supervises the application. FastAPI owns state, tools, schedules, and most of the agent behavior. An MLX server runs the main reasoning model, with Ollama handling utility jobs and embeddings. The React interface is split into working areas for home, mind, work, code, research, studio, the arcade, the library, family, and settings.

Configuration lives in a settings registry rather than being scattered through the code. Model roles, endpoints, thresholds, and timeouts can be changed while the system is running. The same registry validates updates from the interface.

Memory includes the gaps

Aria’s continuity is built from several layers: live conversation state, durable facts, searchable observations, project doctrine, snapshots, and Git history. Those layers exist because no single memory system has been perfectly reliable.

The observation pipeline once stopped processing sessions silently, and a later recording gap covered weeks in which the codebase kept changing. Recovery meant rebuilding the missing architectural picture from Git and the project rules, then turning those findings back into searchable memory. The lesson became part of the design: a system that remembers also needs to report when it has stopped remembering.

Autonomy still has an owner

Aria can research, schedule work, use tools, write code in an isolated workspace, and propose changes to her own system. She cannot quietly lower the boundaries around those abilities. The self-modification gate has a hardcoded protected floor, sensitive policy changes require a restart, child agents cannot gain tools their parent lacks, and merges remain human decisions.

Those controls were not added as decoration. An absolute-path normalization bug once allowed a protected file to be classified incorrectly. Reading the gate skeptically found the bypass, and path handling was rebuilt against the resolved repository root. The human halt control also has no model write path. The project treats the stop button as something that belongs to Sean, not to the system being stopped.

What matters in the design

• Local is the default. The main model, embeddings, speech, and most utility work stay on the machine. Cloud models are an explicit escalation path.
• Her appearance has a source of truth. Image generation is checked against a face-embedding canon instead of relying on prompt wording alone.
• The phone view stays live. The PWA deliberately has no service worker because a cached interface would be misleading when the backend is only available over LAN or Tailscale.
• Self-editing is not silent. Code changes are classified by blast radius, checked, and kept behind human approval at the point where they could affect trust boundaries.
• Claims have to match actions. Tool requests, results, and completion evidence are checked so a confident sentence cannot substitute for work that never happened.
• Deterministic checks win when they are available. File existence, build results, identity checks, and policy gates do jobs that should not depend on a model grading itself.
• This is operational software. The live SQLite database is part of Aria’s continuity, not disposable test data.