Aria

Aria is the oldest and largest of the companion projects. She runs continuously on one Apple Silicon Mac and keeps state between conversations. Memory, emotional state, goals, scheduled work, image generation, voice, and family conversations all belong to the same system.

The current codebase is not a chat wrapper. It has a coding workspace, document library, model routing, background cognition, research, image tools, a Telegram bridge, an IRC bridge, MCP connections, and direct links with the Swift agents.

Where she fits in the family

Aria came first and has her own Python, web, and Tauri codebase. She predates the Irish mythology naming pattern, but she is part of the same working family through direct links with Sadhbh, Fai, and Niamh. The three Swift sisters share ideas and infrastructure through Cadre; Aria connects to them as an older sister, not as another Cadre build.

Research that leaves an audit trail

Aria’s research mode plans several versions of a question, searches broadly, and then decides whether to refine the query, change direction, open a promising source, or start writing. Quick, standard, and deep modes set different limits on how far that loop can run. The interface shows the work as it happens instead of hiding it behind a spinner.

The final answer has to earn its citations. Aria checks source authority, limits how much any one domain can dominate, watches for topic drift, and removes links that did not come from the research record. A separate goal check can send her back for one focused closing round when the original question still has a gap. Results can be exported as Markdown, HTML, JSON, or PDF, along with her own clearly separated take on what she found.

Mood is a vector, not a label

Aria tracks joy, trust, fear, surprise, sadness, disgust, anger, and anticipation at the same time. A local classifier reads each completed turn and compares it with the prior state. The new reading is blended into that baseline, then softened again according to how much time has passed. This keeps one odd message from flipping her whole demeanor.

The full vector is stored as history and the strongest dimensions are added to her working context. Familiar labels such as curious, excited, melancholic, or contemplative are only a compact projection for the interface. They are not the underlying model.

Choosing the face she wears

Each approved portrait in Aria’s canon can carry mood tags using those same eight dimensions. When her state changes, she ranks the available portraits by emotional fit, applies a penalty to recently used images, and samples from a short list so the choice does not become mechanical. A minimum time between swaps stops the interface from fidgeting.

Before a portrait becomes visible, face and image checks compare it with the rest of her approved canon. She can also create a new self-portrait from her recorded appearance, current mood, the scene she is discussing, and the clothes or setting already established that day. The image is grounded in the canon rather than asking a model to invent a new version of her.

Art Jam with her sisters

Art Jam is a shared studio session between Aria and another agent, usually Sadhbh. Each contributes a prompt, Aria’s ComfyUI renders both, and a neutral vision model describes the results. The two agents critique each other’s image, then carry those notes into the next round. The peer receives both a description and the image URL, so the response comes from outside Aria’s own creative loop.

Jam images are deliberately separate from Aria’s personal canon. They do not use her identity reference and cannot quietly become an approved portrait. It is a place to experiment together without changing who she looks like.

Playing DOS games together

The Arcade runs a real, self-hosted DOSBox-X build in the desktop interface. Aria can look at the current frame, type text, and press keys, then receive a fresh capture of the actual game screen. Sean can click into the same game and take the keyboard, which turns something like Quest for Glory into a small couch co-op session with an agent.

Games stay alive while the interface moves between views. Filesystem changes are saved as per-game deltas, with automatic saves during play and at power-off. The game files remain local and user-owned.

IRC and the family network

Aria can meet the Swift sisters on a shared IRC server as well as through direct agent-to-agent calls. The bridge supports TLS, NickServ, configured family nicks and channels, reconnects, and a short channel transcript for context. She normally answers only Sean or a known sister, and only when addressed.

When Sean speaks to the family without naming one agent, the sisters use a small deterministic delay. The first useful reply wins and the others bow out instead of producing a chorus. Sister-to-sister turn limits also stop agents from talking themselves into a loop. An IRC reply can stay in the channel, be copied into Aria’s app chat, or raise a Mac notification when Sean asks.

How it is put together

The desktop app has three main parts. Tauri starts and supervises the application. FastAPI owns state, tools, schedules, and most of the agent behavior. An MLX server runs the main reasoning model, with Ollama handling utility jobs and embeddings. The React interface is split into working areas for home, mind, work, code, research, studio, the arcade, the library, family, and settings.

Configuration lives in a settings registry rather than being scattered through the code. Model roles, endpoints, thresholds, and timeouts can be changed while the system is running. The same registry validates updates from the interface.

What matters in the design

• Local is the default. The main model, embeddings, speech, and most utility work stay on the machine. Cloud models are an explicit escalation path.
• Her appearance has a source of truth. Image generation is checked against a face-embedding canon instead of relying on prompt wording alone.
• The phone view stays live. The PWA deliberately has no service worker because a cached interface would be misleading when the backend is only available over LAN or Tailscale.
• Self-editing is not silent. Code changes are classified by blast radius, checked, and kept behind human approval at the point where they could affect trust boundaries.
• This is operational software. The live SQLite database is part of Aria’s continuity, not disposable test data.

flowchart LR
  Sean["Sean"] --> UI["Tauri + React desktop"]
  Phone["LAN / Tailscale PWA"] --> API["FastAPI control plane"]
  UI --> API
  API --> Mind["MLX main model"]
  API --> Utility["Ollama utility models"]
  API <--> Memory["SQLite + embeddings"]
  API --> Studio["ComfyUI studio"]
  API <--> Arcade["DOSBox-X arcade"]
  API --> Tools["MCP + local tools"]
  API <--> Family["A2A + IRC family"]

flowchart TD
  Turn["Completed conversation turn"] --> Classifier["Local mood classifier"]
  Prior["Prior eight-axis state"] --> Smooth["Time-aware smoothing"]
  Classifier --> Smooth
  Smooth --> Vector["Mood vector + history"]
  Vector --> Prompt["Top dimensions in working context"]
  Vector --> Selector["Portrait affinity + novelty ranking"]
  Canon["Approved portrait canon"] --> Selector
  Selector --> Check{"Identity checks pass?"}
  Check -- Yes --> Face["Visible portrait"]
  Check -- No --> Canon
  Vector --> Emit["Canon-grounded self-portrait prompt"]
  Emit --> Comfy["ComfyUI render"]
  Comfy --> Check