Executive summary
AvaOS is my answer to a practical question: what would it take to make AI genuinely useful for sensitive professional work without surrendering control of the data, the runtime, or the decision process? I designed it as a portable, local-first operating system for persistent AI teammates. Its central principle is simple: human amplified, not replaced.
Ava, the system’s Chief of Staff, can coordinate conversations, tasks, memory, approvals, and specialist teammates. But Ava is not the owner, and the language model is not the system of record. The model is treated as an untrusted reasoning component inside a deterministic governance harness. Durable memory requires owner approval. Sensitive actions have explicit boundaries. Important events are auditable.
The project grew from a threat model and domain model into a tested runtime foundation, a loopback API, a local inference stack, and an owner-facing operating workflow. Along the way, Windows security controls and a subtle UTF-8 streaming defect exposed failures that only appeared on real hardware. Fixing those failures became part of the design story, not something to hide.
Project at a glance
| Dimension | Details |
|---|---|
| Challenge | Create a persistent AI organization that stays private, portable, model-agnostic, and under one human owner’s control. |
| Core environment | Windows 11 daily driver, Python 3.12, external 1 TB SSD runtime, NVIDIA RTX 5060 8 GB. The source repository configures a local Qwen3 14B quantized model; see the evidence snapshot for an unresolved difference between that and a later runtime observation. |
| Engineering scope | Threat modeling, domain design, runtime separation, SQLite state, approvals, audit, API, local inference, tests, and operational validation. |
| Documented maturity | Private alpha with an operational backend and owner workflow. The polished desktop interface and several integrations remain in progress. |
The problem
Most AI assistants optimize for convenience. They assume cloud connectivity, vendor-controlled state, opaque data handling, and a chat session that forgets the organization around it. That can be acceptable for low-risk prompts. It is a poor fit for work that needs durable context, defensible controls, and clear ownership.
I wanted a system that could act more like a small organization than a disposable chatbot. It needed a Chief of Staff, specialist roles, governed memory, task coordination, and a record of what happened. It also had to operate locally enough that private work did not depend on sending every prompt to an external service.
That led to five non-negotiable design goals:
- Owner authority: one human owner remains the final authority for durable memory, configuration, and sensitive actions.
- Local-first privacy: core workflows can run without a cloud dependency, with loopback-only services and explicit data boundaries.
- Portability: the runtime can live on an external SSD while source code and host-specific work remain separated.
- Model neutrality: models are interchangeable components, not the identity or authority of the system.
- Governed persistence: memory, approvals, task state, and audit history survive sessions without becoming an uncontrolled prompt dump.
Scope and non-goals
AvaOS is deliberately narrow. It is a personal portfolio and research platform, not an employer production system and not a multi-tenant SaaS product. The private-alpha scope excludes unattended internet operation, automatic publishing or email, production deployment authority, self-modifying production code, and unrestricted agent swarms. Those boundaries keep the engineering problem honest: prove the control plane before expanding autonomy.
Architecture and trust model
The architecture separates probabilistic reasoning from deterministic authority. The model can propose, summarize, and coordinate. The governance layer decides what is permitted, what must be approved, what becomes durable, and what must be recorded.
| Layer | Purpose | Trust boundary |
|---|---|---|
| Owner interface | Conversations, approvals, task review, health, and configuration. | Authenticated local owner only. |
| Governance plane | Authorization, approval workflow, identity, policy checks, and fail-closed decisions. | Deterministic code is authoritative. |
| Ava orchestration | Chief of Staff behavior, conversation state, task routing, teammate coordination. | May recommend; may not silently elevate authority. |
| Model runtime | Local inference using a swappable language model and serving adapter. | Treat model output as untrusted input. |
| Persistent state | SQLite-backed memory, conversations, tasks, configuration, and append-only audit. | Schema-controlled writes with locking and migrations. |
| Tool and data boundary | Host workspaces, future integrations, and explicitly exposed capabilities. | Least privilege; private keys and secrets stay outside the portable runtime. |
Core request flow
- The owner submits a request through the local interface or API.
- The governance plane authenticates the owner, validates the request, and establishes the permitted scope.
- Ava assembles only the relevant conversation, memory, teammate, and task context.
- The selected local model produces candidate output inside that constrained context.
- The runtime validates and streams the response, records state transitions, and emits audit events.
- Anything that would become durable memory remains a proposal until the owner approves or rejects it.
Key engineering decisions
Start with the threat model
Before building the interface, I modeled thirty threats and documented ten security invariants. The first design pass classified twenty-six threats as mitigated, two as accepted external dependencies, and two as deferred. That exercise defined the MVP around only the entities needed to prove control: organization, human owner, runtime configuration state, durable memory, approval record, and append-only audit event.
This reversed the usual chatbot build order. The first milestone was not a polished chat window. It was a runtime that knew who the owner was, could distinguish a proposal from an approved fact, and could explain what it had done.
Separate source, runtime, and host work
AvaOS treats code, portable personal state, and host-specific work as different classes of data. Source code can be reviewed and versioned. The portable runtime can move with the owner. Company or client work can remain in a host-bound workspace instead of being copied onto the portable drive. That separation reduces accidental mixing and makes Safe Disconnect a concrete engineering requirement rather than a slogan.
Make memory an approval workflow
Ava does not silently write arbitrary model output into long-term memory. A memory request becomes a proposal with provenance and status. The owner can approve, reject, supersede, or retire it. Retrieval then uses only governed records. This turns memory from a convenience feature into a controlled data lifecycle.
Keep the API local and explicit
The backend exposes a versioned local API for overview, teammates, conversations, messages, tasks, approvals, activity, and boardroom state. Services bind to loopback, require an owner token, and use explicit conversation states such as submitted, thinking, responding, complete, and failed. That state machine gives the interface a trustworthy answer to a basic question: what is the system doing right now?
Design for model replacement
The language model is selected through configuration and adapters. Ava’s identity, memory, authority, and audit history do not live inside model weights. That makes it possible to evaluate a new model without redesigning the organization or handing the new model inherited authority.
Build evolution
The project advanced in deliberate layers:
- Foundation: portable configuration, schema migrations, OS-bound owner identity, append-only audit, governed memory, health inspection, and process locking.
- Organization: a persistent Boardroom roster with Ava as Chief of Staff and specialist teammates, each with explicit roles rather than vague agent prompts.
- Operations: task lifecycle, conversation lifecycle, approvals, activity history, and CLI workflows for diagnosis and control.
- Local API: an authenticated loopback service that exposes the operational model without making the browser the source of truth.
- Inference and stabilization: local model serving, governed execution, integrity checks, trust enrollment, warm-runtime experimentation, Windows security compatibility, streaming correctness, failure-state handling, and performance measurement on the real machine.
The hard parts
Windows security blocked the obvious inference path
The cold-start inference path used a privately signed executable. On the target Windows machine, Smart App Control denied that binary even though the signature and private trust chain were valid. Windows Code Integrity events confirmed the denial. The resident path, which loaded the model through a reputable Python process, was permitted, but an over-broad preflight check wrapped both paths and prevented the permitted route from becoming reachable.
The fix was architectural, not cosmetic: I scoped the Smart App Control preflight to the cold adapter only. That preserved the fail-closed behavior for the blocked executable while allowing the resident path to operate under enforced Windows security. The result was a usable local backend without asking the owner to weaken platform protections.
Lesson: cryptographic validity and operating-system reputation policy are different trust decisions. A local AI runtime must respect both.
Streaming failed on real Unicode boundaries
The owner interface later rejected some otherwise valid responses with a security-validation error. The root cause was not the model. UTF-8 bytes were being decoded at token boundaries, and a multibyte character split across chunks became the replacement character U+FFFD. The response validator correctly refused to accept corrupted text.
I changed the streaming layer to reassemble incomplete byte sequences before decoding and kept the validator strict. That fixed the user-visible failure without lowering the security bar.
Lesson: streaming is a data-integrity problem as much as a user-experience feature. Real validation has to include multilingual text and arbitrary chunk boundaries.
Validation and results
AvaOS has been validated as an operational backend on its intended Windows runtime. The evidence combines automated tests, command-line diagnostics, API checks, Windows event evidence, and owner walkthroughs across conversation, memory, task, and teammate workflows.
Measurement conditions, and what is unresolved about them. These are historical figures from an August 2026 milestone. They are not a measurement of the current runtime, and nothing here was re-run or independently reproduced for this write-up.
The test count is the full-suite gate as recorded in the project history at that milestone. The warm-start, generation, and cold-start figures come from an owner validation walkthrough on one machine with one owner; that run’s exact date and runtime profile were not recorded.
The model those figures were produced under is genuinely unresolved. The source repository configures Qwen3-14B-Q4_K_M. A later owner observation of a running system reported a low_resource profile, a model-size constraint of 3B or less, and no GPU offload, which may be packaged or external runtime state that the source repository does not represent. Until the two are reconciled, treat the latency and throughput numbers as belonging to an unconfirmed configuration. None of them are benchmarks, and no other hardware was measured.
| Evidence | Recorded result | What it demonstrates |
|---|---|---|
| Automated suite | 4,269 passed; 6 skipped, as recorded at the governed-inference milestone. Not re-run here. | Broad regression coverage across runtime, storage, governance, API, and inference behavior at that point in the project. |
| Governed memory | Proposal, owner approval, and retrieval in a later conversation. | Persistence works without silent model-authored memory. |
| Conversation state | Submitted through complete or failed, with live progress states. | The UI can represent real backend state rather than fake activity. |
| Windows security | Resident path validated with Smart App Control enforced. | The system can operate without disabling a platform security control. |
| Warm start | New-conversation time to first token reached about 1.25 seconds with most model state reused. | The resident runtime materially improves responsiveness. |
| Generation | Roughly 4 to 5.5 tokens per second, depending on context and run. | Short interactions are usable; long answers remain throughput-bound. |
| Cold start | About 77 seconds in the recorded owner validation. | Model load remains a visible operational cost. |
| GPU state | CUDA detected, but executable trust and runtime support prevented validated offload; GPU remained idle. A later runtime observation also reported no GPU offload. | Hardware acceleration is supported by design but not yet proven in the target environment. |
What the validation proved
- Ava can identify her operating environment and coordinate a specialist teammate through the task system.
- Approved memory can cross conversation boundaries while preserving owner authority and provenance.
- The runtime can fail closed when Windows blocks an executable, then recover through a permitted architecture path.
- The API, CLI, database, and inference layers agree on the same operational state.
- Performance limits were measured and published with their conditions rather than hidden behind a polished demo, including where those conditions have since turned out to be unclear.
Security and governance controls
The security model is intentionally layered. No single control is asked to carry the whole system.
- OS-bound owner identity and owner-token authentication for local services.
- Loopback-only API exposure for the private-alpha control plane.
- Explicit approvals for durable memory and other owner-governed changes.
- Append-only audit events for security-relevant and state-changing actions.
- Schema migrations, write locking, and health checks for persistent state.
- Integrity and trust checks for model-serving components, with fail-closed behavior.
- Least-privilege boundaries between source, portable runtime state, and host workspaces.
- Public trust material may enter the runtime; private signing keys and passphrases remain on separate encrypted offline media and never enter Git.
Current limitations
The backend is ahead of the interface. AvaOS is a private alpha, not a finished desktop product. The strongest current value is the governed operating foundation and the evidence that it survives contact with a real Windows security environment.
- The polished owner desktop interface, animated chat behavior, avatars, and Ava’s Office view are still being built.
- Safe Disconnect is a permanent portability requirement, but the complete detach-and-verify workflow is not yet implemented.
- Validated GPU offload is still blocked by executable trust and runtime compatibility. Long responses therefore remain CPU-throughput-bound.
- A Discord adapter is implemented and merged, and a live Gateway has been exercised during owner acceptance. That acceptance is not complete, so Discord is not described here as validated. Broader external connections and expanded teammate workflows remain backlog items.
- Which model the runtime actually loads is an open question. The source repository configures Qwen3-14B-Q4_K_M, while a later owner observation of a running system reported a low_resource profile, a model-size constraint of 3B or less, and no GPU offload. Until that is reconciled, the performance figures above cannot be attributed to a specific model with confidence.
- The system is not designed as a general-purpose enterprise platform, unattended internet operator, or multi-tenant SaaS service.
What this project demonstrates
AvaOS shows how I approach AI engineering when privacy, security, and operational reality matter. I start with authority and threat boundaries, encode them in a domain model, build deterministic controls around probabilistic components, and validate the result on the machine that actually has to run it.
The work spans security architecture, Python systems engineering, data modeling, API design, local inference, Windows trust behavior, testing, and product thinking. It also demonstrates a willingness to keep inconvenient evidence in the story. A blocked executable, an idle GPU, and a corrupted Unicode stream are not presentation failures. They are the places where the real engineering became visible.
AI-assisted development, human-owned decisions
AI tools supported ideation, code review, debugging, test design, and documentation throughout the project. They did not own the architecture or the acceptance criteria. I defined the constraints, selected the tradeoffs, reviewed the output, ran the validations, and remained accountable for every security and product decision. That division of labor mirrors AvaOS itself: AI amplifies the human owner; it does not replace ownership.
Next phase
- Complete the owner-facing desktop experience and connect it cleanly to the existing conversation state machine.
- Finish Safe Disconnect so the portable runtime can prove that it has released files, processes, and state before the drive is removed.
- Establish a trusted and repeatable GPU path, then measure latency, throughput, power, and failure behavior against the CPU baseline.
- Expand teammate coordination and Boardroom workflows without weakening approval, memory, or audit boundaries.
- Package the architecture, threat model, and operational evidence into reproducible public documentation while keeping secrets and private working data out of the repository.
Closing
AvaOS is not a chatbot with a larger prompt. It is an attempt to make an AI organization governable: persistent enough to be useful, constrained enough to be trusted, portable enough to be owned, and transparent enough to be challenged. The project is still evolving, but its foundation already proves the central idea: useful AI autonomy starts with clear human authority.