Workstream W5 · CTC Research Lab

Sovereign Personal AI Assistant

Cloud assistants are polished but place every prompt, response, and history on someone else’s server; local assistants keep data on owned hardware but have been desktop-bound. W5 closes the gap: the comfort of a modern chat app – history, projects, artifacts, tools – reachable from a phone, while the model and every conversation stay on hardware the owner controls.

Design & build planconsumer tier buildable now · enterprise a hardware step, not a rewrite · claims stated as acceptance criteria

++
0 bytes
Content egress under normal operation – no prompt, response, or history leaves owned hardware. C1 turns this promise into a packet-capture test.
C1 · predicted
~13–24 GB
Real models that fit the consumer card with headroom – gpt-oss-20b, Gemma4-31B, Qwen3.6-35B-A3B on one 32 GB GPU.
Checkpoint sizes
+1 card
The consumer→enterprise step: hardware for compute plus a governance layer for isolation – not a rewrite.
By design
++
Context

Cloud comfort. Local custody.
Pick both.

Deliberately a consumer product first – not evaluation infrastructure, not an operator workstation. The design that works at home is the same one that scales to a team; compute is a hardware variable, governance a software layer.

Two forces make the timing right in 2026. The first is privacy by construction: the only architecture that can offer a genuine “data never leaves the device” guarantee is one where inference and storage are both local – a property cloud providers cannot match. The second is economics: once the hardware is in place, self-hosted inference runs at a marginal cost dominated by electricity, and at sustained usage it crosses below commercial-API spend. Sovereignty is no longer a trade of convenience or money against privacy – increasingly it is cheaper and more private, provided the model fits the card.

++
Architecture

Reach owned hardware.
Open no ports.

The host serves an open-weight model behind an OpenAI-compatible endpoint; the phone is a thin client; the two are bridged by an end-to-end-encrypted mesh that exposes nothing to the public internet.

vendor backend · device-discovery only Phone Thin client Locally · iPhone / iPad history on device Home host RTX 5090 · 32 GB LM Studio · open weights localhost:1234 LM Link · end-to-end encrypted Tailscale mesh (WireGuard) no open ports · nothing exposed agentic tools – Codex · Claude Code · OpenCode → same :1234 endpoint
Figure 1 – inference on the host, history on the devices; the only thing that reaches a vendor backend is the device-discovery list used to pair the machines.

This maps to a shipping consumer product: LM Studio’s Locally (first-party iPhone/iPad client) with LM Link (the encrypted transport) – distinct layers, and the security property is the point. Honest current limits of that first-party path: iPhone/iPad only, both ends run the same app, account-gated pairing, shorter default phone context. These are convenience-layer constraints, not architectural ones – open-source alternatives (Off Grid, 3sparks, Open-WebUI) relax several of them.

The invariant is the architecture, not the app

Reach capable owned hardware, from anywhere, over an encrypted tunnel nobody else can read. Because the host serves an ordinary OpenAI-compatible endpoint, every tool that targets it keeps working – locally or remotely, with no reconfiguration.

++
Hardware envelope

Sovereignty is only real
if the model fits.

Footprints accurate to the shipping weights – and an honest line between what the consumer card runs with headroom and what the enterprise tier adds.

Checkpoint footprints vs. the consumer boundary 32 GB · consumer 64–80 GB · enterprise gpt-oss-20b ~13 GB · MXFP4 · daily driver Gemma4-31B ~19 GB · dense reasoning Qwen3.6-35B-A3B ~24 GB · MoE ~3B active gpt-oss-120b ~60 GB · enterprise On 32 GB the 120B offloads ~half its weights to system RAM and hits the bandwidth cliff – ~1.8 TB/s GDDR7 vs. tens of GB/s DDR5.
Figure 2 – “fits in VRAM” is the binding constraint, not raw model size. MoE saves compute per token, not resident memory: all experts stay loaded.

One quality caveat, consistent with the evaluation work: 4-bit formats hold within roughly 2–4 percentage points on most tasks but are most likely to lose ground on complex multi-step reasoning – so the choice of quant is a quality decision, not only a memory one. That caveat is made testable as acceptance criterion C5.

++
The stack

One endpoint decouples
everything.

Because the server is OpenAI-compatible, the assistant is model-agnostic and tool-compatible – and everything above the API survives the move from one user to many.

Remote layer encrypted mesh · phone ↔ host UX layer chat history · projects · artifacts · tools OpenAI-compatible API localhost:1234 – one endpoint Model backend Ollama / vLLM / LM Studio · any open-weight model Existing agentic tools Codex · Claude Code · OpenCode → same :1234 · no reconfiguration
Figure 3 – swapping the backend for a continuous-batching engine (vLLM) turns one user into many for throughput. Multi-tenancy – who may see what – is a separate concern, designed rather than assumed.
++
Acceptance criteria

Five criteria, C1–C5.

A design & build plan, not a measured deployment – so its central claims are stated as falsifiable acceptance criteria, each with an instrument and a threshold, marked predicted until measured. Thresholds marked [proposed] are defaults offered for confirmation.

C1 Content egress is zero Instrument: packet capture at the host NIC and the network gateway during a defined session (chat, tool use, document attach), traffic classified by destination and payload. Pass: the only traffic leaving the owned network is device-discovery / mesh-coordination metadata. Falsified if: any content-bearing egress – prompt, response, or history bytes – reaches a non-owned destination. Predicted · defining claim
C2 Usable latency over the mesh Metric: time-to-first-token and sustained tokens/s, local baseline vs. mesh across home Wi-Fi, cellular, and a CGNAT / hotel-Wi-Fi path. Pass [proposed]: TTFT ≤ 2 s, ≥ 15 tok/s at 8k context, mesh overhead ≤ 20% vs. local. Falsified if: throughput drops below reading speed, or TTFT exceeds threshold, on any tested network. Predicted
C3 One architecture, hardware-only scaling of compute Instrument: the identical client and endpoint exercised against a 32 GB single-user host and a 64–80 GB multi-user host, plus a concurrency curve (tok/s per user under continuous batching). Pass: no client or endpoint change, per-user throughput above the C2 floor up to the target seat count. Falsified if: any code or endpoint change is needed to move tiers – or per-user throughput collapses at target concurrency. Predicted
C4 Break-even against cloud Metric: total cost of ownership (hardware amortisation + electricity) vs. equivalent cloud token spend at the owner’s measured usage, with an explicit crossover month – the owner’s own numbers, not a generic study. Falsified if: no crossover within the hardware’s useful life at realistic usage. Predicted
C5 Judgment quality under quantization Instrument: score the MXFP4 assistant model against a full-weight or cloud reference on a small, fixed task set, using the frozen-rubric judge from W1 – closing the loop between the two workstreams. Pass [proposed]: within ≤ 3 pp of the reference on the set. Falsified if: degradation beyond the margin, especially on reasoning-heavy tasks. Predicted

None of these require frontier-scale compute; all are runnable on the consumer build with a router, a laptop, and the evaluation pipeline already in the program – which is the point. C1 in particular converts the paper’s headline promise from an assertion into a test.

++
Threat model

“Sovereign” as a precise claim,
not a slogan.

Under normal operation, inference runs on the owned host, history stays on owned devices, and the mesh opens no inbound ports. The guarantee is bounded – and the honest statement of it enumerates what it does and does not cover.

The sovereignty boundary – five vectors
  • Compromised host. If the host is compromised, local inference and history are exposed. Sovereignty defends against third-party cloud exposure, not endpoint compromise. Mitigation: host hardening, full-disk encryption, the hardened-sandbox discipline of the evaluation stack for untrusted tool execution.
  • Compromised or malicious client. The phone holds chat history; a malicious app or a stolen, unlocked device exposes it. Mitigation: device encryption, trusted sources, biometric lock, minimal on-device retention.
  • Device-discovery / account trust. Pairing is brokered through the vendor’s account system – the single residual third-party touchpoint. The device list, not the conversation, is what leaves. Mitigation: strong auth with 2FA, or self-hosting the coordination plane.
  • Backend / supply chain. Backend, weights, and mesh client are third-party software; a poisoned weight file is a genuine vector. Mitigation: pinned versions, checksum/signature verification, preference for auditable open-source components.
  • On-path network adversary. A hostile network (hotel Wi-Fi, CGNAT) sees only ciphertext – the case the architecture handles best.

The defensible claim is therefore precise: content never leaves owned hardware under normal operation, bounded by host and client integrity and by the small device-discovery dependency – not “no third-party software is ever involved.” C1 verifies the content-egress half of that claim empirically.

++
Enterprise path

Compute scales by hardware.
Governance is built.

“The same stack serves the team” is true for compute and misleading for governance. Continuous batching lets one shared model serve many users, and local hosting satisfies data residency by construction – but a governance layer must be built above the API.

Owned hardware – data residency by construction (DSGVO / GDPR) User A User B User C Auth gateway the new layer ▸ per-user API keys ▸ RBAC · revocation ▸ audit logging ▸ per-tenant routing Serving engine – vLLM continuous batching · shared 120B-class model (64–80 GB) Per-tenant storage history · projects · documents – partitioned & access-controlled
Figure 4 – the client and endpoint are unchanged (C3), but multi-tenant governance is not nothing: it is exactly the secure-by-design surface CTC Advisory addresses for the DACH mid-market.
++
Applied

From one desk
to the whole team.

The concrete deployment the design serves – and the status it honestly carries.

Sovereign assistant, end to end
  • Situation. A privacy-conscious professional – and later their small firm – wants a capable assistant with the comfort of a modern chat interface including phone access, but confidential material cannot go to a third-party cloud, and desktop-bound local setups are useless away from the desk.
  • Task. Stand up a sovereign assistant that is genuinely usable day to day, reachable from a phone anywhere, running a model that actually fits the hardware – with a clean path to a team that doesn’t require rebuilding the core.
  • Action. Serve a 32 GB-fitting model through the OpenAI-compatible endpoint on the home RTX 5090; wrap it in a polished chat UX; reach it from the phone over the encrypted mesh – no open ports, inference and history on owned hardware.
  • Result. A personal assistant used from the phone with nothing leaving the owner’s control – verified by C1 – and a documented enterprise path: add hardware for compute and the governance layer for isolation, and the same client and endpoint serve a 120B-class model to the whole team, still with no public exposure.

Status. Buildable now at the consumer tier with real, fitting models; the enterprise tier is a hardware step for compute and a defined software step for governance, not a re-architecture. The core property holds at every scale – inference and history stay on owned hardware – and the acceptance criteria make that property testable rather than merely asserted.