Skip to content

System Architecture

High-Level Diagram

Frontend (Dashboard) ↓ REST API Backend (FastAPI) ↓ Internal Modules GPU / LLM / Bot / System Stats

Frontend

  • Pure HTML/CSS/JS
  • Panels for Chat, Models, System Monitor, Virtual Machines, Network Services, Game Servers, Discord Bot, Diagnostics, Backend Logs, and Settings
  • Polls backend for telemetry and logs
  • Sends chat messages and confirms the currently loaded model
  • An explicit diagnostic request in dashboard chat runs the same operator-only observe-once action as the Diagnostics panel; the local model receives that read-only evidence to explain the result
  • Sends explicitly selected web-research requests and displays source links
  • Displays backend online/offline status

Backend

  • FastAPI application
  • Modular design:
    • main.py assembles the FastAPI app and keeps the main:app server entry point stable.
    • api/runtime.py initializes shared configuration, clients, stores, and common request helpers.
    • api/middleware.py applies authentication, authorization, CSRF/origin checks, CORS, and request logging.
    • api/routes/ groups authentication, chat, memory, research, and operations endpoints by responsibility.
    • system_stats.py, model_manager.py, bot_manager.py, and gpu/ provide focused service and platform modules.
  • The current model manager controls one loopback llama.cpp server and restarts it when switching models. A resident Utility AI with routed Household/Maintenance profiles is the proposed single-GPU design, not an implemented capability; simultaneous VRAM use remains unverified.
  • Provides REST endpoints for the implemented dashboard features. The VM panel has a read-only Proxmox inventory endpoint using a backend-only, least-privilege API token; there are no VM management operations. Network service panels remain planning-only UI.
  • Streams local chat replies over WebSocket (/ws/chat)
  • Filters raw <think>...</think> blocks server-side from standard and streaming assistant output. Assistant transcript writes, reads, and model context use the same filter; future Utility, Household, and Maintenance routing must preserve this policy. Raw chain-of-thought is not a user-facing feature; full reasoning visibility is reserved for a future, explicitly gated developer/debug mode, which is not implemented.
  • Enforces dashboard account roles for REST/WebSocket routes and uses a separate scoped bearer credential for designated Discord-bot routes
  • Persists private per-account conversation history and explicitly admin-published shared insights in a separate SQLite database. Admin chat and research receive current-admin identity and bounded shared notes; the AI Interface also receives shared notes. Other roles and Discord service requests do not.
  • Persists one separate admin-shared AI Interface transcript with attributed admin turns. Only the explicit diagnostics observation is executable from this console; private AI Chat transcripts are never loaded into it.
  • Persists bounded observe-only diagnostic observations, incident state, and audit events in that local database, restoring open incidents after restart; diagnostics remain manually triggered and cannot execute repairs or commands.
  • Serves bundled reviewed diagnostic runbooks through bounded local lexical retrieval. Historical incidents are not included in search, and retrieval does not invoke the model or execute procedure steps.
  • Contains deterministic in-memory maintenance/remediation simulations and proposal-only Proxmox VM, network-service, and game-server contracts. They have no live service, process, shell, Proxmox, database, or scheduler adapter.
  • Application updates: Stage 1 (admin-uploaded, signed bundle verified and staged, never applied) is implemented in release_bundle.py and api/routes/updates.py; a signed test bundle verified on the nested lab (2026-11-05). Applying is planned: the target design uses a separate least-privilege updater so the API can restart safely; its rootless-versus-brokered privilege model remains undecided. The updater is limited to the backend/dashboard services; host packages, Proxmox, GPU runtimes, secrets, persistent user data, and model weights are out of scope. A separate signed model-package flow is planned to verify and install model assets into the persistent model directory without coupling them to application updates or rollback; it is not implemented.

Web Research

  • The user explicitly enables Research Web for a message; ordinary chat stays local and does not search the web.
  • FastAPI queries a self-hosted SearXNG instance, passes a small set of untrusted search excerpts and bounded retrieved page text to the local llama.cpp model, and streams a cited summary with source links.
  • Retrieves at most three HTML pages per query, with a one-megabyte response cap and five-second timeout per page. Only public IPs on ports 80/443 are allowed; DNS results are pinned to the connection and every redirect is validated. Unsafe, unavailable, and non-HTML pages use the search excerpt instead. Page text and excerpts remain untrusted model evidence.
  • AI Chat consumes research through authenticated Server-Sent Events; the REST endpoint remains available for complete JSON responses. Completed research turns and their citation/page-fetch status are persisted to the account-owned conversation.
  • SearXNG must enable JSON output and be configured through SEARXNG_BASE_URL; the Linux/Proxmox service deployment remains pending.
  • Website-building assistance is planned as workspace-scoped work, requiring user approval before file changes or command execution.

Discord Bot

  • Python bot connected to backend
  • Slash commands for chat, model selection, diagnostics
  • Logs streamed to dashboard

Conversational History

  • ai-backend/memory_store.py stores dashboard conversations and messages in SQLite, separately from authentication records.
  • Conversation ownership is scoped to the signed-in identity provider and username; other accounts cannot enumerate, read, or delete those records.
  • The dashboard can start, resume, and delete conversations, and resumes the account's most recently updated conversation after sign-in. Discord bot requests remain stateless and are not written to dashboard history.
  • Recent context is limited to 20 messages and 12,000 characters. The database retains conversation messages until the owner deletes them.
  • Admins can explicitly publish and remove operational notes in a separate shared-insights table. Only admins can read or manage them; admins receive at most 20 newest notes and 8,000 characters of approved notes in chat and research context. The Discord service credential and non-admins receive no shared admin context.
  • Account transcripts remain private and are never automatically extracted into shared notes. Shared notes are curated prompt context, not model training. There is no automatic fact extraction or semantic/vector retrieval. Diagnostic runbook retrieval is a separate local lexical prototype; embeddings and historical-case retrieval remain unimplemented.
  • The AI Interface uses one fixed shared conversation owned by a synthetic system identity. Only admin-only API routes can access it. Each console user turn stores its authenticated admin attribution; the current admin receives that shared history and approved notes as prompt context, never another account's AI Chat transcript.

GPU Pipeline

  • Windows testing uses llama.cpp Vulkan for local Qwen2.5 1.5B chat and reads RX 6800 XT load and dedicated-memory usage from live Windows counters.
  • Linux AMD GPU telemetry is implemented using the amdgpu DRM sysfs utilization and VRAM counters, with optional hwmon sensors. This is independent of ROCm/HIP inference and still requires target-host validation.

Proxmox Deployment

  • The dashboard web server, FastAPI backend, and AI runtime are planned to run directly on the Proxmox host.
  • Host GPU telemetry is intended to read the RX 6800 XT assigned to maintenance AI; the Linux provider's device identity and readings remain unverified on the physical target.
  • The RX 6800 XT is intended for direct Proxmox host use; GPU passthrough to a VM is not part of the deployment plan. Host workloads may share GPU resources.
  • Linux runtime, GPU telemetry, service management, and safe remote UI/API access remain to be validated. See the Linux/Proxmox transition runbook.

Planned Infrastructure Management

The VM panel can display a live read-only Proxmox VM/container inventory when a dedicated API token is configured on the backend. This integration reads only the cluster resource inventory; it does not expose credentials to browser code or offer management actions. In the nested Proxmox lab, the configured token successfully retrieved a guest record on 2026-10-16; temporary Linux backend and dashboard service startup/stop checks also passed with loopback-only bindings. These checks do not establish physical-host acceptance or persistent service/reboot recovery. Network Services remains planning-only. The intended future VM panel may add provisioning, configuration, power controls, task status, and a browser console for guest recovery. Any such operations require narrowly scoped Proxmox API permissions, explicit authorization and confirmation, audit records, and tested recovery.

The primary console requirement is recovering and administering guests when their normal remote access is unavailable. Design the guest console flow around Proxmox console access, authenticated sessions, correct terminal rendering/input, and reliable reconnect/error handling.

The Network Services panel has UI planning views for DHCPv4, DHCPv6, DNS, web hosting, and routing/firewall; nothing is configured or connected. Services may be hosted inside dedicated VMs rather than on the Proxmox host. The initial integration target is observe-only discovery and health/approved metrics for a service and its owning VM, with no guest-service or network changes. The eventual telemetry interface (for example, a restricted guest agent/API or standard exporter) and its permissions must be selected and reviewed; Proxmox inventory alone does not establish service health. Final service placement, address scopes, segmentation, and edge-router ownership remain undecided. DHCP and routing/firewall operations can sever management access; keep safeguards and an independent recovery path.

Any later capability to act on observed network services requires a separate admin-authorized maturity phase, explicit per-action limits, audit evidence, independent verification, and tested rollback/out-of-band recovery. Full autonomy is a distinct later gate and must never be enabled or expanded by the AI itself. No network-service mutation API or autonomous action path is currently implemented.

The Game Servers panel is planned to manage isolated game-server workloads. A family Minecraft server is a candidate first use case, not a selected deployment. Choose edition/version and management tooling before integrating live controls; plan player access/allowlisting, resource isolation, world backups, and restore validation.