Technology

Inference infrastructure

Multiple hosts with NVIDIA GPU acceleration, connected via a private network mesh without public ports. Model routing through a proxy with failover chains: local models for sensitive workloads, cloud connection as escalation path for complex reasoning tasks.

Profiles per use case (Coder, Reasoning, Concierge, Embedding, Vision), switchable without code changes. Authentication runs through the operating system's secure key store — no API keys in the repository, none exported as environment variables. An extension of this pattern was submitted as a contribution to the Hermes open-source agent framework.

RAG with privacy by construction

Vector-based knowledge base with hard separation between customer-released content and internal materials. The separation is architecturally guaranteed, not by filter: every indexer instance writes only to its own collection and sees only its own mount path. An accidental leak is impossible by design.

Reindex runs incrementally based on versioning diffs. The pipeline is test-driven with dedicated privacy probes — semantically internal queries must consistently return empty for the public collection.

AI agent pattern

Clear architecture across all our agents: tag contract in the model output instead of free-form tool calls, consent flow before every action, stateless on the frontend with renderer-side session history. No personal data persisted.

The pattern is robust with models of any size — from cloud frontier models to local mid-tier models — and fits the GDPR tone: the system asks first, acts second. The concierge agent on this website follows exactly this pattern.

Operations and resilience

Service landscape is container-orchestrated with health checks, restart policies, and failover per service. When a standard stack has a bug, we fix it — example: a custom build of the local inference engine due to a platform-specific crash, tracked in parallel as a pull request to the upstream project.

Voice pipeline with hardware-accelerated transcription, real-time capable even for longer audio sequences. Image generation pipeline for internal diagrams and marketing material runs on the same stack.

Engineering discipline

Binding conventions across all projects:

• Concept before implementation — no major code change without a prior architecture document and approval.
• Working implementations are not changed without explicit assignment — refactoring is not justification for risk.
• Mandatory documents per project: architecture, tech decisions with reasoning, progress log, security, commit conventions.
• License management with traffic-light logic (Apache-2.0/MIT/BSD approved, GPL/AGPL only with exception) — automated check per dependency.
• Test-driven pipelines as a requirement for merges, not as a wish.