Outside-in verification
Uptime Kuma probes public applications, edge routing, TLS, and expected content. Synthetic failures prove the full path works before a real outage depends on it.
Back to climacs.net
HARP Project
Evidence-first AI reliability, not autonomous remediation
HARP — Homelab & Hybrid AI Reliability Platform
HARP helps infrastructure teams investigate incidents using evidence from their own runbooks, RCAs, monitoring data and operational documentation. It produces cited evidence packs for engineers and AI assistants while keeping every state-changing action under human control.
Under the hood: deterministic BM25 retrieval, optional OpenRAG semantic search, read-only MCP access, optional model routing through the wider architecture (e.g. LiteLLM), plus workflow state and human approval.
Core capabilities
BM25 + hybrid searchEvidence packs with citationsRead-only MCP agent surfaceNero-Camp workflow & approval stateHuman approval before remediationEval harness + 42 golden queries
Optional ecosystem
OpenRAG semantic retrievalLiteLLM model routing (wider architecture)n8n workflow glue (optional) Control-room architecture read-only first
HARPBM25 · hybrid search · evidence packs · MCP
AI clientsCodex, Cursor, Claude Desktop
SignalsUptime Kuma · Alertmanager · Prometheus
LiteLLMmodel routing (wider architecture)
OpenRAGoptional semantic retrieval
Nero-Campworkflow and approval state
Git + runbooksRCAs, docs, sessions, evals
BM25: 41/42
Retrieval pass rate
98% on 42-query Sprint 11 baseline
Hybrid: 41/42
Retrieval pass rate
98% with OpenRAG enabled
v0.6.0
HARP platform
API + container packaging release
0.5.0
MCP interface
Read-only tools live
BM25 and hybrid retrieval both reached 41/42 on the verified Sprint 11 baseline — separate measurements, tied result.
Proof
How HARP proves itself.
Synthetic testing checks whether the complete path works before a real incident depends on it.
Inside-out verification
Prometheus and Alertmanager supply cluster and infrastructure alerts. Internal signals normalize into the same downstream triage flow as external probes.
Retrieval regression verification
A 42-query golden set checks whether expected runbooks and RCAs continue to appear at useful ranks. This measures retrieval quality, not general LLM answer quality.
Core Message
The reliability boundary is the product.
HARP is built around a clear split: evidence, workflow state, retrieval, model routing, event glue, and approvals each have one owner.
HARP is evidence
HARP turns Git-tracked runbooks, RCAs, session logs, alerts, metrics, and generated evidence packs into searchable reliability memory with citations and confidence signals.
MCP is read-only
AI tools can ask HARP for context through MCP tools and resources. They do not get shell access, Git writes, kubectl mutation, or remediation power.
Nero owns approval
Nero-Camp owns workflow state, approval, audit timeline, and the human decision record. HARP provides evidence; Nero records the operational decision.
Scope Boundary
What HARP is not.
The deliberate scope is as important as the feature set. These are hard boundaries, not roadmap gaps.
Not autonomous remediation
HARP does not restart services, apply patches, or execute kubectl mutations. Evidence-only means evidence-only.
Not a general chatbot
HARP has no broad operational credentials. Every answer must cite a repo-relative file path or live evidence.
Not a SaaS product
HARP is self-hosted, content-agnostic, and profile-driven. Point it at any Git runbook repo and it indexes that repo.
Not a replacement for monitoring
HARP does not replace Prometheus, Grafana, Alertmanager, or Argo CD. It is a reliability knowledge layer that sits alongside them.
Talk Track
How to explain HARP.
01
The problem
Incident context is scattered across alerts, logs, runbooks, dashboards, and old RCAs. Copying that manually into AI chats creates weak grounding.
02
A concrete example
A public service becomes unreachable, or disk space crosses a warning threshold. Two signal paths—outside-in and inside-out—both normalize into one triage flow.
03
What HARP produces
Cited evidence packs: runbook excerpts, RCA matches, confidence signals, and safe diagnostic suggestions grounded in your own documentation.
04
The human boundary
Nero-Camp records task state and approval. No state-changing action runs automatically. The operator reviews, decides, and records the outcome.
05
Proof and measurements
Synthetic monitoring exercises the full path. A 42-query golden set regression-tests retrieval. BM25 and hybrid modes pass at 98% on the verified baseline.
06
Technical depth
Deterministic BM25 retrieval, optional OpenRAG semantic search, read-only MCP, LiteLLM routing in the wider architecture, and workflow state under human control.
Modules
Explain each part one by one.
HARP
EvidenceSearch API, evidence renderer, MCP server, policy labels, eval harness, and OpenRAG connector.
Explain HARP →OpenRAG
SemanticMilvus-backed semantic retrieval for symptom-language discovery beyond exact keyword matches.
Explain OpenRAG →MCP
Agent APIA constrained tool/resource/prompt surface for AI clients that need context, not production power.
Explain MCP →LiteLLM
RoutingModel gateway in the wider reliability architecture—not required for HARP core retrieval.
Explain LiteLLM →Nero-Camp
ApprovalTask state, source records, timelines, approval gates, and human closeout decisions.
Explain Nero-Camp →Authorship
What I built vs. what I integrated.
A clear split between components I authored from scratch and components I selected, configured, and integrated into the architecture.
| Component | Status | Evidence |
|---|---|---|
| HARP search engine | Built | Pure-Python BM25 + RRF hybrid merge in harp/core/ |
| Evidence pack generator | Built | Alert payload → cited Markdown with action tiers |
| MCP server | Built | FastMCP 3.x, 5 tools, 4 resources, 3 prompts |
| Eval harness | Built | 42 golden queries, MRR scoring, harp/evals/ |
| RAG export connector | Built | Provider-neutral interface with redaction layer |
| Synthetic monitoring gateway | Built | Normalizes Uptime Kuma + Alertmanager into shared triage flow |
| OpenRAG | Integrated | Self-hosted Milvus + BGE-small embeddings; optional semantic layer |
| LiteLLM gateway | Integrated in wider architecture | Model routing for AI remediation agent; HARP core retrieval does not require it |
| Nero-Camp | Integrated | Workflow/approval state layer; HARP feeds evidence only |
| Prometheus + Alertmanager | Integrated | Inside-out alert signal source |
| Uptime Kuma | Integrated | Outside-in synthetic probe signal source |
| n8n | Available / optional | Workflow glue for event routing and low-risk automation; not part of HARP's core retrieval or approval boundary |
Enterprise Mapping
How this maps to enterprise platform engineering.
Each homelab component maps to a recognised enterprise capability. The homelab is the proof of concept; the concepts are production-grade.
| HARP Component | Enterprise Equivalent | Value |
|---|---|---|
| HARP search (BM25 + RRF) | Enterprise RAG retrieval layer | Grounded context for AI agents and on-call engineers |
| MCP server (read-only) | API gateway / agent contract | Safe, scoped access surface for AI clients |
| Evidence packs + citations | Audit-grade answer grounding | Replaces ungrounded LLM output with traceable docs |
| Action tiers (R/P/X/D) | Policy-as-code guardrails | Maps to enterprise change-management tiers |
| Nero-Camp approval loop | Workflow orchestration + ITSM | Human-in-the-loop control plane for risky actions |
| Eval harness + 42 golden queries | Retrieval-quality regression testing | Detects when known searches stop returning the expected operational evidence |
| OpenRAG connector | Vector store integration | Pluggable semantic backend behind HARP API |
| Redaction layer | Data-loss prevention (DLP) | Strips secrets before export to external RAG backends |