Skip to content

ADR-0034 — Read-Only GraphQL as a Complementary Read Path

Status

Accepted — Adopted as a complementary path; not the default (July 2026). Confirmed & merged to mainline (PR #1, September 2026) — the two open caveats at adoption (the ~2× tool-call cost and the simple-lookup over-routing) were both resolved: the cost vanished on the deepagents 0.7.5 upgrade and the over-routing was fixed by the anchor-object routing rule, with the win 3×-replicated (see updated Evidence). Repository: https://github.com/FinnMacCumail/ollamaDeepAgents (src/tools/netbox_graphql.py, src/skills/netbox-graphql/)

Context

NetBox Labs' commercial Platform MCP Server makes infrastructure "agent-native" (~100 tools, a sandboxed "Code Mode", RBAC-scoped sessions) but is NetBox Cloud only. This project runs a self-hosted community instance under a strict data-privacy mandate that rules out shipping topology to a managed service. The community MCP server exposes four generic read tools and, critically, cannot express cross-model relationship filters — the netbox-mcp-filters skill exists to work around this by decomposing cross-domain questions into brittle multi-step sequences.

An appraisal of the Cloud product found its layers reproduce very unevenly on self-hosted NetBox: the Knowledge layer (Agent Skills) is open-source and portable; the ~100-tool Access layer mostly maps to the community REST/GraphQL API; the Governance layer is commercial and — being write-safety machinery — irrelevant to a read-only query agent. NetBox's GraphQL API (read-only, Strawberry-based since 4.3, present in the open-source edition) collapses "cross-domain reads" into a single server-side join.

Decision

Add a read-only netbox_graphql tool (plus netbox_graphql_schema introspection) as a complementary read path, taught by a routing skill; keep it strictly read-only and not the default. Build it PRP-first in two phases split by validation method.

  • Scope: read-only, no CRUD. This is a hard constraint (privacy + safety), and it removes most of the Cloud surface (writes, bulk, branching, governance).
  • PRP 1 — mechanism (unit-tested). Standalone tool; read-only enforced by graphql-core AST inspection (mutations/subscriptions rejected before any HTTP), never regex. Endpoint is read-only server-side, so the primary safety surface is query cost (depth/size/timeout). Bypasses the MCP filter-validator by design. 23 tests; live-verified.
  • PRP 2 — behaviour (A/B-tested). A routing skill + one system-prompt line. The skill teaches grammar + runtime introspection, not a fixed schema, so it generalizes to any NetBox type (root = snake_case(model)_list; IDs bare; strings {exact:}/{in_list:}; nested filters; and netbox_graphql_schema(<Type>) for anything unfamiliar, including plugin objects).

Evidence

Generalization was validated on an out-of-benchmark domain (circuits → provider → terminations, absent from the dataset): the agent answered via a pure introspect-then-query path with zero MCP fallback — confirming the skill teaches a method, not a memorised schema.

A/B (GraphQL-enabled vs MCP-only; same models; corrected netbox-benchmark-v4; reference-grounded correctness judge from ADR-0033). The result was measured in three rounds; each removed a caveat:

Round Combined correctness Tool-call cost Over-routing Runs
1. deepagents 0.6.10 0.667 → 0.75 ~2× more present 1
2. re-measured on 0.7.5 0.667 → 0.75 cost-neutral / cheaper present 1
3. + anchor-object routing, replicated → ≈0.82 cost-neutral fixed 3
  • The site-comparison IP-allocation query — which hallucinated a different fabricated utilization % on every MCP-only run (7.7 / 17.6 / 100 / 23.2%) — scored 0.5 → 1.0 correctness on both models with GraphQL (server-side join + prefix-membership reasoning avoids the misattribution); this held on every one of the 3 replication runs.
  • The ~2× tool-call cost was a 0.6.10 artifact: on 0.7.5 the leaner default prompts made the agent cost-neutral (flash 18.5 → 12.8 tool calls with GraphQL).
  • The over-routing was fixed by reframing routing around the anchor-object rule ("one named object → MCP, even across models"); device-detail recovered (flash 3/3 runs = 1.0, pro 2/3 and MCP-routed; the one run that slipped to GraphQL scored 0.5 — proving the mechanism).

Consequences

Positive

  • Recreates the Cloud product's cross-domain reads on-prem, read-only, and private.
  • A measured correctness win on exactly the query class that was hallucinating.
  • Generalizes to any NetBox object via runtime introspection; does not overfit the benchmark.
  • One high-value tool replaces the intent of "~100 tools" for the read use case.

Negative / limitations (status at Sept-2026 confirmation)

  • ~2× tool calls — RESOLVED. This was a deepagents 0.6.10 artifact; on 0.7.5 the GraphQL path is cost-neutral (sometimes cheaper). See ADR-0035.
  • Soft routing over-applies GraphQL to simple lookups — RESOLVED by the anchor-object routing rule (device-detail recovered to 1.0, MCP-routed). Residual: pro's routing is 2/3 deterministic — the mechanical LLMToolSelectorMiddleware gate is the escalation if airtight routing is ever required, not needed at 2/3.
  • Aggregate needs ≥3 runs (single-run variance) — SATISFIED; 3× replicated, combined ≈0.82.
  • Remaining (unrelated to GraphQL routing): tenant-site-summary is a persistent model-accuracy weak spot (pro ~0.47) — a separate investigation, not a routing issue.

Relationship to ADR-0032

The Cloud "Code Mode" is the same sandboxed-code-orchestration pattern as the QuickJS/PTC middleware deferred in ADR-0032 — and deferred for the same reason (a single-source sequential workload doesn't benefit). GraphQL is the right lever for this workload: it does the join server-side in one request, needing neither a sandbox nor a ≥10-tool surface. Where PTC was "not now," GraphQL is "yes, complementary."

References