Reported issues for Agentic HIL
Pod holds 18 of 29 problems reported by people outside the maintainer team. Issues filed by the project's own owners, members and collaborators are excluded entirely — a maintainer's release checklist is not a warning to a prospective user.
Back to Agentic HIL.
Most discussed
setup has no non-interactive completion path for agents whose host blocks self-configuration
Every eval agent that stopped at the approval gate named the same blocker: setup writes the agent's own user-level MCP registration and skill, exactly what host permission classifiers catch. A first-class escape would convert the stop into a partially configured bench in one step: agentic-hil init already finishes the project half, so the missing piece is machine-readable output naming the exact remaining command (agentic-hil agent-install --agent <agent>), or a setup --project-only alia
Read the thread · 2026-08-12 · closed · 2 comments
Bench proof: the hot-plug read leaves a running core untouched (SysTick five-pair measurement)
#342 shipped the ST-Link typed-debug reads on the least intrusive connect the CLI documents (mode=HOTPLUG, no reset=), and deliberately claims documented, not measured. This issue tracks the measurement that upgrades the claim, on a real bench with a Nucleo attached, everything through the MCP tools:
flash_firmwarean ELF whose firmware increments a RAM counter (e.g.volatile uint32_t uptime_msinSysTick_Handler); put the symbol indebug.allowed_symbols; let the board run about
Read the thread · 2026-08-30 · closed · 1 comment
An MCP conflict names the file but not what it found there
mcp_config_conflict says an unmanaged entry exists and names the file. It does not say what it found there, and that is the one fact that settles what to do next.
A real case: an operator's ~/.claude.json held
{
"type": "stdio",
"command": "<a wrapper script under the operator's own ~/.claude>",
"args": []
}
The refusal is correct: that entry points at a program the operator chose, and rewriting it would hand the hardware gate to something else. But from the report alone
Read the thread · 2026-08-18 · closed · 1 comment
setup's result should shout that the agent must restart before the tools exist
Lived through three recording takes: setup --agent claude-code succeeds, and the session that ran it then waits on MCP tools that cannot appear, because the host reads registrations at session start. The docs say it since #227, but the one place the agent is guaranteed to look is the setup RESULT itself. setup and agent-install should carry restart_required: true plus one sentence ("the
Read the thread · 2026-08-14 · closed · 1 comment
Test plans gate dump_memory and read_symbol on the openocd backend, though both tools answer on stlink
Follow-up from #342. test_reactor.py (around line 1834) admits dump_memory and read_symbol plan steps only when the configured debugger is type: openocd, although both underlying tools have answered on the ST-Link backend since 0.18.0, and since #342 they answer non-intrusively. The coverage workflow that motivated #342 (dump RAM-resident measurement data before a run ends) therefore works through the MCP tools but still cannot be written as a plan step on an ST-Link bench.
This needs a
Read the thread · 2026-08-29 · closed · 0 comments
debuggers..type is not settable over MCP, and the lock no longer protects anything its siblings do not already open
A bench needed to switch its dut debugger from the CubeProgrammer backend to OpenOCD (to reach the typed-debug family, #342) and could not do it over MCP: debuggers.<name>.type is deliberately absent from the project_config_set keys, so the operator had to edit config.yaml by hand while executable, interface_cfg and target_cfg, the three fields that fully determine what binary runs with what scripts, are all writable behind allow_config_description_write.
The documented reason for the
Read the thread · 2026-08-29 · closed · 0 comments
upgrade_failed has no ERROR_CATALOGUE entry, so its refusal renders no What-to-do section
Noticed during #326. Refusals of type upgrade_failed have no entry in ERROR_CATALOGUE (src/agentic_hil/knowledge.py), so the rendered refusal carries Details and whatever next_steps the producing code attached, but no standing What-to-do section like other refusal types get. #326 worked around it with next_steps because knowledge.py was outside its scope.
Add the catalogue entry: likely causes (network or index unreachable, TLS-intercepting proxy per the #326 signatures, package manag
Read the thread · 2026-08-20 · closed · 0 comments
Behind a TLS proxy the dry-run resolution query fails and the already-current guard is defeated
Follow-up from #326, which deliberately retried only the manager's install invocation. The pip and uv pip routes first ask the index with a dry-run resolution to honour the #191 guard (do not replace an installation that is already current). Behind a TLS-intercepting proxy that query fails like everything else, reads as "cannot tell", and falls through to the mutating install; the install then fails on the same proxy, is retried with the system store by #326's change, and succeeds. End state c
Read the thread · 2026-08-20 · closed · 0 comments
Most recent
known_runs and prune_run_records abort the whole listing when one record vanishes between glob and stat
Noticed while fixing #312. known_runs and prune_run_records (src/agentic_hil/runlifecycle.py) sort a glob of record files by st_mtime; a record removed between the glob and the stat (a run finishing and pruning itself, another process cleaning up) raises out of the stat and the outer except OSError abandons the whole listing, so one vanished file hides every other run.
A listing should tolerate members disappearing while it is being taken: stat each candidate individually, skip the ones
Read the thread · 2026-08-20 · closed · 0 comments
plugin manifests: marketplace and plugin descriptions still lead with the genre, not the operations
#321 moved the PyPI summary and the MCP registry description to operations-first wording, and #319 does the same for the docs site's meta description. Two summary surfaces are left in the pre-#321 voice, and they are the ones a Claude Code user reads in the plugin marketplace before deciding to install:
.claude-plugin/marketplace.json: "Safe embedded firmware development with hardware-in-the-loop targets via policy-gated MCP tools." Names no operation at all.- `plugins/agentic-hil/.claude-
Read the thread · 2026-08-20 · closed · 0 comments
docs: mkdocs site_description still names the genre, not the operations
#316 sharpens the PyPI summary and the MCP registry description to name the operations (probe, flash, reset, UART and CAN on a real STM32 or other embedded target, policy-gated) instead of the genre. The branch for it also adds a Documentation project URL pointing at the rendered docs site, so that site becomes one click from the package page.
The docs site's own <meta name="description"> comes from site_description in mkdocs.yml, and that string is still the shape the other surfaces ju
Read the thread · 2026-08-20 · closed · 0 comments
Captured streams are redacted by key name only, and secrets live in their values
Follow-up to #314, noticed while giving captured streams a body in the rendering: redact_sensitive masks by key name only. A secret that appears inside the VALUE of a captured stream is not masked, and package managers put secrets there: pip and uv print index URLs, and a private index URL routinely carries https://user:token@host/simple/. The machine document has always published those streams under --json, so this is not a new exposure class, but the rendering now shows them to a person
Read the thread · 2026-08-20 · closed · 0 comments
A stop over MCP can race the detached run's state file on a slow runner
tests/test_reactor_mcp_tools.py::test_a_stop_asked_over_mcp_ends_a_waiting_run failed once on Python 3.10 on macos-latest (run 32343626219, first attempt) and passed on the other 17 legs of the same commit and on the rerun. A rerun making it green does not close it; that is how a race stays.
agentic_hil.config.ConfigError: Test run state could not be read.
(raised through the FileNotFoundError branch while the test waited on the stopped run)
earlier in the same test: ConfigError: Conf
[Read the thread](https://github.com/agentic-hil/agentic-hil/issues/312) · 2026-08-20 · closed · 0 comments
### The pytest plugin's deprecated config selector fails the run, and no page says so
The pytest plugin accepts `--agentic-hil-config` and the matching `agentic_hil_config` ini key. `src/agentic_hil/pytest_plugin.py` registers both as deprecated selectors:
```python
group.addoption(
"--agentic-hil-config",
action="store",
default=None,
help="Deprecated config selector; must resolve to the discovered authoritative config.",
)
parser.addini("agentic_hil_config", help="Deprecated Agentic HIL config selector.", default=None)
The agentic_hil_config fixture does
Read the thread · 2026-08-19 · closed · 0 comments
docs/mcp-hosts.md still carries a verification date from the 0.3.0 release
docs/mcp-hosts.md line 5 says:
This guide was verified against the linked host documentation on 2026-07-14.
That line was last touched by 0737afd, the v0.3.0 release commit of 2026-07-20. The package is now at 0.16.1.dev0, and the changelog records 0.4.0 through 0.16.0 between then and now, all inside the last five weeks. The page itself moved eight times in that window, including f96edbb and fb5c70a on 2026-08-09, cbd7e06 on 2026-08-15 and 0eec572 on 2026-08-16. The file changed
Read the thread · 2026-08-19 · closed · 0 comments
The installer shows the operator the machine document on the one path where the report is all they get
Reported from a real run: the installer's step 4 failed and printed the whole agent_install document as JSON at the operator.
Two things caused it, and only one of them is fixed by upgrading.
The run was on 0.11.0, which predates the rendering entirely, and the installer had kept that copy rather than upgrading it (#291). But even on 0.16.0 the same path prints the same JSON, because install.sh and install.ps1 capture the command's output in order to decide whether the step succeeded, an
Read the thread · 2026-08-18 · closed · 0 comments
The test plan format is not published as a resource, so an agent reads it out of site-packages
Observed in a real first-run session (2026-08-17, the README demo take): asked to pin a test, the agent issued nine Bash calls in a row to locate the installed package, dump schemas/testconfig.schema.json by hand, and grep cli.py/test_reactor.py for the default plan filename. It was following the published rule (facts about this server are resources; never its installed package) and found no resource that answers the one question it had. resources/list carries config-shape, config-schema, errors
Read the thread · 2026-08-17 · closed · 0 comments
Human-readable CLI output when a human is reading, JSON only for machines
Every CLI frontend prints one JSON document, which is right for the agents, scripts and the installer that parse it, and wrong for the operator the output is often addressed to: 'agentic-hil init --force' answers a person with a wall of JSON whose one actionable sentence is buried in a summary field. Rule: when stdout is a terminal, render for a person (the summary as prose, steps and warnings as lines, next_step prominent, refusals with their remediation list); when stdout is piped or redirecte
Read the thread · 2026-08-17 · closed · 0 comments
The remaining reports are on the project's issue tracker.