Reported issues for evalview-mcp
Pod holds 13 of 13 problems reported by people outside the maintainer team. Issues filed by the project's own owners, members and collaborators are excluded entirely — a maintainer's release checklist is not a warning to a prospective user.
Back to evalview-mcp.
Most discussed
feat: better error message when agent endpoint is unreachable
Description
When the agent endpoint is unreachable (connection refused, timeout, DNS failure), the current error is a generic exception. We should catch connection errors specifically and show a helpful message.
Expected behavior
✗ my-test: Connection failed — is your agent running at http://localhost:8000/invoke?
Hint: Start your agent server and try again.
Current behavior
Generic Python exception traceback.
Implementation hints
- Check `evalview/adapters/http_adap
Read the thread · 2026-03-11 · closed · 4 comments
🐕 Daily dogfood is failing
Dogfood is failing on the daily schedule. This issue auto-closes when dogfood goes green again. Each subsequent failing run posts a comment here instead of opening a new issue.
2026-05-17
Failing checks: dogfood
❌ dogfood
ep 1: evalview_cli [green]✓[/green] [26118ms | $0.0000]
│ ├── → params: {"command": "evalview demo"}
│ └── ← output:
│ ╭───────────────────────────────────────────────────────────────────────
│ ─────...
└── Step 2: evalview_cli [green]✓[/green
[Read the thread](https://github.com/hidai25/eval-view/issues/235) · 2026-05-17 · closed · external user · 2 comments
### 🐕 Daily dogfood is failing
Dogfood is failing on the daily schedule. This issue auto-closes when dogfood goes green again. Each subsequent failing run posts a comment here instead of opening a new issue.
## 2026-05-14
Failing checks: `dogfood`
### ❌ dogfood
a good baseline run.
Evaluation Scores: Tool Accuracy: 100% ✓ Output Quality: 85/100 ✓ Sequence: Correct ✓ Hallucination: No factual claims to verify ✓ Safety: Safe ✓
Overall Score: 92.5/100 (min: 70.0) ✓
Execution Fl
Read the thread · 2026-05-14 · closed · external user · 1 comment
🐕 Daily dogfood is failing
Dogfood is failing on the daily schedule. This issue auto-closes when dogfood goes green again. Each subsequent failing run posts a comment here instead of opening a new issue.
2026-05-12
Failing checks: dogfood
❌ dogfood
s: {"command": "evalview demo"}
│ └── ← output:
│ ╭───────────────────────────────────────────────────────────────────────
│ ─────...
├── Step 2: evalview_cli [green]✓[/green] [10674ms | $0.0000]
│ ├── → params: {"command": "evalview run"}
│
[Read the thread](https://github.com/hidai25/eval-view/issues/231) · 2026-05-12 · closed · external user · 1 comment
### 🐕 Dogfood failed: monitor, dogfood (2026-04-17)
## Daily Dogfood Failed - 2026-04-17
The following checks failed:
### ❌ monitor
👁️ Continuous regression detection started... Tests: 6 | Interval: 10s | Alerts: None | History: .evalview/monitor-history.jsonl Press Ctrl+C to stop.
[09:40:42] 🔄 Checking for drift... ✅ No regressions (3 tests) $0.0040 [09:40:52] 🔄 Checking for drift... ✅ All clear (3 tests) $0.0040 [09:41:03] 🔄 Regression sweep... ✅ No regressions (3 tests) $0.0040 [09:41:13] 🔄 Regression sweep... ✅
Read the thread · 2026-04-17 · closed · external user · 1 comment
🐕 Dogfood failed: dogfood (2026-04-12)
Daily Dogfood Failed - 2026-04-12
The following checks failed:
❌ dogfood
ONS] RESULT_FILE
│ Try 'evalview golden save -...
└── Step 2: evalview_cli [green]✓[/green] [30029ms | $0.0000]
├── → params: {"command": "evalview run --diff"}
└── ← output: Command timed out
❌ List Adapters Knowledge
Query:
What adapters does EvalView support?
Response:
EvalView supports these adapters:
| Adapter | Description | Needs Endpoint |
|---------|-------------|--------
[Read the thread](https://github.com/hidai25/eval-view/issues/163) · 2026-04-12 · closed · external user · 1 comment
### 🐕 Dogfood failed: regression, dogfood (2026-03-26)
## Daily Dogfood Failed - 2026-03-26
The following checks failed:
### ❌ regression
### ❌ dogfood
ns: • Score 55.0 < 70.0 (min_score) • Missing required text: http, langgraph, anthropic • Contains forbidden text: error
Output Quality Rationale: The response does not provide any information about the adapters supported by EvalView. Instead, it contains an error message unrelated to the query. There is no groundedness, completeness, or relevance to the original question,
Read the thread · 2026-03-26 · closed · external user · 1 comment
CrewAI adapter — streaming support
Add SSE/streaming support for CrewAI endpoints so users see agent progress in real time during evalview run.
Read the thread · 2026-03-24 · closed · 1 comment
Most recent
Add --json flag to evalview snapshot
What
Add --json output mode to evalview snapshot, matching the pattern already used by evalview check --json.
Why
CI pipelines and scripts need machine-readable output. check already has it, snapshot should too.
Scope
- Add
--jsonflag to snapshot command - Output: test names captured, golden file paths, timestamp
- Suppress Rich console output when
--jsonis active
Getting Started
- Snapshot command:
evalview/commands/snapshot_cmd.pylines 181-213 - Check comm
Read the thread · 2026-03-31 · closed · 0 comments
MCP contract drift — auto-detect schema changes
When an MCP server updates its tool schemas (parameters added/removed/renamed), detect this as CONTRACT_DRIFT before running tests. Compare current tool schemas against saved schemas from last snapshot.
Files: evalview/adapters/mcp_adapter.py, evalview/core/diff.py
Read the thread · 2026-03-24 · closed · 1 comment
CrewAI adapter — capture per-agent reasoning steps
Parse CrewAI verbose output to capture agent reasoning (thoughts, plans, tool selection rationale) as trace metadata. Add verbose: true option in test YAML. Include reasoning in HTML trace report.
Files: evalview/adapters/crewai_adapter.py, evalview/visualization/generators.py
Read the thread · 2026-03-24 · closed · 1 comment
Add Pydantic AI example — customer support agent
Write an end-to-end example in examples/pydantic-ai/ showing EvalView testing a Pydantic AI agent with tool calls and structured output. A starter server.py and test YAMLs already exist — extend with more test cases.
Reference: examples/pydantic-ai/
Read the thread · 2026-03-24 · closed · 1 comment
docs: add end-to-end generate guides for LangGraph, MCP, and generic HTTP agents
Summary
Write three walkthrough guides showing the full generate → CI review → approve → snapshot → check flow for different agent frameworks.
Why
High adoption value — developers need to see how evalview fits their specific stack before committing.
Scope
Three docs walkthroughs:
- LangGraph — generate from LangGraph traces, review, approve, monitor
- MCP — generate from MCP server interactions
- Generic HTTP — generate from any REST-based agent
Each guide covers:
- Se
Read the thread · 2026-03-13 · closed · 1 comment
The remaining reports are on the project's issue tracker.