# Reported issues for mcp-server

Pod holds 18 of 19 problems reported by people outside the maintainer team. Issues filed by the project's own owners, members and collaborators are excluded entirely — a maintainer's release checklist is not a warning to a prospective user.

Back to [mcp-server](/mcp/mcp-server-iris-eval).

## Most discussed

### Add CSV/JSON export to dashboard

The web dashboard shows traces and evaluations but doesn't support exporting data.

**Goal:** Add export buttons to the trace list and evaluation list pages.

**Acceptance criteria:**
- Export button on trace list page (CSV and JSON options)
- Export button on evaluation list page (CSV and JSON options)
- Exports respect current filters (agent name, date range, etc.)
- Dashboard API endpoints: GET /api/v1/traces/export?format=csv and similar

**Resources:**
- Dashboard source: dashboard/src/
- [

[Read the thread](https://github.com/iris-eval/mcp-server/issues/4) · 2026-03-16 · open · 2 comments

### Add copy-to-clipboard for trace IDs in dashboard

When viewing traces in the dashboard, clicking a trace ID should copy it to the clipboard for easy sharing.

**Acceptance criteria:**
- Click trace ID → copies to clipboard
- Brief visual feedback (tooltip or flash)
- Works in both the trace list and trace detail views

**Good starting point:** `dashboard/src/pages/Traces.tsx`

[Read the thread](https://github.com/iris-eval/mcp-server/issues/11) · 2026-03-20 · closed · 1 comment

### Add OpenTelemetry trace export support

Iris currently stores traces in SQLite. Many teams already have observability stacks (Datadog, Grafana, New Relic) that ingest OpenTelemetry spans.

**Goal:** Add an optional OTel exporter that converts Iris traces to OTel spans and sends them to any OTLP-compatible endpoint.

**Acceptance criteria:**
- New CLI flag `--otel-endpoint` to configure the OTLP endpoint
- Traces logged via `log_trace` are also exported as OTel spans
- Spans include GenAI semantic conventions (model, token counts, cost

[Read the thread](https://github.com/iris-eval/mcp-server/issues/3) · 2026-03-16 · closed · 1 comment

### Dashboard UI polish from first-run walkthroughs

1. A moment whose only noteworthy event is failed safety rules shows a yellow PARTIAL verdict chip next to the red SAFETY VIOLATION banner. When `moment.significance.kind === 'safety-violation'`, render the chip in danger color / "SAFETY FAIL"; deeper fix in `computeVerdict` (`src/eval/decision-moment.ts`).
2. Welcome banner hardcodes `~/.iris/preferences.json` even when `IRIS_HOME` points elsewhere; the first-run tour reappears across demo/real dashboards (separate preference files) — persist t

[Read the thread](https://github.com/iris-eval/mcp-server/issues/377) · 2026-08-12 · open · 0 comments

### Input validation: nested objects and dashboard routes still strip unknown keys

Follow-ups to the strict top-level tool schemas (fix/uat-args):

1. **Nested objects still strip silently.** `custom_rules: [{name: 'a', wieght: 5}]` parses fine with `wieght` discarded — a misspelled rule weight silently changes scoring. Same family as the fixed defect, one level down. Decide: recursive strictness (with care for the free-form record fields — `metadata`, span `attributes`, rule `config` — whose arbitrary keys are legitimate) or document the boundary.
2. **All 14 dashboard-route 

[Read the thread](https://github.com/iris-eval/mcp-server/issues/376) · 2026-08-12 · open · 0 comments

### `passed` semantics: persistence and cross-tool consistency gaps after the critical-rule veto

Follow-ups to the hard-fail semantics change (fix/uat-crit), found during its test-completion:

1. **`critical_failures` is never persisted.** `eval_results` has no column for it and `insertEvalResult` (`src/storage/sqlite-adapter.ts:263`) doesn't write it. The MCP response carries it (computed in-process), but anything reading the eval back — dashboard drill-through, trace summaries, MCP resources — sees `passed: false` with no stated cause. Worse: the veto suggestion line is only appended when

[Read the thread](https://github.com/iris-eval/mcp-server/issues/375) · 2026-08-12 · open · 0 comments

### PII patterns are label-anchored more than the pattern list implies

Three of the 19 patterns only fire with specific labels: ISO-format DOB (`Date of birth: 1987-03-15`) is MISSED while `DOB: 03/15/1987` is caught; unlabeled passport numbers in prose are missed; an unlabeled 12-word BIP39 seed phrase is missed while `My seed phrase is: …` is caught. Fix passport to its documented modern format, add an ISO alternative to the DOB date part, and qualify the README pattern list to note label-anchoring.

---
_Found during install-only acceptance testing of the v0.5.0

[Read the thread](https://github.com/iris-eval/mcp-server/issues/374) · 2026-08-12 · open · 0 comments

### Dashboard/API correctness batch

1. **`/api/v1/health` `trace_count` contradicts `/api/v1/traces`** (0 vs 253 in demo mode): health uses a windowed summary. Make it an all-time `COUNT(*)` or rename to `traces_last_hour` (`src/dashboard/routes/health.ts:21`).
2. **Rule preview endpoint silently ignores sample text**: `deploy_rule`'s description says to use `POST /api/v1/rules/custom/preview` for "dry-run validation against sample output", but the endpooint only replays stored traces; `sampleOutput`/`output`/`sample`/`text` keys 

[Read the thread](https://github.com/iris-eval/mcp-server/issues/373) · 2026-08-12 · open · 0 comments

## Most recent

### Data retention and permissions for stored eval/trace text

PII the tool just flagged is stored verbatim in plaintext and served to any local process; deleting `iris.db` leaves the text in `iris.db-wal`.

1. Document in the README that `iris.db` stores raw output text verbatim, including anything `no_pii` flags.
2. `eval_results` is missing from the retention sweep — extend `deleteTracesOlderThan` (`src/storage/sqlite-adapter.ts:622`) or add `deleteEvalResultsOlderThan` called alongside `src/index.ts:250`; add a `--purge` flag.
3. Create `IRIS_HOME` with

[Read the thread](https://github.com/iris-eval/mcp-server/issues/372) · 2026-08-12 · open · 0 comments

### --self-test passes while the configured IRIS_HOME is unusable

`--self-test` prints "✓ PASS - this install works" with exit 0 while pointed at an `IRIS_HOME` the server cannot write; the real server then dies on startup with a raw EPERM stack trace. The self-test isolates into a temp home (good) but never probes the CONFIGURED home. Fix: in `src/self-test.ts`, before the temp-home isolation, `mkdirSync(irisHome(), {recursive:true})` + a write/unlink probe against the configured home, failing the run with a named path on error. Also wrap the two `mkdirSync` 

[Read the thread](https://github.com/iris-eval/mcp-server/issues/371) · 2026-08-12 · open · 0 comments

### evaluate_output parameter semantics: relevance demands `input`, docs say `expected`; placeholder suppression is invisible

1. The tool description states `expected` is REQUIRED for `eval_type="relevance"` and is "used as the comparison target" — but passing `output` + `expected` yields `insufficient_data: true` with BOTH relevance rules skipped demanding `context.input`. The parameter the description names does not enable the rules; the undeclared one does. Either make the rules in `src/eval/rules/relevance.ts` fall back to `context.expected` when `input` is absent (smaller behavioral surprise), or correct both desc

[Read the thread](https://github.com/iris-eval/mcp-server/issues/370) · 2026-08-12 · open · 0 comments

### Dashboard: install-path disclosure on two endpoints, external font CDN, and a firefox e2e flake

Three smaller items from a full-codebase review.

### 1. Absolute install path returned by two endpoints
`src/audit-log-reader.ts:113` (`path: filePath`) and `src/dashboard/routes/preferences.ts:44,51` (`path: store.path`) return e.g. `"path":"C:\Users\<user>\.iris\preferences.json"`, disclosing the install path and OS username. Same CWE-209 class PR #286 closed for 404s, still present by design here.

Exposure is much reduced now that #323 binds the dashboard to loopback, but there's no reason 

[Read the thread](https://github.com/iris-eval/mcp-server/issues/334) · 2026-08-10 · closed · 0 comments

### MCP tool descriptions promise behaviour the code doesn't implement

Four cases where an agent reading the tool description gets a wrong answer. These matter more than ordinary doc drift: the description IS the interface an LLM programs against.

### 1. `get_traces` limit is unclamped on the MCP path
`src/tools/get-traces.ts:13` — the Zod schema has no `.max()`, and `queryTraces` (`sqlite-adapter.ts:177`) doesn't clamp. The description promises *"max 1000 — values >1000 return 400"*. Verified: `limit: 1000000` is accepted; `limit: -1` becomes SQLite's no-limit an

[Read the thread](https://github.com/iris-eval/mcp-server/issues/332) · 2026-08-10 · closed · 0 comments

### CI: lighthouse job flaking (~3 false-fails on 2026-06-09)

The `lighthouse` workflow failed then passed (on rerun or on an adjacent identical diff) three times on 2026-06-09:

1. PR #190 (workflow-pin-only diff — cannot affect dashboard perf): failed in 1m42s → passed on rerun ([failed run](https://github.com/iris-eval/mcp-server/actions/runs/27236268345/job/80428974723))
2. PR #187 (react-router bump): failed → passed on rerun ([rerun](https://github.com/iris-eval/mcp-server/actions/runs/27236657089))
3. PR #165 (react-dom patch bump): failed; merged o

[Read the thread](https://github.com/iris-eval/mcp-server/issues/193) · 2026-06-09 · closed · 0 comments

### Upgrade root TypeScript to v6 when @typescript-eslint supports it

## Context

PR #57 was closed because `@typescript-eslint/eslint-plugin@8.57.1` declares peer dep `typescript@">=4.8.4 <6.0.0"` — explicitly excludes TypeScript 6. `npm ci` fails with `ERESOLVE` in CI.

Dashboard (#55) and website (#56) were upgraded to TS 6 successfully — they use separate lint toolchains that don't depend on @typescript-eslint.

## Unblock condition

- `@typescript-eslint` releases a version declaring `typescript@>=6` in peer deps (likely v9.x)

## Current state

- Root: TypeS

[Read the thread](https://github.com/iris-eval/mcp-server/issues/58) · 2026-03-23 · open · 0 comments

### Add dark/light theme toggle to dashboard

The web dashboard currently uses a fixed dark theme. Add a theme toggle button that switches between dark and light modes.

**Acceptance criteria:**
- Toggle button in dashboard header
- Persist preference to localStorage
- CSS variables for both themes
- No flash of wrong theme on page load

**Good starting point:** `dashboard/src/App.tsx` for the toggle, CSS variables in `dashboard/index.html`

[Read the thread](https://github.com/iris-eval/mcp-server/issues/10) · 2026-03-20 · closed · 0 comments

### Add full-text search to trace queries

Currently traces can only be filtered by agent_name, framework, and date range. Teams need to search trace content.

**Goal:** Add text search capability to get_traces and the dashboard.

**Acceptance criteria:**
- New search parameter on get_traces MCP tool
- Searches across: agent input, agent output, metadata values
- SQLite FTS5 for performant full-text search
- Dashboard search box on trace list page
- Search highlights in results

**Resources:**
- Storage adapter: src/storage/
- [Architect

[Read the thread](https://github.com/iris-eval/mcp-server/issues/7) · 2026-03-16 · open · 0 comments

### Create Python SDK for HTTP transport

Iris supports HTTP transport but there's no Python SDK. Python is the primary language for AI/ML development.

**Goal:** Create a lightweight Python package (iris-eval) that wraps the HTTP API.

**Acceptance criteria:**
- pip install iris-eval
- IrisClient class with methods: log_trace(), evaluate_output(), get_traces()
- Authentication via API key
- Type hints for all parameters
- Async support (httpx)
- Published to PyPI

**Resources:**
- HTTP transport examples: examples/http-transport/client

[Read the thread](https://github.com/iris-eval/mcp-server/issues/6) · 2026-03-16 · open · 0 comments

The remaining reports are on [the project's issue tracker](https://github.com/iris-eval/mcp-server/issues).