# Reported issues for AgentLens

Pod holds 16 of 16 problems reported by people outside the maintainer team. Issues filed by the project's own owners, members and collaborators are excluded entirely — a maintainer's release checklist is not a warning to a prospective user.

Back to [AgentLens](/mcp/agentlens).

## Most discussed

### Roadmap: agent control plane (Tier 2–3)

This is the cross-repo roadmap for agentkitai, derived from the competitive analysis (memory · observability · guardrails). It tracks Tier 2–3 work across **agentlens** (observe), **agentgate** (enforce/identify), and **lore** (remember).

## Strategic thesis
The three products map onto the emerging **"agent control plane"** category. The defensible, underserved wedge is **governance / audit / compliance, unified by agent identity** — exactly where agentkitai already holds two rare assets (Agent

[Read the thread](https://github.com/agentkitai/agentlens/issues/56) · 2026-06-22 · open · 9 comments

### Eval & prompt stack — compliance evals on the tamper-evident audit trail

**Roadmap:** Tier 3 — Close the eval/prompt quality gap vs Langfuse/LangSmith/Phoenix/Braintrust by leading with a defensible, compliance-first slice: LLM-as-judge scoring anchored to the hash-chained audit trail for tamper-evident evidence. From the agentkitai competitive analysis. Borrowed from: Langfuse (evals, LLM judges, datasets), LangSmith (prompt management, versioning), Phoenix (evaluators), Braintrust (experiments + guardrails gates).

## Why
AgentLens today captures a pristine audi

[Read the thread](https://github.com/agentkitai/agentlens/issues/55) · 2026-06-22 · closed · 4 comments

### LiveEval: compliance/guardrail scorer over live traffic

Narrow follow-up from #267. The registry scorers (regex/contains/exact_match/llm_judge) are now wired for live eval. Compliance is scored via a separate path (lib/eval/compliance.ts — tool allow/deny lists over the trace's tool_call events), not a registry IScorer. **No milestone.**

- Adapt the compliance scoring into an IScorer (or a live-specific runner) that reads the sampled session's tool_call events and evaluates against a configured policy (builtin-evaluators' compliance configs).
- Regi

[Read the thread](https://github.com/agentkitai/agentlens/issues/274) · 2026-07-01 · closed · 0 comments

### RFC 3161: offline CMS/cert-chain verification of timestamp tokens

Narrow follow-up from #99. Requesting + storing the signed TSA token shipped (PR #268: DER TimeStampReq, POST to TSA, store token + PKIStatus + genTime). This adds **offline cryptographic verification** of a stored token. **No milestone.**

- Parse the timeStampToken (CMS SignedData → TSTInfo) and verify: the messageImprint matches the subject hash, the TSA signature is valid, and the signing cert chains to a trusted TSA root.
- A trusted-cert store (bundled roots + configurable) and CMS/ASN.1 p

[Read the thread](https://github.com/agentkitai/agentlens/issues/269) · 2026-07-01 · closed · 0 comments

### Evaluator sampling (online eval)

From #224 (d). **No milestone.** Run evaluators on a sampled fraction of live traffic. Needs a LiveEvalEngine first (sampler + async evaluator runner over the ingest stream, writing scores back as annotations). Prereq-gated → graduate as its own effort once the online-eval engine exists.

[Read the thread](https://github.com/agentkitai/agentlens/issues/254) · 2026-06-30 · closed · 0 comments

### Integrations stretch: JS LlamaIndex + Haystack (follow-up from #152)

**Follow-up from #152.** The two STRETCH integration targets, explicitly out of #152's acceptance. **No milestone** (does not gate Phase 7).

#152 delivered the must-haves: Python LlamaIndex (#210) + JS LangChain/LangGraph (#209), plus the existing Python LangChain. Acceptance met.

## Remaining (stretch)
- **JS LlamaIndex** instrumentation in `packages/sdk` (mirror the JS LangChain handler `src/langchain.ts` → `Instrumentation.capture`; LlamaIndex.TS uses a callback/instrumentation handler).
- 

[Read the thread](https://github.com/agentkitai/agentlens/issues/211) · 2026-06-30 · closed · 0 comments

### Epic: Full Postgres runtime parity — port SQLite-only features to the Postgres path

## Problem

AgentLens presents a dual-dialect storage layer (SQLite default, Postgres for scale), but **runtime parity is far from complete**. Authoritative audit (2026-06-30):

- **53 tables exist in SQLite** (`db/migrate.sqlite.ts`), but **only 21 are ever created in Postgres** (`db/drizzle/*.sql`).
- **`PostgresEventStore` (`db/postgres-store.ts`) implements only the core observability layer**: events, sessions, agents, alert rules/history, analytics, retention, stats. It has **zero** methods

[Read the thread](https://github.com/agentkitai/agentlens/issues/172) · 2026-06-30 · closed · 0 comments

### Scheduled blob-storage exports (S3/GCS/Azure) + NDJSON/JSONL

**Gap (tier-2, should-have):** Only **on-demand** signed exports; no scheduled exports to blob storage, and no NDJSON/JSONL.

**Evidence (audit):** ❌ scheduled blob-storage exports, ⚠️ NDJSON. Langfuse offers hourly/daily/weekly exports to S3/GCS/Azure (incl. JSONL). Files: `routes/compliance.ts`, `lib/export-signing.ts`.

**Scope**
- [ ] Cron-driven export job → S3/GCS/Azure sink (config per org/project).
- [ ] NDJSON / JSONL format (fine-tuning friendly) alongside JSON/CSV.
- [ ] Reuse the exi

[Read the thread](https://github.com/agentkitai/agentlens/issues/151) · 2026-06-29 · closed · 0 comments

## Most recent

### End-user (userId) attribution + per-user analytics

**Gap (tier-2, should-have):** No first-class **end-user (`userId`)** dimension. Analytics breaks down by agent/model/session/tool but **not by user**.

**Evidence (audit):** `routes/analytics.ts` — "No user-level breakdown"; only feedback `subjectId` HMAC attribution exists. Langfuse treats users as first-class (`data-model`).

**Scope**
- [ ] Thread a `userId` attribute through ingest (SDK + OTLP), stored on events/traces (verified where possible, else supplied).
- [ ] Per-user analytics break

[Read the thread](https://github.com/agentkitai/agentlens/issues/149) · 2026-06-29 · closed · 0 comments

### Prompt runtime primitives: variables, chat type, config, placeholders

**Gap (tier-1, must-have):** Prompt management is **versioning-only**. Missing the runtime primitives that make prompts usable and drive a playground.

**Evidence (audit):** ❌ variables (`{{var}}`), ❌ message placeholders, ❌ config, ❌ composability, ⚠️ chat prompt type — none found in `packages/server/src/db/prompt-store.ts`. Langfuse `features/variables|message-placeholders|config|composability|folders`.

**Scope**
- [ ] `{{variable}}` compilation (SDK + server) with typed/declared variables.
-

[Read the thread](https://github.com/agentkitai/agentlens/issues/145) · 2026-06-29 · closed · 0 comments

### LLM Playground (interactive prompt run + side-by-side compare)

**Gap (tier-1, must-have):** **No LLM Playground** — the single most-cited missing feature vs Langfuse. No interactive prompt run/compare UI exists.

**Evidence (audit):** inventory "Absent entirely"; Langfuse `features/playground`, side-by-side compare, tool-calling, open-in-playground.

**Scope**
- [ ] Playground UI: edit a prompt, fill variables, pick model/params, run, see output + tokens/cost/latency.
- [ ] Server execution endpoint using the LLM-connection store.
- [ ] Side-by-side compari

[Read the thread](https://github.com/agentkitai/agentlens/issues/144) · 2026-06-29 · closed · 0 comments

### LLM connections / bring-your-own model-key store

**Gap (tier-1, must-have):** AgentLens has **no server-side LLM-connection / bring-your-own model-key store**. The SDKs only instrument the user's *own* in-app provider calls — the server can't itself call a provider. This blocks the Playground and server-side prompt/evaluator execution.

**Evidence (audit):** Langfuse `administration/llm-connection` has no AgentLens equivalent; no connection/key store found in `packages/server`.

**Scope**
- [ ] Encrypted per-(org/project) provider connection s

[Read the thread](https://github.com/agentkitai/agentlens/issues/143) · 2026-06-29 · closed · 0 comments

### Phase 6 — Teams, RBAC, SSO & exports

> **Epic — AgentLens feature roadmap.** Tracked under its phase milestone; see #56 for the cross-repo roadmap and #55 for the eval/prompt stack.

## Goal
Turn AgentLens's two divergent identity stacks (the OSS `tenant_id` model and the Postgres `cloud/` org model) into one first-class **org → project → member** hierarchy with unified role-based access, enterprise **SSO/SAML** login, and **asymmetrically signed, third-party-verifiable** data exports plus signed outbound webhooks — every record

[Read the thread](https://github.com/agentkitai/agentlens/issues/125) · 2026-06-28 · closed · 0 comments

### Phase 3 — Scores & human feedback

> **Epic — AgentLens feature roadmap.** Tracked under its phase milestone; see #56 for the cross-repo roadmap and #55 for the eval/prompt stack.

## Goal
Add a first-class human-and-end-user scoring layer to AgentLens: a public, read-friendly scores API that unifies automated and human scores, annotation queues for human review of sessions/traces, and end-user feedback ingestion attached to sessions. Every human score and piece of feedback is stamped with a verified annotator/agent identity a

[Read the thread](https://github.com/agentkitai/agentlens/issues/122) · 2026-06-28 · closed · 0 comments

### Phase 0 — Nested execution traces

> **Epic — AgentLens feature roadmap.** Tracked under its phase milestone; see #56 for the cross-repo roadmap and #55 for the eval/prompt stack.

## Goal
Preserve the parent/child span structure AgentLens already receives over OTLP — instead of flattening every span to a timestamp-ordered event list — and render it as a collapsible execution tree with a latency waterfall, where each node is agent-attributed and audit-grade.

## Why it matters
Today an agent run that arrives as a clean `tra

[Read the thread](https://github.com/agentkitai/agentlens/issues/119) · 2026-06-28 · closed · 0 comments

### Billing-grade spend (Slice B): verify agent identity on the OTLP ingest path

Parent epic: **agentkitai/agentgate#24**. **Slice B of 3.** Depends on Slice A (verified-id column + billing-grade mode). Design doc: \`docs/billing-grade-spend.md\`.

## Problem
The OTLP ingest path (`/v1/traces`, `routes/otlp.ts::extractAgentId`) reads `agentlens.agentId` / `service.name` **straight from untrusted span attributes** — no token, no signature, no server-authoritative stamp. Anyone POSTing OTLP can attribute cost to any agent. This is the spoofability called out in agentgate#24 an

[Read the thread](https://github.com/agentkitai/agentlens/issues/88) · 2026-06-25 · closed · 0 comments

The remaining reports are on [the project's issue tracker](https://github.com/agentkitai/agentlens/issues).