Reported issues for baepsae

Pod holds 24 of 50 problems reported by people outside the maintainer team. Issues filed by the project's own owners, members and collaborators are excluded entirely — a maintainer's release checklist is not a warning to a prospective user.

Back to baepsae.

Most discussed

Accessibility permission error is too ambiguous for macOS app automation

Summary

Using mcp-baepsae for macOS app automation, the initial Accessibility permission failure mode is difficult to diagnose in real usage.

The current error explains that Accessibility permission is required, but in practice it is still ambiguous which process actually needs the permission for a given launch mode (npx, local native binary, terminal host, MCP client app).

This caused repeated trial-and-error before describe-ui started working.

Environment

macOS 26.2
Xcode 17

Read the thread · 2026-03-18 · closed · 8 comments

[Roadmap 00] mcp-baepsae 개선 순차 백로그

목적

현재 상태의 mcp-baepsae 를 기준으로, 기능 확대보다 먼저 신뢰도 / 정합성 / 테스트 가시성 / workflow 안정성을 회복하는 순서로 개선 백로그를 정리합니다.

이 이슈는 실행 순서를 정리하는 메타 이슈입니다.

Phase 0 — 공개 surface 와 정합성 복구

#45 공개 MCP API 표면 정합성 복구
#46 tool manifest 자동 생성 및 문서 드리프트 CI 차단

Phase 1 — 테스트/진단 기반 강화

#47 real smoke test 가시성 개선: capability tagging, skip taxonomy, environment diagnostics
#48 MCP/Native 공통 오류 모델과 진단 코드 체계 도입
#44 Accessibility permission error is too ambiguous for macOS app automation

Phase 2

Read the thread · 2026-03-20 · closed · 2 comments

Tab bar buttons not exposed in accessibility tree for simulator

Problem

When using analyze_ui or query_ui on iOS simulator apps with SwiftUI TabView, the individual tab buttons are not exposed as separate accessibility elements. Only a parent AXGroup with text=Tab Bar is visible.

Accessibility tree output

AXGroup text=Tab Bar

Individual tabs (e.g., "Basic", "Scroll", "Drag") cannot be found by:

query_ui(query="Basic") — not found in accessibility tree
tap(label="Basic") — fails
tap(id="Basic") — fails

Workaro

Read the thread · 2026-02-21 · closed · 2 comments

Preserve real native CLI coverage alongside fake harness contract tests

Background

Recent contract-test stabilization work moved several routing checks to a fake native harness so CI can verify argv forwarding without depending on slow real native execution.

That improves CI stability, but it also creates a risk: if too many tests rely only on the fake harness, the default CI path can stop validating whether the real Swift native CLI still accepts the routed subcommands and flags.

Problem

The fake harness is good for:

verifying argument forwarding
simu

Read the thread · 2026-03-22 · closed · 1 comment

[Roadmap 05] `type_text` 입력 정책 정리: submission method, clipboard side effect, result reporting

배경

현재 simulator 대상 type_text 는 실사용 안정성을 위해 paste 기반 경로를 우선 사용하지만, 그 대가로 시스템 clipboard 를 건드립니다.

또한 입력 정책이 다음 축으로 분산되어 있습니다.

simulator vs macOS target
auto / paste / keyboard
autocorrect 회피 (#37)
clipboard 복구/경합 가능성

즉, 현재는 기능은 동작하지만 API semantics 가 충분히 명시적이지 않습니다.

목표

입력 관련 정책을 MCP surface 에서 명확히 드러내고, 호출자가 실제로 어떤 방식으로 입력이 수행되었는지 알 수 있게 합니다.

작업 범위

1. method semantics 명확화

auto, paste, keyboard 의 정확한 의미 문서화
simulator/macOS target 별 default 정책 재점검
필요 시 `

Read the thread · 2026-03-20 · closed · 1 comment

[Roadmap 04] MCP/Native 공통 오류 모델과 진단 코드 체계 도입

배경

현재 mcp-baepsae 는 { content: text[], isError: true } 형태로 실패를 잘 감싸고 있지만, 상위 자동화 계층에서 기계적으로 다루기에는 아직 오류 모델이 약합니다.

특히 다음 문제가 있습니다.

같은 텍스트 에러라도 분류가 어려움
permission / target / selector / native-unavailable 를 구조적으로 구분하기 어려움
stderr 원문과 사용자용 메시지가 섞임
재시도 가능한 오류와 즉시 실패해야 하는 오류를 구분하기 어려움

목표

TypeScript MCP layer 와 Swift native layer 사이에 공통 오류 taxonomy 를 도입해, LLM/automation/테스트가 실패를 안정적으로 분기 처리할 수 있게 합니다.

제안

오류 코드 예시

invalid_arguments
permission_denied
`target_not_fo

Read the thread · 2026-03-20 · closed · 1 comment

[Roadmap 03] real smoke test 가시성 개선: capability tagging, skip taxonomy, environment diagnostics

배경

현재 npm run test:real 는 환경 의존성이 높아 많은 테스트가 skip 됩니다.

최근 로컬 실행에서도:

contract test 는 통과
real test 는 booted simulator 없음 / Accessibility permission 없음 등의 이유로 대다수 skip

문제는 skip 자체보다도, 무엇이 실제 검증되었고 무엇이 환경 때문에 건너뛰어졌는지 구조적으로 보이지 않는다는 점입니다.

목표

test:real 결과만 봐도 다음을 바로 알 수 있게 합니다.

어떤 capability 가 충족되었는지
어떤 테스트가 왜 skip 되었는지
어떤 선행 환경을 준비하면 되는지
CI/로컬에서 어떤 subset 을 돌려야 하는지

작업 범위

1. capability tagging

테스트를 capability 별로 명시적으로 구분합니다.

예시:

simctl-only
`booted-simul

Read the thread · 2026-03-20 · closed · 1 comment

[Roadmap 02] tool manifest 자동 생성 및 문서 드리프트 CI 차단

배경

공개 tool surface 는 한번 맞추는 것만으로는 충분하지 않습니다. 현재처럼 README 와 실제 registry 가 다시 어긋나면 같은 문제가 반복됩니다.

목표

공개 tool 목록의 source of truth 를 코드에서 자동으로 만들고, 문서/테스트가 이 목록에서 벗어나면 CI 에서 즉시 실패하게 합니다.

제안

1. authoritative manifest 도입

다음 중 하나를 선택합니다.

빌드 시점에 src/tools/* 를 읽어 tool-manifest.json 생성
테스트에서 서버를 실제로 띄워 listTools() 결과를 snapshot/fixture 로 저장
둘 다 사용하되 하나를 canonical source 로 선언

2. drift check 추가

다음을 자동 검증합니다.

README / README-KR 의 tool table
contract test 의 expected tool

Read the thread · 2026-03-20 · closed · 1 comment

Most recent

feat: menu state monitoring (kAXMenuOpenedNotification / kAXMenuClosedNotification)

Problem

menu_action and context_menu_action use fixed delays (200ms) to wait for menus/submenus to open. This is fragile — too short for slow systems, too long for fast ones.

Proposed Solution

Use menu notifications for reliable menu interaction:

Notifications

kAXMenuOpenedNotification — menu appeared
kAXMenuClosedNotification — menu dismissed
kAXMenuItemSelectedNotification — menu item was selected

Integration

Replace Thread.sleep(forTimeInterval: 0.2) in

Read the thread · 2026-03-31 · closed · 0 comments

feat: app focus tracking via kAXApplicationActivatedNotification

Problem

activate_app sends activation request and polls isActive for 1 second. But there's no way to track when focus actually changes between apps, or when the target app loses focus unexpectedly during automation.

Proposed Solution

AXObserver-based app focus tracking:

Notifications

kAXApplicationActivatedNotification — app gained focus
kAXApplicationDeactivatedNotification — app lost focus
kAXApplicationHiddenNotification — app was hidden
`kAXApplicationShownNot

Read the thread · 2026-03-31 · closed · 0 comments

feat: text change monitoring via kAXValueChangedNotification

Problem

No way to know when text content changes in a target app. Automation workflows that type text and need to verify the result must poll with read_ui_value, which is slow and wasteful.

Proposed Solution

AXObserver-based text change monitoring:

watch_text_changes(bundleId: "com.apple.dt.Xcode", id: "sourceEditor", timeout: 10)

Watches for:

kAXValueChangedNotification — text content changed
kAXSelectedTextChangedNotification — selection changed
`kAXSelectedChi

Read the thread · 2026-03-31 · closed · 0 comments

feat: window lifecycle notifications (kAXWindowCreatedNotification, kAXSheetCreatedNotification)

Problem

detect_dialog only checks current state (snapshot). If a dialog appears AFTER the check, it's missed. wait_for_ui polls for elements but can't efficiently detect new windows/sheets.

Proposed Solution

Add AXObserver-based window/sheet watching as a native command:

baepsae-native watch-windows --bundle-id com.apple.dt.Xcode --timeout 30

Watches for:

kAXWindowCreatedNotification — new window appeared
kAXSheetCreatedNotification — new sheet/dialog appeared
`k

Read the thread · 2026-03-31 · closed · 0 comments

feat: paginated child access for large UI trees (AXUIElementCopyAttributeValues)

Problem

analyze_ui fetches ALL children via kAXChildrenAttribute, which loads the entire child array at once. For apps with thousands of UI elements (Xcode project navigator, large tables), this is slow and memory-heavy.

Proposed Solution

Use AXUIElementCopyAttributeValues for paginated access:

// Get child count without loading all children
var count: CFIndex = 0
AXUIElementGetAttributeValueCount(element, kAXChildrenAttribute, &count)

// Load page by page
let pageSize 

[Read the thread](https://github.com/oozoofrog/mcp-baepsae/issues/90) · 2026-03-31 · closed · 0 comments

### feat: is_settable check in set_ui_value with better error messages

## Problem

`set_ui_value` fails with opaque AXError codes when the target attribute is read-only. Users get "AXError -25205" instead of a helpful message.

## Proposed Solution

Before calling `AXUIElementSetAttributeValue`, check `AXUIElementIsAttributeSettable`:

```swift
var settable: DarwinBoolean = false
AXUIElementIsAttributeSettable(element, attribute, &settable)
if !settable.boolValue {
    throw NativeError.commandFailed(
        "Attribute '\(attribute)' is not writable on this elemen

[Read the thread](https://github.com/oozoofrog/mcp-baepsae/issues/89) · 2026-03-31 · closed · 0 comments

### feat: enumerate_ui — dynamic attribute/action/parameterized-attribute discovery

## Problem

When automating unfamiliar apps, there's no way to discover what AX attributes, actions, and parameterized attributes a specific element supports. Users must guess or read Apple documentation.

## Proposed Solution

New `enumerate_ui` tool that dumps all capabilities of a target element:

enumerate_ui(bundleId: "com.apple.dt.Xcode", id: "sourceEditor")


**Output:**
```json
{
  "attributes": [
    {"name": "AXRole", "settable": false},
    {"name": "AXValue", "settable": true}

[Read the thread](https://github.com/oozoofrog/mcp-baepsae/issues/88) · 2026-03-31 · closed · 0 comments

### feat: add selectedTextRange read + dynamic attribute enumeration

## Problem

Two related gaps in AX attribute coverage:

### 1. selectedTextRange read is missing
`set_ui_value(attribute: "selectedTextRange")` is implemented, but **reading** the current selection range is not. `read_ui_value` supports `selectedText` (string) but not the range (location, length).

### 2. No dynamic attribute enumeration
Cannot discover what attributes an element supports at runtime. Useful for debugging and exploring unfamiliar apps.

## Proposed Solution

### selectedTextRange

[Read the thread](https://github.com/oozoofrog/mcp-baepsae/issues/87) · 2026-03-31 · closed · 0 comments

### feat: add AXUIElementSetMessagingTimeout for heavy app resilience

## Problem

When automating heavy apps (Xcode during build, large Electron apps), AX calls can time out with the default ~6 second system timeout. This causes `cannotComplete` errors that are indistinguishable from real failures.

## Proposed Solution

1. **Set global timeout** at native binary startup:
```swift
// In resolveTarget or accessibilityRootElement
AXUIElementSetMessagingTimeout(appElement, 10.0) // 10 seconds

Expose as MCP parameter — optional axTimeout on tools that us

Read the thread · 2026-03-31 · closed · 0 comments

feat: add AXObserver for real-time UI change notifications

Problem

wait_for_ui currently polls query_ui at intervals (default 500ms). This is:

Wasteful (repeated full tree traversal)
Slow (up to 500ms latency)
Unreliable (can miss transient states)

Proposed Solution

Add AXObserver-based notification watching as an optional backend for wait_for_ui:

// AXObserver watches for specific notifications
var observer: AXObserver?
AXObserverCreate(pid, callback, &observer)
AXObserverAddNotification(observer, element, kAXWindowCreate

[Read the thread](https://github.com/oozoofrog/mcp-baepsae/issues/85) · 2026-03-31 · closed · 0 comments

### feat: add AXUIElementCopyElementAtPosition for coordinate-to-element hit testing

## Problem

No way to find which AX element is at a specific screen coordinate. This is critical for:
- **SwiftUI NSHostingView workaround** — standard `kAXChildrenAttribute` traversal misses SwiftUI content hidden behind NSHostingView layers
- **Coordinate-based element identification** — given a click point, find what element is there
- **Debugging** — "what element is at this pixel?"

## Proposed Solution

Add a `hit_test` tool using `AXUIElementCopyElementAtPosition`:

hit_test(bundleId:

Read the thread · 2026-03-31 · closed · 0 comments

feat: text selection and cursor control via AX API

Problem

Cannot programmatically select text, move cursor, or get cursor position in text editors. This limits automation of code editing workflows.

Proposed Solution

Extend read_ui_value and add set_ui_value for text manipulation:

Reading (already partially done)

read_ui_value(attribute: "selectedText") ✅ implemented
read_ui_value(attribute: "insertionPoint") ✅ implemented
read_ui_value(attribute: "numberOfCharacters") ✅ implemented

Writing (new)

set_ui_val

[Read the thread](https://github.com/oozoofrog/mcp-baepsae/issues/82) · 2026-03-31 · closed · 0 comments

### feat: alert/sheet/dialog detection tool

## Problem

No way to detect if a modal dialog, sheet, or alert is currently presented. Automation workflows can get stuck when an unexpected dialog appears.

## Proposed Solution

Add a `detect_dialog` tool that checks for modal overlays:

detect_dialog(bundleId: "com.apple.dt.Xcode")

Returns: { hasDialog: true, type: "sheet", title: "Save changes?" }


### Implementation

Check AX tree for:
- Windows with `kAXSubroleAttribute == "AXDialog"` or `"AXSheet"` or `"AXSystemDialog"`
- `kAXM

[Read the thread](https://github.com/oozoofrog/mcp-baepsae/issues/81) · 2026-03-31 · closed · 0 comments

### feat: context menu item selection after right_click

## Problem

`right_click` can trigger a context menu, but there is no tool to select an item from the resulting context menu. Users must manually find and click items.

## Proposed Solution

Add a `context_menu_action` tool or extend `menu_action` to work with context menus:

right_click(bundleId: "com.apple.dt.Xcode", id: "sourceEditor") context_menu_action(bundleId: "com.apple.dt.Xcode", item: "Refactor > Rename")


### Implementation Notes

Context menus in AX appear as:
- A new `AXMen

[Read the thread](https://github.com/oozoofrog/mcp-baepsae/issues/80) · 2026-03-31 · closed · 0 comments

### feat: add human-readable key names to key/key_combo/key_sequence

## Problem

All keyboard tools (`key`, `key_combo`, `key_sequence`) only accept raw HID keycodes (0-255). AI agents and users must know that Command=55, Shift=56, etc. This is error-prone.

## Proposed Solution

Accept human-readable key names alongside numeric keycodes:

key_combo(modifiers: ["command", "shift"], key: "o")

equivalent to

key_combo(modifiers: [55, 56], key: 31)


### Name mapping (subset)

| Name | Keycode | Name | Keycode |
|------|---------|------|---------|
| command/

[Read the thread](https://github.com/oozoofrog/mcp-baepsae/issues/79) · 2026-03-31 · closed · 0 comments

### fix: submenu navigation fails — AX kAXPressAction does not open submenus

## Problem

`menu_action` with `>` separator for submenu navigation (added in #77) fails at depth 2. When pressing a menu item that has a submenu (e.g., File > **New** > File...), `kAXPressAction` executes the item instead of opening its submenu.

## Root Cause

macOS AX menus with submenus require **mouse hover** to reveal the submenu, not `kAXPressAction`. The current implementation uses `AXUIElementPerformAction(kAXPressAction)` for intermediate menu items, which either:
- Executes the item d

[Read the thread](https://github.com/oozoofrog/mcp-baepsae/issues/78) · 2026-03-31 · closed · 0 comments

The remaining reports are on [the project's issue tracker](https://github.com/oozoofrog/mcp-baepsae/issues).

Reported issues for baepsae

Most discussed

Accessibility permission error is too ambiguous for macOS app automation

Summary

Environment

[Roadmap 00] mcp-baepsae 개선 순차 백로그

목적

Phase 0 — 공개 surface 와 정합성 복구

Phase 1 — 테스트/진단 기반 강화

Phase 2

Tab bar buttons not exposed in accessibility tree for simulator

Problem

Accessibility tree output

Workaro

Preserve real native CLI coverage alongside fake harness contract tests

Background

Problem

[Roadmap 05] type_text 입력 정책 정리: submission method, clipboard side effect, result reporting

배경

목표

작업 범위

1. method semantics 명확화

[Roadmap 04] MCP/Native 공통 오류 모델과 진단 코드 체계 도입

배경

목표

제안

오류 코드 예시

[Roadmap 03] real smoke test 가시성 개선: capability tagging, skip taxonomy, environment diagnostics

배경

목표

작업 범위

1. capability tagging

[Roadmap 02] tool manifest 자동 생성 및 문서 드리프트 CI 차단

배경

목표

제안

1. authoritative manifest 도입

2. drift check 추가

Most recent

feat: menu state monitoring (kAXMenuOpenedNotification / kAXMenuClosedNotification)

Problem

Proposed Solution

Notifications

Integration

feat: app focus tracking via kAXApplicationActivatedNotification

Problem

Proposed Solution

Notifications

feat: text change monitoring via kAXValueChangedNotification

Problem

Proposed Solution

feat: window lifecycle notifications (kAXWindowCreatedNotification, kAXSheetCreatedNotification)

Problem

Proposed Solution

feat: paginated child access for large UI trees (AXUIElementCopyAttributeValues)

Problem

Proposed Solution

feat: add AXObserver for real-time UI change notifications

Problem

Proposed Solution

feat: text selection and cursor control via AX API

Problem

Proposed Solution

Reading (already partially done)

Writing (new)

Returns: { hasDialog: true, type: "sheet", title: "Save changes?" }

equivalent to

[Roadmap 05] `type_text` 입력 정책 정리: submission method, clipboard side effect, result reporting