Docs: add CLAUDE.md build/run/drive loop; fix Vita3K MCP Wayland env + frame storage
- New CLAUDE.md documenting the build -> install -> launch -> drive -> logs loop. - session_start must force SDL_VIDEODRIVER=wayland + empty DISPLAY, else Vita3K renders to the host X display (blank frames, dropped input). - Native build now targets build/ instead of build-native/. - Capture frames into agent-run/frames/ (wiped each session) and clear the /tmp/kwin-mcp-screenshots-* folder afterwards instead of leaving them in /tmp. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Sads
+23
-11
@@ -68,18 +68,29 @@ tool works there. (Connecting to the live desktop with `session_connect` does
|
||||
the only reliable way to capture, and it is headless: the human cannot watch it
|
||||
live on their monitor.)
|
||||
|
||||
- `session_start` with `app_command: "vita3k -r VIMM00001"` **and
|
||||
`keep_screenshots: true`** — starts an isolated Wayland session, launches the
|
||||
app, and keeps every captured frame on disk after `session_stop`. Give it ~12s
|
||||
to boot.
|
||||
- **Before starting, wipe the per-session frame folder** so each run is clean:
|
||||
`rm -rf agent-run/frames && mkdir -p agent-run/frames`. (`agent-run/` is
|
||||
git-ignored; `frames/` is the throwaway capture folder — keepers live in
|
||||
`agent-run/` proper and are never touched.)
|
||||
- `session_start` with `app_command: "vita3k -r VIMM00001"`,
|
||||
`keep_screenshots: true`, **and `env: {"SDL_VIDEODRIVER": "wayland", "DISPLAY":
|
||||
""}`** — starts an isolated Wayland session, launches the app, and keeps every
|
||||
captured frame on disk after `session_stop`. Without that `env` the frames come
|
||||
back blank white and input is dropped (host `DISPLAY=:1` sends SDL to X11). Give
|
||||
it ~12s to boot.
|
||||
- **`focus_window` with `app_name: "Vita3K"` BEFORE sending keys.** Without focus
|
||||
the keypresses are dropped. Presses also drop intermittently — re-focus and
|
||||
re-press. Send one key per call; rapid bursts get coalesced/dropped.
|
||||
- `keyboard_key` with `screenshot_after_ms: [ms]` after **every** command, so the
|
||||
frame folder is a complete visual record of the run for the human to scrub.
|
||||
- `session_stop` when done (and `pkill -9 -f vita3k`). Frames survive in
|
||||
`/tmp/kwin-mcp-screenshots-*/`; copy them to `./agent-run/` (git-ignored) and
|
||||
tell the human where they are.
|
||||
- `session_stop` when done (and `pkill -9 -f vita3k`). kwin-mcp always writes
|
||||
frames to `/tmp/kwin-mcp-screenshots-*/` (that path is not configurable); don't
|
||||
leave them there — **move them into the repo and clear the tmp folder**:
|
||||
```bash
|
||||
cp /tmp/kwin-mcp-screenshots-*/*.png agent-run/frames/ 2>/dev/null
|
||||
rm -rf /tmp/kwin-mcp-screenshots-*
|
||||
```
|
||||
Then tell the human the frames are in `agent-run/frames/`.
|
||||
|
||||
### Zero-context-cost capture (the standing rule)
|
||||
|
||||
@@ -110,7 +121,8 @@ Vita3K maps host keys → Vita buttons. The ones this app uses:
|
||||
|
||||
### Typical verify loop (capture everything, Read nothing)
|
||||
|
||||
1. `session_start` (`keep_screenshots: true`) → wait ~12s.
|
||||
1. `rm -rf agent-run/frames && mkdir -p agent-run/frames`, then `session_start`
|
||||
(`keep_screenshots: true`, `env` forcing Wayland) → wait ~12s.
|
||||
2. `focus_window "Vita3K"`.
|
||||
3. `keyboard_key "Right"` (with `screenshot_after_ms`) … stepping to the target
|
||||
tile, one key per call.
|
||||
@@ -119,6 +131,6 @@ Vita3K maps host keys → Vita buttons. The ones this app uses:
|
||||
- `~/.local/share/Vita3K/Vita3K/ux0/data/vitaimmich/log.txt` (the app's own log
|
||||
— records page fetches, video open/play, scans, errors)
|
||||
- MCP `read_app_log` with the launched PID, or `/tmp/vita3k-runtime.log`.
|
||||
6. On `session_stop`, the captured frames remain in
|
||||
`/tmp/kwin-mcp-screenshots-*/` — hand that folder (or a copy in `./agent-run/`)
|
||||
to the human as the visual record.
|
||||
6. On `session_stop`, move the frames out of `/tmp` and clear it:
|
||||
`cp /tmp/kwin-mcp-screenshots-*/*.png agent-run/frames/ 2>/dev/null && rm -rf /tmp/kwin-mcp-screenshots-*`.
|
||||
Hand `agent-run/frames/` to the human as the visual record.
|
||||
|
||||
@@ -0,0 +1,160 @@
|
||||
# CLAUDE.md — vitaImmich
|
||||
|
||||
PS Vita Immich client (PoC). C sources in `src/` (`main.c`, vendored `jsmn.h`).
|
||||
Built with the VitaSDK toolchain into a `.vpk`, run in the **Vita3K** emulator,
|
||||
and driven headlessly through the **kwin-mcp** server. This file is the
|
||||
end-to-end loop: **build → install → launch → drive → read logs.** For the
|
||||
narrative version see `AgenticLoop.md`; full key map is in `controls.MD`.
|
||||
|
||||
App title id: **`VIMM00001`**.
|
||||
|
||||
---
|
||||
|
||||
## 1. Build the changes → a `.vpk`
|
||||
|
||||
After editing anything in `src/` (or `CMakeLists.txt`), rebuild.
|
||||
|
||||
**Preferred — Docker (reproducible), writes `build/vitaImmich.vpk`:**
|
||||
|
||||
```bash
|
||||
./build.sh
|
||||
```
|
||||
|
||||
> `build/` ends up root-owned. If the Docker runtime is broken (containerd shim
|
||||
> `readlink /proc/self/exe: no such file or directory`), use the native fallback.
|
||||
|
||||
**Fallback — native host SDK, also writes `build/vitaImmich.vpk`:**
|
||||
|
||||
```bash
|
||||
export VITASDK="$HOME/vitasdk"
|
||||
export PATH="$VITASDK/bin:$PATH"
|
||||
cmake -B build -DCMAKE_BUILD_TYPE=Release \
|
||||
-DCMAKE_TOOLCHAIN_FILE="$VITASDK/share/vita.toolchain.cmake" .
|
||||
cmake --build build --parallel "$(nproc)"
|
||||
```
|
||||
|
||||
> If a prior Docker build left `build/` root-owned, clear it first
|
||||
> (`sudo rm -rf build`) so the native build can write there.
|
||||
|
||||
Whichever path you use, the next step installs that `.vpk`.
|
||||
|
||||
---
|
||||
|
||||
## 2. Install the rebuilt VPK into Vita3K
|
||||
|
||||
Installing = launch Vita3K once with the `.vpk` as an argument; it extracts and
|
||||
auto-boots. `deploy.sh` does install-then-launch in one shot:
|
||||
|
||||
```bash
|
||||
./deploy.sh # installs ./build/vitaImmich.vpk, then launches VIMM00001 detached
|
||||
```
|
||||
|
||||
Gotchas (these are why it must run from a script, not inline):
|
||||
|
||||
- Background it from a **script file** with a `sleep`/`wait` then `pkill`. Inline
|
||||
backgrounding from the agent's shell does not persist.
|
||||
- Guard `pkill` with `|| true` — a no-match exit code aborts the chain.
|
||||
- Vita3K preserves the **zip's internal mtime** on extract. To confirm a *fresh*
|
||||
install, compare the installed `eboot.bin` mtime against the vpk entry, not
|
||||
wall-clock:
|
||||
|
||||
```bash
|
||||
unzip -l build/vitaImmich.vpk | grep eboot
|
||||
stat -c 'eboot: %y' ~/.local/share/Vita3K/Vita3K/ux0/app/VIMM00001/eboot.bin
|
||||
```
|
||||
|
||||
- Wipe a stale install: `vita3k -d VIMM00001`.
|
||||
|
||||
> `deploy.sh` launches detached on the **live desktop**. For agent-driven
|
||||
> verification with screenshots, **skip `deploy.sh`'s launch** and instead launch
|
||||
> inside the MCP's isolated session (step 3) — only do the *install* half first
|
||||
> (`vita3k ./build/vitaImmich.vpk` for ~4s, then `pkill`).
|
||||
|
||||
---
|
||||
|
||||
## 3. Launch + drive via kwin-mcp (screenshots + keystrokes)
|
||||
|
||||
Launch the app **inside kwin-mcp's own isolated KWin session** — its screenshot
|
||||
tool only works there. `session_connect` to the live desktop does **not** capture
|
||||
(spectacle can't save non-interactively; KWin's `ScreenShot2` DBus rejects
|
||||
unauthorized callers). The isolated session is headless: the human cannot watch
|
||||
it live, which is exactly why we capture every frame to disk.
|
||||
|
||||
0. **Wipe the per-session frame folder first:**
|
||||
`rm -rf agent-run/frames && mkdir -p agent-run/frames`. (`agent-run/` is
|
||||
git-ignored; `frames/` is the throwaway capture folder — your curated keepers
|
||||
stay in `agent-run/` proper and are never touched.)
|
||||
1. **`session_start`** with `app_command: "vita3k -r VIMM00001"`,
|
||||
`keep_screenshots: true`, **and `env: {"SDL_VIDEODRIVER": "wayland",
|
||||
"DISPLAY": ""}`** — starts the isolated session, launches the app, and keeps
|
||||
every captured frame after `session_stop`. Give it **~12s** to boot.
|
||||
|
||||
> **The `env` is mandatory.** The session inherits the host's `DISPLAY=:1`, so
|
||||
> without it Vita3K's SDL backend picks X11 and renders to the *host* display —
|
||||
> the captured frames come back **blank white**, `focus_window "Vita3K"` fails
|
||||
> ("no application matching"), and keystrokes never reach the app. Forcing
|
||||
> `SDL_VIDEODRIVER=wayland` and clearing `DISPLAY` makes Vita3K render into the
|
||||
> MCP's Wayland session, where capture and input both work. Sanity check:
|
||||
> `list_windows` should show a `Vita3K` window and frames should be ~300–500 KB,
|
||||
> not ~8 KB.
|
||||
2. **`focus_window` with `app_name: "Vita3K"` BEFORE every key.** Without focus,
|
||||
keypresses are dropped. They also drop intermittently — re-focus and re-press.
|
||||
**One key per call**; rapid bursts get coalesced/dropped.
|
||||
3. **`keyboard_key`** with `screenshot_after_ms: <ms>` after **every** command,
|
||||
so the frame folder is a complete visual record.
|
||||
4. **`session_stop`** when done, then `pkill -9 -f vita3k`. kwin-mcp always writes
|
||||
frames to `/tmp/kwin-mcp-screenshots-*/` (not configurable) — move them into the
|
||||
repo and clear tmp, then tell the human where they are:
|
||||
```bash
|
||||
cp /tmp/kwin-mcp-screenshots-*/*.png agent-run/frames/ 2>/dev/null
|
||||
rm -rf /tmp/kwin-mcp-screenshots-*
|
||||
```
|
||||
|
||||
### Key map (this app's actions; full list in `controls.MD`)
|
||||
|
||||
| Vita button | host key (kwin-mcp) | in-app action |
|
||||
|-------------|------------------------------|----------------------|
|
||||
| D-pad | `Right` `Left` `Up` `Down` | move grid / browse |
|
||||
| Cross (X) | `x` | view / play / replay |
|
||||
| Circle (O) | `c` | back |
|
||||
| Select | `Shift_R` | sync |
|
||||
| Start | `Return` | exit |
|
||||
|
||||
---
|
||||
|
||||
## 4. Read the logs — verify from TEXT, not frames
|
||||
|
||||
**Capturing a frame is free** (PNGs on disk are not in context). **Reading a PNG
|
||||
back into context is the expensive part.** So capture every command, but **do not
|
||||
Read frames back** during the loop. Verify progress from the app's own **text**
|
||||
log instead:
|
||||
|
||||
```
|
||||
~/.local/share/Vita3K/Vita3K/ux0/data/vitaimmich/log.txt
|
||||
```
|
||||
|
||||
This is the app's own log — it records page fetches, video open/play, scans, and
|
||||
errors. Also available:
|
||||
|
||||
- MCP **`read_app_log`** with the launched PID.
|
||||
- `/tmp/vita3k-runtime.log` (Vita3K runtime stdout/stderr from `deploy.sh`).
|
||||
- `/tmp/vita3k-install.log` (install output).
|
||||
|
||||
**Only Read a frame** when a screenshot is genuinely the only way to resolve an
|
||||
ambiguity (rare). Reading every frame is what bloats context — that's what this
|
||||
rule avoids.
|
||||
|
||||
### Typical verify loop
|
||||
|
||||
1. Build (step 1) → install half of step 2.
|
||||
2. `rm -rf agent-run/frames && mkdir -p agent-run/frames`, then `session_start`
|
||||
(`keep_screenshots: true`, `env` forcing Wayland) → wait ~12s.
|
||||
3. `focus_window "Vita3K"`.
|
||||
4. `keyboard_key "Right"` (+ `screenshot_after_ms`), stepping to the target tile,
|
||||
one key per call; `x` to open, `x` again to play.
|
||||
5. Confirm what happened from `log.txt` — **not** the frames.
|
||||
6. `session_stop` + `pkill`; move frames into the repo and clear tmp
|
||||
(`cp /tmp/kwin-mcp-screenshots-*/*.png agent-run/frames/ 2>/dev/null && rm -rf /tmp/kwin-mcp-screenshots-*`),
|
||||
then hand `agent-run/frames/` to the human as the visual record.
|
||||
|
||||
---
|
||||
Reference in New Issue
Block a user