Video bitrate probe, loading throbbers, boot spinner, transparent logo

- Video: probe the MP4 header (range request for moov) before downloading
  the whole clip; if the average bitrate overruns the declared H.264 level
  the Vita's decoder enforces, show the "too high" message up front instead
  of downloading then failing at playback. Verdict validated against the
  real demo videos (e.g. e75d495e: Level 3.1 @ 18.8 Mbps -> rejected).
- Error screens: word-wrap title + detail so long messages no longer run
  off the edges; the bitrate message now spells out the server-side fix
  (Immich transcoding policy / bitrate cap).
- Boot: one continuous spinner. A single boot worker thread does sign-in +
  first page + local scan while the main thread animates one logo with a
  never-resetting frame counter, instead of three separate static screens.
- Throbbers: the cloud/server-info page and smart search now fetch on a
  worker thread and animate a throbber instead of freezing the UI.
- Logo: swap the in-app logo for a clean 200x200 transparent PNG (the old
  one was matted from the white-background icon and had fringe); keep the
  launcher icon's white background. Linear-filter the logo + cloud badges
  so they scale/rotate smoothly.
- Drop AgenticLoop.md (and its CLAUDE.md reference).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Sads
2026-06-14 00:48:42 +02:00
co-authored by Claude Opus 4.8
parent 341d913346
commit 351804819b
8 changed files with 442 additions and 185 deletions
-136
View File
@@ -1,136 +0,0 @@
# Agentic loop: build, run & drive vitaImmich in Vita3K
This is the loop an agent uses to make a change, run it in the Vita3K emulator,
and verify it by sending input + reading screenshots — all headless, on KDE
Plasma (Wayland).
## One-time setup
- VitaSDK (community): native toolchain at `~/vitasdk` (`arm-vita-eabi-gcc`,
`vita-pack-vpk`, toolchain at `share/vita.toolchain.cmake`).
- Vita3K installed and on `PATH` (`/usr/bin/vita3k`).
- The MCP that drives the GUI: `uv tool install kwin-mcp` (configured in
`.mcp.json` as the `kwin-mcp` server).
## 1. Build the VPK → `build/vitaImmich.vpk`
Preferred (Docker, reproducible):
```bash
./build.sh # writes build/vitaImmich.vpk (build/ ends up root-owned)
```
Fallback when the Docker runtime is broken (e.g. containerd shim error
`readlink /proc/self/exe: no such file or directory`), build natively with the
host SDK into `build-native/` (git-ignored):
```bash
export VITASDK="$HOME/vitasdk"
export PATH=$VITASDK/bin:$PATH
cmake -B build-native -DCMAKE_BUILD_TYPE=Release \
-DCMAKE_TOOLCHAIN_FILE=$VITASDK/share/vita.toolchain.cmake .
cmake --build build-native --parallel "$(nproc)"
```
## 2. Install the VPK into Vita3K
Installing = launch Vita3K once with the `.vpk` as an argument; it extracts and
auto-boots. Gotchas:
- Run it **from a script file**, backgrounded with a wait then killed. Inline
backgrounding from the agent's shell does not persist.
- Guard `pkill` with `|| true` — a no-match exit code aborts the chain.
- Vita3K extracts files preserving the **zip's internal mtime**, so to confirm a
fresh install compare the installed `eboot.bin` mtime against
`unzip -l <vpk> | grep eboot`, not wall-clock time.
```bash
#!/bin/bash
cd "$(git rev-parse --show-toplevel)"
pkill -9 -f vita3k 2>/dev/null || true
sleep 1
vita3k ./build-native/vitaImmich.vpk >/tmp/vinst.log 2>&1 & # or ./build/...
sleep 8
pkill -9 -f vita3k 2>/dev/null || true
sleep 1
grep -i "successfully" /tmp/vinst.log | tail -1
stat -c 'eboot: %y' ~/.local/share/Vita3K/Vita3K/ux0/app/VIMM00001/eboot.bin
```
App title id: **`VIMM00001`**. To wipe a stale install: `vita3k -d VIMM00001`.
## 3. Launch + drive via kwin-mcp
Launch the app **inside the MCP's own isolated KWin session** — its screenshot
tool works there. (Connecting to the live desktop with `session_connect` does
*not* work for capture: `spectacle` can't save non-interactively, and KWin's
`ScreenShot2` DBus API rejects unauthorized callers — so the isolated session is
the only reliable way to capture, and it is headless: the human cannot watch it
live on their monitor.)
- **Before starting, wipe the per-session frame folder** so each run is clean:
`rm -rf agent-run/frames && mkdir -p agent-run/frames`. (`agent-run/` is
git-ignored; `frames/` is the throwaway capture folder — keepers live in
`agent-run/` proper and are never touched.)
- `session_start` with `app_command: "vita3k -r VIMM00001"`,
`keep_screenshots: true`, **and `env: {"SDL_VIDEODRIVER": "wayland", "DISPLAY":
""}`** — starts an isolated Wayland session, launches the app, and keeps every
captured frame on disk after `session_stop`. Without that `env` the frames come
back blank white and input is dropped (host `DISPLAY=:1` sends SDL to X11). Give
it ~12s to boot.
- **`focus_window` with `app_name: "Vita3K"` BEFORE sending keys.** Without focus
the keypresses are dropped. Presses also drop intermittently — re-focus and
re-press. Send one key per call; rapid bursts get coalesced/dropped.
- `keyboard_key` with `screenshot_after_ms: [ms]` after **every** command, so the
frame folder is a complete visual record of the run for the human to scrub.
- `session_stop` when done (and `pkill -9 -f vita3k`). kwin-mcp always writes
frames to `/tmp/kwin-mcp-screenshots-*/` (that path is not configurable); don't
leave them there — **move them into the repo and clear the tmp folder**:
```bash
cp /tmp/kwin-mcp-screenshots-*/*.png agent-run/frames/ 2>/dev/null
rm -rf /tmp/kwin-mcp-screenshots-*
```
Then tell the human the frames are in `agent-run/frames/`.
### Zero-context-cost capture (the standing rule)
Capturing a frame (writing the PNG) costs the agent **nothing** — files on disk
are not in context. The cost is only incurred when the agent **Reads** a PNG
back. So:
- **Capture every command** (`screenshot_after_ms`) — free, and it gives the
human the full record.
- **Do not Read frames back** into context during the loop. Verify progress from
the app's own **text** log instead (cheap), not from images:
`~/.local/share/Vita3K/Vita3K/ux0/data/vitaimmich/log.txt`.
- Only Read a frame when a screenshot is genuinely the only way to resolve an
ambiguity (rare). Reading every frame is what bloats context and slows the
session — that is exactly what this rule avoids.
### Key mapping (full list in `controls.MD`)
Vita3K maps host keys → Vita buttons. The ones this app uses:
| Vita button | host key (kwin-mcp) | in-app action |
|-------------|---------------------|--------------------------|
| D-pad | `Right` `Left` `Up` `Down` | move grid / browse |
| Cross (X) | `x` | view / play / replay |
| Circle (O) | `c` | back |
| Select | `Shift_R` | sync |
| Start | `Return` | exit |
### Typical verify loop (capture everything, Read nothing)
1. `rm -rf agent-run/frames && mkdir -p agent-run/frames`, then `session_start`
(`keep_screenshots: true`, `env` forcing Wayland) → wait ~12s.
2. `focus_window "Vita3K"`.
3. `keyboard_key "Right"` (with `screenshot_after_ms`) … stepping to the target
tile, one key per call.
4. `keyboard_key "x"` to open, `x` again to play.
5. Confirm what happened from the **text** log, not the frames:
- `~/.local/share/Vita3K/Vita3K/ux0/data/vitaimmich/log.txt` (the app's own log
— records page fetches, video open/play, scans, errors)
- MCP `read_app_log` with the launched PID, or `/tmp/vita3k-runtime.log`.
6. On `session_stop`, move the frames out of `/tmp` and clear it:
`cp /tmp/kwin-mcp-screenshots-*/*.png agent-run/frames/ 2>/dev/null && rm -rf /tmp/kwin-mcp-screenshots-*`.
Hand `agent-run/frames/` to the human as the visual record.
+2 -2
View File
@@ -10,8 +10,8 @@ just it); the parts are textual slices, so add new code to the relevant part,
not to CMake. `jsmn.h` is vendored. not to CMake. `jsmn.h` is vendored.
Built with the VitaSDK toolchain into a `.vpk`, run in the **Vita3K** emulator, Built with the VitaSDK toolchain into a `.vpk`, run in the **Vita3K** emulator,
and driven headlessly through the **kwin-mcp** server. This file is the and driven headlessly through the **kwin-mcp** server. This file is the
end-to-end loop: **build → install → launch → drive → read logs.** For the end-to-end loop: **build → install → launch → drive → read logs.** Full key
narrative version see `AgenticLoop.md`; full key map is in `controls.MD`. map is in `controls.MD`.
App title id: **`VIMM00001`**. App title id: **`VIMM00001`**.
Binary file not shown.

Before

Width:  |  Height:  |  Size: 11 KiB

After

Width:  |  Height:  |  Size: 8.6 KiB

+131 -36
View File
@@ -11,15 +11,48 @@
enum { MODE_GRID, MODE_DETAIL, MODE_CLOUD, MODE_CLOUD_DETAILS }; enum { MODE_GRID, MODE_DETAIL, MODE_CLOUD, MODE_CLOUD_DETAILS };
/* the first library page is fetched on this thread so the main thread can /* the whole boot workload (auto sign-in, first library page, local media scan)
* spin the loading logo while it waits */ * runs on this one thread so the main thread can animate a single, continuous
static volatile int g_init_load_done; * spinner from launch to grid instead of flashing a separate static screen per
static int init_load_thread(SceSize args, void *argp) * step. g_boot_status is the caption it shows; g_boot_done signals completion;
* g_boot_login_failed asks the main thread to drop to the interactive login. */
static char g_boot_status[48] = "Loading...";
static volatile int g_boot_done;
static volatile int g_boot_login_failed;
static int boot_thread(SceSize args, void *argp)
{ {
(void)args; (void)argp; (void)args; (void)argp;
fetch_page(0); /* stored email/password (no API key, not already signed in): try once */
if (!g_apikey[0] && g_email[0] && !g_token[0]) {
snprintf(g_boot_status, sizeof(g_boot_status), "Signing in...");
char lerr[160];
if (do_login(lerr, sizeof(lerr)) != 0) {
g_boot_login_failed = 1;
__sync_synchronize(); __sync_synchronize();
g_init_load_done = 1; g_boot_done = 1;
return 0;
}
}
snprintf(g_boot_status, sizeof(g_boot_status), "Loading your library...");
fetch_page(0);
snprintf(g_boot_status, sizeof(g_boot_status), "Scanning local media...");
scan_local_media();
/* reserve headroom so a resume-from-sleep rescan can append new camera
* files without reallocating while the worker/sync threads read */
grow_locals(g_local_count + 512);
rebuild_display();
__sync_synchronize();
g_boot_done = 1;
return 0;
}
/* fetch the cloud page's server info off the main thread, so the page can
* animate a throbber while the request is in flight (fetch_server_info sets
* g_srv_state to 1/-1 when it finishes) */
static int srvinfo_thread(SceSize args, void *argp)
{
(void)args; (void)argp;
fetch_server_info();
return 0; return 0;
} }
@@ -69,6 +102,7 @@ static void reset_for_account_change(void)
g_search_active = 0; g_search_active = 0;
g_search_count = 0; g_search_count = 0;
g_srv_state = 0; /* re-fetch the cloud page's server info */ g_srv_state = 0; /* re-fetch the cloud page's server info */
g_srv_fetching = 0;
__sync_synchronize(); __sync_synchronize();
/* reload from the new account (sync still paused), then resume */ /* reload from the new account (sync still paused), then resume */
@@ -89,6 +123,14 @@ int main(void)
g_ic_server = vita2d_load_PNG_file("app0:cloud_server.png"); g_ic_server = vita2d_load_PNG_file("app0:cloud_server.png");
g_ic_device = vita2d_load_PNG_file("app0:cloud_device.png"); g_ic_device = vita2d_load_PNG_file("app0:cloud_device.png");
g_ic_both = vita2d_load_PNG_file("app0:cloud_both.png"); g_ic_both = vita2d_load_PNG_file("app0:cloud_both.png");
/* linear (bilinear) sampling so these scale/rotate smoothly instead of
* showing jagged stair-stepped edges (point sampling is the default) */
vita2d_texture *icons[] = { g_logo, g_ic_server, g_ic_device, g_ic_both };
for (unsigned i = 0; i < sizeof(icons) / sizeof(icons[0]); i++)
if (icons[i])
vita2d_texture_set_filters(icons[i],
SCE_GXM_TEXTURE_FILTER_LINEAR,
SCE_GXM_TEXTURE_FILTER_LINEAR);
/* on-screen keyboard (smart search). the IME runs as a common dialog; /* on-screen keyboard (smart search). the IME runs as a common dialog;
* the config tells it the system language + enter/cancel button map */ * the config tells it the system language + enter/cancel button map */
@@ -164,30 +206,40 @@ int main(void)
log_line("server='%s' len=%d hex=%s", g_server, n, hex); log_line("server='%s' len=%d hex=%s", g_server, n, hex);
} }
/* auth: API key needs nothing; email/password from config is tried once; /* auth: an API key needs nothing; missing config drops to interactive
* anything missing or a failed login drops to the interactive sign-in */ * sign-in here. stored email/password is tried inside the boot thread. */
if (!have_cfg) { if (!have_cfg)
login_screen(); login_screen();
} else if (!g_apikey[0] && g_email[0]) {
draw_loading("Signing in...", 0);
char lerr[160];
if (do_login(lerr, sizeof(lerr)) != 0)
login_screen();
}
/* spinning-logo loading screen while the first page loads on a thread */ /* one continuous spinner: the boot thread signs in (if needed), fetches
g_init_load_done = 0; * the first page and scans local media while the main thread animates the
SceUID lt = sceKernelCreateThread("init_load", init_load_thread, * logo with a single, never-resetting frame counter. */
g_boot_done = 0;
g_boot_login_failed = 0;
SceUID bt = sceKernelCreateThread("boot", boot_thread,
0x10000100, 256 * 1024, 0, 0, NULL); 0x10000100, 256 * 1024, 0, 0, NULL);
if (lt >= 0) { if (bt >= 0) {
sceKernelStartThread(lt, 0, NULL); sceKernelStartThread(bt, 0, NULL);
unsigned int lf = 0; unsigned int bf = 0;
while (!g_init_load_done) { while (!g_boot_done) {
draw_loading("Loading your library...", lf++); draw_loading(g_boot_status, bf++);
sceDisplayWaitVblankStart(); sceDisplayWaitVblankStart();
} }
} else { } else {
fetch_page(1); /* fallback: blocking, no spinner */ /* fallback: run the same steps blocking, without the spinner */
fetch_page(1);
scan_local_media();
grow_locals(g_local_count + 512);
rebuild_display();
}
/* a stored-credential sign-in failed: drop to interactive login (main
* thread), then load blocking */
if (g_boot_login_failed) {
login_screen();
fetch_page(1);
scan_local_media();
grow_locals(g_local_count + 512);
rebuild_display();
} }
if (g_asset_count == 0) if (g_asset_count == 0)
fatal_error(NULL, "Server returned no assets (see log.txt)"); fatal_error(NULL, "Server returned no assets (see log.txt)");
@@ -197,15 +249,6 @@ int main(void)
if (worker >= 0) if (worker >= 0)
sceKernelStartThread(worker, 0, NULL); sceKernelStartThread(worker, 0, NULL);
/* scan the Vita's own camera media (fast: paths + stat only), build
* the merged timeline, then let the sync thread hash + dup-check it */
draw_loading("Scanning local media...", 0);
scan_local_media();
/* reserve headroom so a resume-from-sleep rescan can append new camera
* files without reallocating while the worker/sync threads read */
grow_locals(g_local_count + 512);
rebuild_display();
SceUID syncw = sceKernelCreateThread("sync_worker", sync_thread, SceUID syncw = sceKernelCreateThread("sync_worker", sync_thread,
0x10000100, 256 * 1024, 0, 0, NULL); 0x10000100, 256 * 1024, 0, 0, NULL);
if (syncw >= 0) if (syncw >= 0)
@@ -628,8 +671,38 @@ int main(void)
g_search_active ? g_search_query : "", g_search_active ? g_search_query : "",
q, sizeof(q), 0); q, sizeof(q), 0);
if (ok && q[0]) { if (ok && q[0]) {
/* run the search on a worker thread and
* spin a throbber while it's in flight */
snprintf(g_search_pending,
sizeof(g_search_pending), "%s", q);
g_search_done = 0;
SceUID stid = sceKernelCreateThread("search",
search_thread, 0x10000100,
128 * 1024, 0, 0, NULL);
int nres;
if (stid >= 0) {
sceKernelStartThread(stid, 0, NULL);
char cap[160];
snprintf(cap, sizeof(cap),
"Searching \"%s\"...", q);
unsigned int sf = 0;
while (!g_search_done) {
vita2d_start_drawing();
vita2d_clear_screen();
draw_throbber(SCREEN_W / 2.0f,
SCREEN_H / 2.0f - 16.0f,
22.0f, sf++);
draw_centered(SCREEN_H / 2 + 36,
RGBA8(200, 200, 200, 255),
cap);
vita2d_end_drawing();
vita2d_swap_buffers();
}
nres = g_search_result;
} else {
show_status("Searching \"%s\"...", q); show_status("Searching \"%s\"...", q);
int nres = run_smart_search(q); nres = run_smart_search(q);
}
if (nres >= 0) { if (nres >= 0) {
sel = 0; sel = 0;
scroll = target = 0; scroll = target = 0;
@@ -1277,8 +1350,30 @@ pf_skip:
/* server + backup overview, opened from the cloud /* server + backup overview, opened from the cloud
* button on the search bar */ * button on the search bar */
if (g_srv_state == 0) { if (g_srv_state == 0) {
show_status("Loading server info..."); /* fetch on a worker thread so the page animates a
fetch_server_info(); * throbber instead of freezing on the request */
if (!g_srv_fetching) {
g_srv_fetching = 1;
SceUID st = sceKernelCreateThread("srvinfo",
srvinfo_thread, 0x10000100,
64 * 1024, 0, 0, NULL);
if (st >= 0)
sceKernelStartThread(st, 0, NULL);
else
fetch_server_info(); /* fallback */
}
vita2d_start_drawing();
vita2d_clear_screen();
draw_throbber(SCREEN_W / 2.0f,
SCREEN_H / 2.0f - 16.0f, 22.0f, frame);
draw_centered(SCREEN_H / 2 + 36,
RGBA8(200, 200, 200, 255),
"Loading server info...");
if (pressed & SCE_CTRL_CIRCLE)
mode = MODE_GRID;
vita2d_end_drawing();
vita2d_swap_buffers();
continue;
} }
int total = g_local_count; int total = g_local_count;
+53
View File
@@ -64,6 +64,59 @@ static void draw_centered(int y, uint32_t color, const char *text)
draw_text((SCREEN_W - w) / 2, y, color, 1.0f, text); draw_text((SCREEN_W - w) / 2, y, color, 1.0f, text);
} }
/* draw `text` centered and word-wrapped so no line exceeds maxw pixels,
* stacking lines line_h px apart. honours explicit '\n' as a hard break.
* returns the y just past the last line drawn. used by the error screens,
* whose messages used to be one centered line that ran off both edges. */
static int draw_centered_wrapped(int y, uint32_t color, const char *text,
int maxw, int line_h)
{
char line[256];
line[0] = '\0';
const char *p = text;
for (;;) {
while (*p == ' ') /* collapse runs of spaces between words */
p++;
const char *w = p;
while (*p && *p != ' ' && *p != '\n')
p++;
int wlen = (int)(p - w);
int forced_nl = (*p == '\n');
int end = (*p == '\0');
if (wlen > 0) {
char cand[256];
if (line[0])
snprintf(cand, sizeof(cand), "%s %.*s", line, wlen, w);
else
snprintf(cand, sizeof(cand), "%.*s", wlen, w);
/* place the word if it fits, or if the line is empty (a
* single over-long word goes on its own line rather than
* looping forever) */
if (!line[0] || text_width(1.0f, cand) <= maxw) {
snprintf(line, sizeof(line), "%s", cand);
} else {
draw_centered(y, color, line);
y += line_h;
snprintf(line, sizeof(line), "%.*s", wlen, w);
}
}
if (forced_nl) {
draw_centered(y, color, line);
y += line_h;
line[0] = '\0';
p++; /* consume the '\n' */
}
if (end) {
if (line[0]) {
draw_centered(y, color, line);
y += line_h;
}
return y;
}
}
}
/* startup loading screen: the Immich logo spinning, with text below it */ /* startup loading screen: the Immich logo spinning, with text below it */
static void draw_loading(const char *text, unsigned int frame) static void draw_loading(const char *text, unsigned int frame)
{ {
+10 -1
View File
@@ -74,10 +74,19 @@ static char g_search_query[128];
static int *g_search_idx; static int *g_search_idx;
static int g_search_count; static int g_search_count;
static int g_search_cap; static int g_search_cap;
/* smart search runs on a worker thread so the grid can animate a throbber
* while the request is in flight; g_search_pending is the query it runs,
* g_search_done/g_search_result report completion + its return value */
static char g_search_pending[128];
static volatile int g_search_done;
static volatile int g_search_result;
/* server info shown on the cloud/backup page; fetched lazily the first time /* server info shown on the cloud/backup page; fetched lazily the first time
* the page is opened. g_srv_state: 0 not fetched, 1 ok, -1 failed. */ * the page is opened. g_srv_state: 0 not fetched, 1 ok, -1 failed.
* g_srv_fetching: a worker thread is fetching it now (so the page animates a
* throbber instead of blocking the main thread on the request). */
static int g_srv_state; static int g_srv_state;
static volatile int g_srv_fetching;
static char g_srv_use[40]; /* disk used, human ("1.4 TiB") */ static char g_srv_use[40]; /* disk used, human ("1.4 TiB") */
static char g_srv_total[40]; /* disk size, human ("1.8 TiB") */ static char g_srv_total[40]; /* disk size, human ("1.8 TiB") */
static int g_srv_pct; /* disk usage percentage */ static int g_srv_pct; /* disk usage percentage */
+13
View File
@@ -871,6 +871,19 @@ static int run_smart_search(const char *query)
return g_search_count; return g_search_count;
} }
/* run the (blocking) smart search off the main thread so the grid can animate
* a throbber while the request is in flight. the main thread only draws the
* throbber meanwhile, so it never touches the display model this rebuilds. */
static int search_thread(SceSize args, void *argp)
{
(void)args; (void)argp;
int r = run_smart_search(g_search_pending);
__sync_synchronize();
g_search_result = r;
g_search_done = 1;
return 0;
}
static void draw_hud(const char *text) static void draw_hud(const char *text)
{ {
(void)text; /* bottom HUD bar removed */ (void)text; /* bottom HUD bar removed */
+233 -10
View File
@@ -18,7 +18,16 @@
#define ALIGN_UP(x, a) (((x) + ((a) - 1)) & ~((a) - 1)) #define ALIGN_UP(x, a) (((x) + ((a) - 1)) & ~((a) - 1))
/* draw an error screen until O is pressed */ /* the server-side fix for a too-high-bitrate clip, appended to the relevant
* error messages so the user knows what to change on their Immich install */
#define SERVER_FIX_STEPS \
"To fix it on the server: in Immich open Administration -> Settings -> " \
"Video Transcoding, set Transcode policy to \"All videos\" and Max " \
"bitrate to 10000k (or Target resolution to 720p), save, then run the " \
"\"Transcode video\" job under Administration -> Jobs."
/* draw an error screen until O is pressed. title and detail are word-wrapped
* to the screen, so long messages no longer run off the edges. */
static void show_blocking_error(const char *title, const char *detail) static void show_blocking_error(const char *title, const char *detail)
{ {
SceCtrlData pad; SceCtrlData pad;
@@ -26,9 +35,11 @@ static void show_blocking_error(const char *title, const char *detail)
sceCtrlPeekBufferPositive(0, &pad, 1); sceCtrlPeekBufferPositive(0, &pad, 1);
vita2d_start_drawing(); vita2d_start_drawing();
vita2d_clear_screen(); vita2d_clear_screen();
draw_centered(230, RGBA8(255, 80, 80, 255), title); int y = draw_centered_wrapped(150, RGBA8(255, 80, 80, 255),
title, SCREEN_W - 80, 34);
if (detail && detail[0]) if (detail && detail[0])
draw_centered(280, RGBA8(200, 200, 200, 255), detail); draw_centered_wrapped(y + 16, RGBA8(205, 205, 205, 255),
detail, SCREEN_W - 80, 30);
draw_centered(510, RGBA8(160, 160, 160, 255), "O back"); draw_centered(510, RGBA8(160, 160, 160, 255), "O back");
vita2d_end_drawing(); vita2d_end_drawing();
vita2d_swap_buffers(); vita2d_swap_buffers();
@@ -688,13 +699,12 @@ replay:
* problem. Tell the user instead of holding on a black frame. */ * problem. Tell the user instead of holding on a black frame. */
log_line("video: active but 0 frames decoded (HW decoder " log_line("video: active but 0 frames decoded (HW decoder "
"rejected the stream, or running under Vita3K)"); "rejected the stream, or running under Vita3K)");
show_blocking_error("This video can't be decoded", show_blocking_error("This video's bitrate is too high to play",
"The player started but produced no frames. " "The player started but produced no frames. On a real "
"On a real Vita this usually means the clip's " "Vita this means the clip's bitrate exceeds the H.264 "
"bitrate exceeds the H.264 level it declares - " "level it declares, so the hardware decoder rejects it. "
"re-transcode it in Immich at a lower bitrate " SERVER_FIX_STEPS
"or 720p. (The Vita3K emulator decodes no video " "\n(The Vita3K emulator also decodes no video at all.)");
"at all.)");
} }
/* reached the end of the video: hold on the last frame with a replay /* reached the end of the video: hold on the last frame with a replay
@@ -745,6 +755,193 @@ replay:
__sync_synchronize(); __sync_synchronize();
} }
/* ---- pre-download bitrate probe ------------------------------------------
* Before pulling the whole stream we fetch just the MP4 header (the small
* `moov` box) and decide whether the Vita's hardware H.264 decoder will accept
* it. Everything we need is in `moov`: `avcC` carries the declared H.264 level
* (which the decoder sizes its buffers from and refuses to overrun), and the
* total file size (from the range response) over the `mvhd` duration gives the
* average bitrate. If that bitrate exceeds the level's ceiling we show the
* "too high" message immediately instead of downloading the whole clip only to
* fail at playback. Immich's transcodes are faststart (moov at the front), so
* a small head request usually has it; if not, we try the tail. The estimate
* is approximate (average vs the decoder's peak/buffer limit), so this only
* blocks clear cases and otherwise defers to the real playback attempt. */
enum { PROBE_OK = 0, PROBE_TOO_HIGH = 1, PROBE_UNKNOWN = -1 };
#define PROBE_HEAD_BYTES (256 * 1024)
static uint32_t rd_be32(const unsigned char *p)
{
return ((uint32_t)p[0] << 24) | ((uint32_t)p[1] << 16) |
((uint32_t)p[2] << 8) | p[3];
}
static uint64_t rd_be64(const unsigned char *p)
{
return ((uint64_t)rd_be32(p) << 32) | rd_be32(p + 4);
}
/* locate a 4CC box tag in a buffer; returns a pointer to the tag (payload
* follows immediately after the 4 bytes), or NULL. a plain scan is enough for
* the tags we want (avcC/mvhd/moov) — collisions in real files are negligible. */
static const unsigned char *find_tag(const unsigned char *buf, size_t len,
const char *tag)
{
if (len >= 4)
for (size_t i = 0; i + 4 <= len; i++)
if (buf[i] == (unsigned char)tag[0] &&
!memcmp(buf + i, tag, 4))
return buf + i;
return NULL;
}
/* the H.264 level's bitrate ceiling in Mbps (High-profile MaxBR). the decoder
* keys its buffer sizing off this, so a stream above it is what gets rejected. */
static double level_ceiling_mbps(int lvl)
{
switch (lvl) {
case 30: return 12.5;
case 31: return 17.5;
case 32: return 25.0;
case 40: case 41: case 42: return 62.5;
case 50: return 168.75;
case 51: case 52: return 300.0;
default: return lvl >= 51 ? 300.0 : lvl >= 40 ? 62.5 : 17.5;
}
}
/* capture the total file size from a 206's "Content-Range: bytes a-b/TOTAL" */
static size_t probe_hdr_cb(char *b, size_t s, size_t n, void *ud)
{
size_t len = s * n;
long long *total = ud;
if (len > 14 && !strncasecmp(b, "Content-Range:", 14)) {
char *slash = memchr(b, '/', len);
if (slash)
*total = strtoll(slash + 1, NULL, 10);
}
return len;
}
/* range GET [start,end] of the asset's playback stream into `out`; sets
* *total to the full file size from Content-Range. returns the HTTP status,
* or -1 on transport error. */
static int http_get_range(int idx, long start, long end, membuf *out,
long long *total)
{
char url[700];
snprintf(url, sizeof(url), "%s/api/assets/%s/video/playback",
g_server, g_asset_ids[idx]);
CURL *curl = curl_easy_init();
if (!curl)
return -1;
char keyhdr[300];
auth_header(keyhdr, sizeof(keyhdr));
struct curl_slist *hdrs = curl_slist_append(NULL, keyhdr);
char range[64];
snprintf(range, sizeof(range), "%ld-%ld", start, end);
out->data = NULL;
out->size = 0;
curl_easy_setopt(curl, CURLOPT_URL, url);
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, hdrs);
curl_easy_setopt(curl, CURLOPT_RANGE, range);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_cb);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, out);
curl_easy_setopt(curl, CURLOPT_HEADERFUNCTION, probe_hdr_cb);
curl_easy_setopt(curl, CURLOPT_HEADERDATA, total);
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
curl_easy_setopt(curl, CURLOPT_CONNECTTIMEOUT, 10L);
curl_easy_setopt(curl, CURLOPT_TIMEOUT, 20L);
curl_easy_setopt(curl, CURLOPT_USERAGENT, "vitaImmich/0.1 (PS Vita)");
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0L);
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, 0L);
if (g_resolve_list)
curl_easy_setopt(curl, CURLOPT_RESOLVE, g_resolve_list);
net_lock();
CURLcode res = curl_easy_perform(curl);
net_unlock();
long code = 0;
if (res == CURLE_OK)
curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &code);
curl_slist_free_all(hdrs);
curl_easy_cleanup(curl);
if (res != CURLE_OK) {
free(out->data);
out->data = NULL;
return -1;
}
return (int)code;
}
/* decide from the header whether the clip is decodable. on TOO_HIGH, fills
* *level (level_idc, e.g. 31) and *mbps (estimated average bitrate). */
static int probe_video_bitrate(int idx, int *level, double *mbps)
{
membuf head;
long long total = -1;
int code = http_get_range(idx, 0, PROBE_HEAD_BYTES - 1, &head, &total);
/* need a real partial response (206) with a known total to estimate */
if (code != 206 || total <= 0) {
log_line("video probe %s: no usable range (code=%d total=%lld)",
g_asset_ids[idx], code, total);
free(head.data);
return PROBE_UNKNOWN;
}
const unsigned char *sbuf = (const unsigned char *)head.data;
size_t slen = head.size;
membuf tail;
tail.data = NULL;
/* faststart files have moov up front; otherwise grab the tail */
if (!find_tag(sbuf, slen, "moov") && total > (long long)head.size) {
long ts = total > PROBE_HEAD_BYTES ? (long)(total - PROBE_HEAD_BYTES) : 0;
long long t2 = -1;
if (http_get_range(idx, ts, total - 1, &tail, &t2) == 206) {
sbuf = (const unsigned char *)tail.data;
slen = tail.size;
}
}
const unsigned char *avcc = find_tag(sbuf, slen, "avcC");
const unsigned char *mvhd = find_tag(sbuf, slen, "mvhd");
int verdict = PROBE_UNKNOWN;
/* avcc payload[3] = level_idc; mvhd (v1) reads a 64-bit field at +24,
* so the payload must have 32 bytes available */
if (avcc && avcc + 4 + 4 <= sbuf + slen && mvhd && mvhd + 4 + 32 <= sbuf + slen) {
int lvl = avcc[4 + 3];
const unsigned char *m = mvhd + 4;
uint32_t timescale;
uint64_t duration;
if (m[0] == 1) { /* version 1: 64-bit times */
timescale = rd_be32(m + 20);
duration = rd_be64(m + 24);
} else { /* version 0: 32-bit times */
timescale = rd_be32(m + 12);
duration = rd_be32(m + 16);
}
double secs = timescale ? (double)duration / timescale : 0;
if (secs > 0.5 && lvl > 0) {
double est = (double)total * 8.0 / secs / 1e6;
double ceil_mbps = level_ceiling_mbps(lvl);
verdict = est > ceil_mbps ? PROBE_TOO_HIGH : PROBE_OK;
*level = lvl;
*mbps = est;
log_line("video probe %s: level %d.%d, ~%.1f Mbps "
"(%lld B / %.1fs), ceiling %.1f -> %s",
g_asset_ids[idx], lvl / 10, lvl % 10, est,
(long long)total, secs, ceil_mbps,
verdict == PROBE_TOO_HIGH ? "reject" : "ok");
}
}
if (verdict == PROBE_UNKNOWN)
log_line("video probe %s: header incomplete (avcC=%p mvhd=%p) - "
"deferring to playback", g_asset_ids[idx],
(const void *)avcc, (const void *)mvhd);
free(head.data);
free(tail.data);
return verdict;
}
/* download + play asset idx, cleaning up the temp file afterwards */ /* download + play asset idx, cleaning up the temp file afterwards */
static void view_video(int idx) static void view_video(int idx)
{ {
@@ -755,6 +952,32 @@ static void view_video(int idx)
g_dl_bg = g_thumb[idx]; g_dl_bg = g_thumb[idx];
g_video_rot = ori_to_deg(g_asset_rot[idx]); g_video_rot = ori_to_deg(g_asset_rot[idx]);
/* check the header before committing to the full download, so an
* unplayable (too-high-bitrate) clip is caught up front */
vita2d_start_drawing();
vita2d_clear_screen();
if (g_dl_bg) {
draw_texture_fitted(g_dl_bg, 0, 0, SCREEN_W, SCREEN_H);
vita2d_draw_rectangle(0, 0, SCREEN_W, SCREEN_H, RGBA8(0, 0, 0, 130));
}
draw_centered(SCREEN_H / 2, RGBA8(255, 255, 255, 255), "Checking video…");
vita2d_end_drawing();
vita2d_swap_buffers();
int level = 0;
double mbps = 0;
if (probe_video_bitrate(idx, &level, &mbps) == PROBE_TOO_HIGH) {
char det[512];
snprintf(det, sizeof(det),
"This clip is about %.0f Mbps but declares H.264 Level "
"%d.%d, whose bitrate ceiling the Vita's hardware decoder "
"enforces. " SERVER_FIX_STEPS,
mbps, level / 10, level % 10);
show_blocking_error("This video's bitrate is too high to play", det);
g_dl_bg = NULL;
return;
}
char err[160]; char err[160];
int r = download_video(idx, err, sizeof(err)); int r = download_video(idx, err, sizeof(err));
if (r == 0) { if (r == 0) {