Skip to content

feat(automation): drive the game by script from an [auto:ui] log - #158

Merged
NoRain211 merged 13 commits into
NoRain211:mainfrom
g-freeks:automation-driver
Oct 27, 2026
Merged

NoRain211 merged 13 commits into
NoRain211:mainfrom
g-freeks:automation-driver

Conversation

@g-freeks

@g-freeks g-freeks commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Summary

This lets a script play the game and check where it is, so features can be tested without someone at the pad. No linked issue.

RECOMP_AUTO_LOG=1 (recomp-runtime/ui_log.c) prints an [auto:ui] line when the game changes screen or moves a cursor. It never writes guest memory. It covers:

  • the scene and the menu screen;
  • the save select, and the game's list widgets as numbered lists with the game's own item text. List 0 holds the main, pool, hotel and Casino menus; list 2 holds the save prompt;
  • the island place (map, shop, items, Collection, pool or hotel, rest, Casino), the part of the day and the map location;
  • the shop's cursor and its Buy/Wrap prompts;
  • the items and Collection screens' tabs and cursors;
  • a new vacation's opening scene and character select;
  • a conversation flag (dialog open|closed).

Every line meant for scripts starts with [auto:<source>], so ^\[auto: picks them out.

tools/drive_game.py sends pad presses through the runner's existing --input-paused-analog-file. After each press it waits for the log line that shows the press landed, and presses again when the game ignored one. A session folder under private/ keeps the game running between commands.

  • Steps: press, wait, until, goto (map), select (list menu), rest, buy and shot.
  • Routes from launch: route shop|map|pool|items|hotel [--vacation N], and route new [--vacation N] [--girl NAME], which starts a new vacation, skips the opening scene, picks the girl and gets through the welcome dialogs.
  • Saves: they are copied at start and put back on stop if the run changed them.
  • Screenshots: RECOMP_SCREENSHOT_TRIGGER saves the next frame whenever a given file appears, so scripts need no window capture.

Addresses, the log's line grammar and how each address was found (RAM dumps diffed across screens and cursor positions, each checked against a screenshot) are in docs/ui-log.md. Usage, the game's day flow and its pitfalls are in docs/automation.md. The pitfalls include the pool menu hiding itself, an A on the hotel menu entering the Casino, and the welcome tour ending in the Accessory Shop.

Seven commits: the log and driver, the hotel's night, starting a new vacation, the CHANGELOG entry, then three fixes: waits look only after the newest press, the controller-mode preference is put back after a run, and start puts back the saves of a run that ended without stop.

Verification

Linux (CachyOS), on this branch rebased onto ab007e4:

  • python -m unittest discover -s tools -p "test_*.py" passes (1 skipped).

    • test_drive_game.py runs the driver against a fake game thread: lost presses, the shared mark, goto direction, waking the pool menu, rest, buy, finish_dialogs, refusing a slot with a save, and restoring saves.
  • python tools/public_export.py verify --require-public-tree passes.

  • Public runtime build and ctest: 41/41 with GCC. (Host clang passed 39/39 on the earlier base 3243911; not rerun.)

    • ui_log_test covers every line kind on synthetic memory.
    • screenshot_test covers the trigger file.
  • MSVC and llvm-mingw were not run locally; there is no Windows toolchain here.

    • The same three commits before the rebase passed public-ci on the fork (custody, runtime-tests, runtime-tests-clang, runtime-tests-macos).
    • public-ci on this PR covers the rebased commits.
  • Full-program build from a user-owned disc, on the earlier base 3243911:

    • lifter 7adaf210a89814c5ca4444e4876fccb51b938878;
    • generation parity exact-local-match;
    • executable SHA-256 053d44e885fa33c1d15d909a533f39dfbd976e97eeaf67e4fdef8438ea7e5c54.
  • Observed in the game on the earlier base 3243911 (not rerun since), each route from a fresh launch to its natural end:

    Route Ends at Time
    route shop Sports Shop list, cursor line 16 s
    route items items screen 18 s
    route hotel rests until night, through the gift dialogs, to the hotel menu 53 s
    route new --girl Tina opening, character select, welcome, Accessory Shop 71 s
    • Earlier runs on the fork covered buy (Buy and Wrap prompts) and the Collection and Casino screens.
    • Runs that wrote the save (night, new vacation) had it restored by stop. The restored saves matched a snapshot taken before the runs.

Not verified: the tool's Windows process handling (taskkill, tasklist, process groups) was written but not run on Windows.

Contribution checklist

  • The change follows AGENTS.md and CONTRIBUTING.md.
  • No XBE, generated game C, retail instruction bytes, extracted assets or
    filenames, saves, BIOS data, private paths, or run evidence are included.
  • Third-party material and its license are identified, if applicable. (None.)

🤖 Generated with Claude Code

@norain211-mira

norain211-mira Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

Mira PR Walkthrough

The PR description covers a full automation feature: an [auto:ui] UI-state log in the runtime (RECOMP_AUTO_LOG), the tools/drive_game.py script driver with routes and save restore, a screenshot trigger, and docs. The diff provided here contains only one file, docs/automation.md, with a small edit (3 lines added, 3 removed) in one hunk around line 107. It's probably a wording or accuracy fix to the usage or pitfalls notes. Because the change is documentation only, it carries almost no runtime risk.

Confidence: 5/5   ◉◉◉◉◉   Safe to merge
  • The visible change is a 3-line documentation edit with no code or build impact.

1 file reviewed


Comment @norain211-mira help to get the list of available commands and usage tips.

@norain211-mira norain211-mira Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mira Review Summary

This PR has one blocker: in tools/drive_game.py:148, start overwrites a save backup left over from an earlier run that was never restored, so the player's original saves can be permanently lost if a previous run crashed or the route failed. This needs to be fixed before merge.

Key Issues

Issue Location
🔴 start deletes an earlier run's save backup that was never restored, so the player's original saves can be lost after a crash or a failed route tools/drive_game.py:148

Comment thread tools/drive_game.py Resolved
g-freeks and others added 7 commits October 24, 2026 19:45
RECOMP_AUTO_LOG=1 prints an [auto:ui] line whenever the game changes
screen or moves a cursor: scene, menu screen, save select, the shared
list menu (main and pool menus), island place, map location, the shop's
cursor and Buy/Wrap prompts, and the items screen's tab and cursor.
Every automation line uses the [auto:<source>] prefix so tools can pick
them out.

tools/drive_game.py presses pad buttons through the runner's paused
input file and waits for the line that shows each press landed. It has
routes to the shop, island map, pool and items screen and a buy step,
keeps the player's saves, and takes screenshots through the new
RECOMP_SCREENSHOT_TRIGGER without a key.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The [auto:ui] log now names the part of the day (0x41EFEB) and the
island places past the shops and the pool: the Collection (6), a rest
at the pool (7) and the Casino (8). Place 5 is the pool by day and the
hotel at night. The Collection screen logs its tab and its cursor over
all pages (0x31CC7D, 0x31CE0C); the items cursor is read as a word.

drive_game.py gains a rest step and a hotel route: it rests at the
pool until night, moves through the night's gift dialogs with A and
stops at the hotel's menu, walking back from the Casino if a press got
there first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The [auto:ui] log now covers the game's whole array of list widgets
from 0x5C5800 as numbered lists ("list K cursor C of N: TEXT"), so the
save prompt reads the game's own items (Continue, Options, Delete a
saved game; New Game and Options for an empty vacation) instead of a
guessed table. It names a new vacation's opening scene (island place
17) and character select (place 0, "pick cursor C NAME"), and logs
"dialog open|closed" from the conversation flag at 0x5C5FED.

drive_game.py gains "route new [--vacation N] [--girl NAME]": it
refuses a vacation that has a save, skips the opening with START, picks
the girl by name and presses A only while a dialog is open, so the
welcome ends in the Accessory Shop where its tour leads and no press
goes past it. Continuing routes take --vacation too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A press did not move the mark, so lines an earlier press command caused
stayed after it. `until up "Options"` run after two separate Down presses
matched the Options line the first Down had logged, pressed nothing and
let the next A confirm "Delete a saved game" instead. A press now moves
the mark to the end of the log before it presses, so a wait after it only
sees what followed it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Controller Settings writes the Analog/Digital choice to
.recomp-storage/controller-mode-v1, outside the saves the driver backs
up. A scripted run that changed the mode left the player's preference
changed. start now records the file (or its absence) and stop puts it
back unless --keep-save is given.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
start deleted saves-at-start and copied the current saves as the new
baseline. When the last run wrote the saves and then crashed or was killed
without stop, that threw away the only copy of the player's saves. start
now runs the same put-back as stop first; the controller-mode preference
had the same gap and gets the same fix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
g-freeks and others added 2 commits October 24, 2026 21:10
A scripted start while a person played the same disc failed at save
recovery, but its session kept the pid and a backup of the saves taken
at that moment. The next stop or start would have put that backup back
over everything the player saved since. start now refuses while another
runner has the same game open (Linux, through /proc), and a game that
exits before it polls the pad leaves no backup behind. Adds ls and rs
(thumbstick clicks) to the buttons.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Find another runner on the same disc through PowerShell on Windows and ps on macOS, check a running game without tasklist, take screenshots by the runtime's saved line so Windows can move the file, detach the game from the console, and resolve the session folder.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 17 files

Tip: cubic can generate docs of your entire codebase and keep them up to date. Try it here.

View guided diff | Re-trigger cubic

Comment thread tools/drive_game.py Resolved
Comment thread tools/drive_game.py Resolved
Comment thread recomp-runtime/screenshot_test.c Resolved
Comment thread docs/free-camera.md Resolved
Comment thread tools/test_drive_game.py Resolved
Comment thread recomp-runtime/screenshot.c Resolved
Comment thread docs/ui-log.md Resolved
Comment thread CHANGELOG.md Resolved
Comment thread docs/ui-log.md Resolved
Comment thread docs/automation.md Resolved
g-freeks and others added 3 commits October 25, 2026 13:33
Driver:
- A pid read from session.json is trusted only if it is still a runner on
  the session's disc, so stop cannot end a process that reused it.
- A run that starts with no saves (a new vacation) has the saves it made
  moved aside on stop; before, they stayed.
- stop puts back only once per run. A second stop after a finished run
  used to put the old backup back over what the player saved since.
- --xbe now applies beside the selected build's runner.
- shot NAME rejects paths; same_tree counts file/folder mismatches; a
  game that exits before polling also deletes the backup folder.

Screenshot trigger: Windows reads the path as UTF-16 and keeps it as
UTF-8; a trigger file that cannot be removed turns the trigger off
instead of requesting a screenshot every frame; the test clears a stale
trigger first.

Tests cover the fresh profile, the second stop, pid identity, shot
names, a wrapped purchase, a full new vacation and --xbe. Docs describe
press without a wait, held frames, frame numbering and what is
read-only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fake game read the press file while the driver renamed it into place. Windows refused with a sharing violation, the thread died, and later presses went unread (1 in 24 runs under load). The runner already retries this error.
At night the island map offers only the hotel, so goto pressed for over a minute and then reported it could not reach the place. Routes other than map now stop when the map logs day part night.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 3 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

View guided diff | Re-trigger cubic

Comment thread docs/automation.md Resolved
@NoRain211
NoRain211 merged commit bc684f8 into NoRain211:main Oct 27, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants