cookbook(action): add RoboCasa mobile-manipulation post-training recipe - #318
cookbook(action): add RoboCasa mobile-manipulation post-training recipe#318hyzhou-nv wants to merge 1 commit into
Conversation
|
Thanks! In general it LGTM, but can we keep this recipe focused on training? So files are organized in the same way of other sft recipes. Maybe we can move the eval/dataset related scripts to cosmos-framework and add the usage guidance in the readme? |
0a8d25e to
2161358
Compare
Thanks for the suggestion! Both eval and dataset scripts are moved to cosmos-framework repo. |
|
Seems a lot of changed unrelated to robocasa is added? |
2161358 to
9079a60
Compare
Adds a RoboCasa recipe to the action finetune cookbook, organized like the DROID and LIBERO ones: a training launcher, the Nano SFT TOML, and README coverage. Evaluation and dataset conversion live in cosmos-framework, so the recipe folder holds training files only. RoboCasa is mobile manipulation, so the base is not fixed. The recipe uses a 15-D raw action contract that passes the native base command through unchanged, an agentview_left | eye_in_hand composite at full resolution, and EEF proprioception as a clean condition token. `TASK_SET` selects the split (18 target/atomic, 65 pretrain/atomic, or all 300); the names resolve to the task tuples in `robocasa_lerobot_dataset.py`, which stays the single source of truth. Conversion has to run before training rather than being optional setup: RoboCasa publishes LeRobot v2.1 and the pinned lerobot is v3.0 only and rejects that layout. The README documents the ordering and where the converter and evaluator live. Requires the matching cosmos-framework change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
9079a60 to
3f3a82e
Compare
Sorry about that. I mistakenly created the PR from an outdated commit. I've rebased and force-pushed. |
2 participants
Adds a complete RoboCasa recipe to the action finetune cookbook, following the DROID/LIBERO layout: dataset conversion, training launcher, closed-loop evaluation, and the Nano SFT TOML.
RoboCasa publishes LeRobot v2.1 while the pinned lerobot is v3.0-only, so a one-time conversion step is included; it works on copies and is idempotent.
Evaluation follows the official protocol -- a policy server per GPU with the simulator client in a separate venv, since robosuite and cosmos-framework cannot share one environment.
Requires the matching cosmos-framework change that registers the RoboCasa dataset and embodiment.