Skip to content

cookbook: add Cosmos3-Edge TensorRT-Edge-LLM reasoner and policy notebooks - #326

Open
ConstBob wants to merge 1 commit into
NVIDIA:mainfrom
ConstBob:docs/edge-trt-edge-llm
Open

cookbook: add Cosmos3-Edge TensorRT-Edge-LLM reasoner and policy notebooks#326
ConstBob wants to merge 1 commit into
NVIDIA:mainfrom
ConstBob:docs/edge-trt-edge-llm

Conversation

@ConstBob

Copy link
Copy Markdown
Collaborator

Summary

Testing

Tested on an x86 developer GPU (RTX PRO 6000, SM120) with TensorRT-Edge-LLM v0.10.0. Both added notebooks in this PR were run top-to-bottom (run_with_trt_edge_llm.ipynb and run_policy_with_trt_edge_llm.ipynb).

  • Reasoner: export --task reasoningllm_build + visual_buildllm_inference on robot_153.jpg. Pipeline finished.
  • Policy: export --task policycosmos3_policy_buildcosmos3_policy_inference. [1,16,10], finite=true, ~195 ms (runtime smoke).

On this SM120 box the pipeline succeeded for both notebooks (export, engine build, and C++ inference finished; policy returned [1,16,10] with finite=true). Reasoner captions were not usable (text did not match robot_153.jpg); policy was only checked as a finite action tensor, not DROID quality.

Likely cause: TensorRT engines and ViT/FMHA kernels are SM-specific. The device used for this testing is RTX PRO 6000. Official Edge-LLM targets are Thor / Spark / Orin, etc. I do not have an Official Edge-LLM device (Jetson Thor / DRIVE Thor / DGX Spark / Orin), so caption or policy quality on those SKUs are not checked. x86 is Developer-only; on this SM120 box reasoner captions were not usable and should not be treated as gold. Re-run is needed on Thor, or Spark) before treating quality as validated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant