Repository navigation
SiteObject.in_box is not rotation-invariant and can cause false-negative results #145
Description
Activity
Corroborating this from a different angle. I run 350 episodes across all ten libero_object tasks with a paired design: every attack episode against a benign twin at the same task and seed, and the benign arm fires 2 of 50 when it should fire 0.
I had assumed that it was entirely my own predicate that was uncalibrated, and I filed it against myself on that basis. Reading this, some of it may be in_box rather than in my code, because the two cases where my benign arm tripped are the two tasks where the object is placed at an angle rather than axis-aligned.
I have not separated the two causes yet. If helpful, I can run the benign arm again with the object pose logged per step and post the distribution of in_box outcomes against yaw, which would show whether the false negatives cluster at particular rotations or are spread. That is a CPU-only run for me, so it costs nothing but wall clock.
The reason this matters beyond a flaky eval: anyone using LIBERO to measure a safety property rather than task success inherits the false negative directly into their rate. A false-negative success check reads as a failure, and if you are counting failures as breaches, you get a number that is too high without any way to see it.
Problem
SiteObject.in_box()does not correctly account for the orientation of a site. When a container such as a basket is rotated during a rollout, an object whose center is inside the visualcontain_regioncan still be classified as outside, causing a false-negative task success result.The relevant code is:
https://github.com/Lifelong-Robot-Learning/LIBERO/blob/master/libero/libero/envs/objects/site_object.py#L35-L56
The source already notes that this size transformation
np.abs(site_rotation @ site.size)is "hacky".As an extreme case, for a box with equal x/y half-sizes (e.g. the basket in LIBERO_OBJECT), when it is rotated by 45 degrees, one of the computed world-space half-extents becomes 0.
Expected behavior
A box site represents an oriented box. Its containment result should remain unchanged if the site and the point are transformed by the same rigid-body pose.
For a site position
c, orientationR, half-sizes, and world pointp, the point should first be transformed into site-local coordinates:Containment can then be tested against the local half-sizes:
This preserves the current center-point approximation and the existing 1 cm lower-bound tolerance, while making the contain region follow arbitrary basket translation and rotation, including roll and pitch.