feat(mmdet): compile decoding parameters into the runner - #12
Closed
201815054 wants to merge 8 commits into
Closed
Conversation
Export wrote <name>.postproc.json -- anchor scales, head convolution layout,
normalisation values -- and the runner read it at startup as a third
positional argument:
run_mmdet <gguf> <input> <postproc.json> <out.bin> [size]
Once an architecture is fixed those values never change again. They are
constants, and they were being shipped as input, parsed on every run, and
carried as a third file that has to stay in step with the second. A .gguf and
a .postproc.json from different exports load without complaint and decode
wrongly.
mmdet_to_pt.py now emits <name>.postproc.h: a generated `mmdet_params()`
returning the same values, compiled into the runner alongside the head
component and the backbone graph.
run_mmdet <gguf> <input> <out.bin> [size]
Deployment is the executable and the weights. Nothing is read at startup and
no pair can be mismatched.
build_mmdet_cpp.sh takes the header as an optional second argument and
otherwise finds *.postproc.h in the generated directory. A second argument
that is not a header is still read as an architecture name, so existing
invocations keep working. frcnn_to_pt.py and the two-stage runners keep their
own sidecar; only the single-stage path changes here.
Verified on RetinaNet R18 at 512: detections are byte-identical to the same
model run through the previous JSON path.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Documents the path the previous commit completes: backbone and neck run as a compiled ggml graph, while the head, decoding and post-processing are C++ assembled from library primitives -- detection heads carry control flow that depends on the data, and tracing records only the path one input happened to take. Covers the export frontend, the interface a compiled backbone module must satisfy, the runner, the components in tools/detect, the decoder API in postproc.h, the two-stage and instance-segmentation variants, ByteTrack, and the fields of the generated parameters. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The runner wrote raw float32 and printed only a count, so the only way to see
what a detector produced was to write a script. Two additions:
- run_mmdet prints the highest-scoring detections as a table. VISP_PRINT_DETS
sets how many, 0 turns it off. The file it writes is unchanged -- still raw
float32, still what a reference comparison needs.
- tools/verify/draw_boxes.py draws a detections file onto the image it came
from, scaling boxes back from the square input the detector ran on.
The README is rewritten in English and brought in line with the guide: the
parameters are a generated function rather than a sidecar, the runner takes one
argument fewer, and the sections on heads, two-stage detectors, segmentation
and tracking describe what each part does rather than restating accuracy
figures that belong with the harnesses producing them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The runner wrote raw float32 and nothing else, so seeing what a detector had
found meant writing a script. Every other command-line entry point in this
repository produces an image; the detector runner now does too.
The output extension decides:
run_mmdet model.gguf image.jpg detected.png 512 # boxes drawn on the image
run_mmdet model.gguf image.jpg boxes.bin 512 # raw float32
Raw output is unchanged and one extension away, because comparing against a
reference implementation needs the numbers, not a picture.
tools/detect/draw.h holds the drawing: box outlines, one colour per class, no
text. Keeping a font out of the runner costs nothing, because the runner also
prints the highest-scoring detections as a table -- the image carries where,
the table carries what. VISP_DRAW_THRESHOLD and VISP_PRINT_DETS adjust both.
tools/verify/draw_boxes.py stays for drawing a .bin that was kept for
comparison, and does add class names and scores as text.
Verified on RetinaNet R18 at 512: the .bin path is still byte-identical to the
previous output, and the image path draws boxes scaled back to the original
resolution.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… header Running the documented steps from an empty directory found two gaps. The build step was shown without the parameters header, but export writes it next to backbone.pt rather than into the generated directory, so the command as printed could not find it. The header is now named in the command, and the script says what to pass when it is missing instead of reporting an empty path. The compile step did not say which resolution to use. Tracing records the operations for one input shape, so a graph built at a different size aborts in ggml_can_repeat once a tensor of the wrong extent reaches a residual addition -- a failure a long way from its cause. Both documents now say to compile at the --size given to export. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
201815054
force-pushed
the
mmdet/params-function
branch
from
August 6, 2026 09:05
5f1744b to
97a631e
Compare
Extracting every shell command from both documents and comparing them found one that would fail. The VISP_BUILD example in the guide -- the form to use when the library was configured somewhere other than build/ -- omitted the parameters header, so copying that line hit the same "no parameters header" stop that the plain form had already been fixed for. The README gained the same note in the same shape, since a reader who built elsewhere had nothing to go on there at all. Nine of the ten commands are now identical between the two. The remaining differences are deliberate: the guide shows an argument-form line for run_mmdet that the README does not need, and spells the config path as /path/to/... where the README uses a bare file name. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Removes bold and italic markers from the prose. Structure -- headings, tables, code, links, lists -- is untouched; only the markers a reader sees as noise when looking at the source are gone. Also removes the link to overview.md, which lives in a different pull request. This one has to make sense on its own. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
repr(float('inf')) is "inf", so the float-literal helper produced "inff",
which C++ does not accept -- the generated header would fail to compile.
Configs reach this. FCOS bounds its last regression range with INF
(regress_ranges=((-1, 64), ..., (512, INF))), so any head whose parameters
include such a range hits it as soon as those values are emitted.
Non-finite values now become INFINITY / -INFINITY / NAN, and the generated
header includes <cmath> for them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Collaborator
Author
|
Closing while a larger issue is investigated: with a trained checkpoint the pipeline does not reproduce the reference detections. Reproduced on main, so it predates these changes, but there is no point reviewing documentation for a flow whose end-to-end correctness is in question. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three things, in the order they build on each other: the decoding parameters stop being a file, the runner produces something a person can look at, and the guide describes what results.
1. Parameters become a generated function
Export wrote
<name>.postproc.json— anchor scales, head convolution layout, normalisation values — and the runner read it at startup as a third positional argument:Once an architecture is fixed those values never change again. They are constants, and they were being shipped as input, parsed on every run, and carried as a third file that has to stay in step with the second. A
.ggufand a.postproc.jsonfrom different exports load without complaint and decode wrongly.mmdet_to_pt.pyemits<name>.postproc.hinstead:compiled into the runner alongside the head component and the backbone graph.
build_mmdet_cpp.shtakes the header as an optional second argument and otherwise finds*.postproc.hin the generated directory. A second argument that is not a header is still read as an architecture name, so existing invocations keep working.frcnn_to_pt.pyand the two-stage runners keep their own sidecar; only the single-stage path changes here.2. The result is an image by default
The runner wrote raw
float32and printed a count, so seeing what a detector had found meant writing a script. Every other command-line entry point in this repository produces an image; this one now does too. The output extension decides:Raw output is unchanged and one extension away, because comparing against a reference implementation needs the numbers, not a picture.
tools/detect/draw.hholds the drawing: box outlines, one colour per class, no text. Keeping a font out of the runner costs nothing, because the runner also prints the highest-scoring detections as a table — the image carries where, the table carries what.VISP_DRAW_THRESHOLD0.3VISP_PRINT_DETS0turns the table offtools/verify/draw_boxes.pydraws a.binkept for comparison, and does add class names and scores as text.3. Documentation
docs/mmdet-detectors.mddocuments the whole path: the export frontend, the interface a compiled backbone module must satisfy, the runner, the components intools/detect, the decoder API inpostproc.h, the two-stage and instance-segmentation variants, ByteTrack, and the fields of the generated parameters.tools/README.mdis rewritten in English and describes how the directory is arranged rather than restating the guide.Verification
Run from an empty directory following the documented steps, on RetinaNet R18 at 512:
backbone.pt+backbone.postproc.hlibvisioncpp.binoutputcmp)VISP_PRINT_DETS=0VISP_DRAW_THRESHOLD=0.84draw_boxes.pyagrees on the same fileTwo documentation defects surfaced during that run and are fixed here:
backbone.ptrather than into the generated directory, so the command as printed could not find it.ggml_can_repeatonce a tensor of the wrong extent reaches a residual addition — a failure a long way from its cause.Independent of #11 and #13.
🤖 Generated with Claude Code