Skip to content

feat(mmdet): compile decoding parameters into the runner - #12

Closed
201815054 wants to merge 8 commits into
mainfrom
mmdet/params-function
Closed

feat(mmdet): compile decoding parameters into the runner#12
201815054 wants to merge 8 commits into
mainfrom
mmdet/params-function

Conversation

@201815054

@201815054 201815054 commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Three things, in the order they build on each other: the decoding parameters stop being a file, the runner produces something a person can look at, and the guide describes what results.

1. Parameters become a generated function

Export wrote <name>.postproc.json — anchor scales, head convolution layout, normalisation values — and the runner read it at startup as a third positional argument:

run_mmdet <gguf> <input> <postproc.json> <out.bin> [size]

Once an architecture is fixed those values never change again. They are constants, and they were being shipped as input, parsed on every run, and carried as a third file that has to stay in step with the second. A .gguf and a .postproc.json from different exports load without complaint and decode wrongly.

mmdet_to_pt.py emits <name>.postproc.h instead:

// Generated by mmdet_to_pt.py — do not edit.
// source: retinanet_r18_fpn_1x_coco.py
inline mmdet_cfg mmdet_params() {
    mmdet_cfg c;
    c.head.stacked_convs = 4;
    c.det.strides = {8.0f, 16.0f, 32.0f, 64.0f, 128.0f};
    /* ... */
    return c;
}

compiled into the runner alongside the head component and the backbone graph.

Before After
Files to deploy 3 2
Read at startup GGUF + JSON GGUF
Mismatched pair possible yes, silently no

build_mmdet_cpp.sh takes the header as an optional second argument and otherwise finds *.postproc.h in the generated directory. A second argument that is not a header is still read as an architecture name, so existing invocations keep working. frcnn_to_pt.py and the two-stage runners keep their own sidecar; only the single-stage path changes here.

2. The result is an image by default

The runner wrote raw float32 and printed a count, so seeing what a detector had found meant writing a script. Every other command-line entry point in this repository produces an image; this one now does too. The output extension decides:

run_mmdet model.gguf image.jpg detected.png 512   # boxes drawn on the image
run_mmdet model.gguf image.jpg boxes.bin     512   # raw float32

Raw output is unchanged and one extension away, because comparing against a reference implementation needs the numbers, not a picture.

tools/detect/draw.h holds the drawing: box outlines, one colour per class, no text. Keeping a font out of the runner costs nothing, because the runner also prints the highest-scoring detections as a table — the image carries where, the table carries what.

Variable Effect
VISP_DRAW_THRESHOLD Minimum score to draw. Default 0.3
VISP_PRINT_DETS How many rows to print. 0 turns the table off

tools/verify/draw_boxes.py draws a .bin kept for comparison, and does add class names and scores as text.

3. Documentation

docs/mmdet-detectors.md documents the whole path: the export frontend, the interface a compiled backbone module must satisfy, the runner, the components in tools/detect, the decoder API in postproc.h, the two-stage and instance-segmentation variants, ByteTrack, and the fields of the generated parameters.

tools/README.md is rewritten in English and describes how the directory is arranged rather than restating the guide.

Verification

Run from an empty directory following the documented steps, on RetinaNet R18 at 512:

Step Result
Export backbone.pt + backbone.postproc.h
Compile at 512 136 tensors
Build runner links against libvisioncpp
Run, image output 100 detections, boxes drawn
Run, .bin output byte-identical to the previous JSON path (cmp)
VISP_PRINT_DETS=0 table suppressed
VISP_DRAW_THRESHOLD=0.84 6 drawn; draw_boxes.py agrees on the same file

Two documentation defects surfaced during that run and are fixed here:

  • The build step was shown without the parameters header, but export writes it next to backbone.pt rather than into the generated directory, so the command as printed could not find it.
  • The compile step did not say which resolution to use. A graph built at a different size aborts in ggml_can_repeat once a tensor of the wrong extent reaches a residual addition — a failure a long way from its cause.

Independent of #11 and #13.

🤖 Generated with Claude Code

eunchae and others added 2 commits August 6, 2026 17:37
Export wrote <name>.postproc.json -- anchor scales, head convolution layout,
normalisation values -- and the runner read it at startup as a third
positional argument:

    run_mmdet <gguf> <input> <postproc.json> <out.bin> [size]

Once an architecture is fixed those values never change again. They are
constants, and they were being shipped as input, parsed on every run, and
carried as a third file that has to stay in step with the second. A .gguf and
a .postproc.json from different exports load without complaint and decode
wrongly.

mmdet_to_pt.py now emits <name>.postproc.h: a generated `mmdet_params()`
returning the same values, compiled into the runner alongside the head
component and the backbone graph.

    run_mmdet <gguf> <input> <out.bin> [size]

Deployment is the executable and the weights. Nothing is read at startup and
no pair can be mismatched.

build_mmdet_cpp.sh takes the header as an optional second argument and
otherwise finds *.postproc.h in the generated directory. A second argument
that is not a header is still read as an architecture name, so existing
invocations keep working. frcnn_to_pt.py and the two-stage runners keep their
own sidecar; only the single-stage path changes here.

Verified on RetinaNet R18 at 512: detections are byte-identical to the same
model run through the previous JSON path.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Documents the path the previous commit completes: backbone and neck run as a
compiled ggml graph, while the head, decoding and post-processing are C++
assembled from library primitives -- detection heads carry control flow that
depends on the data, and tracing records only the path one input happened to
take.

Covers the export frontend, the interface a compiled backbone module must
satisfy, the runner, the components in tools/detect, the decoder API in
postproc.h, the two-stage and instance-segmentation variants, ByteTrack, and
the fields of the generated parameters.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
eunchae and others added 2 commits August 6, 2026 17:50
The runner wrote raw float32 and printed only a count, so the only way to see
what a detector produced was to write a script. Two additions:

  - run_mmdet prints the highest-scoring detections as a table. VISP_PRINT_DETS
    sets how many, 0 turns it off. The file it writes is unchanged -- still raw
    float32, still what a reference comparison needs.
  - tools/verify/draw_boxes.py draws a detections file onto the image it came
    from, scaling boxes back from the square input the detector ran on.

The README is rewritten in English and brought in line with the guide: the
parameters are a generated function rather than a sidecar, the runner takes one
argument fewer, and the sections on heads, two-stage detectors, segmentation
and tracking describe what each part does rather than restating accuracy
figures that belong with the harnesses producing them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The runner wrote raw float32 and nothing else, so seeing what a detector had
found meant writing a script. Every other command-line entry point in this
repository produces an image; the detector runner now does too.

The output extension decides:

    run_mmdet model.gguf image.jpg detected.png 512   # boxes drawn on the image
    run_mmdet model.gguf image.jpg boxes.bin     512   # raw float32

Raw output is unchanged and one extension away, because comparing against a
reference implementation needs the numbers, not a picture.

tools/detect/draw.h holds the drawing: box outlines, one colour per class, no
text. Keeping a font out of the runner costs nothing, because the runner also
prints the highest-scoring detections as a table -- the image carries where,
the table carries what. VISP_DRAW_THRESHOLD and VISP_PRINT_DETS adjust both.

tools/verify/draw_boxes.py stays for drawing a .bin that was kept for
comparison, and does add class names and scores as text.

Verified on RetinaNet R18 at 512: the .bin path is still byte-identical to the
previous output, and the image path draws boxes scaled back to the original
resolution.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… header

Running the documented steps from an empty directory found two gaps.

The build step was shown without the parameters header, but export writes it
next to backbone.pt rather than into the generated directory, so the command
as printed could not find it. The header is now named in the command, and the
script says what to pass when it is missing instead of reporting an empty
path.

The compile step did not say which resolution to use. Tracing records the
operations for one input shape, so a graph built at a different size aborts
in ggml_can_repeat once a tensor of the wrong extent reaches a residual
addition -- a failure a long way from its cause. Both documents now say to
compile at the --size given to export.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@201815054
201815054 force-pushed the mmdet/params-function branch from 5f1744b to 97a631e Compare August 6, 2026 09:05
eunchae and others added 3 commits August 7, 2026 11:01
Extracting every shell command from both documents and comparing them found
one that would fail. The VISP_BUILD example in the guide -- the form to use
when the library was configured somewhere other than build/ -- omitted the
parameters header, so copying that line hit the same "no parameters header"
stop that the plain form had already been fixed for.

The README gained the same note in the same shape, since a reader who built
elsewhere had nothing to go on there at all.

Nine of the ten commands are now identical between the two. The remaining
differences are deliberate: the guide shows an argument-form line for
run_mmdet that the README does not need, and spells the config path as
/path/to/... where the README uses a bare file name.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Removes bold and italic markers from the prose. Structure -- headings, tables,
code, links, lists -- is untouched; only the markers a reader sees as noise
when looking at the source are gone.

Also removes the link to overview.md, which lives in a different pull request.
This one has to make sense on its own.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
repr(float('inf')) is "inf", so the float-literal helper produced "inff",
which C++ does not accept -- the generated header would fail to compile.

Configs reach this. FCOS bounds its last regression range with INF
(regress_ranges=((-1, 64), ..., (512, INF))), so any head whose parameters
include such a range hits it as soon as those values are emitted.

Non-finite values now become INFINITY / -INFINITY / NAN, and the generated
header includes <cmath> for them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@201815054

201815054 commented Aug 7, 2026

Copy link
Copy Markdown
Collaborator Author

Closing while a larger issue is investigated: with a trained checkpoint the pipeline does not reproduce the reference detections. Reproduced on main, so it predates these changes, but there is no point reviewing documentation for a flow whose end-to-end correctness is in question.

@201815054 201815054 closed this Aug 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant