Commit Graph
13 Commits
Author SHA1 Message Date
nghianguyenandClaude Opus 5 cd99fcf6d9 feat(ANSODEngine): run a sealed model on TensorRT without unsealing it
A .ansmodel reaching the GPU path went to ModelOptimizer, which knows one
way to open a model — unzip it — and failed on the first byte. Even past
that, the TensorRT engine only ever parsed an ONNX file, so the one thing
the format exists to prevent, a decrypted model on disk, was the only way
it could have run.

The builder now takes its bytes from memory when it has them. The path it
is given still names the plan that gets cached beside it, because a plan
is a built artefact and carries no model: the second run loads it and
never needs the ONNX at all. For a package the plan lands in the folder
the zip flow used, so a model converted from a zip reuses the plan that
zip already built.

OptimizeModelStr recognises a package and skips the unzip-and-optimise
leg entirely: the handle below it opens the package in memory and builds
the plan from the decrypted bytes, which is the same work by a shorter
road.

Still on the file path: the other TensorRT handlers (segmentation, pose,
classification). Only the YOLO detector reads a package today.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 09:38:45 +10:00
nghianguyen a43a99834a Complete RTMPOSE 2026-05-14 21:16:35 +10:00
nghianguyen ee8f0d1163 Suppot RTMPOSE classes 2026-05-14 17:48:09 +10:00
nghianguyen 7e772f76bc Improve ANSCV 2026-04-21 09:26:02 +10:00
nghianguyen 3418090042 - Remove ALPR_OD Tracker and voting system.
- Disable debugview
2026-04-19 23:15:50 +10:00
nghianguyen 51fe507dfd Fix unregister issue 2026-04-19 14:47:29 +10:00
nghianguyen 0a8aaed215 Add CPU/GPU gate and support new ANSALPR using OCR 2026-04-12 17:16:16 +10:00
nghianguyen eb863bf510 Support getEngine 2026-04-09 11:00:34 +10:00
nghianguyen eeb205779a Fix model optimisation 2026-04-09 08:09:02 +10:00
nghianguyen 69787b0ff0 Fix AMD and OpenVINO 2026-04-08 13:45:52 +10:00
nghianguyen e134ebdf15 Use software decoder by default 2026-04-04 20:19:54 +11:00
nghianguyen c1b919ec47 Fix setting GPU behaviour:
Condition	maxSlotsPerGpu	Behavior
OptimizeModelStr	0	Bypass: non-shared temporary engine
1 GPU	1	Single slot, no round-robin
>1 GPU, VRAM < 24 GB	1	Round-robin: 1 slot per GPU
>1 GPU, VRAM >= 24 GB	-1	Elastic: on-demand slot growth
2026-03-30 09:59:09 +11:00
nghianguyen 8a2e721058 Refactor project structure 2026-03-28 19:56:39 +11:00