Commit Graph
8 Commits
Author SHA1 Message Date
nghianguyenandClaude Opus 4.8 d2479b9b07 GpuMemoryGuard: scale VRAM safety margin to card size
The margin was a fixed 768 MiB — 3% of a 24 GiB compute card but 19% of
a 4 GiB card (RTX A400), where free VRAM never clears it, so CanLoad()
refused every engine and UnderPressure() skipped every frame. Now
total/8 clamped to [192, 768] MiB: cards >= 6 GiB keep exactly the old
margin, only small cards change. ANSCENTER_GPU_MARGIN_MB / SetMargin()
still override (0 = auto). Refusal log now reports total VRAM alongside
free, which is what made this diagnosable.

Verified on the A400 test PC: detector + recognizer load and match.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 13:10:10 +10:00
nghianguyenandClaude Fable 5 a4ea532f36 Fix multi-GPU inference on Blackwell (dual RTX 5080): four stacked fixes
Verified on dual RTX 5080: 355/355 detections, both GPUs in round-robin,
zero CUDA errors. Requires TensorRT >= 10.16.1 in SharedCore (10.15.1 has
a Myelin l2cm crash when creating a context on a second Blackwell device).

1. blobFromGpuMats: thread_local GpuMat caches (tl_blob/tl_floatImg) were
   device-blind — a pool thread warming GPU[1] then GPU[0] reused device-1
   memory with a device-0 stream, causing an async illegal memory access
   that poisoned every CUDA context (sticky 700). Track owning device and
   rebuild the cache on device switch.

2. runInference: pool round-robin can dispatch to a slot on a different
   GPU than the one the caller prepared inputs on. Detect foreign input
   GpuMats via cudaPointerGetAttributes and migrate them through a pinned
   host staging buffer (two-hop D2H+H2D; direct cross-device cudaMemcpy2D
   is unsupported and GeForce has no P2P).

3. warmUp: now pins its own CUDA device before uploading dummy inputs and
   returns success/failure; loadNetwork rejects engines whose warmup fails
   so broken slots can never join the pool silently.

4. Blackwell builder workaround: setMaxAuxStreams(0) on SM 12.x plus an
   .aux0 engine-cache suffix so pre-workaround caches rebuild.

Also: ANSFR-UnitTest gains --gpu N flag (sets CUDA_VISIBLE_DEVICES) for
single-GPU bisection, and rich diagnostics on input-copy failures.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 20:00:19 +10:00
nghianguyen b3436a61ee Prevent OOM 2026-06-18 05:55:31 +10:00
nghianguyen ef2a122fec Add logs to custom models and model optimisation 2026-04-25 13:17:36 +10:00
nghianguyen 57cc8e0a56 Fix double stop in ANSVideoPlayer 2026-04-22 10:10:16 +10:00
nghianguyenandClaude Opus 4.6 8d2db78cee Remove [Engine] and [EnginePoolManager] debug log messages
Cleaned up verbose engine telemetry emitted to stdout/stderr and the
Windows Event Viewer. Removes logEngineEvent/logEvent calls (and their
diagnostic-only locals) across the TensorRT engine load, build, run,
multi-GPU, and pool-manager paths, plus the now-unused logEvent helper
in EnginePoolManager.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 18:09:08 +10:00
nghianguyen e134ebdf15 Use software decoder by default 2026-04-04 20:19:54 +11:00
nghianguyen 7b4134133c Initial setup for CLion 2026-03-28 16:54:11 +11:00