a4ea532f3627bc6a4fd5dca19ceeb4cdb07c48d9
Verified on dual RTX 5080: 355/355 detections, both GPUs in round-robin, zero CUDA errors. Requires TensorRT >= 10.16.1 in SharedCore (10.15.1 has a Myelin l2cm crash when creating a context on a second Blackwell device). 1. blobFromGpuMats: thread_local GpuMat caches (tl_blob/tl_floatImg) were device-blind — a pool thread warming GPU[1] then GPU[0] reused device-1 memory with a device-0 stream, causing an async illegal memory access that poisoned every CUDA context (sticky 700). Track owning device and rebuild the cache on device switch. 2. runInference: pool round-robin can dispatch to a slot on a different GPU than the one the caller prepared inputs on. Detect foreign input GpuMats via cudaPointerGetAttributes and migrate them through a pinned host staging buffer (two-hop D2H+H2D; direct cross-device cudaMemcpy2D is unsupported and GeForce has no P2P). 3. warmUp: now pins its own CUDA device before uploading dummy inputs and returns success/failure; loadNetwork rejects engines whose warmup fails so broken slots can never join the pool silently. 4. Blackwell builder workaround: setMaxAuxStreams(0) on SM 12.x plus an .aux0 engine-cache suffix so pre-workaround caches rebuild. Also: ANSFR-UnitTest gains --gpu N flag (sets CUDA_VISIBLE_DEVICES) for single-GPU bisection, and rich diagnostics on input-copy failures. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Description
No description provided
13 MiB
Languages
C
55.3%
C++
43.4%
Python
0.5%
CMake
0.3%
Objective-C++
0.2%
Other
0.2%