38 Commits
Author SHA1 Message Date
nghianguyen 214f598e18 Support Generate Presigned URL for S3 2026-05-19 10:37:46 +10:00
nghianguyen 26f9b07ac1 Add adaptive padding to improve tracking 2026-05-14 22:01:07 +10:00
nghianguyen a43a99834a Complete RTMPOSE 2026-05-14 21:16:35 +10:00
nghianguyen 71404affb7 Correct logic in RTMPOSE when using generic model to detect person 2026-05-14 19:33:31 +10:00
nghianguyen 52ec13a8e3 improve pose 2026-05-14 19:17:09 +10:00
nghianguyen 8e4d9a9e5d Add person model into RTMPOSE 2026-05-14 19:05:20 +10:00
nghianguyen ee8f0d1163 Suppot RTMPOSE classes 2026-05-14 17:48:09 +10:00
nghianguyen eb1244c936 Support get global IP address 2026-05-09 21:55:01 +10:00
nghianguyen cffdeb223b Update get info 2026-05-09 20:57:13 +10:00
nghianguyen a94bf88a5b Fix other ANSCV classes 2026-05-09 16:47:46 +10:00
nghianguyen aa323a07a3 Fix debug 2026-05-07 23:35:18 +10:00
nghianguyen 75693cadd9 Add jitter buffer 2026-05-07 22:36:27 +10:00
nghianguyen 2e0fee5e3d Fix VideoPlayer from stop after just few frames 2026-05-07 18:17:33 +10:00
nghianguyen a526aa6cdb Fix ALPR order issue 2026-05-06 19:55:52 +10:00
nghianguyen 66c67dff26 Fix onnx runtime loader for custom model 2026-05-05 16:23:01 +10:00
nghianguyen b0002b4a4b Update ALPR developer guide 2026-05-05 04:41:33 +10:00
nghianguyen d03c2c1138 disable ALPR checker by default 2026-05-05 04:37:23 +10:00
nghianguyen 91d81cbd88 fix onnx yolo race condition 2026-05-04 23:45:05 +10:00
nghianguyen 03a63142ec Disable stabilized feature by default. Need ,more time to test 2026-05-04 09:39:07 +10:00
nghianguyen fa6d95de82 Update test app 2026-05-03 16:30:39 +10:00
nghianguyen 4214507e3b Test ALPR model (v1.3) 2026-05-03 07:16:32 +10:00
nghianguyen cff2a90fad Support multiple GPUs 2026-05-02 12:24:53 +10:00
nghianguyen 8359511895 Update unit test 2026-05-01 20:33:09 +10:00
nghianguyen fc9cf9671b Fix ANSONNXYOLO:
1. core/ANSLicensingSystem/ANSLicense.h:268 — added default initializers to ModelConfig POD members (inpWidth = 0, inpHeight = 0, etc.). This was the actual bug: CreateANSODHandle declares a local ModelConfig and only sets a few fields, leaving inpWidth/inpHeight as indeterminate stack memory. On your machine that memory consistently held ~80, slipping past the <=0 → 640 guard and driving the ORT session at 80×80 — which crashes YOLOv11's /model.12/Concat because 2·ceil(80/32)=6 ≠ ceil(80/16)=5.

2. modules/ANSODEngine/ANSONNXYOLO.h:75 — ONNXYOLO::setInputShape() now snaps width/height up to the next multiple of 32 (with a 64-pixel floor). Defense in depth: even if a future caller hands in a non-stride-32 size, the FPN concat alignment is preserved.

3. modules/ANSODEngine/ANSONNXYOLO.cpp — two diagnostic additions:

WarmUpEngine bails with a clear log line if the warm-up dims aren't a multiple of 32, instead of letting the model.12 error fire and mask the real cause.
transform() logs the input shape, canvas shape, source size, and dynamic flag once per session in DebugView — so any future shape-mismatch crash points at the offending HxW directly.
2026-05-01 20:24:08 +10:00
nghianguyen 1c8abf2a88 Fix AWS MinIO 2026-05-01 11:16:47 +10:00
nghianguyen c4ec6f0ed2 Add AMD 16FP conversion plan 2026-05-01 10:09:24 +10:00
nghianguyen 91efbb9a8f Fix AMD ALPR_OCR and custom model 2026-05-01 07:53:18 +10:00
nghianguyen e1f37ec767 Add ANSALPR country format 2026-04-28 19:23:46 +10:00
nghianguyen cd5d6f1923 Remove spaces, ".", and "-" from ALPR 2026-04-28 17:49:37 +10:00
nghianguyen 234f2c68a2 bucket prewarm + batch padding 2026-04-28 17:12:27 +10:00
nghianguyen 4bab357c72 Stress test on multiple streams 2026-04-28 13:49:57 +10:00
nghianguyen dcf974c35c Fix AMD by changing from GetTensorData<T>() to GetTensorMutableData<T>() 2026-04-28 13:25:02 +10:00
nghianguyen f4b74c837e protect sensitive information from logs 2026-04-26 07:53:37 +10:00
nghianguyen da94e7f6da strip sensitive information from logs 2026-04-26 07:35:30 +10:00
nghianguyen f81737ffc6 Add debug and log functions to ANSLIB 2026-04-26 06:59:42 +10:00
nghianguyen 947398483a Unit test before memory leak check 2026-04-26 06:14:44 +10:00
nghianguyen bab94bf9bc Add log and change logger policy 2026-04-25 20:29:58 +10:00
nghianguyen ef2a122fec Add logs to custom models and model optimisation 2026-04-25 13:17:36 +10:00
93 changed files with 13289 additions and 1549 deletions
+1
View File
@@ -0,0 +1 @@
{"sessionId":"b4ba6a5c-5267-4098-8e7c-152012d3df22","pid":5176,"acquiredAt":1776991716766}
+127 -1
View File
@@ -106,7 +106,133 @@
"Bash(dumpbin //dependents \"C:\\\\\\\\ProgramData\\\\\\\\ANSCENTER\\\\\\\\Shared\\\\\\\\ANSLicensingSystem.dll\")",
"Bash(grep -v \"//\")",
"Bash(grep -v \"^ *//\")",
"Bash(python strip_folder_lock.py)"
"Bash(python strip_folder_lock.py)",
"Bash(tasklist //FI \"IMAGENAME eq ANSLPR-UnitTest.exe\")",
"Bash(taskkill //PID 41856 //F)",
"PowerShell(Stop-Process -Id 41856 -Force)",
"PowerShell(Start-Process -FilePath \"taskkill.exe\" -ArgumentList \"/PID\",\"41856\",\"/F\" -Verb RunAs -Wait)",
"Bash(npm root *)",
"Bash(npm list *)",
"Bash(npm install *)",
"Bash(NODE_PATH=\"C:/home/alex/.npm-global/node_modules\" node build_anslib_logging_guide.js)",
"Bash(python \"C:/Users/nghia/AppData/Roaming/Claude/local-agent-mode-sessions/skills-plugin/d8e35aa4-a14e-4e20-b921-ba1b9a3cce86/cdda7cc8-a1c7-42ff-98b4-473ec3e8b9fb/skills/docx/scripts/office/validate.py\" \"C:/Projects/CLionProjects/ANSCORE/docs/ANSLIB_Logging_Guide.docx\")",
"Bash(python \"C:/Users/nghia/AppData/Roaming/Claude/local-agent-mode-sessions/skills-plugin/d8e35aa4-a14e-4e20-b921-ba1b9a3cce86/cdda7cc8-a1c7-42ff-98b4-473ec3e8b9fb/skills/docx/scripts/office/soffice.py\" --headless --convert-to pdf --outdir \"C:/Projects/CLionProjects/ANSCORE/docs\" \"C:/Projects/CLionProjects/ANSCORE/docs/ANSLIB_Logging_Guide.docx\")",
"Bash(sort -t: -k1 -u)",
"Bash(git -C C:/Projects/CLionProjects/ANSCORE diff core/ANSLicensingSystem/ANSLicense.cpp)",
"Bash(awk 'NR>=154022 && NR<=156428 {print}' \"C:/Users/nghia/Downloads/AVNET-8845HS2.log\")",
"Bash(awk 'NR>=156350 && NR<=156428 {print}' \"C:/Users/nghia/Downloads/AVNET-8845HS2.log\")",
"Bash(git -C C:/Projects/CLionProjects/ANSCORE log --oneline -10 -- modules/ANSFR/OpenVINODeviceConfig.h modules/ANSFR/OpenVINODeviceConfig.cpp)",
"Bash(git -C C:/Projects/CLionProjects/ANSCORE log --oneline -5 -- core/ANSLibsLoader/EPLoader.cpp)",
"Bash(git -C C:/Projects/CLionProjects/ANSCORE show --stat 69787b0)",
"Bash(git -C C:/Projects/CLionProjects/ANSCORE show --stat 97d8149)",
"Bash(git -C C:/Projects/CLionProjects/ANSCORE show 97d8149 -- engines/ONNXEngine/OpenVINODeviceConfig.h modules/ANSFR/ANSFR.cpp)",
"Bash(awk 'NR>=7106 && NR<=7250 && /ONNXYOLO|ANSCUSTOM|DetectObjects/ {print}' \"C:/Users/nghia/Downloads/AVNET-8845HS5.log\")",
"Bash(awk 'NR>=5850 && NR<=7115 && /ONNXYOLO|ANSCUSTOM_Infer|ANSONNXCL_pp|cls\\\\] calling/' \"C:/Users/nghia/Downloads/AVNET-8845HS5.log\")",
"Bash(git -C C:/Projects/CLionProjects/ANSCORE show --stat 3a527d2)",
"Bash(git -C C:/Projects/CLionProjects/ANSCORE show 3a527d2 -- modules/ANSODEngine/ANSCUSTOMDETECTOR.cpp modules/ANSODEngine/ANSONNXYOLO.cpp modules/ANSODEngine/ANSONNXCL.cpp)",
"Bash(git -C C:/Projects/CLionProjects/ANSCORE show 3a527d2 -- modules/ANSODEngine/ANSEngineCommon.h)",
"Bash(git -C C:/Projects/CLionProjects/ANSCORE diff HEAD modules/ANSODEngine/ANSONNXYOLO.cpp modules/ANSODEngine/ANSONNXCL.cpp)",
"Bash(git -C C:/Projects/CLionProjects/ANSCORE diff HEAD modules/ANSODEngine/ANSONNXYOLO.cpp)",
"Bash(git -C C:/Projects/CLionProjects/ANSCORE diff HEAD modules/ANSODEngine/ANSONNXCL.cpp)",
"Bash(git -C C:/Projects/CLionProjects/ANSCORE log --all --oneline --since='3 weeks ago' -- modules/ANSODEngine/ANSONNXYOLO.cpp modules/ANSODEngine/ANSONNXCL.cpp engines/ONNXEngine/ONNXEngine.cpp engines/ONNXEngine/ONNXEngine.h)",
"Bash(git -C C:/Projects/CLionProjects/ANSCORE show --stat 844d739)",
"Bash(git -C C:/Projects/CLionProjects/ANSCORE log -p --grep=\"mutex\\\\|lock\\\\|hang\\\\|stall\\\\|concurrent\\\\|thread\" --since=\"3 weeks ago\" -- modules/ANSODEngine/ANSONNXYOLO.cpp)",
"Bash(git -C C:/Projects/CLionProjects/ANSCORE show 844d739 -- modules/ANSODEngine/ANSONNXYOLO.cpp)",
"Bash(awk '$2>=835 && $2<=860 && /1853990010|ANSCUSTOM|ANSONNXCL|ANSONNXYOLO/' \"C:/Users/nghia/Downloads/AVNET-8845HS7.log\")",
"Bash(awk '$2>=836 && $2<=837 && /ANSCUSTOM|ANSONNXCL/' \"C:/Users/nghia/Downloads/AVNET-8845HS7.log\")",
"Bash(awk '$2>=836.0 && $2<=836.9' \"C:/Users/nghia/Downloads/AVNET-8845HS7.log\")",
"Bash(grep -nB1 -A3 \"DirectML\\\\|DisableMemPattern\\\\|SetExecutionMode\\\\|ExtendedOptimization\\\\|ORT_SEQUENTIAL\" \"C:/Projects/CLionProjects/ANSCORE/engines/ONNXEngine/ONNXEngine.cpp\")",
"Bash(grep -nE \"^\\\\}$|warmupModel\\\\\\(|#if 0|#endif|ANSONNXCL_legacy_Init\" C:/Projects/CLionProjects/ANSCORE/modules/ANSODEngine/ANSONNXCL.cpp)",
"Bash(git -C C:/Projects/CLionProjects/ANSCORE restore modules/ANSODEngine/ANSONNXCL.h modules/ANSODEngine/ANSONNXCL.cpp)",
"Bash(git -C C:/Projects/CLionProjects/ANSCORE status --short modules/)",
"Bash(git -C C:/Projects/CLionProjects/ANSCORE diff --stat HEAD modules/ANSODEngine/ANSONNXCL.cpp modules/ANSODEngine/ANSONNXOBB.cpp modules/ANSODEngine/ANSONNXPOSE.cpp modules/ANSODEngine/ANSONNXSEG.cpp modules/ANSODEngine/ANSYOLO12OD.cpp engines/ONNXEngine/ONNXEngine.cpp engines/ONNXEngine/ONNXSAM3.cpp)",
"Bash(awk -F'[:\\(\\)]' '{gsub\\(/^ +| +$/,\"\",$2\\); split\\($2,a,\" \"\\); ms=a[1]; if \\(ms+0 >= 500\\) print $0}')",
"Bash(awk -F'[:\\(\\)]' '{split\\($2,a,\" \"\\); ms=a[1]; if \\(ms+0 >= 100\\) print $0}')",
"Bash(grep -cE \"^\\\\s*Plate: [^ ]+$\" \"C:/Users/nghia/Downloads/Log4.txt\")",
"Bash(grep -oE \"Plate: [^ ]+$\" \"C:/Users/nghia/Downloads/Log5.txt\")",
"Bash(awk -F'\\\\t' '{print $3}')",
"Bash(awk -F'\\\\t' '{print $3}' \"C:/Users/nghia/Downloads/AVNET-8845HS_twoconsoleapps.log\")",
"Bash(awk -F'\\\\t' '{print $3}' \"C:/Users/nghia/Downloads/AVNET-8845HS_issue.log\")",
"Bash(awk -F'\\\\t' '$2+0 >= 49 && $2+0 <= 52 {print NR\": \"$0}' \"C:/Users/nghia/Downloads/AVNET-8845HS_issue.log\")",
"Bash(awk -F'\\\\t' 'BEGIN{prev=0} {t=$2+0; if\\(t-prev>5\\){print NR\": GAP=\"t-prev\"s \"$0} prev=t}' \"C:/Users/nghia/Downloads/AVNET-8845HS3.log\")",
"Bash(awk -F'\\\\t' '{print $3}' \"C:/Users/nghia/Downloads/AVNET-8845HS3.log\")",
"Bash(awk -F'\\\\t' '{print $3}' \"C:/Users/nghia/Downloads/AVNET-8845HS4_5.log\")",
"Bash(awk -F'\\\\t' '$2+0>=20 && $2+0<=22' \"C:/Users/nghia/Downloads/AVNET-8845HS4_5.log\")",
"Bash(awk -F'\\\\t' '$2+0>=20.5 && $2+0<=21.5' \"C:/Users/nghia/Downloads/AVNET-8845HS4_5.log\")",
"Bash(awk -F'\\\\t' '$2+0>=21.0 && $2+0<=21.7' \"C:/Users/nghia/Downloads/AVNET-8845HS4_5.log\")",
"Bash(awk -F'\\\\t' '$2+0>=21.05 && $2+0<=21.23' \"C:/Users/nghia/Downloads/AVNET-8845HS4_5.log\")",
"Bash(awk -F'\\\\t' '$2+0>=17 && $2+0<=20' \"C:/Users/nghia/Downloads/AVNET-8845HS6.log\")",
"Bash(awk -F'\\\\t' '$2+0>=17.4 && $2+0<=17.7' \"C:/Users/nghia/Downloads/AVNET-8845HS6.log\")",
"Bash(awk -F'\\\\t' '$2+0>=12 && $2+0<=18' \"C:/Users/nghia/Downloads/AVNET-8845HS6.log\")",
"Bash(git -C \"C:/Projects/CLionProjects/ANSCORE\" status)",
"Bash(git -C \"C:/Projects/CLionProjects/ANSCORE\" diff --stat modules/ANSODEngine/ANSONNXCL.h modules/ANSODEngine/ANSONNXCL.cpp modules/ANSODEngine/ANSONNXYOLO.cpp)",
"Bash(awk -F'\\\\t' '{print $4}' \"/c/Users/nghia/Downloads/AVNET-8845HS7.log\")",
"Bash(\"C:/Users/nghia/AppData/Local/Programs/CLion 2026.1/bin/cmake/win/x64/bin/cmake.exe\" --build C:/Projects/CLionProjects/ANSCORE/cmake-build-release --target ANSLPR-UnitTest -j 30)",
"Bash(cmd.exe /c '\"C:\\\\Program Files\\\\Microsoft Visual Studio\\\\2022\\\\Community\\\\VC\\\\Auxiliary\\\\Build\\\\vcvars64.bat\" >NUL && \"C:\\\\Users\\\\nghia\\\\AppData\\\\Local\\\\Programs\\\\CLion 2026.1\\\\bin\\\\cmake\\\\win\\\\x64\\\\bin\\\\cmake.exe\" --build C:/Projects/CLionProjects/ANSCORE/cmake-build-release --target ANSLPR-UnitTest -j 30')",
"Bash(cmd.exe //c 'call \"C:\\\\Program Files\\\\Microsoft Visual Studio\\\\2022\\\\Community\\\\VC\\\\Auxiliary\\\\Build\\\\vcvars64.bat\" >NUL && \"C:\\\\Users\\\\nghia\\\\AppData\\\\Local\\\\Programs\\\\CLion 2026.1\\\\bin\\\\cmake\\\\win\\\\x64\\\\bin\\\\cmake.exe\" --build \"C:\\\\Projects\\\\CLionProjects\\\\ANSCORE\\\\cmake-build-release\" --target ANSLPR-UnitTest -j 30')",
"PowerShell($env:VS_VARS = '\"C:\\\\Program Files\\\\Microsoft Visual Studio\\\\2022\\\\Community\\\\VC\\\\Auxiliary\\\\Build\\\\vcvars64.bat\"'; $cmd = \"call $env:VS_VARS && `\"C:\\\\Users\\\\nghia\\\\AppData\\\\Local\\\\Programs\\\\CLion 2026.1\\\\bin\\\\cmake\\\\win\\\\x64\\\\bin\\\\cmake.exe`\" --build `\"C:\\\\Projects\\\\CLionProjects\\\\ANSCORE\\\\cmake-build-release`\" --target ANSLPR-UnitTest -j 30\"; cmd.exe /c $cmd 2>&1 | Select-Object -Last 60)",
"PowerShell(cmd.exe /c $cmd 2>&1)",
"PowerShell($cmd = 'call \"C:\\\\Program Files\\\\Microsoft Visual Studio\\\\2022\\\\Community\\\\VC\\\\Auxiliary\\\\Build\\\\vcvars64.bat\" >NUL && \"C:\\\\Users\\\\nghia\\\\AppData\\\\Local\\\\Programs\\\\CLion 2026.1\\\\bin\\\\cmake\\\\win\\\\x64\\\\bin\\\\cmake.exe\" --build \"C:\\\\Projects\\\\CLionProjects\\\\ANSCORE\\\\cmake-build-release\" --target ANSLPR-UnitTest -j 30')",
"Bash(command -v dumpbin)",
"Bash(dumpbin /SYMBOLS \"/c/Projects/CLionProjects/ANSCORE/cmake-build-release/bin/ANSOCR.dll\")",
"Bash(\"/c/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.44.35207/bin/Hostx64/x64/dumpbin.exe\" //SYMBOLS \"/c/Projects/CLionProjects/ANSCORE/cmake-build-release/bin/ANSOCR.dll\")",
"Bash(cmake --build . --target ANSODEngine -j 8)",
"PowerShell(@\\(Get-ChildItem -Path \"C:\\\\Program Files\\\\Microsoft Visual Studio\" -Filter \"vcvars64.bat\" -Recurse -ErrorAction SilentlyContinue\\) + @\\(Get-ChildItem -Path \"C:\\\\Program Files \\(x86\\)\\\\Microsoft Visual Studio\" -Filter \"vcvars64.bat\" -Recurse -ErrorAction SilentlyContinue\\) | ForEach-Object { $_.FullName })",
"PowerShell($d = Get-Item C:\\\\Projects\\\\CLionProjects\\\\ANSCORE\\\\cmake-build-release\\\\bin\\\\ANSODEngine.dll; \"$\\($d.Name\\) | $\\($d.Length\\) bytes | $\\($d.LastWriteTime\\)\")",
"Bash(awk '{if \\($0 ~ /=0+$/\\) print \"ZERO\"; else print \"NONZERO\"}')",
"Bash(awk '{print $4}' \"C:\\\\Users\\\\nghia\\\\Downloads\\\\ANSAIBOX-Q1A01S3_memoryleak1.log\")",
"Bash(awk -F'\\\\t' 'NF>=4 && $3 ~ /^[0-9]+$/ {print $3}' \"C:\\\\Users\\\\nghia\\\\Downloads\\\\ANSAIBOX-Q1A01S3_memoryleak1.log\")",
"PowerShell($dll = \"C:\\\\Projects\\\\CLionProjects\\\\ANSCORE\\\\bin\\\\anslicensing.dll\"; if \\(Test-Path $dll\\) { Get-Process | Where-Object { try { $_.Modules | Where-Object { $_.FileName -eq $dll } } catch {} } | Select-Object Id, ProcessName } else { \"DLL not present at $dll\" })",
"PowerShell($p = \"C:\\\\Projects\\\\CLionProjects\\\\ANSCORE\\\\cmake-build-release\\\\bin\\\\anslicensing.dll\"; try { $fs = [System.IO.File]::Open\\($p, 'Open', 'ReadWrite', 'None'\\); $fs.Close\\(\\); \"OK: writable, no other handle\" } catch { \"LOCKED: $\\($_.Exception.Message\\)\" })",
"PowerShell($candidates = @\\(\"handle64.exe\",\"handle.exe\"\\); $found = $null; foreach \\($c in $candidates\\) { $p = Get-Command $c -ErrorAction SilentlyContinue; if \\($p\\) { $found = $p.Source; break } }; if \\($found\\) { Write-Output \"FOUND: $found\"; & $found -nobanner -accepteula \"anslicensing.dll\" 2>$null } else { Write-Output \"handle.exe not in PATH\" })",
"PowerShell(Get-CimInstance Win32_Process -Filter \"ProcessId=22688\")",
"PowerShell(Get-CimInstance Win32_Process -Filter \"ParentProcessId=22688\")",
"Bash(taskkill //F //PID 50352)",
"Bash(awk -F'\\\\t' '/\\\\[TRT_Leak\\\\]/' \"C:/Users/nghia/Downloads/ANSLEGION.log1.log\")",
"Bash(awk -F'\\\\t' '/\\\\[MEDIA_Leak\\\\]/' \"C:/Users/nghia/Downloads/ANSLEGION.log1.log\")",
"Bash(awk -F'\\\\t' '/\\\\[MEDIA_Leak\\\\]/ {print $2, $4}' \"C:/Users/nghia/Downloads/ANSLEGION.log1.log\")",
"Bash(awk 'NR==1 || NR%50==0 || NR>510')",
"Bash(awk -F'\\\\t' '/\\\\[TRT_Leak\\\\]/ {if \\($4 !~ /creates=0 destroys=0 net=0 cached=0/\\) print NR\": \"$0}' \"C:/Users/nghia/Downloads/ANSLEGION.log1.log\")",
"Bash(awk '-F\\\\t' ' *)",
"Bash(awk -F'\\\\t' '/\\\\[TRT_Build\\\\]/' \"C:/Users/nghia/Downloads/ANSLEGION.log1.log\")",
"Bash(awk -F'\\\\t' '/VRAM/ {print $2, $4}' \"C:/Users/nghia/Downloads/ANSLEGION.log1.log\")",
"Bash(awk 'NR==1 || NR%2000==0')",
"Bash(awk -F'\\\\t' '/\\\\[ALPR_Track\\\\]|\\\\[ALPR_TrackId\\\\]/' \"C:/Users/nghia/Downloads/ANSLEGION.log1.log\")",
"Bash(awk -F'\\\\t' '/\\\\[ANSCUSTOM_Infer\\\\]/' \"C:/Users/nghia/Downloads/ANSLEGION.log1.log\")",
"Bash(sort -t= -k2 -n -r)",
"Bash(awk -F'\\\\t' '/\\\\[ALPR_TrackId\\\\]/' \"C:/Users/nghia/Downloads/ANSLEGION.log1.log\")",
"Bash(awk -F'\\\\t' '/\\\\[ANSMOT\\\\]/' \"C:/Users/nghia/Downloads/ANSLEGION.log1.log\")",
"Bash(awk '-F\\\\t' '/\\\\[ANSMOT\\\\] BYTETracker state/ { *)",
"Bash(awk -F'\\\\t' '/setInputShape/' \"C:/Users/nghia/Downloads/ANSLEGION.log1.log\")",
"Bash(awk -F'\\\\t' '/CUDA graph CAPTURE starting/' \"C:/Users/nghia/Downloads/ANSLEGION.log1.log\")",
"Bash(awk '-F\\\\t' '/setInputShape/{ *)",
"Bash(python -m pip install reportlab --quiet)",
"Bash(python build_anscore_devguide_pdf.py)",
"Bash(python -c \"from pypdf import PdfReader; r = PdfReader\\(r'C:/Projects/CLionProjects/ANSCORE/ANSODEngine_Tracker_Stabilization_Guide.pdf'\\); print\\('pages:', len\\(r.pages\\)\\)\")",
"Bash(python -c \"import reportlab.pdfbase.pdfmetrics; print\\('pages: \\(pypdf not available — file size confirms\\)'\\)\")",
"Bash(python -m pip install pypdf --quiet)",
"Bash(python -c \"from pypdf import PdfReader; r = PdfReader\\(r'C:/Projects/CLionProjects/ANSCORE/ANSODEngine_Tracker_Stabilization_Guide.pdf'\\); print\\('pages:', len\\(r.pages\\)\\); print\\('first page text \\(head\\):', r.pages[0].extract_text\\(\\)[:300].replace\\(chr\\(10\\), ' '\\)\\)\")",
"Bash(awk -F'\\\\t' '{print NR}' \"C:/Users/nghia/Downloads/ANSLEGION.log1.log\")",
"Bash(awk -F'\\\\t' 'END {print \"lines:\", NR, \" duration_s:\", $2, \" rate_lines/s:\", NR/$2}' \"C:/Users/nghia/Downloads/ANSLEGION.log1.log\")",
"Bash(grep -nE \"RunInference|RunInferencesBatch|infer\\\\\\\\\\(\\\\\\\\\\)|set_input_tensor|inference_request|m_inferRequest|mutex|lock_guard\" C:/Projects/CLionProjects/ANSCORE/engines/OpenVINOEngine/src/**/*.cpp C:/Projects/CLionProjects/ANSCORE/engines/OpenVINOEngine/include/**/*.h)",
"Bash(grep -nE \"DetectObjects|TryLockWithTimeout|ortRun|->Run\\\\\\\\\\(|m_ortEngine|infer|RunInference\" C:/Projects/CLionProjects/ANSCORE/modules/ANSODEngine/ANSONNXYOLO.cpp)",
"Bash(grep -nE \"ONNXYOLO::detect|ort_session->Run|set_input|detect\\\\\\\\\\(.*const.*Mat|m_inputBuffer|m_input_tensor|allocator|ortInput|run_options|preprocess|postprocess|class ONNXYOLO\" C:/Projects/CLionProjects/ANSCORE/modules/ANSODEngine/ANSONNXYOLO.cpp C:/Projects/CLionProjects/ANSCORE/modules/ANSODEngine/ANSONNXYOLO.h)",
"Bash(ninja ANSODEngine)",
"PowerShell($env:Path = \"C:\\\\Program Files\\\\Microsoft Visual Studio\\\\2022\\\\Community\\\\VC\\\\Tools\\\\Llvm\\\\x64\\\\bin;\" + $env:Path; & \"C:\\\\Program Files\\\\Microsoft Visual Studio\\\\2022\\\\Community\\\\VC\\\\Auxiliary\\\\Build\\\\vcvarsall.bat\" x64 2>&1 | Out-Null; Get-Command ninja -ErrorAction SilentlyContinue | Select-Object Source)",
"PowerShell($ninjaPaths = @\\(\"C:\\\\Program Files\\\\Microsoft Visual Studio\\\\2022\\\\Community\\\\Common7\\\\IDE\\\\CommonExtensions\\\\Microsoft\\\\CMake\\\\Ninja\\\\ninja.exe\", \"C:\\\\ProgramData\\\\chocolatey\\\\bin\\\\ninja.exe\", \"C:\\\\tools\\\\ninja\\\\ninja.exe\"\\); foreach \\($p in $ninjaPaths\\) { if \\(Test-Path $p\\) { Write-Output $p } })",
"PowerShell(cmd /c '\"C:\\\\Program Files\\\\Microsoft Visual Studio\\\\2022\\\\Community\\\\VC\\\\Auxiliary\\\\Build\\\\vcvarsall.bat\" x64 >nul 2>&1 && cd /d C:\\\\Projects\\\\CLionProjects\\\\ANSCORE\\\\cmake-build-release && \"C:\\\\Program Files\\\\Microsoft Visual Studio\\\\2022\\\\Community\\\\Common7\\\\IDE\\\\CommonExtensions\\\\Microsoft\\\\CMake\\\\Ninja\\\\ninja.exe\" ANSODEngine 2>&1')",
"Bash(python build_anslpr_devguide_pdf.py)",
"Bash(where dumpbin.exe)",
"Bash(where llvm-symbolizer.exe)",
"Bash(where cdb.exe)",
"Bash(\"C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/Llvm/x64/bin/llvm-symbolizer.exe\" --obj=\"C:/Projects/AIVISION/ANSFallDetection/ANSCustomFall/Dll1/x64/Release/ANSCustomFall.dll\" 0xb1d1)",
"Bash(cmake --build C:/Projects/CLionProjects/ANSCORE/cmake-build-release --target ANSLicensingSystem -- /m)",
"Bash(cmake --build C:/Projects/CLionProjects/ANSCORE/cmake-build-release --target ANSLicensingSystem)",
"Bash(cmd /c \"\\\\\"C:\\\\\\\\Program Files\\\\\\\\Microsoft Visual Studio\\\\\\\\2022\\\\\\\\Community\\\\\\\\VC\\\\\\\\Auxiliary\\\\\\\\Build\\\\\\\\vcvars64.bat\\\\\" >nul && cmake --build C:\\\\\\\\Projects\\\\\\\\CLionProjects\\\\\\\\ANSCORE\\\\\\\\cmake-build-release --target ANSLicensingSystem 2>&1\")",
"Bash(cmd /c \"call \\\\\"C:\\\\\\\\Program Files\\\\\\\\Microsoft Visual Studio\\\\\\\\2022\\\\\\\\Community\\\\\\\\VC\\\\\\\\Auxiliary\\\\\\\\Build\\\\\\\\vcvars64.bat\\\\\" && cmake --build C:\\\\\\\\Projects\\\\\\\\CLionProjects\\\\\\\\ANSCORE\\\\\\\\cmake-build-release --target ANSLicensingSystem\")",
"PowerShell($vcvars = \"C:\\\\Program Files\\\\Microsoft Visual Studio\\\\2022\\\\Community\\\\VC\\\\Auxiliary\\\\Build\\\\vcvars64.bat\"; cmd /c \"`\"$vcvars`\" && cmake --build C:\\\\Projects\\\\CLionProjects\\\\ANSCORE\\\\cmake-build-release --target ANSLicensingSystem 2>&1\" 2>&1 | Select-Object -Last 120)",
"Bash(grep -rn \"^using namespace std\" \"C:/ANSLibs/anslicensing/\" 2>/dev/null | head -5; grep -rn \"^using namespace std\" \"C:/Projects/CLionProjects/ANSCORE/core/anslicensing/\" 2>/dev/null | head -5)",
"PowerShell(Start-Process powershell -Verb RunAs -ArgumentList '-NoProfile','-Command',\"try { Get-WmiObject MSAcpi_ThermalZoneTemperature -Namespace root\\\\wmi | Select-Object InstanceName,CurrentTemperature | Out-File C:\\\\Projects\\\\CLionProjects\\\\ANSCORE\\\\cmake-build-release\\\\_smoke\\\\thermal.txt } catch { '$_' | Out-File C:\\\\Projects\\\\CLionProjects\\\\ANSCORE\\\\cmake-build-release\\\\_smoke\\\\thermal.txt }\" -Wait; Get-Content C:\\\\Projects\\\\CLionProjects\\\\ANSCORE\\\\cmake-build-release\\\\_smoke\\\\thermal.txt)"
]
}
}
Submodule .claude/worktrees/heuristic-lichterman-bda22b added at 66c67dff26
Submodule .claude/worktrees/lucid-grothendieck-201ca6 added at a526aa6cdb
+452
View File
@@ -0,0 +1,452 @@
# AMD DirectML FP16 Converter — Implementation Plan
**Status:** Not started. Pick this up in a future session.
**Estimated effort:** ~1 week of focused work.
**Owner of context:** This document is self-contained — you don't need the
prior conversation to execute it.
---
## 1. Goal
Recover ~30–50% of per-call inference latency on AMD Radeon iGPUs (DML EP)
by running models in FP16 instead of FP32. Pure C++ implementation —
no Python tooling at any stage. Mirror the existing TensorRT pattern
where `OrtHandlerOptions::trtFP16 = true` causes the engine layer to
build/cache an FP16 plan automatically at session-create time.
The current AMD performance baseline (after the always-serial-on-AMD fix)
is ~5.4 fps for full-frame ALPR. Target after FP16: 7–8 fps.
## 2. Why C++ (and not the standard Python toolchain)
The standard tool for this is `onnxconverter-common.float16.convert_float_to_float16`.
Project rule disallows Python anywhere in the workflow, so we must
re-implement the algorithm in C++ inside ANSCORE.
The DirectML EP itself has no `dml_fp16_enable` runtime knob (unlike TRT's
`trt_fp16_enable`). DML runs whatever precision the ONNX model declares.
So the C++ work has to produce a transformed FP16 ONNX, then hand that
to the DML EP.
## 3. Architecture (mirrors the TRT pattern)
```
TRT path (existing):
trtFP16 = true → ORT TRT EP options → TRT engine builder rewrites
→ engine cached on disk → inference
DML path (new):
dmlFP16 = true → ANSCORE C++ rewrite → cached as <hash>.fp16.onnx
→ DML EP loads cached FP16 model → inference
```
Cache invalidation, per-model granularity, and the "first run is slow"
semantics all match TRT. From the call-site perspective the TRT and DML
options are symmetric.
## 4. Current state of the codebase (relevant files)
These files already exist and are the integration points. Don't recreate
them; modify them.
| File | Relevant content | What this plan adds |
|---|---|---|
| `engines/ONNXEngine/ONNXEngine.h` | `OrtHandlerOptions` struct. AMD section commented "OPEN FOR OPTIMIZATION" — empty today | Add `dmlFP16`, `dmlOptimizedModelDir`, `dmlFP16OpBlockList` fields |
| `engines/ONNXEngine/ONNXEngine.cpp` | `BasicOrtHandler::initialize_handler()` — DML EP branch (search for `EngineType::AMD_GPU`) | Insert path-resolution call before session creation |
| `engines/ONNXEngine/ONNXEngine.cpp` | Already houses `DmlSerialState` + `DmlInferenceGuard` (the always-serial-on-AMD fix from previous work) | Reuse the same TU for FP16 helpers |
| `engines/ONNXEngine/Fp16GraphRewriter.{h,cpp}` | **Does not exist yet** | Create — main implementation of this plan |
| `modules/ANSOCR/ANSONNXOCR/PaddleOCRV5Engine.cpp` | `BuildXxxOcrOptions()` helpers populating per-model `OrtHandlerOptions` | Add `o.dmlFP16 = true/false` per model |
| Caller code in `ANSONNXYOLO`, `ONNXOCRDetector`, `ONNXOCRRecognizer` | Already constructs sessions via `BasicOrtHandler` | No changes — the FP16 swap is invisible to callers |
## 5. Step-by-step plan
### Step 1 — Add AMD knobs to `OrtHandlerOptions` (half day)
In `engines/ONNXEngine/ONNXEngine.h`, replace the AMD placeholder:
```cpp
// ----------------------------------------------------------------
// AMD (DirectML EP) — FP16 graph rewrite
// ----------------------------------------------------------------
// Convert FP32 weights to FP16 before session creation. Mirrors
// trtFP16 semantically: an in-process graph rewrite produces an FP16
// ONNX, cached on disk under dmlOptimizedModelDir, and the DML EP
// loads the cached FP16 model. External inputs/outputs stay FP32 via
// Cast nodes inserted at the boundaries — caller code is unchanged.
// Ignored on every non-DML EP. Default off until per-model accuracy
// validation completes.
bool dmlFP16 = false;
// Where to cache rewritten FP16 ONNX models. Empty → default
// %TEMP%/ANSCENTER/DMLOptimizedCache. Cache key is a hash of
// (source ONNX bytes + dmlFP16 flag + opBlockList). Stale entries
// are silently replaced when source changes.
std::string dmlOptimizedModelDir;
// Op types whose outputs MUST remain FP32 (NMS, TopK, RoiAlign, plus
// any op observed to produce NaN/Inf in FP16 for THIS model). Empty
// = use the built-in default block list (see kDefaultFp16BlockList in
// Fp16GraphRewriter.cpp). Per-model override.
std::vector<std::string> dmlFP16OpBlockList;
```
### Step 2 — Add ONNX protobuf to the build (half day)
You need the `onnx.proto3` schema compiled to C++ via `protoc`. Two
sub-options:
**Option 2A (recommended): vendor the proto file.**
1. Download `onnx.proto3` from https://github.com/onnx/onnx (current
release tag). Put it at `engines/ONNXEngine/proto/onnx.proto3`.
2. Add a CMake `add_custom_command` calling `protoc --cpp_out=...` at
configure time, generating `onnx.pb.{h,cc}` into the build directory.
3. Link `Fp16GraphRewriter` against `protobuf::libprotobuf`.
`libprotobuf` is almost certainly already a transitive dep of
`onnxruntime` — confirm by checking
`C:\ANSLibs\onnxruntime\include` for `google/protobuf/...` headers and
`C:\ANSLibs\protobuf` (or vcpkg's protobuf install). Just expose the
include directories to your new target.
**Option 2B: vendor the full ONNX C++ library.**
Bigger dep but provides typed helpers, validators, and a checker. Skip
unless 2A's hand-rolled approach blows up on edge cases.
### Step 3 — Implement `Fp16GraphRewriter` (~2 days for core)
Create `engines/ONNXEngine/Fp16GraphRewriter.h`:
```cpp
#pragma once
#include <string>
#include <vector>
#include <unordered_set>
namespace ANSCENTER {
// Returns the path to an FP16 version of fp32Path. If cacheDir already
// contains a fresh FP16 cache for this model, returns the cached path
// directly (fast path). Otherwise loads fp32Path, runs the in-memory
// FP16 graph rewrite, writes to cacheDir, and returns the new path.
//
// Cache filename embeds a hash of (fp32 file content + opBlockList) so
// a model update or blocklist change automatically invalidates the
// cache — same pattern as TRT engine cache invalidation.
//
// On any failure (parse error, write error, etc.) returns fp32Path
// unchanged and logs a warning via ANS_DBG. The session creation
// continues with the FP32 model — this function NEVER blocks the
// inference path.
std::string ResolveFp16OnnxPath(
const std::string& fp32Path,
const std::string& cacheDir,
const std::vector<std::string>& opBlockList);
} // namespace ANSCENTER
```
Create `engines/ONNXEngine/Fp16GraphRewriter.cpp`. Key components:
#### 3a. IEEE-754 binary32 → binary16 helper
Standard 30-line implementation, round-to-nearest-even, handle NaN/Inf.
Reference: numpy's `numpy/core/src/npymath/halffloat.c` or
`Eigen/src/Core/arch/Default/Half.h`. Self-contained, no deps.
#### 3b. Default op blocklist
```cpp
static const std::unordered_set<std::string> kDefaultFp16BlockList = {
// Object-detection post-processing — known FP16-unsafe
"NonMaxSuppression", "TopK", "RoiAlign",
// Classical ML ops — never have FP16 implementations
"ArrayFeatureExtractor", "Binarizer", "CastMap",
"CategoryMapper", "DictVectorizer", "FeatureVectorizer",
"Imputer", "LabelEncoder", "LinearClassifier",
"LinearRegressor", "Normalizer", "OneHotEncoder",
"RandomUniform", "SVMClassifier", "SVMRegressor",
"Scaler", "TreeEnsembleClassifier", "TreeEnsembleRegressor",
"ZipMap",
};
```
#### 3c. Initializer conversion
Walk `model.graph().initializer()`. For every `TensorProto` with
`data_type == FLOAT`:
1. Pull the float data (it lives in `raw_data` OR `float_data` —
handle both).
2. Convert each float32 to uint16 binary16.
3. Set `data_type = FLOAT16`, clear `float_data`, write the new bytes
to `raw_data`.
Watch for: external data references (when `data_location ==
EXTERNAL`) — for the YOLO and OCR models in this project, weights are
embedded, but write a guard: if external, skip with a warning.
#### 3d. Cast-node insertion at boundaries
For each external input (entries in `model.graph().input()` that are
NOT initializer names):
1. Note the input's tensor name `orig`.
2. Append a new `NodeProto` to the graph: `Cast(input=orig,
output=orig + "_fp16", to=FLOAT16)`.
3. Walk every existing node in the graph; for any input that equals
`orig`, replace it with `orig + "_fp16"`. Be careful not to rewire
the new Cast node itself (skip the last node added).
For each external output: insert `Cast(FP16 → FP32)` symmetrically.
#### 3e. value_info type updates
For every `ValueInfoProto` in `graph().value_info()`:
- If the tensor is the input to OR output of a block-listed op, leave
its `tensor_type.elem_type` as FP32.
- Otherwise update to FP16.
This step is the most error-prone in practice. When in doubt, leave
elem_type alone — the runtime will accept inferred types. Test
correctness empirically with the validation harness in Step 7.
#### 3f. Subgraphs
If/Loop/Scan ops have nested subgraphs (`AttributeProto` of type
GRAPH). Recurse into each. The OCR recognizer doesn't have these;
neither does YOLO post-Ultralytics-export — but write the recursion
defensively, otherwise this fails silently on future models.
#### 3g. Cache management
Cache filename: `<model_basename>.<sha1_8>.fp16.onnx`, where `sha1_8` is
the first 8 hex chars of `SHA1(fp32_file_bytes || serialized
opBlockList || "v1")`. The "v1" string lets you bust the cache for
all models if you fix a converter bug later.
Cache directory: create on demand with `std::filesystem::create_directories`.
Default: `std::filesystem::temp_directory_path() / "ANSCENTER" /
"DMLOptimizedCache"`.
Lookup: if cache file exists AND its mtime is newer than the source
ONNX, use it. Else regenerate.
### Step 4 — Wire into `initialize_handler` (half day)
In `engines/ONNXEngine/ONNXEngine.cpp`, find the DML branch in
`BasicOrtHandler::initialize_handler()`. Insert path resolution before
session creation:
```cpp
case EngineType::AMD_GPU: {
std::string onnxToLoad = _onnx_path;
if (options.dmlFP16) {
const std::string cacheDir =
options.dmlOptimizedModelDir.empty()
? GetDefaultDmlCacheDir()
: options.dmlOptimizedModelDir;
onnxToLoad = ResolveFp16OnnxPath(
_onnx_path, cacheDir, options.dmlFP16OpBlockList);
ANS_DBG("OrtHandler",
"AMD FP16: source=%s cached=%s",
_onnx_path.c_str(), onnxToLoad.c_str());
}
if (!TryAppendDirectML(opts)) { /* existing fallback */ }
ort_session = std::make_unique<Ort::Session>(
env, onnxToLoad.c_str(), opts);
break;
}
```
`GetDefaultDmlCacheDir()` is a small helper in `Fp16GraphRewriter.cpp`.
### Step 5 — Per-model configuration (1 day)
Update the option-builder functions to opt models into FP16:
`modules/ANSOCR/ANSONNXOCR/PaddleOCRV5Engine.cpp` (and equivalents):
```cpp
OrtHandlerOptions BuildLpDetectorOptions() {
OrtHandlerOptions o;
o.preferTensorRT = true;
o.trtFP16 = true;
o.dmlFP16 = true; // YOLO det validated FP16-safe
return o;
}
OrtHandlerOptions BuildOcrDetectorOptions() {
OrtHandlerOptions o;
o.preferTensorRT = true;
o.trtFP16 = true;
o.dmlFP16 = true; // DBNet text-det validated
return o;
}
OrtHandlerOptions BuildOcrRecognizerOptions() {
OrtHandlerOptions o;
o.preferTensorRT = true;
o.trtFP16 = true;
// OCR recognizer is FP16-sensitive on small text — keep AMD FP32
// until per-plate accuracy validation passes (see Step 7).
o.dmlFP16 = false;
return o;
}
```
Helmet OD (in `ANSCUSTOMDetector` model loading) — set `dmlFP16 = true`.
### Step 6 — DebugView logging (per `MEMORY.md` rule)
Sprinkle `ANS_DBG` markers at every decision/error point in
`Fp16GraphRewriter.cpp`:
- Entry: `ANS_DBG("Fp16Rewriter", "ResolveFp16OnnxPath: src=%s cache=%s",
fp32Path.c_str(), cacheDir.c_str());`
- Cache hit: `ANS_DBG("Fp16Rewriter", "cache HIT %s", cachedPath.c_str());`
- Cache miss → rewrite: `ANS_DBG("Fp16Rewriter",
"cache MISS — rewriting (initializers=%zu nodes=%zu)", ...);`
- Per phase: log initializer-conversion count, Cast-insertion count,
value_info update count.
- On error: `ANS_DBG("Fp16Rewriter",
"ERROR: %s — falling back to FP32 source", ex.what());` and return
the original FP32 path (never block inference).
### Step 7 — Per-model accuracy validation (1–2 days)
You can't ship FP16 without per-model accuracy validation. Build a
small C++ harness — separate executable target, runs against
labeled fixtures from your test data:
```
tools/AccuracyComparator/AccuracyComparator.cpp
- Loads two sessions (FP32 and FP16) on CPU EP for determinism.
- Feeds N representative frames through both.
- For YOLO det: compare box IoU, score deltas, count of boxes lost.
- For OCR det: compare text-region masks (pixel-wise IoU on the prob
map binary at threshold).
- For OCR rec: compare DECODED CTC strings — not raw logits.
A single character delta on a plate is a regression.
- Emit pass/fail against a configurable tolerance.
```
Acceptance criteria:
| Model | Tolerance | What "pass" means |
|---|---|---|
| YOLO LP detector | mean IoU ≥ 0.95 vs FP32 boxes | No boxes lost, score delta < 0.05 |
| YOLO Helmet OD | mean IoU ≥ 0.95 | Same |
| OCR DBNet | mask IoU ≥ 0.92 at threshold | No new false negatives |
| OCR CTC recognizer | char accuracy within 0.5% | If fails, leave AMD FP32 |
Run on at least 500 frames per model from production data.
### Step 8 — Build, deploy, measure (half day)
Build artifacts: `ANSODEngine.dll`, `ANSOCR.dll`. No new DLL — the
rewriter lives inside the existing engine layer.
Deploy to AMD test box. First run: each session takes an extra
2–10 s during initial cache build. Look for `[Fp16Rewriter] cache
MISS — rewriting` lines.
Subsequent runs: instant cache hit. Look for `[Fp16Rewriter] cache HIT`.
Measure:
- Per-call DML latency in `STATS` lines (mean, p99).
- Total fps over a 60-second run.
- Compare to the always-serial-on-AMD baseline (5.4 fps full-frame ALPR).
- Target: ≥ 7 fps.
If a model regresses accuracy in the field (rare with the validation
harness in Step 7, but possible), flip its `dmlFP16` back to `false`
and ship a hotfix. No code change needed beyond the option flag.
## 6. Dependencies summary
| Dep | Where it lives | New or existing |
|---|---|---|
| `libprotobuf` (C++ runtime) | Almost certainly already vendored via onnxruntime; otherwise add via vcpkg or `C:\ANSLibs\protobuf` | Existing, just expose includes |
| `onnx.proto3` schema file | Vendor at `engines/ONNXEngine/proto/onnx.proto3` from ONNX repo | New, ~1 KB file |
| `protoc` compiler | Comes with libprotobuf install | Existing, just call from CMake |
| `<filesystem>` for cache dir | C++17 standard | Existing |
| SHA1 helper for cache key | Pull from any tiny public-domain impl, or use libcrypto if linked | Tiny new file (~80 lines) |
No third-party FP16 libraries. No Python. No new DLLs in deployment.
## 7. Risks and mitigations
| Risk | Likelihood | Mitigation |
|---|---|---|
| Conversion produces NaN/Inf in some op | Medium | Default blocklist + per-model `dmlFP16OpBlockList` override. Always validate with Step 7 harness before flipping a model on. |
| Some FP16 op has no DML kernel → falls back to CPU silently → SLOWER than FP32 | Medium | Profile each model on AMD with ORT's profiling JSON output (`SessionOptions::EnableProfiling`). If a CPU fallback shows up, blocklist the offending op for that model. |
| ONNX schema version mismatch (we vendor proto3, models use proto with newer fields) | Low | Models from this project are exported by Ultralytics + PaddleOCR — both target stable opset 12-17 ranges that proto3 covers fully. Validate by attempting parse on a known-good model during Step 3. |
| Cache directory permissions on customer machines | Low | Default to `%TEMP%` which is always writable; allow override via `dmlOptimizedModelDir`. On any cache-write failure, fall back to FP32 source path (no inference impact). |
| Subgraph (If/Loop/Scan) bug | Low for current models | Recurse defensively in Step 3f; assert if a subgraph is encountered, fail-safe to FP32. |
## 8. Out of scope (do NOT do as part of this plan)
- **Auto mixed precision** (per-op accuracy-driven precision selection).
That requires running a representative dataset through the model to
decide which ops can safely be FP16. Doable in C++ but adds 1+ week.
Defer to a follow-up plan.
- **Model quantization to INT8.** Different problem, different tooling
(calibration data needed).
- **Switching from DML to MIGraphX or ROCm.** Different EP, different
port. Defer.
- **Recovering depth=2 parallelism on AMD.** The always-serial fix is
in place and stable; this plan is purely about per-call speedup
while remaining serial. Combine the two for net throughput gain.
## 9. Project rules to follow (from MEMORY.md)
- **Edit directly in main checkout.** No worktree, no branch.
- **No commits without explicit user request.** When this plan ships,
don't auto-commit.
- **Sprinkle `ANS_DBG` on new code paths** — entry, decisions, errors.
See Step 6 for exact placement.
## 10. Acceptance checklist
When this plan is done, the following must all be true:
- [ ] `OrtHandlerOptions::dmlFP16` exists and is documented inline.
- [ ] `Fp16GraphRewriter.{h,cpp}` exists with `ResolveFp16OnnxPath` exposed.
- [ ] `BasicOrtHandler::initialize_handler` calls the resolver in the
AMD branch when `dmlFP16` is set.
- [ ] At least one production model (LP YOLO det) is opted in via its
option-builder.
- [ ] Accuracy harness in `tools/AccuracyComparator/` exists and produces
pass/fail for the opted-in models.
- [ ] AMD test-box measurement shows per-call DML latency reduced by
≥ 25% on the opted-in model vs FP32 baseline.
- [ ] No regression on Intel or NVIDIA — the FP16 path never executes
on those EPs (`if (options.dmlFP16)` is gated on `EngineType::AMD_GPU`
via the case-branch placement).
- [ ] DebugView logs cleanly: cache HIT / cache MISS / rewrite phases
/ errors all visible.
- [ ] First-run cache-build latency is ≤ 10 s per model on AMD test box.
- [ ] Subsequent-run cache-load latency is ≤ 100 ms per model.
## 11. Open questions to resolve at start
When you start this plan in a future session, answer these first:
1. Is `libprotobuf` already linked into ANSODEngine.dll (transitively
via onnxruntime)? Run `dumpbin /imports ANSODEngine.dll | grep
protobuf` to confirm. If yes, skip the vcpkg/manual install step.
2. Where does `C:\ANSLibs\onnxruntime\include` live exactly? Confirm
path for the protoc include directive.
3. Which ONNX opset version do your YOLO and OCR models target?
Run `Ort::Session` with profiling once and check the model
metadata. This confirms which `onnx.proto3` revision to vendor.
4. Is the initial AMD measurement still ~5.4 fps with the always-serial
fix in place? If something regressed in the meantime, fix that
first before adding FP16.
---
*Last updated: when this plan was authored. No code shipped yet.*
+96 -20
View File
@@ -1,31 +1,64 @@
#include "ANSLIB.h"
#include "ANSLibsLoader.h"
#include <memory>
#include <mutex>
#define LOAD_FUNC(name) \
name##Func = (name##FuncT)GetProcAddress(dllHandle, #name); \
if (!name##Func) { success = false; }
namespace ANSCENTER
{
ANSLIB::ANSLIB() {
namespace {
// Function-pointer typedefs for the ANSLicensingSystem C-facade exports.
// Resolved exactly once during PreloadSharedDllsOnce(); after that the
// pointers are write-once globals — concurrent reads are race-free.
using ANSLogger_LogFnT = void(*)(const char*, const char*, const char*, int);
using ANSCENTER_IsDbgEnFnT = int (*)(void);
using ANSCENTER_DebugLogFnT = void(*)(const char*);
ANSLogger_LogFnT g_ANSLogger_LogInfo = nullptr;
ANSLogger_LogFnT g_ANSLogger_LogError = nullptr;
ANSLogger_LogFnT g_ANSLogger_LogFatal = nullptr;
ANSCENTER_IsDbgEnFnT g_ANSCENTER_IsDebugViewEnabled = nullptr;
ANSCENTER_DebugLogFnT g_ANSCENTER_DebugLog = nullptr;
// Pre-load all ANSCORE DLLs from the Shared folder using full paths.
// This ensures the correct versions are loaded regardless of PATH order
// (e.g., DLHUB_Runtime_Engine may contain older copies on some machines).
// Once loaded, Windows reuses them for all subsequent implicit dependencies.
const char* sharedDir = "C:\\ProgramData\\ANSCENTER\\Shared\\";
const char* preloadDlls[] = {
"ANSLicensingSystem.dll",
"anslicensing.dll",
"ANSMOT.dll",
"ANSODEngine.dll",
nullptr
};
for (int i = 0; preloadDlls[i] != nullptr; i++) {
std::string fullPath = std::string(sharedDir) + preloadDlls[i];
LoadLibraryA(fullPath.c_str());
}
// Run exactly once per process so LoadLibrary refcounts don't grow per
// ANSLIB instance. The DLLs are intentionally pinned for process lifetime.
void PreloadSharedDllsOnce() {
static std::once_flag s_preloadFlag;
std::call_once(s_preloadFlag, []() {
const char* sharedDir = "C:\\ProgramData\\ANSCENTER\\Shared\\";
const char* preloadDlls[] = {
"ANSLicensingSystem.dll",
"anslicensing.dll",
"ANSMOT.dll",
"ANSODEngine.dll",
nullptr
};
for (int i = 0; preloadDlls[i] != nullptr; i++) {
std::string fullPath = std::string(sharedDir) + preloadDlls[i];
LoadLibraryA(fullPath.c_str());
}
// Ensure all shared DLLs (OpenCV, OpenVINO, TRT, ORT) are pre-loaded
ANSCENTER::ANSLibsLoader::Initialize();
// Ensure all shared DLLs (OpenCV, OpenVINO, TRT, ORT) are pre-loaded
ANSCENTER::ANSLibsLoader::Initialize();
// Resolve logging C-facade symbols from the (now-pinned) licensing DLL.
// Use GetModuleHandleA — the DLL is already in this process's module
// list courtesy of the LoadLibraryA above, so refcount is unchanged.
if (HMODULE lic = GetModuleHandleA("ANSLicensingSystem.dll")) {
g_ANSLogger_LogInfo = (ANSLogger_LogFnT) GetProcAddress(lic, "ANSLogger_LogInfo");
g_ANSLogger_LogError = (ANSLogger_LogFnT) GetProcAddress(lic, "ANSLogger_LogError");
g_ANSLogger_LogFatal = (ANSLogger_LogFnT) GetProcAddress(lic, "ANSLogger_LogFatal");
g_ANSCENTER_IsDebugViewEnabled = (ANSCENTER_IsDbgEnFnT) GetProcAddress(lic, "ANSCENTER_IsDebugViewEnabled");
g_ANSCENTER_DebugLog = (ANSCENTER_DebugLogFnT)GetProcAddress(lic, "ANSCENTER_DebugLog");
}
});
}
}
ANSLIB::ANSLIB() {
PreloadSharedDllsOnce();
const char* dllPath = "C:\\ProgramData\\ANSCENTER\\Shared\\ANSODEngine.dll";
dllHandle = LoadLibraryA(dllPath);
@@ -99,6 +132,10 @@ namespace ANSCENTER
return -1;
}
if (CreateANSODHandle_CSFunc) {
if (ANSHandle && ReleaseANSODHandleFunc) {
ReleaseANSODHandleFunc(&ANSHandle);
ANSHandle = nullptr;
}
const char* result = CreateANSODHandle_CSFunc(&ANSHandle, licenseKey, modelFilePath, modelFileZipPassword,
modelThreshold, modelConfThreshold, modelNMSThreshold,
1, modelType, detectionType, loadEngineOnCreation);
@@ -118,9 +155,8 @@ namespace ANSCENTER
return -1;
}
if (RunInference_CPPFunc) {
auto imagePtr = std::make_unique<cv::Mat>(cvImage);
try {
cv::Mat* rawPtr = imagePtr.get();
cv::Mat* rawPtr = &cvImage;
int result = RunInference_CPPFunc(&ANSHandle, &rawPtr, cameraId, detectionResult);
return result;
}
@@ -135,9 +171,8 @@ namespace ANSCENTER
return -1;
}
if (RunInferenceComplete_CPPFunc) {
auto imagePtr = std::make_unique<cv::Mat>(cvImage);
try {
cv::Mat* rawPtr = imagePtr.get();
cv::Mat* rawPtr = &cvImage;
int result = RunInferenceComplete_CPPFunc(&ANSHandle, &rawPtr, cameraId, activeROIMode,detectionResult);
return result;
}
@@ -193,6 +228,10 @@ namespace ANSCENTER
int autoDetectEngine, int modelType, int detectionType, int loadEngineOnCreation, const char* modelFolder,std::string& labelMap)
{
if (!loaded || !LoadModelFromFolderFunc) return -1;
if (ANSHandle && ReleaseANSODHandleFunc) {
ReleaseANSODHandleFunc(&ANSHandle);
ANSHandle = nullptr;
}
return LoadModelFromFolderFunc(&ANSHandle, licenseKey, modelName, className,
detectionScoreThreshold, modelConfThreshold, modelMNSThreshold,
autoDetectEngine, modelType, detectionType, loadEngineOnCreation,modelFolder, labelMap);
@@ -241,4 +280,41 @@ namespace ANSCENTER
if (!loaded || !SetTrackerParametersFunc) return -1;
return SetTrackerParametersFunc(&ANSHandle, trackerParams);
}
// ------------------------------------------------------------------------
// Static logging API. Each call ensures the DLL/symbols are resolved (cheap
// after the first call — std::call_once degenerates to an atomic load) then
// forwards to the C facade. Null-pointer guard makes missing symbols a
// silent no-op rather than a crash.
// ------------------------------------------------------------------------
int ANSLIB::IsDebugViewEnabled() {
PreloadSharedDllsOnce();
return g_ANSCENTER_IsDebugViewEnabled ? g_ANSCENTER_IsDebugViewEnabled() : 0;
}
void ANSLIB::DebugLog(const char* preformattedLine) {
PreloadSharedDllsOnce();
if (!preformattedLine) return;
if (g_ANSCENTER_DebugLog) {
// Preferred path: licensing DLL applies the prefix scrubber and emits.
g_ANSCENTER_DebugLog(preformattedLine);
return;
}
// Fallback for environments where ANSLicensingSystem.dll is missing:
// emit raw to DebugView + stderr without filtering. Better than dropping.
OutputDebugStringA(preformattedLine);
std::fputs(preformattedLine, stderr);
std::fflush(stderr);
}
void ANSLIB::LogInfo(const char* src, const char* msg, const char* file, int line) {
PreloadSharedDllsOnce();
if (g_ANSLogger_LogInfo) g_ANSLogger_LogInfo(src, msg, file, line);
}
void ANSLIB::LogError(const char* src, const char* msg, const char* file, int line) {
PreloadSharedDllsOnce();
if (g_ANSLogger_LogError) g_ANSLogger_LogError(src, msg, file, line);
}
void ANSLIB::LogFatal(const char* src, const char* msg, const char* file, int line) {
PreloadSharedDllsOnce();
if (g_ANSLogger_LogFatal) g_ANSLogger_LogFatal(src, msg, file, line);
}
}
+62 -2
View File
@@ -6,6 +6,7 @@
#include <windows.h>
#include <opencv2/opencv.hpp>
#include <opencv2/core/cuda.hpp>
#include <cstdio>
enum DetectionType {
CLASSIFICATION = 0,
DETECTION = 1,
@@ -49,7 +50,9 @@ enum ModelType {
ONNXSAM3 = 28,
RTSAM3 = 29,
ONNXYOLO = 30,
RTYOLO = 31
RTYOLO = 31,
ONNXRTMPOSE=32,
RTRTMPOSE=33
};
enum TrackerType {
BYTETRACK = 0,
@@ -263,6 +266,10 @@ namespace ANSCENTER {
public:
ANSLIB();
~ANSLIB() noexcept;
ANSLIB(const ANSLIB&) = delete;
ANSLIB& operator=(const ANSLIB&) = delete;
ANSLIB(ANSLIB&&) = delete;
ANSLIB& operator=(ANSLIB&&) = delete;
[[nodiscard]] static ANSLIB* Create();
static void Destroy(ANSLIB* instance);
[[nodiscard]] int Initialize(const char* licenseKey,
@@ -286,6 +293,28 @@ namespace ANSCENTER {
[[nodiscard]] int SetPrompt(const char* text);
[[nodiscard]] int SetTracker(int trackerType, int enableTracker);
[[nodiscard]] int SetTrackerParameters(const char* trackerParams);
// Logging — process-wide, thread-safe, no instance required.
// Routes to SPDLogger inside ANSLicensingSystem.dll via dynamically-resolved
// C exports. Silently no-ops if the DLL or symbols are unavailable.
static void LogInfo (const char* src, const char* msg, const char* file = "", int line = 0);
static void LogError(const char* src, const char* msg, const char* file = "", int line = 0);
static void LogFatal(const char* src, const char* msg, const char* file = "", int line = 0);
// DebugView gate. Wraps ANSCENTER_IsDebugViewEnabled() — non-zero when
// logging should be emitted (env var ANSCENTER_DBGVIEW or sentinel file
// C:\ProgramData\ANSCENTER\ansvisdebugview.txt). Cheap on the hot path.
// Used by the ANSLIB_DBG macro below; can also be called directly to
// gate expensive log-prep work.
static int IsDebugViewEnabled();
// DebugView emit. Hands a pre-formatted "[Tag] body\n" line to
// ANSCENTER_DebugLog inside the licensing DLL, which strips noise path
// prefixes (e.g. C:\ProgramData\ANSCENTER\, C:\ProgramData\<profile>\)
// before writing to OutputDebugStringA + stderr. Used by ANSLIB_DBG.
// If the licensing DLL is unavailable, falls back to raw emit (no
// filtering) so messages aren't lost.
static void DebugLog(const char* preformattedLine);
private:
HMODULE dllHandle = nullptr;
bool loaded = false;
@@ -436,4 +465,35 @@ public:
}
};
#endif
// ANSLIB_DBG: DebugView logging macro mirroring ANS_DBG. Both the gate and the
// emit path are dynamic-loaded inside ANSLIB.dll — consumers don't need to
// link or even know about ANSLicensingSystem.dll. Output goes to BOTH
// OutputDebugStringA (DebugView) and stderr, identical format to ANS_DBG:
// "[tag] formatted\n".
//
// Noise path prefixes (e.g. C:\ProgramData\ANSCENTER\, C:\ProgramData\<profile>\)
// are stripped centrally inside ANSCENTER_DebugLog before emit, so log lines
// like "loaded from C:\ProgramData\Jh7O7nUe7vS\Models\Foo" appear as
// "loaded from Models\Foo".
//
// Usage: ANSLIB_DBG("MyModule", "value=%d ptr=%p", val, ptr);
//
// Hot-path cost when gate is OFF: one indirect call. When gate is ON: snprintf
// to a 1024-byte stack buffer + cross-DLL call into ANSCENTER_DebugLog (which
// scrubs prefixes then emits). No heap, no per-call allocation in the macro.
#ifndef ANSCORE_DEBUGVIEW
#define ANSCORE_DEBUGVIEW 1 // 1 = runtime-gated (default), 0 = hard-off (stripped)
#endif
#if ANSCORE_DEBUGVIEW
#define ANSLIB_DBG(tag, fmt, ...) do { \
if (::ANSCENTER::ANSLIB::IsDebugViewEnabled()) { \
char _ansl_dbg_buf[1024]; \
snprintf(_ansl_dbg_buf, sizeof(_ansl_dbg_buf), "[" tag "] " fmt "\n", ##__VA_ARGS__); \
::ANSCENTER::ANSLIB::DebugLog(_ansl_dbg_buf); \
} \
} while(0)
#else
#define ANSLIB_DBG(tag, fmt, ...) ((void)0)
#endif
#endif
+282
View File
@@ -0,0 +1,282 @@
%PDF-1.4
%“Œ‹ž ReportLab Generated PDF document (opensource)
1 0 obj
<<
/F1 2 0 R /F2 3 0 R /F3 4 0 R /F4 5 0 R /F5 6 0 R /F6 10 0 R
/F7 15 0 R /F8 17 0 R
>>
endobj
2 0 obj
<<
/BaseFont /Helvetica /Encoding /WinAnsiEncoding /Name /F1 /Subtype /Type1 /Type /Font
>>
endobj
3 0 obj
<<
/BaseFont /Helvetica-Bold /Encoding /WinAnsiEncoding /Name /F2 /Subtype /Type1 /Type /Font
>>
endobj
4 0 obj
<<
/BaseFont /Courier /Encoding /WinAnsiEncoding /Name /F3 /Subtype /Type1 /Type /Font
>>
endobj
5 0 obj
<<
/BaseFont /Helvetica-Oblique /Encoding /WinAnsiEncoding /Name /F4 /Subtype /Type1 /Type /Font
>>
endobj
6 0 obj
<<
/BaseFont /Symbol /Name /F5 /Subtype /Type1 /Type /Font
>>
endobj
7 0 obj
<<
/Contents 23 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 22 0 R /Resources <<
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
>> /Rotate 0 /Trans <<
>>
/Type /Page
>>
endobj
8 0 obj
<<
/Contents 24 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 22 0 R /Resources <<
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
>> /Rotate 0 /Trans <<
>>
/Type /Page
>>
endobj
9 0 obj
<<
/Contents 25 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 22 0 R /Resources <<
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
>> /Rotate 0 /Trans <<
>>
/Type /Page
>>
endobj
10 0 obj
<<
/BaseFont /Courier-Bold /Encoding /WinAnsiEncoding /Name /F6 /Subtype /Type1 /Type /Font
>>
endobj
11 0 obj
<<
/Contents 26 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 22 0 R /Resources <<
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
>> /Rotate 0 /Trans <<
>>
/Type /Page
>>
endobj
12 0 obj
<<
/Contents 27 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 22 0 R /Resources <<
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
>> /Rotate 0 /Trans <<
>>
/Type /Page
>>
endobj
13 0 obj
<<
/Contents 28 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 22 0 R /Resources <<
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
>> /Rotate 0 /Trans <<
>>
/Type /Page
>>
endobj
14 0 obj
<<
/Contents 29 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 22 0 R /Resources <<
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
>> /Rotate 0 /Trans <<
>>
/Type /Page
>>
endobj
15 0 obj
<<
/BaseFont /Courier-Oblique /Encoding /WinAnsiEncoding /Name /F7 /Subtype /Type1 /Type /Font
>>
endobj
16 0 obj
<<
/Contents 30 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 22 0 R /Resources <<
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
>> /Rotate 0 /Trans <<
>>
/Type /Page
>>
endobj
17 0 obj
<<
/BaseFont /ZapfDingbats /Name /F8 /Subtype /Type1 /Type /Font
>>
endobj
18 0 obj
<<
/Contents 31 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 22 0 R /Resources <<
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
>> /Rotate 0 /Trans <<
>>
/Type /Page
>>
endobj
19 0 obj
<<
/Contents 32 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 22 0 R /Resources <<
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
>> /Rotate 0 /Trans <<
>>
/Type /Page
>>
endobj
20 0 obj
<<
/PageMode /UseNone /Pages 22 0 R /Type /Catalog
>>
endobj
21 0 obj
<<
/Author (ANSCORE) /CreationDate (D:20260505044053+10'00') /Creator (\(unspecified\)) /Keywords () /ModDate (D:20260505044053+10'00') /Producer (ReportLab PDF Library - \(opensource\))
/Subject (\(unspecified\)) /Title (ANSLPR \204 ALPRChecker & Plate Format Developer Guide) /Trapped /False
>>
endobj
22 0 obj
<<
/Count 10 /Kids [ 7 0 R 8 0 R 9 0 R 11 0 R 12 0 R 13 0 R 14 0 R 16 0 R 18 0 R 19 0 R ] /Type /Pages
>>
endobj
23 0 obj
<<
/Filter [ /ASCII85Decode /FlateDecode ] /Length 2672
>>
stream
Gau0E>?BiE&q9"FeC1!C9oL[]')$>UbJ6g?dL?;Hh1s7H3Pnf0G:T!2(Ee)#motqpa)Pq90KksR=I)odWlq;9_?(B8QMM*B!.UTVq2GOnJ_:)Ua)GF0$eM_KJ"9^'8Codo@XA6*a##-7BAku>`ed_BMg-p["!M^Uq$d@?,/kFs@p?AP#-k)Eo"]4XM+;qaQS?#Kr-3Sg!-<bZ974<SZ/@-*iHK7eC!PX+UhcF:p[";fOg+HlAm\s>*K5FqU6'4rB]r7eBf"=^g5WN0bEpd`[^Ob=WA9caps22"/WK]S_Lf*%:ho+ge;SDWji^QG#kY(CS9Vbs67:6@6,73#[E5qir]IfA\@B>pb@0ro(CM#>)5)'!%Bt'p`=0'38Z@79kYR$Kr6*pj%G>Au5IQNaC)V-D@_*J(FoQMmkkt'tJ@t,!5j\Zbe'0%jgBaL#YttNr'scLBb$ZZ>"[We8CDN\[neV=(*3t7V=.DW%Z@&1<,%+Oh`Cj;eRMRErE"M(\#%`rE;8P<(pd2[LGsUa9e^A[)#Ml:5G!<e@*Tnj\^;?=f&S[JJi*#,=is4KD?[mUmCQscb<\,<8(G))K!fb*@Ko)gkKs$MMb+k'\i^!6OQO%d-mZHV2^3"W@:-%3C7+\7$@+P8*JD#fs+C!6X6-LN04,#5PB#Z$K>V)53rEcV'Bf;bUS,pV/Y_[Oao1JgR-3&h\,^*iPMQ:-YNZmUjWLdN@BtT"e^#m=bKX<cO"7to^k]TbN*+p=cgO,gF!7Tm5+X&DC?q@i<PIbnm](ZZCV;]R->N*1AQS>]R$groeZ[Vt:\h<5fMrbs2_^c<D7cg!@;76L.#E+hL)'HS"I\aLK9P/t#N(eC?X%a?5P_2a%15->=Lom:So1-"sgeGQKEV.]tm#AkJG6>jl#auFU/Y>cM$?B!T)$Pj1SB9Q8ip8@!Fn)o9Wm!b3,d:msMT/N%"p?>L2_8P!`IKY(_-nIA@F3pG8^Kio3Adpej)3af;<S;;CCVu/%T+YiS4cC?i5N!&_$6b12c@Zt('g?!$]C=O*,DM`XY!rN/Td:U=-3B4+BRY.$_r(hikfCL0U:u5;YC8Qkt&b-N@jAN8;'ijHH'&rd@qqgR=LW'?qdB8)$Dcc<9/sW+B!==&5[JGj;YrG5sDcN"jckud&e?j5^_/cgDQqRQLc($*8`OXQ7n2RK:[Y5G:6F>[UY+r)rkbT(fMm+#%OYZ)Uf"mEji<%YAM^Z@%5Z#@&#HPUtLVH=lbZSj%E0$Z9G,(UGW@Q)bu`j5.!H$R%!I?,aVs)'Q0DYn8*UF0&_kc8:tOD[og','[tIXX589h)YYLWR?Z6r2>k=#=f>&(f'fK<Y/:EFe]pD8LXcRi=Uf,SX@ji73/W'+ou_[rZgBI.f9KN6V4ZE:$)'56UHCjUol'M.;H7338TjT*Jld2:JW+@b"aY7eKDtNh"P6G!W[8fKn3:Zis*Fjq#L@d!n88TY99qi$>`;%9,?>fqT^DEOdnXc0i!>B=F=Cq`68D#@T^8NG5n^jd51Y(X,UCMXH+FVChc-@K`n</eZ\4F]YMBs($,&.Z1Y6[%bfs1_i8N!b4YSt?55Me/`B7U!bV[H.M+qno=`.;W*ZM#A`oa&OVY6hep!t(b<oC4,1'dupgW#H4hR^kYbl]IOG,nY%6c'&d4q-jo.:mhUfCAq@8)%i7kV)`&ji$YcVDZ>pb-ucE_2Um4RiX]iE+1fr;XI]Sh<lGU*$/+H=mn'=?CS?R[36QBjM"^a,!05C\^c#1[I(b8XO_qSh-mjsNhDV=20R-`eCQF:rDd@^s&0`_YjG$^I/o[Whrm%CK)d"*dCCd3@Me=^nK_EoRM]A70;RilMiYK>S&PS*mC2fm^t)AS5tYLg4p&!J'hHLmJ0MrNgXOn1(?H.t9KpUfOq1PE2(o-_W%=VTpb=eH+u/#5EmE)AejY<d8ohH[88\bDpr>aLD,o16:gqokXG!YQ4Y/b?qX&u4LT&`@=9kSi"I4lX(El=<QBg)T\TEMZ(R5]fJ@c_IT#%/T"*R2<h-PVQ,L0_S'Jb;36EfrqJAY<3-<Y8=\:i"f"<je&$A(7XW3A-J(F]F.=f*h!csu9MI6E.O8qn=&3mNbBq^i??2BL#53&:?Rqi:i$gMYVocouas]__+`ha^_YNa3Xth,@A+5X";_/YeY_0ppAWna:Qen$_fT,CFt/&:fbLP')!>B%$R5jf=tiOk>uYiu[_3dF[YAGR0r0Vb!A;lrc_"F;2E!O(I/@Yb#TTj;f32V;eRYR8G._]7F`<>ec<D[pkaGU=hd'jgC]OcR&]C#XoSU5*+Ai%i"XlWD8d0[scKS4Gnq4LcF^4]Tk;M<OWR]dZl8VTlu@2]V%eoGoclXH,84c3h4B8P)3Kl;iPQ?1,UfS*[br!eZmG=.XFI<VgYD_aL4qQm,T@VC\;`\bP$Agp]>e:7+u[DNkg!7TDnQSqX8A5`oh-U+V<ok4lDJ#$cX<FosJY^+Gb0$6gV"ig9BHEBfAJn]/j^j7ej,pq@ill_CsALcO%HHeRn=m_%/(Md\b:V(E;fFia)\0gfNsoqBLtecHbKFpm0"G8*5<'^/OINMau'\D&?j#h-Vo*`tg.(IjK]31'$33kH#8B%9V&@+iD^Mh*W^ECVZ+f%nl=fFR(tRq5+XX3^IU%PrP8Cg3L!9C:JE1Js63:eh=QF~>endstream
endobj
24 0 obj
<<
/Filter [ /ASCII85Decode /FlateDecode ] /Length 2090
>>
stream
Gb!;dlYkN9&HA?:K'4R0W>.QIfhRk7Bs>&&dPXb3%RXM)MF:Z`A5T2e[,/%eIA_iH*A'p!*nUZ?B)"FDj*/O*C.*bO97NKY+YTbSk`nUoSP837)^keD4XWHgpV-!.q+>_0R/51:aA<s-,KnMC+&"3M^sJ#`;06<LCDKc@CLlZs_A@PBm"=t`Tr$taC6H:p+InVj!RONSI#4OuQ9[rDP&iNV):Q(J,/u%kUq/oD`&#JV#mBlMJ_mG-kQmf7qdfPk9Fhbnn)5drC-tosLSNgVLnF6,]T]R.W60?A0^_='P):!@GuiBWYeR1NYX;$qR)HoX1F$h";=\YIq6*ktk80K:YGk(H1QOT+PlDHO2Z\'UF`l#/hq4VAcutBXR](!^E<MMt7VKe(3J',k+)a2gNVMJ'*Vb=j4`;,U5JuCA4iJ>?g)0<Be:uNSn=KQX(:eh+#L;m_"86;_A`s<YC'4!;2SU,,dfC<3#)Z.>b&%d1Lg$?%[]=XX`<U=bf7n#Mmps/r'uq3'RA2U,07(TbYsYX20?=qKeEZILqRpEHF/Z%K@Hk]KGDn'FSt[+%iN:>n@$=/1-kG]q`53:Sb^'l_,_n?]+%AVQcLuG`Y@)0SW`@&.>/CW/gB7d/!'qJM?%^48)XRa)56-$I@#pF9FAko.h91HCgko2YU)UYXBNcTV`G7C,-Tri]j"/XN8CgbQ(6/?Z2CZiVR_Xn)MMBf\rTt?/QS`S^F5Q*_odoc)kFG+5W68<Tpu):6H%%:]?_]?B7`'KdXn^i[->5MQ=2M]A"920N+IUV_Y^6+mQ\NA$li9[QD&8[10[pQ!F5'FCH,+5IYKp6fD;f_<O42`kJ_Sk<Wa_pu>)=Q)ilDnL,h"M/'914^!F==f2WX,A,kp7ud:2&U+*6_KGjO>8lfCacDeMEpo\Cj/'J8<p%S'pmfRYSh'GiO3as!;u\F&'Hh#<9R;cc$7?eQ.^6.6io^Btj&.AuSkYa5gtYYcYN++)ja2>@\g3_^HgmR7+aq;lpcoDE++Bk=HH&Jb.^EkmT[lS\u[J&6ZPrj3-QVc)k%TSINoXK'&SjjiXie3WPA8KVp:b/$8KDkb:b9qJT2*TBLH2I6/j7;Ju$VpR,Pdr45?G2YP;O>I%1o(7.U:N$4A*64)64<S_ONEj1TT;"o[+mMfg7?=I<7SQ$m-97^fK,UWdj-WT/4Cc0S7o]M(N6]ldn]HXBqV7+M'r>?&XX.K%),=E]^'a-)8)b&Dqt':Gq\JGV><hD&ok<!E0q*mek-@FZQ7^i[;cYB5V:4ji/#fF?ElObQZ%E5?^@7G!VSepf'H?j"1!RRQ4(H:/>3P@Pk:NO3kk!s=,;&:8ZD7[SHNCkn"1p)Ec2*1l0$1D/Z%eF&"&FjJ1!)'Nin:S,!lf+Z&X7n9Aj(">:E6fJZB.&-3R.8iPB2%?_q5XP&,R956^DltS&rZ$M;eUQMi;8LZqG/%WM"l6Y8iP?0(kL)TrG/Zj8$<n?P(X8a`&`,5=a0T9la@#ETAot)2\#@)WKj7Pn\k/N$Cqn'!qdC%U1C;7gGQCgCs<[ae\8U1T6?''c?&/5oCp:F@mTq+"9&Zm-sKNieHOlp&]#Q;^2uKPLa)+XC8:#Cr]1"&7AZk!=T]cTpQ7aeA:5t_F22mI*#iu0F$(#+e1,u,CIkNHp("(\k]e1D`,ZN,NigE6f?PL\oUZ9`3_n23(f16`)*\aH4iuW*4gIOm0`-S2MgOdf'!8<Y(<)=B:&cPH8BRo-Z;d6s1g>1,3$$E8t,I$Z-u0blPhsh<Lj1Ao;%,ha8F.W>q+?MGJbkEB=aA@?\qggGF_tZ1E8D$rB=YrUKV>,U9I'rq))pm'VuA`U?agPTd]i=;sA_NK_0_5,+\Ef8NkYHF5'Adq=gC(UGc#mZ35MmH+,s#4[M3V8V5Trp/U\EA6U;OTh//,p*8X<*@DINN/,U-iY=O3>9i:*ik<EO*[Fra,#bf$8sol<I=1N`;H,*Ti#B=.]S%@5@U61G#@\gh\.9*iAQ5F3#qlfuMZune9.egoB2Vq)`%$s\2shEbpH(WKTH;"7!Sh^/^tM,^oJjPB'M!S+N>0gEfZ"kOQ*h1VqZh0:`3f~>endstream
endobj
25 0 obj
<<
/Filter [ /ASCII85Decode /FlateDecode ] /Length 1874
>>
stream
Gau`T>BALX'Ro4HSEU]$MM!Vpa=a#t\kT)$[[36#9Bb_%[K%%P;+X$?S_`2;8$Y]GMRdgBg1I3#*K-\6s$#1m.fT8;s(DIQhLh""+8d&L#2n+H3qU#QVH<V#o(/');m0@4RPecc:VBf8l/V7%Z.JOcbuPM5p$utACVZ\*[EUT0GEp'D$?`;sTmdo)[9>'\^YJn3i;@lb1FbkP<`ccG5Ad+&h?85`(j9p3J/&LZM(36BkWO\V*'s&J'/+_u[ZQrR\mLaS@]=7.8"C.Cc@okMs+na&jsQ!>34%qe4P7++WL&ESbi0\Ue^U(DcKnSUE^I4$D^sS(YM]@@pE\3DR1bYBm\A"[Xo>5FlstW@\:.iC$X2K+<(KJHWSiZiKu\qhd<kD`gHC&ZqY_;6-5f;5GI1,>;4%iag?Fqm'I>QS77qEBpBTBO"kV/#J)h4+T6r%Sc+NrO%>8iqo9_#!3fV3petra'<.">41D.37c;HOJN2QVpbVY6^<B_:SNX9Klk#h5%;9Sc20GZ!<W2L^nAYL:Yf6JRTr^Mf;L5P7/#u)t3&(4DfL=FKV=Q3k*j.'VCWEp3/R)[FC4r71>&h9n9+M/4e>"GXobBi5K+H*e54CQBHjpGQY$,6egp$rG;H[k^hfZVOa$Zo>EIh7VmCf<9)M\2/3cA-s]MOm0Hc!t7eEQX=^V;:ZcdGU]-Qb1\V-;F<$*m1LWoB[?Wrc<!U60O+g/&Hq[Q9JO8nrPRL_c^JCIUN/+]-8J'j#g<I%,X(UW*.DU8=<mMA`%:?HKOq24fGO>:9LP'F&j#dXNH=,_lCkGL?CtJb(B.,Njoo7&HGbEYVu5*4q+9T:#SPIV0P,ohud1;V8;5Z1Bg.C+k$HLYPJ<>E1"\[=7tK$PWVXI>5=>j#FLg-(m>f6&OV1\O^btsl8A,]+4O%ONu44V,(qYs!a$;!+MnWNYo1?M"S`^b+KQ*u,^auf$*iOn%>1'8ZZrK,$Y.7^g0>\UacopMG/lLS"UF&(%=]T=%dL[9E-[_N$d$,$l_1^AG9N2hK?L:XV0``Rba9to?kJM!nJ4LU(n-,erN]3<)@^c7os#Jd[ADY5FmFXOOg!!d1D!\iY4LUoc+h[UGMFYB%@tIUWlFXI\QZajc\uS3_dlEW`cKHagj[s^Es9"RHt&"`J?FSbWaAL<EjbGm.?k0.#'7d&gHXDMY<Cal_Sr>rKPun6:/,?odc<PSYPbZjEGF4]\5kmGq/`iGNVVD\C9X=$DO=V&M$3m.G@:;iDmVGHVZl5lP4'Q?1U<JfLJT:`QOI<#HUgJt?UG)t@m7";>=pC`)1_fD8sk?Z["jWio+C3c-(H1crnGNW03XQhd:bK&S5h73oZYRd,g?Z#17U&.=]uW=Lm\I-/]^CiOr3]OOsNaTc9V_lj>RCOC4P."!)t"e@ZlY:7k7Xq?'RFm##"0HZ-q_HJnfIk1,m0g&C(o9JV&Ph"X;4ZPHttl7Lh0n)9`V"+U+<EY<k8MMC!l^<h)S;#'1hh6h.j6P*`2Z)6&U8_$#2tm.X`7"(?\m\e4gT_+s@3qr"bPhHuJ7B".O5T]lhlk@$1.bZ:8fho41Z7UCg?`f"!q4odY&8ehfhQUFs&V`?9:AijLMo7njn6?fl9l@VdG%5^C6r"A%;(KiKul5ccKb?EqckmR'8L6D*V(I0Zr=[.>n8?B"r`Zeubc$/VL,^/"n.56+HecTI38X8;Q1e_QlR>L(q(*#BBY&P(Os,lCTJW0k.ED_[TO4W-/rHT3Qi<s/Fqk>M<&$I4*WT93-pR!Zr0^`<$mg:W)Esh#;,XpS]^Z<#q:Ks>#hR.MfU]ojF4F+HpBe1$8d4-R6c/\5RmMN:d!d?g7(4BO:Js[M$9sTYe7oUo8#OP]Vdf~>endstream
endobj
26 0 obj
<<
/Filter [ /ASCII85Decode /FlateDecode ] /Length 3092
>>
stream
Gb"/(lVlQX(B;ZeEUK_T;@n.I->i2s";c7jUrK+WWUDN->1lMYacU+mqq'`3]6[,9%EV<@iJ%Y*g;eL;p:OE1$%HLZgK=TjT>K8X#$HMV#I5[k0HJ'mo]YV'SB]PX;,U@\!Rn8@b:mI3LVoj^WNp8jW2Kee2F,b$b>:lgk=<%[>`_fk\jR/<a=8EC:]c5">4BpHOh_J6pgHeWZ(:_gGRW'I0RZ;Kp^M_a\58.l(H#*r+mP:1V)2d@#P<Yh-:l2q#R/Iaj:\\LNP/'EEP&N?KHuX3#eV++S6RV$o7b+1&<jXF])bJ9Lced-)dj)G%FoiAiIQ[U@mDNeZ7,s"!W3'5P)@qC>RjFd;(f-3LFu^Ar"Z*5,gO7:rFD;r$,%a1#kI+*kPNe>kh7Z^T_\;\k;oN!$YB`Ldj;0cfI:h!d8$59Eg+LI=@HmYQjF.":8Y_XEJJcjA\G^dUPaAbXM[p!OdLe38`%pe+3+'Bok,<1*oB3;/'Y%eJMWuWgC8b2m>%FB>+JltEInI_oG4"N0uVI#AH9M1k)2-#'$_(HVsM%b1OlccQjKK8cAl*37$o1(r8fCN9@P(OH3/8YS$PaV[i`oAgD&@&RL'DKMP6IeLE3t<[=3HWH-r4hAZ2"R<521]XSX0JnGO/[,<RIOVX=?1L`J(&T+?s`0PWD4/>@:FIS)Xr;n:XSk>Jb_`rMh0IQNO+kKCs6>hProK_0r!'/GT0TK=IGl%)5oR&jfg.J1BX$)?=eQ<W5SloB1k+aHBh`5HgB$JDD5Q(9Nq\AD$$6FDdMVn:o=\e,bAZD!l4VU_Da:=ZCcKJGtm5P>efIQ0QC"A/E45`bSaFi)M2,4VXB01fu4m5XFn)WecFO<@6E6cRVK#.]Y6q&pQ9KQ_l!;Kho2"'g!5M\M0gH_!X(/kH@!`s\Ui::LK7FjnETgbqtDlts5UoR;>B3XD$ZY?$485/]&ab^plP$lj=fAb).l)V@Ugl%YdrcW$!+*fumO>+rET+sCb<KZfWj-41:Qr0gUnc^%6,1,PVe?]L6,c&juYY:;B$(qrAai)Bg'<(06\<9c9%>Bp1&dB/e`_R)VRlmQp:WaI^Ok^+:*>d)#ZZ@J?Oo:@,s]JekiHGh(7a'nA!B*Sb-RX93YHbKASHX3A.1Pk+aAqHP`;BG2:Pu7XM(dteo6`'q^RCKT`(TZa=`'Ia4kFMa[+catqNk_sajNdd`L`>5G(U^LiCN%h"Rtot#nT[$=GCB;VqrkriW`.8_&E`96Ui)a^q"AmZO@H?EoCp,ks,ZS<_lWOO:UN,aP,-feeT5H\PWPJ.jUVND1FJIcaWolfrU53Zf/Kl$SC&*d'VKOI&Us11)<BfNk&,9"]tCoNUN*fN6P!6cV*Fm>lO/oN`(-F<VAIW@9P^t`mP5q@1E9,l+/W,'bIah0nV>">r;rMR[5Ta.O<fTQg@r[Bk,Ai1L[-'"d_0ue8`MZW.22?4kD(e?6t?gf!o\'5TDNHR30pbU5tnM1Y'l/lZ+pjY0,+(+]%n6;r^i/ge1NJ(/kS7'S;okgYESsUdkE:4B:e]P)F-DPXQ7MdknefB=Y[/5b:4+d-t$b=+Y<HN,mVWi;$PnY08`nu]HT!'iG+qWog=h*@d4/rZ@5p).\uAe$e&Dadp>V$;amHqYP/5[V*Fbq7B=Wu$@GU:8+Rb?JVBo,I$IRb(ge1#B3i"JduMjo_4f!s/jXVO8!C'\K2Cg9YpN3[mo5WilCqfmk,[]NH62'XkM@]>&\=bJ?FBO+dM0ScMWUW4Bi$9U=4Pp<kUJW=WM8IMb+N(>YZqol"I_J=Q8'(SBm#eFdFXMpes?\Ss(&Ue%q&FU1-8q-3.V9$V6;@O!.VO)Y)fJ4fUfl'_/mOi=O*:1/Za6N^U0O878CK*'61cu9r12Wr*Bt#BaX?"[8E0rSS4;7lf*S?*RK'86,=nOOOkW[*7q)?1`Y;:6?lUQPuTX+#8Mm(^Um\_*H\.0LD:l3JkbAP+i/[7:NfE)s$r5i#hJDREgH($]@NZ5?1,[F=0Sf?O9Nd[p=j-3)kjC2;F>^`GhW?q\)XZ.MkJ]<o(7Q/KT@iJc/&"*=nk`W?ue?'St:c0_?PTM0R3M`WJ\gtdWG-b["6p0Y+HgtZIUVpfCCOtdBG+*>U[\H*WTHUlf>_l\?*SN;D1"eak*pBZc4ZnKbeIkIe7GuF]!>2EJia.$%BknT8uBM+N`h+6rPuQ%DrYR/b/Op#+Hg*/^(@c?N[fJ6W/%lWdU0_'.[")Cdb%,FU2@G+eqN_aOeM1`6hBT/7-I^<I.A*D*u@%Q[J_??($Kub=.riS=o(XG;E-_5Z8W888;1&1%:ggHEq<3g7cKQ)Sg`;a**g_+uV`,8R$JS#<$W.LfS[U)b9@WUm7Ot-#3Rm1R?<fmh)i8,:5u'+`>`l)Wq"SqDf97c_K<%aL]!E`3Bn_$u&_&c]=5*po>0SHQ<3*lm&PT>l2j3!=pQLZ9!g9Q4#UO%M=nKE.*eTA^cUM"?'-T,lr*'Fr5&A)4-W>FG$iAL#1mL?W=;[2?AEg/0jfUKO+P!*Q+PjA!AZFeQk=T%O*J0&T*@KMI`"OCHX,(/iYIs)]GZ]j,j^tFR26\cYP2)fP1Y'[b:bMX]1^`.YhTD<].563]F4hDGSrXr$o^E'?>uJG6_SE&dOU!3W^O0JRFpT;cc4no_Q;.45`YeBW.g$.S<'cMP&gM38D3^AVu5!C"To]IreFqh3I9m8Z_fl*:jQYgf5$nPMJ0i_"doq?Kk+W$QD^lM_d=Sgp6j;hLoaXK(j.Mma-SgdtH,_%EU9O$AW6DDQrAbHB5FeS\I+/+hPu`WT>4ebK:m'oHO>(<P;R7IHtLJf=6S-nd:5n,'`Za&cqIF5uGA\2-+=HU"074%HNd^WK]1:B"7*3<">-U]#gEdKdAM.2+.%'!AgJl!fFOH!mP,E0M\sc8ad4qKL4AEm'a[WbJJJ-InlX]hX3.K7gY=\5U^CF6O^Ip\F3h@_sD%&+97I,TRDX5>[7dFI\NU_(=5](ML)_7Bs#fLgJ^.Hqe?\2L"%T0Elb7\'NS@jeCc<baL][U_b,pdV;so[R,#W%]QbmG<D29d@\lWpP)o34"HNHdW\h,R~>endstream
endobj
27 0 obj
<<
/Filter [ /ASCII85Decode /FlateDecode ] /Length 2626
>>
stream
Gb!;%CN&7CnU#`(TKlBC_i!<Vn%'pQ\]qs;6'=lgrFDR/,hghO/EoP_=gtp@hB=>qU28#5itQ0cR-&0r3)=S7!Y[eap^S)?T>to"AV"F1GRBaP\1kXO0B9B'cE(cW2`*WYn9bKmh\c.&L&5iC0O;_>!\eG'BNmY;Sj2`3!/fH4MYm;P\:JSN[cp*8G+kQ_I?-)XVcr)kN@"#4P)?Q=+O&[f$Tq?cdbJ91j#Jf"+99"[TQ`tjKGKZ5ErHQg^Vfh0#Pos"epo@01kqlO'`:gs01uT6U`a);MjTcL\.b!7g4o&!K4>Hu-"I4Pi:cGN2u#[Hpd&<&#5h1'\gb'2Lo)0YJV:3R\&!8Fq"MsRPYPpYdu`u6"DX8gTY\#u%TpSHH;]q,]dH^ZicNaan\6N+7@#`soE;2c.o2pO@f&nXTkR^A/CQF!s'?AkHUQQ,Bpb_*"\[<bnnu3YpFsYWCA&)2:PJ[cPnV/NZsUC^(OS<C&bnF!^uJ%X:Rs%ANtmo,EXNUhnc/>>_]P>u&</oY!oHFZ^qWScmXIDf:\%b;K#QHjZ'W6nD>su(?]UR`!6=2Og*B^:Jl!\(F'<E>ol3o#]Xcni4K37.?M,!nHPf;AML!`No?P.S7)J+e:X(DGAeVgDY5k"ck\=;;1Rl1\l!IZKi<=sM42-\<gK$LblN2Ro2Oo(\GqAk#KS_^XM&Ea_J&+UtNGas^SPEm$IY;a%hOJp)P-S_(0He0X*YSm1/1J0L/H#T!X)fRp1dkOd@P==1rLrl70`sno#^9d7+:10L&^=]eO9K4JA.)Y$CI7A(j<LmS[S.k^.2?G@H&NF)#<:(H=e^uVLI?eD?/>4$0IgL+(o1;W(<BCpj.%H99&X%MOrWj9k,obrMN'?oS7dc^_#QrfB<kuCOCu):]Jk#&n!a:?j;L>7Eqg``gRV*oZ=M`+2Jj=!NYQ4gj0V"j"m=R7QQm.I]/gpt,04a/9nEQbO6b/1\Uf*@i5Uk&OjI,Z6G/4b*UV':3u^0;TGpB2I=<j=2NEDbE]B;4MBD#mkml;TIEC^7S;,p7+[mcdVtWdK8_O-`kf*5NIC4IN'5)e["o,#JQ3oF*.^%:=<HEVlFgF,,EUF.@RV"T,P_=*p+JL0^:Z]L[_)rkeeOCUdmlr5A/L'g6Snd(=7h4>3eKc"6dRhtl>8F8dMC>[I_D2!q:,iJC[MG"pS)WY4#\kB'$[l_il2k)KjluuMk\B<*erfr%=/c=Tl4*%/-V:MeSI].p[$9_!+1:OS0;CUGi:?m@mVPaQ6,/B-,s;[[j"aAO.<'brl*6NdU!k/^e7Lt3F2s>JIAIAhBI<7+DQb\!r;89"bA<m,Gb2ib6c0&O[l7Z93_:47>P`qbZ<<'&5SjH^1>2`E,;H&+[?cUVjU9n\VWCTML!r6_(A!&3H$>qHQ=tRbG5>#NEIPrJ>nk@[I#"p4B;Mbf>44*b4aj.@a._YO1?$DmRA(MFC"J4AH?".KUN8WLKfEOK8PW0rQ&b>\^;pt0Z4k'<K,OFkk&M2_C?a&'OqMV^?V.\FXZn1icB'eV%Q(eMAc=r9I!_/^_.oj;"Mk6[m0\8k*7C'KJOr=]#*,I(hMS[iQVG5u1GpZu6mlsm-Y4T^Q`B&bE>u/mX(-Mjf6`s``P/KedYj@M;Og=N8.nF0L\_'aT(F(u,%u`iL)c[Z_U#9&FlV+p(I8qRl?5K-7fPt+g<dn?Z_jROq#!.6`<T_Ip>;Wem<u*kU6f("oX?Zsa(iNHag^<M/)mrHn-,=n3@SI4?UjOR[?TX`qdF-L-[]W3-J\Tr^GP\K@/GTDaT`D`]6XL\ZOQ:<hubG1bYT-A8t[#U=-<S.\L89S74g/B;&``p7($>%mti$:!TQ3%a5M;PLbd-$^Qj=A"8m&fJ"G,rhfsX,<$2VV?P&S!j=_p<<_t>G[SHdQcs=<7L4nH0J\T%_K95gqec=S\F8eEX1P*SsS#d<7OG%0h#=3m^Gcm4*]`hgaqSpAO78rcE:iT66r%fj=JS:qr)&^7,)V$NH2>p'#b0nO?]9"lkfc-E0"HgVG4FKPA$7D<<792sM,`3HE'(lF[8WMDKb+PC`;FCh&m""/RmHi=MTsp=@Q,seV220jD$_buar)D&HM_(TQiK?sX1;On>4>DqQNhl09q9c^nG>kL+-F.II*`P)'E5TE,M<3C72.j7.SV*%9NSsk7\S@J=FdE;XGRWM<5?tD&"K=Tk/qWA?]=0t73J/+a4s!<Ol_m>WSm_hUi$crZCA$Q3i-q2><=`olE3/(pTN&Fp]e)WVX#O-cAa(c+kS99H]<tErJ8%M]\F6^W`MQQ9p0`+J`7pDJ%We9A!PH/K:P!W,7E([)YT:TJ;oA:TQb=61GcsK"j/Jd3#N2dtW1MJ=r>[j"00]a]JOH-,am&kYHXelRGV;SV=f/K4S\>/jZKgN(&\&>!W`-h._!Wa_>2D0!>Ol$<XtCOF1*ZTUk.D;H:(>h@Y53#>\0H][W[h!u+O_b[g8iluEo`LJ`+_V\MI1(S\),dhl)I;)b^4,ALKCeWh5jI5k$>Z@]cPj5RVqTH;Ocfm]B0UiW[UT5ZIVrAD.^;L#dfFr4lL)ghKm&i#d\<hXMC(;(WG^OZTaI\RmL;Q3aM]eB4Z3(cMe2#e$E0~>endstream
endobj
28 0 obj
<<
/Filter [ /ASCII85Decode /FlateDecode ] /Length 2400
>>
stream
GauI8D0+E#&:hOm=59^>FFp5G,Y0teM&kSql6Zufa`Qg.kE$IQ>3O&`ad@J>Dss8CKHp[+4OZ"Cj^*-!<dVI)SdhFSb=LO(3?dQqo'3nS%$QXJ`3;+gN3C?BI.mN_a@Gq.>0h;;%WS#Qeor)9k@(isgQl9Y6o1rrBfJNH3ISqi#nP<j2%Q0mf?m8jmuabe'+SRL$OTn;I*BAW%C+-)Lc/(m/><S/J_HcXRdB\m7k`'9S)i:_6Oq0ppWLhK1e>l%DB"E@qO(X31Q93'()bFqJ6<Z7\Wo"%(M*FS)?6GLIE'l.UuD#(h<hhP805cm&qNjH3c#tO@=7fYV]*bFBpNbeqVR!9Xi?,F9d7-E7o(G(['HYN4&].$<`J$];I)cB<Y+/`]:;1BNL9G?r`W+\Xs<aW"GqZpZssM?gWO97WpB"UK[mGa;%ee>G&/J6C<,qrJdSd9e!B>AZ1j]gEO@a@]96+]JNP\*`W87LD[,(99gg:tYK!U.b&:di'9os]eQVCL\4fM'VmYNQW=#IBEbDE!6^N%64Vkup&)]jM-gWReH2Og_(K?,G,!PeD!pp<!82u!R_s[o-Ao6j]-g5GZd8k:KGJ9ggS,+0#WD1>,lH1J@in"sKG.a<N7AA<,KZ62Yn0HksQbr?S<q@W'/A^qQJfQt-,!Ad79-/[-;pKhWF--?Ns8P+.S:PPd0M/j1le%SOe@Y.1dH)6XgguU&kF3HeIR`CASLE[sLinoQ[U;VV:eXN;2@(b#oQ#F9?a$p(<Roja%9Ll>PNjfMD=gu.Su!1P^]%*eJ5]>/Cc'k2%XZZ5<]Wpf^(e8nP%(8'h0NcUH0=A%Vn8C8>)nH8@i(ri4MYu0-rfaY"5hljJm"?EVO7qnF14>59Y4\!CA!+Cj]:JRU6bT\omK.*mHNl"%RrW0q_mYLT-X<$$aRPrQqpna1QjXi3Lp8?J4D>,&pOPnb'RX,,QU8^Q8?gL?rl@@:D]dR?ohHF*@hI!ktX<@C^s:aH?f(Po]R0]3eK3u'Z13IJm@]o^LgL#5)*'ak!<$*bl.pQ3cLN7<<ONT*A?#Z,j4m*C0:2W[!0>q%sWI]L7]Z;"G0(!$s?kb,=!0YIQToV=V*uOOPhSVQV2jrGV#G@Jb"$H>aRb`QOi+*E53LWSLq:`?<"gGei[HmH0-/G;La%A1V-D%#.tu_6:(1>Hu__Js7L.Wff\o4PQ#@k0juWU^AV(GncmPrj7"O-\4\sWlcj6Vc=,#nlCMu;pcXCg5&E$?7`I]=GKKqm@]1Eue2gnJ]UINI^\:;WVHW(p#tO#KP4@k.0$$.@JCo%+L?ILeO+.QaqnIT.O1O7`M_I5KQ3eAd,M]sT9269IO,9k)r.Icf`_\SL2XV=ieRcLq5N9hEqK(:ZCAm<t>?(+#-*^Uu$!s7dRk#SGL(XOPO!]L7ESW*_5MiCFFcYdc5'Xo3"Aanlj4ADo@85eW&hsnPb?9Yi.W+28`JmQd0UV5[7bZhiNkuVh+s+uHTUp%u-Vs8Wd"PX0;ChSmH)+;hK-Xk=!W3Q-2]17&l!qhu9djK.,4loZ5o7W=QeBk89-$=-]dfEC%[(Ho\=8&o"BuY;Ecei"Ihil*@I$2c6gY58g-pcs=g,4:^(o@,Y@8kVg."^VBu`RVnf\HON4kp?KGOboIK'd(8anTjjdmBt5U=h0O$Fu""X;Xu'CO\"J5u9:YS!W?>nf2K<CQ>^0rfF[l&9>\3t@pM=T)S+,f.BD-rh-IWT.tS6K*+^"ZJ"D2AYPXYD&:WdKlB+b.h0IjFEiAO9Y:+,tL?c%tn=qUo"#Y(S2D$b;As&+X=9cT=ce5BV9#^(87IUFM]@dmg\j-Et"2gU9u_o.EpLb@@-BXqU<FP_fmcUB\6<l?]mXV/k9j#W[VPR5XP?-r:["DkhB17R@)adYAUoj=^QrZX7q*-"6]@s$hN<hT&]0KR:^f&)+u1RM^S[M[aSq'7GTLJ5%7q?/c^fF+EQZITo!;c*u$u)HmgS=$O_8?PLF0Kf7)2]B7*Hc$`dBVRAALcDds]fnDT-D`+!dbR2KTnr%Pc#'4hVA$Wn+jr&VW++QB">ij\PlGbS5a\WlU3p%pRO39D7_5].tk:\4"n1SAK\R?[;J"+=ikUD#'@'@npEI,8.!(M&8CNgp)eCDk84&`F6W%@\!'BEqOrUkZeIKMFt_!!pkDVAFhU4CLg?r(I1;Q?%&!JddHkl6i1oXCm]\>9t`-/P3JX8?*q+V=JYnh>_sJbHS9"R6oL$a*N:u+j2Fmau'^Vq^o#=N5O7jR)h!:+Yj2bmILHm+XWcIP3(]PBR'PpGlHgqldbK/]l9[iF+tdW_pSFJ.#9MUMr%_e8%WeIr]Z8d`jrb<cJ+8VFd8=8&aTnWS><Qig4E9aP[V\NddsY;Hph&Oli'N%rW@VR25^~>endstream
endobj
29 0 obj
<<
/Filter [ /ASCII85Decode /FlateDecode ] /Length 2659
>>
stream
Gau0E>>sRl&q9"Fo\5%)0U@*Cc.70]!]/"8A/WJ4_l6V+&rmSe34@F>\87]XIp2D#[M,.sScf\LP.kgV\%=jR"E]#)p$&RZI\"=.B**b'C^J&09UOYOea2MG,Ee+p`gTX3S&3gTRUb(Tk@$;E,UF5O=p(UAmb)4[`0"`%DApG1K16<:&fW[NG&03oA-J_iQ3a-;VqA]kR,G3-Ou3pB_?,5'KHGog0`(mWT;45Ln_L%';-.G?\8>L\WN4Iq:i!M9eg%1,\D&9KPk"j,Fa]T&LojfXJ-<VaVKM![V&H#-[GUQtg_91t`B?U.&T7f.-X?HkibA0Z!*[3c!Z&M&(PPM3[7U,p@V)=%$,Mg%K8CsR@/'t&/6Td:&+Db2`T>6_OeYX@MM/<YhGNPnJnU!-bE\)72g>!4:Y^:dPa;Z\:*>NmP<aEt`?WGJKoj&W57_P8N"T%X2t<B0MC<agMdt]IFVP5SSo^,=nt&d96<LMROfc.EDfpE.s+Ba:E0j]@de?SB[fQ%/V^B1JB=IbRX3VtuPY4B9,ofg4*=!QHoH*V"3<`b?(T/0V0m%oqj+l&]L$Q)/i6lYY4ifuPbGV7YX9/>(!ZnM_;[Vhb=p,[l_l:KL*kS3HFDTW<;1"\d6TIqd^`;)IV7Hi.eAT^bj\H.3ht\,0X:N_M+dspD%+*c<:idYAA/:BqqBU]:iSr[A6h44>.]:X:Sc`'?Q<9d,&P+d0d4BV+]/,^H1u"iZ3<].9F7`4Dl8Y5HS+\#X;gcNQaCRKc/Rgg=K&%hah#Jg`oaH>tDik+3XXp/Wk1n7O'245jJ)q'*O/hiI&ra!I,:k3cG3)ab8.Cq.pD^DNZ,KW,L.oSGr7k)5<kBG!0FosLcu2DF0J`o<As9YD-VZ]%qZL$mhHpN?f!)c$V,^UHBI]GiQNcmmCP^J\dNFGXpaL%=QmWqJGo:n&7uk!8B?AF0=*sKXZBotWh:gs3=e[-7]1FGM-KKWJOY0l)pXtOemUAK%QeG$Dhu75kq-,VhV7a)a"ITHC`XLI")DhNY[c*mCk((Xh!Ri^(Yei.5H;uOqAPfpcbN$ZmP#QQ!pNQQ8EJ.BEM2R(&9ZFMtF[^cE2V`dB6FQbp<ipLE4Cg6u#M#[&47oZ0H>R&,a:9>oHF4f1'cm!KLL#%1$*/2V/*jG+4HQA;Z;HIQNfc>-p6Uc5s6D:`A'p0.G7H<4Z^^V#r#"$WDdI9;C?'*bqNfNO[>r4Q62C]EN=G?o^EAJM]7F=>"[V:"UemZ'@tCe$-*Y$sWpK4XJN%Q^/iJlIM.FNe`u82*0'7p5Hn)'"/o!'h`k>@I#$&n]?4D"-C$IE<bq^W[5Mq_!gubr]O=t9rn)gr%gTopgepse[-<]GJb"siSUfALEF;F7TFG)\\e>nAQiBu9gl0Ld9U04+![M8/kJ]'6,9W<&Y^XMmE2`44+I1Z0]/pIZsAUWGrKG5&hGRuB1":P#D@&$%T(N@>0SEA,NcmOJbV?L+B)>uMk!jKaYHgJ9=!"8(<B;lk#)cMoU,"S?#"S5#S&KI(8=B;j*LuRPV-Y0F%-?r:+q-2'_F19dhYHWuLLG?t[Qm8+RA+]Y,nuFN7F7ETj>\@j`-fU[L37Tm\>F(NQ\lXY<'':EjCe58GN&Nojgt$?*'T2,f3F@E(q_O&FeQt`c8Fb441&WlG(C*Q/X-e@@`rellhiTV3_EK('1_J;fWq+8<S'aT?G'gb:Qq7c%60R\^/dc[>@FUa+b@%0"kZB\9#:lM6>TVTqna42Dnq\hUl?7QTX[!+3<[W*`/jH=SYO.)*]hWKnJ0eE(KN'jbZ/fr`%[L?%P"_V,74SG!g=:MReOuR[XU_]$E8<')U_:XhV8Ef>SG2KJcdWiRW<SngRW-B8C_MVp,,]tj"Y\S_,2'ZcX->?o>mJm/Xn!kX;n33^/W%*e5-:EpEU\F>$-m\IVT![9>b(Q.;7t#L]%j[C=?`sKLp5ib8A!d<;/P5F`*::(s/E30Nh%nJSAiG?"!Jt1/>^b"NHQ?=_T0@.?bBV#q1pUkARJ5C`sVCi)s`/>]j<&5\rDVpK\SDT\E[4-OPgCobmSVa?[_`\a1h:QEk9XF<jkWuePkFK8@ai%OnW3B:(66(a"h)hIE1FQaiCdd!Kp7T8\X5,dH^`f(O&)=BJ4\'&n:0qM\P.^loaRL%o]1=E!Tknn)qmUmfbR/hlj4qe&cJ,)YVd!3pm&&M"h;;=62l"=`kOG<aOr_jeN;fFmJ@pm(N1#%@kSa$qPCm;Z)'-i`#?@G0&Sb-T`iperbcO8,o!p)&>FfGmjfS%(U<sA-i>WE>80hhodH0&(c,&\pg:,9r9:Wn_<_YEWVGC&$u/Tbr&gVH^Id5]q/R00thmppZEC<"'j;H%-j:K9hj-:8Fpdd<Vii8OOisV,\4`<b&>Wf@EMT!Smed8,%4)RJP&KWB?=j)SORV(%]foHU`_.R@fAZMiTYQYMib(4biNnkr"g_CjiIm:5I6HF=7GO2O5C7NTl-bWDHm#bqCCEUT%IlXQCDH#o`$/m?I+]4k,BBSjs+Cb+!EdpU^#mrd1OF`itfS=1@IlSZ;Db4A.NRDX3H-K4]l[4H,N2V-?n&DIa'e:Z9,OAe[]"M*aJX@1O&pndo'5AYdS</AGU7b24\%/T:b/?r8kIHD16lh:`GGo#FA6)Du~>endstream
endobj
30 0 obj
<<
/Filter [ /ASCII85Decode /FlateDecode ] /Length 1317
>>
stream
Gau`Rd;l!?'Rf^Wgt@H7aZ,/'KRdsr))o0!]:5\Nf\7e"F=>3:3uD4aPKhud+M]r;N88Xq-FVl*k?]07F:J>Q/lI)n.tF8SE.1B/:m`ne0Z7@J%3O`-^IG"=Bt;<Z[`*f%*''N&2,Ao<AgT6L5h'm)iIEKIE,%ad5Do&Xe,a0&6n,S&N&!p-/5I!oIT9l3CQ?6m)!'gU-6QA7,1f\k=>e99:,6N&YXOVg2/+^]8X*HK6sm8'KOE%uEseo22IX<t#dfFrr$pbp&tHc8/'Le1^sD<"Jl7dTgEuVkfH6DPiQE%i\<s&R2_0(l%DIaWX;k5mSR7iDj:@PQ+b9*MKE9;M]T#$&/sCD=abA\$E@)imBc,_D\W+)&e?P59l`;S8bA:DFc6aBP@NBa.!;R`[$<?Ml:d0C7R49<lYkFk,BoG)o>G,0sp%!r^bkeZ'J4E%VP%k&IcsKLBIOqItW\Tn%oDoq3jgK4uXF2@N=GEg;$Gij2AKY#hq-93o14^Rl]K[W<(\d,^@Go6]l(udDUN&O8m9p@)?8B/QJV)t]<11F!FgR@fUiBEhj_rRHZ)+Gb=ZP_BK?>dF(LO%c$ra_,,,kA%,2'miI9mcP&+G@]"$=.70"`UT@(KZ7cXj-N@d.a(f0!]fp:X'e-=g.9L"%8Q"+6G$TJ<`1VEV$[Y9Utk9O&kYWc$,3!M"XK%X1u!V)qJ\DQL8OEIJkCX9dcs!:N%iNId'4*C.1'1dgQ:$9:0[B,a[1dKle`.U_ZIbUPk,fDCb5;;Y>QI"'a92T!u)<,!nc27@p`BfAGioj'BH];F*9jh#[:inHLcb?Z[Q64Yfbmr%SdrY]njOFPRLb<Nm]MRuBU$#UXher%[U/]*">s,8&$d?g?Tcd1-:p'Jd\`R]%me+>ii6oLHeLh.Z`XA?q]hLa,2"01s3SDMJef=,sRW,kfX8-:8$1=a]W*(48;70WSo00Lm=b5h)I`kggkJ_rZ-3LgZej$Hu/TdW2i_tEgT^A26ogD^p<]uc%NgYF/5\\-03;L>U[GE]Y(P(cc)7s6cIaM]S#P:2:619pON,tgNaVTrV/6(n*YD2^D>&-_me]CO"SW;(\fUS@EF3r"3[7b7$SpMJs@!S_j0WbN?RB/8#P+kc5,>-luP_`CI`DCj]p%@)F(a#0JpNUa25Zc3AN)PdDXp+LD-Z_:Q=l(qcri5tFH@j$tnS*eA_AGTjGk2a^%$uGLXq_;3:O1.KFqZr,TO*ga(a2`D[(hWpk?TkX2nMPi;X3h>1gb=AmL(eKRibi06\.6hT/5,_(SufZZemU>82^FI92cX+065]Y1X;i"a~>endstream
endobj
31 0 obj
<<
/Filter [ /ASCII85Decode /FlateDecode ] /Length 3004
>>
stream
Gb!l!>BALh(4Q"]30fiH)'eI%JF)]VmFX3dAWi"sAfm>@WCA_kM,8!!,maD6pRINL[C@&8PHCgQ:pI+9S3DAhhn^ADq!lSSGRe[UJBL"b4<Fb@n3SLU_)o.ISGdp!<@^.:a7Y)+LMkU_H;Wb^5+HfO0^sZH3.B6K]a1-'e;O5WW9G2JoJX_m0UUl45'VW=,,,0C^(\>tAmB<e3N+.P+B$9n*&<ZH7&9`##+o>nP$%Mp5Ps(>q_`Q`1S##H8gh>/fB^o$`6>j7+g`<cmfcL._%OhbcJ+bM1"TU);e/:S0e\sRWgtFekE&6&_9g7")91l((!Y<c:Sd"#nE-[8+`R,Bj8I5]a'Z?.L!7)Rp%Z3]@CIlOSk"Bdk4ASM1H]>rk4ImqJE@NqO?s6?&Ai>CpjOjrhi.U>If7]b>8(*L]O&>cR9,7sJ6l/b\]b9UCOT]0cmn/()Tb+.EuMNBX2dH+[gtFlf"9qQ89/R,Tu?=UEbTtGOgSXY+/*D-GE0#UH,u8l5*FX&<%]LKcLm:/[hepU]tHDrp>__P&'suU[sD:c>-C:XE\3tjhY#IJ/]HO<X7;i;DTc5(MD+X'*<1K217%p[>4[HEX<HY,0">(fK48<'c@fZUk/OqcmdL\6J)D2C/=-^_?f-61Gc*bE@W'r;_?8uF*idEO)peKs4r-rFe5aYSQ0.O'bYucGb)CY\;-=@Ar4Q!(gD^$8F[*)7r6ocJ3OZJ>DS*;M'AM#>Oqr,n11j4NBF2D$>*36<MOEV)m1/L=%B'*J,aLNkk,]-s;e^$,ca/#ll4:5W"s9hO?8ImWRS)sD_")`iodaf3O]1TErq>aHc+s$4`Uffilc"e20!],*S8a3`BsUH!r/8(-6ZE@E]OihO[92aj"oj%!2S=L?,W^jc.k71X]2;"-)^A]H7Ps;:GDQ1#WU<+EWfhc&B41B3?pc;R%du6$L_HZYHT&?]^s7`j$.pN?ld\8*prf-]Aj\K6.4mH<B@#YhT(=O,4OO)FfR4Bf9rC2c-%aDH9'fgpp;?hGb!Qa%ehIlu*@a`'5Jhp+SL4Gk)i8,/_5`lhREr[B&e#r797aptUcV,#c9Z-ZYVZs@6LjT"JP.=4r1Kb;*E9Xl]^3tS793u9N@kAmL569enQ4f+$^QT'BIlR#WHQj%g<rV>]$XQ320-c::8jBg*O'(r<n\l]>1&I'\qkDN;[9J@-loeU]k@cK6stkq+J]tn#V+F/6s>aU_]U^iV+;4Q5nAdN&n)t.r`l+Hb&PB<-/YhP/qBd.80u>$cO(me4r9Toa)k5$Le3NMis$,ci?&?i]5T!Q1fl3Gd)[Uk,!(:3^cqt_]uW\6_(Lm8:rR<0]>ikU5%=s7jB,)&W8^]F.,_VpGt>KCApW"AX7m-Xid\T9peN9_Z5pLmLUWqRpM[<1ZFT5mhEAdJ,HV&QjEEZt"=UI;e="`K%n5B8*8bs%d.Til%8:(P;>O7-HG/$q^os&?HOV8eVr,CU_Y#6dA@nfS2<e-^-f]3rG&"ITTHBKtf6?R9^)DQ'ATnA*jLLuEH[bT,g'8%fZmR"sj3Y2F>$("cjft;sIE,MHBmk;K4*Z8JbMuZ]gjP2]\rFI=J2Q%f"O].3OgJs*dKOHkNQ1aVAViZJdD`%\UNS3/*0iM(@Hrm_>@(-oB?0"Vk]\44TN2`^huZ7FBmV$GpOl>t'5fo?(V-(lAId$\E_4*#?dtYO2Plq+2tP6G,ho1##C[9=!$QS*;c"X/]oEJoqddV>5T-SZ>,!L,NuCR!Y=%#.C*30SI8Nq/1/*fSAko7k@'g(%0Yo%u4pu`f5/eWT)6V'h7KFSXM-QdnPq8dg^_A]DiVB'%Hq`=,<^Rio)0a:PX@Vn3]lD<+\,RCTP"\/8?mQ3%A6,(@/Z-lT9+.%2X,T]X?,q)7DANLM/1E9B<lKkDqQO.r<4qWeA)sM7J@-42lUOo:)kOsG9Xf#8Wr0][L"\:<S<5UU&j/tCI!qf+c6@S<pEek/q#BrR'D5Y7O5[Y8Fe]MC.K+A3%W;Wr9rsq)UFNr(n6i-KqLiVn]=!@#,mO]aXKTn,gNiogG\mN>b)(S^1bU&_rd@N0f0d+*mmRJNp;S;OFh(Z+(5HrJ'c]B;_*/9H@]e.[:-KAgpEWQ'8]`aq+&/=J3qq+0Vs!Km\JF:EoF/TFS`=[Ej%%S((Ih3GSY^C[k(gdq-;6RXISKcr&*T^,G5Q"#^OAM0p\aMo/2&O,HBZ*+_h>Q6`:H[n+7AP<(QEXG54L]2o$BO(RQ*ke"eh:"@-PIR#nrF<P_'/EnHr;*%f@lT(q(%(8>5h_bf\O3Y#[[)8::WD5lZ*-oSoSn-M7o<*u/V\)7A-F"F9%N&u,Lo?O(6M&L(kPP:&uJ6)!f4X.<rSrej\i'K#;6AuPfp)t@B0*jIF#ag%^:M7IQtKAnbqH;*tYnI"B>E44-moEt:R=E`=YVuWh&0\f/1-0_fqr2)AYf4p?r]C.R8asbjo%3Lk4.aK5Z=o$%+$N(hhf]>EL._R7r;iiHc!d^`u=DnUPkY&1D>.4r$R.IjfX5Wk/^%oA1Y.&$bEJ!*7.p:9,CMpJRW-_9Hq=/Q3og*]F7L9tnb"FI;5\F*$dPV7q=5'C)7Ho'\9k=Dr(MY*/KS%RoA1B9N<A&SlnVtg.M9&a7\7\qTE>+XOhTho4\QBuCU@o"CaDragQQi%ZTu[>^CUrgjV7V\AX]uURl7UY\ag[P$=NT&C6@:qu[+m2p:g2EU;'f26$FdsPgo4)[gUa_,eXbN01D*e1LZ,LiFMiga$]SlJm/$JLm(m]W'BATeC=b95]Cu)k#3nHKV[44W1Og'jW+i`'H7dNH-4-.cMP\(Z6$pr!kgS=o"VGpO't6&iTRGXF`UHXp8cp]*QZ(T^D(Nl'7!>g1DMF*mSKLgeI.1B4lJsNViZ6"mCD8pT?m(;(9GG`qF.`he>p*J&\7oLk_BagTjlPCK<';kL1WS^K_\W+5P)0XRgMi85oa,bTF?9g,CF5,`oIep0g9#Z[p;,G^!*'+A:B~>endstream
endobj
32 0 obj
<<
/Filter [ /ASCII85Decode /FlateDecode ] /Length 1756
>>
stream
Gau0DD/\/e&H88.ETi2.Bk'T]S#$9+O?,tBU1i#ol^)l&=IHVI;D4=!]%O=f^*>)KP3^2LJ_Go+nP4<&g`):t@,dNOh_5(U21a4l!lT]4*/J)a0R4%-PG><OUuHJdL2#>4"XD>PI*X&4*g-u]Z&(UF_m0B&G[S%Qmo^k`(.Td:1hpio,cJQ]@Oi1b<%tP8!617E<^=L<3+q>-9KpATL"-&%^FfD8iE@JX"4oI*e">WHQ>M1/ndN\s>o6`TWe'sR'IpjTO6O,un-.'H"B(Hn7\D%GJWHh$qCOJng60]tP)6f9B:!.U&';aO^h2KW.G/\OOP=bqo26gB9!1D_hYJt5^d:mJRdF1bjA&@bi,qNbUWGiSOYLn4i#p-O@,LRW&5][999t<K6]J'O7o=+/`&G)uL>#Km>ck!cKu#()dj5flA*3ABAM>,=F9V-R\*t$m=clM]o9]ndS3WOnAdlCb,#/s'j5'2r?tl[QW"!<t2WFA_#%'",Tf*nqmkPKK$CG!6m$Y;f"=iGhAf?^H.2I*1+7*i^\4Z!5c9N(#r.:J)_a=s!?fHICMHDq#),b'ni8(*_W5?nHXt7)-GB`TNk4\e7$$dO]9Nn1XRS=%7$'63P,AQE+W:*cl#U9V]AL`QB^ebhl(FP^:[LHiNmYP@h_&R6u$Ca+?#(Y0$GXHP@SmhG\%)]1\Ybr_@Wi*c6(c.<&A=04k!V22i+bQ:`dF7sD\9I/k,J;VHPf!B8-9pOOZe;\akaDao6tf4E*!hn-a:dE6\en&S6;@]MFh/Nun+ZRAVdXIh#\)s(r`OAbei9F2Oe7*CGeLP&ougoOWa`0CgGd:8)&7.W!lO(P(k:3*P$783g#'o'B6F'W,SptFr!d<VZ<RA>-9^:,Q[X7uYL%3p3c+][)oE[dA5I+C9qWcj!TjP[fKL>5kmtnp%FFGpO^;[:c%K:jB35s=JUrSS?jG<dh3Pk.-*To#?U'-[H@4<=]>`OMP$\p@i6u_\eIQTfn+2\#"XXOp#T_lGkhqgRIn^Gk?Q\5p]kSnBA3DQe7h0a#A@;eVBA%)U[TFR_,U_;(#p3rG**ZVmq:Lea0$.smln6CtO[tg,eg)4o>82Ib$!rDFXU/:j1l-+0dIEQZRthpPLWsY::F`NXk=K*cP6s&'18]sd.n(Fe-XSEHl!CYkb6>?O<F:=,-95ta,bIUP@.Pl>`Pm4-O58Wm)S7",b$!Oi?hlUiC?g9@6E+h\n\MBoK03<ZIppc8cMt"f*"KXZcsM:iOFO1mN3aI#S=ACF_s#J>DK&p#4DI_nj%lt^I&^mlA(qma5M<ti1u$p+C4n].gI=bC)Jl$W[o[LN`fn]2)TL'pgZHHYZ#&TKLH$;RkE(#K?EmkH7`CgJ$XpDD9b=L.'<U%9Z<aHZ[<:b8)i.pJp_j$LQ)T/5N_=ooKMjAM?+mS`,E9?egio"s+j*iId.F"M8O5bGHOth>`'X@F6FCgr\.IZF>YoR4`9C/R""k^R(K&0'lmr<4"<@7_S-M(>OTrp1>Fki&p4Ui#UQ``pO;^QRJO#>a&muFt8TGOsbSFpjO#ALAZIq!$Z[0o0[En5JrD/sN%!^B=<0)?Q,@7gX];JSEodXp58VH4XQLRY;4QkoBND`gf>ho5X["N7u(Y:`**f@6+S*`jN0KCc;#P0XDklUgB_,_1O+ckmOJLjRX&Cn9Q6'#tcSn^^E`V;AQ$1n)0'A,pka6BX5l'HMlrc4@0n]d[1dSBl/GEZUXGs^OjdPjMt8,i[4M:L#~>endstream
endobj
xref
0 33
0000000000 65535 f
0000000061 00000 n
0000000168 00000 n
0000000275 00000 n
0000000387 00000 n
0000000492 00000 n
0000000607 00000 n
0000000684 00000 n
0000000889 00000 n
0000001094 00000 n
0000001299 00000 n
0000001410 00000 n
0000001616 00000 n
0000001822 00000 n
0000002028 00000 n
0000002234 00000 n
0000002348 00000 n
0000002554 00000 n
0000002638 00000 n
0000002844 00000 n
0000003050 00000 n
0000003120 00000 n
0000003436 00000 n
0000003558 00000 n
0000006322 00000 n
0000008504 00000 n
0000010470 00000 n
0000013654 00000 n
0000016372 00000 n
0000018864 00000 n
0000021615 00000 n
0000023024 00000 n
0000026120 00000 n
trailer
<<
/ID
[<704d8733fff16b81e99fe2145e253d08><704d8733fff16b81e99fe2145e253d08>]
% ReportLab generated PDF document -- digest (opensource)
/Info 21 0 R
/Root 20 0 R
/Size 33
>>
startxref
27968
%%EOF
+149
View File
@@ -0,0 +1,149 @@
%PDF-1.4
%“Œ‹ž ReportLab Generated PDF document (opensource)
1 0 obj
<<
/F1 2 0 R /F2 3 0 R /F3 4 0 R /F4 5 0 R /F5 8 0 R
>>
endobj
2 0 obj
<<
/BaseFont /Helvetica /Encoding /WinAnsiEncoding /Name /F1 /Subtype /Type1 /Type /Font
>>
endobj
3 0 obj
<<
/BaseFont /Helvetica-Bold /Encoding /WinAnsiEncoding /Name /F2 /Subtype /Type1 /Type /Font
>>
endobj
4 0 obj
<<
/BaseFont /Courier /Encoding /WinAnsiEncoding /Name /F3 /Subtype /Type1 /Type /Font
>>
endobj
5 0 obj
<<
/BaseFont /Symbol /Name /F4 /Subtype /Type1 /Type /Font
>>
endobj
6 0 obj
<<
/Contents 14 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 13 0 R /Resources <<
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
>> /Rotate 0 /Trans <<
>>
/Type /Page
>>
endobj
7 0 obj
<<
/Contents 15 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 13 0 R /Resources <<
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
>> /Rotate 0 /Trans <<
>>
/Type /Page
>>
endobj
8 0 obj
<<
/BaseFont /Helvetica-Oblique /Encoding /WinAnsiEncoding /Name /F5 /Subtype /Type1 /Type /Font
>>
endobj
9 0 obj
<<
/Contents 16 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 13 0 R /Resources <<
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
>> /Rotate 0 /Trans <<
>>
/Type /Page
>>
endobj
10 0 obj
<<
/Contents 17 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 13 0 R /Resources <<
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
>> /Rotate 0 /Trans <<
>>
/Type /Page
>>
endobj
11 0 obj
<<
/PageMode /UseNone /Pages 13 0 R /Type /Catalog
>>
endobj
12 0 obj
<<
/Author (ANSCORE) /CreationDate (D:20260504092543+10'00') /Creator (\(unspecified\)) /Keywords () /ModDate (D:20260504092543+10'00') /Producer (ReportLab PDF Library - \(opensource\))
/Subject (\(unspecified\)) /Title (ANSODEngine \204 Tracker & Stabilization Developer Guide) /Trapped /False
>>
endobj
13 0 obj
<<
/Count 4 /Kids [ 6 0 R 7 0 R 9 0 R 10 0 R ] /Type /Pages
>>
endobj
14 0 obj
<<
/Filter [ /ASCII85Decode /FlateDecode ] /Length 2736
>>
stream
Gb!#^D0+_q%fW&,JPS5Y-II_#MsYW_+\eKPp,:#NENdt3cZBkZMRA]q=/%Y:rq_526qOiDU2E".:D?_M5nF%4#@IfSp`]4=g&QN8JG1u*#_dsC`6p5>I"_=TjGst;pTu<Z5u!-O`%ptait*2cA/n,S0C$>^"-e\Zmr'V9WF2VCQ:]:,kgZ,"idbo.;hn*jGm<[,md^:&*^'`?K-eL'Mo8p"c4]mg4EC0F#_P?I*lJXqZS*kg[k)e0MpJX&#.,IqJ]EW`MY\!8R`%(bAoS,@-:2$JB(?S%8TC_^5EhkW6JU>JgP+tqjKrBME^Y$b3@3jb6.T%?;J2/@6X@;QK2q-4BbR8Bq;#NZE,d%HMQR%@?MZiinJ"eiaWBlRJRo`oG>E6#V6d"ciF29BrQ"dk&2W+trDtkpQi@WnYdhG3"3+F=nm0"VSQVrKjpk-ds6[NspEWZS7_4[14jjohdYNp4i4HU_6cr][_"B_aasQga#\!(G"6;=RS+dOc[h&C&]^TJ;6e[#B'qB'k@nH0DH/L(?VIIV_N7MCOS-"-N"mMJP`?8*A5"#D^>]fgs;=BRXJ&EYjMT*?5pC.IW'8Ko.R;1bg!kq_'_GNk/:9!gVU0YAjS5*l*lrn5h0Qr-0VGj]GN_ZQ^_`>+I@GlWf4N/]:i`MHu-I'0d1Vq-8I?sEPB2b>A.L'=rL-^]8*pVg(,4mk?@Dp7*YE!;cbR+I>L:M\uZpuA#)#DI\8+Fkf^6LW'7%_jE)8l>+hBaZVZ!S@D@$\\$C\fcVC`Z$Hmiji_W2="*#jY.Nqd1.\fg6MRq<IY\hCEf^^O!PB=EHqOV+,7S,spWin*g*13+c`2=0X"SGK8b8PoGJIaEYGSI'jKO=8tGdT'TRi[O`+/VhVq>DD*?#1EaWOLH38m`&\!83V[QPY"/uPl\2L6X\8&,*N'm/kVF8)ilrprD_IBSVm,$U(hd%jUe$Y=$EW4mS")pJr=FmL:=V7,m,k2&A:s[.(OsUd:[![eMR<cgQ<"_UO5-WE"8d<#]4&jrKdb6jb9WcIW#tR(o82nZ$o5^[2^&jT_RLk_L'%rNS04JN.>$ER$r>MBIusl/qVJ0a2TgK6*mJLT@TEce.L6js.g0[""Cg,RJHS?j:^j>^A9o#%e]>is[2lQ]l#_OZgR&HRA[EPDh8Pn\qB8N^+qFHan/UEdX;m&%#8\CEG_b=(N3SIWqZ',T*c@%*22e-i,(M`uNgB,VA;IkeS-5G0[nW5+X>i<gJVRJ,=hn2s");kC($LW\C4ACcs2,![!+%CrQCeDEPTaIbliPsmeZ!Zc^gZ=o;9U3:8HC'^%:]eorA_leZreI.1`>:&&Qfj*V`K4Z4b^]Z;T?6QL(_W6d2+_JgkZ&Nff"U@+c^6"W>D!T:baEa4(OV52f91Hr5HqY[$SA9r:Jt-(p)8)>:M^5oJ&sAC.IJ[YkUu3pk$+XO><N;V.huGUN06eOW8j-g[\X*4U^>)$6P;@.'5fh4S9Vc(R[.tW#k;Zn\-d$C'8Y^"aW>pBb^tQj8T<8;Y#.dFbMMX^l^XHG&]-0p,_NsE`\^]N[h.o1SP0.V$E!Lmc`S.;rS,;3iQg3dL;nXpWIkI"K(/rg^+G^HpK)Qo?EY4UC)D/q^jltDB/p3Ao%aNZrU[5@7u?3b2c3147s79-\)O5h+$a6F;#_L7B<g84V'7`ENH%Ug$W%<_Z"eS"d-j6$m-&lpTR;Y^C+%\e#q_%SW5Q(L5CB;Z#_6"*:aC'LkS*q:+Z;MP=sgmaQ?Ne7_Y&1V%a#jhCesO/d#sST`UCCRNH&!k]LR;oGMC#PmSn05#]Kg#]bqG6U2\iXb_0NCXj+S\rku\Wl!NoIV3IIOd#(nL[Ht"&LjQhbFbZ&BdU)Hr2I+)oIAY0B<]C/ktDDg$.<pOM\T-3m()ZKXWrkC\g#m[^fR8M^>C)Jl%hfmWW.eTW4]7YAGt2#`0dHq?Q#@db6/C:=Jsn5h:7A@?8FPJouoBJl792%9b-rZKK*Po&IU.M*BZ1P.@rE6EZbuF9\A?kW69^>FeGEDr8]3mL5Ck&WI7g=P`!EL/(M.!g[n4M.s$\"&agj2hdYTQN^HV'fAZ^5n7$d'.;g$`j(FM80b""o>nP1?+i3%>JMeAV1u3t^e145e-JXmN7T:%Bb7t0lTs2C[;1'F'Q0A?Wob-?qhf#gUn628O@k0Dgd2Z+[.m#YUe#J?p)I+GBVBS%.;51$\BVR$`Rg!M0/'q"mY?\UgAZ)Drq26f['oeAo;EM4pTW:OJDlrDhft]qlJGVH-LBlt9CPpJtG`E"JQJLo(m9@0#GW;Rb_\uME+LM0SM&;3uL=ap$Np?qEM`mr#6B09:7rM[bN6&*d`ImklJXY"9U!C/>RVM6W;3./,FXE;L;uO"FAY,!8.rI:nQEY%FLc-00#Mh)>p"8It1J2M&_HglULO2r&IrJbp<GrJ]Fh!>QYE@#sZ?i(U!Uf=Xp.AsBR1If8h,LrLWG>b*$9KgrQ!ZId=N'mMJ*p,o4;/N'o@lq.=.P?4(#.,97\uhaa4+-GVJa,k*=d&gVPW.$\M=j>JPU0,@0kUqbJGB2(R_8@c-.UnhKt-\9gA,8W.<>$q_VT0i]@Qa+ZgJhI[ob5MdZkRkh7ZLC/k@'bW+b&D"-_)PP0'^'^,BkZ(ZVHZq]0hgcTF&p2FtAlJRa4"c&lY***p``)%`t<^4g7CM;!Gi?YO8cpFng/2O5t(>hb&.%+[*OW^i@O;`q-PJ7+Xl!k;~>endstream
endobj
15 0 obj
<<
/Filter [ /ASCII85Decode /FlateDecode ] /Length 924
>>
stream
Gatm:9lo#B&A@Zcp./)U8?sf$a!OO0l.7c.4`lF:j+uVm=MtuOl*'R0?t2.8&>l4R61!n(^>5[t"r%:m^Z.>i70r)F)))QZ'UU!E_U!09%\<l@j6:lCE^t0>Uec@e)6Q=_Tu(5u#D>/>B+g"NIE=Xn*X7GBl7`^$Et#m93FrJ!";hII+3Y(n!OK"lUi6fUBjicU,HXrMR>R2^asSPa&3`F*>,67mUQJSB>jf-8o:q);PnrZe%`4E_F6^U[(DmFTV[-Ibb");&+1gtc5lp'I$.anXn1g0+=,G4&D.MMqEVNt`Ja?Ri!g_gO>_i(5DNk`t?/mSL!R)5CRQ$RT1qnMp]Jbo/fhtQc,.PgW66<2*V+P7j%lu:D+[R]#N)ATdeh6k9pYe1kNL%/;W6M(3-Yh/,qKW4CJ_W_=M,+%<2DJp,A@ML1`$at6!seb`B:ZqA@qSC05iCZX\"HH;%sFgXDXNc:]6=U"^";ib>:e=nKhgXG*l)VgXBjW$b&0O@L)3(kX75iMjei^CCoR8+!FY8=DK"724+J<IlX6h3b[$E9UGo,O.@'8>/X6_P@O"8^JNGjm5[?fOCX,lth.`-UWDm?m/V-^t<s;usfGVW+/i-"3ALa;iI@os]$m(g".9$K9m!;0-YTTn1\fWT<^-u0]=&ro.AGDPdmPu(ODee*T4irU7JGQ+JkAnc5YCIjY<^rGDVif8),n-=`Lp`,GXY()HF55cpI4bIRarm9AFjSh)Kdr?b>g_9d=HD'O*-&>JlJd3+r,Pr2#GjOjQ[%d>,u[>cXJ-[qF8Uid`rV;'AU)6.O*OX]/QmI"$+JLgHk!4CB>]+D>8C-Hj3g>b#28pC2B3Yg4+N3$8(3.?g*GB!d4k+!M/Q,0WF<VGA\cna_go)9i3ZTI<GjAWf3KqJY:!?<2s#S+?Z?6jdJ~>endstream
endobj
16 0 obj
<<
/Filter [ /ASCII85Decode /FlateDecode ] /Length 3062
>>
stream
GauHNlYkN;'*"Q<@^$piC6L%korrPJm%J9Z4kqr4g>i`GX:L%pe$`F_nsXUSm"bPoDb.QY]rYHT5n_A:Aqp3mk=H!`J#N3*>lhTdJ;Y?RPek6WW2Z9Un/e_S:4>F$ktGlg'5#l*7^f'R[S?9T*7fs\:h'Vh1L*Cdcn4"Wfa'Ko';S3@2?l:FO9G`$-l07%TNO>2a53R5pj[On+,-rm2Vs>3Hm;t6_\[QB/VWtQ5BVRFDFop>2r0Yp,d6rW@BTN;Y/Udj#25R>3d?B-`':j[%ma15n-'<dTObRB0s]E=HQH8L*G[e0*WnSfK;C6\B_-W?s2!#hjaFc6L#?G<.]&8cTRjRTCf``X`S[-9Lm-N5ZGF!*I[_A;HQsKROf^E//3%8hCcKXG,>Iut:9doH$;%/ZXs@P.mO%RJf0Ml<p6&:%#p8m)VRke;>hPh1JE8'9Qf"ahN"Q0Y:?k.*p:*L<L5R!*JFXGIHeTZ\UeOR`'ASKHUi62C/moTBX*#Y47L\*`#f?Z=fCM"pmRs.NFCiK%4!C1+et!;MfM80*.>*cnGOKqbO+]kh/MopX[lpL481^HsPK]*Y`Jn#/11msl*4jPu=P]na=O`>;=99"%m.hfs!@korL*g^d%E,.rhf;ij%8:cm`m^E<]X3M_5g+"=2e3pKl;lNLB,P1N4!I/+&Mn44;lbn+L6r4(AL(_Af$=sPJ]\LaHleu,>ApsD+"h[1B6#3G(Po2[T0A76:YOH/UZ>@6")L?@dA>&%Rl\LfKtgtiFPA+k74UQr,MPVe/,p5$56$"^rca]soMo=InL9!82*f=APKMLQQrf_R:Bib^1k8BTJ5I1<Z2r!<<CmhI+*)t_Ya%u)dHkp/kW0#\C,C9.LHdYAH.t5t%p2'4FF&ZB`aK^gn'LOopg^CR&a8.gr:cSPMtDT$^@Goq'@u_;0BhB=%B!k",QV:R[l7smT"2b01I>:D\XZ$(B)lLr]ULZ2#D=3pq'E@j1JBQS/usE"IqZ`Yj5'J`r41LZ)GIUkQOY4p("t?6FdEO_rn.Yqbu"L>eX>a/6)<tXRrp"MM:VqiCnQts4psL&*,YTV,B(<\!Pe\J\Vau]J9Zp0Q;K@%!Q4sbo-!V`1E=(2D];20+<^i7jLZh1q_K9g3J<pKd.a5D7$j=16S"S.KmZtpg`"*MDu(1U_)-)Dk2)smno;m?UAC/Hk+Fd@2.tE'=ODqB>9M`aZtiDHC=d5"0;-`*4!)oGd1.DK>X&grI6@+</-5$i^h%g3pgT)+T^^4BII`)Q$#hE+()qoZ?[3;<=>.?/\9>*TgN'#-c07b"joEQ=@D#=\!5mjsp&NMdD%*@Ca5Qmg%ICU>=N9W.i[KQE;=Mho$=X-s[6R$+RH#Pn;%t\ZB)JYD.b6(C,?.8_-BAane!$%=:6Nu5#AtR[lIJ`-TU+"(Mj';D<8n<]9@=Nm9+@jRAAFUhU6X#4RXQrL%)"?=2;3Z&!iC\A<g^6[mPljhgfm5%Ybe(WU#!PbrfE;/X?D2_!+C.BdGF.hO!?]9r4=]ciDumeH=00WPJ[eGX#2`,<YI7;^_:1/.NfC.Pi+P^NJ/\B#0K,Y_*(W#ld!FUG:(R8YDhn_*_*uDgXq#8BfZdP*06sR\8%Wj@JG9:Dg67%Zf=-<Em+s0"jTkTMObhbQOnjq?LuhT.ab+*Cflp$=G+_S:VAs6a700=d@D=.0RthueD@o]!;S^j3FDRfcO)HUIi\=J2%cRY10!UF)_Am'O`%Ithi5_h88&N)"sH>0,'#g=Rik$rq8b:@J_ebp';McCi8LZ(D'DY#AhU3aai[5T[O!#B<Rlt`ojtNd+]b_;0r(\Kamr)ML_P5IeYP)J+7gYN,.)aALG6In&)h"VjA\U=@_f(d@$6[p<T17;*?-;>W+pP06a:)ilpRE*GlCMXGO.^@e9L\KNAhXQ%'-qIV]:J_30()BfN_>Wc.2n+3u[:I-f+?SYi709g79iZ1q]4/TtBrp/Ub[@N(6V6Lb^#k3m2cT;VgVDCod4!l>'jn$n:4:qM=ft?eQ-g/igL:oL>nmgL'&kKWU:DAdkF4+FUFKrB[c>1^MQIA)t:Y_eHO=h5BI*#K1-#YGm<]hVJ,L*LZ,tEQ:R_g=osVY1:r#A"75tX$rSqT-),-LFbm!dgjK:>"rqmgTJ&qDRHAb1ZAcj$:7</<f$?F+Ikr-k[N*c,ojZd4OUCq\a&_+YCC8K1GQS`U2df)q)=6=0if!;P+p-kTHDHVg/ds?@l&Wc"o\s`8n9/#aoHSG6",IUBb)>"?%j_aP8i&'V(BYjKc?QEH_5":^WLku!`?ap.pd8SiPH+NT%^&)[bN3FPEjqE(3&'hX#6N[^sEU`:`'U'6sK+A7q2Og9=0gf!-r4eg!Sjd3!1Wt#Ad5QP@-:pr.88^fq2Rn/4Dn8o>=_<CY;\ap`b\S5=rn"hQ2Q\CoQ*&j6@HXk/7/b`s8U$Z1@.WOIN"mFW9uD:OG&48]EnZeXXd>Zu"m!IjChN;6%<pcuBap@A6jN/S(KsWa+CMc?dDnp#mbR,IM`l+FgN+RaUkgFpiK+.<ukL_\o*FUBch"(VY@[,&+(:;NuVOU,^`<ISPE@kH-E7_Y+UFpbtkNVgb/Cr_][R$O1fJ:N/!oQBr/\)LFP/o9+K16XEY77&q1+/YrTWH5D*t/d'J#A@+b/nUa*F"Tisb7-TP#,dN1l7-9LBlr9^@4_3T4M#sM:A+C\-&k=ufq;*<SA3KuZZrj^2b?P2%;A.J;Qo_Dq?9`F'7HT01OCb(=#nhclI$scH'n>!kro<Sj"R$ocanH:K-DE)^>eU)]XSm[A5$gsZQJP9`qnj:$FBV,`C@3mb>[5@9Q=aRidWC8erIk.L;O8ubjW_=8r`?(OM&?JsKn4!tTWt.FF*RRcYq9f1,&X9)7^K]GcMjF6[^XVZ*2+Z_UW$mUPP0e*pU#1JJr@`3:3q0=:R1bM4AJ99Ynh8dr%*;'o?.[/L?2Bu.@9,.6A&;cadV'4WI-SS[NQudiI]BRMTV02^Eq+;5k#!:BiTCu."(E@mj%AZI9O&OdZIfjE[D08QWCk$*($m-f5U:FRW3QI~>endstream
endobj
17 0 obj
<<
/Filter [ /ASCII85Decode /FlateDecode ] /Length 1615
>>
stream
Gatm;lYiOf&HD15@Qot=6J1C6WMu,KX/iDF-*\q"F`lERK"5?Vnjn**lT<:A%JYM?Sh5;?)<UIYg"@5E#(B/Bn#l_/k&<=*5UaiD+I6o]r"Il3-\"i]_eH`o5u!,dlVQDY*.mW&=nda>GRb/6kFX*4%IXsY.W8#uY'Diq_1E%&/D&B#oh7%Ka>r%G:S(B8)UrNqJ=%a#^&gWs=dLoX4S^,$)=Xa6Zn(1r%-Cn_qb\r@?=83lg@*W(09:<a?:spK^QSVj3'](W1+o`ZjUZ+bBb[KC9311<M%0Im0;YI(M>q!q/CK4fZD?E:ZbFqZq1%u5MY*JmZmta8/<+m.i%Ipf($W><e-gSQEHnKLfo?7/3jImV7e"a++YsjD"JuecSN?Oc[oNgo/^)9D,Ocb@9=X#c<V1(NhdN48**E%H&B<JFkBf`fZoGrNJ[/R9UR&#N3R;(5G/Mb,f:WhnWZWqQI2VoT:fp)T+onUg$coBUF^(mU;1)fhU&s!jb=,8'&pSgE*bX2\7lMu_R*oVg8dVApbX<q/milCp.<2tid2G+%%?<hQo5qM$9D0,G)CgL2<p43-a%',1Y>q(q8!r@44!qB24"rp2H]=K:Etq,\L:RKd`Y4L)]RVI>0Eoo2_Ha18>]R2@ReMP)":.EJ`p<\+W.DOg"1DVZLK,h]F"-N]#PY?/_ZdI/`XU6kW;O(dQIk(MBu@8+"A5CPoh^p]ILDG.GcN4FZb";=irhHpW*GN-A8,dhS]J*5%LXNngJ[6X?eLQC*=5o=;gSU7>['-lcTP-O.!!%'[Rk>l,c;N/*L;(dofAXjW)2MNZBJ"cGGJ`6SZ<C7^fjoMH%dFPp]V0qm[#%S7AAu]j!.C=\/s@M]8X;'8o[_t&ph-".?gWgUO/6J5.V7+eull00&#6/rt">U+T"o)T46&t^r'QRq4^I_82ttV[I[H4B6b+8CUD\@D"mQb(^8r-)X`Ocem"aOgL7:.*XIF?d.NqiG/lsN,pb7UX^EFVT&;oXp7EaKS52u4@e]ZeU$isb)$R=r\#i+k?RskfB3fp[6dPh8%Fi^9#@KE[POa(IXu5<oa\C#Yp:?enj-X'N*V8050ad;^@UZmNgl7s!4l@8.&k>;[%L.!PYHg3Jn+5#hNe7$^rk;-LB53MWp-UHP*=J>Fq/<3Oa22hoD2qYd:.!ean5>I/0#?a72EHd;Q9Bh1F4DSqm@&\6F@g`3P-Ark;"1V63'Dl"+dbDobgldR)CgWm_"Gsl5a*+/0I43BDQ-G/>%;@5Xuk"d%(<Kdg;2aQ1[WQZC$1gBb#L`'7EP5]SFSCfDkb+b-d.\*=,].8@QY.])1]E/^a#jX7-(liD=n1eIURHGq$1:.ZDYM<QLa%@c*5Ib['*@:/b<,.C`O2#m.<MjIW":A8ng(_ik,in[#"5t*DC7(#MFYp*6e28`LD;DM`d>1J"`^u*Jm(I"tV?G$n>]-nl<Pb@W?S0N+SL78CWK9jG[l87t$EH5"$.:0!mW4d"A&&Aj*QYX'tYso1*2BS=Uu0Yp@#Nrj!X)3.5:^N37)l#9R0:[*!KX@E0Q,nXS4[D'i`1W"a"a0nL-,FkP1"^"3-5X!d9tI-TutXB%:k08F5YjBkBnE$t~>endstream
endobj
xref
0 18
0000000000 65535 f
0000000061 00000 n
0000000132 00000 n
0000000239 00000 n
0000000351 00000 n
0000000456 00000 n
0000000533 00000 n
0000000738 00000 n
0000000943 00000 n
0000001058 00000 n
0000001263 00000 n
0000001469 00000 n
0000001539 00000 n
0000001857 00000 n
0000001936 00000 n
0000004764 00000 n
0000005779 00000 n
0000008933 00000 n
trailer
<<
/ID
[<5308ad286f7ed61e1c31de3c2e9f2ace><5308ad286f7ed61e1c31de3c2e9f2ace>]
% ReportLab generated PDF document -- digest (opensource)
/Info 12 0 R
/Root 11 0 R
/Size 18
>>
startxref
10640
%%EOF
+72
View File
@@ -0,0 +1,72 @@
#pragma once
// av_frame_ptr.h — RAII smart pointer for FFmpeg AVFrame*.
//
// Why this exists
// ---------------
// CVideoPlayer::getNV12Frame() / getCudaHWFrame() return cloned AVFrame* and
// the contract is "caller must av_frame_free()". That contract is comment-only
// — nothing in the type system enforces it, so any early return, exception,
// or refactor that drops the pointer leaks the AVFrame *and* the underlying
// NV12/YUV420P/YUVJ420P buffers (~3 MB per 1080p frame, ~50 MB/s under load).
//
// AvFramePtr wraps the raw pointer in std::unique_ptr<AVFrame, AvFrameDeleter>
// so the destructor frees it automatically — leak-by-omission becomes a
// compile-time impossibility on every internal code path that adopts this type.
//
// Scope: INTERNAL TO ANSCORE ONLY.
// -------------------------------
// Do NOT change DLL-exported function signatures to use AvFramePtr.
// std::unique_ptr cannot cross a DLL boundary safely (different runtimes,
// different STL implementations, different allocators). The public
// AVFrame*-returning APIs in ANSRTSP.h / ANSSRT.h / etc. must remain raw
// pointers for ABI stability.
//
// Use this type at the internal *consumer* call site:
//
// AvFramePtr nv12{ player->getNV12Frame() };
// if (nv12) {
// doSomething(nv12.get()); // borrowed access
// } // auto-freed on scope exit
//
// To transfer ownership into ANSCV's pendingFree pipeline (the
// gpu_frame_attach / gpu_frame_attach_cuda functions, which take raw
// AVFrame* and store/defer them), call .release():
//
// AvFramePtr nv12{ player->getNV12Frame() };
// gpu_frame_attach(mat, nv12.release(), gpuIdx, pts);
//
// Every .release() in the codebase is a deliberate ownership hand-off —
// `git grep '\.release()'` over the migrated files gives an audit list.
extern "C" {
#include <libavutil/frame.h>
}
#include <memory>
namespace ansmedia {
// Custom deleter that calls av_frame_free (which takes AVFrame**).
// noexcept because deleters must not throw — av_frame_free is a C function
// and never throws, but tagging makes the contract explicit.
struct AvFrameDeleter {
void operator()(AVFrame* f) const noexcept {
if (f) {
av_frame_free(&f); // takes AVFrame**, sets *f to nullptr internally
}
}
};
// Owning smart pointer for AVFrame*. Frees on destruction via av_frame_free.
//
// Construction:
// AvFramePtr p; // empty
// AvFramePtr p{ av_frame_clone(src) }; // adopt raw clone
//
// Ownership transfer (e.g. handing off to gpu_frame_attach_cuda):
// takes_ownership(p.release()); // raw pointer escapes, no auto-free
//
// Copy is disallowed (unique_ptr semantics). Move is fine.
using AvFramePtr = std::unique_ptr<AVFrame, AvFrameDeleter>;
} // namespace ansmedia
+10 -3
View File
@@ -59,6 +59,7 @@ CFilePlayer::CFilePlayer()
, m_bSeek(0)
, m_dSeekPos(0)
, m_nNalLength(0)
, m_bLooped(FALSE)
, m_pFormatContext(NULL)
, m_hReadThread(0)
, m_hVideoThread(0)
@@ -280,8 +281,9 @@ BOOL CFilePlayer::play()
{
return TRUE;
}
m_bPlaying = TRUE;
m_bLooped = FALSE; // fresh playback — no wrap yet
if (m_nVideoIndex >= 0)
{
@@ -440,12 +442,17 @@ BOOL CFilePlayer::readFrame()
return FALSE;
}
}
// File wrapped — flag it so the consumer (ANSVideoPlayer) can detect
// EOF without watching m_nCurPos, which is non-monotonic during normal
// playback (decode-order packet PTS + audio thread also writes to it).
m_bLooped = TRUE;
if (av_read_frame(m_pFormatContext, &pkt) != 0)
{
return FALSE;
}
}
}
else if (0 != rret)
{
return FALSE;
+8 -1
View File
@@ -53,6 +53,12 @@ public:
BOOL seek(int pos);
int64 getElapse() {return m_nCurPos;};
int64 getDuration() {return m_nDuration;}
// Set inside readFrame() when av_read_frame returns AVERROR_EOF and the
// reader auto-loops via av_seek_frame. Caller (ANSVideoPlayer) polls this
// to detect EOF without relying on m_nCurPos, which is non-monotonic
// (decode-order packet PTS + audio thread also writes to it).
BOOL hasLooped() {return m_bLooped;}
void clearLooped() {m_bLooped = FALSE;}
int getVideoCodec();
int getAudioCodec();
BOOL onRecord();
@@ -79,7 +85,8 @@ private:
int m_bSeek; // seek request
double m_dSeekPos; // seek position, nit is millisecond
int m_nNalLength;
BOOL m_bLooped; // set by readFrame() when EOF triggered an av_seek_frame back to start
AVFormatContext * m_pFormatContext; // format context
pthread_t m_hReadThread; // read thread
+40 -15
View File
@@ -943,18 +943,20 @@ BOOL CVideoDecoder::decode(AVPacket* pkt)
if (!m_bInited)
{
log_print(HT_LOG_ERR, "%s, Decoder not initialized\r\n", __FUNCTION__);
// Hot-path pre-check; per-packet during early frames if decode() races
// with init.
ANS_DBG("MEDIA_Decode", "decode(pkt) called but decoder not initialized");
return FALSE;
}
if (!m_bRunning)
{
log_print(HT_LOG_WARN, "%s, Decoder not running\r\n", __FUNCTION__);
ANS_DBG("MEDIA_Decode", "decode(pkt) called but decoder not running");
return FALSE;
}
if (!m_pContext) {
log_print(HT_LOG_ERR, "%s, Context is NULL\r\n", __FUNCTION__);
ANS_DBG("MEDIA_Decode", "decode(pkt) called but m_pContext is NULL");
return FALSE;
}
@@ -968,7 +970,12 @@ BOOL CVideoDecoder::decode(AVPacket* pkt)
{
if (!readFrame())
{
log_print(HT_LOG_ERR, "%s, Failed to read frame during retry %d\r\n", __FUNCTION__, retryCount);
// Hot-path error: was log_print() which fflushes per call and can
// spike per-packet during stream stress. ANS_DBG is silent in
// production; the per-frame error count is also visible via
// MEDIA_Pulse drop(err=) so observability is preserved.
ANS_DBG("MEDIA_Decode",
"send_packet retry %d: readFrame failed", retryCount);
return FALSE;
}
Sleep(1); // Reduced sleep time
@@ -980,7 +987,8 @@ BOOL CVideoDecoder::decode(AVPacket* pkt)
{
char error_buf[AV_ERROR_MAX_STRING_SIZE];
av_strerror(ret, error_buf, sizeof(error_buf));
log_print(HT_LOG_ERR, "%s, avcodec_send_packet failed: %s (ret=%d)\r\n", __FUNCTION__, error_buf, ret);
ANS_DBG("MEDIA_Decode",
"avcodec_send_packet failed: %s (ret=%d)", error_buf, ret);
return FALSE;
}
@@ -993,25 +1001,29 @@ BOOL CVideoDecoder::decode(uint8* data, int len, int64_t pts)
if (!m_bInited)
{
log_print(HT_LOG_ERR, "%s, Decoder not initialized\r\n", __FUNCTION__);
// Hot-path: per-packet if decoder isn't initialised. Was log_print
// which fflushes per call.
ANS_DBG("MEDIA_Decode", "decode called but decoder not initialized");
return FALSE;
}
if (!m_bRunning)
{
log_print(HT_LOG_WARN, "%s, Decoder not running\r\n", __FUNCTION__);
// Per-packet during start/stop transitions.
ANS_DBG("MEDIA_Decode", "decode called but decoder not running");
return FALSE;
}
if (!data || len <= 0) {
log_print(HT_LOG_ERR, "%s, Invalid input data\r\n", __FUNCTION__);
ANS_DBG("MEDIA_Decode", "decode called with invalid input (data=%p len=%d)",
(void*)data, len);
return FALSE;
}
// Allocate packet
AVPacket* packet = av_packet_alloc();
if (!packet) {
log_print(HT_LOG_ERR, "%s, Failed to allocate AVPacket\r\n", __FUNCTION__);
ANS_DBG("MEDIA_Decode", "av_packet_alloc failed (OOM?)");
return FALSE;
}
@@ -1020,7 +1032,8 @@ BOOL CVideoDecoder::decode(uint8* data, int len, int64_t pts)
if (ret < 0) {
char error_buf[AV_ERROR_MAX_STRING_SIZE];
av_strerror(ret, error_buf, sizeof(error_buf));
log_print(HT_LOG_ERR, "%s, Failed to allocate packet data: %s\r\n", __FUNCTION__, error_buf);
ANS_DBG("MEDIA_Decode",
"av_new_packet failed (len=%d): %s", len, error_buf);
av_packet_free(&packet);
return FALSE;
}
@@ -1061,13 +1074,21 @@ BOOL CVideoDecoder::readFrame()
else if (ret < 0) {
char error_buf[AV_ERROR_MAX_STRING_SIZE];
av_strerror(ret, error_buf, sizeof(error_buf));
log_print(HT_LOG_ERR, "%s, avcodec_receive_frame failed: %s (ret=%d)\r\n", __FUNCTION__, error_buf, ret);
// Hot-path error during stream stress (UDP loss, corrupt I-frame).
// Was log_print(HT_LOG_ERR, ...) which fflushes per call and could
// fire ~25/sec on a degraded stream — observed as sudden GetImage
// slowness. ANS_DBG is silent in production; pulse counter
// MEDIA_Pulse drop(err=) preserves observability.
ANS_DBG("MEDIA_Decode",
"avcodec_receive_frame failed: %s (ret=%d)", error_buf, ret);
return FALSE;
}
// Check if we got a valid frame
if (!m_pFrame || m_pFrame->width <= 0 || m_pFrame->height <= 0) {
log_print(HT_LOG_WARN, "%s, Received invalid frame\r\n", __FUNCTION__);
ANS_DBG("MEDIA_Decode", "received invalid frame (w=%d h=%d)",
m_pFrame ? m_pFrame->width : -1,
m_pFrame ? m_pFrame->height : -1);
av_frame_unref(m_pFrame);
continue;
}
@@ -1090,8 +1111,10 @@ BOOL CVideoDecoder::readFrame()
{
char error_buf[AV_ERROR_MAX_STRING_SIZE];
av_strerror(ret, error_buf, sizeof(error_buf));
log_print(HT_LOG_ERR, "%s, Error transferring hardware frame to system memory: %s (ret=%d)\r\n",
__FUNCTION__, error_buf, ret);
// Hot-path: HW→SW transfer failure. Per-frame on degraded HW
// decoder. Gated to avoid fflush spike during stress.
ANS_DBG("MEDIA_Decode",
"hwframe_transfer_data failed: %s (ret=%d)", error_buf, ret);
av_frame_unref(m_pFrame);
if (m_pCudaHWFrame) av_frame_free(&m_pCudaHWFrame);
continue;
@@ -1139,7 +1162,9 @@ int CVideoDecoder::render(AVFrame* frame)
m_pCallback(frame, m_pUserdata);
}
catch (...) {
log_print(HT_LOG_ERR, "%s, Exception in callback function\r\n", __FUNCTION__);
// Hot-path: per-frame if onVideoFrame ever throws. Was log_print
// which fflushes per call.
ANS_DBG("MEDIA_Decode", "exception in onVideoFrame callback");
return 0;
}
}
File diff suppressed because it is too large Load Diff
+51 -1
View File
@@ -11,12 +11,14 @@
#include "media_util.h"
#include <list>
#include <string>
#include <array>
#include <opencv2/imgproc.hpp>
#include <opencv2/highgui.hpp>
#include <opencv2/opencv.hpp>
#include <turbojpeg.h>
#include <atomic>
#include <chrono>
#include "ANSLicense.h" // ANS_DBG (used by inline FrameQueue::pushFrame below)
// Leak diagnostics — net counters surfaced in MEDIA_Leak heartbeat.
// Defined in video_player.cpp; also incremented from FrameQueue here.
@@ -51,6 +53,16 @@ private:
uint64_t m_frameSeq = 0; // Sequence counter — increments on each new frame push
public:
// Pulse counter for evicted-unread frames (instrumentation, not behavior).
// Incremented every time a push has to drop the oldest frame because the
// queue hit maxFrames AND the oldest never got read. With getLatestFrame()
// the consumer reads the back, so anything pushed before the consumer's
// last read sits in the middle/front and gets silently evicted here.
static std::atomic<uint64_t>& evictedCounter() {
static std::atomic<uint64_t> g{0};
return g;
}
// Push a new frame to the queue (removes oldest if full)
void pushFrame(AVFrame* newFrame) {
if (!newFrame) return;
@@ -60,7 +72,9 @@ public:
// Clone the frame to ensure proper ownership transfer
AVFrame* frameCopy = av_frame_clone(newFrame);
if (!frameCopy) {
std::cerr << "Failed to clone AVFrame!" << std::endl;
// av_frame_clone failure (basically OOM). Gated to avoid spamming
// stderr during memory pressure.
ANS_DBG("MEDIA_Queue", "av_frame_clone failed (OOM?)");
return;
}
g_queueClones.fetch_add(1, std::memory_order_relaxed);
@@ -74,6 +88,7 @@ public:
frameQueue.pop();
av_frame_free(&oldFrame);
g_queueFrees.fetch_add(1, std::memory_order_relaxed);
evictedCounter().fetch_add(1, std::memory_order_relaxed);
}
}
@@ -177,6 +192,16 @@ public:
virtual void setImageQuality(int mode) { m_nImageQuality = mode; }
void setTargetFPS(double intervalMs); // Set minimum interval between processed frames in ms (0 = no limit, 100 = ~10 FPS)
double getLastFrameAgeMs(); // Milliseconds since last frame arrived from decoder (0 if no frame yet)
// Jitter buffer: hold decoded frames for ~ms before delivering to consumer.
// Smooths network/decoder jitter at the cost of added end-to-end latency.
// 0 disables (today's getLatest semantics). Default 200 ms.
// Adaptive: if the consumer is slower than the producer, drop-ahead kicks in
// and bypasses the buffer (latency stays ~0 frames).
// Note: applies only to the BGR cv::Mat returned by getImage(); the NV12/CUDA
// fast-path frames (m_currentNV12Frame / m_currentCudaHWFrame) remain
// updated at push time so inference always sees the latest.
void setJitterBufferMs(int ms);
int getJitterBufferMs() const { return m_jitterBufferMs; }
virtual void setRtpMulticast(BOOL flag) {}
virtual void setRtpOverUdp(BOOL flag) {}
@@ -317,6 +342,31 @@ protected:
std::chrono::steady_clock::time_point m_lastDecoderFrameTime;
bool m_lastDecoderFrameTimeSet = false;
// --- Jitter buffer (BGR delivery only) ---
// Default 200 ms — absorbs typical network/decoder hiccups.
int m_jitterBufferMs = 200;
// Observed inter-arrival interval, ms. Seeds at 40 ms (=25 fps) and is
// refined to a rolling median of the last m_arrivalCount samples.
double m_observedFrameIntervalMs = 40.0;
// Ring buffer of recent arrival ticks (steady_clock raw ticks). Used to
// compute the rolling median frame interval. Sized at 32 — covers >1 s at
// 30 fps so a single GOP-boundary jitter doesn't dominate.
static constexpr size_t kArrivalRingSize = 32;
std::array<long long, kArrivalRingSize> m_arrivalTicks{};
size_t m_arrivalIdx = 0;
size_t m_arrivalCount = 0;
// Wall-clock of the last frame released to the consumer. Used for rate-paced
// delivery — if the consumer polls faster than the producer rate, between
// releases we return the cached frame so the buffer doesn't drain instantly.
std::chrono::steady_clock::time_point m_lastReleaseTime;
bool m_lastReleaseTimeSet = false;
// Fill-phase: when true, hold releases until queue depth reaches target.
// Set true at startup and again whenever the queue drains to empty mid-stream.
// This is what makes the buffer actually buffer — without it, pacing alone
// can't grow the queue from empty (producer and consumer would just match
// rates with depth=0 forever).
bool m_jitterBufferRefilling = true;
BOOL m_bPlaying;
BOOL m_bPaused;
+437
View File
@@ -0,0 +1,437 @@
"""Generate the ANSODEngine Tracker / Stabilization developer-guide PDF.
Run:
python build_anscore_devguide_pdf.py
Output:
ANSODEngine_Tracker_Stabilization_Guide.pdf (next to this script)
"""
from pathlib import Path
from reportlab.lib import colors
from reportlab.lib.enums import TA_LEFT
from reportlab.lib.pagesizes import A4
from reportlab.lib.styles import ParagraphStyle, getSampleStyleSheet
from reportlab.lib.units import mm
from reportlab.platypus import (
PageBreak,
Paragraph,
SimpleDocTemplate,
Spacer,
Table,
TableStyle,
)
OUTPUT = Path(__file__).with_name("ANSODEngine_Tracker_Stabilization_Guide.pdf")
# --------------------------------------------------------------------------- #
# Styles #
# --------------------------------------------------------------------------- #
styles = getSampleStyleSheet()
H1 = ParagraphStyle(
"H1", parent=styles["Heading1"],
fontName="Helvetica-Bold", fontSize=18, leading=22,
spaceBefore=4, spaceAfter=10, textColor=colors.HexColor("#0B3D91"),
)
H2 = ParagraphStyle(
"H2", parent=styles["Heading2"],
fontName="Helvetica-Bold", fontSize=13, leading=16,
spaceBefore=14, spaceAfter=6, textColor=colors.HexColor("#0B3D91"),
)
H3 = ParagraphStyle(
"H3", parent=styles["Heading3"],
fontName="Helvetica-Bold", fontSize=11, leading=14,
spaceBefore=10, spaceAfter=4, textColor=colors.HexColor("#222222"),
)
BODY = ParagraphStyle(
"Body", parent=styles["BodyText"],
fontName="Helvetica", fontSize=9.5, leading=13,
spaceBefore=2, spaceAfter=4, alignment=TA_LEFT,
)
NOTE = ParagraphStyle(
"Note", parent=BODY,
leftIndent=10, fontSize=9, textColor=colors.HexColor("#444444"),
)
CODE = ParagraphStyle(
"Code", parent=BODY,
fontName="Courier", fontSize=8.5, leading=11,
leftIndent=8, rightIndent=8, spaceBefore=4, spaceAfter=8,
backColor=colors.HexColor("#F4F4F4"),
borderColor=colors.HexColor("#D0D0D0"), borderWidth=0.5,
borderPadding=4,
)
TABLE_HEADER = ParagraphStyle(
"TH", parent=BODY, fontName="Helvetica-Bold", fontSize=9, leading=12,
textColor=colors.white,
)
TABLE_CELL = ParagraphStyle(
"TC", parent=BODY, fontSize=9, leading=12, spaceBefore=0, spaceAfter=0,
)
TABLE_CELL_MONO = ParagraphStyle(
"TCM", parent=TABLE_CELL, fontName="Courier", fontSize=8.5,
)
# --------------------------------------------------------------------------- #
# Helpers #
# --------------------------------------------------------------------------- #
def p(text, style=BODY):
return Paragraph(text, style)
def code_block(lines):
"""Multi-line code block, preserves indentation by replacing spaces with &nbsp;."""
body = "<br/>".join(
line.replace(" ", "&nbsp;").replace("<", "&lt;").replace(">", "&gt;")
for line in lines
)
# restore tag rendering for our own tags
body = body.replace("&lt;br/&gt;", "<br/>")
return Paragraph(body, CODE)
def make_table(header, rows, col_widths=None, mono_cols=None):
"""rows: list[list[str]]. mono_cols: indices that should use monospace."""
mono_cols = mono_cols or set()
def cell(text, col_idx):
style = TABLE_CELL_MONO if col_idx in mono_cols else TABLE_CELL
return Paragraph(text, style)
data = [[Paragraph(h, TABLE_HEADER) for h in header]]
for row in rows:
data.append([cell(c, i) for i, c in enumerate(row)])
t = Table(data, colWidths=col_widths, repeatRows=1)
t.setStyle(TableStyle([
("BACKGROUND", (0, 0), (-1, 0), colors.HexColor("#0B3D91")),
("TEXTCOLOR", (0, 0), (-1, 0), colors.white),
("ALIGN", (0, 0), (-1, -1), "LEFT"),
("VALIGN", (0, 0), (-1, -1), "TOP"),
("FONTSIZE", (0, 0), (-1, -1), 9),
("BOTTOMPADDING", (0, 0), (-1, 0), 6),
("TOPPADDING", (0, 0), (-1, 0), 6),
("BOTTOMPADDING", (0, 1), (-1, -1), 4),
("TOPPADDING", (0, 1), (-1, -1), 4),
("LEFTPADDING", (0, 0), (-1, -1), 5),
("RIGHTPADDING", (0, 0), (-1, -1), 5),
("ROWBACKGROUNDS", (0, 1), (-1, -1),
[colors.white, colors.HexColor("#F4F7FB")]),
("GRID", (0, 0), (-1, -1), 0.4, colors.HexColor("#B0B7C3")),
]))
return t
# --------------------------------------------------------------------------- #
# Document content #
# --------------------------------------------------------------------------- #
def build_story():
s = []
# Title
s.append(p(
"ANSODEngine &mdash; Tracker &amp; Stabilization Developer Guide",
H1,
))
s.append(p(
"Module: <b>ANSODEngine</b> &nbsp;&middot;&nbsp; "
"Base class: <b>ANSCENTER::ANSODBase</b> &nbsp;&middot;&nbsp; "
"Affects every OD engine subclass "
"(ANSRTYOLO, ANSONNXYOLO, ANSYOLOV12RTOD, etc.)",
NOTE,
))
s.append(Spacer(1, 6))
# ── 1. Default state after model load ─────────────────────────────────
s.append(p("1. Default state after model load", H2))
s.append(p(
"Every engine derived from <b>ANSODBase</b> finishes "
"<font face='Courier'>Initialize</font> / "
"<font face='Courier'>LoadModel</font> / "
"<font face='Courier'>LoadModelFromFolder</font> "
"with these defaults:"
))
s.append(make_table(
["Feature", "Default", "Source of default"],
[
["Tracker (BYTETrack / UCMC / OCSORT)",
"<b>ON</b>",
"Loaders auto-call SetTracker(BYTETRACK, true) "
"(ANSODEngine.cpp:77, 110, 139, 174, 246, 260)"],
["Stabilization (ghost injection + EMA + hysteresis + "
"class consistency)",
"<b>OFF</b>",
"Member initializer _stabilizationEnabled = false; "
"no auto-enable"],
],
col_widths=[55*mm, 18*mm, 95*mm],
))
s.append(Spacer(1, 4))
s.append(p(
"<b>Notes.</b> Tracker assigns a <font face='Courier'>trackId</font> "
"to detections and (when stabilization is on) lets the tracker "
"predict bridging &ldquo;ghost&rdquo; boxes. "
"Stabilization is opt-in and requires the tracker. "
"It cannot run standalone.",
NOTE,
))
# ── 2. Public setters &mdash; invariant table ─────────────────────────
s.append(p("2. Public setters &mdash; invariant table", H2))
s.append(p(
"The single invariant maintained by the API: "
"<b>stabilization requires tracker</b> "
"(<font face='Courier'>stab=true &rArr; tracker=true</font>)."
))
s.append(make_table(
["Call", "_trackerEnabled", "_stabilizationEnabled", "Why"],
[
["SetTracker(type, true)",
"&rarr; true",
"unchanged",
"Tracker is independently useful. Don&rsquo;t side-effect stab."],
["SetTracker(type, false)",
"&rarr; false",
"<b>&rarr; false</b> (forced)",
"Stabilization can&rsquo;t run without tracker. Forced clear "
"keeps the flag honest."],
["SetStabilization(true, hist, miss)",
"<b>&rarr; true</b> (auto if off)",
"&rarr; true",
"Stabilization needs tracker. Auto-enables it."],
["SetStabilization(false)",
"unchanged",
"&rarr; false",
"Tracker stays usable on its own."],
["SetStabilizationParameters(json)",
"unchanged",
"unchanged",
"Tunes thresholds only; does not toggle either flag."],
["SetTrackerParameters(json)",
"unchanged",
"unchanged",
"Tunes tracker params only; no-op if tracker is off."],
],
col_widths=[55*mm, 28*mm, 32*mm, 53*mm],
mono_cols={0},
))
# ── 3. Reachable runtime states ───────────────────────────────────────
s.append(p("3. Reachable runtime states", H2))
s.append(make_table(
["_trackerEnabled", "_stabilizationEnabled", "What runs at inference"],
[
["false", "false",
"Raw detections only. No trackId. No ghosts."],
["true", "false",
"Detections + trackId assignment. <b>No ghosts, no EMA, "
"no hysteresis.</b> &nbsp;&larr; default"],
["true", "true",
"Full pipeline: tracking + ghost injection + EMA confidence + "
"hysteresis + class consistency + stale-history pruning."],
["false", "true",
"<b>Unreachable</b> &mdash; invariant prevents it."],
],
col_widths=[35*mm, 42*mm, 91*mm],
mono_cols={0, 1},
))
# ── 4. Caller recipes ─────────────────────────────────────────────────
s.append(p("4. How to opt in / out (caller recipes)", H2))
s.append(make_table(
["Goal", "Sequence"],
[
["Default (tracker only, no stabilization)",
"Nothing. Just load the model."],
["Tracker only, explicit",
"SetTracker(BYTETRACK, true)"],
["Tracker + stabilization",
"SetStabilization(true, 20, 5) &nbsp; (auto-enables tracker)"],
["Tracker + stabilization with custom thresholds",
"SetStabilization(true) then SetStabilizationParameters(json)"],
["Disable everything",
"SetTracker(BYTETRACK, false) &nbsp; (also clears stabilization)"],
["Switch tracker type without losing stabilization",
"SetTracker(UCMC, true) then SetStabilization(true) "
"if it was on"],
],
col_widths=[70*mm, 98*mm],
mono_cols={1},
))
s.append(PageBreak())
# ── 5. Stabilization parameters ──────────────────────────────────────
s.append(p("5. Stabilization parameters", H2))
s.append(p(
"Apply only when stabilization is on. "
"<font face='Courier'>SetStabilization(enabled, historySize, "
"maxMisses)</font> arguments:"
))
s.append(make_table(
["Param", "Default", "Min / clamp", "Effect"],
[
["historySize", "20", "&ge; 5",
"Frames retained in _detectionQueue per camera."],
["maxMisses", "5", "&ge; 1",
"Frames a ghost may persist before pruning. "
"History pruned at 2&times;maxMisses unseen frames."],
],
col_widths=[30*mm, 18*mm, 22*mm, 98*mm],
mono_cols={0},
))
s.append(Spacer(1, 6))
s.append(p(
"<font face='Courier'>SetStabilizationParameters(json)</font> "
"&mdash; all keys optional:",
))
s.append(make_table(
["Key", "Default", "Range", "Purpose"],
[
["hysteresis_enter", "model threshold", "float, 0 = auto",
"Confidence needed for a <i>new</i> track to appear."],
["hysteresis_keep", "0.65 &times; enter", "float, 0 = auto",
"Lower threshold to <i>keep</i> an established track visible."],
["ema_alpha", "0.30", "[0.01, 1.0]",
"EMA weight for new observations (lower = smoother)."],
["track_boost_min_frames", "10", "&ge; 1",
"Frames before a track is &ldquo;established&rdquo; and gets "
"confidence boost."],
["track_boost_amount", "0.05", "[0.0, 0.5]",
"Confidence bonus for established tracks."],
["class_consistency_frames", "5", "&ge; 1",
"Consecutive frames of a new class required before accepting "
"a class switch on an established track."],
["confidence_decay", "0.85", "[0.1, 1.0]",
"Per-miss multiplier on ghost confidence."],
["min_confidence", "0.15", "[0.0, 1.0]",
"Floor below which ghosts are dropped."],
],
col_widths=[45*mm, 28*mm, 28*mm, 67*mm],
mono_cols={0},
))
# ── 6. Performance & memory ─────────────────────────────────────────
s.append(p("6. Performance &amp; memory characteristics", H2))
s.append(make_table(
["Aspect", "Tracker only", "Tracker + Stabilization"],
[
["Per-frame CPU cost",
"BYTETrack association, ~O(N&middot;M)",
"+ EMA / hysteresis / class-consistency loop O(N) + "
"history-prune O(K)"],
["Mutex acquisitions per inference",
"1 extra inside ApplyTracking",
"2 extra (ApplyTracking + StabilizeDetections) on the "
"engine&rsquo;s _mutex"],
["Result vector size",
"= real detections",
"= real detections + ghost objects in grace window"],
["Per-camera state",
"ANSMOT instance + BYTETrack removed_stracks_ "
"(capped at 1000)",
"+ _trackHistories (pruned at unseen &gt; 2&times;maxMisses) "
"+ _detectionQueue (capped at historySize)"],
["Bounded?",
"Yes &mdash; caps in BYTETrack (kRemovedCap=1000)",
"Yes &mdash; explicit prune in StabilizeDetections Phase 3"],
],
col_widths=[50*mm, 55*mm, 63*mm],
))
# ── 7. Diagnostic logging ────────────────────────────────────────────
s.append(p("7. Diagnostic logging", H2))
s.append(p(
"Every flag transition emits a DebugView line via "
"<font face='Courier'>ANS_DBG(\"ANSOD_StabState\", &hellip;)</font>:"
))
s.append(code_block([
"[ANSOD_StabState] this=<engine_ptr> "
"SetTracker(true,type=N) tracker=0->1 stab=0 (unchanged)",
"[ANSOD_StabState] this=<engine_ptr> "
"SetTracker(false) tracker=1->0 stab=1->0",
"[ANSOD_StabState] this=<engine_ptr> "
"SetStabilization(true,hist=20,miss=5) tracker=1->1 stab=0->1",
"[ANSOD_StabState] this=<engine_ptr> "
"SetStabilization(false) tracker=1 (unchanged) stab=1->0",
]))
s.append(p(
"Capture with DebugView (or any ANSCORE log) and "
"<font face='Courier'>grep ANSOD_StabState</font> to audit which "
"engine instances run with which flags in production.",
NOTE,
))
# ── 8. Behavior change vs previous build ─────────────────────────────
s.append(p("8. Behavior change vs. previous build", H2))
s.append(p(
"For developers updating from the prior version of ANSCORE."
))
s.append(make_table(
["Behavior", "Before", "After"],
[
["SetTracker(true) side-effect on stabilization",
"Auto-set stabilization to true",
"<b>No effect on stabilization</b>"],
["SetTracker(false) side-effect on stabilization",
"None (could leave stab=true while tracker=false)",
"<b>Forced stabilization to false</b>"],
["Default after model load",
"tracker=ON, stabilization=ON",
"tracker=ON, <b>stabilization=OFF</b>"],
["SetStabilization(true) auto-enable tracker",
"Yes",
"Yes (unchanged)"],
["SetStabilization(false) side-effect on tracker",
"None",
"None (unchanged)"],
],
col_widths=[60*mm, 53*mm, 55*mm],
))
s.append(Spacer(1, 6))
s.append(p(
"<b>Migration note.</b> If your application relied on the old "
"auto-enable to get stabilization for free, add an explicit "
"<font face='Courier'>SetStabilization(true)</font> call after "
"model load. ANSLPR&rsquo;s defensive "
"<font face='Courier'>SetStabilization(false)</font> calls remain "
"correct (now redundant but harmless).",
NOTE,
))
# ── Footer / version note ────────────────────────────────────────────
s.append(Spacer(1, 14))
s.append(p(
"<i>This guide reflects the ANSODEngine API as of the &lsquo;tracker/"
"stabilization invariant&rsquo; cleanup. File references point to "
"modules/ANSODEngine/ANSODEngine.cpp and "
"modules/ANSODEngine/ANSEngineCommon.h.</i>",
NOTE,
))
return s
# --------------------------------------------------------------------------- #
# Build #
# --------------------------------------------------------------------------- #
def main():
doc = SimpleDocTemplate(
str(OUTPUT),
pagesize=A4,
leftMargin=18*mm, rightMargin=18*mm,
topMargin=18*mm, bottomMargin=18*mm,
title="ANSODEngine — Tracker & Stabilization Developer Guide",
author="ANSCORE",
)
story = build_story()
doc.build(story)
print(f"Wrote {OUTPUT}")
if __name__ == "__main__":
main()
+951
View File
@@ -0,0 +1,951 @@
"""Generate the ANSLPR developer-guide PDF.
Covers two features:
1. ALPRChecker — tracker-based voting / text stabilization
2. Plate format validation — SetPlateFormat / SetPlateFormats
Run:
python build_anslpr_devguide_pdf.py
Output:
ANSLPR_ALPRChecker_PlateFormat_Guide.pdf (next to this script)
"""
from pathlib import Path
from reportlab.lib import colors
from reportlab.lib.enums import TA_LEFT
from reportlab.lib.pagesizes import A4
from reportlab.lib.styles import ParagraphStyle, getSampleStyleSheet
from reportlab.lib.units import mm
from reportlab.platypus import (
PageBreak,
Paragraph,
SimpleDocTemplate,
Spacer,
Table,
TableStyle,
)
OUTPUT = Path(__file__).with_name("ANSLPR_ALPRChecker_PlateFormat_Guide.pdf")
# --------------------------------------------------------------------------- #
# Styles #
# --------------------------------------------------------------------------- #
styles = getSampleStyleSheet()
H1 = ParagraphStyle(
"H1", parent=styles["Heading1"],
fontName="Helvetica-Bold", fontSize=18, leading=22,
spaceBefore=4, spaceAfter=10, textColor=colors.HexColor("#0B3D91"),
)
H2 = ParagraphStyle(
"H2", parent=styles["Heading2"],
fontName="Helvetica-Bold", fontSize=13, leading=16,
spaceBefore=14, spaceAfter=6, textColor=colors.HexColor("#0B3D91"),
)
H3 = ParagraphStyle(
"H3", parent=styles["Heading3"],
fontName="Helvetica-Bold", fontSize=11, leading=14,
spaceBefore=10, spaceAfter=4, textColor=colors.HexColor("#222222"),
)
BODY = ParagraphStyle(
"Body", parent=styles["BodyText"],
fontName="Helvetica", fontSize=9.5, leading=13,
spaceBefore=2, spaceAfter=4, alignment=TA_LEFT,
)
NOTE = ParagraphStyle(
"Note", parent=BODY,
leftIndent=10, fontSize=9, textColor=colors.HexColor("#444444"),
)
CODE = ParagraphStyle(
"Code", parent=BODY,
fontName="Courier", fontSize=8.5, leading=11,
leftIndent=8, rightIndent=8, spaceBefore=4, spaceAfter=8,
backColor=colors.HexColor("#F4F4F4"),
borderColor=colors.HexColor("#D0D0D0"), borderWidth=0.5,
borderPadding=4,
)
TABLE_HEADER = ParagraphStyle(
"TH", parent=BODY, fontName="Helvetica-Bold", fontSize=9, leading=12,
textColor=colors.white,
)
TABLE_CELL = ParagraphStyle(
"TC", parent=BODY, fontSize=9, leading=12, spaceBefore=0, spaceAfter=0,
)
TABLE_CELL_MONO = ParagraphStyle(
"TCM", parent=TABLE_CELL, fontName="Courier", fontSize=8.5,
)
# --------------------------------------------------------------------------- #
# Helpers #
# --------------------------------------------------------------------------- #
def p(text, style=BODY):
return Paragraph(text, style)
def code_block(lines):
body = "<br/>".join(
line.replace(" ", "&nbsp;").replace("<", "&lt;").replace(">", "&gt;")
for line in lines
)
body = body.replace("&lt;br/&gt;", "<br/>")
return Paragraph(body, CODE)
def make_table(header, rows, col_widths=None, mono_cols=None):
mono_cols = mono_cols or set()
def cell(text, col_idx):
style = TABLE_CELL_MONO if col_idx in mono_cols else TABLE_CELL
return Paragraph(text, style)
data = [[Paragraph(h, TABLE_HEADER) for h in header]]
for row in rows:
data.append([cell(c, i) for i, c in enumerate(row)])
t = Table(data, colWidths=col_widths, repeatRows=1)
t.setStyle(TableStyle([
("BACKGROUND", (0, 0), (-1, 0), colors.HexColor("#0B3D91")),
("TEXTCOLOR", (0, 0), (-1, 0), colors.white),
("ALIGN", (0, 0), (-1, -1), "LEFT"),
("VALIGN", (0, 0), (-1, -1), "TOP"),
("FONTSIZE", (0, 0), (-1, -1), 9),
("BOTTOMPADDING", (0, 0), (-1, 0), 6),
("TOPPADDING", (0, 0), (-1, 0), 6),
("BOTTOMPADDING", (0, 1), (-1, -1), 4),
("TOPPADDING", (0, 1), (-1, -1), 4),
("LEFTPADDING", (0, 0), (-1, -1), 5),
("RIGHTPADDING", (0, 0), (-1, -1), 5),
("ROWBACKGROUNDS", (0, 1), (-1, -1),
[colors.white, colors.HexColor("#F4F7FB")]),
("GRID", (0, 0), (-1, -1), 0.4, colors.HexColor("#B0B7C3")),
]))
return t
# --------------------------------------------------------------------------- #
# Document content #
# --------------------------------------------------------------------------- #
def build_story():
s = []
# ====================================================================
# Title
# ====================================================================
s.append(p(
"ANSLPR &mdash; ALPRChecker &amp; Plate Format Developer Guide",
H1,
))
s.append(p(
"Module: <b>ANSLPR</b> &nbsp;&middot;&nbsp; "
"Base class: <b>ANSCENTER::ANSALPR</b> &nbsp;&middot;&nbsp; "
"Affects every engine subclass "
"(<b>ANSALPR_OD</b>, <b>ANSALPR_OCR</b>, <b>ANSALPR_CPU</b>)",
NOTE,
))
s.append(Spacer(1, 6))
s.append(p(
"This guide documents two independent post-processing features in the "
"ANSLPR pipeline: the <b>ALPRChecker</b> (per-track majority voting + "
"lock + skip-when-locked) and the <b>plate format validator</b> "
"(<font face='Courier'>SetPlateFormat</font> / "
"<font face='Courier'>SetPlateFormats</font>). "
"They share neither state nor toggle; the format validator runs as a "
"filter <i>before</i> the ALPRChecker voter sees any text."
))
# ====================================================================
# PART 1 — ALPRChecker
# ====================================================================
s.append(p("Part 1 &mdash; ALPRChecker (tracker-based voting)", H1))
# ── 1.1 Purpose ──────────────────────────────────────────────────────
s.append(p("1.1 Purpose", H2))
s.append(p(
"Convert noisy per-frame OCR strings for the same physical plate "
"(&ldquo;50H 304.07&rdquo;, &ldquo;50H304O7&rdquo;, &ldquo;50H304.07&rdquo;&hellip;) "
"into a single stable <i>locked</i> text. The mechanism is layered:"
))
s.append(p(
"<b>track identity</b> &rarr; <b>rolling history per track</b> &rarr; "
"<b>recency-weighted majority vote</b> &rarr; <b>stable lock with "
"edit-distance noise rejection</b> &rarr; <b>periodic re-verify</b> "
"so wrong locks can self-correct."
))
# ── 1.2 Default state ────────────────────────────────────────────────
s.append(p("1.2 Default state", H2))
s.append(make_table(
["Flag", "Default", "Source"],
[
["_enableALPRChecker",
"<b>false</b> (opt-in)",
"Member initializer in ANSLPR.h "
"(<font face='Courier'>std::atomic&lt;bool&gt; "
"_enableALPRChecker{false}</font>)"],
["Full-frame auto-detect",
"engages after a few pixel-identical frames "
"(only relevant once enabled)",
"shouldUseALPRChecker / ImageSizeTracker"],
["Skip-when-locked re-verify period",
"<b>30 frames</b>",
"tryReuseLockedText default <font face='Courier'>reverifyEvery=30</font>"],
],
col_widths=[55*mm, 50*mm, 63*mm],
mono_cols={0},
))
s.append(Spacer(1, 4))
s.append(p(
"<b>Out of the box:</b> ALPRChecker is <b>OFF</b>. The pipeline emits "
"raw recognizer text per frame &mdash; no voting, no lock, no "
"skip-when-locked, no spatial dedup. Callers must opt in with "
"<font face='Courier'>SetALPRCheckerEnabled(true)</font> "
"(or <font face='Courier'>ANSALPR_SetALPRCheckerEnabled(h, 1)</font>) "
"to enable cross-frame stabilization. Once enabled, full-frame "
"auto-detect still gates it on per camera, so pre-cropped pipeline "
"calls remain stateless even when the flag is on.",
NOTE,
))
# ── 1.3 Public API ───────────────────────────────────────────────────
s.append(p("1.3 Public API", H2))
s.append(p("<b>C++ class API</b> (<font face='Courier'>ANSCENTER::ANSALPR</font>):"))
s.append(make_table(
["Method", "Effect"],
[
["void SetALPRCheckerEnabled(bool enable)",
"Atomic flip of <font face='Courier'>_enableALPRChecker</font>. "
"Logs <font face='Courier'>[ALPR_Checker]</font> with prev/new."],
["bool IsALPRCheckerEnabled() const",
"Atomic read. Logs the current value."],
],
col_widths=[80*mm, 88*mm],
mono_cols={0},
))
s.append(Spacer(1, 4))
s.append(p("<b>C exports</b> (LabVIEW / .NET / other clients):"))
s.append(make_table(
["Function", "Args", "Return"],
[
["ANSALPR_SetALPRCheckerEnabled",
"Handle, int enable (0|1)",
"1 OK / -1 invalid handle / 0 exception"],
["ANSALPR_GetALPRCheckerEnabled",
"Handle",
"1 enabled / 0 disabled / -1 invalid handle"],
],
col_widths=[60*mm, 55*mm, 53*mm],
mono_cols={0, 1},
))
# ── 1.4 State (data structures) ──────────────────────────────────────
s.append(p("1.4 Internal state (per ANSALPR instance)", H2))
s.append(code_block([
"class ALPRChecker {",
" int maxFrames = 60; // rolling history depth",
" int minVotesToStabilize = 3; // identical reads to lock",
" float iouMatchThreshold = 0.30f; // spatial match (legacy API)",
" std::recursive_mutex _mutex; // protects all maps",
"",
" // trackId-based primary path",
" unordered_map<string, unordered_map<int, TrackedPlateById>>",
" trackedPlatesById;",
"",
" // spatial-IoU legacy path",
" unordered_map<string, vector<TrackedPlate>> trackedPlates;",
"};",
"",
"struct TrackedPlateById {",
" int trackId;",
" deque<string> textHistory; // RAW reads, capped",
" string lockedText; // \"\" until locked",
" int lockCount;",
" int framesSinceLastSeen; // for prune",
" int framesSinceVerify; // for skip-when-locked",
"};",
]))
# ── 1.5 Algorithm ────────────────────────────────────────────────────
s.append(p("1.5 Core algorithm &mdash; checkPlateByTrackId", H2))
s.append(p(
"Primary entry point. Hybrid identity strategy: trackId hash lookup "
"(O(1) common case), Levenshtein-1 fallback for lost tracks.",
))
s.append(code_block([
"LOCK _mutex",
"if detectedPlate empty: return \"\"",
"",
"plates = trackedPlatesById[cameraId]",
"for (id, p) in plates: p.framesSinceLastSeen++ // age all",
"",
"// Periodic prune: every 30 calls, drop entries unseen > 180 frames",
"_pruneCounterById++",
"if _pruneCounterById >= 30 and plates.size() > 20:",
" drop entries where framesSinceLastSeen > maxFrames * 3",
"",
"// PRIMARY: O(1) trackId hit",
"if trackId in plates:",
" tp = plates[trackId]",
" tp.framesSinceLastSeen = 0",
" tp.textHistory.push_back(detectedPlate) // RAW",
" if size > maxFrames: pop_front()",
"",
" voted = majorityVote(tp.textHistory)",
"",
" if tp.lockedText empty:",
" if count(history == voted) >= minVotesToStabilize: // 3",
" tp.lockedText = voted // LOCK",
" return voted",
" else:",
" if Levenshtein(detectedPlate, lockedText) == 0:",
" tp.lockCount++; return lockedText",
" elif Levenshtein(voted, lockedText) > 1:",
" // Re-lock if 3 of last 6 votes agree on new text",
" if count(recent6 == voted) >= 3:",
" tp.lockedText = voted // RE-LOCK",
" return lockedText",
" return lockedText // emit lock, ignore noise",
"",
"// FALLBACK: trackId changed (occlusion, ID switch). Migrate.",
"for (id, p) in plates:",
" if Levenshtein(detectedPlate, p.lockedText) <= 1:",
" migrate p to plates[trackId]; erase old id",
" return p.lockedText",
" elif p.lockedText empty and any of last 3 history within edit-1:",
" migrate; return majorityVote(migrated.history)",
"",
"// No match anywhere: fresh entry",
"plates[trackId] = { trackId, history=[detectedPlate], locked=\"\" }",
"return detectedPlate",
]))
s.append(p("1.6 majorityVote", H3))
s.append(p(
"Recency-weighted vote with Levenshtein-1 group boosting. Newest "
"entries get weight 2.0, oldest 1.0. Variants within edit distance 1 "
"donate 0.5&times; their score to each candidate, so 1-character "
"OCR drift fuses instead of competing."
))
s.append(code_block([
"weightedFreq = {}",
"for i in 0..n-1:",
" weight = 1.0 + i / max(1, n-1) // newest=2.0, oldest=1.0",
" weightedFreq[history[i]] += weight",
"",
"for kv in weightedFreq:",
" total = kv.score",
" for other in weightedFreq:",
" if abs(len(kv) - len(other)) <= 1",
" and Levenshtein(kv, other) == 1:",
" total += other.score * 0.5",
" track best",
"return best",
]))
s.append(p("1.7 tryReuseLockedText &mdash; skip-when-locked fast path", H3))
s.append(p(
"Saves ~50&ndash;95&nbsp;ms per locked plate per frame by skipping "
"the OCR call entirely when a stable lock exists. Periodic re-verify "
"(every 30 frames by default) lets one OCR run through so a wrong "
"lock can self-correct via the normal voting path."
))
s.append(code_block([
"LOCK _mutex",
"tp = trackedPlatesById[cameraId][trackId] // miss -> return \"\"",
"if tp.lockedText empty: return \"\" // not locked yet",
"",
"if reverifyEvery > 0 and tp.framesSinceVerify >= reverifyEvery:",
" tp.framesSinceVerify = 0",
" return \"\" // signal: caller runs OCR this frame",
"",
"tp.framesSinceVerify++",
"tp.framesSinceLastSeen = 0 // critical: keep entry alive across",
" // long skip-when-locked windows",
"return tp.lockedText // skip OCR; emit lock directly",
]))
# ── 1.8 Pipeline integration ─────────────────────────────────────────
s.append(PageBreak())
s.append(p("1.8 Pipeline integration", H2))
s.append(p(
"<b>Pipeline mode always bypasses the ALPRChecker, regardless of "
"how <font face='Courier'>_enableALPRChecker</font> is set.</b> Two "
"independent mechanisms enforce this:"
))
s.append(p(
"<b>Mechanism 1 &mdash; dedicated batch entry point (hard bypass).</b> "
"<font face='Courier'>ANSALPR_RunInferencesBatch_LV</font> / "
"<font face='Courier'>_LV_V2</font> "
"(<font face='Courier'>dllmain.cpp:836, 1405</font>) routes through "
"<font face='Courier'>engine-&gt;RunInferencesBatch(...)</font>. The "
"implementations at "
"<font face='Courier'>ANSALPR_OD::RunInferencesBatch</font> "
"(line 2637) and "
"<font face='Courier'>ANSALPR_OCR::RunInferencesBatch</font> "
"(line 1863) contain <b>zero references</b> to the checker &mdash; "
"no <font face='Courier'>shouldUseALPRChecker</font>, no "
"<font face='Courier'>checkPlateByTrackId</font>, no "
"<font face='Courier'>tryReuseLockedText</font>, no "
"<font face='Courier'>ensureUniquePlateText</font>. The flag is "
"never read on this path. Header contract at "
"<font face='Courier'>ANSLPR.h:163-178</font>: "
"&ldquo;Tracker, voting, spatial dedup, and per-camera "
"accumulating state are all bypassed.&rdquo;"
))
s.append(p(
"<b>Mechanism 2 &mdash; auto-detect floors on the regular "
"<font face='Courier'>RunInference</font> path.</b> "
"<font face='Courier'>shouldUseALPRChecker</font> applies a chain "
"of gates; the last two reject pipeline crops even when the "
"flag is on:"
))
s.append(code_block([
"if !_enableTrackerVoting: return false // OD master (1)",
"if !_enableALPRChecker.load(relaxed): return false // user flag (2)",
"if imageSize.width < MIN_FULLFRAME_WIDTH: return false // HARD FLOOR (3)",
" // = 1000 px",
"tracker = _imageSizeTrackers[cameraId]",
"if imageSize == tracker.lastSize: // CONSISTENCY (4)",
" if ++tracker.consistentCount >= CONFIRM_THRESHOLD: // = 5 frames",
" tracker.detectedFullFrame = true",
"else:",
" tracker.lastSize = imageSize; consistentCount = 1 // reset on any",
" tracker.detectedFullFrame = false // mismatch",
"return tracker.detectedFullFrame",
]))
s.append(p(
"Gates (3) and (4) make the bypass automatic for cropped inputs:"
))
s.append(make_table(
["Pipeline indicator", "Effect", "Independent of flag?"],
[
["<font face='Courier'>width &lt; 1000 px</font> "
"(<font face='Courier'>MIN_FULLFRAME_WIDTH</font>)",
"Returns <font face='Courier'>false</font> immediately, "
"before the auto-detect state machine runs.",
"<b>Yes</b> &mdash; rejects all small crops unconditionally "
"after the flag check passes."],
["Frame size varies between calls",
"<font face='Courier'>consistentCount</font> resets to 1 on "
"every mismatch; <font face='Courier'>detectedFullFrame</font> "
"stays <font face='Courier'>false</font>. Pre-cropped vehicle "
"ROIs (e.g. 496&times;453, 497&times;455, &hellip;) never "
"accumulate consistency.",
"<b>Yes</b> &mdash; the consistency requirement is independent "
"of the user flag."],
],
col_widths=[60*mm, 70*mm, 38*mm],
))
s.append(Spacer(1, 4))
s.append(p(
"<b>Practical guarantee.</b> Even if a caller sets "
"<font face='Courier'>SetALPRCheckerEnabled(true)</font> and feeds "
"pipeline crops, voting stays off per camera because the auto-detect "
"never flips <font face='Courier'>detectedFullFrame</font> to true. "
"Mixing full-frame and pipeline calls on different cameras works "
"cleanly: the per-camera <font face='Courier'>_imageSizeTrackers</font> "
"entry tracks each camera's mode independently.",
NOTE,
))
s.append(p("Caller path &times; flag matrix", H3))
s.append(make_table(
["Caller path", "Input characteristic",
"Flag = OFF", "Flag = ON"],
[
["<font face='Courier'>RunInferencesBatch</font> "
"(pipeline batch, LV/V2)",
"any size, any consistency",
"<b>bypassed</b> (hard)",
"<b>bypassed</b> (hard, flag never read)"],
["<font face='Courier'>RunInference</font>",
"full video frames "
"(&ge; 1000 px, consistent size)",
"bypassed",
"active after 5 consistent frames"],
["<font face='Courier'>RunInference</font>",
"cropped vehicle ROI "
"(&lt; 1000 px)",
"bypassed",
"<b>bypassed by MIN_FULLFRAME_WIDTH floor</b>"],
["<font face='Courier'>RunInference</font>",
"varying-size crops "
"(&ge; 1000 px but inconsistent)",
"bypassed",
"<b>bypassed by consistency reset</b>"],
],
col_widths=[55*mm, 50*mm, 30*mm, 33*mm],
mono_cols={2, 3},
))
s.append(Spacer(1, 4))
s.append(p(
"In effect, the flag answers a single question: <i>&ldquo;if the "
"caller is sending real video frames from this camera, should I "
"stabilize the output?&rdquo;</i> It has no effect on pipeline / "
"cropped inputs &mdash; those are always raw OCR.",
NOTE,
))
s.append(p("1.9 OCR engine wiring (per frame)", H3))
s.append(code_block([
"useChecker = shouldUseALPRChecker(frameSize, cameraId) // once",
"",
"for each detected plate box:",
" if useChecker:",
" locked = tryReuseLockedText(cam, trackId)",
" if locked not empty:",
" emit { className=locked, +color }; SKIP CROP",
" continue",
" build crop(s) (padded, optionally rectified, top/bottom split)",
"",
"ONE batched recognizer call across all crops",
"",
"for each plate (non-skipped):",
" text = StripPlateArtifacts(top + \" \" + bottom)",
" if Country == JAPAN and IsJapaneseIncomplete(text):",
" text = recover kana from bottom-half tile",
" text = StripPlateSeparators(text)",
" text = AnalyseLicensePlateText(text) // <-- format filter",
" if text empty: continue // (drops invalid)",
"",
" if useChecker:",
" className = checkPlateByTrackId(cam, text, trackId)",
" else:",
" className = text // raw",
" emit { className, +color }",
"",
"if useChecker:",
" ensureUniquePlateText(output, cam) // spatial dedup",
]))
s.append(p("1.10 OD engine wiring", H3))
s.append(p(
"Same gate at every <font face='Courier'>alprChecker.checkPlateByTrackId</font> "
"call. The OD path doesn't use skip-when-locked because it already "
"runs character classification per detection &mdash; nothing to skip. "
"Note OD also has a separate master "
"<font face='Courier'>_enableTrackerVoting</font> "
"flag checked <i>before</i> <font face='Courier'>_enableALPRChecker</font>; "
"if that one is off, the disabled-path applies regardless."
))
# ── 1.11 Disabled mode ──────────────────────────────────────────────
s.append(p("1.11 Disabled mode (default) vs enabled &mdash; what runs", H2))
s.append(make_table(
["Stage", "Enabled (opt-in)", "Disabled (default)"],
[
["LP detection",
"Runs", "Runs"],
["ROI cropping + 5% padding",
"Runs", "Runs"],
["Japan rectification + kana recovery",
"Runs (Japan only)", "Runs (Japan only)"],
["Batched recognizer call",
"Runs", "Runs"],
["StripPlateArtifacts / StripPlateSeparators",
"Runs", "Runs"],
["AnalyseLicensePlateText (format filter)",
"Runs (independent toggle)", "Runs (independent toggle)"],
["Skip-when-locked (tryReuseLockedText)",
"Runs", "<b>Bypassed</b>"],
["Per-track voting (checkPlateByTrackId)",
"Runs", "<b>Bypassed</b> &mdash; raw recognizer text"],
["Spatial dedup (ensureUniquePlateText)",
"Runs", "<b>Bypassed</b>"],
["trackedPlatesById state",
"Populated and pruned", "Untouched (existing entries frozen)"],
],
col_widths=[60*mm, 53*mm, 55*mm],
))
s.append(Spacer(1, 4))
s.append(p(
"<b>Cost.</b> Disabling skip-when-locked adds ~50&ndash;95&nbsp;ms "
"per locked plate per frame because every plate now goes through "
"OCR every frame.",
NOTE,
))
# ── 1.12 Thread safety ──────────────────────────────────────────────
s.append(p("1.12 Thread safety", H2))
s.append(make_table(
["Concern", "Resolution"],
[
["Toggle race between config thread and inference thread",
"<font face='Courier'>_enableALPRChecker</font> is "
"<font face='Courier'>std::atomic&lt;bool&gt;</font>; "
"store/load use <font face='Courier'>memory_order_relaxed</font>. "
"Tear-free, no fence cost."],
["Concurrent track map mutation",
"<font face='Courier'>ALPRChecker::_mutex</font> "
"(recursive) wraps every read &amp; write of "
"<font face='Courier'>trackedPlatesById</font> / "
"<font face='Courier'>trackedPlates</font>."],
["Concurrent spatial dedup state",
"<font face='Courier'>_plateIdentitiesMutex</font> in "
"<font face='Courier'>ANSALPR_OCR</font> protects "
"<font face='Courier'>_plateIdentities</font>."],
["Toggle while OCR is in flight",
"Safe. The in-flight frame uses whichever value it observed; "
"the next frame picks up the new value."],
],
col_widths=[60*mm, 108*mm],
))
# ── 1.13 Diagnostic logging ─────────────────────────────────────────
s.append(p("1.13 Diagnostic logging (DebugView)", H2))
s.append(make_table(
["Tag", "Emitted by"],
[
["[ALPR_Checker]",
"Set/Get accessors and C-export wrappers — every flag transition "
"and every read."],
["[ALPR_TrackId]",
"checkPlateByTrackId — LOCKED, RE-LOCK, MIGRATED, "
"noise, NEW; tryReuseLockedText — SKIP-LOCKED, REVERIFY."],
["[ALPR_OCR_Checker]",
"shouldUseALPRChecker — full-frame vs pipeline mode auto-detect "
"transitions per camera."],
["[ALPR_Format]",
"AnalyseLicensePlateText — REJECT lines when a plate doesn't "
"match any configured format (see Part 2)."],
["[OCR_Leak]",
"Once-per-60s heartbeat with sizes of each map — useful for "
"spotting unbounded growth in production."],
],
col_widths=[40*mm, 128*mm],
mono_cols={0},
))
s.append(Spacer(1, 4))
s.append(p(
"Tags are gated behind <font face='Courier'>ANSCORE_DEBUGVIEW</font> "
"and <font face='Courier'>ANSCENTER_IsDebugViewEnabled()</font>. "
"Filter <i>DebugView</i> on the bracketed tag to isolate any layer.",
NOTE,
))
# ====================================================================
# PART 2 — Plate format
# ====================================================================
s.append(PageBreak())
s.append(p("Part 2 &mdash; Plate format validation", H1))
# ── 2.1 Purpose ──────────────────────────────────────────────────────
s.append(p("2.1 Purpose", H2))
s.append(p(
"Reject OCR output that doesn't match any configured plate pattern. "
"Acts as a hard filter <b>upstream</b> of the ALPRChecker voter, so "
"malformed reads never enter <font face='Courier'>textHistory</font> "
"and a track never locks on garbage."
))
# ── 2.2 Pattern syntax ───────────────────────────────────────────────
s.append(p("2.2 Pattern syntax", H2))
s.append(p(
"Patterns are simple character templates. Each character in the "
"pattern matches one character in the candidate plate; lengths must "
"match exactly."
))
s.append(make_table(
["Token", "Matches", "Notes"],
[
["d", "Any digit (<font face='Courier'>0&ndash;9</font>)",
"<font face='Courier'>std::isdigit</font>"],
["l", "Any letter (<font face='Courier'>A&ndash;Z, a&ndash;z</font>)",
"<font face='Courier'>std::isalpha</font>. Plate text is "
"uppercased before matching."],
["any other char",
"Literal &mdash; must match exactly",
"Use uppercase literals (e.g. <font face='Courier'>M</font>, "
"<font face='Courier'>D</font>, <font face='Courier'>NG</font>) "
"since the plate is uppercased first."],
],
col_widths=[20*mm, 80*mm, 68*mm],
mono_cols={0},
))
s.append(Spacer(1, 4))
s.append(p(
"<b>Empty format list</b> = no validation; all plates accepted "
"(<font face='Courier'>MatchesPlateFormat</font> returns true). "
"Calling <font face='Courier'>SetPlateFormats(emptyList)</font> "
"disables filtering.",
NOTE,
))
s.append(p("2.3 Examples", H3))
s.append(make_table(
["Pattern", "Matches", "Doesn't match"],
[
["ddlddddd",
"30A12345 &nbsp;|&nbsp; 51F67890",
"30A1234 (too short) &nbsp;|&nbsp; 30AA12345 "
"(2nd letter where digit expected)"],
["lldddd",
"AB1234 &nbsp;|&nbsp; QC9876",
"A12345 (1st char must be letter) &nbsp;|&nbsp; AB12345 "
"(too long)"],
["dddddNGdd",
"12345NG67",
"12345NN67 (literal mismatch)"],
["ddMDdddddd",
"12MD345678",
"12mD345678 (case mismatch on literal &mdash; plate is "
"uppercased, so 'D' must be 'D')"],
],
col_widths=[35*mm, 65*mm, 68*mm],
mono_cols={0, 1, 2},
))
# ── 2.4 Default formats per country ─────────────────────────────────
s.append(p("2.4 Default formats per country", H2))
s.append(p(
"<font face='Courier'>ApplyCountryDefaultPlateFormats()</font> "
"populates the format list automatically based on "
"<font face='Courier'>_country</font>. Called at the end of "
"<font face='Courier'>Initialize()</font> (after country.txt is "
"read) and from <font face='Courier'>SetCountry()</font>, so a "
"runtime country switch picks up the right format set on the very "
"next frame."
))
s.append(make_table(
["Country", "Formats installed"],
[
["VIETNAM",
"ddlddddd, ddldddd, ddldddddd, ddllddddd, ddMDdddddd, "
"dddddNGdd, dddddQTdd, dddddCVdd, dddddNNdd, lldddd "
"(11 patterns covering car / motorbike / military / "
"diplomatic / police)"],
["JAPAN, CHINA, AUSTRALIA, USA, INDONESIA, &hellip;",
"<i>Empty</i> &mdash; no validation. "
"<font face='Courier'>AnalyseLicensePlateText</font> is also "
"a near-no-op for non-Vietnam countries (preserves kana / "
"kanji / region-specific punctuation verbatim)."],
],
col_widths=[55*mm, 113*mm],
))
# ── 2.5 Public API ──────────────────────────────────────────────────
s.append(p("2.5 Public API", H2))
s.append(p("<b>C++ class API</b> (<font face='Courier'>ANSCENTER::ANSALPR</font>):"))
s.append(make_table(
["Method", "Effect"],
[
["void SetPlateFormat(const string&amp; format)",
"Replace the list with a single pattern."],
["void SetPlateFormats(const vector&lt;string&gt;&amp; formats)",
"Replace the entire list. Pass empty vector to disable validation."],
["vector&lt;string&gt; GetPlateFormats() const",
"Returns the current list."],
["virtual void SetCountry(Country c)",
"Calls ApplyCountryDefaultPlateFormats() so the list resets to "
"the country's defaults."],
],
col_widths=[80*mm, 88*mm],
mono_cols={0},
))
s.append(Spacer(1, 4))
s.append(p("<b>C exports</b>:"))
s.append(make_table(
["Function", "Args", "Return"],
[
["ANSALPR_SetFormat",
"Handle, const char* format",
"1 OK / -1 invalid handle / 0 exception"],
["ANSALPR_SetFormats",
"Handle, const char* formats "
"(<i>semicolon-separated</i>, e.g. \"ddlddddd;ddldddd;lldddd\")",
"1 OK / -1 invalid handle / 0 exception"],
["ANSALPR_GetFormats",
"Handle, LStrHandle outFormats",
"Count written, semicolon-separated"],
["ANSALPR_SetCountry",
"Handle, int country "
"(0=VN, 1=CN, 2=AU, 3=US, 4=ID, 5=JP)",
"1 OK / -1 invalid handle / 0 exception"],
],
col_widths=[55*mm, 65*mm, 48*mm],
mono_cols={0, 1},
))
# ── 2.6 Where it runs ───────────────────────────────────────────────
s.append(PageBreak())
s.append(p("2.6 Where it runs in the pipeline", H2))
s.append(p(
"Sequential post-processing inside the per-plate loop &mdash; "
"<b>before</b> the ALPRChecker voter:"
))
s.append(code_block([
"topText, bottomText = OCR results per row",
"combinedText = StripPlateArtifacts(top + \" \" + bottom) // °, ○, screws",
"[Japan] kana recovery if hiragana missing",
"combinedText = StripPlateSeparators(combinedText) // -, comma, ws",
" v",
"combinedText = AnalyseLicensePlateText(combinedText) // <-- PART 2",
" 1. keep only alnum chars",
" 2. uppercase",
" 3. if _plateFormats not empty:",
" if no pattern matches: log REJECT, return \"\"",
" return cleaned plate",
" v",
"if combinedText empty: continue // dropped — voter never sees it",
" v",
"if useChecker: // <-- PART 1",
" className = checkPlateByTrackId(cam, combinedText, trackId)",
"else:",
" className = combinedText",
]))
s.append(Spacer(1, 4))
s.append(p(
"<b>Consequence.</b> When a format list is configured, the voter's "
"<font face='Courier'>textHistory</font> contains only format-valid "
"strings. <font face='Courier'>lockedText</font> is therefore "
"always format-valid by construction. A track that consistently "
"produces format-invalid OCR will never lock &mdash; it just never "
"reaches the voter, and no output is emitted for that plate.",
NOTE,
))
# ── 2.7 Country interaction ─────────────────────────────────────────
s.append(p("2.7 Country interaction", H2))
s.append(p(
"<font face='Courier'>AnalyseLicensePlateText</font> behaves "
"differently per country (see "
"<font face='Courier'>ANSALPR_CPU.cpp</font>, "
"<font face='Courier'>ANSALPR_OCR.cpp</font>, "
"<font face='Courier'>ANSALPR_OD.cpp</font>):"
))
s.append(make_table(
["Country", "AnalyseLicensePlateText behavior"],
[
["VIETNAM",
"Strips non-alnum, uppercases, applies suburb-code remapping "
"(13&rarr;73, 42&rarr;62, 44&rarr;64, &hellip;), then validates "
"against <font face='Courier'>_plateFormats</font>. Mismatch &rarr; \"\""],
["Other countries",
"Returns input unchanged so kana / kanji / region punctuation "
"flow through verbatim. <font face='Courier'>_plateFormats</font> "
"is empty by default for these countries; even if you populate "
"it manually, the format check still runs in OCR (same code "
"path) but is currently a no-op for non-Vietnam in OD."],
],
col_widths=[35*mm, 133*mm],
))
# ── 2.8 Combined behavior matrix ────────────────────────────────────
s.append(p("2.8 ALPRChecker &times; format combined behavior", H2))
s.append(make_table(
["Format set", "ALPRChecker", "Behavior"],
[
["empty",
"OFF",
"Raw OCR every frame. No filter, no voting. "
"Highest noise, highest recall."],
["empty",
"ON",
"Voted/locked text. Voter learns from raw recognizer output "
"&mdash; can lock on bad reads if they're consistent."],
["configured",
"OFF",
"<b>Default for Vietnam.</b> Format-filtered raw OCR per frame. "
"Invalid reads dropped, valid ones emitted as-is "
"(no cross-frame memory)."],
["configured",
"ON",
"<b>Recommended for stable cameras.</b> Format filters bad "
"reads out, voter locks the surviving canonical text. Best "
"stability, lowest false-positive rate. Opt in with "
"<font face='Courier'>SetALPRCheckerEnabled(true)</font>."],
],
col_widths=[35*mm, 30*mm, 103*mm],
mono_cols={0, 1},
))
s.append(Spacer(1, 4))
s.append(p(
"<b>Toggling at runtime.</b> Both flags are independent and "
"thread-safe. Existing locks survive a format-set change &mdash; "
"they're not retroactively re-validated. To force a clean slate, "
"destroy and recreate the ANSALPR handle (or wait for the prune "
"pass to expire stale tracks).",
NOTE,
))
# ── 2.9 Caller recipes ──────────────────────────────────────────────
s.append(p("2.9 Caller recipes", H2))
s.append(make_table(
["Goal", "C export sequence"],
[
["Default pipeline (raw OCR + format filter, no voting)",
"Nothing &mdash; defaults are fine."],
["Enable tracker voting + skip-when-locked",
"ANSALPR_SetALPRCheckerEnabled(h, 1)"],
["Switch country at runtime",
"ANSALPR_SetCountry(h, 5) &nbsp; // Japan; resets formats"],
["Vietnam-only police plates",
"ANSALPR_SetFormats(h, \"dddddCVdd\")"],
["Vietnam plus a custom long-form pattern",
"ANSALPR_SetFormats(h, \"ddlddddd;ddldddd;dddddCVdd;ddldddddddd\")"],
["Disable format validation entirely",
"ANSALPR_SetFormats(h, \"\") &nbsp; // empty -> accept all"],
["Disable tracker voting (revert to raw OCR every frame)",
"ANSALPR_SetALPRCheckerEnabled(h, 0)"],
["Read current state",
"ANSALPR_GetALPRCheckerEnabled(h); ANSALPR_GetFormats(h, &amp;buf)"],
],
col_widths=[68*mm, 100*mm],
mono_cols={1},
))
# ── 2.10 Diagnostic logging ─────────────────────────────────────────
s.append(p("2.10 Diagnostic logging (DebugView)", H2))
s.append(code_block([
"[ANSLPR] ANSALPR_SetFormat: HandlePtr=0x... format=ddlddddd",
"[ANSLPR] ANSALPR_SetFormats: HandlePtr=0x... formats=ddlddddd;lldddd",
"[ANSLPR] ANSALPR_SetCountry: HandlePtr=0x... country=0",
"[ALPR_Format] REJECT plate='30AA1234' (no _plateFormats match)",
]))
s.append(p(
"Filter on <font face='Courier'>[ALPR_Format]</font> to count how "
"many plates the format filter is rejecting per camera per minute. "
"A high reject rate with a low ALPR result count means the format "
"list is too tight for what the recognizer is producing.",
NOTE,
))
# ── Footer ──────────────────────────────────────────────────────────
s.append(Spacer(1, 14))
s.append(p(
"<i>This guide reflects the ANSLPR API as of the ALPRChecker atomic "
"+ getter export change. File references point to "
"modules/ANSLPR/ANSLPR.{h,cpp}, modules/ANSLPR/ANSLPR_OCR.{h,cpp}, "
"modules/ANSLPR/ANSLPR_OD.{h,cpp}, modules/ANSLPR/ANSLPR_CPU.cpp, "
"and modules/ANSLPR/dllmain.cpp.</i>",
NOTE,
))
return s
# --------------------------------------------------------------------------- #
# Build #
# --------------------------------------------------------------------------- #
def main():
doc = SimpleDocTemplate(
str(OUTPUT),
pagesize=A4,
leftMargin=18*mm, rightMargin=18*mm,
topMargin=18*mm, bottomMargin=18*mm,
title="ANSLPR — ALPRChecker & Plate Format Developer Guide",
author="ANSCORE",
)
story = build_story()
doc.build(story)
print(f"Wrote {OUTPUT}")
if __name__ == "__main__":
main()
File diff suppressed because it is too large Load Diff
+267 -34
View File
@@ -32,34 +32,32 @@
// filesystem check runs at most once per ~2 s.
extern "C" ANSLICENSE_API int ANSCENTER_IsDebugViewEnabled(void);
// C-linkage facade over SPDLogger so consumers can dynamic-load these via
// GetProcAddress without depending on the C++ class ABI. Each call routes to
// the process-wide SPDLogger singleton (thread-safe magic-static init).
extern "C" {
ANSLICENSE_API void ANSLogger_LogInfo (const char* src, const char* msg, const char* file, int line);
ANSLICENSE_API void ANSLogger_LogError(const char* src, const char* msg, const char* file, int line);
ANSLICENSE_API void ANSLogger_LogFatal(const char* src, const char* msg, const char* file, int line);
// DebugView choke point shared by ANS_DBG and ANSLIB_DBG. Strips noise
// path prefixes before emitting to OutputDebugStringA + stderr.
ANSLICENSE_API void ANSCENTER_DebugLog(const char* preformattedLine);
}
// ANS_DBG: Debug logging macro for DebugView (OutputDebugStringA on Windows).
// Usage: ANS_DBG("MyModule", "value=%d ptr=%p", val, ptr);
// Output: [MyModule] value=42 ptr=0x1234
// NOTE: We avoid #include <windows.h> here to prevent winsock.h/winsock2.h
// conflicts. Instead, forward-declare OutputDebugStringA directly.
#if ANSCORE_DEBUGVIEW && defined(_WIN32)
extern "C" __declspec(dllimport) void __stdcall OutputDebugStringA(const char* lpOutputString);
// Emit to BOTH DebugView (OutputDebugStringA) AND stderr so console apps
// (ANSCV-UnitTest, ANSLPR-UnitTest, ANSUtilities_test, etc.) show the message
// without needing to attach DebugView. fputs + fflush keeps lines atomic even
// when multiple threads log concurrently.
//
// Routes through ANSCENTER_DebugLog so noise path prefixes (e.g. the
// ProgramData profile dir) are stripped centrally before emit. The macro just
// formats on the caller's stack and hands off the buffer.
#if ANSCORE_DEBUGVIEW
#define ANS_DBG(tag, fmt, ...) do { \
if (ANSCENTER_IsDebugViewEnabled()) { \
char _ans_dbg_buf[1024]; \
snprintf(_ans_dbg_buf, sizeof(_ans_dbg_buf), "[" tag "] " fmt "\n", ##__VA_ARGS__); \
OutputDebugStringA(_ans_dbg_buf); \
fputs(_ans_dbg_buf, stderr); \
fflush(stderr); \
} \
} while(0)
#elif ANSCORE_DEBUGVIEW
// Non-Windows: stderr only.
#define ANS_DBG(tag, fmt, ...) do { \
if (ANSCENTER_IsDebugViewEnabled()) { \
char _ans_dbg_buf[1024]; \
snprintf(_ans_dbg_buf, sizeof(_ans_dbg_buf), "[" tag "] " fmt "\n", ##__VA_ARGS__); \
fputs(_ans_dbg_buf, stderr); \
fflush(stderr); \
ANSCENTER_DebugLog(_ans_dbg_buf); \
} \
} while(0)
#else
@@ -243,7 +241,9 @@ namespace ANSCENTER
ONNXSAM3=28,
RTSAM3 = 29,
ONNXYOLO = 30,
RTYOLO = 31
RTYOLO = 31,
ONNXRTMPOSE=32,
RTRTMPOSE=33
};
enum TrackerType {
BYTETRACK = 0,
@@ -268,18 +268,28 @@ namespace ANSCENTER
UNLICENSE = 3
};
struct ModelConfig {
DetectionType detectionType;
ModelType modelType;
PrecisionType precisionType;
int inpWidth;
int inpHeight;
float modelConfThreshold;
float modelMNSThreshold;
float detectionScoreThreshold;
int numKPS; // Number of pose keypoint
float kpsThreshold;
bool autoGPUDetection;
float unknownPersonThreshold;
// All POD members default-initialized. Several callers (notably
// CreateANSODHandle in ANSODEngine/dllmain.cpp) declare a local
// ModelConfig and only set a subset of fields before passing it
// to ANSODBase::Initialize. Without these defaults inpWidth /
// inpHeight enter Initialize() as indeterminate stack values; the
// existing `<=0 → 640` guard in ANSONNXYOLO::Initialize only
// catches non-positive garbage, so a small positive stack value
// (e.g. 80) would slip through and end up driving the ORT
// session at 80x80 — which crashes YOLOv11 in /model.12/Concat
// because 2*ceil(80/32) ≠ ceil(80/16) in the FPN.
DetectionType detectionType{};
ModelType modelType{};
PrecisionType precisionType{};
int inpWidth = 0;
int inpHeight = 0;
float modelConfThreshold = 0.0f;
float modelMNSThreshold = 0.0f;
float detectionScoreThreshold = 0.0f;
int numKPS = 0; // Number of pose keypoint
float kpsThreshold = 0.0f;
bool autoGPUDetection = false;
float unknownPersonThreshold = 0.0f;
int gpuOptBatchSize = 1; // The batch size which should be optimized for.
int gpuMaxBatchSize = 1; // Maximum allowable batch size
int gpuDeviceIndex = 0; // GPU device index
@@ -309,6 +319,110 @@ namespace ANSCENTER
int enableFeature;
};
// ------------------------------------------------------------------------
// Real-time system telemetry. Snapshot of CPU / GPU / RAM / disk / network
// usage at the time of the call.
//
// Cheap to call frequently — values are cached for ~500 ms via a process-
// wide PDH sampler held in ANSLicense.cpp. The first call after process
// start blocks for ~150 ms because PDH counters require two samples to
// produce a delta-rate reading; subsequent calls are sub-millisecond.
//
// Fields the platform could not provide are set to -1 (or -1.0). Common
// reasons:
// - cpuTempDegC: ACPI thermal zone is unavailable / blocked on most
// consumer hardware. Use a vendor SDK (NVML / ADL / IPMI) for real
// thermals if you need them.
// - netLoadPercent: no network adapter advertises a non-zero
// "Current Bandwidth" (e.g. all interfaces are loopback / virtual).
// - valIGPU_MB: Windows < 10 19H1 does not expose GPU Process Memory
// PDH counters.
//
// Memory / disk units use 1024 base (1 GB == 2^30 bytes); throughput uses
// MB == 2^20 bytes for parity with Task Manager.
// ------------------------------------------------------------------------
struct SystemHealthInfo {
// CPU
// cpuLoadPercent — 0..100 always. Source: PDH "% Processor Time".
// The right value for "what fraction of CPU time
// am I spending in non-idle threads?".
// cpuUtilizationPercent — turbo-aware. Source: PDH "% Processor
// Utility". Can exceed 100% during sustained turbo
// boost. This is what Task Manager displays.
// Pick one to bind to the LabVIEW indicator; cpuUtilizationPercent
// matches what users see in TM but breaks fixed-range UI scaling.
double cpuLoadPercent = -1.0;
double cpuUtilizationPercent = -1.0;
double cpuTempDegC = -1.0;
double cpuFreqGHz = -1.0; // current clock (turbo-aware)
std::string cpuModel; // e.g. "Intel(R) Core(TM) i9-..."
// System memory
double memTotalGB = -1.0;
double memoryUsedGB = -1.0;
double memoryFreeGB = -1.0;
double rssPercent = -1.0; // current process WS / total physical
double swapPercent = -1.0; // page file used / page file limit
// Current process
int64_t valIRSS_MB = -1; // working-set size (MB)
int64_t valSwap_MB = -1; // private commit / page-file usage (MB)
int64_t valIGPU_MB = -1; // dedicated GPU memory used by us (MB)
// GPU — values report the *primary* adapter only. On hybrid systems
// (e.g. Iris Xe + RTX 4090) the primary is picked deterministically:
// NVIDIA discrete > AMD discrete > whichever has the most VRAM. Per-
// adapter numbers are available in gpuAdaptersDetail below.
double gpuLoadPercent = -1.0; // 0..100 on the primary adapter only
std::string gpuName; // primary adapter model name
// Per-adapter rollup; one row per physical GPU, comma-separated.
// Row format: <name>|<vendor>|<load%>|<processMemMB>|<vramMB>
// Vendors: NVIDIA / AMD / Intel / Microsoft / Other
// Numbers use the C locale ('.' decimal, 2dp for load%).
std::string gpuAdaptersDetail;
// Disk (system drive — the volume holding %SystemRoot%)
double diskLoadPercent = -1.0;
double diskReadSpeedMBs = -1.0;
double diskWriteSpeedMBs = -1.0;
double diskUsedGB = -1.0;
double diskTotalGB = -1.0;
// Network (sum across all non-loopback interfaces)
double netLoadPercent = -1.0;
double netReceiveSpeedMBs = -1.0;
double netSendSpeedMBs = -1.0;
};
// ------------------------------------------------------------------------
// Public IP + IP-based geolocation, sourced from ipapi.co/json/.
//
// One HTTPS GET per refresh; results cached for 60 minutes (success) or
// 60 seconds (failure backoff). Never throws, never blocks more than 5 s.
//
// `ok == false` means the data could not be obtained — `error` carries
// the reason ("HTTP 429", "WinHttpConnect failed", "JSON parse: ...",
// etc.). Latitude/longitude are 0,0 when unknown.
//
// Privacy note: this performs an outbound HTTPS request to a third-party
// service (ipapi.co). Each request reveals the device's public IP to that
// service. Use the simpler GetPublicIPAddress() if you only need the IP
// and prefer no third-party HTTP hop.
// ------------------------------------------------------------------------
struct PublicNetworkInfo {
bool ok = false;
std::string publicIP; // e.g. "203.0.113.42"
std::string country; // full name, e.g. "Vietnam"
std::string countryCode; // ISO 2-letter, e.g. "VN"
std::string region; // state / province
std::string city;
std::string isp; // organization / provider name
std::string timezone; // IANA tz, e.g. "Asia/Ho_Chi_Minh"
double latitude = 0.0;
double longitude = 0.0;
std::string error; // empty on success
};
class ANSLICENSE_API SPDLogger
@@ -410,6 +524,25 @@ namespace ANSCENTER
public:
[[nodiscard]] static bool LicenseVerification(std::string licenseKey, int productId, std::string registrationName);
[[nodiscard]] static EngineType CheckHardwareInformation();
[[nodiscard]] static SystemHealthInfo GetSystemHealthInformation();
// Returns the device's public IPv4 address as seen from the internet,
// or an empty string if it can't be resolved (no connectivity, DNS
// blocked, etc). Result is cached for 10 minutes — subsequent calls
// are a memory read.
//
// Implementation uses a single DNS query to OpenDNS's special name
// 'myip.opendns.com' (the resolver echoes back the client's seen IP
// in an A record). No HTTP, no third-party REST, no database — works
// on any network where outbound DNS to 208.67.222.222 is allowed
// (which is the vast majority, including most corporate firewalls).
[[nodiscard]] static std::string GetPublicIPAddress();
// Public IP + geolocation in one shot via ipapi.co/json/. See the
// PublicNetworkInfo doc-comment for caching, failure semantics, and
// privacy implications. If you only need the IP, prefer the simpler
// GetPublicIPAddress() above (no HTTP, no third-party REST).
[[nodiscard]] static PublicNetworkInfo GetPublicNetworkInformation();
[[nodiscard]] static std::string ListLicenses();
[[nodiscard]] static int ActivateTrialLicense(std::string productName);
[[nodiscard]] static int DeactivateLicense(std::string productName);
@@ -447,6 +580,106 @@ extern "C" __declspec(dllexport) int ExtractModelZipFile(const char* modelZipFil
extern "C" __declspec(dllexport) int ZipFolderToZipFile(const char* modelFolderSource,const char* modelZipFileDes, const char* password);
extern "C" __declspec(dllexport) int PrepareEdgeModel(const char* modelZipFile, const char* zipFilePassword, const char* edgePassword, const char* modelName, const char* edgeSerialNumber, LStrHandle pathArray);
extern "C" __declspec(dllexport) int GetSystemHardwareInformation(LStrHandle systemHardwareInformation);
// SystemHealthParameters cluster, one scalar at a time. Pass nullptr for any
// field you do not care about. Returns 1 on success, 0 on failure. Cached
// internally so calling at LabVIEW UI rate (e.g. 1 Hz) is essentially free.
extern "C" __declspec(dllexport) int GetSystemHealthParameters(
double* cpuLoadPercent,
double* cpuTempDegC,
double* cpuFreqGHz,
LStrHandle cpuModel,
double* memTotalGB,
double* rssPercent,
double* swapPercent,
double* gpuLoadPercent,
int64_t* valIRSS_MB,
int64_t* valSwap_MB,
int64_t* valIGPU_MB,
double* memoryUsedGB,
double* memoryFreeGB,
double* diskLoadPercent,
double* diskReadSpeedMBs,
double* diskWriteSpeedMBs,
double* diskUsedGB,
double* diskTotalGB,
double* netLoadPercent,
double* netReceiveSpeedMBs,
double* netSendSpeedMBs,
LStrHandle gpuName, // primary adapter name (e.g. "NVIDIA RTX 4090"); pass nullptr to skip
LStrHandle gpuAdaptersDetail, // per-adapter rollup; pass nullptr to skip — see SystemHealthInfo for format
double* cpuUtilizationPercent); // turbo-aware "% Processor Utility"; matches Task Manager (may exceed 100)
// Same payload as GetSystemHealthParameters, packed into a single string with
// ';' separators in this exact order:
// 1 cpuLoadPercent (%; 0..100, "% Processor Time")
// 2 cpuTempDegC (°C, -1 if unavailable)
// 3 cpuFreqGHz (GHz)
// 4 cpuModel (string; any literal ';' is rewritten to ',')
// 5 memTotalGB
// 6 rssPercent (%)
// 7 swapPercent (%)
// 8 gpuLoadPercent (%)
// 9 valIRSS_MB (current process working set, MB)
// 10 valSwap_MB (current process page-file usage, MB)
// 11 valIGPU_MB (current process GPU dedicated memory, MB)
// 12 memoryUsedGB
// 13 memoryFreeGB
// 14 diskLoadPercent (%)
// 15 diskReadSpeedMBs
// 16 diskWriteSpeedMBs
// 17 diskUsedGB
// 18 diskTotalGB
// 19 netLoadPercent (%)
// 20 netReceiveSpeedMBs
// 21 netSendSpeedMBs
// 22 gpuName (primary adapter model, e.g. "NVIDIA RTX 4090")
// 23 gpuAdaptersDetail (per-adapter rollup — see SystemHealthInfo struct)
// 24 cpuUtilizationPercent (%; turbo-aware, may exceed 100; matches Task Manager)
//
// New fields are appended at the end so existing parsers stay valid. Fields
// 1..23 are stable.
// Numbers use the C locale ('.' decimal point) and 2-decimal fixed format.
// Any literal ';' in gpuName / gpuAdaptersDetail is rewritten to ',' so the
// outer separator is never ambiguous.
// Returns 1 on success, 0 on failure.
extern "C" __declspec(dllexport) int GetSystemHealthParametersString(LStrHandle outString);
// Writes the device's public IPv4 (e.g. "203.0.113.42") into the LStrHandle.
// Returns 1 on success, 0 if the IP could not be resolved. Cached for 10 min.
extern "C" __declspec(dllexport) int GetPublicIPAddress(LStrHandle ipv4);
// Public IP + IP-based geolocation via ipapi.co/json/. Each pointer is
// optional; pass nullptr to skip a field. `errorMsg` carries a human-readable
// reason when the call fails (no internet, rate limit, etc.). Returns 1 on
// success, 0 on failure. Cached for 1 hour on success; 1-minute back-off on
// failure so a flapping network doesn't get hammered.
extern "C" __declspec(dllexport) int GetPublicNetworkParameters(
LStrHandle publicIP,
LStrHandle country,
LStrHandle countryCode,
LStrHandle region,
LStrHandle city,
LStrHandle isp,
LStrHandle timeZone,
double* latitude,
double* longitude,
LStrHandle errorMsg);
// Same payload packed as a single ';'-separated string in this exact order:
// 1 publicIP
// 2 country (full name)
// 3 countryCode (ISO 2-letter)
// 4 region
// 5 city
// 6 isp
// 7 timezone (IANA, e.g. "Asia/Ho_Chi_Minh")
// 8 latitude (6dp, C locale)
// 9 longitude (6dp, C locale)
// Any literal ';' inside string fields is rewritten to ',' so the separator
// stays unambiguous. Returns 1 on success, 0 on failure (string still written
// — values just empty/0).
extern "C" __declspec(dllexport) int GetPublicNetworkParametersString(LStrHandle outString);
extern "C" __declspec(dllexport) int ReleaseLoggers();
extern "C" __declspec(dllexport) int ListANSLicenses(std::string &listLicense);
extern "C" __declspec(dllexport) int ListANSLicenses_LV(LStrHandle ansProductLicenses);
+4
View File
@@ -52,6 +52,10 @@ target_link_libraries(ANSLicensingSystem PRIVATE
bz2.lib zip.lib
libboost_log-vc143-mt-x64-1_82.lib
shlwapi.lib winhttp.lib
pdh.lib psapi.lib
wbemuuid.lib oleaut32.lib ole32.lib
dxgi.lib
dnsapi.lib
${WIN_COMMON_LIBS}
)
+234
View File
@@ -2,6 +2,10 @@
#include "pch.h"
#include "ANSLicense.h"
#include "Utility.h"
#include <sstream>
#include <iomanip>
#include <algorithm>
#include <locale>
BOOL APIENTRY DllMain( HMODULE hModule,
DWORD ul_reason_for_call,
@@ -369,6 +373,236 @@ extern "C" __declspec(dllexport) int GetSystemHardwareInformation(LStrHandle sys
}
}
// SystemHealthParameters cluster, decomposed into individual scalar outputs so
// LabVIEW can wire each indicator straight to a Call Library Function Node.
// Any pointer may be nullptr (LabVIEW will pass nullptr for cluster fields the
// caller hasn't bound). cpuModel is optional but recommended; pass an LStrHandle
// or nullptr to skip. Returns 1 on success, 0 on hard failure.
extern "C" __declspec(dllexport) int GetSystemHealthParameters(
double* cpuLoadPercent,
double* cpuTempDegC,
double* cpuFreqGHz,
LStrHandle cpuModel,
double* memTotalGB,
double* rssPercent,
double* swapPercent,
double* gpuLoadPercent,
int64_t* valIRSS_MB,
int64_t* valSwap_MB,
int64_t* valIGPU_MB,
double* memoryUsedGB,
double* memoryFreeGB,
double* diskLoadPercent,
double* diskReadSpeedMBs,
double* diskWriteSpeedMBs,
double* diskUsedGB,
double* diskTotalGB,
double* netLoadPercent,
double* netReceiveSpeedMBs,
double* netSendSpeedMBs,
LStrHandle gpuName,
LStrHandle gpuAdaptersDetail,
double* cpuUtilizationPercent)
{
try {
const ANSCENTER::SystemHealthInfo info =
ANSCENTER::ANSLicenseHelper::GetSystemHealthInformation();
auto writeLStr = [](LStrHandle handle, const std::string& s) {
if (!handle) return;
const int size = static_cast<int>(s.length());
const MgErr err = DSSetHandleSize(handle, sizeof(int32) + size * sizeof(uChar));
if (err == noErr) {
(*handle)->cnt = size;
if (size > 0) memcpy((*handle)->str, s.c_str(), size);
}
};
if (cpuLoadPercent) *cpuLoadPercent = info.cpuLoadPercent;
if (cpuTempDegC) *cpuTempDegC = info.cpuTempDegC;
if (cpuUtilizationPercent) *cpuUtilizationPercent = info.cpuUtilizationPercent;
if (cpuFreqGHz) *cpuFreqGHz = info.cpuFreqGHz;
if (memTotalGB) *memTotalGB = info.memTotalGB;
if (rssPercent) *rssPercent = info.rssPercent;
if (swapPercent) *swapPercent = info.swapPercent;
if (gpuLoadPercent) *gpuLoadPercent = info.gpuLoadPercent;
if (valIRSS_MB) *valIRSS_MB = info.valIRSS_MB;
if (valSwap_MB) *valSwap_MB = info.valSwap_MB;
if (valIGPU_MB) *valIGPU_MB = info.valIGPU_MB;
if (memoryUsedGB) *memoryUsedGB = info.memoryUsedGB;
if (memoryFreeGB) *memoryFreeGB = info.memoryFreeGB;
if (diskLoadPercent) *diskLoadPercent = info.diskLoadPercent;
if (diskReadSpeedMBs) *diskReadSpeedMBs = info.diskReadSpeedMBs;
if (diskWriteSpeedMBs) *diskWriteSpeedMBs = info.diskWriteSpeedMBs;
if (diskUsedGB) *diskUsedGB = info.diskUsedGB;
if (diskTotalGB) *diskTotalGB = info.diskTotalGB;
if (netLoadPercent) *netLoadPercent = info.netLoadPercent;
if (netReceiveSpeedMBs) *netReceiveSpeedMBs = info.netReceiveSpeedMBs;
if (netSendSpeedMBs) *netSendSpeedMBs = info.netSendSpeedMBs;
writeLStr(cpuModel, info.cpuModel);
writeLStr(gpuName, info.gpuName);
writeLStr(gpuAdaptersDetail, info.gpuAdaptersDetail);
return 1;
} catch (...) {
return 0;
}
}
extern "C" __declspec(dllexport) int GetPublicIPAddress(LStrHandle ipv4) {
try {
if (!ipv4) return 0;
const std::string s = ANSCENTER::ANSLicenseHelper::GetPublicIPAddress();
const int size = static_cast<int>(s.length());
const MgErr err = DSSetHandleSize(ipv4, sizeof(int32) + size * sizeof(uChar));
if (err != noErr) return 0;
(*ipv4)->cnt = size;
if (size > 0) memcpy((*ipv4)->str, s.c_str(), size);
return s.empty() ? 0 : 1;
} catch (...) {
return 0;
}
}
namespace {
inline void WriteLStr(LStrHandle h, const std::string& s) {
if (!h) return;
const int size = static_cast<int>(s.length());
const MgErr err = DSSetHandleSize(h, sizeof(int32) + size * sizeof(uChar));
if (err == noErr) {
(*h)->cnt = size;
if (size > 0) memcpy((*h)->str, s.c_str(), size);
}
}
}
extern "C" __declspec(dllexport) int GetPublicNetworkParameters(
LStrHandle publicIP,
LStrHandle country,
LStrHandle countryCode,
LStrHandle region,
LStrHandle city,
LStrHandle isp,
LStrHandle timeZone,
double* latitude,
double* longitude,
LStrHandle errorMsg)
{
try {
const ANSCENTER::PublicNetworkInfo info =
ANSCENTER::ANSLicenseHelper::GetPublicNetworkInformation();
WriteLStr(publicIP, info.publicIP);
WriteLStr(country, info.country);
WriteLStr(countryCode, info.countryCode);
WriteLStr(region, info.region);
WriteLStr(city, info.city);
WriteLStr(isp, info.isp);
WriteLStr(timeZone, info.timezone);
WriteLStr(errorMsg, info.error);
if (latitude) *latitude = info.latitude;
if (longitude) *longitude = info.longitude;
return info.ok ? 1 : 0;
} catch (...) {
return 0;
}
}
extern "C" __declspec(dllexport) int GetPublicNetworkParametersString(LStrHandle outString) {
try {
if (!outString) return 0;
const ANSCENTER::PublicNetworkInfo info =
ANSCENTER::ANSLicenseHelper::GetPublicNetworkInformation();
auto sanitize = [](std::string s) {
std::replace(s.begin(), s.end(), ';', ',');
return s;
};
std::ostringstream os;
os.imbue(std::locale::classic());
os << std::fixed << std::setprecision(6);
os << sanitize(info.publicIP) << ';'
<< sanitize(info.country) << ';'
<< sanitize(info.countryCode) << ';'
<< sanitize(info.region) << ';'
<< sanitize(info.city) << ';'
<< sanitize(info.isp) << ';'
<< sanitize(info.timezone) << ';'
<< info.latitude << ';'
<< info.longitude;
const std::string s = os.str();
const int size = static_cast<int>(s.length());
const MgErr err = DSSetHandleSize(outString, sizeof(int32) + size * sizeof(uChar));
if (err != noErr) return 0;
(*outString)->cnt = size;
if (size > 0) memcpy((*outString)->str, s.c_str(), size);
return info.ok ? 1 : 0;
} catch (...) {
return 0;
}
}
extern "C" __declspec(dllexport) int GetSystemHealthParametersString(LStrHandle outString) {
try {
if (!outString) return 0;
const ANSCENTER::SystemHealthInfo info =
ANSCENTER::ANSLicenseHelper::GetSystemHealthInformation();
// Keep ';' as a clean separator: rewrite any literal ';' inside any
// free-form string field (rare, but vendor strings aren't safe by
// contract).
auto sanitize = [](std::string s) {
std::replace(s.begin(), s.end(), ';', ',');
return s;
};
const std::string model = sanitize(info.cpuModel);
const std::string gpuName_ = sanitize(info.gpuName);
const std::string gpuDetail = sanitize(info.gpuAdaptersDetail);
std::ostringstream os;
os.imbue(std::locale::classic()); // force '.' decimal regardless of system locale
os << std::fixed << std::setprecision(2);
os << info.cpuLoadPercent << ';'
<< info.cpuTempDegC << ';'
<< info.cpuFreqGHz << ';'
<< model << ';'
<< info.memTotalGB << ';'
<< info.rssPercent << ';'
<< info.swapPercent << ';'
<< info.gpuLoadPercent << ';'
<< info.valIRSS_MB << ';'
<< info.valSwap_MB << ';'
<< info.valIGPU_MB << ';'
<< info.memoryUsedGB << ';'
<< info.memoryFreeGB << ';'
<< info.diskLoadPercent << ';'
<< info.diskReadSpeedMBs << ';'
<< info.diskWriteSpeedMBs << ';'
<< info.diskUsedGB << ';'
<< info.diskTotalGB << ';'
<< info.netLoadPercent << ';'
<< info.netReceiveSpeedMBs << ';'
<< info.netSendSpeedMBs << ';'
<< gpuName_ << ';'
<< gpuDetail << ';'
<< info.cpuUtilizationPercent;
const std::string s = os.str();
const int size = static_cast<int>(s.length());
const MgErr err = DSSetHandleSize(outString, sizeof(int32) + size * sizeof(uChar));
if (err != noErr) return 0;
(*outString)->cnt = size;
if (size > 0) memcpy((*outString)->str, s.c_str(), size);
return 1;
} catch (...) {
return 0;
}
}
extern "C" __declspec(dllexport) int ReleaseLoggers() {
try {
bool result = ANSCENTER::ANSLSHelper::ReleaseLogger();
Binary file not shown.
+412
View File
@@ -0,0 +1,412 @@
// Build ANSLIB Logging Guide.docx — examples document for developers.
// Run: node build_anslib_logging_guide.js
const fs = require('fs');
const path = require('path');
const {
Document, Packer, Paragraph, TextRun, Table, TableRow, TableCell,
AlignmentType, LevelFormat, HeadingLevel, BorderStyle, WidthType,
ShadingType, PageOrientation
} = require('docx');
// -------- helpers ----------------------------------------------------------
const FONT_BODY = "Calibri";
const FONT_CODE = "Consolas";
function p(text, opts = {}) {
return new Paragraph({
spacing: { after: 120 },
...opts,
children: [new TextRun({ text, font: FONT_BODY, size: 22, ...(opts.run || {}) })],
});
}
function h1(text) {
return new Paragraph({
heading: HeadingLevel.HEADING_1,
spacing: { before: 360, after: 200 },
children: [new TextRun({ text, font: FONT_BODY, size: 36, bold: true, color: "1F3864" })],
});
}
function h2(text) {
return new Paragraph({
heading: HeadingLevel.HEADING_2,
spacing: { before: 280, after: 160 },
children: [new TextRun({ text, font: FONT_BODY, size: 28, bold: true, color: "2E74B5" })],
});
}
function h3(text) {
return new Paragraph({
heading: HeadingLevel.HEADING_3,
spacing: { before: 200, after: 100 },
children: [new TextRun({ text, font: FONT_BODY, size: 24, bold: true, color: "404040" })],
});
}
// Single-line inline-code style (uses gray-ish background via shading isn't applied to runs;
// we just use a monospace font + light shading via a wrapping single-cell table when block).
function codeLine(text) {
return new TextRun({ text, font: FONT_CODE, size: 20 });
}
// Multi-line code block — render as a one-cell table with light-gray fill,
// each source line becomes a paragraph inside the cell.
function codeBlock(source) {
const lines = source.replace(/\r\n/g, "\n").split("\n");
const paras = lines.map(line =>
new Paragraph({
spacing: { after: 0, line: 260 },
children: [new TextRun({ text: line || " ", font: FONT_CODE, size: 20 })],
})
);
return new Table({
width: { size: 9360, type: WidthType.DXA },
columnWidths: [9360],
rows: [
new TableRow({
children: [
new TableCell({
width: { size: 9360, type: WidthType.DXA },
shading: { fill: "F4F4F4", type: ShadingType.CLEAR },
margins: { top: 120, bottom: 120, left: 180, right: 180 },
borders: {
top: { style: BorderStyle.SINGLE, size: 4, color: "DDDDDD" },
bottom: { style: BorderStyle.SINGLE, size: 4, color: "DDDDDD" },
left: { style: BorderStyle.SINGLE, size: 4, color: "DDDDDD" },
right: { style: BorderStyle.SINGLE, size: 4, color: "DDDDDD" },
},
children: paras,
}),
],
}),
],
});
}
function bullet(text, runs = null) {
return new Paragraph({
numbering: { reference: "bullets", level: 0 },
spacing: { after: 80 },
children: runs || [new TextRun({ text, font: FONT_BODY, size: 22 })],
});
}
function spacer() {
return new Paragraph({ spacing: { after: 120 }, children: [new TextRun(" ")] });
}
// ---- table builder for the cheat sheets -----------------------------------
const CELL_BORDER = { style: BorderStyle.SINGLE, size: 4, color: "BFBFBF" };
const CELL_BORDERS = { top: CELL_BORDER, bottom: CELL_BORDER, left: CELL_BORDER, right: CELL_BORDER };
function headerCell(text, width) {
return new TableCell({
width: { size: width, type: WidthType.DXA },
borders: CELL_BORDERS,
shading: { fill: "1F3864", type: ShadingType.CLEAR },
margins: { top: 100, bottom: 100, left: 140, right: 140 },
children: [new Paragraph({
children: [new TextRun({ text, font: FONT_BODY, size: 22, bold: true, color: "FFFFFF" })],
})],
});
}
function bodyCell(text, width, opts = {}) {
const runs = Array.isArray(text)
? text
: [new TextRun({ text, font: opts.code ? FONT_CODE : FONT_BODY, size: 20 })];
return new TableCell({
width: { size: width, type: WidthType.DXA },
borders: CELL_BORDERS,
shading: opts.shade ? { fill: opts.shade, type: ShadingType.CLEAR } : undefined,
margins: { top: 80, bottom: 80, left: 140, right: 140 },
children: [new Paragraph({ children: runs })],
});
}
function sheetTable(columnWidths, headers, rows) {
const total = columnWidths.reduce((a, b) => a + b, 0);
return new Table({
width: { size: total, type: WidthType.DXA },
columnWidths,
rows: [
new TableRow({
tableHeader: true,
children: headers.map((h, i) => headerCell(h, columnWidths[i])),
}),
...rows.map(row =>
new TableRow({
children: row.map((c, i) => {
if (c && typeof c === 'object' && c.runs) {
return bodyCell(c.runs, columnWidths[i], c.opts || {});
}
return bodyCell(c, columnWidths[i]);
}),
})
),
],
});
}
// =========================================================================
// Document content
// =========================================================================
const children = [];
// ---------- Title ----------
children.push(new Paragraph({
spacing: { after: 80 },
alignment: AlignmentType.LEFT,
children: [new TextRun({ text: "ANSLIB Logging API", font: FONT_BODY, size: 48, bold: true, color: "1F3864" })],
}));
children.push(new Paragraph({
spacing: { after: 360 },
children: [new TextRun({ text: "Developer guide — usage examples for ANSLIB_DBG, LogInfo, LogError, LogFatal", font: FONT_BODY, size: 24, italics: true, color: "595959" })],
}));
// ---------- Overview ----------
children.push(h1("Overview"));
children.push(p("ANSLIB exposes a small logging API that consumers can use without linking against ANSLicensingSystem.dll or any spdlog headers. All symbols are dynamically loaded inside ANSLIB; users only need to include ANSLIB.h and link ANSLIB.lib."));
children.push(p("Two flavours are provided:"));
children.push(bullet("ANSLIB_DBG — a printf-style macro for trace breadcrumbs that can be turned on or off remotely (DebugView-style logging)."));
children.push(bullet("ANSCENTER::ANSLIB::LogInfo / LogError / LogFatal — leveled logging that writes through SPDLogger to the rolling log file."));
// ---------- Setup ----------
children.push(h1("Setup"));
children.push(p("Same prerequisites for every example below. Add this to any consumer translation unit:"));
children.push(codeBlock(
`#include "ANSLIB.h" // brings everything: static methods + ANSLIB_DBG macro
// Optional alias to shorten call sites:
using LOG = ANSCENTER::ANSLIB;`));
children.push(p("No other includes, no extra .lib to link beyond ANSLIB.lib."));
// ---------- Example 1 ----------
children.push(h1("Example 1 — Function entry/exit (ANSLIB_DBG, the workhorse)"));
children.push(codeBlock(
`int LoadConfig(const std::string& path) {
ANSLIB_DBG("Config", "LoadConfig entry path=%s", path.c_str());
if (path.empty()) {
ANSLIB_DBG("Config", "empty path, bailing");
return -1;
}
int rc = ParseFile(path);
ANSLIB_DBG("Config", "ParseFile returned rc=%d", rc);
return rc;
}`));
children.push(p("DebugView output (only when the gate is on):"));
children.push(codeBlock(
`[Config] LoadConfig entry path=C:\\models\\car.cfg
[Config] ParseFile returned rc=0`));
children.push(p("This is the drop-in replacement for ANS_DBG. Use it for breadcrumb logging that you want to be able to flip on/off remotely."));
// ---------- Example 2 ----------
children.push(h1("Example 2 — Error path (LogError, persisted to spdlog file)"));
children.push(p("When something goes wrong and you want it in the rolling log file (not just DebugView), use LogError:"));
children.push(codeBlock(
`int InitEngine(const std::string& modelPath) {
auto* engine = ANSCENTER::ANSLIB::Create();
if (!engine) {
LOG::LogError("InitEngine", "ANSLIB::Create returned null", __FILE__, __LINE__);
return -1;
}
std::string labels;
int rc = engine->Initialize("license-key", modelPath.c_str(), "",
0.5f, 0.5f, 0.4f, 8, 5, 1, labels);
if (rc < 0) {
char detail[256];
snprintf(detail, sizeof(detail),
"Initialize failed rc=%d model=%s", rc, modelPath.c_str());
LOG::LogError("InitEngine", detail, __FILE__, __LINE__);
ANSCENTER::ANSLIB::Destroy(engine);
return rc;
}
LOG::LogInfo("InitEngine", "engine ready", __FILE__, __LINE__);
return 0;
}`));
children.push(p("Rolling log file output (ANSLIB.log or whichever name spdlog has):"));
children.push(codeBlock(
`[2026-04-26 14:32:11.483] [error] [foo.cpp] [42] InitEngine: Initialize failed rc=-1 model=C:\\models\\car.zip
[2026-04-26 14:32:11.484] [info] [foo.cpp] [50] InitEngine: engine ready`));
// ---------- Example 3 ----------
children.push(h1("Example 3 — Fatal / unrecoverable (LogFatal)"));
children.push(p("Use when the process can't continue — e.g. license check failed, GPU lost, mandatory model missing:"));
children.push(codeBlock(
`void OnGpuDeviceLost() {
LOG::LogFatal("Engine",
"CUDA device lost, cannot continue inference",
__FILE__, __LINE__);
std::abort(); // or whatever your shutdown path is
}`));
// ---------- Example 4 ----------
children.push(h1("Example 4 — Mixing both (recommended pattern)"));
children.push(p("Use ANSLIB_DBG for the chatty trace breadcrumbs and LogInfo / LogError / LogFatal for things you want preserved in the file:"));
children.push(codeBlock(
`int RunBatch(const std::vector<cv::Mat>& frames) {
ANSLIB_DBG("Batch", "RunBatch entry frames=%zu", frames.size());
if (frames.empty()) {
LOG::LogError("Batch", "called with empty frame list", __FILE__, __LINE__);
return -1;
}
int processed = 0;
for (const auto& f : frames) {
ANSLIB_DBG("Batch", "processing frame %d (%dx%d)", processed, f.cols, f.rows);
// ...inference...
++processed;
}
char summary[128];
snprintf(summary, sizeof(summary), "RunBatch OK processed=%d", processed);
LOG::LogInfo("Batch", summary, __FILE__, __LINE__);
ANSLIB_DBG("Batch", "RunBatch exit");
return processed;
}`));
children.push(p("DebugView shows the full trace; the log file keeps just the entry/exit summary. This keeps the file small but lets you toggle full tracing remotely when something goes wrong."));
// ---------- Example 5 ----------
children.push(h1("Example 5 — Gating expensive log preparation"));
children.push(p("ANSLIB_DBG already gates the snprintf for you. But if preparing the message itself is expensive (serializing a struct, dumping a Mat, walking a tree), check the gate first manually:"));
children.push(codeBlock(
`void DumpDetections(const std::vector<ANSCENTER::Object>& dets) {
if (!ANSCENTER::ANSLIB::IsDebugViewEnabled()) return; // skip the whole prep
std::string dump;
for (const auto& d : dets) {
dump += d.className + "(" + std::to_string(d.confidence) + ") ";
}
ANSLIB_DBG("Detections", "%zu objects: %s", dets.size(), dump.c_str());
}`));
children.push(p("This skips the entire string-building loop when the gate is off."));
// ---------- Example 6 ----------
children.push(h1("Example 6 — Logging from a class method"));
children.push(p("The __FILE__ / __LINE__ defaults make instance methods cleaner if you don't need them:"));
children.push(codeBlock(
`class MyDetector {
public:
void Process(const cv::Mat& img) {
ANSLIB_DBG("MyDetector", "Process %dx%d", img.cols, img.rows);
if (img.empty()) {
// No file/line — defaults to "" / 0
LOG::LogError("MyDetector", "empty image input");
return;
}
// ...
}
};`));
// ---------- Example 7 ----------
children.push(h1("Example 7 — Macro for \"log everywhere with caller info\" (optional)"));
children.push(p("If you want every LogInfo call to auto-include __FILE__ / __LINE__, define a one-liner in your own code:"));
children.push(codeBlock(
`#define MYAPP_INFO(tag, msg) ::ANSCENTER::ANSLIB::LogInfo (tag, msg, __FILE__, __LINE__)
#define MYAPP_ERROR(tag, msg) ::ANSCENTER::ANSLIB::LogError(tag, msg, __FILE__, __LINE__)
// Usage:
MYAPP_INFO ("Boot", "starting up");
MYAPP_ERROR("Net", "connection refused");`));
children.push(p("This keeps call sites short while preserving file/line tracking in the rolling log."));
// ---------- Cheat sheet 1 ----------
children.push(h1("Cheat sheet — pick the right call"));
children.push(sheetTable(
[3000, 3360, 3000],
["Want to log…", "Use", "Goes to"],
[
["Trace / breadcrumb that's toggled per-incident",
{ runs: [codeLine('ANSLIB_DBG("Tag", "fmt %d", x)')] },
"DebugView + stderr (when gate is on)"],
["Notable event (started, finished, config loaded)",
{ runs: [codeLine('LOG::LogInfo("Tag", "msg", __FILE__, __LINE__)')] },
"spdlog file + console"],
["Recoverable error (will retry, will skip, etc.)",
{ runs: [codeLine('LOG::LogError("Tag", "msg", __FILE__, __LINE__)')] },
"spdlog file + console"],
["Unrecoverable (about to crash / abort)",
{ runs: [codeLine('LOG::LogFatal("Tag", "msg", __FILE__, __LINE__)')] },
"spdlog file + console (critical level)"],
]
));
children.push(spacer());
// ---------- Cheat sheet 2 ----------
children.push(h1("Toggling the DebugView gate at runtime"));
children.push(p("LogInfo / LogError / LogFatal are not gated — they always go to the spdlog file. Only ANSLIB_DBG is gated by the mechanism below."));
children.push(spacer());
children.push(sheetTable(
[4680, 4680],
["Action", "Effect"],
[
[{ runs: [new TextRun({ text: "Drop file ", font: FONT_BODY, size: 20 }),
codeLine("C:\\ProgramData\\ANSCENTER\\ansvisdebugview.txt")] },
{ runs: [new TextRun({ text: "Turns ", font: FONT_BODY, size: 20 }),
codeLine("ANSLIB_DBG"), new TextRun({ text: " (and ", font: FONT_BODY, size: 20 }),
codeLine("ANS_DBG"), new TextRun({ text: ") ON within ~2s", font: FONT_BODY, size: 20 })] }],
["Delete that file", "Turns them OFF within ~2s"],
[{ runs: [codeLine("set ANSCENTER_DBGVIEW=1"), new TextRun({ text: " (or ", font: FONT_BODY, size: 20 }),
codeLine("=0"), new TextRun({ text: ")", font: FONT_BODY, size: 20 })] },
"Forces ON or OFF — overrides the file"],
]
));
// ---------- Operational notes ----------
children.push(h1("Operational notes"));
children.push(bullet("All four entry points (ANSLIB_DBG, LogInfo, LogError, LogFatal) are thread-safe and require no instance — they're static methods that lazy-resolve their underlying DLL symbols on first call."));
children.push(bullet("If ANSLicensingSystem.dll is missing on the target machine, every logging call silently no-ops (no crash, no exception). The application keeps running."));
children.push(bullet("ANSLIB_DBG output format is identical to the original ANS_DBG: \"[tag] formatted\\n\" emitted to both OutputDebugStringA and stderr."));
children.push(bullet("Don't log from static destructors. DLL unload order during process shutdown is undefined on Windows — calling into the licensing DLL after it has been unmapped will crash."));
children.push(bullet("To strip ANSLIB_DBG entirely at compile time (for performance-critical builds), define -DANSCORE_DEBUGVIEW=0 when compiling the consumer translation unit. The macro becomes ((void)0)."));
// =========================================================================
// Document assembly
// =========================================================================
const doc = new Document({
styles: {
default: { document: { run: { font: FONT_BODY, size: 22 } } },
paragraphStyles: [
{ id: "Heading1", name: "Heading 1", basedOn: "Normal", next: "Normal", quickFormat: true,
run: { size: 36, bold: true, font: FONT_BODY, color: "1F3864" },
paragraph: { spacing: { before: 360, after: 200 }, outlineLevel: 0 } },
{ id: "Heading2", name: "Heading 2", basedOn: "Normal", next: "Normal", quickFormat: true,
run: { size: 28, bold: true, font: FONT_BODY, color: "2E74B5" },
paragraph: { spacing: { before: 280, after: 160 }, outlineLevel: 1 } },
{ id: "Heading3", name: "Heading 3", basedOn: "Normal", next: "Normal", quickFormat: true,
run: { size: 24, bold: true, font: FONT_BODY, color: "404040" },
paragraph: { spacing: { before: 200, after: 100 }, outlineLevel: 2 } },
],
},
numbering: {
config: [
{ reference: "bullets",
levels: [{ level: 0, format: LevelFormat.BULLET, text: "•", alignment: AlignmentType.LEFT,
style: { paragraph: { indent: { left: 720, hanging: 360 } } } }] },
],
},
sections: [{
properties: {
page: {
size: { width: 12240, height: 15840 },
margin: { top: 1440, right: 1440, bottom: 1440, left: 1440 },
},
},
children,
}],
});
const outPath = path.join(__dirname, "ANSLIB_Logging_Guide.docx");
Packer.toBuffer(doc).then(buf => {
fs.writeFileSync(outPath, buf);
console.log("Wrote", outPath, "(" + buf.length + " bytes)");
});
+206 -2
View File
@@ -10,9 +10,129 @@
#include <fstream>
#include <cstdlib>
#include <system_error>
#include <atomic>
#include <mutex>
namespace ANSCENTER {
// ====================================================================
// DmlSerialState / DmlInferenceGuard implementation.
//
// Storage is in this single linkage unit (ONNXEngine.dll) so every
// module linking against ONNXEngine sees the same flag and mutex —
// ANSODEngine.dll's DML calls and ANSOCR.dll's DML calls serialize
// against the same process-wide state. See the doc comment on
// DmlSerialState in ONNXEngine.h for the full design rationale.
// ====================================================================
namespace {
// false = parallel (default). Tripped to true (sticky) once any
// DML site sees an exception or a slow Run().
std::atomic<bool> g_dmlSerialMode{ false };
// Held during DML inference when g_dmlSerialMode == true. Plain
// std::mutex (not recursive): one DML Run() at a time across the
// entire process — that's the whole point. Other ORT operations
// (session creation, allocator setup, etc.) DO NOT need this
// mutex and are not gated by it.
std::mutex g_dmlInferenceMutex;
// 1500 ms — comfortably above normal AMD DML latency (50–500 ms)
// and well below WDDM TDR (10 s with TdrDelay=10).
constexpr double kDmlSlowThresholdMs = 1500.0;
}
bool DmlSerialState::IsSerialMode() {
return g_dmlSerialMode.load(std::memory_order_relaxed);
}
double DmlSerialState::SlowThresholdMs() {
return kDmlSlowThresholdMs;
}
void DmlSerialState::TripFromException(const char* context, const char* msg) {
bool expected = false;
if (g_dmlSerialMode.compare_exchange_strong(expected, true)) {
// First caller wins — log the trip event clearly so it's
// obvious in the field which path serialized DML.
std::cerr << "[DmlSerial] TRIPPED from exception in "
<< (context ? context : "?")
<< " - process-wide DML inference is now SERIALIZED. msg="
<< (msg ? msg : "?") << std::endl;
}
}
void DmlSerialState::TripFromLatency(const char* context, double ms) {
bool expected = false;
if (g_dmlSerialMode.compare_exchange_strong(expected, true)) {
std::cerr << "[DmlSerial] TRIPPED from slow inference in "
<< (context ? context : "?")
<< " ms=" << ms
<< " - process-wide DML inference is now SERIALIZED."
<< std::endl;
}
}
void DmlSerialState::AcquireMutex() {
g_dmlInferenceMutex.lock();
}
void DmlSerialState::ReleaseMutex() {
g_dmlInferenceMutex.unlock();
}
DmlInferenceGuard::DmlInferenceGuard(EngineType ep, const char* context)
: _ep(ep), _context(context), _locked(false)
{
// ── Always-serial-on-AMD policy ────────────────────────────
//
// Earlier design (Option A v1) was lazy: parallel by default,
// serialize only after seeing a slow Run() or exception. Field
// testing on AMD Radeon 780M proved this fires too late — by the
// time the latency trip arms, a sibling thread has already
// entered Run() at depth=2 and wedged the D3D12 command queue.
// ORT then crashes inside its own DML EP cleanup when the WDDM
// TDR fires.
//
// The only way to guarantee zero dual-stall on AMD is to never
// let two threads be in flight on DML simultaneously. This
// constructor unconditionally takes the global DML mutex when
// ep == AMD_GPU. Intel (OpenVINO) and NVIDIA (CUDA/TRT) skip
// the lock entirely — guard collapses to a timestamp read +
// comparison, and the destructor is a single comparison. Zero
// measurable overhead on those platforms.
//
// Trade-off: AMD inference becomes serial (depth=1) instead of
// parallel (depth=2). Each individual call still runs on the
// GPU at GPU speeds; only the cross-thread overlap is removed.
// Throughput cost vs broken depth=2 is ~30% lower wall-clock
// throughput per frame, but uptime goes from ~30 s before
// wedge → unbounded.
if (_ep == EngineType::AMD_GPU) {
DmlSerialState::AcquireMutex();
_locked = true;
}
// Start the latency timer AFTER lock acquisition so contention
// on the global mutex doesn't itself trip the slow-Run()
// threshold. We measure GPU Run() time, not queue wait.
_t0 = std::chrono::steady_clock::now();
}
DmlInferenceGuard::~DmlInferenceGuard() {
// Latency-based trip detection: if THIS Run() took longer than
// the slow threshold, future DML inference goes serial. Only
// checked on AMD — Intel/NVIDIA never trip.
if (_ep == EngineType::AMD_GPU) {
const auto t1 = std::chrono::steady_clock::now();
const double ms = std::chrono::duration<double, std::milli>(t1 - _t0).count();
if (ms > kDmlSlowThresholdMs) {
DmlSerialState::TripFromLatency(_context, ms);
}
}
if (_locked) {
DmlSerialState::ReleaseMutex();
}
}
// ====================================================================
// BasicOrtHandler — constructors
// ====================================================================
@@ -365,6 +485,85 @@ namespace ANSCENTER {
return false;
}
// ====================================================================
// Reinitialize — tear down + rebuild session on a new EP.
//
// Used by subclass self-heal paths (ONNXOCRRecognizer /
// ONNXOCRDetector) to recover from Ort::Exception in Run(). The
// typical flow is:
//
// try { ort_session->Run(...); }
// catch (const Ort::Exception&) {
// Reinitialize(EngineType::AMD_GPU); // first attempt: retry DML
// try { ort_session->Run(...); }
// catch (const Ort::Exception&) {
// Reinitialize(EngineType::CPU); // last resort: drop to CPU
// ort_session->Run(...);
// }
// }
//
// After this call returns, ort_session, ort_env, memory_info_handler,
// input_node_names, input_node_dims, output_node_names, etc. are all
// freshly populated for `newEngine`. Subclass state that lives outside
// BasicOrtHandler (e.g. ONNXOCRRecognizer::_warmedUp, ::keys_) is NOT
// touched — the caller should reset whatever flags it owns if they
// become invalid for the new session.
//
// Throws Ort::Exception (or std::exception) on failure to construct
// the new session. Caller should treat that as terminal for the
// current Run() and return an empty result.
// ====================================================================
void BasicOrtHandler::Reinitialize(EngineType newEngine)
{
ANS_DBG("OrtHandler",
"Reinitialize: oldEngine=%d newEngine=%d (model=%ls)",
static_cast<int>(m_engineType),
static_cast<int>(newEngine),
onnx_path ? onnx_path : L"(null)");
// Drop the existing session FIRST so its D3D12 / DML allocations
// are released back to the driver. Order matters: ort_session
// must die before ort_env, otherwise Ort::Session's destructor
// can dereference a dangling env pointer in some EP teardown
// paths (observed on DML 1.15.x).
if (ort_session) {
try { delete ort_session; } catch (...) {}
ort_session = nullptr;
}
if (memory_info_handler) {
try { delete memory_info_handler; } catch (...) {}
memory_info_handler = nullptr;
}
if (ort_env) {
try { delete ort_env; } catch (...) {}
ort_env = nullptr;
}
// Reset shape/name vectors so initialize_handler repopulates
// them from the new session (input names can differ across EPs
// for some models — known DML EP quirk).
input_node_names.clear();
input_node_names_.clear();
input_node_dims.clear();
input_values_handler.clear();
output_node_names.clear();
output_node_names_.clear();
output_node_dims.clear();
input_name = nullptr;
input_tensor_size = 1;
num_outputs = 1;
// Set the requested EP and re-run the standard init. Throws on
// failure — caller's responsibility to handle.
m_engineType = newEngine;
initialize_handler();
ANS_DBG("OrtHandler",
"Reinitialize OK: ort_session=%p engine=%d",
static_cast<void*>(ort_session),
static_cast<int>(m_engineType));
}
// ====================================================================
// initialize_handler
// ====================================================================
@@ -819,7 +1018,10 @@ namespace ANSCENTER {
input_values_handler.clear();
input_values_handler.shrink_to_fit();
const float* vals = output_tensors[0].GetTensorData<float>();
// GetTensorMutableData on DirectML — the const GetTensorData
// triggers a per-call host-readable mapping that on AMD DML
// exhausts a staging-buffer pool after ~8 calls and hangs.
const float* vals = output_tensors[0].GetTensorMutableData<float>();
const unsigned int hidden_dim =
static_cast<unsigned int>(output_node_dims.at(0).at(1));
@@ -1377,7 +1579,9 @@ namespace ANSCENTER {
Ort::RunOptions{ nullptr },
in_names, &input_tensor, 1, out_names, 1);
out_result = post_processing(outputs[0].GetTensorData<float>());
// GetTensorMutableData (not GetTensorData) — see comment in this
// file's other output-read sites; const GetTensorData hangs on AMD DML.
out_result = post_processing(outputs[0].GetTensorMutableData<float>());
}
Ort::Value MOVINET::transform(const cv::Mat& mat)
+118
View File
@@ -8,6 +8,7 @@
#include <typeinfo>
#include <deque>
#include <unordered_map>
#include <chrono> // DmlInferenceGuard latency timing
#include "onnxruntime_cxx_api.h"
#include "opencv2/opencv.hpp"
@@ -337,6 +338,108 @@ namespace ANSCENTER {
// (AMD fields go here — none yet)
};
// ====================================================================
// DmlSerialState — process-wide DirectML inference serialization
//
// AMD Radeon iGPUs (Radeon 680M / 780M, observed) cannot sustain
// 2-way concurrent inference on DirectML. Under sustained
// parallel load the WDDM driver eventually deadlocks the D3D12
// command queue when multiple ORT sessions submit work
// simultaneously, producing a "dual stall" pattern (both threads
// inside ORT::Run() with no exception thrown) and ultimately an
// access violation deep inside the DirectML EP cleanup path.
//
// Policy: every DirectML inference call on AMD_GPU is serialized
// through DmlInferenceGuard's unconditional lock. Lazy "trip on
// first slow Run()" was tried first (a flag flipped to true on
// exception or >1500 ms latency) but field-proven inadequate —
// the trip fires AFTER a slow Run() returns, by which point a
// sibling thread has already entered Run() concurrently and
// wedged. Always-serialize is the only safe answer on AMD.
//
// This class still tracks an "ever-tripped" flag for diagnostics
// — the TripFromException / TripFromLatency calls log a marker
// the first time a DML hiccup is observed, so field traces
// clearly show whether the AMD driver had a degraded event. The
// flag does NOT gate the lock; the lock is unconditional on AMD.
//
// Intel (OpenVINO) and NVIDIA (CUDA/TRT) bypass the lock entirely
// — DmlInferenceGuard short-circuits on ep != AMD_GPU. They keep
// peakDepth=2 parallel inference unchanged; zero behavioral
// change on those platforms.
//
// Storage lives in the ONNXEngine sources compiled into
// ANSODEngine.dll so all higher-level modules (ANSODEngine,
// ANSOCR, ...) share a single process-global mutex.
// ====================================================================
class ONNXENGINE_API DmlSerialState {
public:
// True when serial mode has been tripped.
static bool IsSerialMode();
// Trip helpers. Both are idempotent — only the first caller
// per process actually flips the flag and emits the
// diagnostic. Called by:
// exception path → ANSONNXYOLO::HandleDmlRunFailure, OCR
// latency path → DmlInferenceGuard destructor on slow Run()
static void TripFromException(const char* context, const char* msg);
static void TripFromLatency (const char* context, double ms);
// 1500 ms — comfortably above normal AMD DML latency and well
// under WDDM TDR.
static double SlowThresholdMs();
// Internal — only DmlInferenceGuard should call these.
static void AcquireMutex();
static void ReleaseMutex();
};
// ====================================================================
// DmlInferenceGuard — RAII guard for DirectML inference call sites
//
// Construct around every Ort::Session::Run() that may execute on
// a DirectML EP. Behavior:
//
// 1. ep != AMD_GPU → entire guard is a no-op (zero overhead on
// Intel / NVIDIA / CPU). The destructor's latency check and
// lock release are also skipped via _ep == AMD_GPU and
// _locked == false guards.
// 2. ep == AMD_GPU → unconditionally acquires the global DML
// mutex; only one DML inference in the entire process runs
// at a time. On scope exit, releases the mutex and (if the
// call exceeded DmlSerialState::SlowThresholdMs()) records a
// diagnostic trip marker — the marker does NOT change
// behavior, since AMD is already serial; it just shows up in
// logs as evidence of a degraded GPU event.
//
// Why unconditional serialization on AMD: empirical testing on
// Radeon 780M shows the WDDM driver wedges when 2 ORT sessions
// submit DML inference concurrently for >~30 s. Lazy "lock only
// after first slow call" was found insufficient — by the time the
// trip fires, a sibling thread is already inside the wedge-prone
// Run() with no way to back out. Always-lock is the only path
// that guarantees the dual-stall window never opens. The cost is
// ~30% lower throughput on AMD (depth=1 vs depth=2) for
// unbounded uptime instead of crashing every 30 s.
// ====================================================================
class ONNXENGINE_API DmlInferenceGuard {
public:
explicit DmlInferenceGuard(EngineType ep, const char* context);
~DmlInferenceGuard();
// Non-copyable / non-movable — RAII scope guard only.
DmlInferenceGuard(const DmlInferenceGuard&) = delete;
DmlInferenceGuard& operator=(const DmlInferenceGuard&) = delete;
DmlInferenceGuard(DmlInferenceGuard&&) = delete;
DmlInferenceGuard& operator=(DmlInferenceGuard&&) = delete;
private:
EngineType _ep;
const char* _context;
bool _locked;
std::chrono::steady_clock::time_point _t0;
};
// ====================================================================
// BasicOrtHandler
// ====================================================================
@@ -410,6 +513,21 @@ namespace ANSCENTER {
// (which require the input name) BEFORE the real session is
// created. Returns an empty string on failure.
static std::string QueryModelInputName(const std::string& onnxPath);
protected:
// Tear down the existing ORT session/env/memory-info and rebuild
// them with `newEngine` as the requested EP. Used by subclass
// self-heal paths (e.g. ONNXOCRRecognizer / ONNXOCRDetector) when
// an Ort::Exception fires from a Run() — the session is recreated
// either on the same EP (transient driver hiccup recovery) or on
// CPU (permanent fallback). Throws on failure; caller should
// treat that as "give up for this call" and return an empty
// result. Subclass-owned auxiliary state (input_node_dims,
// input_node_names, ...) is repopulated by initialize_handler.
// Caller is responsible for any subclass-specific re-init that
// sits on top of the base session (e.g. dictionary loading on
// ONNXOCRRecognizer is not affected because it lives in the
// subclass and survives a session swap).
void Reinitialize(EngineType newEngine);
private:
void initialize_handler();
protected:
+10 -8
View File
@@ -463,7 +463,9 @@ namespace ANSCENTER
auto info = outputs[maskIdx].GetTensorTypeAndShapeInfo();
m_cachedLangMaskShape = info.GetShape();
size_t count = info.GetElementCount();
const bool* data = outputs[maskIdx].GetTensorData<bool>();
// GetTensorMutableData not GetTensorData on DML — const variant
// hangs after ~8 calls. Read-only despite the "Mutable" name.
const bool* data = outputs[maskIdx].GetTensorMutableData<bool>();
m_cachedLangMask.resize(count);
for (size_t i = 0; i < count; ++i)
m_cachedLangMask[i] = data[i] ? 1 : 0;
@@ -474,7 +476,7 @@ namespace ANSCENTER
auto info = outputs[featIdx].GetTensorTypeAndShapeInfo();
m_cachedLangFeaturesShape = info.GetShape();
size_t count = info.GetElementCount();
const float* data = outputs[featIdx].GetTensorData<float>();
const float* data = outputs[featIdx].GetTensorMutableData<float>();
m_cachedLangFeatures.assign(data, data + count);
}
@@ -649,7 +651,7 @@ namespace ANSCENTER
if (elemType == ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT && !shape.empty()) {
size_t numElems = info.GetElementCount();
if (numElems > 0 && numElems < 100000000) {
const float* data = decInputs[di].GetTensorData<float>();
const float* data = decInputs[di].GetTensorMutableData<float>();
double sum = 0;
for (size_t k = 0; k < numElems; ++k) sum += data[k];
double mean = sum / numElems;
@@ -661,14 +663,14 @@ namespace ANSCENTER
// Print bool tensor values (for language_mask)
else if (elemType == ONNX_TENSOR_ELEMENT_DATA_TYPE_BOOL && !shape.empty()) {
size_t numElems = info.GetElementCount();
const bool* data = decInputs[di].GetTensorData<bool>();
const bool* data = decInputs[di].GetTensorMutableData<bool>();
std::cout << " vals:";
for (size_t k = 0; k < std::min(numElems, (size_t)32); ++k)
std::cout << " " << (int)data[k];
}
// Print int64 scalar value
else if (elemType == ONNX_TENSOR_ELEMENT_DATA_TYPE_INT64 && shape.empty()) {
const int64_t* data = decInputs[di].GetTensorData<int64_t>();
const int64_t* data = decInputs[di].GetTensorMutableData<int64_t>();
std::cout << " value=" << data[0];
}
std::cout << std::endl;
@@ -702,10 +704,10 @@ namespace ANSCENTER
auto boxInfo = decOutputs[boxesIdx].GetTensorTypeAndShapeInfo();
auto boxShape = boxInfo.GetShape();
int numBoxes = (boxShape.size() >= 1) ? static_cast<int>(boxShape[0]) : 0;
const float* boxesData = decOutputs[boxesIdx].GetTensorData<float>();
const float* boxesData = decOutputs[boxesIdx].GetTensorMutableData<float>();
// Get scores
const float* scoresData = decOutputs[scoresIdx].GetTensorData<float>();
const float* scoresData = decOutputs[scoresIdx].GetTensorMutableData<float>();
// Get masks
auto maskInfo = decOutputs[masksIdx].GetTensorTypeAndShapeInfo();
@@ -713,7 +715,7 @@ namespace ANSCENTER
// masks shape: [N, 1, H, W]
int maskH = (maskShape.size() >= 3) ? static_cast<int>(maskShape[2]) : 0;
int maskW = (maskShape.size() >= 4) ? static_cast<int>(maskShape[3]) : 0;
const bool* masksData = decOutputs[masksIdx].GetTensorData<bool>();
const bool* masksData = decOutputs[masksIdx].GetTensorMutableData<bool>();
m_maskH = maskH;
m_maskW = maskW;
@@ -2411,6 +2411,12 @@ bool Engine<T>::buildWithRetry(std::string onnxModelPath,
const std::array<float, 3>& divVals,
bool normalize)
{
ANS_DBG("TRT_Build", "buildWithRetry ENTRY onnx=%s normalize=%d optMaxHW=%dx%d optOptHW=%dx%d optMinHW=%dx%d",
onnxModelPath.c_str(), (int)normalize,
m_options.maxInputHeight, m_options.maxInputWidth,
m_options.optInputHeight, m_options.optInputWidth,
m_options.minInputHeight, m_options.minInputWidth);
// -- Quick pre-analysis: detect dynamic spatial dims in ONNX ---------------
bool hasDynamicSpatial = false;
int onnxFixedH = 0, onnxFixedW = 0; // 0 = dynamic (-1 in ONNX)
@@ -2423,6 +2429,10 @@ bool Engine<T>::buildWithRetry(std::string onnxModelPath,
nvonnxparser::createParser(*tempNetwork, m_logger));
std::ifstream onnxFile(onnxModelPath, std::ios::binary | std::ios::ate);
if (!onnxFile.good()) {
ANS_DBG("TRT_Build", "buildWithRetry WARN cannot open ONNX for pre-parse path=%s",
onnxModelPath.c_str());
}
if (onnxFile.good()) {
std::streamsize onnxSize = onnxFile.tellg();
onnxFile.seekg(0, std::ios::beg);
@@ -2432,6 +2442,8 @@ bool Engine<T>::buildWithRetry(std::string onnxModelPath,
bool retryParsed = parseOnnxModelSafe(tempParser.get(),
onnxBuffer.data(), onnxBuffer.size(), &sehRetryParse);
if (sehRetryParse != 0) {
ANS_DBG("TRT_Build", "buildWithRetry WARN SEH=0x%lx during pre-parse — fall through to single build()",
sehRetryParse);
// hasDynamicSpatial stays false → single build() attempt
}
else if (retryParsed && tempNetwork->getNbInputs() > 0) {
@@ -2441,19 +2453,39 @@ bool Engine<T>::buildWithRetry(std::string onnxModelPath,
hasDynamicSpatial = true;
onnxFixedH = (dims.d[2] != -1) ? dims.d[2] : 0;
onnxFixedW = (dims.d[3] != -1) ? dims.d[3] : 0;
ANS_DBG("TRT_Build", "buildWithRetry pre-parse OK nbDims=%d dims=[%d,%d,%d,%d] dynSpatial=%d fixedHW=%dx%d",
dims.nbDims,
(int)dims.d[0], (int)dims.d[1], (int)dims.d[2], (int)dims.d[3],
(int)hasDynamicSpatial, onnxFixedH, onnxFixedW);
} else {
ANS_DBG("TRT_Build", "buildWithRetry WARN nbDims=%d (<4) — treating as fixed", dims.nbDims);
}
} else {
ANS_DBG("TRT_Build", "buildWithRetry WARN parse failed or no inputs (parsed=%d nbInputs=%d)",
(int)retryParsed,
retryParsed ? tempNetwork->getNbInputs() : -1);
}
} else {
ANS_DBG("TRT_Build", "buildWithRetry WARN ONNX read failed bytes=%zd",
(long long)onnxSize);
}
}
} else {
ANS_DBG("TRT_Build", "buildWithRetry SKIP pre-parse (maxInputHW=%dx%d not both >0)",
m_options.maxInputHeight, m_options.maxInputWidth);
}
// -- Fixed-spatial or no dynamic dims: single build attempt ----------------
if (!hasDynamicSpatial) {
ANS_DBG("TRT_Build", "buildWithRetry FIXED_SPATIAL → single buildSafe() attempt");
unsigned long sehBuild = 0;
bool ok = buildSafe(onnxModelPath, subVals, divVals, normalize, &sehBuild);
if (sehBuild != 0) {
ANS_DBG("TRT_Build", "buildWithRetry FAIL fixed-spatial SEH=0x%lx", sehBuild);
return false;
}
ANS_DBG("TRT_Build", "buildWithRetry %s fixed-spatial",
ok ? "SUCCESS" : "FAIL");
return ok;
}
@@ -2482,6 +2514,16 @@ bool Engine<T>::buildWithRetry(std::string onnxModelPath,
if (candidates.back() > 640) candidates.push_back(640);
if (candidates.back() > 320) candidates.push_back(320);
{
std::ostringstream oss;
for (size_t i = 0; i < candidates.size(); ++i) {
if (i) oss << ",";
oss << candidates[i];
}
ANS_DBG("TRT_Build", "buildWithRetry DYNAMIC_SPATIAL dynH=%d dynW=%d maxDynDim=%d candidates=[%s]",
(int)dynamicH, (int)dynamicW, maxDynDim, oss.str().c_str());
}
// Helper: configure m_options for a given candidate
auto setCandidateOptions = [&](int candidate) {
float scale = static_cast<float>(candidate) / maxDynDim;
@@ -2506,16 +2548,28 @@ bool Engine<T>::buildWithRetry(std::string onnxModelPath,
for (size_t attempt = 0; attempt < candidates.size(); ++attempt) {
setCandidateOptions(candidates[attempt]);
ANS_DBG("TRT_Build", "buildWithRetry ATTEMPT %zu/%zu candidate=%d maxHW=%dx%d optHW=%dx%d minHW=%dx%d",
attempt + 1, candidates.size(), candidates[attempt],
m_options.maxInputHeight, m_options.maxInputWidth,
m_options.optInputHeight, m_options.optInputWidth,
m_options.minInputHeight, m_options.minInputWidth);
{
unsigned long sehAttempt = 0;
bool attemptOk = buildSafe(onnxModelPath, subVals, divVals, normalize, &sehAttempt);
if (sehAttempt != 0) {
ANS_DBG("TRT_Build", "buildWithRetry FAIL SEH=0x%lx on candidate=%d — abort (CUDA may be corrupted)",
sehAttempt, candidates[attempt]);
// CUDA context may be corrupted — no point retrying
return false;
}
if (attemptOk) {
ANS_DBG("TRT_Build", "buildWithRetry SUCCESS at attempt %zu/%zu candidate=%d (final maxHW=%dx%d)",
attempt + 1, candidates.size(), candidates[attempt],
m_options.maxInputHeight, m_options.maxInputWidth);
return true;
}
ANS_DBG("TRT_Build", "buildWithRetry attempt %zu FAILED — trying smaller", attempt + 1);
}
}
@@ -2527,6 +2581,8 @@ bool Engine<T>::buildWithRetry(std::string onnxModelPath,
m_options.minInputHeight = origMinH;
m_options.minInputWidth = origMinW;
ANS_DBG("TRT_Build", "buildWithRetry FAIL all %zu candidates exhausted — restored origMaxHW=%dx%d",
candidates.size(), origMaxH, origMaxW);
return false;
}
@@ -2553,6 +2609,9 @@ bool Engine<T>::buildLoadNetwork(
int maxSlotsPerGpu,
double memSafetyFactor)
{
ANS_DBG("TRT_Build", "buildLoadNetwork(6p) ENTRY onnx=%s normalize=%d maxSlotsPerGpu=%d memSafety=%.3f",
onnxModelPath.c_str(), (int)normalize, maxSlotsPerGpu, memSafetyFactor);
// Force single-GPU when: maxSlotsPerGpu==0 (optimizer bypass),
// per-instance forceNoPool, global bypass (OptimizeModelStr),
// exported g_forceNoPool, OR single-GPU system with maxSlotsPerGpu==1.
@@ -2563,18 +2622,32 @@ bool Engine<T>::buildLoadNetwork(
{
extern std::atomic<bool> g_forceNoPool;
int gpuCount = 0;
cudaGetDeviceCount(&gpuCount);
cudaError_t cudaErr = cudaGetDeviceCount(&gpuCount);
if (cudaErr != cudaSuccess) {
ANS_DBG("TRT_Build", "buildLoadNetwork(6p) WARN cudaGetDeviceCount err=%d (%s) — assuming gpuCount=0",
(int)cudaErr, cudaGetErrorString(cudaErr));
}
bool singleGpuNoElastic = (gpuCount <= 1 && maxSlotsPerGpu == 1);
bool gForceNoPool = g_forceNoPool.load(std::memory_order_relaxed);
bool globalBypass = TRTEngineCache::globalBypass().load(std::memory_order_relaxed);
bool noPool = (maxSlotsPerGpu == 0) || m_forceNoPool ||
g_forceNoPool.load(std::memory_order_relaxed) ||
TRTEngineCache::globalBypass().load(std::memory_order_relaxed) ||
singleGpuNoElastic;
gForceNoPool || globalBypass || singleGpuNoElastic;
ANS_DBG("TRT_Build", "buildLoadNetwork(6p) DECISION gpuCount=%d maxSlots==0:%d m_forceNoPool=%d g_forceNoPool=%d globalBypass=%d singleGpuNoElastic=%d → noPool=%d",
gpuCount, (int)(maxSlotsPerGpu == 0), (int)m_forceNoPool,
(int)gForceNoPool, (int)globalBypass, (int)singleGpuNoElastic,
(int)noPool);
if (noPool) {
std::cout << "Info: buildLoadNetwork -- single-GPU forced (maxSlots=" << maxSlotsPerGpu
<< ", forceNoPool=" << m_forceNoPool
<< ", g_forceNoPool=" << g_forceNoPool.load()
<< ", gpuCount=" << gpuCount << ")" << std::endl;
return buildLoadNetwork(onnxModelPath, subVals, divVals, normalize);
ANS_DBG("TRT_Build", "buildLoadNetwork(6p) → single-GPU 4-param overload");
bool ok = buildLoadNetwork(onnxModelPath, subVals, divVals, normalize);
ANS_DBG("TRT_Build", "buildLoadNetwork(6p) %s (single-GPU path)",
ok ? "SUCCESS" : "FAIL");
return ok;
}
}
@@ -2583,11 +2656,16 @@ bool Engine<T>::buildLoadNetwork(
std::cout << "Info: buildLoadNetwork -- activating multi-GPU pool"
<< " (maxSlotsPerGpu=" << maxSlotsPerGpu
<< ", memSafetyFactor=" << memSafetyFactor << ")" << std::endl;
ANS_DBG("TRT_Build", "buildLoadNetwork(6p) → loadSlots multi-GPU pool maxSlotsPerGpu=%d memSafety=%.3f",
maxSlotsPerGpu, memSafetyFactor);
return loadSlots(m_options, onnxModelPath,
subVals, divVals, normalize,
/*fromOnnx=*/true,
maxSlotsPerGpu, memSafetyFactor);
bool ok = loadSlots(m_options, onnxModelPath,
subVals, divVals, normalize,
/*fromOnnx=*/true,
maxSlotsPerGpu, memSafetyFactor);
ANS_DBG("TRT_Build", "buildLoadNetwork(6p) %s (multi-GPU pool path)",
ok ? "SUCCESS" : "FAIL");
return ok;
}
template <typename T>
+90 -53
View File
@@ -1,6 +1,7 @@
#include "ANSFLV.h"
#include "ANSMatRegistry.h"
#include "ANSGpuFrameOps.h"
#include "av_frame_ptr.h" // ansmedia::AvFramePtr — RAII for AVFrame*
#include "ANSCVVendorGate.h" // anscv_vendor_gate::IsNvidiaGpuAvailable()
#include "ANSLicense.h" // ANS_DBG macro
#include <memory>
@@ -223,13 +224,11 @@ namespace ANSCENTER {
}
}
// Honest image-equality check. Same buffer pointer (fast path) or pixel
// compare (fallback). Camera-aliveness uses getLastFrameAgeMs() — separate
// concern. With "same image -> same pts" invariant, callers usually just
// compare pts and skip this function entirely.
bool ANSFLVClient::areImagesIdentical(const cv::Mat& img1, const cv::Mat& img2) {
// Use decoder frame age — returns "stale" only if no decoder output for 5+ seconds.
double ageMs = _playerClient->getLastFrameAgeMs();
if (ageMs > 5000.0) return true; // Truly stale
if (ageMs > 0.0) return false; // Decoder alive
// Fallback for startup (no frame decoded yet)
if (img1.empty() && img2.empty()) return true;
if (img1.empty() || img2.empty()) return false;
if (img1.size() != img2.size() || img1.type() != img2.type()) return false;
@@ -287,17 +286,19 @@ namespace ANSCENTER {
return _pLastFrame;
}
// Use PTS to detect duplicate frames (much faster than pixel comparison)
if (currentPts == _pts && !_pLastFrame.empty()) {
// Duplicate detection via _lastInnerPts (separate from _pts). Invariant:
// same image -> same pts. Camera-alive check uses getLastFrameAgeMs().
if (currentPts == _lastInnerPts && !_pLastFrame.empty()) {
width = _imageWidth;
height = _imageHeight;
pts = _pts;
pts = _pts; // unchanged
return _pLastFrame;
}
// Update PTS first
_pts = currentPts;
pts = currentPts;
// Inner advanced — real new frame. Sync cursor and bump _pts monotonically.
_lastInnerPts = currentPts;
_pts++;
pts = _pts;
// Handle non-rotated case
if (_imageRotateDeg == 0) {
@@ -635,6 +636,14 @@ namespace ANSCENTER {
std::lock_guard<std::recursive_mutex> lock(_mutex);
_playerClient->setTargetFPS(intervalMs); // 0=no limit, 100=~10FPS, 200=~5FPS
}
void ANSFLVClient::SetJitterBufferMs(int ms) {
std::lock_guard<std::recursive_mutex> lock(_mutex);
_playerClient->setJitterBufferMs(ms);
ANS_DBG("FLV_Lifecycle", "SetJitterBufferMs: ms=%d url=%s", ms, _url.c_str());
}
int ANSFLVClient::GetJitterBufferMs() const {
return _playerClient->getJitterBufferMs();
}
void ANSFLVClient::SetNV12FastPath(bool enable) {
std::lock_guard<std::recursive_mutex> lock(_mutex);
_useNV12FastPath = enable;
@@ -729,11 +738,11 @@ extern "C" __declspec(dllexport) int GetFLVStrImage(ANSCENTER::ANSFLVClient * *H
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error getting FLV image: " << e.what() << std::endl;
ANS_DBG("FLV_GetImage", "exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error getting FLV image: Unknown exception." << std::endl;
ANS_DBG("FLV_GetImage", "unknown exception");
return 0;
}
}
@@ -751,27 +760,27 @@ extern "C" __declspec(dllexport) int GetFLVImage(ANSCENTER::ANSFLVClient * *Hand
return 1;
}
else {
std::cerr << "Error resizing jpegImage handle: " << error << std::endl;
ANS_DBG("FLV_GetImage", "DSSetHandleSize failed: error=%d", (int)error);
return 0;
}
}
else {
std::cerr << "No image data retrieved from FLV client." << std::endl;
ANS_DBG("FLV_GetImage", "no image data retrieved (jpegString empty)");
return 0;
}
}
catch (const std::exception& e) {
std::cerr << "Error getting FLV image: " << e.what() << std::endl;
ANS_DBG("FLV_GetImage", "exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error getting FLV image: Unknown exception." << std::endl;
ANS_DBG("FLV_GetImage", "unknown exception");
return 0;
}
}
extern "C" __declspec(dllexport) int GetFLVCVImage(ANSCENTER::ANSFLVClient** Handle, int& width, int& height, int64_t& timeStamp, cv::Mat** image) {
if (!Handle || !*Handle || !image) {
std::cerr << "Error: Invalid input parameters in GetFLVCVImage" << std::endl;
ANS_DBG("FLV_GetImage", "invalid input parameters");
return -1;
}
@@ -785,17 +794,25 @@ extern "C" __declspec(dllexport) int GetFLVCVImage(ANSCENTER::ANSFLVClient** Han
// Thread-safe Mat pointer swap (anscv_mat_replace has its own internal lock)
anscv_mat_replace(image, std::move(img));
// NV12 GPU fast path (optional — disabled by default for stability)
// NV12 GPU fast path (optional — disabled by default for stability).
// AvFramePtr auto-frees clones if we don't reach the .release() handoff.
if ((*Handle)->IsNV12FastPath()) {
int gpuIdx = (*Handle)->GetHWDecodingGpuIndex();
AVFrame* cudaHW = (*Handle)->GetCudaHWFrame();
ansmedia::AvFramePtr cudaHW{ (*Handle)->GetCudaHWFrame() };
if (cudaHW) {
AVFrame* cpuNV12 = (*Handle)->GetNV12Frame();
gpu_frame_attach_cuda(*image, cudaHW, gpuIdx, timeStamp, cpuNV12);
ansmedia::AvFramePtr cpuNV12{ (*Handle)->GetNV12Frame() };
ANS_DBG("MEDIA_AvPtr",
"ANSFLV V1 attach_cuda cudaHW=%p cpuNV12=%p gpuIdx=%d",
(void*)cudaHW.get(), (void*)cpuNV12.get(), gpuIdx);
gpu_frame_attach_cuda(*image, cudaHW.release(), gpuIdx, timeStamp,
cpuNV12.release());
} else {
AVFrame* nv12 = (*Handle)->GetNV12Frame();
ansmedia::AvFramePtr nv12{ (*Handle)->GetNV12Frame() };
if (nv12) {
gpu_frame_attach(*image, nv12, gpuIdx, timeStamp);
ANS_DBG("MEDIA_AvPtr",
"ANSFLV V1 attach nv12=%p gpuIdx=%d",
(void*)nv12.get(), gpuIdx);
gpu_frame_attach(*image, nv12.release(), gpuIdx, timeStamp);
}
}
}
@@ -803,15 +820,15 @@ extern "C" __declspec(dllexport) int GetFLVCVImage(ANSCENTER::ANSFLVClient** Han
return 1;
}
catch (const cv::Exception& e) {
std::cerr << "OpenCV exception in GetFLVCVImage: " << e.what() << std::endl;
ANS_DBG("FLV_GetImage", "OpenCV exception: %s", e.what());
return -2;
}
catch (const std::exception& e) {
std::cerr << "Exception in GetFLVCVImage: " << e.what() << std::endl;
ANS_DBG("FLV_GetImage", "GetFLVCVImage exception: %s", e.what());
return -2;
}
catch (...) {
std::cerr << "Unknown exception in GetFLVCVImage" << std::endl;
ANS_DBG("FLV_GetImage", "GetFLVCVImage unknown exception");
return -2;
}
}
@@ -824,11 +841,11 @@ extern "C" __declspec(dllexport) int StartFLV(ANSCENTER::ANSFLVClient * *Handle)
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error starting FLV client: " << e.what() << std::endl;
ANS_DBG("FLV_Lifecycle", "StartFLV exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error starting FLV client: Unknown exception." << std::endl;
ANS_DBG("FLV_Lifecycle", "StartFLV unknown exception");
return 0;
}
}
@@ -840,11 +857,11 @@ extern "C" __declspec(dllexport) int ReconnectFLV(ANSCENTER::ANSFLVClient * *Han
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error reconnecting FLV client: " << e.what() << std::endl;
ANS_DBG("FLV_Lifecycle", "ReconnectFLV exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error reconnecting FLV client: Unknown exception." << std::endl;
ANS_DBG("FLV_Lifecycle", "ReconnectFLV unknown exception");
return 0;
}
}
@@ -856,11 +873,11 @@ extern "C" __declspec(dllexport) int StopFLV(ANSCENTER::ANSFLVClient * *Handle)
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error stopping FLV client: " << e.what() << std::endl;
ANS_DBG("FLV_Lifecycle", "StopFLV exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error stopping FLV client: Unknown exception." << std::endl;
ANS_DBG("FLV_Lifecycle", "StopFLV unknown exception");
return 0;
}
}
@@ -971,6 +988,18 @@ extern "C" __declspec(dllexport) void SetFLVDisplayResolution(ANSCENTER::ANSFLVC
(*Handle)->SetDisplayResolution(width, height);
} catch (...) { }
}
extern "C" __declspec(dllexport) void SetFLVJitterBuffer(ANSCENTER::ANSFLVClient** Handle, int jitterMs) {
if (Handle == nullptr || *Handle == nullptr) return;
try {
(*Handle)->SetJitterBufferMs(jitterMs);
} catch (...) { }
}
extern "C" __declspec(dllexport) int GetFLVJitterBuffer(ANSCENTER::ANSFLVClient** Handle) {
if (Handle == nullptr || *Handle == nullptr) return -1;
try {
return (*Handle)->GetJitterBufferMs();
} catch (...) { return -1; }
}
extern "C" __declspec(dllexport) void SetFLVTargetFPS(ANSCENTER::ANSFLVClient** Handle, double intervalMs) {
if (Handle == nullptr || *Handle == nullptr) return;
try {
@@ -1003,21 +1032,21 @@ extern "C" __declspec(dllexport) int GetFLVImage_V2(uint64_t handleVal, int& wid
return 1;
}
else {
std::cerr << "Error resizing jpegImage handle: " << error << std::endl;
ANS_DBG("FLV_GetImage", "DSSetHandleSize failed: error=%d", (int)error);
return 0;
}
}
else {
std::cerr << "No image data retrieved from FLV client." << std::endl;
ANS_DBG("FLV_GetImage", "no image data retrieved (jpegString empty)");
return 0;
}
}
catch (const std::exception& e) {
std::cerr << "Error getting FLV image: " << e.what() << std::endl;
ANS_DBG("FLV_GetImage", "exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error getting FLV image: Unknown exception." << std::endl;
ANS_DBG("FLV_GetImage", "unknown exception");
return 0;
}
}
@@ -1032,31 +1061,39 @@ extern "C" __declspec(dllexport) int GetFLVCVImage_V2(uint64_t handleVal, int& w
}
anscv_mat_replace(image, std::move(img));
// Attach NV12 frame for GPU fast-path inference (side-table registry)
// Attach NV12 frame for GPU fast-path inference (side-table registry).
// AvFramePtr auto-frees clones if we don't reach the .release() handoff.
int gpuIdx = h->GetHWDecodingGpuIndex();
AVFrame* cudaHW = h->GetCudaHWFrame();
ansmedia::AvFramePtr cudaHW{ h->GetCudaHWFrame() };
if (cudaHW) {
AVFrame* cpuNV12 = h->GetNV12Frame();
gpu_frame_attach_cuda(*image, cudaHW, gpuIdx, timeStamp, cpuNV12);
ansmedia::AvFramePtr cpuNV12{ h->GetNV12Frame() };
ANS_DBG("MEDIA_AvPtr",
"ANSFLV V2 attach_cuda cudaHW=%p cpuNV12=%p gpuIdx=%d",
(void*)cudaHW.get(), (void*)cpuNV12.get(), gpuIdx);
gpu_frame_attach_cuda(*image, cudaHW.release(), gpuIdx, timeStamp,
cpuNV12.release());
} else {
AVFrame* nv12 = h->GetNV12Frame();
ansmedia::AvFramePtr nv12{ h->GetNV12Frame() };
if (nv12) {
gpu_frame_attach(*image, nv12, gpuIdx, timeStamp);
ANS_DBG("MEDIA_AvPtr",
"ANSFLV V2 attach nv12=%p gpuIdx=%d",
(void*)nv12.get(), gpuIdx);
gpu_frame_attach(*image, nv12.release(), gpuIdx, timeStamp);
}
}
return 1;
}
catch (const cv::Exception& e) {
std::cerr << "OpenCV exception in GetFLVCVImage_V2: " << e.what() << std::endl;
ANS_DBG("FLV_GetImage", "GetFLVCVImage_V2 OpenCV exception: %s", e.what());
return -2;
}
catch (const std::exception& e) {
std::cerr << "Exception in GetFLVCVImage_V2: " << e.what() << std::endl;
ANS_DBG("FLV_GetImage", "GetFLVCVImage_V2 exception: %s", e.what());
return -2;
}
catch (...) {
std::cerr << "Unknown exception in GetFLVCVImage_V2" << std::endl;
ANS_DBG("FLV_GetImage", "GetFLVCVImage_V2 unknown exception");
return -2;
}
}
@@ -1069,11 +1106,11 @@ extern "C" __declspec(dllexport) int StartFLV_V2(uint64_t handleVal) {
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error starting FLV client: " << e.what() << std::endl;
ANS_DBG("FLV_Lifecycle", "StartFLV exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error starting FLV client: Unknown exception." << std::endl;
ANS_DBG("FLV_Lifecycle", "StartFLV unknown exception");
return 0;
}
}
@@ -1086,11 +1123,11 @@ extern "C" __declspec(dllexport) int ReconnectFLV_V2(uint64_t handleVal) {
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error reconnecting FLV client: " << e.what() << std::endl;
ANS_DBG("FLV_Lifecycle", "ReconnectFLV exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error reconnecting FLV client: Unknown exception." << std::endl;
ANS_DBG("FLV_Lifecycle", "ReconnectFLV unknown exception");
return 0;
}
}
@@ -1103,11 +1140,11 @@ extern "C" __declspec(dllexport) int StopFLV_V2(uint64_t handleVal) {
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error stopping FLV client: " << e.what() << std::endl;
ANS_DBG("FLV_Lifecycle", "StopFLV exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error stopping FLV client: Unknown exception." << std::endl;
ANS_DBG("FLV_Lifecycle", "StopFLV unknown exception");
return 0;
}
}
+9
View File
@@ -35,6 +35,9 @@ namespace ANSCENTER
std::string _lastJpegImage;
int _imageWidth, _imageHeight;
int64_t _pts;
// Last currentPts seen from inner getImage(). Used for duplicate
// detection separately from _pts so "same image -> same pts" holds.
int64_t _lastInnerPts = -1;
bool _isPlaying;
bool _useNV12FastPath = false; // false = original stable CPU path, true = NV12 GPU fast path
std::recursive_mutex _mutex;
@@ -73,6 +76,10 @@ namespace ANSCENTER
void SetDisplayResolution(int width, int height); // Set display output size; 0,0 = original (no resize)
void SetImageQuality(int mode); // 0=fast (AI), 1=quality (display)
void SetTargetFPS(double intervalMs); // Set min interval between processed frames in ms (0 = no limit, 100 = ~10 FPS, 200 = ~5 FPS)
// Jitter buffer for the BGR delivery path (forwarded to CVideoPlayer).
// 0 = no buffering (today's getLatest semantics). Default 200 ms.
void SetJitterBufferMs(int ms);
int GetJitterBufferMs() const;
void SetNV12FastPath(bool enable); // true = NV12 GPU fast path, false = original CPU path (stable)
bool IsNV12FastPath() const { return _useNV12FastPath; }
AVFrame* GetNV12Frame(); // Returns cloned NV12 frame for GPU fast-path (caller must av_frame_free)
@@ -114,4 +121,6 @@ extern "C" __declspec(dllexport) void SetFLVImageQuality(ANSCENTER::ANSFLVClient
extern "C" __declspec(dllexport) void SetFLVDisplayResolution(ANSCENTER::ANSFLVClient** Handle, int width, int height);
extern "C" __declspec(dllexport) void SetFLVTargetFPS(ANSCENTER::ANSFLVClient** Handle, double intervalMs);
extern "C" __declspec(dllexport) void SetFLVNV12FastPath(ANSCENTER::ANSFLVClient** Handle, int enable);
extern "C" __declspec(dllexport) void SetFLVJitterBuffer(ANSCENTER::ANSFLVClient** Handle, int jitterMs);
extern "C" __declspec(dllexport) int GetFLVJitterBuffer(ANSCENTER::ANSFLVClient** Handle);
#endif
+81 -43
View File
@@ -1,6 +1,7 @@
#include "ANSFilePlayer.h"
#include "ANSMatRegistry.h"
#include "ANSGpuFrameOps.h"
#include "av_frame_ptr.h" // ansmedia::AvFramePtr — RAII for AVFrame*
#include <memory>
#include <cstdint>
#include <libswscale/swscale.h>
@@ -218,6 +219,14 @@ namespace ANSCENTER {
_displayWidth = width;
_displayHeight = height;
}
void ANSFILEPLAYER::SetJitterBufferMs(int ms) {
std::lock_guard<std::recursive_mutex> lock(_mutex);
if (_playerClient) _playerClient->setJitterBufferMs(ms);
ANS_DBG("FilePlayer_Lifecycle", "SetJitterBufferMs: ms=%d url=%s", ms, _url.c_str());
}
int ANSFILEPLAYER::GetJitterBufferMs() const {
return _playerClient ? _playerClient->getJitterBufferMs() : 0;
}
cv::Mat ANSFILEPLAYER::GetInferenceImage() {
std::lock_guard<std::recursive_mutex> lock(_mutex);
return _inferenceImage;
@@ -323,7 +332,7 @@ extern "C" __declspec(dllexport) int GetFilePlayerStrImage(ANSCENTER::ANSFILEPLA
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error getting image data from FilePlayer client: " << e.what() << std::endl;
ANS_DBG("FilePlayer_GetImage", "exception: %s", e.what());
return -1;
}
catch (...) {
@@ -350,18 +359,18 @@ extern "C" __declspec(dllexport) int GetFilePlayerImage(ANSCENTER::ANSFILEPLAYER
}
else {
// Return failure if there was an error in resizing the handle
std::cerr << "Error resizing jpegImage handle: " << error << std::endl;
ANS_DBG("FilePlayer_GetImage", "DSSetHandleSize failed: error=%d", (int)error);
return 0;
}
}
else {
// If the JPEG image string is empty, return failure
std::cerr << "No image data retrieved from FilePlayer client." << std::endl;
ANS_DBG("FilePlayer_GetImage", "no image data retrieved (jpegString empty)");
return 0;
}
}
catch (const std::exception& e) {
std::cerr << "Error getting image data from FilePlayer client: " << e.what() << std::endl;
ANS_DBG("FilePlayer_GetImage", "exception: %s", e.what());
return -1;
}
catch (...) {
@@ -370,7 +379,7 @@ extern "C" __declspec(dllexport) int GetFilePlayerImage(ANSCENTER::ANSFILEPLAYER
}
extern "C" __declspec(dllexport) int GetFilePlayerCVImage(ANSCENTER::ANSFILEPLAYER** Handle, int& width, int& height, int64_t& timeStamp, cv::Mat** image) {
if (!Handle || !(*Handle) || !image) {
std::cerr << "Error: Invalid input parameters in GetFilePlayerCVImage." << std::endl;
ANS_DBG("FilePlayer_GetImage", "GetFilePlayerCVImage invalid input parameters");
return -1;
}
try {
@@ -382,15 +391,24 @@ extern "C" __declspec(dllexport) int GetFilePlayerCVImage(ANSCENTER::ANSFILEPLAY
// Attach NV12/CUDA frames to registry for inference zero-copy (same as ANSRTSP/ANSVideoPlayer).
// CFilePlayer inherits getNV12Frame()/getCudaHWFrame() from CVideoPlayer.
// AvFramePtr wraps each clone so it auto-frees on any return path before
// the .release() handoff to gpu_frame_attach[_cuda] — leak-by-omission
// becomes impossible if an exception or early return slips in later.
int gpuIdx = (*Handle)->_playerClient->getHWDecodingGpuIndex();
AVFrame* cudaHW = (*Handle)->_playerClient->getCudaHWFrame();
ansmedia::AvFramePtr cudaHW{ (*Handle)->_playerClient->getCudaHWFrame() };
if (cudaHW) {
AVFrame* cpuNV12 = (*Handle)->_playerClient->getNV12Frame();
gpu_frame_attach_cuda(*image, cudaHW, gpuIdx, timeStamp, cpuNV12);
ansmedia::AvFramePtr cpuNV12{ (*Handle)->_playerClient->getNV12Frame() };
ANS_DBG("MEDIA_AvPtr",
"ANSFilePlayer V1 attach_cuda cudaHW=%p cpuNV12=%p gpuIdx=%d",
(void*)cudaHW.get(), (void*)cpuNV12.get(), gpuIdx);
gpu_frame_attach_cuda(*image, cudaHW.release(), gpuIdx, timeStamp, cpuNV12.release());
} else {
AVFrame* nv12 = (*Handle)->_playerClient->getNV12Frame();
ansmedia::AvFramePtr nv12{ (*Handle)->_playerClient->getNV12Frame() };
if (nv12) {
gpu_frame_attach(*image, nv12, gpuIdx, timeStamp);
ANS_DBG("MEDIA_AvPtr",
"ANSFilePlayer V1 attach nv12=%p gpuIdx=%d",
(void*)nv12.get(), gpuIdx);
gpu_frame_attach(*image, nv12.release(), gpuIdx, timeStamp);
} else {
// CPU-only fallback: attach full-res BGR clone for ANSRTYOLO
cv::Mat infImg = (*Handle)->GetInferenceImage();
@@ -429,15 +447,15 @@ extern "C" __declspec(dllexport) int GetFilePlayerCVImage(ANSCENTER::ANSFILEPLAY
return 1;
}
catch (const cv::Exception& e) {
std::cerr << "OpenCV exception in GetFilePlayerCVImage: " << e.what() << std::endl;
ANS_DBG("FilePlayer_GetImage", "GetFilePlayerCVImage OpenCV exception: %s", e.what());
return -2;
}
catch (const std::exception& e) {
std::cerr << "Exception in GetFilePlayerCVImage: " << e.what() << std::endl;
ANS_DBG("FilePlayer_GetImage", "GetFilePlayerCVImage exception: %s", e.what());
return -2;
}
catch (...) {
std::cerr << "Unknown exception in GetFilePlayerCVImage" << std::endl;
ANS_DBG("FilePlayer_GetImage", "GetFilePlayerCVImage unknown exception");
return -2;
}
}
@@ -450,7 +468,7 @@ extern "C" __declspec(dllexport) int StartFilePlayer(ANSCENTER::ANSFILEPLAYER**
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error starting file player: " << e.what() << std::endl;
ANS_DBG("FilePlayer_Lifecycle", "StartFilePlayer exception: %s", e.what());
return -1;
}
catch (...) {
@@ -465,7 +483,7 @@ extern "C" __declspec(dllexport) int ReconnectFilePlayer(ANSCENTER::ANSFILEPLAYE
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error reconnecting file player: " << e.what() << std::endl;
ANS_DBG("FilePlayer_Lifecycle", "ReconnectFilePlayer exception: %s", e.what());
return -1;
}
catch (...) {
@@ -480,7 +498,7 @@ extern "C" __declspec(dllexport) int StopFilePlayer(ANSCENTER::ANSFILEPLAYER** H
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error stopping file player: " << e.what() << std::endl;
ANS_DBG("FilePlayer_Lifecycle", "StopFilePlayer exception: %s", e.what());
return -1;
}
catch (...) {
@@ -495,7 +513,7 @@ extern "C" __declspec(dllexport) int IsFilePlayerPaused(ANSCENTER::ANSFILEPLAYER
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error checking if file player is paused: " << e.what() << std::endl;
ANS_DBG("FilePlayer_Lifecycle", "IsFilePlayerPaused exception: %s", e.what());
return -1;
}
catch (...) {
@@ -510,7 +528,7 @@ extern "C" __declspec(dllexport) int IsFilePlayerRunning(ANSCENTER::ANSFILEPLAYE
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error checking if file player is running: " << e.what() << std::endl;
ANS_DBG("FilePlayer_Lifecycle", "IsFilePlayerRunning exception: %s", e.what());
return -1;
}
catch (...) {
@@ -525,7 +543,7 @@ extern "C" __declspec(dllexport) int IsFilePlayerRecording(ANSCENTER::ANSFILEPLA
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error checking if file player is recording: " << e.what() << std::endl;
ANS_DBG("FilePlayer_Lifecycle", "IsFilePlayerRecording exception: %s", e.what());
return -1;
}
catch (...) {
@@ -549,7 +567,7 @@ extern "C" __declspec(dllexport) void EnableFilePlayerAudioVolume(ANSCENTER::ANS
(*Handle)->EnableAudio(audioStatus);
}
catch (const std::exception& e) {
std::cerr << "Error enabling audio for file player: " << e.what() << std::endl;
ANS_DBG("FilePlayer_Lifecycle", "EnableFilePlayerAudio exception: %s", e.what());
}
catch (...) { }
}
@@ -568,7 +586,7 @@ extern "C" __declspec(dllexport) int SetBBoxFilePlayer(ANSCENTER::ANSFILEPLAYER*
return 1;
}
catch (const std::exception& e) {
std::cerr << "Error setting bounding box for file player: " << e.what() << std::endl;
ANS_DBG("FilePlayer_Cfg", "SetBBoxFilePlayer exception: %s", e.what());
return -1;
}
catch (...) {
@@ -584,7 +602,7 @@ extern "C" __declspec(dllexport) int SetCropFlagFilePlayer(ANSCENTER::ANSFILEPLA
return 1;
}
catch (const std::exception& e) {
std::cerr << "Error setting crop flag for file player: " << e.what() << std::endl;
ANS_DBG("FilePlayer_Cfg", "SetCropFlagFilePlayer exception: %s", e.what());
return -1;
}
catch (...) {
@@ -611,17 +629,17 @@ extern "C" __declspec(dllexport) int GetFilePlayerImage_V2(uint64_t handleVal, i
return 1;
}
else {
std::cerr << "Error resizing jpegImage handle: " << error << std::endl;
ANS_DBG("FilePlayer_GetImage", "DSSetHandleSize failed: error=%d", (int)error);
return 0;
}
}
else {
std::cerr << "No image data retrieved from FilePlayer client." << std::endl;
ANS_DBG("FilePlayer_GetImage", "no image data retrieved (jpegString empty)");
return 0;
}
}
catch (const std::exception& e) {
std::cerr << "Error getting image data from FilePlayer client: " << e.what() << std::endl;
ANS_DBG("FilePlayer_GetImage", "exception: %s", e.what());
return -1;
}
catch (...) {
@@ -639,16 +657,24 @@ extern "C" __declspec(dllexport) int GetFilePlayerCVImage_V2(uint64_t handleVal,
}
anscv_mat_replace(image, std::move(img));
// NV12/CUDA registry attach (same as V1)
// NV12/CUDA registry attach (same as V1).
// AvFramePtr ensures any clone we obtain is freed automatically if we
// don't reach the .release() handoff (e.g. exception, future early return).
int gpuIdx = h->_playerClient->getHWDecodingGpuIndex();
AVFrame* cudaHW = h->_playerClient->getCudaHWFrame();
ansmedia::AvFramePtr cudaHW{ h->_playerClient->getCudaHWFrame() };
if (cudaHW) {
AVFrame* cpuNV12 = h->_playerClient->getNV12Frame();
gpu_frame_attach_cuda(*image, cudaHW, gpuIdx, timeStamp, cpuNV12);
ansmedia::AvFramePtr cpuNV12{ h->_playerClient->getNV12Frame() };
ANS_DBG("MEDIA_AvPtr",
"ANSFilePlayer V2 attach_cuda cudaHW=%p cpuNV12=%p gpuIdx=%d",
(void*)cudaHW.get(), (void*)cpuNV12.get(), gpuIdx);
gpu_frame_attach_cuda(*image, cudaHW.release(), gpuIdx, timeStamp, cpuNV12.release());
} else {
AVFrame* nv12 = h->_playerClient->getNV12Frame();
ansmedia::AvFramePtr nv12{ h->_playerClient->getNV12Frame() };
if (nv12) {
gpu_frame_attach(*image, nv12, gpuIdx, timeStamp);
ANS_DBG("MEDIA_AvPtr",
"ANSFilePlayer V2 attach nv12=%p gpuIdx=%d",
(void*)nv12.get(), gpuIdx);
gpu_frame_attach(*image, nv12.release(), gpuIdx, timeStamp);
} else {
cv::Mat infImg = h->GetInferenceImage();
if (!infImg.empty() && infImg.data != (*image)->data) {
@@ -686,15 +712,15 @@ extern "C" __declspec(dllexport) int GetFilePlayerCVImage_V2(uint64_t handleVal,
return 1;
}
catch (const cv::Exception& e) {
std::cerr << "OpenCV exception in GetFilePlayerCVImage_V2: " << e.what() << std::endl;
ANS_DBG("FilePlayer_GetImage", "GetFilePlayerCVImage_V2 OpenCV exception: %s", e.what());
return -2;
}
catch (const std::exception& e) {
std::cerr << "Exception in GetFilePlayerCVImage_V2: " << e.what() << std::endl;
ANS_DBG("FilePlayer_GetImage", "GetFilePlayerCVImage_V2 exception: %s", e.what());
return -2;
}
catch (...) {
std::cerr << "Unknown exception in GetFilePlayerCVImage_V2" << std::endl;
ANS_DBG("FilePlayer_GetImage", "GetFilePlayerCVImage_V2 unknown exception");
return -2;
}
}
@@ -708,7 +734,7 @@ extern "C" __declspec(dllexport) int StartFilePlayer_V2(uint64_t handleVal) {
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error starting file player: " << e.what() << std::endl;
ANS_DBG("FilePlayer_Lifecycle", "StartFilePlayer exception: %s", e.what());
return -1;
}
catch (...) {
@@ -725,7 +751,7 @@ extern "C" __declspec(dllexport) int ReconnectFilePlayer_V2(uint64_t handleVal)
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error reconnecting file player: " << e.what() << std::endl;
ANS_DBG("FilePlayer_Lifecycle", "ReconnectFilePlayer exception: %s", e.what());
return -1;
}
catch (...) {
@@ -742,7 +768,7 @@ extern "C" __declspec(dllexport) int StopFilePlayer_V2(uint64_t handleVal) {
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error stopping file player: " << e.what() << std::endl;
ANS_DBG("FilePlayer_Lifecycle", "StopFilePlayer exception: %s", e.what());
return -1;
}
catch (...) {
@@ -759,7 +785,7 @@ extern "C" __declspec(dllexport) int IsFilePlayerPaused_V2(uint64_t handleVal) {
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error checking if file player is paused: " << e.what() << std::endl;
ANS_DBG("FilePlayer_Lifecycle", "IsFilePlayerPaused exception: %s", e.what());
return -1;
}
catch (...) {
@@ -776,7 +802,7 @@ extern "C" __declspec(dllexport) int IsFilePlayerRunning_V2(uint64_t handleVal)
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error checking if file player is running: " << e.what() << std::endl;
ANS_DBG("FilePlayer_Lifecycle", "IsFilePlayerRunning exception: %s", e.what());
return -1;
}
catch (...) {
@@ -793,7 +819,7 @@ extern "C" __declspec(dllexport) int IsFilePlayerRecording_V2(uint64_t handleVal
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error checking if file player is recording: " << e.what() << std::endl;
ANS_DBG("FilePlayer_Lifecycle", "IsFilePlayerRecording exception: %s", e.what());
return -1;
}
catch (...) {
@@ -819,7 +845,7 @@ extern "C" __declspec(dllexport) void EnableFilePlayerAudioVolume_V2(uint64_t ha
h->EnableAudio(audioStatus);
}
catch (const std::exception& e) {
std::cerr << "Error enabling audio for file player: " << e.what() << std::endl;
ANS_DBG("FilePlayer_Lifecycle", "EnableFilePlayerAudio exception: %s", e.what());
}
catch (...) { }
}
@@ -842,7 +868,7 @@ extern "C" __declspec(dllexport) int SetBBoxFilePlayer_V2(uint64_t handleVal, in
return 1;
}
catch (const std::exception& e) {
std::cerr << "Error setting bounding box for file player: " << e.what() << std::endl;
ANS_DBG("FilePlayer_Cfg", "SetBBoxFilePlayer exception: %s", e.what());
return -1;
}
catch (...) {
@@ -860,7 +886,7 @@ extern "C" __declspec(dllexport) int SetCropFlagFilePlayer_V2(uint64_t handleVal
return 1;
}
catch (const std::exception& e) {
std::cerr << "Error setting crop flag for file player: " << e.what() << std::endl;
ANS_DBG("FilePlayer_Cfg", "SetCropFlagFilePlayer exception: %s", e.what());
return -1;
}
catch (...) {
@@ -874,3 +900,15 @@ extern "C" __declspec(dllexport) void SetFilePlayerDisplayResolution(ANSCENTER::
(*Handle)->SetDisplayResolution(width, height);
} catch (...) { }
}
extern "C" __declspec(dllexport) void SetFilePlayerJitterBuffer(ANSCENTER::ANSFILEPLAYER** Handle, int jitterMs) {
if (Handle == nullptr || *Handle == nullptr) return;
try {
(*Handle)->SetJitterBufferMs(jitterMs);
} catch (...) { }
}
extern "C" __declspec(dllexport) int GetFilePlayerJitterBuffer(ANSCENTER::ANSFILEPLAYER** Handle) {
if (Handle == nullptr || *Handle == nullptr) return -1;
try {
return (*Handle)->GetJitterBufferMs();
} catch (...) { return -1; }
}
+6
View File
@@ -68,6 +68,10 @@ namespace ANSCENTER
if (mode > 360) _imageRotateDeg = 360;
};
void SetDisplayResolution(int width, int height);
// Jitter buffer for the BGR delivery path (forwarded to CFilePlayer ->
// CVideoPlayer). 0 = no buffering. Default 200 ms (set by CVideoPlayer).
void SetJitterBufferMs(int ms);
int GetJitterBufferMs() const;
[[nodiscard]] cv::Mat GetImage(int& width, int& height, int64_t& pts);
[[nodiscard]] cv::Mat GetInferenceImage();
[[nodiscard]] std::string GetJpegImage(int& width, int& height, int64_t& pts);
@@ -99,6 +103,8 @@ extern "C" __declspec(dllexport) void SetFilePlayerImageRotation(ANSCENTER::ANSF
extern "C" __declspec(dllexport) int SetBBoxFilePlayer(ANSCENTER::ANSFILEPLAYER** Handle, int x, int y, int width, int height);
extern "C" __declspec(dllexport) int SetCropFlagFilePlayer(ANSCENTER::ANSFILEPLAYER** Handle, int cropFlag);
extern "C" __declspec(dllexport) void SetFilePlayerDisplayResolution(ANSCENTER::ANSFILEPLAYER** Handle, int width, int height);
extern "C" __declspec(dllexport) void SetFilePlayerJitterBuffer(ANSCENTER::ANSFILEPLAYER** Handle, int jitterMs);
extern "C" __declspec(dllexport) int GetFilePlayerJitterBuffer(ANSCENTER::ANSFILEPLAYER** Handle);
// V2 functions: accept uint64_t handleVal by value (LabVIEW-safe)
extern "C" __declspec(dllexport) int GetFilePlayerImage_V2(uint64_t handleVal, int& width, int& height, int64_t& timeStamp, LStrHandle jpegImage);
+90 -51
View File
@@ -1,6 +1,7 @@
#include "ANSMJPEG.h"
#include "ANSMatRegistry.h"
#include "ANSGpuFrameOps.h"
#include "av_frame_ptr.h" // ansmedia::AvFramePtr — RAII for AVFrame*
#include "ANSCVVendorGate.h" // anscv_vendor_gate::IsNvidiaGpuAvailable()
#include "ANSLicense.h" // ANS_DBG macro
#include <memory>
@@ -213,11 +214,11 @@ namespace ANSCENTER {
}
}
// Honest image-equality check. Same buffer pointer (fast path) or pixel
// compare (fallback). Camera-aliveness uses getLastFrameAgeMs() — separate
// concern. With "same image -> same pts" invariant, callers usually just
// compare pts and skip this function entirely.
bool ANSMJPEGClient::areImagesIdentical(const cv::Mat& img1, const cv::Mat& img2) {
double ageMs = _playerClient->getLastFrameAgeMs();
if (ageMs > 5000.0) return true;
if (ageMs > 0.0) return false;
if (img1.empty() && img2.empty()) return true;
if (img1.empty() || img2.empty()) return false;
if (img1.size() != img2.size() || img1.type() != img2.type()) return false;
@@ -275,17 +276,19 @@ namespace ANSCENTER {
return _pLastFrame;
}
// Use PTS to detect duplicate frames (much faster than pixel comparison)
if (currentPts == _pts && !_pLastFrame.empty()) {
// Duplicate detection via _lastInnerPts (separate from _pts). Invariant:
// same image -> same pts. Camera-alive check uses getLastFrameAgeMs().
if (currentPts == _lastInnerPts && !_pLastFrame.empty()) {
width = _imageWidth;
height = _imageHeight;
pts = _pts;
pts = _pts; // unchanged
return _pLastFrame;
}
// Update PTS first
_pts = currentPts;
pts = currentPts;
// Inner advanced — real new frame. Sync cursor and bump _pts monotonically.
_lastInnerPts = currentPts;
_pts++;
pts = _pts;
// Handle non-rotated case
if (_imageRotateDeg == 0) {
@@ -633,6 +636,14 @@ namespace ANSCENTER {
std::lock_guard<std::recursive_mutex> lock(_mutex);
_playerClient->setTargetFPS(intervalMs); // 0=no limit, 100=~10FPS, 200=~5FPS
}
void ANSMJPEGClient::SetJitterBufferMs(int ms) {
std::lock_guard<std::recursive_mutex> lock(_mutex);
_playerClient->setJitterBufferMs(ms);
ANS_DBG("MJPEG_Lifecycle", "SetJitterBufferMs: ms=%d url=%s", ms, _url.c_str());
}
int ANSMJPEGClient::GetJitterBufferMs() const {
return _playerClient->getJitterBufferMs();
}
void ANSMJPEGClient::SetNV12FastPath(bool enable) {
std::lock_guard<std::recursive_mutex> lock(_mutex);
_useNV12FastPath = enable;
@@ -727,11 +738,11 @@ extern "C" __declspec(dllexport) int GetMJPEGStrImage(ANSCENTER::ANSMJPEGClient
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error getting MJPEG image: " << e.what() << std::endl;
ANS_DBG("MJPEG_GetImage", "exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error getting MJPEG image: Unknown exception." << std::endl;
ANS_DBG("MJPEG_GetImage", "unknown exception");
return 0;
}
}
@@ -749,28 +760,28 @@ extern "C" __declspec(dllexport) int GetMJPEGImage(ANSCENTER::ANSMJPEGClient * *
return 1;
}
else {
std::cerr << "Error resizing jpegImage handle: " << error << std::endl;
ANS_DBG("MJPEG_GetImage", "DSSetHandleSize failed: error=%d", (int)error);
return 0;
}
}
else {
std::cerr << "No image data retrieved from MJPEG client." << std::endl;
ANS_DBG("MJPEG_GetImage", "no image data retrieved (jpegString empty)");
return 0;
}
}
catch (const std::exception& e) {
std::cerr << "Error getting MJPEG image: " << e.what() << std::endl;
ANS_DBG("MJPEG_GetImage", "exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error getting MJPEG image: Unknown exception." << std::endl;
ANS_DBG("MJPEG_GetImage", "unknown exception");
return 0;
}
}
extern "C" __declspec(dllexport) int GetMJPEGCVImage(ANSCENTER::ANSMJPEGClient** Handle, int& width, int& height, int64_t& timeStamp, cv::Mat** image) {
if (!Handle || !*Handle || !image) {
std::cerr << "Error: Invalid input parameters in GetMJPEGCVImage" << std::endl;
ANS_DBG("MJPEG_GetImage", "invalid input parameters");
return -1;
}
@@ -784,17 +795,25 @@ extern "C" __declspec(dllexport) int GetMJPEGCVImage(ANSCENTER::ANSMJPEGClient**
// Thread-safe Mat pointer swap (anscv_mat_replace has its own internal lock)
anscv_mat_replace(image, std::move(img));
// NV12 GPU fast path (optional — disabled by default for stability)
// NV12 GPU fast path (optional — disabled by default for stability).
// AvFramePtr auto-frees clones if we don't reach the .release() handoff.
if ((*Handle)->IsNV12FastPath()) {
int gpuIdx = (*Handle)->GetHWDecodingGpuIndex();
AVFrame* cudaHW = (*Handle)->GetCudaHWFrame();
ansmedia::AvFramePtr cudaHW{ (*Handle)->GetCudaHWFrame() };
if (cudaHW) {
AVFrame* cpuNV12 = (*Handle)->GetNV12Frame();
gpu_frame_attach_cuda(*image, cudaHW, gpuIdx, timeStamp, cpuNV12);
ansmedia::AvFramePtr cpuNV12{ (*Handle)->GetNV12Frame() };
ANS_DBG("MEDIA_AvPtr",
"ANSMJPEG V1 attach_cuda cudaHW=%p cpuNV12=%p gpuIdx=%d",
(void*)cudaHW.get(), (void*)cpuNV12.get(), gpuIdx);
gpu_frame_attach_cuda(*image, cudaHW.release(), gpuIdx, timeStamp,
cpuNV12.release());
} else {
AVFrame* nv12 = (*Handle)->GetNV12Frame();
ansmedia::AvFramePtr nv12{ (*Handle)->GetNV12Frame() };
if (nv12) {
gpu_frame_attach(*image, nv12, gpuIdx, timeStamp);
ANS_DBG("MEDIA_AvPtr",
"ANSMJPEG V1 attach nv12=%p gpuIdx=%d",
(void*)nv12.get(), gpuIdx);
gpu_frame_attach(*image, nv12.release(), gpuIdx, timeStamp);
}
}
}
@@ -802,15 +821,15 @@ extern "C" __declspec(dllexport) int GetMJPEGCVImage(ANSCENTER::ANSMJPEGClient**
return 1;
}
catch (const cv::Exception& e) {
std::cerr << "OpenCV exception in GetMJPEGCVImage: " << e.what() << std::endl;
ANS_DBG("MJPEG_GetImage", "OpenCV exception: %s", e.what());
return -2;
}
catch (const std::exception& e) {
std::cerr << "Exception in GetMJPEGCVImage: " << e.what() << std::endl;
ANS_DBG("MJPEG_GetImage", "GetMJPEGCVImage exception: %s", e.what());
return -2;
}
catch (...) {
std::cerr << "Unknown exception in GetMJPEGCVImage" << std::endl;
ANS_DBG("MJPEG_GetImage", "GetMJPEGCVImage unknown exception");
return -2;
}
}
@@ -822,11 +841,11 @@ extern "C" __declspec(dllexport) int StartMJPEG(ANSCENTER::ANSMJPEGClient * *Han
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error starting MJPEG client: " << e.what() << std::endl;
ANS_DBG("MJPEG_Lifecycle", "StartMJPEG exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error starting MJPEG client: Unknown exception." << std::endl;
ANS_DBG("MJPEG_Lifecycle", "StartMJPEG unknown exception");
return 0;
}
}
@@ -838,11 +857,11 @@ extern "C" __declspec(dllexport) int ReconnectMJPEG(ANSCENTER::ANSMJPEGClient *
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error reconnecting MJPEG client: " << e.what() << std::endl;
ANS_DBG("MJPEG_Lifecycle", "ReconnectMJPEG exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error reconnecting MJPEG client: Unknown exception." << std::endl;
ANS_DBG("MJPEG_Lifecycle", "ReconnectMJPEG unknown exception");
return 0;
}
}
@@ -854,11 +873,11 @@ extern "C" __declspec(dllexport) int StopMJPEG(ANSCENTER::ANSMJPEGClient * *Hand
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error stopping MJPEG client: " << e.what() << std::endl;
ANS_DBG("MJPEG_Lifecycle", "StopMJPEG exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error stopping MJPEG client: Unknown exception." << std::endl;
ANS_DBG("MJPEG_Lifecycle", "StopMJPEG unknown exception");
return 0;
}
}
@@ -970,6 +989,18 @@ extern "C" __declspec(dllexport) void SetMJPEGDisplayResolution(ANSCENTER::ANSMJ
(*Handle)->SetDisplayResolution(width, height);
} catch (...) { }
}
extern "C" __declspec(dllexport) void SetMJPEGJitterBuffer(ANSCENTER::ANSMJPEGClient** Handle, int jitterMs) {
if (Handle == nullptr || *Handle == nullptr) return;
try {
(*Handle)->SetJitterBufferMs(jitterMs);
} catch (...) { }
}
extern "C" __declspec(dllexport) int GetMJPEGJitterBuffer(ANSCENTER::ANSMJPEGClient** Handle) {
if (Handle == nullptr || *Handle == nullptr) return -1;
try {
return (*Handle)->GetJitterBufferMs();
} catch (...) { return -1; }
}
extern "C" __declspec(dllexport) void SetMJPEGTargetFPS(ANSCENTER::ANSMJPEGClient** Handle, double intervalMs) {
if (Handle == nullptr || *Handle == nullptr) return;
try {
@@ -1001,21 +1032,21 @@ extern "C" __declspec(dllexport) int GetMJPEGImage_V2(uint64_t handleVal, int& w
return 1;
}
else {
std::cerr << "Error resizing jpegImage handle: " << error << std::endl;
ANS_DBG("MJPEG_GetImage", "DSSetHandleSize failed: error=%d", (int)error);
return 0;
}
}
else {
std::cerr << "No image data retrieved from MJPEG client." << std::endl;
ANS_DBG("MJPEG_GetImage", "no image data retrieved (jpegString empty)");
return 0;
}
}
catch (const std::exception& e) {
std::cerr << "Error getting MJPEG image: " << e.what() << std::endl;
ANS_DBG("MJPEG_GetImage", "exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error getting MJPEG image: Unknown exception." << std::endl;
ANS_DBG("MJPEG_GetImage", "unknown exception");
return 0;
}
}
@@ -1030,31 +1061,39 @@ extern "C" __declspec(dllexport) int GetMJPEGCVImage_V2(uint64_t handleVal, int&
}
anscv_mat_replace(image, std::move(img));
// Attach NV12 frame for GPU fast-path inference (side-table registry)
// Attach NV12 frame for GPU fast-path inference (side-table registry).
// AvFramePtr auto-frees clones if we don't reach the .release() handoff.
int gpuIdx = h->GetHWDecodingGpuIndex();
AVFrame* cudaHW = h->GetCudaHWFrame();
ansmedia::AvFramePtr cudaHW{ h->GetCudaHWFrame() };
if (cudaHW) {
AVFrame* cpuNV12 = h->GetNV12Frame();
gpu_frame_attach_cuda(*image, cudaHW, gpuIdx, timeStamp, cpuNV12);
ansmedia::AvFramePtr cpuNV12{ h->GetNV12Frame() };
ANS_DBG("MEDIA_AvPtr",
"ANSMJPEG V2 attach_cuda cudaHW=%p cpuNV12=%p gpuIdx=%d",
(void*)cudaHW.get(), (void*)cpuNV12.get(), gpuIdx);
gpu_frame_attach_cuda(*image, cudaHW.release(), gpuIdx, timeStamp,
cpuNV12.release());
} else {
AVFrame* nv12 = h->GetNV12Frame();
ansmedia::AvFramePtr nv12{ h->GetNV12Frame() };
if (nv12) {
gpu_frame_attach(*image, nv12, gpuIdx, timeStamp);
ANS_DBG("MEDIA_AvPtr",
"ANSMJPEG V2 attach nv12=%p gpuIdx=%d",
(void*)nv12.get(), gpuIdx);
gpu_frame_attach(*image, nv12.release(), gpuIdx, timeStamp);
}
}
return 1;
}
catch (const cv::Exception& e) {
std::cerr << "OpenCV exception in GetMJPEGCVImage_V2: " << e.what() << std::endl;
ANS_DBG("MJPEG_GetImage", "GetMJPEGCVImage_V2 OpenCV exception: %s", e.what());
return -2;
}
catch (const std::exception& e) {
std::cerr << "Exception in GetMJPEGCVImage_V2: " << e.what() << std::endl;
ANS_DBG("MJPEG_GetImage", "GetMJPEGCVImage_V2 exception: %s", e.what());
return -2;
}
catch (...) {
std::cerr << "Unknown exception in GetMJPEGCVImage_V2" << std::endl;
ANS_DBG("MJPEG_GetImage", "GetMJPEGCVImage_V2 unknown exception");
return -2;
}
}
@@ -1067,11 +1106,11 @@ extern "C" __declspec(dllexport) int StartMJPEG_V2(uint64_t handleVal) {
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error starting MJPEG client: " << e.what() << std::endl;
ANS_DBG("MJPEG_Lifecycle", "StartMJPEG exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error starting MJPEG client: Unknown exception." << std::endl;
ANS_DBG("MJPEG_Lifecycle", "StartMJPEG unknown exception");
return 0;
}
}
@@ -1084,11 +1123,11 @@ extern "C" __declspec(dllexport) int ReconnectMJPEG_V2(uint64_t handleVal) {
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error reconnecting MJPEG client: " << e.what() << std::endl;
ANS_DBG("MJPEG_Lifecycle", "ReconnectMJPEG exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error reconnecting MJPEG client: Unknown exception." << std::endl;
ANS_DBG("MJPEG_Lifecycle", "ReconnectMJPEG unknown exception");
return 0;
}
}
@@ -1101,11 +1140,11 @@ extern "C" __declspec(dllexport) int StopMJPEG_V2(uint64_t handleVal) {
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error stopping MJPEG client: " << e.what() << std::endl;
ANS_DBG("MJPEG_Lifecycle", "StopMJPEG exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error stopping MJPEG client: Unknown exception." << std::endl;
ANS_DBG("MJPEG_Lifecycle", "StopMJPEG unknown exception");
return 0;
}
}
+9
View File
@@ -34,6 +34,9 @@ namespace ANSCENTER
std::string _lastJpegImage;
int _imageWidth, _imageHeight;
int64_t _pts;
// Last currentPts seen from inner getImage(). Used for duplicate
// detection separately from _pts so "same image -> same pts" holds.
int64_t _lastInnerPts = -1;
bool _isPlaying;
bool _useNV12FastPath = false;
std::recursive_mutex _mutex;
@@ -72,6 +75,10 @@ namespace ANSCENTER
void SetDisplayResolution(int width, int height); // Set display output size; 0,0 = original (no resize)
void SetImageQuality(int mode); // 0=fast (AI), 1=quality (display)
void SetTargetFPS(double intervalMs); // Set min interval between processed frames in ms (0 = no limit, 100 = ~10 FPS, 200 = ~5 FPS)
// Jitter buffer for the BGR delivery path (forwarded to CVideoPlayer).
// 0 = no buffering (today's getLatest semantics). Default 200 ms.
void SetJitterBufferMs(int ms);
int GetJitterBufferMs() const;
void SetNV12FastPath(bool enable); // true = NV12 GPU fast path, false = original CPU path (stable)
bool IsNV12FastPath() const { return _useNV12FastPath; }
AVFrame* GetNV12Frame(); // Returns cloned NV12 frame for GPU fast-path (caller must av_frame_free)
@@ -114,4 +121,6 @@ extern "C" __declspec(dllexport) void SetMJPEGImageQuality(ANSCENTER::ANSMJPEGCl
extern "C" __declspec(dllexport) void SetMJPEGDisplayResolution(ANSCENTER::ANSMJPEGClient** Handle, int width, int height);
extern "C" __declspec(dllexport) void SetMJPEGTargetFPS(ANSCENTER::ANSMJPEGClient** Handle, double intervalMs);
extern "C" __declspec(dllexport) void SetMJPEGNV12FastPath(ANSCENTER::ANSMJPEGClient** Handle, int enable);
extern "C" __declspec(dllexport) void SetMJPEGJitterBuffer(ANSCENTER::ANSMJPEGClient** Handle, int jitterMs);
extern "C" __declspec(dllexport) int GetMJPEGJitterBuffer(ANSCENTER::ANSMJPEGClient** Handle);
#endif
+105 -53
View File
@@ -1,6 +1,7 @@
#include "ANSRTMP.h"
#include "ANSMatRegistry.h"
#include "ANSGpuFrameOps.h"
#include "av_frame_ptr.h" // ansmedia::AvFramePtr — RAII for AVFrame*
#include "ANSCVVendorGate.h" // anscv_vendor_gate::IsNvidiaGpuAvailable()
#include "ANSLicense.h" // ANS_DBG macro
#include <memory>
@@ -218,14 +219,23 @@ namespace ANSCENTER {
}
}
// Honest image-equality check. Returns true iff the two Mats refer to the
// same pixels. Used by callers who want to know "did I just get the same
// frame back?" — typical pattern after GetImage to skip downstream work
// on rate-limited / buffered duplicates.
//
// Camera-aliveness ("did the decoder go silent?") is a separate concern;
// use GetRTMPLastFrameAgeMs (or _playerClient->getLastFrameAgeMs()) for
// that. With the new pts discipline (same image -> same pts), most callers
// can simply compare pts instead of calling this function at all.
bool ANSRTMPClient::areImagesIdentical(const cv::Mat& img1, const cv::Mat& img2) {
double ageMs = _playerClient->getLastFrameAgeMs();
if (ageMs > 5000.0) return true;
if (ageMs > 0.0) return false;
if (img1.empty() && img2.empty()) return true;
if (img1.empty() || img2.empty()) return false;
if (img1.size() != img2.size() || img1.type() != img2.type()) return false;
// Fast path — same buffer pointer. With the new GetImage discipline,
// the duplicate path returns _pLastFrame so this branch hits and we
// are done in O(1).
if (img1.data == img2.data) return true;
if (img1.isContinuous() && img2.isContinuous()) {
@@ -281,19 +291,25 @@ namespace ANSCENTER {
return _pLastFrame;
}
// Use PTS to detect duplicate frames (much faster than pixel comparison)
// The sequence-based check in getImage() already skips conversion for same frames,
// so if PTS hasn't changed, it's the same frame — no need for expensive memcmp
if (currentPts == _pts && !_pLastFrame.empty()) {
// Duplicate-frame detection. Track the inner player's currentPts in
// _lastInnerPts (separate from our own _pts). On a duplicate, return
// _pLastFrame and DO NOT advance _pts — the invariant is:
// same image -> same pts
// so a caller comparing pts (or calling areImagesIdentical()) can
// recognise rate-limited / buffered duplicates.
// Camera-alive detection should use getLastFrameAgeMs() instead.
if (currentPts == _lastInnerPts && !_pLastFrame.empty()) {
width = _imageWidth;
height = _imageHeight;
pts = _pts;
pts = _pts; // unchanged
return _pLastFrame;
}
// Update PTS first
_pts = currentPts;
pts = currentPts;
// Inner advanced — real new frame. Sync the duplicate cursor and bump
// _pts monotonically (each ++ corresponds to a unique frame).
_lastInnerPts = currentPts;
_pts++;
pts = _pts;
// Handle non-rotated case
if (_imageRotateDeg == 0) {
@@ -647,6 +663,14 @@ namespace ANSCENTER {
std::lock_guard<std::recursive_mutex> lock(_mutex);
_playerClient->setTargetFPS(intervalMs); // 0=no limit, 100=~10FPS, 200=~5FPS
}
void ANSRTMPClient::SetJitterBufferMs(int ms) {
std::lock_guard<std::recursive_mutex> lock(_mutex);
_playerClient->setJitterBufferMs(ms);
ANS_DBG("RTMP_Lifecycle", "SetJitterBufferMs: ms=%d url=%s", ms, _url.c_str());
}
int ANSRTMPClient::GetJitterBufferMs() const {
return _playerClient->getJitterBufferMs();
}
void ANSRTMPClient::SetNV12FastPath(bool enable) {
std::lock_guard<std::recursive_mutex> lock(_mutex);
_useNV12FastPath = enable;
@@ -741,11 +765,11 @@ extern "C" __declspec(dllexport) int GetRTMPtrImage(ANSCENTER::ANSRTMPClient * *
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error getting RTMP image: " << e.what() << std::endl;
ANS_DBG("RTMP_GetImage", "exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error getting RTMP image: Unknown exception." << std::endl;
ANS_DBG("RTMP_GetImage", "unknown exception");
return 0;
}
}
@@ -769,22 +793,22 @@ extern "C" __declspec(dllexport) int GetRTMPImage(ANSCENTER::ANSRTMPClient * *Ha
}
else {
// Return failure if there was an error in resizing the handle
std::cerr << "Error resizing jpegImage handle: " << error << std::endl;
ANS_DBG("RTMP_GetImage", "DSSetHandleSize failed: error=%d", (int)error);
return 0;
}
}
else {
// If the JPEG image string is empty, return failure
std::cerr << "No image data retrieved from RTMP client." << std::endl;
ANS_DBG("RTMP_GetImage", "no image data retrieved (jpegString empty)");
return 0;
}
}
catch (const std::exception& e) {
std::cerr << "Error getting RTMP image: " << e.what() << std::endl;
ANS_DBG("RTMP_GetImage", "exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error getting RTMP image: Unknown exception." << std::endl;
ANS_DBG("RTMP_GetImage", "unknown exception");
return 0;
}
}
@@ -792,7 +816,7 @@ extern "C" __declspec(dllexport) int GetRTMPImage(ANSCENTER::ANSRTMPClient * *Ha
extern "C" __declspec(dllexport) int GetRTMPCVImage(ANSCENTER::ANSRTMPClient** Handle, int& width, int& height, int64_t& timeStamp, cv::Mat** image) {
// Validate input parameters
if (!Handle || !*Handle || !image) {
std::cerr << "Error: Invalid input parameters in GetRTMPCVImage" << std::endl;
ANS_DBG("RTMP_GetImage", "invalid input parameters");
return -1;
}
@@ -808,17 +832,25 @@ extern "C" __declspec(dllexport) int GetRTMPCVImage(ANSCENTER::ANSRTMPClient** H
// Thread-safe Mat pointer swap (anscv_mat_replace has its own internal lock)
anscv_mat_replace(image, std::move(img));
// NV12 GPU fast path (optional — disabled by default for stability)
// NV12 GPU fast path (optional — disabled by default for stability).
// AvFramePtr auto-frees clones if we don't reach the .release() handoff.
if ((*Handle)->IsNV12FastPath()) {
int gpuIdx = (*Handle)->GetHWDecodingGpuIndex();
AVFrame* cudaHW = (*Handle)->GetCudaHWFrame();
ansmedia::AvFramePtr cudaHW{ (*Handle)->GetCudaHWFrame() };
if (cudaHW) {
AVFrame* cpuNV12 = (*Handle)->GetNV12Frame();
gpu_frame_attach_cuda(*image, cudaHW, gpuIdx, timeStamp, cpuNV12);
ansmedia::AvFramePtr cpuNV12{ (*Handle)->GetNV12Frame() };
ANS_DBG("MEDIA_AvPtr",
"ANSRTMP V1 attach_cuda cudaHW=%p cpuNV12=%p gpuIdx=%d",
(void*)cudaHW.get(), (void*)cpuNV12.get(), gpuIdx);
gpu_frame_attach_cuda(*image, cudaHW.release(), gpuIdx, timeStamp,
cpuNV12.release());
} else {
AVFrame* nv12 = (*Handle)->GetNV12Frame();
ansmedia::AvFramePtr nv12{ (*Handle)->GetNV12Frame() };
if (nv12) {
gpu_frame_attach(*image, nv12, gpuIdx, timeStamp);
ANS_DBG("MEDIA_AvPtr",
"ANSRTMP V1 attach nv12=%p gpuIdx=%d",
(void*)nv12.get(), gpuIdx);
gpu_frame_attach(*image, nv12.release(), gpuIdx, timeStamp);
}
}
}
@@ -826,15 +858,15 @@ extern "C" __declspec(dllexport) int GetRTMPCVImage(ANSCENTER::ANSRTMPClient** H
return 1; // Success
}
catch (const cv::Exception& e) {
std::cerr << "OpenCV exception in GetRTMPCVImage: " << e.what() << std::endl;
ANS_DBG("RTMP_GetImage", "OpenCV exception: %s", e.what());
return -2;
}
catch (const std::exception& e) {
std::cerr << "Exception in GetRTMPCVImage: " << e.what() << std::endl;
ANS_DBG("RTMP_GetImage", "GetRTMPCVImage exception: %s", e.what());
return -2;
}
catch (...) {
std::cerr << "Unknown exception in GetRTMPCVImage" << std::endl;
ANS_DBG("RTMP_GetImage", "GetRTMPCVImage unknown exception");
return -2;
}
}
@@ -846,11 +878,11 @@ extern "C" __declspec(dllexport) int StartRTMP(ANSCENTER::ANSRTMPClient * *Handl
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error starting RTMP client: " << e.what() << std::endl;
ANS_DBG("RTMP_Lifecycle", "StartRTMP exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error starting RTMP client: Unknown exception." << std::endl;
ANS_DBG("RTMP_Lifecycle", "StartRTMP unknown exception");
return 0;
}
}
@@ -862,11 +894,11 @@ extern "C" __declspec(dllexport) int ReconnectRTMP(ANSCENTER::ANSRTMPClient * *H
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error reconnecting RTMP client: " << e.what() << std::endl;
ANS_DBG("RTMP_Lifecycle", "ReconnectRTMP exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error reconnecting RTMP client: Unknown exception." << std::endl;
ANS_DBG("RTMP_Lifecycle", "ReconnectRTMP unknown exception");
return 0;
}
}
@@ -878,11 +910,11 @@ extern "C" __declspec(dllexport) int StopRTMP(ANSCENTER::ANSRTMPClient * *Handle
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error stopping RTMP client: " << e.what() << std::endl;
ANS_DBG("RTMP_Lifecycle", "StopRTMP exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error stopping RTMP client: Unknown exception." << std::endl;
ANS_DBG("RTMP_Lifecycle", "StopRTMP unknown exception");
return 0;
}
}
@@ -998,6 +1030,18 @@ extern "C" __declspec(dllexport) void SetRTMPTargetFPS(ANSCENTER::ANSRTMPClient*
(*Handle)->SetTargetFPS(intervalMs);
} catch (...) { }
}
extern "C" __declspec(dllexport) void SetRTMPJitterBuffer(ANSCENTER::ANSRTMPClient** Handle, int jitterMs) {
if (Handle == nullptr || *Handle == nullptr) return;
try {
(*Handle)->SetJitterBufferMs(jitterMs);
} catch (...) { }
}
extern "C" __declspec(dllexport) int GetRTMPJitterBuffer(ANSCENTER::ANSRTMPClient** Handle) {
if (Handle == nullptr || *Handle == nullptr) return -1;
try {
return (*Handle)->GetJitterBufferMs();
} catch (...) { return -1; }
}
extern "C" __declspec(dllexport) void SetRTMPNV12FastPath(ANSCENTER::ANSRTMPClient** Handle, int enable) {
if (Handle == nullptr || *Handle == nullptr) return;
try {
@@ -1024,21 +1068,21 @@ extern "C" __declspec(dllexport) int GetRTMPImage_V2(uint64_t handleVal, int& wi
return 1;
}
else {
std::cerr << "Error resizing jpegImage handle: " << error << std::endl;
ANS_DBG("RTMP_GetImage", "DSSetHandleSize failed: error=%d", (int)error);
return 0;
}
}
else {
std::cerr << "No image data retrieved from RTMP client." << std::endl;
ANS_DBG("RTMP_GetImage", "no image data retrieved (jpegString empty)");
return 0;
}
}
catch (const std::exception& e) {
std::cerr << "Error getting RTMP image: " << e.what() << std::endl;
ANS_DBG("RTMP_GetImage", "exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error getting RTMP image: Unknown exception." << std::endl;
ANS_DBG("RTMP_GetImage", "unknown exception");
return 0;
}
}
@@ -1053,31 +1097,39 @@ extern "C" __declspec(dllexport) int GetRTMPCVImage_V2(uint64_t handleVal, int&
}
anscv_mat_replace(image, std::move(img));
// Attach NV12 frame for GPU fast-path inference (side-table registry)
// Attach NV12 frame for GPU fast-path inference (side-table registry).
// AvFramePtr auto-frees clones if we don't reach the .release() handoff.
int gpuIdx = h->GetHWDecodingGpuIndex();
AVFrame* cudaHW = h->GetCudaHWFrame();
ansmedia::AvFramePtr cudaHW{ h->GetCudaHWFrame() };
if (cudaHW) {
AVFrame* cpuNV12 = h->GetNV12Frame();
gpu_frame_attach_cuda(*image, cudaHW, gpuIdx, timeStamp, cpuNV12);
ansmedia::AvFramePtr cpuNV12{ h->GetNV12Frame() };
ANS_DBG("MEDIA_AvPtr",
"ANSRTMP V2 attach_cuda cudaHW=%p cpuNV12=%p gpuIdx=%d",
(void*)cudaHW.get(), (void*)cpuNV12.get(), gpuIdx);
gpu_frame_attach_cuda(*image, cudaHW.release(), gpuIdx, timeStamp,
cpuNV12.release());
} else {
AVFrame* nv12 = h->GetNV12Frame();
ansmedia::AvFramePtr nv12{ h->GetNV12Frame() };
if (nv12) {
gpu_frame_attach(*image, nv12, gpuIdx, timeStamp);
ANS_DBG("MEDIA_AvPtr",
"ANSRTMP V2 attach nv12=%p gpuIdx=%d",
(void*)nv12.get(), gpuIdx);
gpu_frame_attach(*image, nv12.release(), gpuIdx, timeStamp);
}
}
return 1;
}
catch (const cv::Exception& e) {
std::cerr << "OpenCV exception in GetRTMPCVImage_V2: " << e.what() << std::endl;
ANS_DBG("RTMP_GetImage", "GetRTMPCVImage_V2 OpenCV exception: %s", e.what());
return -2;
}
catch (const std::exception& e) {
std::cerr << "Exception in GetRTMPCVImage_V2: " << e.what() << std::endl;
ANS_DBG("RTMP_GetImage", "GetRTMPCVImage_V2 exception: %s", e.what());
return -2;
}
catch (...) {
std::cerr << "Unknown exception in GetRTMPCVImage_V2" << std::endl;
ANS_DBG("RTMP_GetImage", "GetRTMPCVImage_V2 unknown exception");
return -2;
}
}
@@ -1090,11 +1142,11 @@ extern "C" __declspec(dllexport) int StartRTMP_V2(uint64_t handleVal) {
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error starting RTMP client: " << e.what() << std::endl;
ANS_DBG("RTMP_Lifecycle", "StartRTMP exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error starting RTMP client: Unknown exception." << std::endl;
ANS_DBG("RTMP_Lifecycle", "StartRTMP unknown exception");
return 0;
}
}
@@ -1107,11 +1159,11 @@ extern "C" __declspec(dllexport) int ReconnectRTMP_V2(uint64_t handleVal) {
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error reconnecting RTMP client: " << e.what() << std::endl;
ANS_DBG("RTMP_Lifecycle", "ReconnectRTMP exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error reconnecting RTMP client: Unknown exception." << std::endl;
ANS_DBG("RTMP_Lifecycle", "ReconnectRTMP unknown exception");
return 0;
}
}
@@ -1124,11 +1176,11 @@ extern "C" __declspec(dllexport) int StopRTMP_V2(uint64_t handleVal) {
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error stopping RTMP client: " << e.what() << std::endl;
ANS_DBG("RTMP_Lifecycle", "StopRTMP exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error stopping RTMP client: Unknown exception." << std::endl;
ANS_DBG("RTMP_Lifecycle", "StopRTMP unknown exception");
return 0;
}
}
+12
View File
@@ -35,6 +35,11 @@ namespace ANSCENTER
std::string _lastJpegImage;
int _imageWidth, _imageHeight;
int64_t _pts;
// Last currentPts seen from the inner getImage(). Used for duplicate
// detection separately from _pts so the invariant
// same image -> same pts
// holds for callers comparing pts (or calling areImagesIdentical()).
int64_t _lastInnerPts = -1;
bool _isPlaying;
bool _useNV12FastPath = false;
std::recursive_mutex _mutex;
@@ -73,6 +78,11 @@ namespace ANSCENTER
void SetDisplayResolution(int width, int height); // Set display output size; 0,0 = original (no resize)
void SetImageQuality(int mode); // 0=fast (AI), 1=quality (display)
void SetTargetFPS(double intervalMs); // Set min interval between processed frames in ms (0 = no limit, 100 = ~10 FPS, 200 = ~5 FPS)
// Jitter buffer for the BGR delivery path (forwarded to CVideoPlayer).
// 0 = no buffering (today's getLatest semantics). Default 200 ms (set
// inside CVideoPlayer constructor). Adaptive: slow consumers bypass it.
void SetJitterBufferMs(int ms);
int GetJitterBufferMs() const;
void SetNV12FastPath(bool enable); // true = NV12 GPU fast path, false = original CPU path (stable)
bool IsNV12FastPath() const { return _useNV12FastPath; }
AVFrame* GetNV12Frame(); // Returns cloned NV12 frame for GPU fast-path (caller must av_frame_free)
@@ -113,4 +123,6 @@ extern "C" __declspec(dllexport) void SetRTMPImageQuality(ANSCENTER::ANSRTMPClie
extern "C" __declspec(dllexport) void SetRTMPDisplayResolution(ANSCENTER::ANSRTMPClient** Handle, int width, int height);
extern "C" __declspec(dllexport) void SetRTMPTargetFPS(ANSCENTER::ANSRTMPClient** Handle, double intervalMs);
extern "C" __declspec(dllexport) void SetRTMPNV12FastPath(ANSCENTER::ANSRTMPClient** Handle, int enable);
extern "C" __declspec(dllexport) void SetRTMPJitterBuffer(ANSCENTER::ANSRTMPClient** Handle, int jitterMs);
extern "C" __declspec(dllexport) int GetRTMPJitterBuffer(ANSCENTER::ANSRTMPClient** Handle);
#endif
+207 -96
View File
@@ -2,6 +2,7 @@
#include "ANSMatRegistry.h"
#include "ANSGpuFrameOps.h"
#include "GpuNV12SlotPool.h"
#include "av_frame_ptr.h" // ansmedia::AvFramePtr — RAII for AVFrame*
#include "ANSLicense.h" // ANS_DBG macro
#include "ANSCVVendorGate.h" // anscv_vendor_gate::IsNvidiaGpuAvailable()
#include <memory>
@@ -27,6 +28,51 @@ extern "C"
// Note: per-instance thread safety is handled by ANSRTSPClient::_mutex
// Mat registry thread safety is handled by anscv_mat_replace's internal registry_mutex
// ---------------------------------------------------------------------------
// Freeze diagnostics — once-per-second pulse for ANSRTSPClient::GetImage so
// we can see *which* return path fired during a 1-2s freeze. Filter on
// "RTSP_Pulse" in DebugView. Pairs with the producer/consumer pulse in
// video_player.cpp ("MEDIA_Pulse"): together they let you localize a stall
// to producer (decoder) vs internal player vs ANSRTSP wrapper.
// ---------------------------------------------------------------------------
namespace {
std::atomic<uint64_t> g_rtspPulseGetCalls{0}; // entries to ANSRTSPClient::GetImage
std::atomic<uint64_t> g_rtspPulseNotPlaying{0}; // !_isPlaying
std::atomic<uint64_t> g_rtspPulseEarlyStale{0}; // ageMs >= 5000 → cached early
std::atomic<uint64_t> g_rtspPulseInnerEmpty{0}; // _playerClient->getImage returned empty
std::atomic<uint64_t> g_rtspPulseDupPts{0}; // currentPts == _pts → cached
std::atomic<uint64_t> g_rtspPulseNewFrame{0}; // brand-new frame returned
std::atomic<long long> g_rtspPulseNextEmitTick{0};
inline void emit_rtsp_pulse_if_due(double ageMsLastFrame) {
using clk = std::chrono::steady_clock;
const long long tick = clk::now().time_since_epoch().count();
long long expected = g_rtspPulseNextEmitTick.load(std::memory_order_relaxed);
if (tick < expected) return;
const long long deadline = tick + std::chrono::duration_cast<clk::duration>(
std::chrono::milliseconds(1000)).count();
if (!g_rtspPulseNextEmitTick.compare_exchange_strong(
expected, deadline, std::memory_order_relaxed)) {
return;
}
const uint64_t calls = g_rtspPulseGetCalls.exchange(0, std::memory_order_relaxed);
const uint64_t np = g_rtspPulseNotPlaying.exchange(0, std::memory_order_relaxed);
const uint64_t es = g_rtspPulseEarlyStale.exchange(0, std::memory_order_relaxed);
const uint64_t ie = g_rtspPulseInnerEmpty.exchange(0, std::memory_order_relaxed);
const uint64_t dp = g_rtspPulseDupPts.exchange(0, std::memory_order_relaxed);
const uint64_t nf = g_rtspPulseNewFrame.exchange(0, std::memory_order_relaxed);
// Production: route through ANS_DBG so the pulse is gated by the
// ANSCORE_DEBUGVIEW runtime toggle. Silent by default.
ANS_DBG("RTSP_Pulse",
"[1s] GetImage calls=%llu new=%llu cached(notPlay=%llu earlyStale=%llu "
"innerEmpty=%llu dupPts=%llu) lastFrameAge=%.0fms",
(unsigned long long)calls, (unsigned long long)nf,
(unsigned long long)np, (unsigned long long)es,
(unsigned long long)ie, (unsigned long long)dp,
ageMsLastFrame);
}
} // anonymous namespace
// Debug logging. Define ANSCORE_GPU_DEBUG=1 to enable verbose per-frame logging.
#ifndef RTSP_DBG
#if defined(ANSCORE_GPU_DEBUG) && ANSCORE_GPU_DEBUG
@@ -405,28 +451,31 @@ namespace ANSCENTER {
}
}
// areImagesIdentical: honest image-equality check. Returns true iff the two
// Mats refer to the same pixels. Used by callers who want to know "did I
// just get the same frame back?" — typical pattern after GetImage to skip
// expensive downstream work on rate-limited / buffered duplicates.
//
// For camera-aliveness ("did the decoder go silent?") use
// GetRTSPLastFrameAgeMs() instead — that's a separate concern and was the
// wrong thing to put inside this function before. With the new pts
// discipline (same image -> same pts), most callers can simply compare
// pts instead of calling this function at all.
bool ANSRTSPClient::areImagesIdentical(const cv::Mat& img1, const cv::Mat& img2) {
double ageMs = _playerClient->getLastFrameAgeMs();
if (ageMs > 5000.0) {
ANS_DBG("RTSP_Stale", "FROZEN DETECTED: ageMs=%.1f url=%s playing=%d — camera truly stale",
ageMs, _url.c_str(), (int)_isPlaying);
return true; // Truly stale — no decoder output for 5+ seconds
}
if (ageMs > 0.0) {
return false; // Decoder is receiving frames — camera is alive
}
// ageMs == 0 means no frame has been decoded yet (startup).
// Fall back to pixel comparison for backward compatibility.
if (img1.empty() && img2.empty()) return true;
if (img1.empty() || img2.empty()) return false;
if (img1.size() != img2.size() || img1.type() != img2.type()) return false;
// Same data pointer = same cv::Mat (shallow copy)
// Fast path — same buffer pointer. With the new GetImage logic, the
// duplicate path returns _pLastFrame, so the caller gets back the
// same buffer it received last time → this branch hits and we're done
// in O(1). The vast majority of "is this a duplicate?" queries land
// here.
if (img1.data == img2.data) return true;
// Quick 5-point sampling
// Distinct buffers — fall back to a real pixel comparison. Quick
// 4-point sample first to early-out on obvious mismatches, then a
// full memcmp for confirmation.
if (img1.isContinuous() && img2.isContinuous()) {
const size_t totalBytes = img1.total() * img1.elemSize();
const size_t quarter = totalBytes / 4;
@@ -449,11 +498,17 @@ namespace ANSCENTER {
cv::Mat ANSRTSPClient::GetImage(int& width, int& height, int64_t& pts) {
std::lock_guard<std::recursive_mutex> lock(_mutex);
// RTSP_Pulse: count entry, attempt once-per-second emit on every call so
// the pulse keeps printing even when every path returns cached.
g_rtspPulseGetCalls.fetch_add(1, std::memory_order_relaxed);
emit_rtsp_pulse_if_due(_playerClient ? _playerClient->getLastFrameAgeMs() : -1.0);
// Return last known frame if not playing
if (!_isPlaying) {
width = _imageWidth;
height = _imageHeight;
pts = _pts;
g_rtspPulseNotPlaying.fetch_add(1, std::memory_order_relaxed);
return _pLastFrame; // Shallow copy (fast)
}
@@ -472,6 +527,7 @@ namespace ANSCENTER {
width = _imageWidth;
height = _imageHeight;
pts = _pts;
g_rtspPulseEarlyStale.fetch_add(1, std::memory_order_relaxed);
return _pLastFrame;
}
}
@@ -487,36 +543,43 @@ namespace ANSCENTER {
width = _imageWidth;
height = _imageHeight;
pts = _pts;
g_rtspPulseInnerEmpty.fetch_add(1, std::memory_order_relaxed);
return _pLastFrame;
}
// Use PTS to detect duplicate frames (much faster than pixel comparison)
// The sequence-based check in getImage() already skips conversion for same frames,
// so if PTS hasn't changed, it's the same frame — no need for expensive memcmp
if (currentPts == _pts && !_pLastFrame.empty()) {
// Duplicate-frame detection. We track the inner player's currentPts
// (in _lastInnerPts) separately from our own _pts so that:
// 1. duplicate detection stays correct even when _pts has been
// bumped past currentPts in earlier "new frame" calls
// 2. LabVIEW's invariant "same pts == same image" holds — required
// so a caller comparing pts (or calling areImagesIdentical())
// correctly recognises rate-limited / buffered duplicates.
//
// Camera-alive detection MUST use GetRTSPLastFrameAgeMs() — pts is no
// longer abused as a heartbeat. ageMs is updated for every decoded
// frame inside CVideoPlayer::onVideoFrame, independent of pacing.
if (currentPts == _lastInnerPts && !_pLastFrame.empty()) {
width = _imageWidth;
height = _imageHeight;
// Return timestamp based on decoder frame age so LabVIEW can distinguish
// "rate-limited duplicate" from "camera truly stale".
// If decoder is still receiving frames (age < 5s), advance PTS so LabVIEW
// sees a changing timestamp and doesn't trigger false reconnect.
// If decoder is stale (age > 5s), return same PTS so LabVIEW detects it.
// Optional: log the stale case so DebugView still surfaces it.
double ageMs = _playerClient->getLastFrameAgeMs();
if (ageMs > 0.0 && ageMs < 5000.0) {
// Camera alive but rate-limited — advance PTS to prevent false stale detection
_pts++;
} else if (ageMs >= 5000.0) {
// Camera stale — keep same PTS so LabVIEW triggers reconnect
ANS_DBG("RTSP_GetImage", "STALE PTS: ageMs=%.1f pts=%lld url=%s — not advancing PTS",
if (ageMs >= 5000.0) {
ANS_DBG("RTSP_GetImage", "STALE PTS: ageMs=%.1f pts=%lld url=%s",
ageMs, (long long)_pts, _url.c_str());
}
pts = _pts;
pts = _pts; // unchanged — same image, same pts
g_rtspPulseDupPts.fetch_add(1, std::memory_order_relaxed);
return _pLastFrame;
}
// Update PTS first
_pts = currentPts;
pts = currentPts;
// Inner advanced — real new frame. Sync our duplicate-detection cursor
// and bump _pts monotonically. Each ++ corresponds to a unique frame
// delivered to the consumer, so a caller can use pts as a stable
// "did I get a new frame?" signal.
_lastInnerPts = currentPts;
_pts++;
pts = _pts;
g_rtspPulseNewFrame.fetch_add(1, std::memory_order_relaxed);
// Handle non-rotated case
if (_imageRotateDeg == 0) {
@@ -991,6 +1054,24 @@ namespace ANSCENTER {
std::lock_guard<std::recursive_mutex> lock(_mutex);
_playerClient->setTargetFPS(intervalMs); // 0=no limit, 100=~10FPS, 200=~5FPS
}
void ANSRTSPClient::SetRtpOverUdp(bool useUdp) {
std::lock_guard<std::recursive_mutex> lock(_mutex);
// Forwards to CRtspClient::set_rtp_over_udp via CRtspPlayer. The TRUE/FALSE
// mapping is inverted inside (rtp_tcp = useUdp ? 0 : 1) — see rtsp_cln.cpp:3578.
_playerClient->setRtpOverUdp(useUdp ? TRUE : FALSE);
ANS_DBG("RTSP_Lifecycle",
"SetRtpOverUdp: useUdp=%d url=%s — must be called before Start()",
(int)useUdp, _url.c_str());
}
void ANSRTSPClient::SetJitterBufferMs(int ms) {
std::lock_guard<std::recursive_mutex> lock(_mutex);
_playerClient->setJitterBufferMs(ms);
ANS_DBG("RTSP_Lifecycle",
"SetJitterBufferMs: ms=%d url=%s", ms, _url.c_str());
}
int ANSRTSPClient::GetJitterBufferMs() const {
return _playerClient->getJitterBufferMs();
}
void ANSRTSPClient::SetNV12FastPath(bool enable) {
std::lock_guard<std::recursive_mutex> lock(_mutex);
_useNV12FastPath = enable;
@@ -1127,11 +1208,11 @@ extern "C" __declspec(dllexport) int GetRTSPStrImage(ANSCENTER::ANSRTSPClient *
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error getting RTSP image: " << e.what() << std::endl;
ANS_DBG("RTSP_GetImage", "exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error getting RTSP image: Unknown exception." << std::endl;
ANS_DBG("RTSP_GetImage", "unknown exception");
return 0;
}
}
@@ -1144,7 +1225,7 @@ extern "C" __declspec(dllexport) int GetRTSPCVImage(
{
// Validate input parameters
if (!Handle || !*Handle || !image) {
std::cerr << "Error: Invalid input parameters in GetRTSPCVImage" << std::endl;
ANS_DBG("RTSP_GetImage", "invalid input parameters");
return -1;
}
@@ -1175,7 +1256,12 @@ extern "C" __declspec(dllexport) int GetRTSPCVImage(
(void*)*image, gpuIdx, (int)inFlightGuardHeld);
if (inFlightGuardHeld) {
AVFrame* cudaHW = (*Handle)->GetCudaHWFrame();
// AvFramePtr wraps each clone obtained from the player so it
// auto-frees on every return path before the .release() handoff
// to gpu_frame_attach[_cuda]. This closes the leak window if
// any future early-return / exception slips between the clone
// call and the attach call.
ansmedia::AvFramePtr cudaHW{ (*Handle)->GetCudaHWFrame() };
if (cudaHW) {
RTSP_DBG("[GetRTSPCVImage] cudaHW: %dx%d data[0]=%p data[1]=%p",
cudaHW->width, cudaHW->height,
@@ -1186,13 +1272,20 @@ extern "C" __declspec(dllexport) int GetRTSPCVImage(
gpuIdx, cudaHW->width, cudaHW->height);
// Only fetch CPU NV12 if pool slot unavailable (cross-GPU fallback).
AVFrame* cpuNV12 = slot ? nullptr : (*Handle)->GetNV12Frame();
gpu_frame_attach_cuda(*image, cudaHW, gpuIdx, timeStamp, cpuNV12, slot);
ansmedia::AvFramePtr cpuNV12{ slot ? nullptr : (*Handle)->GetNV12Frame() };
ANS_DBG("MEDIA_AvPtr",
"ANSRTSP attach_cuda cudaHW=%p cpuNV12=%p slot=%p gpuIdx=%d",
(void*)cudaHW.get(), (void*)cpuNV12.get(), (void*)slot, gpuIdx);
gpu_frame_attach_cuda(*image, cudaHW.release(), gpuIdx, timeStamp,
cpuNV12.release(), slot);
} else {
// HW decode not active — try CPU NV12
AVFrame* nv12 = (*Handle)->GetNV12Frame();
ansmedia::AvFramePtr nv12{ (*Handle)->GetNV12Frame() };
if (nv12) {
gpu_frame_attach(*image, nv12, gpuIdx, timeStamp);
ANS_DBG("MEDIA_AvPtr",
"ANSRTSP attach nv12=%p gpuIdx=%d",
(void*)nv12.get(), gpuIdx);
gpu_frame_attach(*image, nv12.release(), gpuIdx, timeStamp);
}
}
@@ -1240,15 +1333,15 @@ extern "C" __declspec(dllexport) int GetRTSPCVImage(
return 1; // Success
}
catch (const cv::Exception& e) {
std::cerr << "OpenCV exception in GetRTSPCVImage: " << e.what() << std::endl;
ANS_DBG("RTSP_GetImage", "OpenCV exception: %s", e.what());
return -2;
}
catch (const std::exception& e) {
std::cerr << "Exception in GetRTSPCVImage: " << e.what() << std::endl;
ANS_DBG("RTSP_GetImage", "exception in GetRTSPCVImage: %s", e.what());
return -2;
}
catch (...) {
std::cerr << "Unknown exception in GetRTSPCVImage" << std::endl;
ANS_DBG("RTSP_GetImage", "unknown exception in GetRTSPCVImage");
return -2;
}
}
@@ -1292,11 +1385,11 @@ extern "C" __declspec(dllexport) int GetRTSPCVImage(
// return 1; // Success
// }
// catch (const cv::Exception& e) {
// std::cerr << "OpenCV exception in GetRTSPCVImage: " << e.what() << std::endl;
// ANS_DBG("RTSP_GetImage", "OpenCV exception: %s", e.what());
// return -2; // Error code for OpenCV exceptions
// }
// catch (const std::exception& e) {
// std::cerr << "Exception in GetRTSPCVImage: " << e.what() << std::endl;
// ANS_DBG("RTSP_GetImage", "exception in GetRTSPCVImage: %s", e.what());
// return -2; // Error code for standard exceptions
// }
// catch (...) {
@@ -1331,7 +1424,7 @@ extern "C" __declspec(dllexport) int GetRTSPCVImage(
// return 1; // Success
// }
// catch (const std::exception& e) {
// std::cerr << "Exception in GetRTSPCVImage: " << e.what() << std::endl;
// ANS_DBG("RTSP_GetImage", "exception in GetRTSPCVImage: %s", e.what());
// return -2; // Error code for exceptions
// }
// catch (...) {
@@ -1360,22 +1453,22 @@ extern "C" __declspec(dllexport) int GetRTSPImage(ANSCENTER::ANSRTSPClient** Han
}
else {
// Return failure if there was an error in resizing the handle
std::cerr << "Error resizing jpegImage handle: " << error << std::endl;
ANS_DBG("RTSP_GetImage", "DSSetHandleSize failed: error=%d", (int)error);
return 0;
}
}
else {
// If the JPEG image string is empty, return failure
std::cerr << "No image data retrieved from FLV client." << std::endl;
ANS_DBG("RTSP_GetImage", "no image data retrieved (jpegString empty)");
return 0;
}
}
catch (const std::exception& e) {
std::cerr << "Error getting RTSP image: " << e.what() << std::endl;
ANS_DBG("RTSP_GetImage", "exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error getting RTSP image: Unknown exception." << std::endl;
ANS_DBG("RTSP_GetImage", "unknown exception");
return 0;
}
}
@@ -1388,11 +1481,11 @@ extern "C" __declspec(dllexport) int StartRTSP(ANSCENTER::ANSRTSPClient **Handle
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error starting RTSP client: " << e.what() << std::endl;
ANS_DBG("RTSP_Lifecycle", "StartRTSP exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error starting RTSP client: Unknown exception." << std::endl;
ANS_DBG("RTSP_Lifecycle", "StartRTSP unknown exception");
return 0;
}
}
@@ -1404,11 +1497,11 @@ extern "C" __declspec(dllexport) int ReconnectRTSP(ANSCENTER::ANSRTSPClient * *H
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error reconnecting RTSP client: " << e.what() << std::endl;
ANS_DBG("RTSP_Lifecycle", "ReconnectRTSP exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error reconnecting RTSP client: Unknown exception." << std::endl;
ANS_DBG("RTSP_Lifecycle", "ReconnectRTSP unknown exception");
return 0;
}
}
@@ -1421,11 +1514,11 @@ extern "C" __declspec(dllexport) int StopRTSP(ANSCENTER::ANSRTSPClient * *Handle
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error stopping RTSP client: " << e.what() << std::endl;
ANS_DBG("RTSP_Lifecycle", "StopRTSP exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error stopping RTSP client: Unknown exception." << std::endl;
ANS_DBG("RTSP_Lifecycle", "StopRTSP unknown exception");
return 0;
}
@@ -1438,11 +1531,11 @@ extern "C" __declspec(dllexport) int PauseRTSP(ANSCENTER::ANSRTSPClient** Handle
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error pausing RTSP client: " << e.what() << std::endl;
ANS_DBG("RTSP_Lifecycle", "PauseRTSP exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error pausing RTSP client: Unknown exception." << std::endl;
ANS_DBG("RTSP_Lifecycle", "PauseRTSP unknown exception");
return 0;
}
}
@@ -1454,11 +1547,11 @@ extern "C" __declspec(dllexport) int IsRTSPPaused(ANSCENTER::ANSRTSPClient** Han
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error checking if RTSP client is paused: " << e.what() << std::endl;
ANS_DBG("RTSP_Lifecycle", "IsRTSPPaused exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error checking if RTSP client is paused: Unknown exception." << std::endl;
ANS_DBG("RTSP_Lifecycle", "IsRTSPPaused unknown exception");
return 0;
}
}
@@ -1470,11 +1563,11 @@ extern "C" __declspec(dllexport) int IsRTSPRunning(ANSCENTER::ANSRTSPClient **Ha
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error checking if RTSP client is running: " << e.what() << std::endl;
ANS_DBG("RTSP_Lifecycle", "IsRTSPRunning exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error checking if RTSP client is running: Unknown exception." << std::endl;
ANS_DBG("RTSP_Lifecycle", "IsRTSPRunning unknown exception");
return 0;
}
}
@@ -1486,11 +1579,11 @@ extern "C" __declspec(dllexport) int IsRTSPRecording(ANSCENTER::ANSRTSPClient **
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error checking if RTSP client is recording: " << e.what() << std::endl;
ANS_DBG("RTSP_Lifecycle", "IsRTSPRecording exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error checking if RTSP client is recording: Unknown exception." << std::endl;
ANS_DBG("RTSP_Lifecycle", "IsRTSPRecording unknown exception");
return 0;
}
}
@@ -1522,11 +1615,11 @@ extern "C" __declspec(dllexport) int SetBBoxRTSP(ANSCENTER::ANSRTSPClient** Hand
return 1;
}
catch (const std::exception& e) {
std::cerr << "Error setting RTSP bounding box: " << e.what() << std::endl;
ANS_DBG("RTSP_Cfg", "SetBBoxRTSP exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error setting RTSP bounding box: Unknown exception." << std::endl;
ANS_DBG("RTSP_Cfg", "SetBBoxRTSP unknown exception");
return 0;
}
}
@@ -1572,6 +1665,24 @@ extern "C" __declspec(dllexport) void SetRTSPTargetFPS(ANSCENTER::ANSRTSPClient*
(*Handle)->SetTargetFPS(intervalMs); // 0=no limit, 100=~10FPS, 200=~5FPS
} catch (...) { }
}
extern "C" __declspec(dllexport) void SetRTSPRtpOverUdp(ANSCENTER::ANSRTSPClient** Handle, int useUdp) {
if (Handle == nullptr || *Handle == nullptr) return;
try {
(*Handle)->SetRtpOverUdp(useUdp != 0);
} catch (...) { }
}
extern "C" __declspec(dllexport) void SetRTSPJitterBuffer(ANSCENTER::ANSRTSPClient** Handle, int jitterMs) {
if (Handle == nullptr || *Handle == nullptr) return;
try {
(*Handle)->SetJitterBufferMs(jitterMs);
} catch (...) { }
}
extern "C" __declspec(dllexport) int GetRTSPJitterBuffer(ANSCENTER::ANSRTSPClient** Handle) {
if (Handle == nullptr || *Handle == nullptr) return -1;
try {
return (*Handle)->GetJitterBufferMs();
} catch (...) { return -1; }
}
extern "C" __declspec(dllexport) void SetRTSPNV12FastPath(ANSCENTER::ANSRTSPClient** Handle, int enable) {
if (Handle == nullptr || *Handle == nullptr) return;
try {
@@ -1593,11 +1704,11 @@ extern "C" __declspec(dllexport) int SetCropFlagRTSP(ANSCENTER::ANSRTSPClient**
return 1;
}
catch (const std::exception& e) {
std::cerr << "Error setting RTSP crop flag: " << e.what() << std::endl;
ANS_DBG("RTSP_Cfg", "SetCropFlagRTSP exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error setting RTSP crop flag: Unknown exception." << std::endl;
ANS_DBG("RTSP_Cfg", "SetCropFlagRTSP unknown exception");
return 0;
}
}
@@ -1622,21 +1733,21 @@ extern "C" __declspec(dllexport) int GetRTSPImage_V2(uint64_t handleVal, int& wi
return 1;
}
else {
std::cerr << "Error resizing jpegImage handle: " << error << std::endl;
ANS_DBG("RTSP_GetImage", "DSSetHandleSize failed: error=%d", (int)error);
return 0;
}
}
else {
std::cerr << "No image data retrieved from FLV client." << std::endl;
ANS_DBG("RTSP_GetImage", "no image data retrieved (jpegString empty)");
return 0;
}
}
catch (const std::exception& e) {
std::cerr << "Error getting RTSP image: " << e.what() << std::endl;
ANS_DBG("RTSP_GetImage", "exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error getting RTSP image: Unknown exception." << std::endl;
ANS_DBG("RTSP_GetImage", "unknown exception");
return 0;
}
}
@@ -1650,7 +1761,7 @@ extern "C" __declspec(dllexport) int GetRTSPCVImage_V2(
{
auto* h = reinterpret_cast<ANSCENTER::ANSRTSPClient*>(handleVal); if (!h) return -1;
if (!image) {
std::cerr << "Error: Invalid input parameters in GetRTSPCVImage_V2" << std::endl;
ANS_DBG("RTSP_GetImage", "GetRTSPCVImage_V2 invalid input parameters");
return -1;
}
try {
@@ -1662,15 +1773,15 @@ extern "C" __declspec(dllexport) int GetRTSPCVImage_V2(
return 1;
}
catch (const cv::Exception& e) {
std::cerr << "OpenCV exception in GetRTSPCVImage_V2: " << e.what() << std::endl;
ANS_DBG("RTSP_GetImage", "GetRTSPCVImage_V2 OpenCV exception: %s", e.what());
return -2;
}
catch (const std::exception& e) {
std::cerr << "Exception in GetRTSPCVImage_V2: " << e.what() << std::endl;
ANS_DBG("RTSP_GetImage", "GetRTSPCVImage_V2 exception: %s", e.what());
return -2;
}
catch (...) {
std::cerr << "Unknown exception in GetRTSPCVImage_V2" << std::endl;
ANS_DBG("RTSP_GetImage", "GetRTSPCVImage_V2 unknown exception");
return -2;
}
}
@@ -1683,11 +1794,11 @@ extern "C" __declspec(dllexport) int StartRTSP_V2(uint64_t handleVal) {
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error starting RTSP client: " << e.what() << std::endl;
ANS_DBG("RTSP_Lifecycle", "StartRTSP exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error starting RTSP client: Unknown exception." << std::endl;
ANS_DBG("RTSP_Lifecycle", "StartRTSP unknown exception");
return 0;
}
}
@@ -1700,11 +1811,11 @@ extern "C" __declspec(dllexport) int ReconnectRTSP_V2(uint64_t handleVal) {
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error reconnecting RTSP client: " << e.what() << std::endl;
ANS_DBG("RTSP_Lifecycle", "ReconnectRTSP exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error reconnecting RTSP client: Unknown exception." << std::endl;
ANS_DBG("RTSP_Lifecycle", "ReconnectRTSP unknown exception");
return 0;
}
}
@@ -1717,11 +1828,11 @@ extern "C" __declspec(dllexport) int StopRTSP_V2(uint64_t handleVal) {
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error stopping RTSP client: " << e.what() << std::endl;
ANS_DBG("RTSP_Lifecycle", "StopRTSP exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error stopping RTSP client: Unknown exception." << std::endl;
ANS_DBG("RTSP_Lifecycle", "StopRTSP unknown exception");
return 0;
}
}
@@ -1734,11 +1845,11 @@ extern "C" __declspec(dllexport) int PauseRTSP_V2(uint64_t handleVal) {
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error pausing RTSP client: " << e.what() << std::endl;
ANS_DBG("RTSP_Lifecycle", "PauseRTSP exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error pausing RTSP client: Unknown exception." << std::endl;
ANS_DBG("RTSP_Lifecycle", "PauseRTSP unknown exception");
return 0;
}
}
@@ -1751,11 +1862,11 @@ extern "C" __declspec(dllexport) int IsRTSPPaused_V2(uint64_t handleVal) {
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error checking if RTSP client is paused: " << e.what() << std::endl;
ANS_DBG("RTSP_Lifecycle", "IsRTSPPaused exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error checking if RTSP client is paused: Unknown exception." << std::endl;
ANS_DBG("RTSP_Lifecycle", "IsRTSPPaused unknown exception");
return 0;
}
}
@@ -1768,11 +1879,11 @@ extern "C" __declspec(dllexport) int IsRTSPRunning_V2(uint64_t handleVal) {
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error checking if RTSP client is running: " << e.what() << std::endl;
ANS_DBG("RTSP_Lifecycle", "IsRTSPRunning exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error checking if RTSP client is running: Unknown exception." << std::endl;
ANS_DBG("RTSP_Lifecycle", "IsRTSPRunning unknown exception");
return 0;
}
}
@@ -1785,11 +1896,11 @@ extern "C" __declspec(dllexport) int IsRTSPRecording_V2(uint64_t handleVal) {
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error checking if RTSP client is recording: " << e.what() << std::endl;
ANS_DBG("RTSP_Lifecycle", "IsRTSPRecording exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error checking if RTSP client is recording: Unknown exception." << std::endl;
ANS_DBG("RTSP_Lifecycle", "IsRTSPRecording unknown exception");
return 0;
}
}
@@ -1828,11 +1939,11 @@ extern "C" __declspec(dllexport) int SetBBoxRTSP_V2(uint64_t handleVal, int x, i
return 1;
}
catch (const std::exception& e) {
std::cerr << "Error setting RTSP bounding box: " << e.what() << std::endl;
ANS_DBG("RTSP_Cfg", "SetBBoxRTSP exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error setting RTSP bounding box: Unknown exception." << std::endl;
ANS_DBG("RTSP_Cfg", "SetBBoxRTSP unknown exception");
return 0;
}
}
@@ -1846,11 +1957,11 @@ extern "C" __declspec(dllexport) int SetCropFlagRTSP_V2(uint64_t handleVal, int
return 1;
}
catch (const std::exception& e) {
std::cerr << "Error setting RTSP crop flag: " << e.what() << std::endl;
ANS_DBG("RTSP_Cfg", "SetCropFlagRTSP exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error setting RTSP crop flag: Unknown exception." << std::endl;
ANS_DBG("RTSP_Cfg", "SetCropFlagRTSP unknown exception");
return 0;
}
}
+23
View File
@@ -37,6 +37,11 @@ namespace ANSCENTER
std::string _lastJpegImage;
int _imageWidth,_imageHeight;
int64_t _pts;
// Last currentPts seen from the inner getImage(). Used for duplicate
// detection separately from _pts (which gets synthetically incremented
// for LabVIEW's staleness check). Keeping these decoupled fixes the
// "every other dup goes through new-frame path" inefficiency.
int64_t _lastInnerPts = -1;
bool _isPlaying;
bool _useNV12FastPath = false; // false = original stable CPU path, true = NV12 GPU fast path
std::recursive_mutex _mutex;
@@ -104,6 +109,21 @@ namespace ANSCENTER
void SetDisplayResolution(int width, int height); // Set display output size; 0,0 = original (no resize)
void SetImageQuality(int mode); // 0=fast (AI), 1=quality (display)
void SetTargetFPS(double intervalMs); // Set min interval between processed frames in ms (0 = no limit, 100 = ~10 FPS, 200 = ~5 FPS)
// Jitter buffer for the BGR delivery path. Holds frames ms milliseconds
// before releasing to absorb network/decoder hiccups. 0 = today's
// getLatest semantics. Default in CVideoPlayer is 200 ms.
// Adaptive: if the consumer is slower than the producer, drop-ahead
// keeps latency bounded — slow callers (LabVIEW polling at 1-5 Hz)
// effectively bypass the buffer.
// Inference path (NV12/CUDA fast-path) is unaffected — it always
// receives the latest decoded frame regardless of this setting.
void SetJitterBufferMs(int ms);
int GetJitterBufferMs() const;
// Transport for RTP packets. true = RTP/UDP (lower latency, no TCP head-of-line
// blocking — recommended on LAN). false = RTP/RTSP/TCP interleaved (default, more
// firewall-friendly but bursts on packet loss / large I-frames).
// MUST be called BEFORE Start() / StartRTSP — the SETUP request is sent inside play().
void SetRtpOverUdp(bool useUdp);
void SetNV12FastPath(bool enable); // true = NV12 GPU fast path (zero-copy inference), false = original CPU path (stable)
bool IsNV12FastPath() const { return _useNV12FastPath; }
double GetLastFrameAgeMs(); // Milliseconds since last frame from decoder (detects truly stale cameras, unaffected by SetTargetFPS)
@@ -145,6 +165,9 @@ extern "C" __declspec(dllexport) int GetRTSPHWDecodingGpuIndex(ANSCENTER::ANSRT
extern "C" __declspec(dllexport) void SetRTSPImageQuality(ANSCENTER::ANSRTSPClient** Handle, int mode);
extern "C" __declspec(dllexport) void SetRTSPDisplayResolution(ANSCENTER::ANSRTSPClient** Handle, int width, int height);
extern "C" __declspec(dllexport) void SetRTSPTargetFPS(ANSCENTER::ANSRTSPClient** Handle, double intervalMs);
extern "C" __declspec(dllexport) void SetRTSPRtpOverUdp(ANSCENTER::ANSRTSPClient** Handle, int useUdp);
extern "C" __declspec(dllexport) void SetRTSPJitterBuffer(ANSCENTER::ANSRTSPClient** Handle, int jitterMs);
extern "C" __declspec(dllexport) int GetRTSPJitterBuffer(ANSCENTER::ANSRTSPClient** Handle);
extern "C" __declspec(dllexport) void SetRTSPNV12FastPath(ANSCENTER::ANSRTSPClient** Handle, int enable);
extern "C" __declspec(dllexport) double GetRTSPLastFrameAgeMs(ANSCENTER::ANSRTSPClient** Handle);
#endif
+91 -54
View File
@@ -1,6 +1,7 @@
#include "ANSSRT.h"
#include "ANSMatRegistry.h"
#include "ANSGpuFrameOps.h"
#include "av_frame_ptr.h" // ansmedia::AvFramePtr — RAII for AVFrame*
#include "ANSCVVendorGate.h" // anscv_vendor_gate::IsNvidiaGpuAvailable()
#include "ANSLicense.h" // ANS_DBG macro
#include <memory>
@@ -226,11 +227,11 @@ namespace ANSCENTER {
}
}
// Honest image-equality check. Same buffer pointer (fast path) or pixel
// compare (fallback). Camera-aliveness uses getLastFrameAgeMs() — separate
// concern. With "same image -> same pts" invariant, callers usually just
// compare pts and skip this function entirely.
bool ANSSRTClient::areImagesIdentical(const cv::Mat& img1, const cv::Mat& img2) {
double ageMs = _playerClient->getLastFrameAgeMs();
if (ageMs > 5000.0) return true;
if (ageMs > 0.0) return false;
if (img1.empty() && img2.empty()) return true;
if (img1.empty() || img2.empty()) return false;
if (img1.size() != img2.size() || img1.type() != img2.type()) return false;
@@ -289,19 +290,19 @@ namespace ANSCENTER {
return _pLastFrame;
}
// Use PTS to detect duplicate frames (much faster than pixel comparison)
// The sequence-based check in getImage() already skips conversion for same frames,
// so if PTS hasn't changed, it's the same frame — no need for expensive memcmp
if (currentPts == _pts && !_pLastFrame.empty()) {
// Duplicate detection via _lastInnerPts (separate from _pts). Invariant:
// same image -> same pts. Camera-alive check uses getLastFrameAgeMs().
if (currentPts == _lastInnerPts && !_pLastFrame.empty()) {
width = _imageWidth;
height = _imageHeight;
pts = _pts;
pts = _pts; // unchanged
return _pLastFrame;
}
// Update PTS first
_pts = currentPts;
pts = currentPts;
// Inner advanced — real new frame. Sync cursor and bump _pts monotonically.
_lastInnerPts = currentPts;
_pts++;
pts = _pts;
// Handle non-rotated case
if (_imageRotateDeg == 0) {
@@ -665,6 +666,14 @@ namespace ANSCENTER {
std::lock_guard<std::recursive_mutex> lock(_mutex);
_playerClient->setTargetFPS(intervalMs); // 0=no limit, 100=~10FPS, 200=~5FPS
}
void ANSSRTClient::SetJitterBufferMs(int ms) {
std::lock_guard<std::recursive_mutex> lock(_mutex);
_playerClient->setJitterBufferMs(ms);
ANS_DBG("SRT_Lifecycle", "SetJitterBufferMs: ms=%d url=%s", ms, _url.c_str());
}
int ANSSRTClient::GetJitterBufferMs() const {
return _playerClient->getJitterBufferMs();
}
void ANSSRTClient::SetNV12FastPath(bool enable) {
std::lock_guard<std::recursive_mutex> lock(_mutex);
_useNV12FastPath = enable;
@@ -759,11 +768,11 @@ extern "C" __declspec(dllexport) int GetSRTStrImage(ANSCENTER::ANSSRTClient * *H
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error getting SRT image: " << e.what() << std::endl;
ANS_DBG("SRT_GetImage", "exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error getting SRT image: Unknown exception." << std::endl;
ANS_DBG("SRT_GetImage", "unknown exception");
return 0;
}
}
@@ -787,22 +796,22 @@ extern "C" __declspec(dllexport) int GetSRTImage(ANSCENTER::ANSSRTClient * *Hand
}
else {
// Return failure if there was an error in resizing the handle
std::cerr << "Error resizing jpegImage handle: " << error << std::endl;
ANS_DBG("SRT_GetImage", "DSSetHandleSize failed: error=%d", (int)error);
return 0;
}
}
else {
// If the JPEG image string is empty, return failure
std::cerr << "No image data retrieved from SRT client." << std::endl;
ANS_DBG("SRT_GetImage", "no image data retrieved (jpegString empty)");
return 0;
}
}
catch (const std::exception& e) {
std::cerr << "Error getting SRT image: " << e.what() << std::endl;
ANS_DBG("SRT_GetImage", "exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error getting SRT image: Unknown exception." << std::endl;
ANS_DBG("SRT_GetImage", "unknown exception");
return 0;
}
}
@@ -810,7 +819,7 @@ extern "C" __declspec(dllexport) int GetSRTImage(ANSCENTER::ANSSRTClient * *Hand
extern "C" __declspec(dllexport) int GetSRTCVImage(ANSCENTER::ANSSRTClient** Handle, int& width, int& height, int64_t& timeStamp, cv::Mat** image) {
// Validate input parameters
if (!Handle || !*Handle || !image) {
std::cerr << "Error: Invalid input parameters in GetSRTCVImage" << std::endl;
ANS_DBG("SRT_GetImage", "GetSRTCVImage invalid input parameters");
return -1;
}
@@ -826,17 +835,25 @@ extern "C" __declspec(dllexport) int GetSRTCVImage(ANSCENTER::ANSSRTClient** Han
// Thread-safe Mat pointer swap (anscv_mat_replace has its own internal lock)
anscv_mat_replace(image, std::move(img));
// NV12 GPU fast path (optional — disabled by default for stability)
// NV12 GPU fast path (optional — disabled by default for stability).
// AvFramePtr auto-frees clones if we don't reach the .release() handoff.
if ((*Handle)->IsNV12FastPath()) {
int gpuIdx = (*Handle)->GetHWDecodingGpuIndex();
AVFrame* cudaHW = (*Handle)->GetCudaHWFrame();
ansmedia::AvFramePtr cudaHW{ (*Handle)->GetCudaHWFrame() };
if (cudaHW) {
AVFrame* cpuNV12 = (*Handle)->GetNV12Frame();
gpu_frame_attach_cuda(*image, cudaHW, gpuIdx, timeStamp, cpuNV12);
ansmedia::AvFramePtr cpuNV12{ (*Handle)->GetNV12Frame() };
ANS_DBG("MEDIA_AvPtr",
"ANSSRT V1 attach_cuda cudaHW=%p cpuNV12=%p gpuIdx=%d",
(void*)cudaHW.get(), (void*)cpuNV12.get(), gpuIdx);
gpu_frame_attach_cuda(*image, cudaHW.release(), gpuIdx, timeStamp,
cpuNV12.release());
} else {
AVFrame* nv12 = (*Handle)->GetNV12Frame();
ansmedia::AvFramePtr nv12{ (*Handle)->GetNV12Frame() };
if (nv12) {
gpu_frame_attach(*image, nv12, gpuIdx, timeStamp);
ANS_DBG("MEDIA_AvPtr",
"ANSSRT V1 attach nv12=%p gpuIdx=%d",
(void*)nv12.get(), gpuIdx);
gpu_frame_attach(*image, nv12.release(), gpuIdx, timeStamp);
}
}
}
@@ -844,15 +861,15 @@ extern "C" __declspec(dllexport) int GetSRTCVImage(ANSCENTER::ANSSRTClient** Han
return 1; // Success
}
catch (const cv::Exception& e) {
std::cerr << "OpenCV exception in GetSRTCVImage: " << e.what() << std::endl;
ANS_DBG("SRT_GetImage", "GetSRTCVImage OpenCV exception: %s", e.what());
return -2;
}
catch (const std::exception& e) {
std::cerr << "Exception in GetSRTCVImage: " << e.what() << std::endl;
ANS_DBG("SRT_GetImage", "GetSRTCVImage exception: %s", e.what());
return -2;
}
catch (...) {
std::cerr << "Unknown exception in GetSRTCVImage" << std::endl;
ANS_DBG("SRT_GetImage", "GetSRTCVImage unknown exception");
return -2;
}
}
@@ -864,11 +881,11 @@ extern "C" __declspec(dllexport) int StartSRT(ANSCENTER::ANSSRTClient * *Handle)
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error starting SRT client: " << e.what() << std::endl;
ANS_DBG("SRT_Lifecycle", "StartSRT exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error starting SRT client: Unknown exception." << std::endl;
ANS_DBG("SRT_Lifecycle", "StartSRT unknown exception");
return 0;
}
}
@@ -880,11 +897,11 @@ extern "C" __declspec(dllexport) int ReconnectSRT(ANSCENTER::ANSSRTClient * *Han
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error reconnecting SRT client: " << e.what() << std::endl;
ANS_DBG("SRT_Lifecycle", "ReconnectSRT exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error reconnecting SRT client: Unknown exception." << std::endl;
ANS_DBG("SRT_Lifecycle", "ReconnectSRT unknown exception");
return 0;
}
}
@@ -896,11 +913,11 @@ extern "C" __declspec(dllexport) int StopSRT(ANSCENTER::ANSSRTClient * *Handle)
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error stopping SRT client: " << e.what() << std::endl;
ANS_DBG("SRT_Lifecycle", "StopSRT exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error stopping SRT client: Unknown exception." << std::endl;
ANS_DBG("SRT_Lifecycle", "StopSRT unknown exception");
return 0;
}
}
@@ -1015,6 +1032,18 @@ extern "C" __declspec(dllexport) void SetSRTTargetFPS(ANSCENTER::ANSSRTClient**
(*Handle)->SetTargetFPS(intervalMs);
} catch (...) { }
}
extern "C" __declspec(dllexport) void SetSRTJitterBuffer(ANSCENTER::ANSSRTClient** Handle, int jitterMs) {
if (Handle == nullptr || *Handle == nullptr) return;
try {
(*Handle)->SetJitterBufferMs(jitterMs);
} catch (...) { }
}
extern "C" __declspec(dllexport) int GetSRTJitterBuffer(ANSCENTER::ANSSRTClient** Handle) {
if (Handle == nullptr || *Handle == nullptr) return -1;
try {
return (*Handle)->GetJitterBufferMs();
} catch (...) { return -1; }
}
extern "C" __declspec(dllexport) void SetSRTNV12FastPath(ANSCENTER::ANSSRTClient** Handle, int enable) {
if (Handle == nullptr || *Handle == nullptr) return;
try {
@@ -1042,21 +1071,21 @@ extern "C" __declspec(dllexport) int GetSRTImage_V2(uint64_t handleVal, int& wid
return 1;
}
else {
std::cerr << "Error resizing jpegImage handle: " << error << std::endl;
ANS_DBG("SRT_GetImage", "DSSetHandleSize failed: error=%d", (int)error);
return 0;
}
}
else {
std::cerr << "No image data retrieved from SRT client." << std::endl;
ANS_DBG("SRT_GetImage", "no image data retrieved (jpegString empty)");
return 0;
}
}
catch (const std::exception& e) {
std::cerr << "Error getting SRT image: " << e.what() << std::endl;
ANS_DBG("SRT_GetImage", "exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error getting SRT image: Unknown exception." << std::endl;
ANS_DBG("SRT_GetImage", "unknown exception");
return 0;
}
}
@@ -1064,7 +1093,7 @@ extern "C" __declspec(dllexport) int GetSRTImage_V2(uint64_t handleVal, int& wid
extern "C" __declspec(dllexport) int GetSRTCVImage_V2(uint64_t handleVal, int& width, int& height, int64_t& timeStamp, cv::Mat** image) {
auto* h = reinterpret_cast<ANSCENTER::ANSSRTClient*>(handleVal); if (!h) return -1;
if (!image) {
std::cerr << "Error: Invalid input parameters in GetSRTCVImage_V2" << std::endl;
ANS_DBG("SRT_GetImage", "GetSRTCVImage_V2 invalid input parameters");
return -1;
}
@@ -1077,31 +1106,39 @@ extern "C" __declspec(dllexport) int GetSRTCVImage_V2(uint64_t handleVal, int& w
anscv_mat_replace(image, std::move(img));
// Attach NV12 frame for GPU fast-path inference (side-table registry)
// Attach NV12 frame for GPU fast-path inference (side-table registry).
// AvFramePtr auto-frees clones if we don't reach the .release() handoff.
int gpuIdx = h->GetHWDecodingGpuIndex();
AVFrame* cudaHW = h->GetCudaHWFrame();
ansmedia::AvFramePtr cudaHW{ h->GetCudaHWFrame() };
if (cudaHW) {
AVFrame* cpuNV12 = h->GetNV12Frame();
gpu_frame_attach_cuda(*image, cudaHW, gpuIdx, timeStamp, cpuNV12);
ansmedia::AvFramePtr cpuNV12{ h->GetNV12Frame() };
ANS_DBG("MEDIA_AvPtr",
"ANSSRT V2 attach_cuda cudaHW=%p cpuNV12=%p gpuIdx=%d",
(void*)cudaHW.get(), (void*)cpuNV12.get(), gpuIdx);
gpu_frame_attach_cuda(*image, cudaHW.release(), gpuIdx, timeStamp,
cpuNV12.release());
} else {
AVFrame* nv12 = h->GetNV12Frame();
ansmedia::AvFramePtr nv12{ h->GetNV12Frame() };
if (nv12) {
gpu_frame_attach(*image, nv12, gpuIdx, timeStamp);
ANS_DBG("MEDIA_AvPtr",
"ANSSRT V2 attach nv12=%p gpuIdx=%d",
(void*)nv12.get(), gpuIdx);
gpu_frame_attach(*image, nv12.release(), gpuIdx, timeStamp);
}
}
return 1;
}
catch (const cv::Exception& e) {
std::cerr << "OpenCV exception in GetSRTCVImage_V2: " << e.what() << std::endl;
ANS_DBG("SRT_GetImage", "GetSRTCVImage_V2 OpenCV exception: %s", e.what());
return -2;
}
catch (const std::exception& e) {
std::cerr << "Exception in GetSRTCVImage_V2: " << e.what() << std::endl;
ANS_DBG("SRT_GetImage", "GetSRTCVImage_V2 exception: %s", e.what());
return -2;
}
catch (...) {
std::cerr << "Unknown exception in GetSRTCVImage_V2" << std::endl;
ANS_DBG("SRT_GetImage", "GetSRTCVImage_V2 unknown exception");
return -2;
}
}
@@ -1114,11 +1151,11 @@ extern "C" __declspec(dllexport) int StartSRT_V2(uint64_t handleVal) {
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error starting SRT client: " << e.what() << std::endl;
ANS_DBG("SRT_Lifecycle", "StartSRT exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error starting SRT client: Unknown exception." << std::endl;
ANS_DBG("SRT_Lifecycle", "StartSRT unknown exception");
return 0;
}
}
@@ -1131,11 +1168,11 @@ extern "C" __declspec(dllexport) int ReconnectSRT_V2(uint64_t handleVal) {
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error reconnecting SRT client: " << e.what() << std::endl;
ANS_DBG("SRT_Lifecycle", "ReconnectSRT exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error reconnecting SRT client: Unknown exception." << std::endl;
ANS_DBG("SRT_Lifecycle", "ReconnectSRT unknown exception");
return 0;
}
}
@@ -1148,11 +1185,11 @@ extern "C" __declspec(dllexport) int StopSRT_V2(uint64_t handleVal) {
else return 0;
}
catch (const std::exception& e) {
std::cerr << "Error stopping SRT client: " << e.what() << std::endl;
ANS_DBG("SRT_Lifecycle", "StopSRT exception: %s", e.what());
return 0;
}
catch (...) {
std::cerr << "Error stopping SRT client: Unknown exception." << std::endl;
ANS_DBG("SRT_Lifecycle", "StopSRT unknown exception");
return 0;
}
}
+9
View File
@@ -34,6 +34,9 @@ namespace ANSCENTER
std::string _lastJpegImage;
int _imageWidth, _imageHeight;
int64_t _pts;
// Last currentPts seen from inner getImage(). Used for duplicate
// detection separately from _pts so "same image -> same pts" holds.
int64_t _lastInnerPts = -1;
bool _isPlaying;
bool _useNV12FastPath = false;
std::recursive_mutex _mutex;
@@ -72,6 +75,10 @@ namespace ANSCENTER
void SetDisplayResolution(int width, int height); // Set display output size; 0,0 = original (no resize)
void SetImageQuality(int mode); // 0=fast (AI), 1=quality (display)
void SetTargetFPS(double intervalMs); // Set min interval between processed frames in ms (0 = no limit, 100 = ~10 FPS, 200 = ~5 FPS)
// Jitter buffer for the BGR delivery path (forwarded to CVideoPlayer).
// 0 = no buffering (today's getLatest semantics). Default 200 ms.
void SetJitterBufferMs(int ms);
int GetJitterBufferMs() const;
void SetNV12FastPath(bool enable); // true = NV12 GPU fast path, false = original CPU path (stable)
bool IsNV12FastPath() const { return _useNV12FastPath; }
AVFrame* GetNV12Frame(); // Returns cloned NV12 frame for GPU fast-path (caller must av_frame_free)
@@ -113,4 +120,6 @@ extern "C" __declspec(dllexport) void SetSRTImageQuality(ANSCENTER::ANSSRTClient
extern "C" __declspec(dllexport) void SetSRTDisplayResolution(ANSCENTER::ANSSRTClient** Handle, int width, int height);
extern "C" __declspec(dllexport) void SetSRTTargetFPS(ANSCENTER::ANSSRTClient** Handle, double intervalMs);
extern "C" __declspec(dllexport) void SetSRTNV12FastPath(ANSCENTER::ANSSRTClient** Handle, int enable);
extern "C" __declspec(dllexport) void SetSRTJitterBuffer(ANSCENTER::ANSSRTClient** Handle, int jitterMs);
extern "C" __declspec(dllexport) int GetSRTJitterBuffer(ANSCENTER::ANSSRTClient** Handle);
#endif
+38 -18
View File
@@ -5,6 +5,7 @@
#include "ANSMatRegistry.h"
#include "ANSGpuFrameRegistry.h"
#include "ANSGpuFrameOps.h" // gpu_frame_attach, gpu_frame_attach_cuda
#include "av_frame_ptr.h" // ansmedia::AvFramePtr — RAII for AVFrame*
#include <memory>
#include <libswscale/swscale.h>
#include <libavutil/imgutils.h>
@@ -251,9 +252,9 @@ namespace ANSCENTER {
// --- HW decode path ---
if (_hwDecodeActive && _hwPlayer) {
_hwPlayer->play(); // starts read/video/audio threads
_hwPlayer->clearLooped(); // drop any wrap state from a prior session
_hwEOF = false;
_hwFrameCount = 0;
_hwLastElapseMs = 0;
_isPlaying = true;
// Wait for first frame outside the mutex to let decode threads run
@@ -311,7 +312,6 @@ namespace ANSCENTER {
_hwCudaAccel = false;
_hwEOF = false;
_hwFrameCount = 0;
_hwLastElapseMs = 0;
}
else {
// --- cv::VideoCapture fallback ---
@@ -525,21 +525,30 @@ namespace ANSCENTER {
_hwCudaAccel = _hwPlayer->isCudaHWAccel();
_totalFrames = static_cast<int64_t>(_hwPlayer->getDuration() / 1000.0 * 30.0);
_fps = 30.0;
std::string hwType = _hwCudaAccel ? "CUDA" : "D3D11VA/DXVA2";
// Decoder mode: CUDA when zero-copy NVDEC is active; otherwise the
// underlying CFilePlayer is running software decode (HW decoding is
// disabled by default in Setup() — saves VRAM). The old log message
// always said "D3D11VA/DXVA2" which was misleading.
const std::string hwType = _hwCudaAccel ? "CUDA" : "SW";
_logger.LogInfo("ANSVIDEOPLAYER::GetImage",
"HW decode ACTIVE (" + hwType + ") GPU[" + std::to_string(_hwGpuIndex) +
"Decode ACTIVE (" + hwType + ") GPU[" + std::to_string(_hwGpuIndex) +
"] " + std::to_string(_resWidth) + "x" + std::to_string(_resHeight),
__FILE__, __LINE__);
ANS_DBG("ANSVP_EOF",
"first frame received: %dx%d totalFrames~=%lld duration=%lldms",
_resWidth, _resHeight,
(long long)_totalFrames, (long long)_hwPlayer->getDuration());
}
// EOF detection: CFilePlayer auto-loops via av_seek_frame on AVERROR_EOF
// (file_player.cpp:434), so ANSVideoPlayer must detect the wrap itself.
// imgPts from getImage() is a monotonic call-counter, not a real PTS —
// use getElapse() (ms of real video position, updated from packet.pts
// in file_player.cpp:608). When the file loops, elapse drops back to ~0.
const int64_t elapseMs = _hwPlayer->getElapse();
if (_hwFrameCount > 10 && _hwLastElapseMs > 0 &&
elapseMs + 500 < _hwLastElapseMs) {
// (file_player.cpp:434). It now flips m_bLooped when that happens; we
// poll hasLooped() to detect the wrap.
//
// Why not getElapse(): m_nCurPos is non-monotonic in normal playback —
// MP4 packets are in decode order so video PTS oscillates with B-frames,
// AND the audio thread also writes m_nCurPos with audio PTS. Either can
// cause a >500ms backwards drop and falsely trip EOF mid-playback.
if (_hwPlayer->hasLooped()) {
_hwEOF = true;
if (_resHeight <= 0 || _resWidth <= 0) { _resHeight = imgH; _resWidth = imgW; }
_previousImage = cv::Mat(_resHeight, _resWidth, CV_8UC3, cv::Scalar(0, 0, 0));
@@ -548,11 +557,12 @@ namespace ANSCENTER {
if (_previousPTS < std::numeric_limits<int64_t>::max()) _previousPTS++;
else _previousPTS = 0;
pts = _previousPTS;
ANS_DBG("ANSVP_EOF",
"hasLooped() -> EOF locked at frameCount=%lld elapseMs=%lld",
(long long)_hwFrameCount, (long long)_hwPlayer->getElapse());
return _previousImage;
}
_hwLastElapseMs = elapseMs;
cv::Mat result = frame; // CFilePlayer returns owned Mat (already cloned internally)
// Apply cropping if enabled
@@ -1010,17 +1020,27 @@ extern "C" __declspec(dllexport) int GetVideoPlayerCVImage(
// When using cv::VideoCapture fallback, attach BGR clone (pixFmt=1000).
if ((*Handle)->_hwDecodeActive && (*Handle)->_hwPlayer) {
// --- HW decode: NV12/CUDA attachment (same pattern as ANSRTSP.cpp:847-862) ---
// AvFramePtr wraps each clone so it auto-frees on any return path
// before the .release() handoff to gpu_frame_attach[_cuda]. This
// makes leak-by-omission impossible if an exception or early return
// is added between the clone and the attach call.
int gpuIdx = (*Handle)->_hwPlayer->getHWDecodingGpuIndex();
AVFrame* cudaHW = (*Handle)->_hwPlayer->getCudaHWFrame();
ansmedia::AvFramePtr cudaHW{ (*Handle)->_hwPlayer->getCudaHWFrame() };
if (cudaHW) {
// CUDA zero-copy: frame data[0]/data[1] are CUDA device pointers.
// Also attach CPU NV12 as fallback for cross-GPU inference.
AVFrame* cpuNV12 = (*Handle)->_hwPlayer->getNV12Frame();
gpu_frame_attach_cuda(*image, cudaHW, gpuIdx, timeStamp, cpuNV12);
ansmedia::AvFramePtr cpuNV12{ (*Handle)->_hwPlayer->getNV12Frame() };
ANS_DBG("MEDIA_AvPtr",
"ANSVideoPlayer attach_cuda cudaHW=%p cpuNV12=%p gpuIdx=%d",
(void*)cudaHW.get(), (void*)cpuNV12.get(), gpuIdx);
gpu_frame_attach_cuda(*image, cudaHW.release(), gpuIdx, timeStamp, cpuNV12.release());
} else {
AVFrame* nv12 = (*Handle)->_hwPlayer->getNV12Frame();
ansmedia::AvFramePtr nv12{ (*Handle)->_hwPlayer->getNV12Frame() };
if (nv12) {
gpu_frame_attach(*image, nv12, gpuIdx, timeStamp);
ANS_DBG("MEDIA_AvPtr",
"ANSVideoPlayer attach nv12=%p gpuIdx=%d",
(void*)nv12.get(), gpuIdx);
gpu_frame_attach(*image, nv12.release(), gpuIdx, timeStamp);
}
}
} else {
+6 -7
View File
@@ -22,12 +22,12 @@ namespace ANSCENTER
{
protected:
cv::VideoCapture cap;
int _resWidth;
int _resHeight;
int64_t _totalFrames;
int64_t _currentFrame;
int64_t _previousPTS;
double _fps;
int _resWidth = 0;
int _resHeight = 0;
int64_t _totalFrames = 0;
int64_t _currentFrame = 0;
int64_t _previousPTS = 0;
double _fps = 0.0;
bool _isPlaying;
std::string _lastJpegImage;
cv::Mat _previousImage;
@@ -50,7 +50,6 @@ namespace ANSCENTER
bool _hwCudaAccel = false; // true = NVIDIA CUDA zero-copy available
bool _hwEOF = false; // true when video reached end of file
int64_t _hwFrameCount = 0; // frame counter for EOF detection
int64_t _hwLastElapseMs = 0; // last CFilePlayer elapse (ms); EOF detected when it wraps backwards
ANSVIDEOPLAYER();
~ANSVIDEOPLAYER() noexcept;
[[nodiscard]] bool Init(std::string licenseKey, std::string url);
+47
View File
@@ -450,6 +450,53 @@ namespace ANSCENTER {
}
}
// Skip-when-locked query — see header for full contract.
//
// Returns the locked plate text if the caller may skip OCR this frame for
// (cameraId, trackId), or "" if the caller must run OCR. Increments the
// re-verify counter on skip; resets it when the counter hits reverifyEvery
// so the next frame falls through to a real OCR run.
std::string ALPRChecker::tryReuseLockedText(const std::string& cameraId,
int trackId,
int reverifyEvery) {
std::lock_guard<std::recursive_mutex> lock(_mutex);
try {
auto camIt = trackedPlatesById.find(cameraId);
if (camIt == trackedPlatesById.end()) return "";
auto& plates = camIt->second;
auto it = plates.find(trackId);
if (it == plates.end()) return "";
auto& tp = it->second;
if (tp.lockedText.empty()) return ""; // not locked yet — caller must run OCR
// Locked. Decide whether to skip this frame's OCR or fall through
// for a periodic re-verify. reverifyEvery <= 0 means "always skip"
// (no re-verify), reverifyEvery == 1 means "always run OCR".
if (reverifyEvery > 0 && tp.framesSinceVerify >= reverifyEvery) {
tp.framesSinceVerify = 0;
ANS_DBG("ALPR_TrackId",
"cam=%s tid=%d REVERIFY (every=%d) — letting OCR through",
cameraId.c_str(), trackId, reverifyEvery);
return ""; // signal: caller should run OCR this frame
}
++tp.framesSinceVerify;
// Keep the entry alive — checkPlateByTrackId is what normally
// resets framesSinceLastSeen, but on skipped frames it's never
// called, and the prune pass would eventually delete this lock.
tp.framesSinceLastSeen = 0;
ANS_DBG("ALPR_TrackId",
"cam=%s tid=%d SKIP-LOCKED text='%s' reverifyIn=%d",
cameraId.c_str(), trackId, tp.lockedText.c_str(),
reverifyEvery > 0 ? (reverifyEvery - tp.framesSinceVerify) : -1);
return tp.lockedText;
}
catch (const std::exception&) {
return ""; // any failure → fall back to running OCR
}
}
//
static void VerifyGlobalANSALPRLicense(const std::string& licenseKey) {
try {
+80 -3
View File
@@ -7,6 +7,7 @@
#include <fstream>
#include <string>
#include <filesystem>
#include <atomic>
#include <opencv2/opencv.hpp>
#include "Utility.h"
#include "ANSLicense.h"
@@ -47,6 +48,14 @@ namespace ANSCENTER
std::string lockedText;
int lockCount = 0;
int framesSinceLastSeen = 0;
// Frames since the last full OCR was used to verify the lock. When
// skip-when-locked is active in ANSALPR_OCR::RunInference, this
// counter increments on every frame whose OCR was skipped; once it
// reaches the re-verify period, the next frame falls through to a
// real OCR run (resetting the counter) so a stale or wrong lock
// can self-correct via majority-vote re-lock. Zero when not in
// use (e.g. raw OCR pass-through, pipeline mode).
int framesSinceVerify = 0;
};
// cameraId → (trackId → tracked plate)
std::unordered_map<std::string, std::unordered_map<int, TrackedPlateById>> trackedPlatesById;
@@ -61,6 +70,26 @@ namespace ANSCENTER
[[nodiscard]] std::string checkPlate(const std::string& cameraId, const std::string& detectedPlate, const cv::Rect& plateBox);
// Hybrid API: trackId as primary identity, Levenshtein fallback for lost tracks
[[nodiscard]] std::string checkPlateByTrackId(const std::string& cameraId, const std::string& detectedPlate, int trackId);
// Skip-when-locked query: returns the locked plate text for (cameraId, trackId)
// if the caller may safely SKIP running OCR this frame, or an empty string if
// the caller must run OCR and feed the result back through checkPlateByTrackId.
//
// Behaviour:
// * Track not found, or found but not locked yet → returns "".
// * Locked, and < `reverifyEvery` frames since the last full OCR for this
// track → returns the locked text and increments the per-track
// re-verify counter. Caller skips the recognizer entirely.
// * Locked, and ≥ `reverifyEvery` frames since the last full OCR →
// returns "" and resets the counter, so the caller runs OCR this frame
// and the regular voting/re-lock logic in checkPlateByTrackId can
// self-correct a wrong lock or notice a vehicle swap on the same trackId.
//
// Side-effect: also resets framesSinceLastSeen so the periodic prune pass
// in checkPlateByTrackId does not retire a track just because we skipped
// OCR on it. Without this the locked plate would silently disappear from
// the registry after a few hundred skipped frames.
[[nodiscard]] std::string tryReuseLockedText(const std::string& cameraId, int trackId, int reverifyEvery = 30);
};
class ANSLPR_API ANSALPR {
@@ -70,6 +99,31 @@ namespace ANSCENTER
bool _debugFlag{ false };
bool _licenseValid{ false };
bool _isInitialized{ false};
// Loading-in-progress flag — distinguishes the "LoadEngine() is still
// running" state from "LoadEngine() never called or failed". Set by
// ModelLoadingGuard at the top of LoadEngine() and cleared on exit
// (success, failure, or exception). RunInference/RunInferencesBatch
// consult this BEFORE _isInitialized so the early frames during the
// 3+ second model-load window don't flood the log with bogus
// "Model is not initialized" errors.
std::atomic<bool> _modelLoading{ false };
// RAII helper: sets _modelLoading=true on construction, false on
// destruction. Use at the top of LoadEngine() so the flag is always
// cleared on every exit path.
struct ModelLoadingGuard {
std::atomic<bool>& flag;
explicit ModelLoadingGuard(std::atomic<bool>& f) : flag(f) {
flag.store(true, std::memory_order_release);
}
~ModelLoadingGuard() {
flag.store(false, std::memory_order_release);
}
ModelLoadingGuard(const ModelLoadingGuard&) = delete;
ModelLoadingGuard& operator=(const ModelLoadingGuard&) = delete;
};
double _detectorThreshold{ 0.5 };
double _ocrThreshold{ 0.5 };
double _colorThreshold{ 0.0 };// If we use the colour information, this threshold should be greater than 0.0
@@ -91,7 +145,16 @@ namespace ANSCENTER
// ALPRChecker toggle (Layer 2 + Layer 3):
// true = enabled for full-frame, auto-disabled for pipeline/crop
// false = force disabled everywhere (raw OCR pass-through)
bool _enableALPRChecker{ true };
// Default OFF — callers must opt in with SetALPRCheckerEnabled(true)
// to get tracker voting / skip-when-locked / spatial dedup. Keeps
// the out-of-the-box pipeline stateless so first-frame results match
// per-frame results exactly, and avoids surprises for callers that
// drive the engine with pre-cropped pipeline inputs.
// Atomic so the inference threads can read it without a lock while
// config threads (LabVIEW/UI/REST) flip it at runtime. Relaxed
// ordering is sufficient: we only need the read to be tear-free, and
// "next frame picks up the new value" is the intended contract.
std::atomic<bool> _enableALPRChecker{ false };
public:
[[nodiscard]] virtual bool Initialize(const std::string& licenseKey, const std::string& modelZipFilePath, const std::string& modelZipPassword, double detectorThreshold, double ocrThreshold, double colourTheshold=0);
@@ -139,8 +202,20 @@ namespace ANSCENTER
/// Enable/disable ALPRChecker (Layer 2 text stabilization + Layer 3 dedup):
/// true = enabled for full-frame, auto-disabled for pipeline/crop
/// false = force disabled everywhere (raw OCR pass-through)
void SetALPRCheckerEnabled(bool enable) { _enableALPRChecker = enable; }
bool IsALPRCheckerEnabled() const { return _enableALPRChecker; }
void SetALPRCheckerEnabled(bool enable) {
const bool prev = _enableALPRChecker.exchange(enable, std::memory_order_relaxed);
ANS_DBG("ALPR_Checker",
"SetALPRCheckerEnabled this=%p prev=%d new=%d (%s)",
(void*)this, prev ? 1 : 0, enable ? 1 : 0,
enable ? "ON" : "OFF");
}
bool IsALPRCheckerEnabled() const {
const bool value = _enableALPRChecker.load(std::memory_order_relaxed);
ANS_DBG("ALPR_Checker",
"IsALPRCheckerEnabled this=%p value=%d (%s)",
(void*)this, value ? 1 : 0, value ? "ON" : "OFF");
return value;
}
virtual void SetCountry(Country country) { _country = country; }
Country GetCountry() const { return _country; }
@@ -208,6 +283,8 @@ extern "C" ANSLPR_API int ANSALPR_GetFormats(ANSCENTER::ANSALPR** Handle, LS
// ALPRChecker: 1 = enabled (full-frame auto-detected), 0 = disabled (raw OCR)
extern "C" ANSLPR_API int ANSALPR_SetALPRCheckerEnabled(ANSCENTER::ANSALPR** Handle, int enable);
// Returns the current ALPRChecker flag: 1 = enabled, 0 = disabled, -1 = invalid handle.
extern "C" ANSLPR_API int ANSALPR_GetALPRCheckerEnabled(ANSCENTER::ANSALPR** Handle);
// Country: 0=VIETNAM, 1=CHINA, 2=AUSTRALIA, 3=USA, 4=INDONESIA, 5=JAPAN
extern "C" ANSLPR_API int ANSALPR_SetCountry(ANSCENTER::ANSALPR** Handle, int country);
+358 -34
View File
@@ -194,6 +194,12 @@ namespace ANSCENTER
else _country = Country::JAPAN; // Default for OCR mode
}
// Install the Vietnam-only plate-format defaults now that the
// country is known. Idempotent — clears `_plateFormats` first.
// SetCountry() also calls this so a runtime country switch picks
// up the right format set without an Initialize / restart.
ApplyCountryDefaultPlateFormats();
// Store the original model zip path — the OCR models (ansocrdec.onnx,
// ansocrcls.onnx, ansocrrec.onnx, dict_ch.txt) are bundled inside the
// same ALPR model zip, so we reuse it for ANSONNXOCR initialization.
@@ -215,6 +221,11 @@ namespace ANSCENTER
bool ANSALPR_OCR::LoadEngine() {
std::lock_guard<std::recursive_mutex> lock(_mutex);
// RAII guard: set _modelLoading=true for the lifetime of this scope.
// RunInference / RunInferencesBatch consult this flag and silently
// defer (instead of logging "Model is not initialized") while we're
// loading the LP detector + OCR engine + colour classifier.
ModelLoadingGuard mlg(_modelLoading);
try {
WriteEventLog("ANSALPR_OCR::LoadEngine: Step 1 - Starting engine load");
this->_logger.LogInfo("ANSALPR_OCR::LoadEngine", "Step 1: Starting engine load", __FILE__, __LINE__);
@@ -866,6 +877,203 @@ namespace ANSCENTER
return collapsed.substr(first, last - first + 1);
}
// Drop visual separators (hyphens, commas, every flavour of whitespace)
// from a plate string so the emitted className is a contiguous token.
// See header comment for full rationale. UTF-8-aware so it handles
// Japanese OCR output that may produce full-width or CJK variants.
std::string ANSALPR_OCR::StripPlateSeparators(const std::string& text) {
if (text.empty()) return text;
std::string stripped;
stripped.reserve(text.size());
size_t pos = 0;
while (pos < text.size()) {
const size_t before = pos;
uint32_t cp = ANSOCRUtility::NextUTF8Codepoint(text, pos);
if (cp == 0 || pos == before) break;
bool drop = false;
switch (cp) {
// --- Spaces (every flavour the recognizer can emit) ---
case 0x0020: // ASCII space
case 0x0009: // horizontal tab
case 0x00A0: // no-break space
case 0x1680: // ogham space mark
case 0x2000: // en quad
case 0x2001: // em quad
case 0x2002: // en space
case 0x2003: // em space
case 0x2004: // three-per-em space
case 0x2005: // four-per-em space
case 0x2006: // six-per-em space
case 0x2007: // figure space
case 0x2008: // punctuation space
case 0x2009: // thin space
case 0x200A: // hair space
case 0x202F: // narrow no-break space
case 0x205F: // medium mathematical space
case 0x3000: //   ideographic space (CJK)
// --- Periods / full stops ---
case 0x002E: // . ASCII full stop / period
case 0xFF0E: // . fullwidth full stop
// --- Hyphens / dashes ---
case 0x002D: // - ASCII hyphen-minus
case 0x2010: // ‐ HYPHEN
case 0x2011: // ‑ NON-BREAKING HYPHEN
case 0x2012: // ‒ FIGURE DASH
case 0x2013: // – EN DASH
case 0x2014: // — EM DASH
case 0x2015: // ― HORIZONTAL BAR
case 0x2212: // − MINUS SIGN
case 0xFF0D: // - fullwidth hyphen-minus
case 0x30FC: // ー katakana-hiragana prolonged sound mark
// (visually similar to a dash and the OCR
// sometimes emits it where a hyphen sits)
drop = true;
break;
default:
break;
}
if (!drop) {
stripped.append(text, before, pos - before);
}
}
return stripped;
}
// ────────────────────────────────────────────────────────────────────
// Country-specific post-processing. See header for full contract.
// Mirrors the ANSALPR_OD pipeline so OD and OCR engines emit the
// same canonical form for Vietnam / Indonesia / Australia / USA,
// while leaving Japan / China text untouched (kana / kanji would
// be destroyed by an ASCII alnum filter, so we pass-through).
// ────────────────────────────────────────────────────────────────────
std::string ANSALPR_OCR::AnalyseLicensePlateText(const std::string& ocrText) {
if (ocrText.empty()) return ocrText;
// Country gate — Vietnam-only. Every other country (Japan, China,
// Indonesia, Australia, USA, …) returns the input unchanged so the
// upstream StripPlateSeparators output flows through verbatim and
// kana / kanji / region-specific punctuation is preserved.
if (_country != Country::VIETNAM) {
return ocrText;
}
try {
// Vietnam: keep only ASCII alphanumerics (StripPlateSeparators
// already removed the common separators; this is a defensive
// second pass that also catches any stray punctuation the OCR
// may have emitted). Uppercase so the format-match literals
// 'M', 'D', 'N', 'G', 'Q', 'T', 'C', 'V' work against the
// uppercase plate output.
std::string analysedLP;
analysedLP.reserve(ocrText.size());
for (char c : ocrText) {
if (std::isalnum(static_cast<unsigned char>(c))) {
analysedLP += c;
}
}
std::transform(analysedLP.begin(), analysedLP.end(), analysedLP.begin(),
[](unsigned char ch) { return static_cast<char>(std::toupper(ch)); });
// Format validation: when `_plateFormats` is configured (which
// it is by default for Vietnam via ApplyCountryDefaultPlateFormats),
// reject plates that don't match any pattern. Empty list = no
// validation (caller cleared it via SetPlateFormats), in which
// case everything is accepted just like ANSALPR_OD does.
if (!analysedLP.empty() && !_plateFormats.empty()
&& !MatchesPlateFormat(analysedLP)) {
ANS_DBG("ALPR_Format",
"REJECT plate='%s' (no _plateFormats match)",
analysedLP.c_str());
return "";
}
return analysedLP;
}
catch (const std::exception& e) {
this->_logger.LogFatal("ANSALPR_OCR::AnalyseLicensePlateText",
e.what(), __FILE__, __LINE__);
return "";
}
}
bool ANSALPR_OCR::MatchesPlateFormat(const std::string& plate) const {
if (_plateFormats.empty()) {
return true; // No formats configured → accept all.
}
for (const auto& format : _plateFormats) {
if (plate.size() != format.size()) continue;
bool matches = true;
for (size_t i = 0; i < format.size(); ++i) {
char f = format[i];
char p = plate[i];
if (f == 'd') {
if (!std::isdigit(static_cast<unsigned char>(p))) { matches = false; break; }
}
else if (f == 'l') {
if (!std::isalpha(static_cast<unsigned char>(p))) { matches = false; break; }
}
else {
// Literal character — exact match required.
if (p != f) { matches = false; break; }
}
}
if (matches) return true;
}
return false;
}
void ANSALPR_OCR::ApplyCountryDefaultPlateFormats() {
// Reset first so repeated SetCountry calls don't accumulate stale
// patterns from previous countries.
_plateFormats.clear();
switch (_country) {
case Country::VIETNAM: {
// Same 11 patterns ANSALPR_OD installs for Vietnam. Keeping
// these identical ensures the OD- and OCR-based pipelines
// accept exactly the same plate strings in production.
// ddlddddd — standard car (e.g. 29A12345)
// ddldddd — short variant
// ddldddddd — extended variant
// ddllddddd — two-letter series
// ddMDdddddd — military
// dddddNGdd — diplomatic NG
// dddddQTdd — diplomatic QT
// dddddCVdd — diplomatic CV
// dddddNNdd — diplomatic NN
// lldddd — special two-letter prefix
_plateFormats.push_back("ddlddddd");
_plateFormats.push_back("ddldddd");
_plateFormats.push_back("ddldddddd");
_plateFormats.push_back("ddllddddd");
_plateFormats.push_back("ddllddddd");
_plateFormats.push_back("ddMDdddddd");
_plateFormats.push_back("dddddNGdd");
_plateFormats.push_back("dddddQTdd");
_plateFormats.push_back("dddddCVdd");
_plateFormats.push_back("dddddNNdd");
_plateFormats.push_back("lldddd");
ANS_DBG("ALPR_Format",
"ApplyCountryDefaultPlateFormats: VIETNAM (%zu patterns)",
_plateFormats.size());
break;
}
case Country::CHINA:
case Country::AUSTRALIA:
case Country::USA:
case Country::INDONESIA:
case Country::JAPAN:
default:
// No default patterns configured for these countries. The
// caller can still install custom formats via SetPlateFormat
// / SetPlateFormats; otherwise format validation accepts all.
ANS_DBG("ALPR_Format",
"ApplyCountryDefaultPlateFormats: country=%d (no default patterns)",
static_cast<int>(_country));
break;
}
}
std::string ANSALPR_OCR::RecoverKanaFromBottomHalf(
const cv::Mat& plateROI, int halfH) const
{
@@ -1023,8 +1231,10 @@ namespace ANSCENTER
// ALPRChecker voting + ensureUniquePlateText dedup.
bool ANSALPR_OCR::shouldUseALPRChecker(const cv::Size& imageSize,
const std::string& cameraId) {
// Force disabled via SetALPRCheckerEnabled(false) → never use.
if (!_enableALPRChecker) return false;
// Force disabled via SetALPRCheckerEnabled(false) → never use. Relaxed
// atomic load: tear-free read is all we need, "next frame sees the new
// value" is the intended contract.
if (!_enableALPRChecker.load(std::memory_order_relaxed)) return false;
// Small images are always pipeline crops — skip auto-detection.
if (imageSize.width < ImageSizeTracker::MIN_FULLFRAME_WIDTH) return false;
@@ -1276,22 +1486,28 @@ namespace ANSCENTER
this->_logger.LogError("ANSALPR_OCR::RunInference", "Invalid license", __FILE__, __LINE__);
return {};
}
if (!_isInitialized) {
this->_logger.LogError("ANSALPR_OCR::RunInference", "Model is not initialized", __FILE__, __LINE__);
// LoadEngine() may be running on another thread (3+ seconds for all
// models). Silently defer instead of spamming "Model is not
// initialized" — the caller will retry on the next frame.
if (_modelLoading.load(std::memory_order_acquire)) {
return {};
}
// Source-of-truth check on the actual engine pointers. If LoadEngine
// failed or Destroy() was called, _lpDetector / _ocrEngine are null
// and we want a real error logged so the operator sees it.
if (!_lpDetector || !_ocrEngine) {
this->_logger.LogError("ANSALPR_OCR::RunInference",
"Model not loaded (LoadEngine not called or failed)",
__FILE__, __LINE__);
return {};
}
if (input.empty() || input.cols < 5 || input.rows < 5) {
this->_logger.LogError("ANSALPR_OCR::RunInference", "Input image is empty or too small", __FILE__, __LINE__);
return {};
}
if (!_lpDetector) {
this->_logger.LogFatal("ANSALPR_OCR::RunInference", "_lpDetector is null", __FILE__, __LINE__);
return {};
}
if (!_ocrEngine) {
this->_logger.LogFatal("ANSALPR_OCR::RunInference", "_ocrEngine is null", __FILE__, __LINE__);
return {};
}
try {
// Convert grayscale to BGR if necessary
@@ -1338,11 +1554,26 @@ namespace ANSCENTER
// lets SetCountry(nonJapan) take effect on the very next
// frame without a restart.
const bool useRectification = (_country == Country::JAPAN);
// Decide once per frame whether the tracker-based correction
// layer should run. Used by both the OCR-skip optimization
// below and by Step 4's checkPlateByTrackId call further down.
// We auto-detect full-frame vs pipeline mode by watching for
// pixel-identical consecutive frames, exactly the same way
// ANSALPR_OD does it.
const bool useChecker = shouldUseALPRChecker(
cv::Size(frameWidth, frameHeight), cameraId);
struct PlateInfo {
size_t origIndex; // into lprOutput
std::vector<size_t> cropIndices; // into allCrops
cv::Mat plateROI; // full (unsplit) ROI, kept for colour + kana recovery
int halfH = 0; // row-split Y inside plateROI (0 = single row)
// Skip-when-locked: if non-empty, this plate's track is already
// locked in alprChecker and we re-used the locked text instead
// of running OCR. Step 4 emits this directly; cropIndices is
// empty for these entries. See ALPRChecker::tryReuseLockedText.
std::string presetClassName;
};
std::vector<cv::Mat> allCrops;
std::vector<PlateInfo> plateInfos;
@@ -1359,6 +1590,34 @@ namespace ANSCENTER
const int height = std::min(frameHeight - y1, box.height);
if (width <= 0 || height <= 0) continue;
// Skip-when-locked fast path. If alprChecker already has a
// locked text for this plate's trackId, re-use it directly and
// don't push any crops onto allCrops. Saves ~95 ms (AMD DML)
// or ~50 ms (Intel OpenVINO) per locked plate per frame. The
// re-verify period inside tryReuseLockedText lets one OCR run
// through every N frames so a wrong lock can self-correct.
//
// Gated on useChecker so pipeline mode (no tracker, no voting)
// always runs raw OCR exactly like before — the optimization
// only applies in the full-frame ALPR pipeline.
if (useChecker) {
std::string locked = alprChecker.tryReuseLockedText(
cameraId, lprOutput[i].trackId);
if (!locked.empty()) {
// Build a minimal plateROI for the colour-detection
// pass in Step 4 — we still want to run colour even
// on skipped plates because it's cached and very
// cheap once the cache is warm.
const cv::Rect safeBox(x1, y1, width, height);
PlateInfo info;
info.origIndex = i;
info.plateROI = frame(safeBox); // non-owning view
info.presetClassName = std::move(locked);
plateInfos.push_back(std::move(info));
continue;
}
}
// Pad the YOLO LP bbox by 5% on each side. Gives the
// rectifier some background for edge detection and helps
// when the detector cropped a character edge.
@@ -1416,28 +1675,50 @@ namespace ANSCENTER
plateInfos.push_back(std::move(info));
}
if (allCrops.empty()) {
// Nothing to emit: no detected plates AND no skipped-locked plates.
if (plateInfos.empty()) {
return {};
}
// Step 3: Single batched recognizer call for every crop.
// Step 3: Single batched recognizer call for every crop that
// wasn't short-circuited by the skip-when-locked fast path.
// ONNXOCRRecognizer groups crops by bucket width and issues
// one ORT Run per bucket — typically 1–2 GPU calls for an
// entire frame regardless of plate count.
auto ocrResults = _ocrEngine->RecognizeTextBatch(allCrops);
// entire frame regardless of plate count. When every plate
// in the frame is already locked, allCrops is empty and we
// skip the recognizer call entirely.
std::vector<std::pair<std::string, float>> ocrResults;
if (!allCrops.empty()) {
ocrResults = _ocrEngine->RecognizeTextBatch(allCrops);
}
// Step 4: Assemble per-plate output
// Step 4: Assemble per-plate output.
std::vector<Object> output;
output.reserve(plateInfos.size());
// Decide once per frame whether the tracker-based correction
// layer should run. We auto-detect full-frame vs pipeline mode
// by watching for pixel-identical consecutive frames, exactly
// the same way ANSALPR_OD does it.
const bool useChecker = shouldUseALPRChecker(
cv::Size(frameWidth, frameHeight), cameraId);
for (const auto& info : plateInfos) {
// Skip-when-locked fast path: this plate's track was already
// locked in alprChecker, no OCR was run, just emit the locked
// text directly. Bypasses CTC / artifact stripping / kana
// recovery / re-vote — all of which would no-op anyway since
// the locked text is already the canonical plate string.
if (!info.presetClassName.empty()) {
Object lprObject = lprOutput[info.origIndex];
lprObject.cameraId = cameraId;
lprObject.className = info.presetClassName;
// Colour detection still runs — it's cached per-(camera,
// plate-text) so subsequent frames are O(1) anyway.
std::string colour = DetectLPColourCached(
info.plateROI, cameraId, lprObject.className);
if (!colour.empty()) {
lprObject.extraInfo = "color:" + colour;
}
output.push_back(std::move(lprObject));
continue;
}
// Reassemble row-by-row so we can target the bottom row
// for kana recovery when the fast path silently dropped
// the hiragana on a Japanese 2-row plate.
@@ -1502,6 +1783,24 @@ namespace ANSCENTER
if (combinedText.empty()) continue;
// Final-pass post-processing: drop hyphens, commas, every
// flavour of whitespace so the emitted plate is a single
// contiguous token. Done BEFORE the voter so:
// 1. textHistory + lockedText store the canonical form
// (better vote convergence — "50H 304.07" and
// "50H304.07" no longer compete as separate strings).
// 2. Skip-when-locked emits stripped text automatically
// via ALPRChecker::tryReuseLockedText, no second strip.
combinedText = StripPlateSeparators(combinedText);
if (combinedText.empty()) continue;
// Vietnam-only: alnum filter + uppercase + format validation
// against `_plateFormats`. No-op on every other country.
// Same canonical form ANSALPR_OD emits, so OD- and OCR-based
// pipelines can be A/B compared on the same input video.
combinedText = AnalyseLicensePlateText(combinedText);
if (combinedText.empty()) continue;
Object lprObject = lprOutput[info.origIndex];
lprObject.cameraId = cameraId;
@@ -1570,19 +1869,25 @@ namespace ANSCENTER
this->_logger.LogError("ANSALPR_OCR::RunInferencesBatch", "Invalid license", __FILE__, __LINE__);
return {};
}
if (!_isInitialized) {
this->_logger.LogError("ANSALPR_OCR::RunInferencesBatch", "Model is not initialized", __FILE__, __LINE__);
// LoadEngine() may be running on another thread (3+ seconds for all
// models). Silently defer instead of spamming "Model is not
// initialized" — the caller will retry on the next frame.
if (_modelLoading.load(std::memory_order_acquire)) {
return {};
}
// Source-of-truth check on the actual engine pointers. If LoadEngine
// failed or Destroy() was called, _lpDetector / _ocrEngine are null
// and we want a real error logged so the operator sees it.
if (!_lpDetector || !_ocrEngine) {
this->_logger.LogError("ANSALPR_OCR::RunInferencesBatch",
"Model not loaded (LoadEngine not called or failed)",
__FILE__, __LINE__);
return {};
}
if (input.empty() || input.cols < 5 || input.rows < 5) return {};
if (!_lpDetector) {
this->_logger.LogFatal("ANSALPR_OCR::RunInferencesBatch", "_lpDetector is null", __FILE__, __LINE__);
return {};
}
if (!_ocrEngine) {
this->_logger.LogFatal("ANSALPR_OCR::RunInferencesBatch", "_ocrEngine is null", __FILE__, __LINE__);
return {};
}
if (vehicleBoxes.empty()) return {};
try {
@@ -1759,6 +2064,21 @@ namespace ANSCENTER
if (combined.empty()) continue;
// Final-pass post-processing — same separator strip the
// full-frame path applies before the voter. Pipeline mode
// has no voter, so we strip just before emit. Keeps the
// className consistent with the full-frame output format
// regardless of which inference path the caller chose.
combined = StripPlateSeparators(combined);
if (combined.empty()) continue;
// Vietnam-only post-processing (alnum + uppercase + format
// validation). Same gate as the full-frame path so OD and
// OCR engines emit identical strings on Vietnam input, and
// non-Vietnam countries pass through unchanged.
combined = AnalyseLicensePlateText(combined);
if (combined.empty()) continue;
Object out = pm.lpObj;
out.className = combined; // raw OCR — no ALPRChecker
out.cameraId = cameraId;
@@ -1867,6 +2187,10 @@ namespace ANSCENTER
if (_ocrEngine) {
_ocrEngine->SetCountry(country);
}
// Re-install country-default plate formats so a runtime country
// switch picks up Vietnam's format list (or clears it when leaving
// Vietnam) without needing an Initialize / restart.
ApplyCountryDefaultPlateFormats();
// Log every SetCountry call so runtime country switches are
// visible and we can confirm the update landed on the right
// handle. The recovery + rectification gates read _country on
+52
View File
@@ -109,6 +109,41 @@ namespace ANSCENTER
// --- Original model zip path (reused for ANSONNXOCR initialization) ---
std::string _modelZipFilePath;
// ────────────────────────────────────────────────────────────────
// Country-specific plate-format processing — Vietnam-only.
//
// When `_country == VIETNAM`, `AnalyseLicensePlateText` strips
// any remaining non-alphanumeric characters, uppercases the
// result, and validates it against `_plateFormats` (the same
// 11-pattern list ANSALPR_OD installs for Vietnam). Plates
// that don't match any configured format are dropped — the
// return value is "" and the caller skips the plate.
//
// Every other country (Japan, China, Indonesia, Australia, USA,
// …) returns the input unchanged so the upstream
// StripPlateSeparators output flows through verbatim, preserving
// kana / kanji / region-specific punctuation.
//
// Hooked in after StripPlateSeparators and before the ALPRChecker
// voter, so locked text + skip-when-locked output are emitted in
// the canonical Vietnam form just like ANSALPR_OD's path.
// ────────────────────────────────────────────────────────────────
[[nodiscard]] std::string AnalyseLicensePlateText(const std::string& ocrText);
// Format match helper — patterns: 'd' = any digit, 'l' = any
// letter, every other character is a literal that must match
// exactly. Returns true when `_plateFormats` is empty (i.e.
// no formats configured → no validation, accept all).
[[nodiscard]] bool MatchesPlateFormat(const std::string& plate) const;
// Reset `_plateFormats` to the default list for the current
// `_country`. Called at the end of Initialize (after country.txt
// is read) and from SetCountry so a runtime country switch picks
// up the right format set on the very next frame. Currently
// populates Vietnam's 11 standard patterns; other countries leave
// the list empty (= no validation).
void ApplyCountryDefaultPlateFormats();
// --- Colour detection helpers ---
[[nodiscard]] std::string DetectLPColourDetector(const cv::Mat& lprROI, const std::string& cameraId);
[[nodiscard]] std::string DetectLPColourCached(const cv::Mat& lprROI, const std::string& cameraId, const std::string& plateText);
@@ -191,6 +226,23 @@ namespace ANSCENTER
// characters are collapsed and leading/trailing spaces trimmed.
[[nodiscard]] static std::string StripPlateArtifacts(const std::string& text);
// Final-pass post-processing: strip the visual separators that some
// countries put between plate sections — spaces, periods, hyphens —
// so the emitted plate string is a contiguous alphanumeric token.
// Examples:
// "84-F1 273.36" → "84F127336"
// "54-0L 7564" → "540L7564"
// "59-N2 363.25" → "59N236325"
// "東京 500あ12-34" → "東京500あ1234"
// Handles ASCII space/period/hyphen-minus plus the common Unicode
// variants (NBSP, ideographic space, em/en dashes, full-width
// period and hyphen, JP prolonged-sound mark) so JP-OCR output
// is normalised the same way as Latin output.
// Applied BEFORE the ALPRChecker voter so voting + locking both
// operate on the canonical stripped form, and the skip-when-locked
// path inherits stripped output without a second strip site.
[[nodiscard]] static std::string StripPlateSeparators(const std::string& text);
// Run recognizer-only on a tight crop of the left portion of the
// bottom half, trying three vertical offsets to absorb row-split
// inaccuracies. Returns the first non-empty result that contains
+139 -245
View File
@@ -356,6 +356,7 @@ namespace ANSCENTER {
constexpr float ROW_SPLIT_MIN_GAP_FACTOR = 0.2f; // maxGap < avgHeight * this => single row
constexpr float ROW_SPLIT_AVGY_FACTOR = 0.4f; // avgY diff must exceed avgHeight * this
constexpr size_t ROW_SPLIT_MIN_GROUP_SIZE = 2; // minimum chars per row
constexpr float Y_CLUSTER_STRONG_FACTOR = 0.7f; // maxYGap > avgHeight * this => Y-cluster reliable, else use PCA
constexpr float DUPLICATE_DIST_THRESHOLD = 5.0f; // pixels: chars closer than this are duplicates
constexpr int DUPLICATE_GRID_SIZE = 10; // spatial hash grid cell size in pixels
constexpr int ASYNC_TIMEOUT_SECONDS = 30; // timeout for async worker threads
@@ -1041,8 +1042,9 @@ namespace ANSCENTER {
// voting and ensureUniquePlateText dedup in all Inference/RunInference paths.
if (!_enableTrackerVoting) return false;
// Force disabled → never use
if (!_enableALPRChecker) return false;
// Force disabled → never use. Relaxed load is fine: tear-free read of a
// single bool is all we need; "next frame sees the new value" is the contract.
if (!_enableALPRChecker.load(std::memory_order_relaxed)) return false;
// Small images are always pipeline crops — skip auto-detection
if (imageSize.width < ImageSizeTracker::MIN_FULLFRAME_WIDTH) return false;
@@ -1239,9 +1241,10 @@ namespace ANSCENTER {
return {};
}
std::string ANSALPR_OD::DetectLicensePlateString(const cv::Mat& lprROI, const std::string& cameraId) {
// No coarse _mutex — _ocrDetector has its own m_inferenceMutex
// No coarse _mutex — _ocrDetector has its own m_inferenceMutex.
// Single-frame and batch paths now share post-processing via
// ProcessSingleOCRResult (PCA-based; robust to plate tilt).
try {
// convert lprROI to greyscale if it is not already
if (lprROI.empty()) {
this->_logger.LogError("ANSALPR_OD::DetectLicensePlateString", "Input image is empty", __FILE__, __LINE__);
return "";
@@ -1255,174 +1258,9 @@ namespace ANSCENTER {
}
std::vector<Object> ocrOutput = _ocrDetector->RunInference(grayLprROI, cameraId);
std::string ocrText = "";
if (ocrOutput.empty()) return ocrText;
//std::cout << "=== OCR Detections ===" << std::endl;
//for (size_t i = 0; i < ocrOutput.size(); ++i) {
// << "' X=" << ocrOutput[i].box.x
// << " Y=" << ocrOutput[i].box.y << std::endl;
//}
// Remove duplicates
std::vector<Object> uniqueOutput;
for (const auto& obj : ocrOutput) {
bool isDuplicate = false;
for (const auto& unique : uniqueOutput) {
if (std::abs(obj.box.x - unique.box.x) < DUPLICATE_DIST_THRESHOLD &&
std::abs(obj.box.y - unique.box.y) < DUPLICATE_DIST_THRESHOLD) {
isDuplicate = true;
break;
}
}
if (!isDuplicate) {
uniqueOutput.push_back(obj);
}
}
//std::cout << "\nAfter removing duplicates: " << uniqueOutput.size() << " chars" << std::endl;
if (uniqueOutput.empty()) return ocrText;
if (uniqueOutput.size() == 1) return uniqueOutput[0].className;
// Calculate average character height for threshold
float avgHeight = 0;
for (const auto& obj : uniqueOutput) {
avgHeight += obj.box.height;
}
avgHeight /= uniqueOutput.size();
// Calculate linear regression Y = mX + b
float sumX = 0, sumY = 0, sumXY = 0, sumX2 = 0;
int n = uniqueOutput.size();
for (const auto& obj : uniqueOutput) {
float x = obj.box.x;
float y = obj.box.y;
sumX += x;
sumY += y;
sumXY += x * y;
sumX2 += x * x;
}
float denominator = n * sumX2 - sumX * sumX;
float slope = (std::abs(denominator) > 1e-6f) ? (n * sumXY - sumX * sumY) / denominator : 0.0f;
float intercept = (n > 0) ? (sumY - slope * sumX) / n : 0.0f;
//std::cout << "Linear regression: Y = " << slope << " * X + " << intercept << std::endl;
// Calculate perpendicular distance from each point to regression line
float a = slope;
float b = -1.0f;
float c = intercept;
float normFactor = std::sqrt(a * a + b * b);
std::vector<std::pair<float, size_t>> distances;
for (size_t i = 0; i < uniqueOutput.size(); ++i) {
float x = uniqueOutput[i].box.x;
float y = uniqueOutput[i].box.y;
float dist = (a * x + b * y + c) / normFactor;
distances.push_back({ dist, i });
}
// Sort by perpendicular distance
std::sort(distances.begin(), distances.end(),
[](const std::pair<float, size_t>& a, const std::pair<float, size_t>& b) {
return a.first < b.first;
});
// Find largest gap in perpendicular distances
float maxGap = 0;
size_t splitIdx = distances.size() / 2;
//std::cout << "\n=== Distance gaps ===" << std::endl;
for (size_t i = 1; i < distances.size(); ++i) {
float gap = distances[i].first - distances[i - 1].first;
//std::cout << "Gap " << i << ": " << gap << std::endl;
if (gap > maxGap) {
maxGap = gap;
splitIdx = i;
}
}
// Check if this is actually a single row
// If max gap is too small relative to character height, it's a single row
std::vector<std::vector<Object>> rows;
if (maxGap < avgHeight * ROW_SPLIT_MIN_GAP_FACTOR) {
// Single row - all characters on one line
rows.resize(1);
rows[0] = uniqueOutput;
}
else {
// Two rows
rows.resize(2);
// Split into two rows based on perpendicular distance
for (size_t i = 0; i < distances.size(); ++i) {
size_t objIdx = distances[i].second;
if (i < splitIdx) {
rows[0].push_back(uniqueOutput[objIdx]);
}
else {
rows[1].push_back(uniqueOutput[objIdx]);
}
}
// Determine which row is on top (lower average Y = top row)
float avgY0 = 0, avgY1 = 0;
for (const auto& obj : rows[0]) {
avgY0 += obj.box.y;
}
for (const auto& obj : rows[1]) {
avgY1 += obj.box.y;
}
avgY0 /= rows[0].size();
avgY1 /= rows[1].size();
//std::cout << "Average Y: Row0=" << avgY0 << " Row1=" << avgY1 << std::endl;
// Swap if needed (top row should be row 0)
if (avgY0 > avgY1) {
//std::cout << "Swapping rows (Row 0 should be on top)" << std::endl;
std::swap(rows[0], rows[1]);
}
}
//std::cout << "\n=== Rows before X-sorting ===" << std::endl;
//for (size_t r = 0; r < rows.size(); ++r) {
// std::cout << "Row " << r << " (" << rows[r].size() << " chars): ";
// for (const auto& obj : rows[r]) {
// }
// std::cout << std::endl;
//}
// Sort each row by X
for (auto& row : rows) {
std::sort(row.begin(), row.end(),
[](const Object& a, const Object& b) {
return a.box.x < b.box.x;
});
}
/*std::cout << "\n=== Rows after X-sorting ===" << std::endl;
for (size_t r = 0; r < rows.size(); ++r) {
std::cout << "Row " << r << ": ";
for (const auto& obj : rows[r]) {
}
std::cout << std::endl;
}*/
// Concatenate
for (const auto& row : rows) {
for (const auto& obj : row) {
ocrText += obj.className;
}
}
//std::cout << "\nFinal text: '" << ocrText << "'" << std::endl;
grayLprROI.release(); // Release the grayscale image to free memory
std::string processedOcrText = AnalyseLicensePlateText(ocrText);
return processedOcrText;
ANS_DBG("OCR_PostProc", "cam=%s entry path=single ocr_dets=%zu lpr=%dx%d",
cameraId.c_str(), ocrOutput.size(), grayLprROI.cols, grayLprROI.rows);
return ProcessSingleOCRResult(ocrOutput, cameraId);
}
catch (std::exception& e) {
this->_logger.LogFatal("ANSALPR_OD::DetectLicensePlateString", e.what(), __FILE__, __LINE__);
@@ -2916,7 +2754,7 @@ namespace ANSCENTER {
// Process OCR results — ProcessSingleOCRResult is pure CPU math
// on ~5-20 characters per plate, so thread overhead dominates.
for (size_t i = 0; i < batchSize; ++i) {
results[i] = ProcessSingleOCRResult(ocrBatchOutputs[i]);
results[i] = ProcessSingleOCRResult(ocrBatchOutputs[i], cameraId);
}
return results;
@@ -2930,8 +2768,9 @@ namespace ANSCENTER {
}
// New helper function - thread-safe, no shared state
std::string ANSALPR_OD::ProcessSingleOCRResult(const std::vector<Object>& ocrOutput) {
std::string ANSALPR_OD::ProcessSingleOCRResult(const std::vector<Object>& ocrOutput, const std::string& cameraId) {
if (ocrOutput.empty()) {
ANS_DBG("OCR_PostProc", "cam=%s empty raw OCR output", cameraId.c_str());
return "";
}
@@ -2977,10 +2816,12 @@ namespace ANSCENTER {
}
if (uniqueOutput.empty()) {
ANS_DBG("OCR_PostProc", "cam=%s all dets dropped as duplicates raw=%zu", cameraId.c_str(), ocrOutput.size());
return "";
}
if (uniqueOutput.size() == 1) {
ANS_DBG("OCR_PostProc", "cam=%s single-char plate cls=%s", cameraId.c_str(), uniqueOutput[0].className.c_str());
return AnalyseLicensePlateText(uniqueOutput[0].className);
}
@@ -2995,17 +2836,85 @@ namespace ANSCENTER {
}
avgHeight /= static_cast<float>(n);
// ---- Mean center ----
// Hybrid row-split strategy:
// - If chars cluster cleanly in Y (maxYGap > avgHeight*Y_CLUSTER_STRONG_FACTOR),
// trust Y-clustering. Handles flat plates with asymmetric layouts where PCA
// gets fooled by cov(x,y) from row-centroid mismatch.
// - Otherwise, the plate is either single-row or significantly tilted; fall
// back to PCA on character centers to find the true reading direction,
// then split rows by perpendicular projection. Handles tilts up to ~45°
// where rows interleave in raw Y.
//
// Failure precedents:
// "30E89916" (78x64, top "30E" centered over wider bottom "89916", flat):
// PCA invents theta≈33° from layout → "30E16899". Y-cluster works.
// "65L149702" (heavily tilted ~28°):
// Y-cluster splits rows mid-row at the largest Y gap → "654970L21".
// PCA correctly finds reading direction.
std::vector<std::pair<float, size_t>> ySorted;
ySorted.reserve(n);
for (int i = 0; i < n; ++i)
ySorted.push_back({ cys[i], static_cast<size_t>(i) });
std::sort(ySorted.begin(), ySorted.end());
float maxYGap = 0.0f;
size_t ySplitIdx = ySorted.size();
const size_t minGroupSize = ROW_SPLIT_MIN_GROUP_SIZE;
for (size_t i = 1; i < ySorted.size(); ++i) {
if (i < minGroupSize || (ySorted.size() - i) < minGroupSize)
continue;
float gap = ySorted[i].first - ySorted[i - 1].first;
if (gap > maxYGap) {
maxYGap = gap;
ySplitIdx = i;
}
}
const float gapThreshold = avgHeight * ROW_SPLIT_MIN_GAP_FACTOR;
const float strongYThreshold = avgHeight * Y_CLUSTER_STRONG_FACTOR;
const bool useYCluster = (maxYGap > strongYThreshold) && (ySplitIdx < ySorted.size());
auto sortByXAndConcat = [&](std::vector<size_t>& row, std::string& out) {
std::sort(row.begin(), row.end(), [&](size_t a, size_t b) {
return uniqueOutput[a].box.x < uniqueOutput[b].box.x;
});
for (size_t idx : row) out += uniqueOutput[idx].className;
};
// ============================================================
// Path A: Y-clustering (flat plates with reliable Y separation)
// ============================================================
if (useYCluster) {
std::vector<size_t> topRow, bottomRow;
topRow.reserve(ySplitIdx);
bottomRow.reserve(ySorted.size() - ySplitIdx);
for (size_t i = 0; i < ySplitIdx; ++i) topRow.push_back(ySorted[i].second);
for (size_t i = ySplitIdx; i < ySorted.size(); ++i) bottomRow.push_back(ySorted[i].second);
std::string ocrText;
ocrText.reserve(n);
sortByXAndConcat(topRow, ocrText);
sortByXAndConcat(bottomRow, ocrText);
std::string finalText = AnalyseLicensePlateText(ocrText);
float avgY0 = 0.0f, avgY1 = 0.0f;
for (size_t idx : topRow) avgY0 += cys[idx];
for (size_t idx : bottomRow) avgY1 += cys[idx];
avgY0 /= static_cast<float>(topRow.size());
avgY1 /= static_cast<float>(bottomRow.size());
ANS_DBG("OCR_PostProc", "cam=%s Y-cluster two-row n=%d split=%zu/%zu maxYGap=%.2f avgYsep=%.2f raw='%s' final='%s'",
cameraId.c_str(), n, topRow.size(), bottomRow.size(),
maxYGap, std::abs(avgY1 - avgY0), ocrText.c_str(), finalText.c_str());
return finalText;
}
// ============================================================
// Path B: PCA (single-row or tilted plates)
// ============================================================
float meanX = 0.0f, meanY = 0.0f;
for (int i = 0; i < n; ++i) {
meanX += cxs[i];
meanY += cys[i];
}
meanX /= n;
meanY /= n;
for (int i = 0; i < n; ++i) { meanX += cxs[i]; meanY += cys[i]; }
meanX /= n; meanY /= n;
// PCA to find the plate's reading direction.
// Robust to large tilt angles unlike OLS regression.
float cov_xx = 0.0f, cov_xy = 0.0f, cov_yy = 0.0f;
for (int i = 0; i < n; ++i) {
float dx = cxs[i] - meanX;
@@ -3016,14 +2925,10 @@ namespace ANSCENTER {
}
float theta = 0.5f * std::atan2(2.0f * cov_xy, cov_xx - cov_yy);
float dirX = std::cos(theta);
float dirY = std::sin(theta);
float dirX = std::cos(theta), dirY = std::sin(theta);
if (dirX < 0) { dirX = -dirX; dirY = -dirY; }
float perpX = -dirY, perpY = dirX;
float perpX = -dirY;
float perpY = dirX;
// Project centers onto both axes
std::vector<float> projAlong(n), projPerp(n);
for (int i = 0; i < n; ++i) {
float dx = cxs[i] - meanX;
@@ -3032,84 +2937,73 @@ namespace ANSCENTER {
projPerp[i] = dx * perpX + dy * perpY;
}
// Row splitting using perpendicular projections
std::vector<std::pair<float, size_t>> perpSorted;
perpSorted.reserve(n);
for (int i = 0; i < n; ++i)
perpSorted.push_back({ projPerp[i], static_cast<size_t>(i) });
std::sort(perpSorted.begin(), perpSorted.end());
// Find largest gap with validation:
// 1. Both groups must have >= 2 chars
// 2. Groups must be vertically separated (avgY check)
float maxGap = 0.0f;
size_t splitIdx = perpSorted.size() / 2;
const size_t minGroupSize = ROW_SPLIT_MIN_GROUP_SIZE;
float maxPerpGap = 0.0f;
size_t perpSplitIdx = perpSorted.size();
for (size_t i = 1; i < perpSorted.size(); ++i) {
float gap = perpSorted[i].first - perpSorted[i - 1].first;
if (i < minGroupSize || (perpSorted.size() - i) < minGroupSize)
continue;
if (gap > maxGap) {
float gap = perpSorted[i].first - perpSorted[i - 1].first;
if (gap > maxPerpGap) {
float avgY_g1 = 0.0f, avgY_g2 = 0.0f;
for (size_t j = 0; j < i; ++j)
avgY_g1 += cys[perpSorted[j].second];
for (size_t j = i; j < perpSorted.size(); ++j)
avgY_g2 += cys[perpSorted[j].second];
for (size_t j = 0; j < i; ++j) avgY_g1 += cys[perpSorted[j].second];
for (size_t j = i; j < perpSorted.size(); ++j) avgY_g2 += cys[perpSorted[j].second];
avgY_g1 /= static_cast<float>(i);
avgY_g2 /= static_cast<float>(perpSorted.size() - i);
if (std::abs(avgY_g2 - avgY_g1) > avgHeight * ROW_SPLIT_AVGY_FACTOR) {
maxGap = gap;
splitIdx = i;
maxPerpGap = gap;
perpSplitIdx = i;
}
}
}
if (maxGap < avgHeight * ROW_SPLIT_MIN_GAP_FACTOR) {
// Single row - sort by projection along reading direction
std::vector<std::pair<float, size_t>> allProj;
for (int i = 0; i < n; ++i)
allProj.push_back({ projAlong[i], static_cast<size_t>(i) });
std::sort(allProj.begin(), allProj.end());
auto sortByProjAlongAndConcat = [&](std::vector<size_t>& row, std::string& out) {
std::sort(row.begin(), row.end(), [&](size_t a, size_t b) {
return projAlong[a] < projAlong[b];
});
for (size_t idx : row) out += uniqueOutput[idx].className;
};
if (maxPerpGap < gapThreshold || perpSplitIdx == perpSorted.size()) {
// Single row along PCA axis
std::vector<size_t> all(n);
for (int i = 0; i < n; ++i) all[i] = static_cast<size_t>(i);
std::string ocrText;
ocrText.reserve(n);
for (const auto& p : allProj)
ocrText += uniqueOutput[p.second].className;
return AnalyseLicensePlateText(ocrText);
sortByProjAlongAndConcat(all, ocrText);
std::string finalText = AnalyseLicensePlateText(ocrText);
ANS_DBG("OCR_PostProc", "cam=%s PCA single-row n=%d theta=%.3f maxPerpGap=%.2f raw='%s' final='%s'",
cameraId.c_str(), n, theta, maxPerpGap, ocrText.c_str(), finalText.c_str());
return finalText;
}
else {
// Two rows
std::vector<std::vector<size_t>> rowIndices(2);
for (size_t i = 0; i < perpSorted.size(); ++i) {
size_t objIdx = perpSorted[i].second;
if (i < splitIdx)
rowIndices[0].push_back(objIdx);
else
rowIndices[1].push_back(objIdx);
}
// Ensure row 0 is top row (lower average center-Y)
float avgY0 = 0.0f, avgY1 = 0.0f;
for (auto idx : rowIndices[0]) avgY0 += cys[idx];
for (auto idx : rowIndices[1]) avgY1 += cys[idx];
avgY0 /= static_cast<float>(rowIndices[0].size());
avgY1 /= static_cast<float>(rowIndices[1].size());
if (avgY0 > avgY1)
std::swap(rowIndices[0], rowIndices[1]);
std::vector<size_t> topRow, bottomRow;
topRow.reserve(perpSplitIdx);
bottomRow.reserve(perpSorted.size() - perpSplitIdx);
for (size_t i = 0; i < perpSplitIdx; ++i) topRow.push_back(perpSorted[i].second);
for (size_t i = perpSplitIdx; i < perpSorted.size(); ++i) bottomRow.push_back(perpSorted[i].second);
// Sort each row by projection along reading direction
std::string ocrText;
ocrText.reserve(n);
for (const auto& row : rowIndices) {
std::vector<std::pair<float, size_t>> rowProj;
for (auto idx : row)
rowProj.push_back({ projAlong[idx], idx });
std::sort(rowProj.begin(), rowProj.end());
for (const auto& p : rowProj)
ocrText += uniqueOutput[p.second].className;
}
return AnalyseLicensePlateText(ocrText);
}
float avgY0 = 0.0f, avgY1 = 0.0f;
for (size_t idx : topRow) avgY0 += cys[idx];
for (size_t idx : bottomRow) avgY1 += cys[idx];
avgY0 /= static_cast<float>(topRow.size());
avgY1 /= static_cast<float>(bottomRow.size());
if (avgY0 > avgY1) std::swap(topRow, bottomRow);
std::string ocrText;
ocrText.reserve(n);
sortByProjAlongAndConcat(topRow, ocrText);
sortByProjAlongAndConcat(bottomRow, ocrText);
std::string finalText = AnalyseLicensePlateText(ocrText);
ANS_DBG("OCR_PostProc", "cam=%s PCA two-row n=%d theta=%.3f split=%zu/%zu maxPerpGap=%.2f avgYsep=%.2f raw='%s' final='%s'",
cameraId.c_str(), n, theta, topRow.size(), bottomRow.size(),
maxPerpGap, std::abs(avgY1 - avgY0), ocrText.c_str(), finalText.c_str());
return finalText;
}
void ANSALPR_OD::ensureUniquePlateText(std::vector<Object>& results, const std::string& cameraId)
+1 -1
View File
@@ -249,7 +249,7 @@ namespace ANSCENTER
const std::string& cameraId) override;
[[nodiscard]] std::vector<std::string> DetectLicensePlateStringBatch(const std::vector<cv::Mat>& lprROIs, const std::string& cameraId);
[[nodiscard]] std::vector<std::string> DetectLPColourDetectorBatch(const std::vector<cv::Mat>& lprROIs, const std::string& cameraId);
[[nodiscard]] std::string ProcessSingleOCRResult(const std::vector<Object>& ocrOutput);
[[nodiscard]] std::string ProcessSingleOCRResult(const std::vector<Object>& ocrOutput, const std::string& cameraId);
[[nodiscard]] bool Destroy() override;
/// Propagate debug flag to all sub-detectors (LPD, OCR, colour)
+29 -1
View File
@@ -965,7 +965,11 @@ extern "C" ANSLPR_API int ANSALPR_SetFormats(ANSCENTER::ANSALPR** Handle, co
extern "C" ANSLPR_API int ANSALPR_SetALPRCheckerEnabled(ANSCENTER::ANSALPR** Handle, int enable) {
ANS_DBG("ANSLPR","ANSALPR_SetALPRCheckerEnabled: HandlePtr=%p, *Handle=%p, enable=%d",
(void*)Handle, (void*)(Handle ? *Handle : nullptr), enable);
if (!Handle || !*Handle) return -1;
if (!Handle || !*Handle) {
ANS_DBG("ALPR_Checker",
"ANSALPR_SetALPRCheckerEnabled: invalid handle, returning -1");
return -1;
}
try {
(*Handle)->SetALPRCheckerEnabled(enable != 0);
ANS_DBG("ALPR_Checker", "SetALPRCheckerEnabled=%d (%s)",
@@ -973,10 +977,34 @@ extern "C" ANSLPR_API int ANSALPR_SetALPRCheckerEnabled(ANSCENTER::ANSALPR**
return 1;
}
catch (...) {
ANS_DBG("ALPR_Checker",
"ANSALPR_SetALPRCheckerEnabled: exception caught, returning 0");
return 0;
}
}
// Returns the current ALPRChecker flag: 1 = enabled, 0 = disabled, -1 = invalid handle.
extern "C" ANSLPR_API int ANSALPR_GetALPRCheckerEnabled(ANSCENTER::ANSALPR** Handle) {
ANS_DBG("ANSLPR","ANSALPR_GetALPRCheckerEnabled: HandlePtr=%p, *Handle=%p",
(void*)Handle, (void*)(Handle ? *Handle : nullptr));
if (!Handle || !*Handle) {
ANS_DBG("ALPR_Checker",
"ANSALPR_GetALPRCheckerEnabled: invalid handle, returning -1");
return -1;
}
try {
const bool enabled = (*Handle)->IsALPRCheckerEnabled();
ANS_DBG("ALPR_Checker", "GetALPRCheckerEnabled=%d (%s)",
enabled ? 1 : 0, enabled ? "ON" : "OFF");
return enabled ? 1 : 0;
}
catch (...) {
ANS_DBG("ALPR_Checker",
"ANSALPR_GetALPRCheckerEnabled: exception caught, returning -1");
return -1;
}
}
extern "C" ANSLPR_API int ANSALPR_SetCountry(ANSCENTER::ANSALPR** Handle, int country) {
ANS_DBG("ANSLPR","ANSALPR_SetCountry: HandlePtr=%p, *Handle=%p, country=%d",
(void*)Handle, (void*)(Handle ? *Handle : nullptr), country);
+196 -23
View File
@@ -2,6 +2,7 @@
#include "include/clipper.h"
#include "ANSGpuFrameRegistry.h"
#include "NV12PreprocessHelper.h"
#include "ANSLicense.h" // ANS_DBG
#include <opencv2/imgproc.hpp>
#include <iostream>
@@ -12,6 +13,130 @@
namespace ANSCENTER {
namespace onnxocr {
// AMD/DirectML latency-based recreate (Path B) — record helper. See
// ONNXOCRRecognizer for the rationale. Arms _slowRecreatePending if
// a Run() exceeded the slow threshold while past warmup and outside
// the cooldown window.
void ONNXOCRDetector::RecordRunLatency(const char* context, double ms) {
if (ms <= kSlowRecreateThresholdMs) return;
if (!_warmedUp) return;
const auto now = std::chrono::steady_clock::now();
const auto sinceLast = std::chrono::duration_cast<std::chrono::milliseconds>(
now - _lastRecreateAt).count();
if (_lastRecreateAt.time_since_epoch().count() != 0 &&
sinceLast < kRecreateCooldownMs) {
ANS_DBG("ANSONNXDet_run",
"%s: slow Run (%.0f ms) but in cooldown (%lldms)",
context, ms, static_cast<long long>(sinceLast));
return;
}
_slowRecreatePending.store(true, std::memory_order_release);
std::cerr << "[ONNXOCRDetector] " << context
<< ": slow Run (" << ms
<< " ms > threshold) — flagged for proactive recreate\n";
ANS_DBG("ANSONNXDet_run",
"%s: slow Run %.0fms > %.0fms — armed proactive recreate",
context, ms, kSlowRecreateThresholdMs);
}
// AMD/DirectML latency-based recreate (Path B) — apply helper. Called
// at entry of public methods. If the slow flag was armed by a prior
// call's RecordRunLatency, swap the session in here so this call uses
// the fresh one.
void ONNXOCRDetector::ApplyPendingSlowRecreate(const char* context) {
if (!_slowRecreatePending.exchange(false, std::memory_order_acquire)) {
return;
}
const EngineType currentEp = getEngineType();
std::cerr << "[ONNXOCRDetector] " << context
<< ": proactive recreate on EP=" << static_cast<int>(currentEp)
<< " (latency-triggered)\n";
ANS_DBG("ANSONNXDet_run",
"%s: proactive recreate on EP=%d", context,
static_cast<int>(currentEp));
try {
Reinitialize(currentEp);
_warmedUp = false;
_lastRecreateAt = std::chrono::steady_clock::now();
std::cerr << "[ONNXOCRDetector] " << context
<< ": proactive recreate OK\n";
ANS_DBG("ANSONNXDet_run", "%s: proactive recreate OK", context);
} catch (const std::exception& e) {
std::cerr << "[ONNXOCRDetector] " << context
<< ": proactive recreate failed: " << e.what() << "\n";
ANS_DBG("ANSONNXDet_run",
"%s: proactive recreate failed (%s)", context, e.what());
} catch (...) {
std::cerr << "[ONNXOCRDetector] " << context
<< ": proactive recreate failed (unknown)\n";
}
}
// AMD/DirectML self-heal helper. See ONNXOCRRecognizer for the full
// rationale. On any Run() failure: recreate session on the same EP
// first (transient driver hiccup recovery — the common case for AMD
// DML 0x80004005 E_FAIL); on a second failure, fall back to CPU EP.
// The current call's result is already lost — caller returns empty;
// the recovered session is used by the NEXT Detect call.
void ONNXOCRDetector::HandleRunFailureAndRecreate(const char* context,
const std::exception& e) {
const EngineType currentEp = getEngineType();
std::cerr << "[ONNXOCRDetector] " << context
<< ": Run() failed on EP=" << static_cast<int>(currentEp)
<< ": " << e.what() << " — recreating session\n";
ANS_DBG("ANSONNXDet_run",
"%s: Run failed EP=%d msg=%.200s — recreating",
context, static_cast<int>(currentEp), e.what());
// Option A: trip process-wide DML serial mode on AMD exception
// (idempotent, no-op on Intel/NVIDIA/CPU).
if (currentEp == EngineType::AMD_GPU) {
ANSCENTER::DmlSerialState::TripFromException(
context, e.what());
}
if (currentEp != EngineType::CPU) {
try {
Reinitialize(currentEp);
_warmedUp = false;
std::cerr << "[ONNXOCRDetector] " << context
<< ": session recreated on EP=" << static_cast<int>(currentEp) << "\n";
ANS_DBG("ANSONNXDet_run",
"%s: session recreated on EP=%d", context,
static_cast<int>(currentEp));
return;
} catch (const std::exception& reErr) {
std::cerr << "[ONNXOCRDetector] " << context
<< ": same-EP recreate failed: " << reErr.what()
<< " — falling back to CPU\n";
ANS_DBG("ANSONNXDet_run",
"%s: same-EP recreate failed (%s); falling back to CPU",
context, reErr.what());
} catch (...) {
std::cerr << "[ONNXOCRDetector] " << context
<< ": same-EP recreate failed (unknown) — falling back to CPU\n";
}
}
try {
Reinitialize(EngineType::CPU);
_warmedUp = false;
std::cerr << "[ONNXOCRDetector] " << context
<< ": fallback session created on CPU\n";
ANS_DBG("ANSONNXDet_run", "%s: CPU fallback OK", context);
} catch (const std::exception& cpuErr) {
std::cerr << "[ONNXOCRDetector] " << context
<< ": CPU fallback ALSO failed: " << cpuErr.what()
<< " — ort_session may be null, future calls will return empty\n";
ANS_DBG("ANSONNXDet_run",
"%s: CPU fallback failed (%s); ort_session may be null",
context, cpuErr.what());
} catch (...) {
std::cerr << "[ONNXOCRDetector] " << context
<< ": CPU fallback ALSO failed (unknown)\n";
}
}
ONNXOCRDetector::ONNXOCRDetector(const std::string& onnx_path, unsigned int num_threads)
: BasicOrtHandler(onnx_path, num_threads) {
}
@@ -55,6 +180,11 @@ std::vector<TextBox> ONNXOCRDetector::Detect(const cv::Mat& srcImage,
return {};
}
// Path B (AMD/DML): apply pending proactive recreate before constructing
// any tensors. See ONNXOCRRecognizer for the full rationale.
ApplyPendingSlowRecreate("Detect");
if (!ort_session) return {}; // recreate may have failed; bail safely
// Try to get full-resolution image from NV12 frame (same pattern as ANSONNXYOLO)
cv::Mat inferenceImage = srcImage;
float bgrScaleX = 1.0f, bgrScaleY = 1.0f;
@@ -89,36 +219,60 @@ std::vector<TextBox> ONNXOCRDetector::Detect(const cv::Mat& srcImage,
*memory_info_handler, inputData.data(), inputData.size(),
inputShape.data(), inputShape.size());
// Run inference
auto outputTensors = ort_session->Run(
Ort::RunOptions{ nullptr },
input_node_names.data(), &inputTensor, 1,
output_node_names.data(), num_outputs);
// Run inference + post-process inside try/catch. AMD/DirectML
// self-heal: on Run() failure (typically 0x80004005 E_FAIL from a
// transient DML driver hiccup) we recreate the session for the next
// call (same EP first, CPU as last resort). This call's result is
// lost — return empty boxes; ALPR pipeline tolerates a dropped
// detection frame, the tracker recovers. Latency is also recorded
// for the Path B proactive-recreate trigger.
try {
const auto t0 = std::chrono::steady_clock::now();
std::vector<Ort::Value> outputTensors;
{
// Option A: serialize DML inference once per-process
// serial mode has been tripped (no-op on Intel/NVIDIA/CPU).
ANSCENTER::DmlInferenceGuard dmlGuard(
m_engineType, "ONNXOCRDetector::Detect");
outputTensors = ort_session->Run(
Ort::RunOptions{ nullptr },
input_node_names.data(), &inputTensor, 1,
output_node_names.data(), num_outputs);
}
const auto t1 = std::chrono::steady_clock::now();
const double runMs =
std::chrono::duration<double, std::milli>(t1 - t0).count();
RecordRunLatency("Detect", runMs);
// Get output data
float* outputData = outputTensors[0].GetTensorMutableData<float>();
auto outputShape = outputTensors[0].GetTensorTypeAndShapeInfo().GetShape();
// Get output data
float* outputData = outputTensors[0].GetTensorMutableData<float>();
auto outputShape = outputTensors[0].GetTensorTypeAndShapeInfo().GetShape();
int outH = static_cast<int>(outputShape[2]);
int outW = static_cast<int>(outputShape[3]);
int outH = static_cast<int>(outputShape[2]);
int outW = static_cast<int>(outputShape[3]);
// Postprocess — detection coords are relative to inferenceImage (full-res),
// then scaled back to srcImage (display-res) coordinates
auto boxes = Postprocess(outputData, outH, outW, ratioH, ratioW,
inferenceImage.rows, inferenceImage.cols,
dbThresh, boxThresh, unclipRatio, useDilation);
// Postprocess — detection coords are relative to inferenceImage (full-res),
// then scaled back to srcImage (display-res) coordinates
auto boxes = Postprocess(outputData, outH, outW, ratioH, ratioW,
inferenceImage.rows, inferenceImage.cols,
dbThresh, boxThresh, unclipRatio, useDilation);
// Rescale box coordinates from full-res to display-res
if (bgrScaleX != 1.0f || bgrScaleY != 1.0f) {
for (auto& box : boxes) {
for (auto& pt : box.points) {
pt.x *= bgrScaleX;
pt.y *= bgrScaleY;
// Rescale box coordinates from full-res to display-res
if (bgrScaleX != 1.0f || bgrScaleY != 1.0f) {
for (auto& box : boxes) {
for (auto& pt : box.points) {
pt.x *= bgrScaleX;
pt.y *= bgrScaleY;
}
}
}
}
return boxes;
return boxes;
}
catch (const Ort::Exception& e) {
HandleRunFailureAndRecreate("Detect", e);
return {};
}
}
std::vector<float> ONNXOCRDetector::Preprocess(const cv::Mat& srcImage, int maxSideLen,
@@ -323,6 +477,8 @@ void ONNXOCRDetector::Warmup() {
// multiple-of-32 rounding. cuDNN caches the algorithm for this
// shape so the first real inference doesn't pay the picker cost.
constexpr int kWarmupSide = 320;
bool warmupFailed = false;
std::string warmupErr;
try {
cv::Mat dummy(kWarmupSide, kWarmupSide, CV_8UC3, cv::Scalar(128, 128, 128));
cv::Mat dummyF;
@@ -347,6 +503,23 @@ void ONNXOCRDetector::Warmup() {
}
catch (const Ort::Exception& e) {
std::cerr << "[ONNXOCRDetector] Warmup failed: " << e.what() << std::endl;
warmupFailed = true;
warmupErr = e.what();
}
// AMD/DML self-heal: if warmup itself threw, the session is dead and
// every Detect() will fail. Recreate now (same EP first, CPU as
// fallback) so production traffic can still serve. Leave _warmedUp
// = false so the recovered session warms up on its first real call.
if (warmupFailed) {
ANS_DBG("ANSONNXDet_run",
"Warmup failed (%.200s); triggering self-heal",
warmupErr.c_str());
try {
std::runtime_error proxy(warmupErr);
HandleRunFailureAndRecreate("Warmup", proxy);
} catch (...) {}
return; // do NOT set _warmedUp; recovered session warms on first real Detect
}
_warmedUp = true;
}
@@ -4,6 +4,8 @@
#include "ONNXEngine.h"
#include <vector>
#include <mutex>
#include <atomic>
#include <chrono>
namespace ANSCENTER {
namespace onnxocr {
@@ -53,6 +55,25 @@ private:
// Expand 4-point box using Clipper offset
std::vector<cv::Point2f> UnclipPolygon(const std::array<cv::Point2f, 4>& box, float unclipRatio);
// AMD/DirectML self-heal helper — see ONNXOCRRecognizer for full
// contract. On Run() failure: recreate session on the same EP first
// (transient hiccup recovery), CPU fallback as last resort. Resets
// _warmedUp so the new session gets re-warmed on next Detect call.
void HandleRunFailureAndRecreate(const char* context, const std::exception& e);
// AMD/DirectML latency-based self-heal (Path B) — see ONNXOCRRecognizer
// for the full rationale. Detector is observed to hit the same
// slow-degradation pattern as the recognizer after a sibling DML
// session takes a fatal: Run() doesn't throw, just gets slow.
// 2 s threshold catches degraded individual Run() (healthy is 50–500 ms);
// 60 s cooldown prevents thrash on permanently broken driver state.
static constexpr double kSlowRecreateThresholdMs = 2000.0;
static constexpr int64_t kRecreateCooldownMs = 60000;
std::atomic<bool> _slowRecreatePending{false};
std::chrono::steady_clock::time_point _lastRecreateAt{};
void RecordRunLatency(const char* context, double ms);
void ApplyPendingSlowRecreate(const char* context);
std::mutex _mutex;
};
+360 -45
View File
@@ -1,4 +1,5 @@
#include "ONNXOCRRecognizer.h"
#include "ANSLicense.h" // ANS_DBG
#include <opencv2/imgproc.hpp>
#include <iostream>
@@ -74,6 +75,17 @@ int ONNXOCRRecognizer::RoundUpToBucket(int resizedW) const {
return imgMaxW_;
}
// Round real batch size UP to the next entry in kRecBatchLadder, clamped to
// kRecMaxBatch. Padded slots in the input tensor are zero-filled and the
// corresponding output rows are discarded after Run(). See the comment on
// kRecBatchLadder in ONNXOCRTypes.h for the rationale.
static int RoundUpToBatchLadder(size_t n) {
for (int b : kRecBatchLadder) {
if (b >= static_cast<int>(n)) return b;
}
return kRecMaxBatch;
}
// Resize + normalize a single crop into a CHW float vector at width
// `bucketW`, padding with zeros on the right when needed. The returned
// vector has exactly 3*imgH_*bucketW elements.
@@ -101,6 +113,165 @@ static std::vector<float> PreprocessCropToBucket(const cv::Mat& crop,
return padded;
}
// AMD/DirectML latency-based recreate (Path B) — record helper.
//
// Called after every successful production-path Run(). If the call
// took longer than kSlowRecreateThresholdMs (5 s default), and the
// session has finished its warmup, and we're outside the cooldown
// window, arm _slowRecreatePending so the NEXT public method will
// perform a session swap before its own Run().
//
// Doesn't actually do the swap here — that has to happen at a clean
// boundary (no in-flight tensors, no held output Ort::Values), which
// is precisely the start of the next public method. Calling
// Reinitialize from inside the middle of an inference path while
// holding output_node_names / inputTensor / outputTensors references
// would invalidate them under our feet.
void ONNXOCRRecognizer::RecordRunLatency(const char* context, double ms) {
if (ms <= kSlowRecreateThresholdMs) return;
if (!_warmedUp) return; // first-shape compile is legitimately slow
// Cooldown gate — if we recreated recently, hold off. Avoids a
// recreate loop if the underlying issue is permanent (e.g. driver
// truly broken) rather than a transient hiccup.
const auto now = std::chrono::steady_clock::now();
const auto sinceLast = std::chrono::duration_cast<std::chrono::milliseconds>(
now - _lastRecreateAt).count();
if (_lastRecreateAt.time_since_epoch().count() != 0 &&
sinceLast < kRecreateCooldownMs) {
ANS_DBG("ANSONNXRec_run",
"%s: slow Run (%.0f ms) but in cooldown (%lldms since last recreate)",
context, ms, static_cast<long long>(sinceLast));
return;
}
_slowRecreatePending.store(true, std::memory_order_release);
std::cerr << "[ONNXOCRRecognizer] " << context
<< ": slow Run (" << ms
<< " ms > threshold) — flagged for proactive session recreate\n";
ANS_DBG("ANSONNXRec_run",
"%s: slow Run %.0fms > %.0fms — armed proactive recreate",
context, ms, kSlowRecreateThresholdMs);
}
// AMD/DirectML latency-based recreate (Path B) — apply helper.
//
// Called at the entry of every public method (Recognize,
// RunBatchAtWidth, Warmup, RecognizeBatch). If the slow flag was
// armed by a previous call's RecordRunLatency, do the session swap
// here — before any of this call's tensors are constructed. Caller
// already holds _mutex, so the new ort_session is exclusively ours
// for the rest of this call.
//
// On Reinitialize failure we leave _slowRecreatePending=false so we
// don't loop forever; the existing exception-based HandleRunFailure-
// AndRecreate path will fall back to CPU on the next actual throw.
void ONNXOCRRecognizer::ApplyPendingSlowRecreate(const char* context) {
if (!_slowRecreatePending.exchange(false, std::memory_order_acquire)) {
return;
}
const EngineType currentEp = getEngineType();
std::cerr << "[ONNXOCRRecognizer] " << context
<< ": proactive recreate on EP=" << static_cast<int>(currentEp)
<< " (latency-triggered)\n";
ANS_DBG("ANSONNXRec_run",
"%s: proactive recreate on EP=%d (latency-triggered)",
context, static_cast<int>(currentEp));
try {
Reinitialize(currentEp);
_warmedUp = false;
_lastRecreateAt = std::chrono::steady_clock::now();
std::cerr << "[ONNXOCRRecognizer] " << context
<< ": proactive recreate succeeded; next call retries on fresh session\n";
ANS_DBG("ANSONNXRec_run", "%s: proactive recreate OK", context);
} catch (const std::exception& e) {
std::cerr << "[ONNXOCRRecognizer] " << context
<< ": proactive recreate failed: " << e.what()
<< " — keeping existing session, exception path will catch real failures\n";
ANS_DBG("ANSONNXRec_run",
"%s: proactive recreate failed (%s)", context, e.what());
} catch (...) {
std::cerr << "[ONNXOCRRecognizer] " << context
<< ": proactive recreate failed (unknown)\n";
}
}
// AMD/DirectML self-heal helper. See header comment for detailed contract.
// In short: on Run() failure, recreate the session on the same EP first
// (transient driver hiccup recovery — the common case); on a second
// failure, fall back to CPU EP. Either path leaves ort_session in a
// usable state for the NEXT Recognize/RunBatchAtWidth call. This call's
// result is already lost — caller returns empty.
void ONNXOCRRecognizer::HandleRunFailureAndRecreate(const char* context,
const std::exception& e) {
const EngineType currentEp = getEngineType();
std::cerr << "[ONNXOCRRecognizer] " << context
<< ": Run() failed on EP=" << static_cast<int>(currentEp)
<< ": " << e.what() << " — recreating session\n";
ANS_DBG("ANSONNXRec_run",
"%s: Run failed EP=%d msg=%.200s — recreating",
context, static_cast<int>(currentEp), e.what());
// Option A: a Run() exception on a DML session is the strongest
// signal the AMD driver can't sustain parallel inference. Trip
// process-wide DML serial mode so future inference calls (from
// any module) take the global mutex. Idempotent — only the
// first caller per process flips the flag. No-op on
// Intel/NVIDIA/CPU.
if (currentEp == EngineType::AMD_GPU) {
ANSCENTER::DmlSerialState::TripFromException(
context, e.what());
}
// Step 1: try to recover on the same EP. Most DML 0x80004005 failures
// are transient driver hiccups; a fresh session typically works. This
// is also a no-op cost on Intel/NVIDIA — those EPs effectively never
// throw from Run() under normal load, so this path is rarely taken.
if (currentEp != EngineType::CPU) {
try {
Reinitialize(currentEp);
_warmedUp = false;
std::cerr << "[ONNXOCRRecognizer] " << context
<< ": session recreated on EP=" << static_cast<int>(currentEp)
<< " — next call will retry on GPU\n";
ANS_DBG("ANSONNXRec_run",
"%s: session recreated on EP=%d (next call retries on GPU)",
context, static_cast<int>(currentEp));
return;
} catch (const std::exception& reCreateErr) {
std::cerr << "[ONNXOCRRecognizer] " << context
<< ": same-EP recreate failed: " << reCreateErr.what()
<< " — falling back to CPU\n";
ANS_DBG("ANSONNXRec_run",
"%s: same-EP recreate failed (%s); falling back to CPU",
context, reCreateErr.what());
} catch (...) {
std::cerr << "[ONNXOCRRecognizer] " << context
<< ": same-EP recreate failed (unknown) — falling back to CPU\n";
}
}
// Step 2: last resort — recreate on CPU. Slower but always works
// when the model itself is valid. Caller's next call uses CPU EP.
try {
Reinitialize(EngineType::CPU);
_warmedUp = false;
std::cerr << "[ONNXOCRRecognizer] " << context
<< ": fallback session created on CPU\n";
ANS_DBG("ANSONNXRec_run", "%s: CPU fallback session created", context);
} catch (const std::exception& cpuErr) {
std::cerr << "[ONNXOCRRecognizer] " << context
<< ": CPU fallback ALSO failed: " << cpuErr.what()
<< " — ort_session likely null, future calls will return empty\n";
ANS_DBG("ANSONNXRec_run",
"%s: CPU fallback failed (%s); ort_session may be null",
context, cpuErr.what());
} catch (...) {
std::cerr << "[ONNXOCRRecognizer] " << context
<< ": CPU fallback ALSO failed (unknown exception)\n";
}
}
TextLine ONNXOCRRecognizer::Recognize(const cv::Mat& croppedImage) {
std::lock_guard<std::mutex> lock(_mutex);
@@ -108,6 +279,12 @@ TextLine ONNXOCRRecognizer::Recognize(const cv::Mat& croppedImage) {
return {};
}
// Path B (AMD/DML): if a previous call observed a slow Run(), it armed
// a proactive-recreate flag. Apply it now — BEFORE this call's tensors
// are built — so the rest of this method runs on a fresh session.
ApplyPendingSlowRecreate("Recognize");
if (!ort_session) return {}; // recreate could have failed; bail safely
try {
// Step 1: aspect-preserving resize to height=imgH_, width capped
// at imgMaxW_. Then round resized width up to the next bucket.
@@ -121,10 +298,24 @@ TextLine ONNXOCRRecognizer::Recognize(const cv::Mat& croppedImage) {
*memory_info_handler, inputData.data(), inputData.size(),
inputShape.data(), inputShape.size());
auto outputTensors = ort_session->Run(
Ort::RunOptions{ nullptr },
input_node_names.data(), &inputTensor, 1,
output_node_names.data(), num_outputs);
const auto t0 = std::chrono::steady_clock::now();
std::vector<Ort::Value> outputTensors;
{
// Option A: serialize DML inference once per-process
// serial mode has been tripped (no-op on Intel/NVIDIA/CPU).
ANSCENTER::DmlInferenceGuard dmlGuard(
m_engineType, "ONNXOCRRecognizer::Recognize");
outputTensors = ort_session->Run(
Ort::RunOptions{ nullptr },
input_node_names.data(), &inputTensor, 1,
output_node_names.data(), num_outputs);
}
const auto t1 = std::chrono::steady_clock::now();
const double runMs =
std::chrono::duration<double, std::milli>(t1 - t0).count();
// Latency-based recreate (Path B): arms the flag for the NEXT call
// if this one was abnormally slow.
RecordRunLatency("Recognize", runMs);
float* outputData = outputTensors[0].GetTensorMutableData<float>();
auto outputShape = outputTensors[0].GetTensorTypeAndShapeInfo().GetShape();
@@ -135,7 +326,11 @@ TextLine ONNXOCRRecognizer::Recognize(const cv::Mat& croppedImage) {
return CTCDecode(outputData, seqLen, numClasses);
}
catch (const Ort::Exception& e) {
std::cerr << "[ONNXOCRRecognizer] Inference failed: " << e.what() << std::endl;
// AMD/DML self-heal: recreate session for next call (same EP first,
// CPU fallback as last resort). This call's output is lost — return
// empty. The OCR pipeline tolerates dropped frames; the ALPR voting
// tracker migrates plate text across frames.
HandleRunFailureAndRecreate("Recognize", e);
return {};
}
}
@@ -146,20 +341,37 @@ void ONNXOCRRecognizer::RunBatchAtWidth(const std::vector<cv::Mat>& crops,
std::vector<TextLine>& out) {
if (crops.empty()) return;
try {
const size_t batchN = crops.size();
const size_t perImage = static_cast<size_t>(3) * imgH_ * bucketW;
// Path B (AMD/DML): apply pending proactive recreate before any tensors
// are constructed for this batch. RecognizeBatch holds _mutex around
// every RunBatchAtWidth chunk it dispatches, so we're safe under the
// same lock the caller already owns.
ApplyPendingSlowRecreate("RunBatchAtWidth");
if (!ort_session) return; // recreate may have failed; abort safely
// Stack N preprocessed crops into one [N,3,H,W] buffer
std::vector<float> batchInput(batchN * perImage, 0.0f);
for (size_t i = 0; i < batchN; ++i) {
try {
// Real number of crops vs. padded batch size submitted to ORT.
// Padding to the ladder keeps the recognizer's input-shape set
// bounded (≤24 distinct shapes) so OpenVINO/DML don't recompile
// kernels mid-stream — see kRecBatchLadder in ONNXOCRTypes.h.
// The padded slots are zeros; their decoded output is discarded.
const size_t realN = crops.size();
const size_t paddedN = static_cast<size_t>(RoundUpToBatchLadder(realN));
const size_t perImage = static_cast<size_t>(3) * imgH_ * bucketW;
ANS_DBG("ANSONNXRec_run",
"RunBatchAtWidth: bucketW=%d realN=%zu paddedN=%zu",
bucketW, realN, paddedN);
// Stack realN preprocessed crops into a [paddedN,3,H,W] buffer.
// The remaining (paddedN - realN) slots stay zero-initialized.
std::vector<float> batchInput(paddedN * perImage, 0.0f);
for (size_t i = 0; i < realN; ++i) {
auto img = PreprocessCropToBucket(crops[i], imgH_, bucketW);
std::memcpy(&batchInput[i * perImage], img.data(),
perImage * sizeof(float));
}
std::array<int64_t, 4> inputShape = {
static_cast<int64_t>(batchN), 3,
static_cast<int64_t>(paddedN), 3,
static_cast<int64_t>(imgH_),
static_cast<int64_t>(bucketW)
};
@@ -167,15 +379,28 @@ void ONNXOCRRecognizer::RunBatchAtWidth(const std::vector<cv::Mat>& crops,
*memory_info_handler, batchInput.data(), batchInput.size(),
inputShape.data(), inputShape.size());
auto outputTensors = ort_session->Run(
Ort::RunOptions{ nullptr },
input_node_names.data(), &inputTensor, 1,
output_node_names.data(), num_outputs);
const auto t0 = std::chrono::steady_clock::now();
std::vector<Ort::Value> outputTensors;
{
// Option A: serialize DML inference once per-process
// serial mode has been tripped (no-op on Intel/NVIDIA/CPU).
ANSCENTER::DmlInferenceGuard dmlGuard(
m_engineType, "ONNXOCRRecognizer::RunBatchAtWidth");
outputTensors = ort_session->Run(
Ort::RunOptions{ nullptr },
input_node_names.data(), &inputTensor, 1,
output_node_names.data(), num_outputs);
}
const auto t1 = std::chrono::steady_clock::now();
const double runMs =
std::chrono::duration<double, std::milli>(t1 - t0).count();
RecordRunLatency("RunBatchAtWidth", runMs);
float* outputData = outputTensors[0].GetTensorMutableData<float>();
auto outputShape = outputTensors[0].GetTensorTypeAndShapeInfo().GetShape();
// Expected output: [N, seqLen, numClasses]
// Expected output: [paddedN, seqLen, numClasses]. We only decode
// the first realN rows; rows [realN..paddedN) are zero-pad noise.
if (outputShape.size() < 3) {
std::cerr << "[ONNXOCRRecognizer] Unexpected batch output rank: "
<< outputShape.size() << std::endl;
@@ -186,17 +411,24 @@ void ONNXOCRRecognizer::RunBatchAtWidth(const std::vector<cv::Mat>& crops,
const int numClasses = static_cast<int>(outputShape[2]);
const size_t perRow = static_cast<size_t>(seqLen) * numClasses;
for (int i = 0; i < outBatch && i < static_cast<int>(batchN); ++i) {
for (size_t i = 0; i < realN && static_cast<int>(i) < outBatch; ++i) {
TextLine tl = CTCDecode(outputData + i * perRow, seqLen, numClasses);
out[origIndices[i]] = std::move(tl);
}
}
catch (const Ort::Exception& e) {
// ORT will throw if the model doesn't support a batch dimension > 1.
// Fall back to per-image inference for this group.
// The original recovery path here covered "model doesn't support
// batch dim >1" — a one-shot decision: drop to single-image and
// try each crop. We keep that, but ALSO add AMD/DML self-heal
// on top: if the per-image fallback ALSO throws, we recreate the
// session (same EP first, then CPU) so the next caller gets a
// healthy session. The current call's results stay empty for
// any crops we couldn't decode — caller tolerates this.
std::cerr << "[ONNXOCRRecognizer] Batch inference failed at bucketW="
<< bucketW << " (" << e.what()
<< ") — falling back to single-image path." << std::endl;
bool anyPerImageFailed = false;
std::string lastPerImageErr;
for (size_t i = 0; i < crops.size(); ++i) {
// Direct call (we already hold _mutex via the public RecognizeBatch
// wrapper). Replicate the single-image preprocessing here to avoid
@@ -209,10 +441,18 @@ void ONNXOCRRecognizer::RunBatchAtWidth(const std::vector<cv::Mat>& crops,
Ort::Value inputTensor = Ort::Value::CreateTensor<float>(
*memory_info_handler, inputData.data(), inputData.size(),
inputShape.data(), inputShape.size());
auto outputTensors = ort_session->Run(
Ort::RunOptions{ nullptr },
input_node_names.data(), &inputTensor, 1,
output_node_names.data(), num_outputs);
std::vector<Ort::Value> outputTensors;
{
// Option A: per-image fallback path also goes
// through the DML serialization gate.
ANSCENTER::DmlInferenceGuard dmlGuard(
m_engineType,
"ONNXOCRRecognizer::RunBatchAtWidth/per-image");
outputTensors = ort_session->Run(
Ort::RunOptions{ nullptr },
input_node_names.data(), &inputTensor, 1,
output_node_names.data(), num_outputs);
}
float* outData = outputTensors[0].GetTensorMutableData<float>();
auto outShape = outputTensors[0].GetTensorTypeAndShapeInfo().GetShape();
int seqLen = static_cast<int>(outShape[1]);
@@ -222,6 +462,20 @@ void ONNXOCRRecognizer::RunBatchAtWidth(const std::vector<cv::Mat>& crops,
std::cerr << "[ONNXOCRRecognizer] Single-image fallback also failed: "
<< e2.what() << std::endl;
out[origIndices[i]] = {};
anyPerImageFailed = true;
lastPerImageErr = e2.what();
}
}
// If even the per-image fallback path is throwing, the session is
// genuinely unhealthy (DML device-lost, not just a batch-shape bug).
// Trigger the AMD/DML self-heal so the NEXT batch lands on a fresh
// session (same EP first, CPU as last resort).
if (anyPerImageFailed) {
try {
std::runtime_error proxy(lastPerImageErr);
HandleRunFailureAndRecreate("RunBatchAtWidth", proxy);
} catch (...) {
// HandleRunFailureAndRecreate already logs; swallow.
}
}
}
@@ -288,30 +542,91 @@ void ONNXOCRRecognizer::Warmup() {
cv::Mat dummy(imgH_ * 2, kRecBucketWidths[kRecNumBuckets - 1] * 2,
CV_8UC3, cv::Scalar(128, 128, 128));
for (int b = 0; b < kRecNumBuckets; ++b) {
const int bucketW = kRecBucketWidths[b];
try {
auto inputData = PreprocessCropToBucket(dummy, imgH_, bucketW);
std::array<int64_t, 4> inputShape = { 1, 3, imgH_, bucketW };
Ort::Value inputTensor = Ort::Value::CreateTensor<float>(
*memory_info_handler, inputData.data(), inputData.size(),
inputShape.data(), inputShape.size());
// Pre-compile every (batchN, bucketW) shape we expect to hit on the
// hot path. Each unique shape costs ~1.7 s to compile on OpenVINO and
// ~150 ms on DirectML; without this prewarm, that compile cost shows
// up as a multi-hundred-ms frame spike the first time a given batch
// count + plate-width combination occurs in the video stream.
//
// We deliberately warm only kRecWarmupBatchLadder ({1,2,4,8}) instead
// of the full kRecBatchLadder ({1,2,4,8,16,24}) — covers >95% of ALPR
// frames while keeping engine-init time bounded. Frames with 9+
// plates pay a one-time spike on first occurrence of batch=16/24.
auto totalT0 = std::chrono::high_resolution_clock::now();
int warmedShapes = 0;
int totalShapes = 0;
std::string lastWarmupErr;
for (int bs = 0; bs < kRecNumWarmupBatchSizes; ++bs) {
const int batchN = kRecWarmupBatchLadder[bs];
for (int b = 0; b < kRecNumBuckets; ++b) {
const int bucketW = kRecBucketWidths[b];
++totalShapes;
try {
const size_t perImage = static_cast<size_t>(3) * imgH_ * bucketW;
std::vector<float> batchInput(static_cast<size_t>(batchN) * perImage, 0.0f);
auto perCrop = PreprocessCropToBucket(dummy, imgH_, bucketW);
for (int i = 0; i < batchN; ++i) {
std::memcpy(&batchInput[static_cast<size_t>(i) * perImage],
perCrop.data(),
perImage * sizeof(float));
}
std::array<int64_t, 4> inputShape = {
static_cast<int64_t>(batchN), 3,
static_cast<int64_t>(imgH_),
static_cast<int64_t>(bucketW)
};
Ort::Value inputTensor = Ort::Value::CreateTensor<float>(
*memory_info_handler, batchInput.data(), batchInput.size(),
inputShape.data(), inputShape.size());
auto t0 = std::chrono::high_resolution_clock::now();
(void)ort_session->Run(
Ort::RunOptions{ nullptr },
input_node_names.data(), &inputTensor, 1,
output_node_names.data(), num_outputs);
auto t1 = std::chrono::high_resolution_clock::now();
double ms = std::chrono::duration<double, std::milli>(t1 - t0).count();
std::cout << "[ONNXOCRRecognizer] Warmup bucketW=" << bucketW
<< " " << ms << " ms" << std::endl;
}
catch (const Ort::Exception& e) {
std::cerr << "[ONNXOCRRecognizer] Warmup failed at bucketW="
<< bucketW << ": " << e.what() << std::endl;
auto t0 = std::chrono::high_resolution_clock::now();
(void)ort_session->Run(
Ort::RunOptions{ nullptr },
input_node_names.data(), &inputTensor, 1,
output_node_names.data(), num_outputs);
auto t1 = std::chrono::high_resolution_clock::now();
double ms = std::chrono::duration<double, std::milli>(t1 - t0).count();
std::cout << "[ONNXOCRRecognizer] Warmup batch=" << batchN
<< " bucketW=" << bucketW
<< " " << ms << " ms" << std::endl;
++warmedShapes;
}
catch (const Ort::Exception& e) {
std::cerr << "[ONNXOCRRecognizer] Warmup failed at batch="
<< batchN << " bucketW=" << bucketW << ": "
<< e.what() << std::endl;
lastWarmupErr = e.what();
}
}
}
auto totalT1 = std::chrono::high_resolution_clock::now();
double totalMs = std::chrono::duration<double, std::milli>(totalT1 - totalT0).count();
std::cout << "[ONNXOCRRecognizer] Warmup complete: " << warmedShapes
<< " shapes in " << totalMs << " ms total" << std::endl;
// AMD/DML self-heal: if NO shape warmed up, the session is dead and
// production traffic will hit Run() failures on every call. Recreate
// it now (same EP first, CPU as fallback) so the pipeline can still
// serve. We deliberately keep _warmedUp = false in this path; the
// next RecognizeBatch will re-trigger Warmup on the recovered session.
//
// We do NOT trigger self-heal on partial failure — losing 2 of 16
// shapes is normal (e.g. some bucket dims aren't supported by all
// EPs); the per-shape catch already guards against that.
if (warmedShapes == 0 && totalShapes > 0) {
std::cerr << "[ONNXOCRRecognizer] Warmup: ALL "
<< totalShapes << " shapes failed — triggering self-heal\n";
ANS_DBG("ANSONNXRec_run",
"Warmup: 0/%d shapes succeeded (last err=%.200s); recreating session",
totalShapes, lastWarmupErr.c_str());
try {
std::runtime_error proxy(lastWarmupErr);
HandleRunFailureAndRecreate("Warmup", proxy);
} catch (...) {}
// Don't set _warmedUp = true; let the recovered session warm up
// itself on the next caller's invocation.
return;
}
_warmedUp = true;
}
@@ -5,6 +5,8 @@
#include <vector>
#include <string>
#include <mutex>
#include <atomic>
#include <chrono>
namespace ANSCENTER {
namespace onnxocr {
@@ -51,6 +53,62 @@ private:
// CTC greedy decode
TextLine CTCDecode(const float* outputData, int seqLen, int numClasses);
// AMD/DirectML self-heal helper. Called by Recognize / RunBatchAtWidth /
// Warmup when ort_session->Run() throws an Ort::Exception. Tries to
// recreate the session on the same EP first (transient driver hiccup
// recovery — the common case for DML 0x80004005 E_FAIL); if that fails,
// falls back to CPU EP. On full failure leaves ort_session in whatever
// state Reinitialize left it (possibly nullptr); the next Recognize
// call will short-circuit on the !ort_session check and return empty.
// Resets _warmedUp so the new session gets re-warmed on first use.
void HandleRunFailureAndRecreate(const char* context, const std::exception& e);
// ----------------------------------------------------------------
// AMD/DirectML latency-based self-heal (Path B).
//
// Observed AMD failure mode: after a transient driver event (e.g.
// helmet detector hits DML E_FAIL and CPU-falls-back), DML stays in
// a degraded state. OCR Run() calls do NOT throw, but their latency
// jumps from 50–200 ms to 1000–7000 ms, often rising over time. The
// exception-based self-heal above never fires because there's no
// exception. This proactive path observes the latency and forces a
// session recreate when a Run() exceeds a sanity threshold — the
// fresh session typically returns to baseline latency.
//
// How it works:
// 1. Each public method records its Run() duration.
// 2. If duration > kSlowRecreateThresholdMs AND we're past warmup
// AND we're past the cooldown window, set _slowRecreatePending.
// 3. The NEXT call to a public method checks the flag at entry; if
// set, it calls Reinitialize(currentEp) BEFORE its Run() — so
// the call that sees the flag uses a fresh session and returns
// to fast latency.
// 4. After a recreate, _lastRecreateMs starts the cooldown timer.
//
// Threshold tuning: 2 s catches the AMD-degraded individual Run()
// pattern observed in production (1–2 s per OCR Run() when DML is
// sick). Healthy individual Run() is 50–500 ms, so 2 s leaves
// comfortable headroom. First-shape compiles on OpenVINO can hit
// ~1.7 s but those happen during Warmup, which is gated out below
// via the _warmedUp check. 60 s cooldown prevents thrash if the
// driver is in a permanently bad state — at most one recreate per
// minute per session.
// ----------------------------------------------------------------
static constexpr double kSlowRecreateThresholdMs = 2000.0;
static constexpr int64_t kRecreateCooldownMs = 60000;
// Note: a slow Run() detected by RecordRunLatency arms this flag;
// ApplyPendingSlowRecreate (called at every public-method entry)
// performs the actual Reinitialize and clears the flag. std::atomic
// is used so the read at entry is lock-free; the write happens under
// the same _mutex the public methods already hold, so the only race
// we're guarding is "another public method picked up the flag first."
std::atomic<bool> _slowRecreatePending{false};
std::chrono::steady_clock::time_point _lastRecreateAt{};
void RecordRunLatency(const char* context, double ms);
void ApplyPendingSlowRecreate(const char* context);
std::vector<std::string> keys_;
int imgH_ = kRecImgH;
int imgMaxW_ = kRecImgMaxW;
+31
View File
@@ -62,6 +62,37 @@ constexpr int kRecBatchSize = 6;
// reasonable for your expected plate count per frame.
constexpr int kRecMaxBatch = 24;
// Runtime batch-padding ladder. Real batch sizes get rounded UP to one of
// these values before the ORT Run() call, with zero-padded slots in the
// input tensor and the padded outputs discarded after CTC decode. The goal
// is to bound the number of distinct (batch, bucketW) input shapes the
// recognizer ever sees: with this ladder × kRecBucketWidths there are at
// most 24 unique shapes total, small enough for OpenVINO/DirectML to keep
// in their per-shape kernel cache instead of recompiling mid-stream.
//
// Without this padding, OpenVINO recompiles the recognizer for every new
// (N, W) combination — measured at ~1.7 s per recompile on Intel Iris Xe,
// which is exactly what produced the 1500–1800 ms frame spikes seen in the
// OPENVINO_GPU video tests.
//
// The last entry MUST equal kRecMaxBatch so the whole ladder stays inside
// the TRT dynamic profile [batch=1..kRecMaxBatch].
constexpr int kRecBatchLadder[] = { 1, 2, 4, 8, 16, 24 };
constexpr int kRecNumBatchSizes = sizeof(kRecBatchLadder) / sizeof(kRecBatchLadder[0]);
static_assert(kRecBatchLadder[kRecNumBatchSizes - 1] == kRecMaxBatch,
"Last kRecBatchLadder entry must equal kRecMaxBatch so that "
"padded shapes stay inside the TRT profile.");
// Warmup batch ladder — a subset of kRecBatchLadder used at engine init to
// pre-compile the most common runtime shapes. Kept smaller than the full
// ladder because each warmup Run on OpenVINO costs ~1.7 s of kernel-compile
// time, and {1,2,4,8} covers the realistic plate-count distribution for
// ALPR (>95% of frames have ≤8 detected plates). Frames with 9+ plates
// will pay a one-time recompile spike on first occurrence of batch=16/24.
constexpr int kRecWarmupBatchLadder[] = { 1, 2, 4, 8 };
constexpr int kRecNumWarmupBatchSizes =
sizeof(kRecWarmupBatchLadder) / sizeof(kRecWarmupBatchLadder[0]);
// A detected text box: 4 corner points (top-left, top-right, bottom-right, bottom-left)
struct TextBox {
std::array<cv::Point2f, 4> points;
+190 -11
View File
@@ -1,15 +1,20 @@
#include "ANSCUSTOMDetector.h"
#include "Utility.h"
#include "EPLoader.h"
#include <windows.h>
namespace ANSCENTER
{
std::string ANSCUSTOMDETECTOR::FindANSCustomLibraryName(const std::string& folderPath, const std::string& keyword) {
ANS_DBG("ANSCUSTOM_FindLib", "ENTRY folder=%s keyword=%s",
folderPath.c_str(), keyword.c_str());
std::string searchPath = folderPath + "\\*.dll";
WIN32_FIND_DATAA findFileData;
HANDLE hFind = FindFirstFileA(searchPath.c_str(), &findFileData);
if (hFind == INVALID_HANDLE_VALUE) {
ANS_DBG("ANSCUSTOM_FindLib", "NO_MATCH (FindFirstFile invalid handle) folder=%s",
folderPath.c_str());
return ""; // Return an empty string if no file is found
}
@@ -17,39 +22,68 @@ namespace ANSCENTER
std::string fileName = findFileData.cFileName;
if (fileName.find(keyword) != std::string::npos) {
FindClose(hFind); // Close the handle before returning
return folderPath + "\\" + fileName; // Return the full path
std::string fullPath = folderPath + "\\" + fileName;
ANS_DBG("ANSCUSTOM_FindLib", "FOUND %s", fullPath.c_str());
return fullPath; // Return the full path
}
} while (FindNextFileA(hFind, &findFileData) != 0);
FindClose(hFind); // Close the handle after finishing the loop
ANS_DBG("ANSCUSTOM_FindLib", "NO_MATCH folder=%s keyword=%s",
folderPath.c_str(), keyword.c_str());
return ""; // Return an empty string if no match is found
}
bool ANSCUSTOMDETECTOR::OptimizeModel(bool fp16, std::string& optimizedModelFolder) {
ANS_DBG("ANSCUSTOM_Optimize", "ENTRY this=%p fp16=%d", (void*)this, (int)fp16);
std::lock_guard<std::recursive_mutex> lock(_mutex);
if (!ANSODBase::OptimizeModel(fp16, optimizedModelFolder)) {
ANS_DBG("ANSCUSTOM_Optimize", "ERROR base OptimizeModel failed this=%p", (void*)this);
return false;
}
optimizedModelFolder = _modelFolder;
return _customDetector->OptimizeModel(fp16);
if (!_customDetector) {
ANS_DBG("ANSCUSTOM_Optimize", "ERROR _customDetector null this=%p", (void*)this);
return false;
}
bool ok = _customDetector->OptimizeModel(fp16);
ANS_DBG("ANSCUSTOM_Optimize", "%s this=%p folder=%s",
ok ? "SUCCESS" : "FAILED", (void*)this, optimizedModelFolder.c_str());
return ok;
}
bool ANSCUSTOMDETECTOR::LoadModel(const std::string& modelZipFilePath, const std::string& modelZipPassword) {
ANS_DBG("ANSCUSTOM_LoadModel", "ENTRY this=%p zip=%s", (void*)this, modelZipFilePath.c_str());
std::lock_guard<std::recursive_mutex> lock(_mutex);
ModelLoadingGuard mlg(_modelLoading);
try {
bool result = ANSODBase::LoadModel(modelZipFilePath, modelZipPassword);
if (!result) return false;
if (!result) {
ANS_DBG("ANSCUSTOM_LoadModel", "ERROR base LoadModel failed this=%p zip=%s",
(void*)this, modelZipFilePath.c_str());
return false;
}
std::string labelMap;
CreateCustomDetector();
if (!_customDetector) {
ANS_DBG("ANSCUSTOM_LoadModel", "ERROR _customDetector null after Create this=%p", (void*)this);
return false;
}
//_customDetector->SetLoadEngineOnCreate(this->_loadEngineOnCreation);
return _customDetector->Initialize(_modelFolder,_modelConfig.detectionScoreThreshold, labelMap);
bool ok = _customDetector->Initialize(_modelFolder,_modelConfig.detectionScoreThreshold, labelMap);
ANS_DBG("ANSCUSTOM_LoadModel", "%s this=%p folder=%s thr=%.3f",
ok ? "SUCCESS" : "FAILED", (void*)this, _modelFolder.c_str(),
_modelConfig.detectionScoreThreshold);
return ok;
}
catch (std::exception& e) {
ANS_DBG("ANSCUSTOM_LoadModel", "ERROR exception this=%p: %s", (void*)this, e.what());
this->_logger.LogFatal("ANSCUSTOMDETECTOR::LoadModel", e.what(), __FILE__, __LINE__);
return false;
}
}
bool ANSCUSTOMDETECTOR::LoadModelFromFolder(std::string licenseKey, ModelConfig modelConfig, std::string modelName, std::string className, const std::string& modelFolder, std::string& labelMap) {
ANS_DBG("ANSCUSTOM_LoadFolder", "ENTRY this=%p model=%s class=%s folder=%s",
(void*)this, modelName.c_str(), className.c_str(), modelFolder.c_str());
std::lock_guard<std::recursive_mutex> lock(_mutex);
ModelLoadingGuard mlg(_modelLoading);
try {
@@ -61,19 +95,32 @@ namespace ANSCENTER
if (_modelConfig.modelConfThreshold < 0.2)
_modelConfig.modelConfThreshold = 0.5;
CreateCustomDetector();
if (!_customDetector) {
ANS_DBG("ANSCUSTOM_LoadFolder", "ERROR _customDetector null after Create this=%p", (void*)this);
_isInitialized = false;
return false;
}
_customDetector->SetLoadEngineOnCreate(this->_loadEngineOnCreation);
_isInitialized = _customDetector->Initialize(_modelFolder, _modelConfig.detectionScoreThreshold, labelMap);
ANS_DBG("ANSCUSTOM_LoadFolder", "%s this=%p folder=%s thr=%.3f loadOnCreate=%d",
_isInitialized ? "SUCCESS" : "FAILED",
(void*)this, _modelFolder.c_str(),
_modelConfig.detectionScoreThreshold,
(int)this->_loadEngineOnCreation);
return _isInitialized;
}
catch (std::exception& e) {
ANS_DBG("ANSCUSTOM_LoadFolder", "ERROR exception this=%p: %s", (void*)this, e.what());
this->_logger.LogFatal("ANSCUSTOMDETECTOR::LoadModel", e.what(), __FILE__, __LINE__);
return false;
}
}
bool ANSCUSTOMDETECTOR::ConfigureParameters(Params& param) {
ANS_DBG("ANSCUSTOM_Configure", "ENTRY this=%p", (void*)this);
std::lock_guard<std::recursive_mutex> lock(_mutex);
try {
if (!_customDetector) {
ANS_DBG("ANSCUSTOM_Configure", "ERROR _customDetector null this=%p", (void*)this);
this->_logger.LogError("ANSCUSTOMDETECTOR::ConfigureParamaters", "Custom detector is not initialized", __FILE__, __LINE__);
return false;
}
@@ -131,16 +178,27 @@ namespace ANSCENTER
}
param.ROI_Values.push_back(roiValue);
}
ANS_DBG("ANSCUSTOM_Configure", "SUCCESS this=%p ROI_Config=%zu Options=%zu Params=%zu ROI_Values=%zu",
(void*)this, param.ROI_Config.size(), param.ROI_Options.size(),
param.Parameters.size(), param.ROI_Values.size());
return true;
}
catch (std::exception& e) {
ANS_DBG("ANSCUSTOM_Configure", "ERROR exception this=%p: %s", (void*)this, e.what());
this->_logger.LogFatal("ANSCUSTOMDETECTOR::ConfigureParamaters", e.what(), __FILE__, __LINE__);
return false;
}
}
bool ANSCUSTOMDETECTOR::SetParameters(const Params& param) {
ANS_DBG("ANSCUSTOM_SetParams", "ENTRY this=%p ROI_Config=%zu Options=%zu Params=%zu ROI_Values=%zu",
(void*)this, param.ROI_Config.size(), param.ROI_Options.size(),
param.Parameters.size(), param.ROI_Values.size());
std::lock_guard<std::recursive_mutex> lock(_mutex);
try {
if (!_customDetector) {
ANS_DBG("ANSCUSTOM_SetParams", "ERROR _customDetector null this=%p", (void*)this);
return false;
}
// Convert the vector of Params to a vector of CustomParams
ANSODBase::SetParameters(param);
CustomParams customParams;
@@ -193,61 +251,131 @@ namespace ANSCENTER
}
customParams.ROI_Values.push_back(roiValue);
}
return _customDetector->SetParameters(customParams);
bool ok = _customDetector->SetParameters(customParams);
ANS_DBG("ANSCUSTOM_SetParams", "%s this=%p", ok ? "SUCCESS" : "FAILED", (void*)this);
return ok;
}
catch (std::exception& e) {
ANS_DBG("ANSCUSTOM_SetParams", "ERROR exception this=%p: %s", (void*)this, e.what());
this->_logger.LogFatal("ANSCUSTOMDETECTOR::SetParamaters", e.what(), __FILE__, __LINE__);
return false;
}
}
bool ANSCUSTOMDETECTOR::CreateCustomDetector() {
ANS_DBG("ANSCUSTOM_CreateDet", "ENTRY this=%p folder=%s prevDet=%p prevHMod=%p",
(void*)this, _modelFolder.c_str(),
(void*)_customDetector.get(), (void*)hMod);
std::lock_guard<std::recursive_mutex> lock(_mutex);
try {
Destroy();
//1. Load dynamic library
_modelFilePath = FindANSCustomLibraryName(_modelFolder, "ANSCustom");//
_modelFilePath = FindANSCustomLibraryName(_modelFolder, "ANSCustom");//
if (!FileExist(_modelFilePath)) {
_modelFilePath = CreateFilePath(_modelFolder, "ANSCustomCode.dll"); // This is the dynamic library
}
//2. Check if the file exists again
if (!FileExist(_modelFilePath)) {
ANS_DBG("ANSCUSTOM_CreateDet", "ERROR DLL not found path=%s", _modelFilePath.c_str());
this->_logger.LogFatal("ANSCUSTOMDETECTOR::Initialize", "Cannot find ANSCustomCode.dll", __FILE__, __LINE__);
return false;
}
// 2b. Pre-load ANSCENTER's onnxruntime.dll into the process before
// LoadLibrary on the custom plugin. The plugin DLL has a static
// import of "onnxruntime.dll"; if no module with that base name is
// loaded yet, the Windows loader falls back to the safe DLL search
// path and binds to C:\Windows\System32\onnxruntime.dll (the
// Windows ML build, ORT 1.17). That older runtime does not support
// the API version the plugin's headers ask for, so its static-init
// of Ort::Global<void>::api_ becomes nullptr and the first Ort::
// call inside the plugin crashes. Once EPLoader has loaded the
// matching ORT, by-name match in the loader binds the plugin's
// import to it instead.
try {
const auto& ep = ANSCENTER::EPLoader::Current();
ANS_DBG("ANSCUSTOM_CreateDet",
"EP_PRELOADED type=%s dir=%s fromSubdir=%d",
ANSCENTER::EPLoader::EngineTypeName(ep.type),
ep.libraryDir.c_str(),
ep.fromSubdir ? 1 : 0);
}
catch (const std::exception& e) {
ANS_DBG("ANSCUSTOM_CreateDet",
"ERROR EPLoader::Current threw — plugin may bind to wrong onnxruntime.dll: %s",
e.what());
this->_logger.LogFatal("ANSCUSTOMDETECTOR::Initialize",
(std::string("EPLoader::Current failed: ") + e.what()).c_str(),
__FILE__, __LINE__);
return false;
}
// 3. Load the library
//
// Use LoadLibraryExW with LOAD_LIBRARY_SEARCH_DEFAULT_DIRS so the
// plugin's transitive imports (notably "onnxruntime.dll") are
// resolved against directories added with AddDllDirectory — i.e.
// the staging dir EPLoader copied the matching ORT into. Plain
// LoadLibrary uses legacy safe-DLL search, which only consults
// app dir + System32 + PATH and ignores AddDllDirectory entirely;
// that path picks up C:\Windows\System32\onnxruntime.dll (the
// Windows ML build, ORT 1.17), and the plugin's static-init of
// Ort::Global<void>::api_ becomes nullptr — first Ort:: call
// crashes. LOAD_LIBRARY_SEARCH_DLL_LOAD_DIR also lets a custom
// model bundle its own deps next to the plugin DLL if it wants.
std::wstring dllPath = String2WString(_modelFilePath);
hMod = LoadLibrary(dllPath.c_str());
hMod = LoadLibraryExW(
dllPath.c_str(),
nullptr,
LOAD_LIBRARY_SEARCH_DEFAULT_DIRS | LOAD_LIBRARY_SEARCH_DLL_LOAD_DIR);
if (!hMod) {
DWORD err = GetLastError();
ANS_DBG("ANSCUSTOM_CreateDet", "ERROR LoadLibraryEx failed path=%s err=%lu",
_modelFilePath.c_str(), (unsigned long)err);
this->_logger.LogFatal("ANSCUSTOMDETECTOR::Initialize", "Cannot open library", __FILE__, __LINE__);
return false;
}
ANS_DBG("ANSCUSTOM_CreateDet", "LIB_LOADED hMod=%p path=%s", (void*)hMod, _modelFilePath.c_str());
CreatePluginFunc createPlugin = (CreatePluginFunc)GetProcAddress(hMod, "Create");
if (!createPlugin) {
ANS_DBG("ANSCUSTOM_CreateDet", "ERROR GetProcAddress(Create) failed hMod=%p — FreeLibrary",
(void*)hMod);
this->_logger.LogFatal("ANSCUSTOMDETECTOR::Initialize.", "Cannot load Create", __FILE__, __LINE__);
FreeLibrary(hMod);
hMod = nullptr;
return false;
}
_customDetector = std::unique_ptr<IANSCustomClass>(createPlugin());
if (!_customDetector) {
ANS_DBG("ANSCUSTOM_CreateDet", "ERROR createPlugin returned null hMod=%p — FreeLibrary",
(void*)hMod);
this->_logger.LogFatal("ANSCUSTOMDETECTOR::Initialize", "Cannot create custom detector", __FILE__, __LINE__);
FreeLibrary(hMod);
hMod = nullptr;
return false;
}
ANS_DBG("ANSCUSTOM_CreateDet", "SUCCESS this=%p detector=%p hMod=%p",
(void*)this, (void*)_customDetector.get(), (void*)hMod);
return true;
}
catch (std::exception& e) {
ANS_DBG("ANSCUSTOM_CreateDet", "ERROR exception this=%p: %s", (void*)this, e.what());
this->_logger.LogFatal("ANSCUSTOMDETECTOR::Initialize", e.what(), __FILE__, __LINE__);
return false;
}
}
bool ANSCUSTOMDETECTOR::Initialize(std::string licenseKey, ModelConfig modelConfig, const std::string& modelZipFilePath, const std::string& modelZipPassword, std::string& labelMap) {
ANS_DBG("ANSCUSTOM_Initialize", "ENTRY this=%p zip=%s",
(void*)this, modelZipFilePath.c_str());
std::lock_guard<std::recursive_mutex> lock(_mutex);
ModelLoadingGuard mlg(_modelLoading);
try {
bool result = ANSODBase::Initialize(licenseKey, modelConfig, modelZipFilePath, modelZipPassword, labelMap);
if (!result) return false;
if (!result) {
ANS_DBG("ANSCUSTOM_Initialize", "ERROR base Initialize failed this=%p zip=%s",
(void*)this, modelZipFilePath.c_str());
return false;
}
_modelConfig = modelConfig;
if (_modelConfig.modelMNSThreshold < 0.2)
_modelConfig.modelMNSThreshold = 0.5;
@@ -257,12 +385,23 @@ namespace ANSCENTER
labelMap.clear();
//1. Create custom detector
CreateCustomDetector();
if (!_customDetector) {
ANS_DBG("ANSCUSTOM_Initialize", "ERROR _customDetector null after Create this=%p", (void*)this);
_isInitialized = false;
return false;
}
//2. Load initial model
_customDetector->SetLoadEngineOnCreate(this->_loadEngineOnCreation);
_isInitialized = _customDetector->Initialize(_modelFolder, _modelConfig.detectionScoreThreshold,labelMap);
ANS_DBG("ANSCUSTOM_Initialize", "%s this=%p folder=%s thr=%.3f loadOnCreate=%d",
_isInitialized ? "SUCCESS" : "FAILED",
(void*)this, _modelFolder.c_str(),
_modelConfig.detectionScoreThreshold,
(int)this->_loadEngineOnCreation);
return _isInitialized;
}
catch (std::exception& e) {
ANS_DBG("ANSCUSTOM_Initialize", "ERROR exception this=%p: %s", (void*)this, e.what());
this->_logger.LogFatal("ANSCUSTOMDETECTOR::Initialize", e.what(), __FILE__, __LINE__);
return false;
}
@@ -271,24 +410,42 @@ namespace ANSCENTER
return RunInference(input, "CustomCam");
}
std::vector<Object> ANSCUSTOMDETECTOR::RunInference(const cv::Mat& input,const std::string& camera_id) {
if (!PreInferenceCheck("ANSCUSTOMDETECTOR::RunInference")) return {};
ANS_DBG("ANSCUSTOM_Infer", "ENTRY this=%p cam=%s in=%dx%d type=%d",
(void*)this, camera_id.c_str(), input.cols, input.rows, input.type());
if (!PreInferenceCheck("ANSCUSTOMDETECTOR::RunInference")) {
ANS_DBG("ANSCUSTOM_Infer", "EARLY_OUT preInferenceCheck failed this=%p cam=%s",
(void*)this, camera_id.c_str());
return {};
}
std::vector<Object> result;
try {
// Early returns - good practice maintained
if (!_licenseValid) {
ANS_DBG("ANSCUSTOM_Infer", "ERROR invalid license this=%p cam=%s",
(void*)this, camera_id.c_str());
this->_logger.LogError("ANSCUSTOMDETECTOR::RunInference",
"Invalid License", __FILE__, __LINE__);
return result;
}
if (!_isInitialized) {
ANS_DBG("ANSCUSTOM_Infer", "ERROR not initialized this=%p cam=%s",
(void*)this, camera_id.c_str());
this->_logger.LogError("ANSCUSTOMDETECTOR::RunInference",
"Model is not initialized", __FILE__, __LINE__);
return result;
}
if (input.empty() || input.cols < 10 || input.rows < 10) {
ANS_DBG("ANSCUSTOM_Infer", "EARLY_OUT input too small this=%p cam=%s %dx%d",
(void*)this, camera_id.c_str(), input.cols, input.rows);
return result;
}
if (!_customDetector) {
ANS_DBG("ANSCUSTOM_Infer", "ERROR _customDetector null this=%p cam=%s",
(void*)this, camera_id.c_str());
return result;
}
@@ -320,9 +477,14 @@ namespace ANSCENTER
if (_stabilizationEnabled) result = StabilizeDetections(result, camera_id);
}
ANS_DBG("ANSCUSTOM_Infer", "DONE this=%p cam=%s det=%zu tracked=%d stab=%d",
(void*)this, camera_id.c_str(), result.size(),
(int)_trackerEnabled, (int)_stabilizationEnabled);
return result; // RVO applies, no copy
}
catch (const std::exception& e) { // Catch by const reference
ANS_DBG("ANSCUSTOM_Infer", "ERROR exception this=%p cam=%s: %s",
(void*)this, camera_id.c_str(), e.what());
this->_logger.LogFatal("ANSCUSTOMDETECTOR::RunInference",
e.what(), __FILE__, __LINE__);
return {}; // Return empty vector
@@ -330,26 +492,43 @@ namespace ANSCENTER
}
ANSCUSTOMDETECTOR::~ANSCUSTOMDETECTOR() {
ANS_DBG("ANSCUSTOM_Dtor", "ENTRY this=%p detector=%p hMod=%p",
(void*)this, (void*)_customDetector.get(), (void*)hMod);
try {
//this->_logger.LogDebug("ANSCUSTOMDETECTOR::~ANSCUSTOMDETECTOR()", "Release ANSCUSTOMDETECTOR ", __FILE__, __LINE__);
// Auto-release plugin object before unloading its DLL.
// Destroy() itself swallows exceptions, but wrap defensively
// so the destructor never throws.
Destroy();
}
catch (std::exception& e) {
catch (const std::exception& e) {
ANS_DBG("ANSCUSTOM_Dtor", "ERROR exception this=%p: %s", (void*)this, e.what());
this->_logger.LogFatal("ANSCUSTOMDETECTOR::~ANSCUSTOMDETECTOR()", e.what(), __FILE__, __LINE__);
}
catch (...) {
ANS_DBG("ANSCUSTOM_Dtor", "ERROR unknown exception this=%p", (void*)this);
}
ANS_DBG("ANSCUSTOM_Dtor", "EXIT this=%p detector=%p hMod=%p",
(void*)this, (void*)_customDetector.get(), (void*)hMod);
}
bool ANSCUSTOMDETECTOR::Destroy() {
ANS_DBG("ANSCUSTOM_Destroy", "ENTRY this=%p detector=%p hMod=%p",
(void*)this, (void*)_customDetector.get(), (void*)hMod);
try {
if (_customDetector) {
ANS_DBG("ANSCUSTOM_Destroy", "DESTROY detector=%p", (void*)_customDetector.get());
_customDetector->Destroy();
_customDetector.reset();
}
if (hMod) {
ANS_DBG("ANSCUSTOM_Destroy", "FREE_LIBRARY hMod=%p", (void*)hMod);
FreeLibrary(hMod);
hMod = nullptr;
}
ANS_DBG("ANSCUSTOM_Destroy", "SUCCESS this=%p", (void*)this);
return true;
}
catch (std::exception& e) {
ANS_DBG("ANSCUSTOM_Destroy", "ERROR exception this=%p: %s", (void*)this, e.what());
this->_logger.LogFatal("ANSCUSTOMDETECTOR::Destroy", e.what(), __FILE__, __LINE__);
return false;
}
+1 -1
View File
@@ -24,7 +24,7 @@ namespace ANSCENTER {
std::unique_ptr<IANSCustomClass> _customDetector;
HMODULE hMod= nullptr;
std::string FindANSCustomLibraryName(const std::string& folderPath, const std::string& keyword);
bool CreateCustomDetector();
bool CreateCustomDetector();
};
}
#endif
+51
View File
@@ -36,6 +36,57 @@ const int MAX_MISSING_SCREEN = 1000;
namespace ANSCENTER
{
// ────────────────────────────────────────────────────────────────────────
// DirectML fatal-error detection
//
// AMD's DirectML EP can fail in several distinct ways that all leave the
// Ort::Session permanently broken and require a fresh CPU-fallback session:
//
// 887A0005 DXGI_ERROR_DEVICE_REMOVED — adapter killed (RDP attach,
// TDR, sleep/wake, driver swap). GetDeviceRemovedReason gives
// the sub-cause but the device is gone either way.
// 80004005 E_FAIL — generic D3D12 / DML failure. Seen from
// DmlGraphFusionHelper::ExecuteOperator and
// DmlCommandRecorder::ExecuteCommandList on AMD when the
// driver couldn't compile/execute a fused subgraph or when
// TDR fires mid-command-list. No structured sub-cause
// available — must treat as device-fatal.
//
// Branding strings ("DmlFusedNode", "DmlGraphFusionHelper",
// "DmlCommandRecorder") catch future failure modes that surface the same
// way without us having to chase each new HRESULT.
//
// Used by every catch site that wraps Ort::Session::Run on AMD DML.
// Centralised here so a new failure mode is handled once, not 4+ times.
// ────────────────────────────────────────────────────────────────────────
inline bool IsDmlFatalError(const std::string& msg) {
return msg.find("887A0005") != std::string::npos
|| msg.find("80004005") != std::string::npos
|| msg.find("DmlFusedNode") != std::string::npos
|| msg.find("DmlGraphFusionHelper") != std::string::npos
|| msg.find("DmlCommandRecorder") != std::string::npos
|| msg.find("DmlExecutionProvider") != std::string::npos;
}
// ────────────────────────────────────────────────────────────────────────
// IsDmlDeviceRemoved — narrowly checks ONLY 887A0005
// (DXGI_ERROR_DEVICE_REMOVED). This is the truly unrecoverable case:
// the D3D12 adapter is gone (driver crashed, RDP attach killed the
// device, sleep/wake torched the WDDM context). No amount of session
// recreation will bring it back — only a process restart or CPU
// fallback can recover.
//
// In contrast, 80004005 (E_FAIL) from DmlGraphFusionHelper /
// DmlCommandRecorder is intermittent on AMD when the driver chokes on
// a fused subgraph or TDR fires mid-command-list. A fresh DML
// session typically clears it and resumes GPU inference — that's the
// stay-on-GPU recovery path used by ANSONNXYOLO's catch blocks.
// ────────────────────────────────────────────────────────────────────────
inline bool IsDmlDeviceRemoved(const std::string& msg) {
return msg.find("887A0005") != std::string::npos
|| msg.find("DXGI_ERROR_DEVICE_REMOVED") != std::string::npos;
}
template <typename T>
typename std::enable_if<std::is_arithmetic<T>::value, T>::type
inline clamp(const T& value, const T& low, const T& high)
+112 -22
View File
@@ -418,10 +418,24 @@ namespace ANSCENTER
}
}
const bool prevTracker = _trackerEnabled;
const bool prevStab = _stabilizationEnabled;
_trackerEnabled = enabled;
_trackerType = trackerType;
if (!enabled) return true;
if (!enabled) {
// Invariant: stabilization requires the tracker. Disabling the
// tracker forces stabilization off so the flag does not lie —
// otherwise StabilizeDetections is dead code (gated on
// _trackerEnabled in the inference path) but introspection
// would still report stabilization as on.
_stabilizationEnabled = false;
ANS_DBG("ANSOD_StabState",
"this=%p SetTracker(false) tracker=%d->0 stab=%d->0",
(void*)this, (int)prevTracker, (int)prevStab);
return true;
}
// Store config so per-camera trackers can be created lazily
_trackerMotType = 1; // default BYTETRACK
@@ -447,10 +461,14 @@ namespace ANSCENTER
}
}
// Auto-enable stabilization with defaults when tracker is enabled
if (!_stabilizationEnabled) {
_stabilizationEnabled = true;
}
// Note: stabilization is NOT auto-enabled here. Tracker can run
// without stabilization (raw trackId assignment, no ghost
// injection / EMA / hysteresis). Callers that want stabilization
// must opt in explicitly via SetStabilization(true), which
// auto-enables the tracker if needed.
ANS_DBG("ANSOD_StabState",
"this=%p SetTracker(true,type=%d) tracker=%d->1 stab=%d (unchanged)",
(void*)this, (int)trackerType, (int)prevTracker, (int)prevStab);
return true;
}
@@ -463,8 +481,13 @@ namespace ANSCENTER
bool ANSODBase::SetStabilization(bool enabled, int historySize, int maxMisses) {
std::lock_guard<std::recursive_mutex> lock(_mutex);
try {
const bool prevTracker = _trackerEnabled;
const bool prevStab = _stabilizationEnabled;
if (enabled) {
// Auto-enable tracker if not already enabled
// Invariant: stabilization requires tracker. Auto-enable
// tracker if currently off so the flag is meaningful at
// runtime (StabilizeDetections is gated on _trackerEnabled).
if (!_trackerEnabled) {
if (!SetTracker(_trackerType, true)) {
_logger.LogError("ANSODBase::SetStabilization",
@@ -476,9 +499,18 @@ namespace ANSCENTER
_stabilizationQueueSize = static_cast<size_t>(std::max(historySize, 5));
_stabilizationMaxConsecutiveMisses = std::max(maxMisses, 1);
QUEUE_SIZE = _stabilizationQueueSize;
ANS_DBG("ANSOD_StabState",
"this=%p SetStabilization(true,hist=%d,miss=%d) tracker=%d->%d stab=%d->1",
(void*)this, historySize, maxMisses,
(int)prevTracker, (int)_trackerEnabled, (int)prevStab);
}
else {
// Disabling stabilization leaves the tracker untouched —
// tracker is independently useful (raw trackId assignment).
_stabilizationEnabled = false;
ANS_DBG("ANSOD_StabState",
"this=%p SetStabilization(false) tracker=%d (unchanged) stab=%d->0",
(void*)this, (int)prevTracker, (int)prevStab);
}
return true;
}
@@ -562,18 +594,52 @@ namespace ANSCENTER
}
}
// Step 1: Convert Object[] → TrackerObject[]
// Adaptive padding: tiny objects produce near-zero inter-frame IoU,
// so ByteTrack/IoU matching drops them. We pad small boxes only
// when feeding the tracker; detections[i].box is never modified
// and the un-padded size is recovered for ghost rendering.
// Thresholds are internal by design — no user-facing knob.
constexpr int SMALL_THRESHOLD_PX = 32; // below this: full pad
constexpr int LARGE_THRESHOLD_PX = 96; // at/above this: no pad
constexpr float MAX_PAD_RATIO = 0.5f; // applied to smallest
auto computePadRatio = [](int maxDim) -> float {
if (maxDim >= LARGE_THRESHOLD_PX) return 0.0f;
if (maxDim <= SMALL_THRESHOLD_PX) return MAX_PAD_RATIO;
float t = static_cast<float>(LARGE_THRESHOLD_PX - maxDim) /
static_cast<float>(LARGE_THRESHOLD_PX - SMALL_THRESHOLD_PX);
return MAX_PAD_RATIO * t;
};
auto padBox = [](const cv::Rect& b, float r) -> cv::Rect {
if (r <= 0.0f) return b;
int pw = static_cast<int>(b.width * r);
int ph = static_cast<int>(b.height * r);
return cv::Rect(b.x - pw, b.y - ph,
b.width + 2 * pw, b.height + 2 * ph);
};
// Step 1: Convert Object[] → TrackerObject[] using adaptive padding.
// paddedBoxes[i] mirrors detections[i] so step-3 matching happens
// in padded space without altering the original detection boxes.
std::vector<TrackerObject> trackerInput;
std::vector<cv::Rect> paddedBoxes;
trackerInput.reserve(detections.size());
paddedBoxes.reserve(detections.size());
int paddedCount = 0;
for (const auto& obj : detections) {
int maxDim = std::max(obj.box.width, obj.box.height);
float ratio = computePadRatio(maxDim);
cv::Rect padded = padBox(obj.box, ratio);
paddedBoxes.push_back(padded);
if (ratio > 0.0f) ++paddedCount;
TrackerObject tobj{};
tobj.track_id = 0;
tobj.class_id = obj.classId;
tobj.prob = obj.confidence;
tobj.x = static_cast<float>(obj.box.x);
tobj.y = static_cast<float>(obj.box.y);
tobj.width = static_cast<float>(obj.box.width);
tobj.height = static_cast<float>(obj.box.height);
tobj.x = static_cast<float>(padded.x);
tobj.y = static_cast<float>(padded.y);
tobj.width = static_cast<float>(padded.width);
tobj.height = static_cast<float>(padded.height);
tobj.left = tobj.x;
tobj.top = tobj.y;
tobj.right = tobj.x + tobj.width;
@@ -581,20 +647,26 @@ namespace ANSCENTER
tobj.object_id = camera_id;
trackerInput.push_back(tobj);
}
ANS_DBG("ANSOD_Tracker",
"this=%p cam=%s ApplyTracking dets=%d padded=%d",
(void*)this, camera_id.c_str(),
(int)detections.size(), paddedCount);
// Step 2: Run per-camera tracker
// Step 2: Run per-camera tracker (operates in padded space)
auto trackerOutput = camera._tracker->UpdateTracker(0, trackerInput);
// Step 3: IoU-match tracked objects back to detections, assign trackId
// Step 3: IoU-match tracker output to detections in padded space,
// then assign trackId to the matching detection. detections[i].box
// remains the original un-padded box.
for (const auto& tobj : trackerOutput) {
cv::Rect trkBox(static_cast<int>(tobj.x), static_cast<int>(tobj.y),
static_cast<int>(tobj.width), static_cast<int>(tobj.height));
float bestIoU = 0.f;
int bestIdx = -1;
for (int i = 0; i < static_cast<int>(detections.size()); ++i) {
cv::Rect inter = trkBox & detections[i].box;
for (int i = 0; i < static_cast<int>(paddedBoxes.size()); ++i) {
cv::Rect inter = trkBox & paddedBoxes[i];
float interArea = static_cast<float>(inter.area());
float unionArea = static_cast<float>(trkBox.area() + detections[i].box.area()) - interArea;
float unionArea = static_cast<float>(trkBox.area() + paddedBoxes[i].area()) - interArea;
float iou = (unionArea > 0.f) ? interArea / unionArea : 0.f;
if (iou > bestIoU) {
bestIoU = iou;
@@ -606,15 +678,16 @@ namespace ANSCENTER
}
}
// Step 4: If stabilization enabled, inject ghost objects for unmatched tracker predictions
// Step 4: If stabilization enabled, inject ghost objects for unmatched
// tracker predictions. Ghost.box uses the last-known un-padded size
// from track history at the tracker's predicted center — so users
// never see the inflated padded box that lives inside the tracker.
if (_stabilizationEnabled) {
// Collect trackIds already assigned to real detections
std::set<int> matchedTrackIds;
for (const auto& det : detections) {
if (det.trackId > 0) matchedTrackIds.insert(det.trackId);
}
// Tracker-predicted objects not matched to any detection are "ghosts"
for (const auto& tobj : trackerOutput) {
if (tobj.track_id <= 0) continue;
if (matchedTrackIds.count(tobj.track_id) > 0) continue;
@@ -623,9 +696,26 @@ namespace ANSCENTER
ghost.trackId = tobj.track_id;
ghost.classId = tobj.class_id;
ghost.confidence = tobj.prob;
ghost.box = cv::Rect(
static_cast<int>(tobj.x), static_cast<int>(tobj.y),
static_cast<int>(tobj.width), static_cast<int>(tobj.height));
// Prefer history's un-padded size; fall back to tracker box
// only when no history exists (brand-new track that ghosts
// before any confirmed real detection — rare edge case).
auto histIt = camera._trackHistories.find(tobj.track_id);
if (histIt != camera._trackHistories.end() &&
histIt->second.lastBox.width > 0 &&
histIt->second.lastBox.height > 0) {
const cv::Rect& last = histIt->second.lastBox;
int cx = static_cast<int>(tobj.x + tobj.width * 0.5f);
int cy = static_cast<int>(tobj.y + tobj.height * 0.5f);
ghost.box = cv::Rect(cx - last.width / 2,
cy - last.height / 2,
last.width, last.height);
}
else {
ghost.box = cv::Rect(
static_cast<int>(tobj.x), static_cast<int>(tobj.y),
static_cast<int>(tobj.width), static_cast<int>(tobj.height));
}
ghost.cameraId = camera_id;
TagStabilized(ghost.extraInfo);
detections.push_back(ghost);
+171 -201
View File
@@ -1,6 +1,8 @@
#include"ANSONNXCL.h"
#include "ANSONNXYOLO.h" // ANSONNXYOLO is the delegate target — see facade comment in ANSONNXCL.h
#include "EPLoader.h"
#include "OpenVINODeviceConfig.h"
#include <typeinfo> // typeid(e).name() in ANS_DBG diagnostics
namespace ANSCENTER
{
@@ -574,24 +576,41 @@ namespace ANSCENTER
return false;
}
}
std::vector<Object> ANSONNXCL::postprocess(const std::vector<Ort::Value>& outputTensors, const std::string& camera_id) {
std::vector<Object> ANSONNXCL::postprocess(std::vector<Ort::Value>& outputTensors, const std::string& camera_id) {
ANS_DBG("ANSONNXCL_pp", "ENTRY tensors=%zu cam=%s this=%p",
outputTensors.size(), camera_id.c_str(), (void*)this);
std::lock_guard<std::recursive_mutex> lock(_mutex);
ANS_DBG("ANSONNXCL_pp", "lock acquired this=%p", (void*)this);
try {
std::vector<Object> output;
if (outputTensors.empty()) {
ANS_DBG("ANSONNXCL_pp", "EARLY-RETURN empty tensors");
this->_logger.LogError("ANSONNXCL::postprocess", "No output tensors", __FILE__, __LINE__);
return {};
}
const float* rawOutput = outputTensors[0].GetTensorData<float>();
ANS_DBG("ANSONNXCL_pp", "GetTensorMutableData<float>");
// GetTensorMutableData (not GetTensorData) on DirectML. The const
// GetTensorData triggers a per-call host-readable mapping that on
// AMD DML exhausts a small staging-buffer pool after ~8 calls and
// blocks indefinitely. GetTensorMutableData returns the existing
// host-accessible pointer directly with no per-call mapping cost.
// Same pattern used by every output-tensor read in ANSONNXYOLO
// and engines/ONNXEngine. Safe on all EPs (CUDA/OpenVINO/CPU);
// we read the data only, never mutate it.
const float* rawOutput = outputTensors[0].GetTensorMutableData<float>();
if (!rawOutput) {
ANS_DBG("ANSONNXCL_pp", "EARLY-RETURN rawOutput=null");
this->_logger.LogError("ANSONNXCL::postprocess", "rawOutput pointer is null", __FILE__, __LINE__);
return {};
}
ANS_DBG("ANSONNXCL_pp", "GetTensorTypeAndShapeInfo + GetShape");
const std::vector<int64_t> outputShape = outputTensors[0].GetTensorTypeAndShapeInfo().GetShape();
size_t numScores = vectorProduct(outputShape);
ANS_DBG("ANSONNXCL_pp", "shape.size=%zu numScores=%zu",
outputShape.size(), numScores);
std::ostringstream oss_shape;
oss_shape << "Output tensor shape: [";
@@ -602,6 +621,8 @@ namespace ANSCENTER
DEBUG_PRINT(oss_shape.str());
// CRITICAL: Check for NaN/Inf in raw output
ANS_DBG("ANSONNXCL_pp", "NaN/Inf scan begin (limit=%zu)",
std::min(numScores, size_t(100)));
bool hasNaN = false;
for (size_t i = 0; i < std::min(numScores, size_t(100)); ++i) {
if (std::isnan(rawOutput[i]) || std::isinf(rawOutput[i])) {
@@ -614,6 +635,7 @@ namespace ANSCENTER
}
if (hasNaN) {
ANS_DBG("ANSONNXCL_pp", "EARLY-RETURN NaN/Inf");
this->_logger.LogError("ANSONNXCL::postprocess",
"Model produced NaN/Inf values - input may be corrupted or model is broken",
__FILE__, __LINE__);
@@ -621,7 +643,10 @@ namespace ANSCENTER
}
int currentNumClasses = numClasses_ > 0 ? numClasses_ : static_cast<int>(_classes.size());
ANS_DBG("ANSONNXCL_pp", "currentNumClasses=%d (numClasses_=%d _classes.size=%zu)",
currentNumClasses, numClasses_, _classes.size());
if (currentNumClasses <= 0) {
ANS_DBG("ANSONNXCL_pp", "EARLY-RETURN no classes");
this->_logger.LogError("ANSONNXCL::postprocess", "No valid number of classes", __FILE__, __LINE__);
return {};
}
@@ -635,6 +660,7 @@ namespace ANSCENTER
DEBUG_PRINT(oss_scores.str());
// Initialize scores with zeros
ANS_DBG("ANSONNXCL_pp", "alloc scores vec n=%d", currentNumClasses);
std::vector<float> scores(currentNumClasses, 0.0f);
int validScores = 0;
@@ -658,9 +684,12 @@ namespace ANSCENTER
}
}
else {
ANS_DBG("ANSONNXCL_pp", "EARLY-RETURN unsupported shape size=%zu",
outputShape.size());
this->_logger.LogError("ANSONNXCL::postprocess", "Unsupported output shape", __FILE__, __LINE__);
return {};
}
ANS_DBG("ANSONNXCL_pp", "scores filled validScores=%d", validScores);
// Find maximum score
int bestClassId = -1;
@@ -674,9 +703,11 @@ namespace ANSCENTER
}
if (bestClassId == -1) {
ANS_DBG("ANSONNXCL_pp", "EARLY-RETURN bestClassId=-1");
this->_logger.LogError("ANSONNXCL::postprocess", "Could not determine best class ID", __FILE__, __LINE__);
return {};
}
ANS_DBG("ANSONNXCL_pp", "argmax bestClassId=%d maxScore=%g", bestClassId, maxScore);
// Check if output is already a probability distribution (sums to ~1.0).
// Some ONNX models include a Softmax layer in the graph; applying
@@ -689,6 +720,8 @@ namespace ANSCENTER
}
const bool alreadyNormalized = (allNonNeg && rawSum > 0.9f && rawSum < 1.1f);
ANS_DBG("ANSONNXCL_pp", "alloc probabilities vec n=%d normalized=%d",
currentNumClasses, (int)alreadyNormalized);
std::vector<float> probabilities(currentNumClasses, 0.0f);
float confidence = 0.0f;
if (alreadyNormalized) {
@@ -704,9 +737,11 @@ namespace ANSCENTER
}
confidence = sumExp > 0 ? probabilities[bestClassId] / sumExp : 0.0f;
}
ANS_DBG("ANSONNXCL_pp", "softmax done confidence=%g", confidence);
// CRITICAL: Validate final confidence
if (std::isnan(confidence) || std::isinf(confidence)) {
ANS_DBG("ANSONNXCL_pp", "EARLY-RETURN confidence NaN/Inf");
this->_logger.LogError("ANSONNXCL::postprocess",
"Final confidence is NaN/Inf: " + std::to_string(confidence),
__FILE__, __LINE__);
@@ -720,6 +755,8 @@ namespace ANSCENTER
else if (bestClassId >= 0) {
className = "ClassID_" + std::to_string(bestClassId);
}
ANS_DBG("ANSONNXCL_pp", "className resolved id=%d name=%s",
bestClassId, className.c_str());
Object detection;
if (m_imgWidth > 20 && m_imgHeight > 20) {
@@ -728,6 +765,8 @@ namespace ANSCENTER
else {
detection.box = cv::Rect(0, 0, m_imgWidth, m_imgHeight);
}
ANS_DBG("ANSONNXCL_pp", "RectToNormalizedPolygon w=%g h=%g",
m_imgWidth, m_imgHeight);
detection.polygon = ANSUtilityHelper::RectToNormalizedPolygon(detection.box, m_imgWidth, m_imgHeight);
detection.classId = bestClassId;
detection.className = className;
@@ -735,13 +774,21 @@ namespace ANSCENTER
detection.cameraId = camera_id;
output.push_back(detection);
ANS_DBG("ANSONNXCL_pp", "SUCCESS detections=%zu", output.size());
return output;
}
catch (const std::exception& e) {
ANS_DBG("ANSONNXCL_pp", "FATAL exception type=%s msg=%s",
typeid(e).name(), e.what());
this->_logger.LogFatal("ANSONNXCL::postprocess", e.what(), __FILE__, __LINE__);
return {};
}
}
catch (...) {
ANS_DBG("ANSONNXCL_pp", "FATAL unknown exception (non-std)");
this->_logger.LogFatal("ANSONNXCL::postprocess", "unknown non-std exception", __FILE__, __LINE__);
return {};
}
}
std::vector<Object> ANSONNXCL::classify(const cv::Mat& image, const std::string& camera_id) {
std::lock_guard<std::recursive_mutex> lock(_mutex);
@@ -830,9 +877,50 @@ namespace ANSCENTER
);
}
catch (const Ort::Exception& e) {
const std::string msg = e.what();
this->_logger.LogError("ANSONNXCL::classify",
"ONNX Runtime Exception during Run(): " + std::string(e.what()), __FILE__, __LINE__);
"ONNX Runtime Exception during Run(): " + msg, __FILE__, __LINE__);
delete[] blobPtr; // Clean up on error
blobPtr = nullptr;
// DML device-lost / DML graph-execution failure → tear down the
// dead DirectML session and rebuild on CPU. Without this, every
// subsequent classify() call hits the same dead session and
// floods the event log (16k+ entries observed on AMD/RDP).
if (IsDmlFatalError(msg) && !_dmlDeviceLost) {
_dmlDeviceLost = true;
_logger.LogFatal("ANSONNXCL::classify",
std::string("DirectML fatal error — recreating session on CPU. msg=")
+ msg.substr(0, 256),
__FILE__, __LINE__);
ANS_DBG("ANSONNXCL", "DML fatal — rebuild on CPU msg=%s", msg.c_str());
try {
// Order matters: drop AllocatedStringPtrs (which depend
// on the live session's allocator) BEFORE tearing down
// session_, sessionOptions_, env_. Init() will repopulate.
inputNodeNameAllocatedStrings_.clear();
outputNodeNameAllocatedStrings_.clear();
inputNames_.clear();
outputNames_.clear();
session_ = Ort::Session{ nullptr };
sessionOptions_ = Ort::SessionOptions{ nullptr };
env_ = Ort::Env{ nullptr };
if (Init(_modelFilePath, inputImageShape_, /*useGPU=*/false)) {
_logger.LogInfo("ANSONNXCL::classify",
"CPU fallback session created successfully",
__FILE__, __LINE__);
ANS_DBG("ANSONNXCL", "CPU fallback OK");
} else {
_logger.LogFatal("ANSONNXCL::classify",
"CPU fallback session creation failed",
__FILE__, __LINE__);
}
} catch (const std::exception& re) {
_logger.LogFatal("ANSONNXCL::classify",
std::string("CPU fallback exception: ") + re.what(),
__FILE__, __LINE__);
}
}
return {};
}
@@ -848,9 +936,17 @@ namespace ANSCENTER
// Postprocess
try {
return postprocess(outputTensors, camera_id);
ANS_DBG("ANSONNXCL_cls", "calling postprocess outputTensors=%zu",
outputTensors.size());
auto pp = postprocess(outputTensors, camera_id);
ANS_DBG("ANSONNXCL_cls", "postprocess returned detections=%zu", pp.size());
return pp;
}
catch (const std::exception& e) {
// If we land here, the throw escaped postprocess's own catch —
// most likely from its lock_guard line (outside its try block).
ANS_DBG("ANSONNXCL_cls", "postprocess THREW (escaped its catch) type=%s msg=%s",
typeid(e).name(), e.what());
this->_logger.LogError("ANSONNXCL::classify",
"Exception during postprocessing: " + std::string(e.what()), __FILE__, __LINE__);
return {};
@@ -868,216 +964,90 @@ namespace ANSCENTER
return {};
}
}
bool ANSONNXCL::OptimizeModel(bool fp16, std::string & optimizedModelFolder) {
if (!ANSODBase::OptimizeModel(fp16, optimizedModelFolder)) {
return false;
}
return true;
}
bool ANSONNXCL::Initialize(std::string licenseKey, ModelConfig modelConfig, const std::string& modelZipFilePath, const std::string& modelZipPassword, std::string& labelMap) {
std::lock_guard<std::recursive_mutex> lock(_mutex);
ModelLoadingGuard mlg(_modelLoading);
try {
_modelLoadValid = false;
bool result = ANSODBase::Initialize(licenseKey, modelConfig, modelZipFilePath, modelZipPassword, labelMap);
if (!result) return false;
// Parsing for YOLO only here
_modelConfig = modelConfig;
_modelConfig.detectionType = ANSCENTER::DetectionType::CLASSIFICATION;
_modelConfig.modelType = ModelType::ONNXCL;
_modelConfig.inpHeight = 224;
_modelConfig.inpWidth = 224;
if (_modelConfig.modelMNSThreshold < 0.2)
_modelConfig.modelMNSThreshold = 0.5;
if (_modelConfig.modelConfThreshold < 0.2)
_modelConfig.modelConfThreshold = 0.5;
if (_modelConfig.numKPS <= 0 || _modelConfig.numKPS > 133) // 133 = COCO wholebody max
_modelConfig.numKPS = 17;
if (_modelConfig.kpsThreshold == 0)_modelConfig.kpsThreshold = 0.5; // If not define
_fp16 = (modelConfig.precisionType == PrecisionType::FP16);
if (FileExist(_modelConfigFile)) {
ModelType modelType;
std::vector<int> inputShape;
_classes = ANSUtilityHelper::GetConfigFileContent(_modelConfigFile, modelType, inputShape);
if (inputShape.size() == 2) {
if (inputShape[0] > 0)_modelConfig.inpHeight = inputShape[0];
if (inputShape[1] > 0)_modelConfig.inpWidth = inputShape[1];
}
// ────────────────────────────────────────────────────────────────────────
// Public-method facade: delegate everything to the wrapped ANSONNXYOLO.
//
// This is a verification harness — ANSONNXYOLO already covers the
// classification path (auto-detected from model output shape) and ships
// with the safe DML config + DML-fatal-error recovery. By routing every
// ANSONNXCL public call through ANSONNXYOLO we can A/B test whether
// ANSONNXCL's own session/postprocess logic is the actual cause of the
// AMD same-process DML races we saw with the custom helmet model.
//
// The dead code below the facade (Init / preprocess / postprocess /
// classify / Ort::* members in the header) is intentionally preserved
// so the original implementation is one git-revert away if needed.
// ────────────────────────────────────────────────────────────────────────
namespace {
// Lazy-create the delegate on first public call. Not thread-safe in
// the strict sense, but the public methods all take _mutex first via
// the underlying ANSODBase, so concurrent first-time creation is
// serialised in practice.
inline ANSONNXYOLO& GetDelegate(std::unique_ptr<ANSONNXYOLO>& slot) {
if (!slot) {
slot = std::make_unique<ANSONNXYOLO>();
ANS_DBG("ANSONNXCL_Facade",
"delegate created (ANSONNXYOLO=%p)", (void*)slot.get());
}
else {// This is old version of model zip file
_modelFilePath = CreateFilePath(_modelFolder, "train_last.onnx");
_classFilePath = CreateFilePath(_modelFolder, "classes.names");
std::ifstream isValidFileName(_classFilePath);
if (!isValidFileName)
{
this->_logger.LogDebug("ANSONNXCL::Initialize. Load classes from string", _classFilePath, __FILE__, __LINE__);
LoadClassesFromString();
}
else {
this->_logger.LogDebug("ANSONNXCL::Initialize. Load classes from file", _classFilePath, __FILE__, __LINE__);
LoadClassesFromFile();
}
}
// 1. Load labelMap and engine
labelMap.clear();
if (!_classes.empty())
labelMap = VectorToCommaSeparatedString(_classes);
// 2. Initialize ONNX Runtime session
cv::Size targetInputShape= cv::Size(_modelConfig.inpWidth, _modelConfig.inpHeight);
result = Init(_modelFilePath, targetInputShape,true);
_modelLoadValid = true;
_isInitialized = true;
return result;
}
catch (const std::exception& e) {
this->_logger.LogFatal("ANSONNXCL::Initialize", e.what(), __FILE__, __LINE__);
return false;
return *slot;
}
}
bool ANSONNXCL::LoadModel(const std::string& modelZipFilePath, const std::string& modelZipPassword) {
std::lock_guard<std::recursive_mutex> lock(_mutex);
ModelLoadingGuard mlg(_modelLoading);
try {
bool result = ANSODBase::LoadModel(modelZipFilePath, modelZipPassword);
if (!result) return false;
_modelConfig.detectionType = ANSCENTER::DetectionType::CLASSIFICATION;
_modelConfig.modelType = ModelType::TENSORRT;
_modelConfig.inpHeight = 224;
_modelConfig.inpWidth = 224;
if (_modelConfig.modelMNSThreshold < 0.2)
_modelConfig.modelMNSThreshold = 0.5;
if (_modelConfig.modelConfThreshold < 0.2)
_modelConfig.modelConfThreshold = 0.5;
if (_modelConfig.numKPS <= 0 || _modelConfig.numKPS > 133) // 133 = COCO wholebody max
_modelConfig.numKPS = 17;
if (_modelConfig.kpsThreshold == 0)_modelConfig.kpsThreshold = 0.5; // If not define
// if (_modelConfig.precisionType == PrecisionType::FP16)_fp16 = true;
_fp16 = true; // Load Model from Here
// 0. Check if the configuration file exist
if (FileExist(_modelConfigFile)) {
ModelType modelType;
std::vector<int> inputShape;
_classes = ANSUtilityHelper::GetConfigFileContent(_modelConfigFile, modelType, inputShape);
if (inputShape.size() == 2) {
if (inputShape[0] > 0)_modelConfig.inpHeight = inputShape[0];
if (inputShape[1] > 0)_modelConfig.inpWidth = inputShape[1];
}
}
else {// This is old version of model zip file
_modelFilePath = CreateFilePath(_modelFolder, "train_last.onnx");
_classFilePath = CreateFilePath(_modelFolder, "classes.names");
std::ifstream isValidFileName(_classFilePath);
if (!isValidFileName)
{
this->_logger.LogDebug("ANSONNXCL::Initialize. Load classes from string", _classFilePath, __FILE__, __LINE__);
LoadClassesFromString();
}
else {
this->_logger.LogDebug("ANSONNXCL::Initialize. Load classes from file", _classFilePath, __FILE__, __LINE__);
LoadClassesFromFile();
}
}
// Initialize ONNX Runtime session
cv::Size targetInputShape = cv::Size(_modelConfig.inpWidth, _modelConfig.inpHeight);
result = Init(_modelFilePath, targetInputShape,true);
_modelLoadValid = true;
_isInitialized = true;
return result;
}
catch (const std::exception& e) {
this->_logger.LogFatal("ANSONNXCL::LoadModel", e.what(), __FILE__, __LINE__);
return false;
}
bool ANSONNXCL::OptimizeModel(bool fp16, std::string& optimizedModelFolder) {
ANS_DBG("ANSONNXCL_Facade", "OptimizeModel → ANSONNXYOLO");
return GetDelegate(_yoloDelegate).OptimizeModel(fp16, optimizedModelFolder);
}
bool ANSONNXCL::LoadModelFromFolder(std::string licenseKey, ModelConfig modelConfig, std::string modelName, std::string className, const std::string& modelFolder, std::string& labelMap) {
std::lock_guard<std::recursive_mutex> lock(_mutex);
ModelLoadingGuard mlg(_modelLoading);
try {
bool result = ANSODBase::LoadModelFromFolder(licenseKey, modelConfig, modelName, className, modelFolder, labelMap);
if (!result) return false;
std::string _modelName = modelName;
if (_modelName.empty()) {
_modelName = "train_last";
}
std::string modelFullName = _modelName + ".onnx";
// Parsing for YOLO only here
_modelConfig = modelConfig;
_modelConfig.detectionType = ANSCENTER::DetectionType::CLASSIFICATION;
_modelConfig.modelType = ModelType::TENSORRT;
_modelConfig.inpHeight = 224;
_modelConfig.inpWidth = 224;
if (_modelConfig.modelMNSThreshold < 0.2)
_modelConfig.modelMNSThreshold = 0.5;
if (_modelConfig.modelConfThreshold < 0.2)
_modelConfig.modelConfThreshold = 0.5;
if (_modelConfig.numKPS <= 0 || _modelConfig.numKPS > 133) // 133 = COCO wholebody max
_modelConfig.numKPS = 17;
if (_modelConfig.kpsThreshold == 0)_modelConfig.kpsThreshold = 0.5; // If not define
_fp16 = true; // Load Model from Here
// 0. Check if the configuration file exist
if (FileExist(_modelConfigFile)) {
ModelType modelType;
std::vector<int> inputShape;
_classes = ANSUtilityHelper::GetConfigFileContent(_modelConfigFile, modelType, inputShape);
if (inputShape.size() == 2) {
if (inputShape[0] > 0)_modelConfig.inpHeight = inputShape[0];
if (inputShape[1] > 0)_modelConfig.inpWidth = inputShape[1];
}
}
else {// This is old version of model zip file
_modelFilePath = CreateFilePath(_modelFolder, modelFullName);
_classFilePath = CreateFilePath(_modelFolder, className);
std::ifstream isValidFileName(_classFilePath);
if (!isValidFileName)
{
this->_logger.LogDebug("ANSONNXCL::Initialize. Load classes from string", _classFilePath, __FILE__, __LINE__);
LoadClassesFromString();
}
else {
this->_logger.LogDebug("ANSONNXCL::Initialize. Load classes from file", _classFilePath, __FILE__, __LINE__);
LoadClassesFromFile();
}
}
// 1. Load labelMap and engine
labelMap.clear();
if (!_classes.empty())
labelMap = VectorToCommaSeparatedString(_classes);
// 2. Initialize ONNX Runtime session
cv::Size targetInputShape = cv::Size(_modelConfig.inpWidth, _modelConfig.inpHeight);
result = Init(_modelFilePath, targetInputShape,true);
_modelLoadValid = true;
_isInitialized = true;
return result;
}
catch (const std::exception& e) {
this->_logger.LogFatal("ANSONNXCL::LoadModelFromFolder", e.what(), __FILE__, __LINE__);
return false;
}
bool ANSONNXCL::Initialize(std::string licenseKey, ModelConfig modelConfig,
const std::string& modelZipFilePath,
const std::string& modelZipPassword,
std::string& labelMap) {
ANS_DBG("ANSONNXCL_Facade", "Initialize → ANSONNXYOLO zip=%s",
modelZipFilePath.c_str());
return GetDelegate(_yoloDelegate).Initialize(
licenseKey, modelConfig, modelZipFilePath, modelZipPassword, labelMap);
}
bool ANSONNXCL::LoadModel(const std::string& modelZipFilePath,
const std::string& modelZipPassword) {
ANS_DBG("ANSONNXCL_Facade", "LoadModel → ANSONNXYOLO zip=%s",
modelZipFilePath.c_str());
return GetDelegate(_yoloDelegate).LoadModel(modelZipFilePath, modelZipPassword);
}
bool ANSONNXCL::LoadModelFromFolder(std::string licenseKey, ModelConfig modelConfig,
std::string modelName, std::string className,
const std::string& modelFolder,
std::string& labelMap) {
ANS_DBG("ANSONNXCL_Facade",
"LoadModelFromFolder → ANSONNXYOLO modelName=%s folder=%s",
modelName.c_str(), modelFolder.c_str());
return GetDelegate(_yoloDelegate).LoadModelFromFolder(
licenseKey, modelConfig, modelName, className, modelFolder, labelMap);
}
bool ANSONNXCL::Destroy() {
return true;
if (_yoloDelegate) {
ANS_DBG("ANSONNXCL_Facade", "Destroy → ANSONNXYOLO");
return _yoloDelegate->Destroy();
}
return true;
}
ANSONNXCL::~ANSONNXCL() {
// Explicit Destroy() so the delegate's session is torn down before
// the unique_ptr destructor frees the object. unique_ptr<ANSONNXYOLO>
// still releases the heap memory afterwards.
Destroy();
}
std::vector<Object> ANSONNXCL::RunInference(const cv::Mat& input, const std::string& camera_id) {
if (!PreInferenceCheck("ANSONNXCL::RunInference")) return {};
try {
std::vector<Object> result;
if (input.empty()) return result;
if ((input.cols < 5) || (input.rows < 5)) return result;
return classify(input, camera_id);
}
catch (const std::exception& e) {
this->_logger.LogFatal("ANSONNXCL::RunInference", e.what(), __FILE__, __LINE__);
return {};
}
std::vector<Object> ANSONNXCL::RunInference(const cv::Mat& input,
const std::string& camera_id) {
if (input.empty()) return {};
if (input.cols < 5 || input.rows < 5) return {};
return GetDelegate(_yoloDelegate).RunInference(input, camera_id);
}
std::vector<Object> ANSONNXCL::RunInference(const cv::Mat& inputImgBGR) {
return RunInference(inputImgBGR, "CustomCam");
}
+36 -1
View File
@@ -3,8 +3,26 @@
#pragma once
#include "ANSEngineCommon.h"
#include "engine.h"
#include <memory>
namespace ANSCENTER {
// Forward declaration: avoid pulling ANSONNXYOLO.h into every consumer of
// ANSONNXCL.h. Definition is included in ANSONNXCL.cpp.
class ANSONNXYOLO;
// ────────────────────────────────────────────────────────────────────────
// ANSONNXCL — verification facade over ANSONNXYOLO.
//
// ANSONNXYOLO already auto-detects classification models (single 2D
// output [B, nc]) and routes through its own classification postprocess.
// It also has the safe DML config (DisableMemPattern + ORT_SEQUENTIAL +
// EXTENDED) and the IsDmlFatalError catch-and-rebuild path.
//
// We delegate every public method here to an internal ANSONNXYOLO so we
// can A/B test whether ANSONNXCL's own session/postprocess code is the
// real cause of the AMD same-process DML races we saw with the custom
// helmet model. Public ABI is unchanged — external callers don't move.
// ────────────────────────────────────────────────────────────────────────
class ANSENGINE_API ANSONNXCL :public ANSODBase {
public:
virtual bool Initialize(std::string licenseKey, ModelConfig modelConfig, const std::string& modelZipFilePath, const std::string& modelZipPassword, std::string& labelMap) override;
@@ -15,10 +33,23 @@ namespace ANSCENTER {
std::vector<Object> RunInference(const cv::Mat& input, const std::string& camera_id);
bool Destroy();
~ANSONNXCL();
private:
// The wrapped engine. Lazy-initialised on first public call so we
// don't pay the cost (and pull in the dependency) for callers who
// never use this class.
std::unique_ptr<ANSONNXYOLO> _yoloDelegate;
private:
std::string _modelFilePath;
bool _modelLoadValid;
bool _fp16{ false };
// DML device-lost recovery: when DirectML fails fatally
// (HRESULT 887A0005 device-removed, 80004005 graph/command-recorder
// E_FAIL, etc. — see ANSCENTER::IsDmlFatalError) the Ort::Session
// is permanently broken. We detect once, recreate on CPU via
// Init(_modelFilePath, inputImageShape_, /*useGPU=*/false), and
// suppress further error-log flooding via this flag.
bool _dmlDeviceLost{ false };
size_t vectorProduct(const std::vector<int64_t>& vector);
bool preprocessImageToTensor(const cv::Mat& image, cv::Mat& outImage,
const cv::Size& targetShape,
@@ -28,7 +59,11 @@ namespace ANSCENTER {
void warmupModel();
bool Init(const std::string& modelPath, const cv::Size& targetInputShape, bool useGPU = true);
bool preprocess(const cv::Mat& image, float*& blob, std::vector<int64_t>& inputTensorShape);
std::vector<Object> postprocess(const std::vector<Ort::Value>& outputTensors, const std::string& camera_id);
// outputTensors is non-const because GetTensorMutableData() (the
// ORT API that doesn't hang on AMD DirectML) requires a non-const
// Ort::Value receiver. See comment at the GetTensorMutableData
// call site in postprocess() for the full rationale.
std::vector<Object> postprocess(std::vector<Ort::Value>& outputTensors, const std::string& camera_id);
std::vector<Object> classify(const cv::Mat& image, const std::string& camera_id);
private:
+5 -3
View File
@@ -1089,7 +1089,7 @@ namespace ANSCENTER {
std::vector<Object> ANSONNXOBB::postprocess(
const cv::Size& originalImageSize,
const cv::Size& resizedImageShape,
const std::vector<Ort::Value>& outputTensors,
std::vector<Ort::Value>& outputTensors,
int topk,
const std::string& camera_id)
{
@@ -1103,8 +1103,10 @@ namespace ANSCENTER {
return {};
}
// Extract output tensor data and shape [1, num_features, num_detections]
const float* rawOutput = outputTensors[0].GetTensorData<float>();
// Extract output tensor data and shape [1, num_features, num_detections].
// GetTensorMutableData (not GetTensorData) on DML — const variant
// hangs on AMD after ~8 calls. Read-only despite the name.
const float* rawOutput = outputTensors[0].GetTensorMutableData<float>();
const std::vector<int64_t> outputShape = outputTensors[0].GetTensorTypeAndShapeInfo().GetShape();
if (outputShape.size() < 3) {
+3 -1
View File
@@ -74,10 +74,12 @@ namespace ANSCENTER {
void warmupModel();
bool Init(const std::string& modelPath, bool useGPU=true, int deviceId = 0);
cv::Mat preprocess(const cv::Mat& image, float*& blob, std::vector<int64_t>& inputTensorShape);
// outputTensors is non-const because GetTensorMutableData() requires
// a non-const Ort::Value receiver — see ANSONNXCL.h for full note.
std::vector<Object> postprocess(
const cv::Size& originalImageSize,
const cv::Size& resizedImageShape,
const std::vector<Ort::Value>& outputTensors, int topk,
std::vector<Ort::Value>& outputTensors, int topk,
const std::string& camera_id);
std::vector<Object> detect(const cv::Mat& image, const std::string& camera_id);
private:
+4 -2
View File
@@ -759,7 +759,7 @@ namespace ANSCENTER {
std::vector<Object> ANSONNXPOSE::postprocess(
const cv::Size& originalImageSize,
const cv::Size& resizedImageShape,
const std::vector<Ort::Value>& outputTensors,
std::vector<Ort::Value>& outputTensors,
const std::string& camera_id)
{
std::lock_guard<std::recursive_mutex> lock(_mutex);
@@ -773,7 +773,9 @@ namespace ANSCENTER {
return {};
}
const float* rawOutput = outputTensors[0].GetTensorData<float>();
// GetTensorMutableData (not GetTensorData) on DML — const variant
// hangs on AMD after ~8 calls. Read-only despite the name.
const float* rawOutput = outputTensors[0].GetTensorMutableData<float>();
if (!rawOutput) {
this->_logger.LogError("ANSONNXPOSE::postprocess", "rawOutput pointer is null", __FILE__, __LINE__);
return {};
+3 -1
View File
@@ -41,8 +41,10 @@ namespace ANSCENTER {
void warmupModel();
bool Init(const std::string& modelPath, bool useGPU=true, int deviceId = 0);
cv::Mat preprocess(const cv::Mat& image, float*& blob, std::vector<int64_t>& inputTensorShape);
// outputTensors is non-const because GetTensorMutableData() requires
// a non-const Ort::Value receiver — see ANSONNXCL.h for full note.
std::vector<Object> postprocess(const cv::Size& originalImageSize, const cv::Size& resizedImageShape,
const std::vector<Ort::Value>& outputTensors, const std::string& camera_id);
std::vector<Ort::Value>& outputTensors, const std::string& camera_id);
std::vector<Object> detect(const cv::Mat& image, const std::string& camera_id);
private:
static std::atomic<int> instanceCounter_; // Thread-safe counter
+871
View File
@@ -0,0 +1,871 @@
#include "ANSONNXRTMPOSE.h"
#include "Utility.h"
#include "ANSLicense.h" // ANS_DBG macro for DebugView
#include <algorithm>
#include <cmath>
#include <chrono>
#include <sstream>
namespace {
// ── IoU dedup for end-to-end YOLO person detector output ──────────────
//
// Ultralytics E2E ONNX exports bake NMS into the graph, but its IoU
// threshold is set at export time (default ≈ 0.7) and we can't override
// it from outside. For a top-down pose pipeline that's too lax — pairs
// of overlapping bboxes survive and we end up running RTMPose twice on
// the same person.
//
// This filter walks the detector's output (already sorted by confidence
// in the E2E format), keeping each box only if its IoU with every
// already-kept box is below kPersonDedupIoU. O(N²) is fine because N is
// capped at 300 and typically <10 in practice.
constexpr float kPersonDedupIoU = 0.45f;
inline float boxIoU(const cv::Rect& a, const cv::Rect& b) {
const cv::Rect inter = a & b;
if (inter.area() <= 0) return 0.f;
const int uni = a.area() + b.area() - inter.area();
return uni > 0 ? static_cast<float>(inter.area()) / static_cast<float>(uni) : 0.f;
}
std::vector<ANSCENTER::Object> dedupByIoU(
const std::vector<ANSCENTER::Object>& persons, float iouThr) {
std::vector<ANSCENTER::Object> kept;
kept.reserve(persons.size());
for (const auto& p : persons) {
bool isDup = false;
for (const auto& k : kept) {
if (boxIoU(p.box, k.box) > iouThr) { isDup = true; break; }
}
if (!isDup) kept.push_back(p);
}
return kept;
}
} // anonymous namespace
namespace ANSCENTER {
// ====================================================================
// ONNXRtmPose — BasicOrtHandler subclass
// ====================================================================
ONNXRtmPose::ONNXRtmPose(const std::string& _onnx_path, unsigned int _num_threads)
: BasicOrtHandler(_onnx_path, _num_threads)
{
resolveInputShape();
}
ONNXRtmPose::ONNXRtmPose(const std::string& _onnx_path, EngineType engineType,
unsigned int _num_threads)
: BasicOrtHandler(_onnx_path, engineType, _num_threads)
{
resolveInputShape();
}
// ------------------------------------------------------------------
// resolveInputShape — read input/output dims off the model.
// Input : 1x3xHxW (RTMPose body-2d default 1x3x256x192)
// Output : simcc_x [1, K, ext_w], simcc_y [1, K, ext_h]
// The keypoint count K is read off the first output's dim[1].
// ------------------------------------------------------------------
void ONNXRtmPose::resolveInputShape() {
if (input_node_dims.size() >= 4) {
const int h = static_cast<int>(input_node_dims[2]);
const int w = static_cast<int>(input_node_dims[3]);
if (h > 0) m_input_h = h;
if (w > 0) m_input_w = w;
}
if (!output_node_dims.empty() && output_node_dims[0].size() >= 2) {
const int k = static_cast<int>(output_node_dims[0][1]);
if (k > 0) m_num_points = k;
}
ANS_DBG("ONNXRtmPose", "resolveInputShape: %dx%d kps=%d", m_input_w, m_input_h, m_num_points);
}
// ------------------------------------------------------------------
// GetAffineTransform — direct port of utils.h.
// ------------------------------------------------------------------
cv::Mat ONNXRtmPose::GetAffineTransform(float center_x, float center_y,
float scale_w, float scale_h,
int output_w, int output_h, bool inverse)
{
cv::Point2f src1(center_x, center_y);
cv::Point2f src2(center_x, center_y - scale_w * 0.5f);
cv::Point2f src3(src2.x - (src1.y - src2.y),
src2.y + (src1.x - src2.x));
const float dcx = output_w * 0.5f;
const float dcy = output_h * 0.5f;
cv::Point2f dst1(dcx, dcy);
cv::Point2f dst2(dcx, dcy - output_w * 0.5f);
cv::Point2f dst3(dst2.x - (dst1.y - dst2.y),
dst2.y + (dst1.x - dst2.x));
cv::Point2f srcPts[3] = { src1, src2, src3 };
cv::Point2f dstPts[3] = { dst1, dst2, dst3 };
return inverse ? cv::getAffineTransform(dstPts, srcPts)
: cv::getAffineTransform(srcPts, dstPts);
}
// ------------------------------------------------------------------
// CropImageByDetectBox — aspect-fit + 1.2x context expansion.
// Direct port of rtmpose_onnxruntime.cpp::CropImageByDetectBox.
// ------------------------------------------------------------------
void ONNXRtmPose::CropImageByDetectBox(const cv::Mat& image, const cv::Rect& box,
cv::Mat& crop, cv::Mat& affine_rev)
{
float w = static_cast<float>(box.width);
float h = static_cast<float>(box.height);
const float cx = box.x + w * 0.5f;
const float cy = box.y + h * 0.5f;
const float aspect = static_cast<float>(m_input_w) / static_cast<float>(m_input_h);
if (w > aspect * h) h = w / aspect;
else w = h * aspect;
const float scale_w = w * 1.2f;
const float scale_h = h * 1.2f;
cv::Mat affine = GetAffineTransform(cx, cy, scale_w, scale_h,
m_input_w, m_input_h, false);
affine_rev = GetAffineTransform(cx, cy, scale_w, scale_h,
m_input_w, m_input_h, true);
cv::warpAffine(image, crop, affine,
cv::Size(m_input_w, m_input_h), cv::INTER_LINEAR);
}
// ------------------------------------------------------------------
// transformInto — input is a 192x256 BGR person crop. Convert to
// RGB, ImageNet-normalize, lay out CHW into outBuffer, build the
// Ort::Value view over outBuffer.data(). Mirrors the design of
// ONNXYOLO::transformInto: per-call backing storage is owned by the
// caller so two threads sharing one ONNXRtmPose can preprocess and
// Run() concurrently without racing on input_values_handler.
// ------------------------------------------------------------------
Ort::Value ONNXRtmPose::transformInto(const cv::Mat& cropBGR,
std::vector<float>& outBuffer,
std::vector<int64_t>& outDims)
{
cv::Mat rgb;
cv::cvtColor(cropBGR, rgb, cv::COLOR_BGR2RGB);
cv::Mat floatImg;
rgb.convertTo(floatImg, CV_32FC3);
const int H = floatImg.rows;
const int W = floatImg.cols;
const size_t HW = static_cast<size_t>(H) * W;
outDims = { 1, 3, static_cast<int64_t>(H), static_cast<int64_t>(W) };
outBuffer.resize(3 * HW);
float* data = outBuffer.data();
// HWC -> CHW + (pixel - mean) / std. RGB channel order.
for (int y = 0; y < H; ++y) {
const float* row = floatImg.ptr<float>(y);
for (int x = 0; x < W; ++x) {
const float r = row[x * 3 + 0];
const float g = row[x * 3 + 1];
const float b = row[x * 3 + 2];
data[0 * HW + y * W + x] = (r - MEAN_R) / STD_R;
data[1 * HW + y * W + x] = (g - MEAN_G) / STD_G;
data[2 * HW + y * W + x] = (b - MEAN_B) / STD_B;
}
}
return Ort::Value::CreateTensor<float>(
*memory_info_handler,
outBuffer.data(),
outBuffer.size(),
outDims.data(),
outDims.size());
}
// BasicOrtHandler pure-virtual override. Thin shim that writes into
// the shared base members (legacy single-threaded caller path).
Ort::Value ONNXRtmPose::transform(const cv::Mat& mat)
{
Ort::Value v = transformInto(mat, input_values_handler, input_node_dims);
input_tensor_size = input_values_handler.size();
return v;
}
// RTMPose is single-person per call — batched inference doesn't apply.
Ort::Value ONNXRtmPose::transformBatch(const std::vector<cv::Mat>& /*images*/)
{
throw std::runtime_error("ONNXRtmPose::transformBatch: not supported (single-person model)");
}
// ------------------------------------------------------------------
// estimate — full per-bbox pipeline.
// ------------------------------------------------------------------
Object ONNXRtmPose::estimate(const cv::Mat& image, const cv::Rect& box)
{
Object obj{};
if (image.empty() || box.width <= 0 || box.height <= 0) {
ANS_DBG("ONNXRtmPose", "estimate: invalid input empty=%d box=%dx%d",
image.empty() ? 1 : 0, box.width, box.height);
return obj;
}
try {
// Affine crop -> 192x256 BGR person image, plus inverse-affine
// to map output coords back to original image.
cv::Mat crop, affine_reverse;
CropImageByDetectBox(image, box, crop, affine_reverse);
if (crop.empty()) {
ANS_DBG("ONNXRtmPose", "estimate: warpAffine produced empty crop");
return obj;
}
// Per-call backing storage so multiple threads sharing one
// ONNXRtmPose can Run() in parallel — see comment on
// transformInto.
std::vector<float> inputBuffer;
std::vector<int64_t> inputDims;
Ort::Value inputTensor = transformInto(crop, inputBuffer, inputDims);
auto outputTensors = ort_session->Run(
Ort::RunOptions{ nullptr },
input_node_names.data(),
&inputTensor, 1,
output_node_names.data(),
num_outputs);
if (outputTensors.size() < 2) {
ANS_DBG("ONNXRtmPose", "estimate: expected 2 outputs (simcc_x,simcc_y), got %zu",
outputTensors.size());
return obj;
}
// Decode SimCC. Disambiguate which output is X vs Y by length
// (X output's last dim is proportional to input W; Y's to H).
float* out0 = outputTensors[0].GetTensorMutableData<float>();
float* out1 = outputTensors[1].GetTensorMutableData<float>();
auto shape0 = outputTensors[0].GetTensorTypeAndShapeInfo().GetShape();
auto shape1 = outputTensors[1].GetTensorTypeAndShapeInfo().GetShape();
if (shape0.size() < 3 || shape1.size() < 3 ||
shape0[1] != shape1[1] || shape0[1] <= 0) {
ANS_DBG("ONNXRtmPose", "estimate: bad output shapes (rank or K mismatch)");
return obj;
}
const int joint_num = static_cast<int>(shape0[1]);
int W0 = static_cast<int>(shape0[2]);
int H1 = static_cast<int>(shape1[2]);
float* simcc_x = out0;
float* simcc_y = out1;
int extW = W0;
int extH = H1;
// If the first output looks H-shaped (closer to 2*input_h than
// 2*input_w), swap so simcc_x always indexes the W-axis.
const float ratioFirstW = (m_input_w > 0) ? float(W0) / float(m_input_w) : 0.f;
const float ratioFirstH = (m_input_h > 0) ? float(W0) / float(m_input_h) : 0.f;
if (std::abs(ratioFirstW - 2.f) > std::abs(ratioFirstH - 2.f)) {
std::swap(simcc_x, simcc_y);
extW = H1;
extH = W0;
}
const double m0 = affine_reverse.at<double>(0, 0);
const double m1 = affine_reverse.at<double>(0, 1);
const double m2 = affine_reverse.at<double>(0, 2);
const double m3 = affine_reverse.at<double>(1, 0);
const double m4 = affine_reverse.at<double>(1, 1);
const double m5 = affine_reverse.at<double>(1, 2);
const float imgW = static_cast<float>(image.cols);
const float imgH = static_cast<float>(image.rows);
std::vector<float> kps;
kps.reserve(static_cast<size_t>(joint_num) * 3);
std::vector<cv::Point2f> polygon;
polygon.reserve(joint_num);
std::stringstream xss, yss;
float meanScore = 0.f;
int meanCount = 0;
for (int i = 0; i < joint_num; ++i) {
const float* x_ptr = simcc_x + static_cast<size_t>(i) * extW;
const float* y_ptr = simcc_y + static_cast<size_t>(i) * extH;
const int px = static_cast<int>(std::max_element(x_ptr, x_ptr + extW) - x_ptr);
const int py = static_cast<int>(std::max_element(y_ptr, y_ptr + extH) - y_ptr);
const float score = std::max(x_ptr[px], y_ptr[py]);
// SimCC oversampling factor 2 — same constant as the reference impl.
const double raw_x = px * 0.5;
const double raw_y = py * 0.5;
float wx = static_cast<float>(m0 * raw_x + m1 * raw_y + m2);
float wy = static_cast<float>(m3 * raw_x + m4 * raw_y + m5);
wx = std::clamp(wx, 0.f, imgW - 1.f);
wy = std::clamp(wy, 0.f, imgH - 1.f);
kps.push_back(wx);
kps.push_back(wy);
kps.push_back(score);
polygon.emplace_back(wx, wy);
xss << wx;
yss << wy;
if (i + 1 < joint_num) { xss << ";"; yss << ";"; }
if (score > 0.f) { meanScore += score; ++meanCount; }
}
obj.classId = 0;
obj.confidence = (meanCount > 0) ? meanScore / meanCount : 0.f;
obj.box = box;
obj.kps = std::move(kps);
obj.polygon = std::move(polygon);
obj.extraInfo = xss.str() + "|" + yss.str();
return obj;
}
catch (const std::exception& e) {
ANS_DBG("ONNXRtmPose", "estimate: EXCEPTION '%s'", e.what());
return obj;
}
}
// ====================================================================
// ANSONNXRTMPOSE — ANSODBase wrapper
// ====================================================================
ANSONNXRTMPOSE::~ANSONNXRTMPOSE() {
try { Destroy(); }
catch (const std::exception& e) {
_logger.LogError("ANSONNXRTMPOSE::~ANSONNXRTMPOSE", e.what(), __FILE__, __LINE__);
}
}
bool ANSONNXRTMPOSE::Destroy() {
try {
if (m_personDetector) {
m_personDetector->Destroy();
m_personDetector.reset();
}
m_ortEngine.reset();
ANS_DBG("ANSONNXRTMPOSE", "Destroy: engine released");
return true;
}
catch (const std::exception& e) {
_logger.LogError("ANSONNXRTMPOSE::Destroy", e.what(), __FILE__, __LINE__);
return false;
}
}
bool ANSONNXRTMPOSE::OptimizeModel(bool fp16, std::string& optimizedModelFolder) {
if (!ANSODBase::OptimizeModel(fp16, optimizedModelFolder)) return false;
// ONNX models don't need an explicit precision-conversion pass here;
// the EP picks FP16 / FP32 at session-creation time. Mirrors
// ANSONNXYOLO::OptimizeModel which is also a no-op beyond folder
// resolution.
optimizedModelFolder = _modelFolder;
// Also bring up the bundled person-detector if present so its ORT
// session is constructed (and any cached engine is materialized)
// alongside the pose model. Mirrors the TRT counterpart's behavior.
(void)TryLoadPersonDetector(_modelFolder);
return true;
}
bool ANSONNXRTMPOSE::InitOrtEngine() {
try {
if (!FileExist(_modelFilePath)) {
_logger.LogError("ANSONNXRTMPOSE::InitOrtEngine",
"Model file does not exist: " + _modelFilePath, __FILE__, __LINE__);
ANS_DBG("ANSONNXRTMPOSE", "InitOrtEngine: missing onnx '%s'", _modelFilePath.c_str());
return false;
}
m_ortEngine = std::make_shared<ONNXRtmPose>(_modelFilePath);
ANS_DBG("ANSONNXRTMPOSE", "InitOrtEngine: ready %dx%d kps=%d",
m_ortEngine->inputWidth(), m_ortEngine->inputHeight(),
m_ortEngine->numKeypoints());
return true;
}
catch (const std::exception& e) {
_logger.LogFatal("ANSONNXRTMPOSE::InitOrtEngine", e.what(), __FILE__, __LINE__);
ANS_DBG("ANSONNXRTMPOSE", "InitOrtEngine: EXCEPTION '%s'", e.what());
return false;
}
}
bool ANSONNXRTMPOSE::InitOrtEngine(ANSCENTER::EngineType engineType) {
try {
if (!FileExist(_modelFilePath)) {
_logger.LogError("ANSONNXRTMPOSE::InitOrtEngine",
"Model file does not exist: " + _modelFilePath, __FILE__, __LINE__);
return false;
}
m_ortEngine = std::make_shared<ONNXRtmPose>(_modelFilePath, engineType);
ANS_DBG("ANSONNXRTMPOSE", "InitOrtEngine(EP=%d): ready %dx%d kps=%d",
static_cast<int>(engineType),
m_ortEngine->inputWidth(), m_ortEngine->inputHeight(),
m_ortEngine->numKeypoints());
return true;
}
catch (const std::exception& e) {
_logger.LogFatal("ANSONNXRTMPOSE::InitOrtEngine", e.what(), __FILE__, __LINE__);
return false;
}
}
// ------------------------------------------------------------------
// WarmUpEngine — single dummy inference to amortise first-call cost
// (CUDA arena alloc / cuDNN algo selection / DML shader compile).
// Non-fatal on failure: the engine is fully loaded before this runs.
// ------------------------------------------------------------------
void ANSONNXRTMPOSE::WarmUpEngine() {
if (!m_ortEngine) return;
const auto et = m_ortEngine->getEngineType();
if (et != EngineType::NVIDIA_GPU && et != EngineType::AMD_GPU) {
ANS_DBG("ANSONNXRTMPOSE", "Warm-up skipped (CPU/OpenVINO EP)");
return;
}
try {
const int W = m_ortEngine->inputWidth();
const int H = m_ortEngine->inputHeight();
cv::Mat dummy(H, W, CV_8UC3, cv::Scalar(128, 128, 128));
const cv::Rect wholeImage(0, 0, W, H);
auto t0 = std::chrono::steady_clock::now();
(void)m_ortEngine->estimate(dummy, wholeImage);
auto t1 = std::chrono::steady_clock::now();
auto ms = std::chrono::duration_cast<std::chrono::milliseconds>(t1 - t0).count();
ANS_DBG("ANSONNXRTMPOSE", "Warm-up done: %lld ms (%dx%d)", (long long)ms, W, H);
}
catch (const std::exception& e) {
_logger.LogWarn("ANSONNXRTMPOSE::WarmUpEngine",
std::string("Warm-up skipped (non-fatal): ") + e.what(),
__FILE__, __LINE__);
ANS_DBG("ANSONNXRTMPOSE", "Warm-up: EXCEPTION '%s'", e.what());
}
catch (...) {
_logger.LogWarn("ANSONNXRTMPOSE::WarmUpEngine",
"Warm-up skipped (unknown exception)", __FILE__, __LINE__);
}
}
bool ANSONNXRTMPOSE::Initialize(std::string licenseKey, ModelConfig modelConfig,
const std::string& modelZipFilePath,
const std::string& modelZipPassword,
std::string& labelMap)
{
std::lock_guard<std::recursive_mutex> lock(_mutex);
ModelLoadingGuard mlg(_modelLoading);
ANS_DBG("ANSONNXRTMPOSE", "Initialize: zip='%s'", modelZipFilePath.c_str());
try {
_modelLoadValid = false;
bool result = ANSODBase::Initialize(licenseKey, modelConfig,
modelZipFilePath, modelZipPassword, labelMap);
if (!result) return false;
_modelConfig = modelConfig;
_modelConfig.modelType = ModelType::ONNXRTMPOSE;
if (_modelConfig.inpHeight <= 0) _modelConfig.inpHeight = 256;
if (_modelConfig.inpWidth <= 0) _modelConfig.inpWidth = 192;
if (_modelConfig.numKPS <= 0 || _modelConfig.numKPS > 133)
_modelConfig.numKPS = 26;
if (_modelConfig.kpsThreshold <= 0)
_modelConfig.kpsThreshold = KPS_THRESHOLD;
KPS_THRESHOLD = _modelConfig.kpsThreshold;
_modelFilePath = CreateFilePath(_modelFolder, "train_last.onnx");
if (FileExist(_modelConfigFile)) {
ModelType modelType;
std::vector<int> inputShape;
_classes = ANSUtilityHelper::GetConfigFileContent(_modelConfigFile, modelType, inputShape);
if (inputShape.size() == 2) {
if (inputShape[0] > 0) _modelConfig.inpHeight = inputShape[0];
if (inputShape[1] > 0) _modelConfig.inpWidth = inputShape[1];
}
}
else {
_classFilePath = CreateFilePath(_modelFolder, "classes.names");
std::ifstream isValid(_classFilePath);
if (!isValid) LoadClassesFromString();
else LoadClassesFromFile();
}
labelMap.clear();
if (!_classes.empty())
labelMap = VectorToCommaSeparatedString(_classes);
if (this->_loadEngineOnCreation) {
if (!InitOrtEngine()) {
_logger.LogError("ANSONNXRTMPOSE::Initialize",
"Failed to create ONNX Runtime engine: " + _modelFilePath,
__FILE__, __LINE__);
return false;
}
// Sync ModelConfig.numKPS to whatever the model actually has,
// so consumers reading GetModelConfig() see the truth.
if (m_ortEngine && m_ortEngine->numKeypoints() > 0)
_modelConfig.numKPS = m_ortEngine->numKeypoints();
WarmUpEngine();
}
// Top-down pipeline: try to load the bundled person detector.
// Falls back gracefully (whole-image pose) when the files are missing.
(void)TryLoadPersonDetector(_modelFolder);
_modelLoadValid = true;
_isInitialized = true;
return true;
}
catch (const std::exception& e) {
_logger.LogFatal("ANSONNXRTMPOSE::Initialize", e.what(), __FILE__, __LINE__);
ANS_DBG("ANSONNXRTMPOSE", "Initialize: EXCEPTION '%s'", e.what());
return false;
}
}
bool ANSONNXRTMPOSE::LoadModel(const std::string& modelZipFilePath,
const std::string& modelZipPassword)
{
std::lock_guard<std::recursive_mutex> lock(_mutex);
ModelLoadingGuard mlg(_modelLoading);
ANS_DBG("ANSONNXRTMPOSE", "LoadModel: zip='%s'", modelZipFilePath.c_str());
try {
bool result = ANSODBase::LoadModel(modelZipFilePath, modelZipPassword);
if (!result) return false;
_modelConfig.modelType = ModelType::ONNXRTMPOSE;
if (_modelConfig.inpHeight <= 0) _modelConfig.inpHeight = 256;
if (_modelConfig.inpWidth <= 0) _modelConfig.inpWidth = 192;
if (_modelConfig.numKPS <= 0 || _modelConfig.numKPS > 133)
_modelConfig.numKPS = 26;
if (_modelConfig.kpsThreshold <= 0)
_modelConfig.kpsThreshold = KPS_THRESHOLD;
KPS_THRESHOLD = _modelConfig.kpsThreshold;
_modelFilePath = CreateFilePath(_modelFolder, "train_last.onnx");
if (FileExist(_modelConfigFile)) {
ModelType modelType;
std::vector<int> inputShape;
_classes = ANSUtilityHelper::GetConfigFileContent(_modelConfigFile, modelType, inputShape);
if (inputShape.size() == 2) {
if (inputShape[0] > 0) _modelConfig.inpHeight = inputShape[0];
if (inputShape[1] > 0) _modelConfig.inpWidth = inputShape[1];
}
}
else {
_classFilePath = CreateFilePath(_modelFolder, "classes.names");
std::ifstream isValid(_classFilePath);
if (!isValid) LoadClassesFromString();
else LoadClassesFromFile();
}
if (this->_loadEngineOnCreation) {
if (!InitOrtEngine()) { _modelLoadValid = false; return false; }
if (m_ortEngine && m_ortEngine->numKeypoints() > 0)
_modelConfig.numKPS = m_ortEngine->numKeypoints();
WarmUpEngine();
}
(void)TryLoadPersonDetector(_modelFolder);
_modelLoadValid = true;
_isInitialized = true;
return true;
}
catch (const std::exception& e) {
_logger.LogFatal("ANSONNXRTMPOSE::LoadModel", e.what(), __FILE__, __LINE__);
ANS_DBG("ANSONNXRTMPOSE", "LoadModel: EXCEPTION '%s'", e.what());
return false;
}
}
bool ANSONNXRTMPOSE::LoadModelFromFolder(std::string licenseKey, ModelConfig modelConfig,
std::string modelName, std::string className,
const std::string& modelFolder,
std::string& labelMap)
{
std::lock_guard<std::recursive_mutex> lock(_mutex);
ModelLoadingGuard mlg(_modelLoading);
ANS_DBG("ANSONNXRTMPOSE", "LoadModelFromFolder: folder='%s' model='%s'",
modelFolder.c_str(), modelName.c_str());
try {
bool result = ANSODBase::LoadModelFromFolder(licenseKey, modelConfig,
modelName, className,
modelFolder, labelMap);
if (!result) return false;
_modelConfig = modelConfig;
_modelConfig.modelType = ModelType::ONNXRTMPOSE;
if (_modelConfig.inpHeight <= 0) _modelConfig.inpHeight = 256;
if (_modelConfig.inpWidth <= 0) _modelConfig.inpWidth = 192;
if (_modelConfig.numKPS <= 0 || _modelConfig.numKPS > 133)
_modelConfig.numKPS = 26;
if (_modelConfig.kpsThreshold <= 0)
_modelConfig.kpsThreshold = KPS_THRESHOLD;
KPS_THRESHOLD = _modelConfig.kpsThreshold;
std::string _modelName = modelName;
if (_modelName.empty()) _modelName = "train_last";
std::string modelFullName = _modelName + ".onnx";
_modelFilePath = CreateFilePath(_modelFolder, modelFullName);
if (FileExist(_modelConfigFile)) {
ModelType modelType;
std::vector<int> inputShape;
_classes = ANSUtilityHelper::GetConfigFileContent(_modelConfigFile, modelType, inputShape);
if (inputShape.size() == 2) {
if (inputShape[0] > 0) _modelConfig.inpHeight = inputShape[0];
if (inputShape[1] > 0) _modelConfig.inpWidth = inputShape[1];
}
}
else {
_classFilePath = CreateFilePath(_modelFolder, className);
std::ifstream isValid(_classFilePath);
if (!isValid) LoadClassesFromString();
else LoadClassesFromFile();
}
labelMap.clear();
if (!_classes.empty())
labelMap = VectorToCommaSeparatedString(_classes);
if (this->_loadEngineOnCreation) {
if (!InitOrtEngine()) { _modelLoadValid = false; return false; }
if (m_ortEngine && m_ortEngine->numKeypoints() > 0)
_modelConfig.numKPS = m_ortEngine->numKeypoints();
WarmUpEngine();
}
(void)TryLoadPersonDetector(_modelFolder);
_modelLoadValid = true;
_isInitialized = true;
return true;
}
catch (const std::exception& e) {
_logger.LogFatal("ANSONNXRTMPOSE::LoadModelFromFolder", e.what(), __FILE__, __LINE__);
ANS_DBG("ANSONNXRTMPOSE", "LoadModelFromFolder: EXCEPTION '%s'", e.what());
return false;
}
}
std::vector<Object> ANSONNXRTMPOSE::RunInference(const cv::Mat& inputImgBGR) {
return RunInference(inputImgBGR, "");
}
std::vector<Object> ANSONNXRTMPOSE::RunInference(const cv::Mat& inputImgBGR,
const std::string& camera_id)
{
if (_modelLoading.load()) return {};
{
auto lock = TryLockWithTimeout("ANSONNXRTMPOSE::RunInference");
if (!lock.owns_lock()) return {};
if (!_modelLoadValid) {
_logger.LogError("ANSONNXRTMPOSE::RunInference", "Model not loaded", __FILE__, __LINE__);
return {};
}
if (!_licenseValid) {
_logger.LogError("ANSONNXRTMPOSE::RunInference", "Invalid license", __FILE__, __LINE__);
return {};
}
if (!_isInitialized) {
_logger.LogError("ANSONNXRTMPOSE::RunInference", "Model not initialized", __FILE__, __LINE__);
return {};
}
if (inputImgBGR.empty() || inputImgBGR.cols < 10 || inputImgBGR.rows < 10)
return {};
}
try { return DetectObjects(inputImgBGR, camera_id); }
catch (const std::exception& e) {
_logger.LogFatal("ANSONNXRTMPOSE::RunInference", e.what(), __FILE__, __LINE__);
ANS_DBG("ANSONNXRTMPOSE", "RunInference: EXCEPTION '%s'", e.what());
return {};
}
}
// ------------------------------------------------------------------
// DetectObjects — two pipelines:
//
// (a) Top-down (preferred) — m_personDetector loaded from the
// bundled person.onnx + person.names. Detect people first,
// then run RTMPose ONCE per person bbox. This is what RTMPose
// is designed for; works on multi-person scenes with small /
// distant subjects.
//
// (b) Standalone fallback — no detector files in the model folder.
// Treat the whole image as a single person crop. Only sensible
// for inputs that are already a person (selfies, single-subject
// fitness clips, unit-test crops). Higher-level callers can
// also use ANSODBase::RunInference(input, vector<Rect>, cam)
// which crops each bbox and routes back here per crop.
// ------------------------------------------------------------------
std::vector<Object> ANSONNXRTMPOSE::DetectObjects(const cv::Mat& inputImage,
const std::string& camera_id)
{
try {
// Snapshot engine handle + detector under brief lock; dereference
// outside (mirrors ANSONNXYOLO::DetectObjects so concurrent
// inference threads don't serialise on _mutex for the full
// ORT Run() chain).
std::shared_ptr<ONNXRtmPose> engine;
ANSONNXYOLO* personDetector = nullptr; // raw view; lifetime owned by m_personDetector
std::string firstClass;
{
auto lk = TryLockWithTimeout("ANSONNXRTMPOSE::DetectObjects");
if (!lk.owns_lock()) return {};
if (!m_ortEngine) {
_logger.LogError("ANSONNXRTMPOSE::DetectObjects",
"ORT engine is null", __FILE__, __LINE__);
return {};
}
engine = m_ortEngine; // ref-count++
personDetector = m_personDetector.get();
if (!_classes.empty()) firstClass = _classes.front();
}
// --- Top-down path ---
if (personDetector) {
std::vector<Object> detections = personDetector->RunInference(inputImage, camera_id);
if (detections.empty()) {
ANS_DBG("ANSONNXRTMPOSE", "DetectObjects: detector found 0 detections cam='%s'",
camera_id.c_str());
return {};
}
// person.onnx may be a multi-class detector (e.g. a generic
// COCO YOLO with 80 classes). The COCO 'person' class is id=0
// by convention. Drop everything else BEFORE pose / dedup so
// we don't waste cycles on cars, dogs, signs, etc.
std::vector<Object> persons;
persons.reserve(detections.size());
for (auto& d : detections) {
if (d.classId != 0) continue;
if (d.box.width <= 0 || d.box.height <= 0) continue;
persons.push_back(std::move(d));
}
if (persons.empty()) {
ANS_DBG("ANSONNXRTMPOSE",
"DetectObjects: %zu detections but 0 persons (classId==0) cam='%s'",
detections.size(), camera_id.c_str());
return {};
}
// IoU dedup — Ultralytics E2E NMS in person.onnx is too lax
// (training-time IoU≈0.7 default), letting near-duplicate
// boxes through. Drop duplicates BEFORE running pose so we
// don't pay for two RTMPose inferences on the same subject.
const size_t beforeDedup = persons.size();
persons = dedupByIoU(persons, kPersonDedupIoU);
if (persons.size() != beforeDedup) {
ANS_DBG("ANSONNXRTMPOSE", "DetectObjects: dedup %zu -> %zu (IoU>%g) cam='%s'",
beforeDedup, persons.size(), kPersonDedupIoU, camera_id.c_str());
}
std::vector<Object> out;
out.reserve(persons.size());
for (const auto& person : persons) {
Object pose = engine->estimate(inputImage, person.box);
if (pose.kps.empty()) continue;
pose.cameraId = camera_id;
pose.trackId = person.trackId;
pose.confidence = person.confidence;
pose.className = !person.className.empty() ? person.className
: (firstClass.empty() ? std::string("person") : firstClass);
out.push_back(std::move(pose));
}
ANS_DBG("ANSONNXRTMPOSE", "DetectObjects: %zu persons -> %zu poses cam='%s'",
persons.size(), out.size(), camera_id.c_str());
return out;
}
// --- Standalone fallback path ---
const cv::Rect wholeImage(0, 0, inputImage.cols, inputImage.rows);
Object obj = engine->estimate(inputImage, wholeImage);
if (obj.kps.empty()) return {};
obj.cameraId = camera_id;
obj.className = firstClass.empty() ? std::string("person") : firstClass;
std::vector<Object> out;
out.push_back(std::move(obj));
ANS_DBG("ANSONNXRTMPOSE", "DetectObjects (standalone): kps=%d cam='%s'",
engine->numKeypoints(), camera_id.c_str());
return out;
}
catch (const std::exception& e) {
_logger.LogFatal("ANSONNXRTMPOSE::DetectObjects", e.what(), __FILE__, __LINE__);
ANS_DBG("ANSONNXRTMPOSE", "DetectObjects: EXCEPTION '%s'", e.what());
return {};
}
}
// ====================================================================
// TryLoadPersonDetector
//
// Look for person.onnx + person.names alongside the pose model. If
// both exist, instantiate an ANSONNXYOLO front-end and load it via
// LoadModelFromFolder (which creates the ORT session). On success
// m_personDetector is non-null and DetectObjects switches to the
// proper top-down pipeline.
//
// Missing files are NOT an error — the wrapper is designed to work
// standalone for single-person test inputs. We log a debug line so
// it's clear which path is active and return false.
// ====================================================================
bool ANSONNXRTMPOSE::TryLoadPersonDetector(const std::string& modelFolder) {
const std::string personOnnx = CreateFilePath(modelFolder, "person.onnx");
const std::string personNames = CreateFilePath(modelFolder, "person.names");
if (!FileExist(personOnnx) || !FileExist(personNames)) {
ANS_DBG("ANSONNXRTMPOSE",
"TryLoadPersonDetector: person.onnx/person.names not in '%s' "
"-- standalone mode (whole-image pose)", modelFolder.c_str());
return false;
}
if (m_personDetector) {
ANS_DBG("ANSONNXRTMPOSE", "TryLoadPersonDetector: already loaded, skipping");
return true;
}
try {
auto detector = std::make_unique<ANSONNXYOLO>();
detector->SetLoadEngineOnCreation(this->_loadEngineOnCreation);
ModelConfig personCfg{};
personCfg.detectionType = DetectionType::DETECTION;
personCfg.modelType = ModelType::ONNXYOLO;
personCfg.precisionType = PrecisionType::FP32;
personCfg.inpHeight = 640;
personCfg.inpWidth = 640;
personCfg.detectionScoreThreshold = 0.5f;
personCfg.modelConfThreshold = 0.5f;
personCfg.modelMNSThreshold = 0.45f;
personCfg.numKPS = 0;
personCfg.gpuOptBatchSize = 1;
personCfg.gpuMaxBatchSize = 1;
std::string personLabelMap;
const bool ok = detector->LoadModelFromFolder(
/*licenseKey*/ _licenseKey,
personCfg,
/*modelName*/ "person",
/*className*/ "person.names",
/*modelFolder*/ modelFolder,
personLabelMap);
if (!ok) {
_logger.LogWarn("ANSONNXRTMPOSE::TryLoadPersonDetector",
"Person detector LoadModelFromFolder FAILED -- falling back to whole-image pose",
__FILE__, __LINE__);
ANS_DBG("ANSONNXRTMPOSE", "TryLoadPersonDetector: detector load FAILED");
return false;
}
m_personDetector = std::move(detector);
ANS_DBG("ANSONNXRTMPOSE",
"TryLoadPersonDetector: detector loaded OK from '%s' (labels='%s')",
modelFolder.c_str(), personLabelMap.c_str());
return true;
}
catch (const std::exception& e) {
_logger.LogWarn("ANSONNXRTMPOSE::TryLoadPersonDetector", e.what(), __FILE__, __LINE__);
ANS_DBG("ANSONNXRTMPOSE", "TryLoadPersonDetector: EXCEPTION '%s'", e.what());
m_personDetector.reset();
return false;
}
}
} // namespace ANSCENTER
+175
View File
@@ -0,0 +1,175 @@
#ifndef ANSONNXRTMPOSE_H
#define ANSONNXRTMPOSE_H
#pragma once
#include "ANSEngineCommon.h"
#include "ONNXEngine.h"
#include "ANSONNXYOLO.h" // internal person-detector front-end (top-down pipeline)
namespace ANSCENTER {
// ====================================================================
// ONNXRtmPose — BasicOrtHandler subclass for RTMPose (SimCC) ONNX
//
// Ported from C:\Projects\AIVISION\RTM_POSE\OnnxRuntime-CPU\
// OnnxRunTime-CPU\rtmpose_onnxruntime.cpp but using BasicOrtHandler
// so the session benefits from the shared EP loader (CUDA / TensorRT
// / DirectML / OpenVINO / CPU auto-detect), per-session high-perf
// options, EP fallback recovery, and the wstring path handling that
// BasicOrtHandler centralises.
//
// RTMPose is a top-down single-person pose estimator:
// Input : person crop, 192x256 RGB, ImageNet-normalized
// Output : two SimCC heatmaps
// simcc_x [1, K, ext_w]
// simcc_y [1, K, ext_h]
// K = number of keypoints (e.g. 17 / 26 / 133)
// ext_w / ext_h = SimCC oversampled dims (typ. 2 * input)
//
// Class name is mixed-case (not ONNXRTMPOSE) to avoid a collision with
// the unscoped enum value ModelType::ONNXRTMPOSE in ANSLicense.h —
// identical spelling would shadow the type.
// ====================================================================
class ONNXENGINE_API ONNXRtmPose : public BasicOrtHandler
{
public:
explicit ONNXRtmPose(const std::string& _onnx_path,
unsigned int _num_threads = 1);
explicit ONNXRtmPose(const std::string& _onnx_path,
EngineType engineType,
unsigned int _num_threads = 1);
~ONNXRtmPose() override = default;
// Full per-bbox pipeline. Returns one Object with kps in original-
// image coords (kps layout: [x, y, score] * K). Empty kps means
// the call short-circuited (invalid box, runtime error, etc.).
Object estimate(const cv::Mat& image, const cv::Rect& box);
// Read-back accessors so the wrapper can introspect once the
// session is alive (used to populate _modelConfig.numKPS, log a
// sanity line, etc.).
int inputWidth() const { return m_input_w; }
int inputHeight() const { return m_input_h; }
int numKeypoints() const { return m_num_points; }
private:
// Override the BasicOrtHandler pure virtuals. transform() takes a
// pre-cropped 192x256 BGR person image and produces the normalized
// CHW tensor. transformBatch() throws — RTMPose is single-person.
Ort::Value transform(const cv::Mat& mat) override;
Ort::Value transformBatch(const std::vector<cv::Mat>& images) override;
// Per-call preprocess that writes into a caller-provided buffer +
// dims vector instead of the BasicOrtHandler members (mirrors
// ONNXYOLO::transformInto). Lets concurrent threads sharing one
// ONNXRtmPose instance preprocess and infer in parallel without
// racing on input_values_handler.
Ort::Value transformInto(const cv::Mat& cropBGR,
std::vector<float>& outBuffer,
std::vector<int64_t>& outDims);
// Affine helpers — direct port of utils.h::GetAffineTransform
// and rtmpose_onnxruntime.cpp::CropImageByDetectBox.
static cv::Mat GetAffineTransform(float center_x, float center_y,
float scale_w, float scale_h,
int output_w, int output_h,
bool inverse = false);
void CropImageByDetectBox(const cv::Mat& image, const cv::Rect& box,
cv::Mat& crop, cv::Mat& affine_rev);
// Read input geometry off the model (1x3xHxW). Falls back to
// 192x256 (the canonical RTMPose body-2d shape) if the ONNX
// export uses dynamic dims.
void resolveInputShape();
// Fixed RTMPose model geometry (read off the model at construction;
// defaults match the reference impl).
int m_input_w{ 192 };
int m_input_h{ 256 };
int m_num_points{ 26 };
// ImageNet normalization. Matches the IMAGE_MEAN / IMAGE_STD
// constants in C:\Projects\AIVISION\RTM_POSE\OnnxRuntime-CPU\
// OnnxRunTime-CPU\rtmpose_utils.h.
const float MEAN_R{ 123.675f };
const float MEAN_G{ 116.28f };
const float MEAN_B{ 103.53f };
const float STD_R { 58.395f };
const float STD_G { 57.12f };
const float STD_B { 57.375f };
};
// ====================================================================
// ANSONNXRTMPOSE — ANSODBase wrapper around ONNXRtmPose
//
// Mirrors the structure of ANSONNXYOLO. Owns a shared_ptr<ONNXRtmPose>
// so a concurrent inference thread can copy the handle under a brief
// lock and dereference it lock-free, which lets multiple AI tasks
// share one ANSONNXRTMPOSE without serialising on the engine mutex.
// ====================================================================
class ANSENGINE_API ANSONNXRTMPOSE : public ANSODBase {
public:
bool Initialize(std::string licenseKey, ModelConfig modelConfig,
const std::string& modelZipFilePath,
const std::string& modelZipPassword,
std::string& labelMap) override;
bool LoadModel(const std::string& modelZipFilePath,
const std::string& modelZipPassword) override;
bool LoadModelFromFolder(std::string licenseKey, ModelConfig modelConfig,
std::string modelName, std::string className,
const std::string& modelFolder,
std::string& labelMap) override;
bool OptimizeModel(bool fp16, std::string& optimizedModelFolder) override;
std::vector<Object> RunInference(const cv::Mat& input) override;
std::vector<Object> RunInference(const cv::Mat& input,
const std::string& camera_id) override;
bool Destroy() override;
~ANSONNXRTMPOSE();
private:
std::string _modelFilePath;
bool _modelLoadValid{ false };
// Keypoint confidence floor. The SimCC max score is emitted as
// kps[2] regardless; this is just the threshold used to compute
// the per-Object confidence average.
float KPS_THRESHOLD{ 0.3f };
// ORT engine. shared_ptr (mirrors ANSONNXYOLO) so an inference
// thread holding a local copy keeps the underlying session alive
// even if a future CPU-fallback / EP-rebuild path resets the
// member here on another thread.
std::shared_ptr<ONNXRtmPose> m_ortEngine;
// ── Optional internal person detector (top-down pipeline) ───────
// RTMPose is a top-down single-person estimator. If the model
// folder bundles person.onnx + person.names alongside the pose
// model, we auto-load an ANSONNXYOLO person detector and run pose
// ONCE PER DETECTED PERSON. When those files are missing the
// wrapper falls back to whole-image pose so single-person test
// inputs (selfies, fitness clips) still work.
std::unique_ptr<ANSONNXYOLO> m_personDetector;
// Look for person.onnx + person.names in `modelFolder`. Loads
// m_personDetector and returns true on success; returns false
// (without logging an error) if either file is missing — caller
// should treat that as "no detector, fall back to whole-image".
bool TryLoadPersonDetector(const std::string& modelFolder);
bool InitOrtEngine();
bool InitOrtEngine(ANSCENTER::EngineType engineType);
void WarmUpEngine();
// Per-bbox pipeline used by RunInference. Pulled out as its own
// method so the future bbox-overload (ANSODBase::RunInference with
// a vector<cv::Rect>) can route here directly.
std::vector<Object> DetectObjects(const cv::Mat& inputImage,
const std::string& camera_id);
};
} // namespace ANSCENTER
#endif
+6 -4
View File
@@ -726,7 +726,7 @@ namespace ANSCENTER {
std::vector<Object> ANSONNXSEG::postprocess(
const cv::Size& origSize,
const cv::Size& letterboxSize,
const std::vector<Ort::Value>& outputs,
std::vector<Ort::Value>& outputs,
const std::string& camera_id)
{
std::lock_guard<std::recursive_mutex> lock(_mutex);
@@ -738,9 +738,11 @@ namespace ANSCENTER {
std::to_string(outputs.size()));
}
// Extract output tensors
const float* detections = outputs[0].GetTensorData<float>();
const float* prototypes = outputs[1].GetTensorData<float>();
// Extract output tensors. GetTensorMutableData (not GetTensorData)
// on DML — const variant hangs on AMD after ~8 calls. Read-only
// despite the name.
const float* detections = outputs[0].GetTensorMutableData<float>();
const float* prototypes = outputs[1].GetTensorMutableData<float>();
// Get tensor shapes
auto detectionShape = outputs[0].GetTensorTypeAndShapeInfo().GetShape(); // [1, 116, N]
+3 -1
View File
@@ -51,8 +51,10 @@ namespace ANSCENTER {
void warmupModel();
bool Init(const std::string& modelPath, bool useGPU=true, int deviceId = 0);
cv::Mat preprocess(const cv::Mat& image,float*& blobPtr,std::vector<int64_t>& inputTensorShape);
// outputs is non-const because GetTensorMutableData() requires a
// non-const Ort::Value receiver — see ANSONNXCL.h for full note.
std::vector<Object> postprocess(const cv::Size& origSize,const cv::Size& letterboxSize,
const std::vector<Ort::Value>& outputs, const std::string& camera_id);
std::vector<Ort::Value>& outputs, const std::string& camera_id);
std::vector<Object> segment(const cv::Mat& image, const std::string& camera_id);
std::vector<cv::Point2f> maskToPolygon(const cv::Mat& binaryMask,
const cv::Rect& boundingBox,
+382 -125
View File
@@ -82,9 +82,15 @@ namespace ANSCENTER {
}
// ------------------------------------------------------------------
// transform — BGR → RGB, letterbox, /255, HWC→CHW
// transformInto — BGR → RGB, letterbox, /255, HWC→CHW into a
// caller-provided buffer + dims vector. Thread-safe across instances:
// each call's tensor backing memory is unique to the caller, so two
// threads sharing one ONNXYOLO can preprocess in parallel without
// racing on a shared input_values_handler.
// ------------------------------------------------------------------
Ort::Value ONNXYOLO::transform(const cv::Mat& mat)
Ort::Value ONNXYOLO::transformInto(const cv::Mat& mat,
std::vector<float>& outBuffer,
std::vector<int64_t>& outDims)
{
// Grayscale → BGR if needed
cv::Mat bgrMat;
@@ -94,7 +100,10 @@ namespace ANSCENTER {
bgrMat = mat;
}
// Check if model is classification (first output has 2 dims: [B, nc])
// Check if model is classification (first output has 2 dims: [B, nc]).
// output_node_dims is populated once at session creation in
// BasicOrtHandler::initialize_handler and never mutated by this
// class, so reading it concurrently from multiple threads is safe.
const bool isClassification = !output_node_dims.empty()
&& output_node_dims[0].size() == 2;
@@ -116,34 +125,50 @@ namespace ANSCENTER {
const int width = canvas.cols;
const size_t imageSize = static_cast<size_t>(height) * width;
input_node_dims = { 1, 3, height, width };
input_tensor_size = 1 * 3 * imageSize;
input_values_handler.resize(input_tensor_size);
// Diagnostics — log the very first transformInto() per ONNXYOLO so a
// future ORT shape-mismatch failure (e.g. /model.12/Concat for
// non-stride-32 inputs) points at the offending HxW directly.
// One line per session — does not flood DebugView.
static thread_local const ONNXYOLO* s_loggedFirst = nullptr;
if (s_loggedFirst != this) {
s_loggedFirst = this;
ANS_DBG("ONNXYOLO",
"transformInto[first]: inputImageShape=(%dx%d) canvas=%dx%d cls=%d src=%dx%d dyn=%d",
inputImageShape.width, inputImageShape.height, width, height,
isClassification ? 1 : 0, mat.cols, mat.rows,
isDynamicInputShape ? 1 : 0);
}
outDims = { 1, 3, height, width };
const size_t tensorSize = static_cast<size_t>(1) * 3 * imageSize;
outBuffer.resize(tensorSize);
std::vector<cv::Mat> channelMats(channels);
for (int c = 0; c < channels; ++c) {
channelMats[c] = cv::Mat(height, width, CV_32FC1,
input_values_handler.data() + c * imageSize);
outBuffer.data() + c * imageSize);
}
cv::split(canvas, channelMats);
return Ort::Value::CreateTensor<float>(
*memory_info_handler,
input_values_handler.data(),
input_tensor_size,
input_node_dims.data(),
input_node_dims.size());
outBuffer.data(),
tensorSize,
outDims.data(),
outDims.size());
}
Ort::Value ONNXYOLO::transformBatch(const std::vector<cv::Mat>& images)
Ort::Value ONNXYOLO::transformBatchInto(const std::vector<cv::Mat>& images,
std::vector<float>& outBuffer,
std::vector<int64_t>& outDims)
{
if (images.empty())
throw std::runtime_error("ONNXYOLO::transformBatch: empty input");
throw std::runtime_error("ONNXYOLO::transformBatchInto: empty input");
const size_t N = images.size();
// Preprocess all images: letterbox → BGR→RGB → float → /255
// Check if model is classification (first output has 2 dims: [B, nc])
// (output_node_dims is immutable post-init — safe to read here.)
const bool isClassification = !output_node_dims.empty()
&& output_node_dims[0].size() == 2;
@@ -151,7 +176,7 @@ namespace ANSCENTER {
batch.reserve(N);
for (const auto& img : images) {
if (img.empty())
throw std::runtime_error("ONNXYOLO::transformBatch: empty image in batch");
throw std::runtime_error("ONNXYOLO::transformBatchInto: empty image in batch");
// Grayscale → BGR if needed
cv::Mat bgrImg;
@@ -178,31 +203,54 @@ namespace ANSCENTER {
const int width = batch[0].cols;
const size_t imageSize = static_cast<size_t>(height) * width;
input_node_dims = {
outDims = {
static_cast<int64_t>(N), 3,
static_cast<int64_t>(height),
static_cast<int64_t>(width)
};
input_tensor_size = N * 3 * imageSize;
input_values_handler.resize(input_tensor_size);
const size_t tensorSize = N * 3 * imageSize;
outBuffer.resize(tensorSize);
// Pack each image into CHW layout (same as transform() for single image)
// Pack each image into CHW layout (same as transformInto() for single image)
for (size_t b = 0; b < N; ++b) {
const size_t batchOffset = b * 3 * imageSize;
std::vector<cv::Mat> channelMats(3);
for (int c = 0; c < 3; ++c) {
channelMats[c] = cv::Mat(height, width, CV_32FC1,
input_values_handler.data() + batchOffset + c * imageSize);
outBuffer.data() + batchOffset + c * imageSize);
}
cv::split(batch[b], channelMats);
}
return Ort::Value::CreateTensor<float>(
*memory_info_handler,
input_values_handler.data(),
input_tensor_size,
input_node_dims.data(),
input_node_dims.size());
outBuffer.data(),
tensorSize,
outDims.data(),
outDims.size());
}
// ------------------------------------------------------------------
// transform / transformBatch — pure-virtual overrides required by
// BasicOrtHandler. detect() / detectBatch() do NOT call these (they
// call transformInto / transformBatchInto with per-call buffers for
// thread safety). Kept as thin shims that delegate to the *Into
// variants but write the result into the shared base members
// (input_values_handler / input_node_dims). Any external caller that
// happens to invoke these directly gets the legacy behaviour.
// ------------------------------------------------------------------
Ort::Value ONNXYOLO::transform(const cv::Mat& mat)
{
Ort::Value v = transformInto(mat, input_values_handler, input_node_dims);
input_tensor_size = input_values_handler.size();
return v;
}
Ort::Value ONNXYOLO::transformBatch(const std::vector<cv::Mat>& images)
{
Ort::Value v = transformBatchInto(images, input_values_handler, input_node_dims);
input_tensor_size = input_values_handler.size();
return v;
}
// ------------------------------------------------------------------
@@ -227,7 +275,15 @@ namespace ANSCENTER {
if (image.empty())
return {};
Ort::Value inputTensor = transform(image);
// Per-call backing storage for the input tensor. These MUST live
// until ort_session->Run() returns — the Ort::Value is a non-owning
// view into inputBuffer.data(). Local lifetime guarantees no other
// thread can clobber our buffer mid-Run, which is what enables the
// 4 AI tasks sharing one ONNXYOLO to truly run inference in
// parallel on Intel/OpenVINO EP.
std::vector<float> inputBuffer;
std::vector<int64_t> inputDims;
Ort::Value inputTensor = transformInto(image, inputBuffer, inputDims);
auto outputTensors = ort_session->Run(
Ort::RunOptions{ nullptr },
@@ -268,9 +324,14 @@ namespace ANSCENTER {
}
}
// resizedShape is the canvas size we letterboxed into. Pre-option-3
// this came from input_node_dims[3]/[2] which transform() had just
// overwritten — that's no longer thread-safe (other camera's call
// may have moved on). Use our local dims instead; they hold what
// transformInto() actually produced for THIS image.
const cv::Size resizedShape(
static_cast<int>(input_node_dims[3]),
static_cast<int>(input_node_dims[2]));
static_cast<int>(inputDims[3]),
static_cast<int>(inputDims[2]));
const size_t numOutputs = outputTensors.size();
@@ -1403,8 +1464,11 @@ namespace ANSCENTER {
const size_t N = images.size();
// Fallback to sequential if model has fixed batch=1
// (input_node_dims[0] == 1 and not dynamic (-1))
// Fallback to sequential if the MODEL has fixed batch=1. We read
// input_node_dims here only to inspect the model's static input
// shape — option-3 detect()/detectBatch() no longer overwrite
// input_node_dims, so it retains whatever initialize_handler
// populated from the loaded ONNX graph.
if (input_node_dims.size() >= 1 && input_node_dims[0] == 1) {
std::vector<std::vector<Object>> results(N);
for (size_t i = 0; i < N; ++i)
@@ -1419,8 +1483,12 @@ namespace ANSCENTER {
for (const auto& img : images)
originalSizes.push_back(img.size());
// Batch preprocess + single inference call
Ort::Value inputTensor = transformBatch(images);
// Per-call backing storage — see detect() for the full thread-safety
// rationale. These outlive ort_session->Run() because they are
// local to this function.
std::vector<float> inputBuffer;
std::vector<int64_t> inputDims;
Ort::Value inputTensor = transformBatchInto(images, inputBuffer, inputDims);
auto outputTensors = ort_session->Run(
Ort::RunOptions{ nullptr },
@@ -1446,9 +1514,11 @@ namespace ANSCENTER {
}
}
// Use the per-call dims, not the (now-immutable) member, for the
// same reason as detect().
const cv::Size resizedShape(
static_cast<int>(input_node_dims[3]),
static_cast<int>(input_node_dims[2]));
static_cast<int>(inputDims[3]),
static_cast<int>(inputDims[2]));
// Determine task type from output shapes (same logic as detect())
const size_t numOutputs = outputTensors.size();
@@ -1610,7 +1680,7 @@ namespace ANSCENTER {
"Model file does not exist: " + _modelFilePath, __FILE__, __LINE__);
return false;
}
m_ortEngine = std::make_unique<ONNXYOLO>(_modelFilePath);
m_ortEngine = std::make_shared<ONNXYOLO>(_modelFilePath);
return true;
}
catch (const std::exception& e) {
@@ -1626,7 +1696,7 @@ namespace ANSCENTER {
"Model file does not exist: " + _modelFilePath, __FILE__, __LINE__);
return false;
}
m_ortEngine = std::make_unique<ONNXYOLO>(_modelFilePath, engineType);
m_ortEngine = std::make_shared<ONNXYOLO>(_modelFilePath, engineType);
return true;
}
catch (const std::exception& e) {
@@ -1638,26 +1708,35 @@ namespace ANSCENTER {
// ========================================================================
// WarmUpEngine — run a dummy inference after session creation.
//
// Scope: **NVIDIA (CUDA EP) only.** On first inference, the CUDA EP
// allocates its memory arena (capped at 2 GB via BasicOrtHandler config),
// resolves cuDNN convolution algorithms, and populates the kernel launch
// cache. Running one dummy inference at load time amortises this cost
// so the first real frame doesn't see a latency spike.
// Scope: NVIDIA (CUDA EP) AND AMD (DirectML EP).
// • NVIDIA: amortises CUDA arena allocation, cuDNN convolution algo
// selection, and kernel-launch cache population.
// • AMD: amortises D3D12 PSO compilation and first-time DML kernel
// binding. Empirically ~150 ms on DirectML (see comment in
// ONNXOCRRecognizer::Warmup). Without this, that compile cost
// surfaces inside the first live Run() — which on AMD has been
// observed to race with concurrent DML traffic from other sessions
// (e.g. helmet detector running while ALPR's _lpDetector fires its
// first ever Run()) and trigger DmlGraphFusionHelper crashes /
// 887A0006 DEVICE_HUNG. Moving the heavy first-run work into
// LoadEngine — where only the loading thread is active — sidesteps
// this timing window.
//
// Explicitly disabled on AMD, Intel and CPU:
// • AMD (DirectML) — calling detect() at load time has been observed
// to hit a multi-terabyte cv::Mat allocation inside postprocessLegacy
// on AMD RDNA iGPUs when DirectML returns garbage output tensor
// dims. ONNXYOLO::detect() now has an output-shape sanity guard
// that catches this at runtime, so the warm-up would add risk
// without benefit. Earlier builds enabled warm-up specifically for
// Radeon 680M TDR mitigation; that workaround is obsolete with
// current DirectML 1.15.x drivers.
// • Intel (OpenVINO) — running detect() at load time has been
// observed to expose latent heap-corruption bugs
// (ntdll +0x1176e5 / STATUS_HEAP_CORRUPTION 0xc0000374).
// • CPU EP — no shader compile or kernel cache to warm up; the first
// real frame has the same latency as any subsequent frame.
// Skipped for:
// • Intel (OpenVINO EP) — OpenVINO compiles the full plan at session
// creation, so first Run() has no extra compile cost. Calling
// detect() at load time has also exposed latent heap-corruption
// bugs (ntdll +0x1176e5 / STATUS_HEAP_CORRUPTION 0xc0000374).
// • CPU EP — no shader / kernel cache to warm up.
//
// History note: AMD warm-up was disabled in earlier builds after
// observing multi-terabyte cv::Mat allocations from garbage DirectML
// output dims. ONNXYOLO::detect() now has an output-shape sanity
// guard (see line ~257, kMaxOutputDim = 1,000,000) that catches that
// case before postprocessLegacy can allocate, so re-enabling AMD
// warm-up is safe. The exception-swallowing catch blocks below
// guarantee that any warm-up failure is non-fatal — the session is
// already loaded and real inference will still work.
//
// Non-fatal on failure: if warm-up itself throws, regular inference
// still works — the engine is fully loaded before WarmUpEngine runs.
@@ -1665,9 +1744,11 @@ namespace ANSCENTER {
void ANSONNXYOLO::WarmUpEngine() {
if (!m_ortEngine) return;
// Gate strictly on NVIDIA_GPU. Every other EP is a no-op.
if (m_ortEngine->getEngineType() != EngineType::NVIDIA_GPU) {
ANS_DBG("ONNXYOLO", "Warm-up skipped (non-NVIDIA EP)");
// Warm up on NVIDIA and AMD (both have lazy first-Run() compile cost).
// Skip on CPU and OpenVINO — see scope comment above.
const auto et = m_ortEngine->getEngineType();
if (et != EngineType::NVIDIA_GPU && et != EngineType::AMD_GPU) {
ANS_DBG("ONNXYOLO", "Warm-up skipped (CPU/OpenVINO EP)");
return;
}
@@ -1691,12 +1772,26 @@ namespace ANSCENTER {
const int w = std::clamp(rawW, kMinDim, kMaxDim);
const int h = std::clamp(rawH, kMinDim, kMaxDim);
// Stride alignment guard — YOLOv8/v11/v12 FPN concats only line
// up when H,W are multiples of 32. A non-aligned warm-up size
// would crash inside /model.12/Concat with a 5-vs-6 shape error
// and the warning would mask any real engine load failure.
if ((w % 32) != 0 || (h % 32) != 0) {
_logger.LogWarn("ANSONNXYOLO::WarmUpEngine",
"Warm-up skipped — dims not multiple of stride 32 ("
+ std::to_string(w) + "x" + std::to_string(h) + ")",
__FILE__, __LINE__);
return;
}
try {
// Mid-gray BGR image matches the letterbox fill colour used in
// preprocessing (114,114,114 ~ 128) and avoids degenerate inputs.
cv::Mat dummy(h, w, CV_8UC3, cv::Scalar(128, 128, 128));
ANS_DBG("ONNXYOLO", "Warm-up: running 1 dummy CUDA inference (%dx%d)", w, h);
ANS_DBG("ONNXYOLO", "Warm-up: running 1 dummy %s inference (%dx%d) cfg=(%dx%d) raw=(%dx%d)",
(et == EngineType::NVIDIA_GPU) ? "CUDA" : "DirectML",
w, h, _modelConfig.inpWidth, _modelConfig.inpHeight, rawW, rawH);
auto t0 = std::chrono::steady_clock::now();
(void)m_ortEngine->detect(dummy, _classes,
@@ -1975,6 +2070,150 @@ namespace ANSCENTER {
}
}
// ====================================================================
// HandleDmlRunFailure — stay-on-DML recovery (replaces CPU-fallback).
//
// Called from every catch block that wraps an Ort::Session::Run on a
// DirectML EP. The contract is documented on the member declaration
// in ANSONNXYOLO.h — quick summary:
//
// * CAS on _dmlRecreateInProgress elects ONE recreate owner per
// driver hiccup; other concurrent failures just bail.
// * Cooldown (kDmlRecreateCooldownMs) prevents thrashing if E_FAIL
// keeps firing back to back.
// * IsDmlDeviceRemoved (887A0005) → CPU fallback, last resort.
// * After kMaxConsecutiveDmlFailures recreates without an
// intervening successful inference → CPU fallback.
// * Otherwise: reset() + InitOrtEngine(<original DML EP>) +
// WarmUpEngine() → fresh DML session, GPU inference resumes.
//
// Successful inferences reset _dmlConsecutiveFailures to 0 in
// DetectObjects / DetectObjectsBatch, so isolated transient hiccups
// do not accumulate toward the CPU-fallback threshold.
// ====================================================================
void ANSONNXYOLO::HandleDmlRunFailure(const char* context, const std::string& msg)
{
// Option A: a DML exception is the strongest possible signal
// that the AMD driver can't sustain parallel inference. Trip
// process-wide DML serial mode immediately so the NEXT
// inference attempt (from any thread, in this DLL or another)
// takes the global mutex and runs one-at-a-time. This is
// idempotent — only the first caller per process actually
// flips the flag. Done before the CAS so even the CAS losers
// (concurrent threads catching the same fatal) contribute the
// signal — but in practice the first one wins.
ANSCENTER::DmlSerialState::TripFromException(context, msg.c_str());
// CAS — only one thread owns the recreate at a time. Other
// concurrent DML failures just return and retry on the next call.
bool expected = false;
if (!_dmlRecreateInProgress.compare_exchange_strong(expected, true)) {
ANS_DBG("ONNXYOLO", "%s: DML fatal — another thread is recreating, skip",
context);
return;
}
// Hold the engine mutex for the entire recreate so concurrent
// inference threads (which use a shared_ptr local copy of the
// engine) don't see a half-built session.
std::lock_guard<std::recursive_mutex> lk(_mutex);
try {
// Cooldown — if we just recreated within the window, the
// driver is in a prolonged bad state; don't burn time
// recreating again immediately. Other threads will keep
// returning {} until either the next call past cooldown
// succeeds or we hit the CPU-fallback threshold.
const auto now = std::chrono::steady_clock::now();
const auto sinceLastMs = std::chrono::duration_cast<std::chrono::milliseconds>(
now - _lastDmlRecreateAt).count();
if (_lastDmlRecreateAt.time_since_epoch().count() != 0
&& sinceLastMs < kDmlRecreateCooldownMs) {
ANS_DBG("ONNXYOLO",
"%s: DML fatal — within cooldown (%lld ms < %lld), skip",
context, (long long)sinceLastMs, (long long)kDmlRecreateCooldownMs);
_dmlRecreateInProgress.store(false);
return;
}
_lastDmlRecreateAt = now;
// Decide target EP for the new session:
// * 887A0005 device-removed → CPU is the only option
// * Too many consecutive failures → CPU as last resort
// * Otherwise → recreate on the SAME DML EP we were on
ANSCENTER::EngineType originalEp = ANSCENTER::EngineType::CPU;
if (m_ortEngine) {
originalEp = m_ortEngine->getEngineType();
}
const int failures = _dmlConsecutiveFailures.fetch_add(1) + 1;
const bool deviceRemoved = ANSCENTER::IsDmlDeviceRemoved(msg);
const bool exceededRetries = failures > kMaxConsecutiveDmlFailures;
ANSCENTER::EngineType targetEp = originalEp;
const char* reason = "stay-on-DML retry";
if (deviceRemoved) {
targetEp = ANSCENTER::EngineType::CPU;
reason = "device-removed (887A0005) → CPU fallback";
} else if (exceededRetries) {
targetEp = ANSCENTER::EngineType::CPU;
reason = "exceeded retry budget → CPU fallback";
}
_logger.LogFatal(context,
std::string("DirectML fatal — ") + reason
+ " (failure #" + std::to_string(failures)
+ "). msg=" + msg.substr(0, 256),
__FILE__, __LINE__);
ANS_DBG("ONNXYOLO",
"%s: DML fatal recreate — %s, failures=%d, originalEp=%d, targetEp=%d",
context, reason, failures, (int)originalEp, (int)targetEp);
// reset() releases the member's reference; concurrent inference
// threads holding a shared_ptr local copy keep their old
// engine alive until they return — no use-after-free (B2).
m_ortEngine.reset();
if (!InitOrtEngine(targetEp)) {
_logger.LogFatal(context,
"DML recreate failed — m_ortEngine left null", __FILE__, __LINE__);
ANS_DBG("ONNXYOLO", "%s: InitOrtEngine(%d) failed", context, (int)targetEp);
_dmlRecreateInProgress.store(false);
return;
}
// Warm up the new session so the first real call doesn't pay
// the algorithm-selection / shader-compile tax. AMD DML
// recreate WITHOUT warmup typically hits 1-2 s on the very
// first inference, which can re-trigger our 2 s slow watchdog
// and produce a false alarm.
try {
WarmUpEngine();
} catch (const std::exception& we) {
ANS_DBG("ONNXYOLO", "%s: WarmUpEngine after recreate threw: %s",
context, we.what());
}
// If we landed on CPU as a last resort, leave the failure
// counter pinned so we don't oscillate back to DML on the
// next exception. Otherwise (we're back on DML) DON'T reset
// the counter here — it gets reset by a *successful*
// inference in DetectObjects / DetectObjectsBatch.
if (targetEp == ANSCENTER::EngineType::CPU
&& originalEp != ANSCENTER::EngineType::CPU) {
_logger.LogInfo(context,
"Permanently switched to CPU EP — DML retry budget exhausted "
"or device removed. Restart the process to retry GPU.",
__FILE__, __LINE__);
}
}
catch (const std::exception& re) {
_logger.LogFatal(context,
std::string("DML recreate exception: ") + re.what(),
__FILE__, __LINE__);
}
_dmlRecreateInProgress.store(false);
}
std::vector<Object> ANSONNXYOLO::DetectObjects(const cv::Mat& inputImage,
const std::string& camera_id)
{
@@ -1985,16 +2224,27 @@ namespace ANSCENTER {
return {};
}
// ── Snapshot config under a brief lock ──────────────────
// ── Snapshot config + engine handle under a brief lock ──
// Only hold _mutex long enough to validate state and copy
// the parameters needed for inference. The actual ORT
// detect() call runs outside the lock so that concurrent
// Initialize/LoadModel calls are not blocked for the full
// duration of inference.
//
// B2: We also snapshot a shared_ptr COPY of m_ortEngine
// here. The lock-free detect() call below dereferences
// the LOCAL copy (`engine`), not the member. If a
// concurrent thread catches a DML fatal and does
// m_ortEngine.reset() under the same lock, that only
// releases the member's reference — our local copy keeps
// the engine alive until this function returns. This
// closes the use-after-free race that produced the
// 0xc0000005 crash at ANSODEngine.dll +0x111fd0.
float probThresh, nmsThresh;
int numKps;
std::vector<std::string> classes;
bool trackerEnabled, stabilizationEnabled;
std::shared_ptr<ONNXYOLO> engine;
{
ANS_DBG("ONNXYOLO", "DetectObjects: cam=%s acquiring mutex...", camera_id.c_str());
auto lk = TryLockWithTimeout("ANSONNXYOLO::DetectObjects");
@@ -2007,13 +2257,14 @@ namespace ANSCENTER {
return {};
}
// Snapshot parameters while locked
// Snapshot parameters + engine handle while locked
probThresh = PROBABILITY_THRESHOLD;
nmsThresh = NMS_THRESHOLD;
numKps = NUM_KPS;
classes = _classes;
trackerEnabled = _trackerEnabled;
stabilizationEnabled = _stabilizationEnabled;
engine = m_ortEngine; // ref-count++
}
// ── _mutex released — heavy work below runs lock-free ───
@@ -2047,11 +2298,28 @@ namespace ANSCENTER {
}
}
// Run ORT inference — no mutex held, this is the expensive call
auto results = m_ortEngine->detect(inferenceImage, classes,
probThresh,
nmsThresh,
numKps);
// Run ORT inference — no engine mutex held, this is the
// expensive call.
// B2: dereference the LOCAL shared_ptr `engine`, not the
// member m_ortEngine. A concurrent recreate thread may
// reset() m_ortEngine while we're inside detect() — the
// local ref-count keeps the underlying ONNXYOLO alive
// until this function returns.
//
// Option A: DmlInferenceGuard wraps the call to (a)
// serialize on the global DML mutex IF DML serial mode has
// been tripped, and (b) trip serial mode if THIS call's
// latency exceeds the slow threshold (1.5 s — well below
// WDDM TDR). No-op on Intel/NVIDIA/CPU.
std::vector<Object> results;
{
DmlInferenceGuard dmlGuard(engine->getEngineType(),
"ANSONNXYOLO::DetectObjects");
results = engine->detect(inferenceImage, classes,
probThresh,
nmsThresh,
numKps);
}
// --- Rescale coordinates from full-res back to display-res ---
if (bgrScaleX != 1.0f || bgrScaleY != 1.0f) {
@@ -2075,42 +2343,38 @@ namespace ANSCENTER {
for (auto& obj : results)
obj.cameraId = camera_id;
// Tracking/stabilization (ApplyTracking has its own lock)
if (trackerEnabled && !m_ortEngine->lastWasClassification) {
// Tracking/stabilization (ApplyTracking has its own lock).
// B2: read lastWasClassification from the LOCAL engine copy
// — using m_ortEngine here would re-introduce the race we
// just closed for detect().
if (trackerEnabled && !engine->lastWasClassification) {
results = ApplyTracking(results, camera_id);
if (stabilizationEnabled) results = StabilizeDetections(results, camera_id);
}
// A successful inference clears the DML failure budget so
// isolated transient hiccups (one E_FAIL every few minutes)
// never accumulate toward kMaxConsecutiveDmlFailures and
// wrongly demote us to CPU. Only sustained back-to-back
// failures with no successful inference between them
// exhaust the budget.
_dmlConsecutiveFailures.store(0, std::memory_order_relaxed);
return results;
}
catch (const std::exception& e) {
const std::string msg = e.what();
if (msg.find("887A0005") != std::string::npos) {
if (!_dmlDeviceLost) {
_dmlDeviceLost = true;
_logger.LogFatal("ANSONNXYOLO::DetectObjects",
"DirectML GPU device lost (887A0005) — attempting CPU fallback",
__FILE__, __LINE__);
ANS_DBG("ONNXYOLO", "DML device lost — recreating session on CPU");
try {
std::lock_guard<std::recursive_mutex> lk(_mutex);
m_ortEngine.reset();
if (InitOrtEngine(ANSCENTER::EngineType::CPU)) {
_logger.LogInfo("ANSONNXYOLO::DetectObjects",
"CPU fallback session created successfully",
__FILE__, __LINE__);
ANS_DBG("ONNXYOLO", "CPU fallback OK");
} else {
_logger.LogFatal("ANSONNXYOLO::DetectObjects",
"CPU fallback session creation failed",
__FILE__, __LINE__);
}
} catch (const std::exception& re) {
_logger.LogFatal("ANSONNXYOLO::DetectObjects",
std::string("CPU fallback exception: ") + re.what(),
__FILE__, __LINE__);
}
}
// Any DML-fatal HRESULT or DML branded error → rebuild on CPU.
// See IsDmlFatalError() in ANSEngineCommon.h for the full set
// (887A0005 device-removed, 80004005 generic E_FAIL from
// DmlGraphFusionHelper / DmlCommandRecorder, etc.).
if (IsDmlFatalError(msg)) {
// Stay-on-DML retry — see HandleDmlRunFailure for full
// contract. Tries to rebuild the session on the same
// DML EP first; only falls back to CPU on
// device-removed (887A0005) or after the retry budget
// is exhausted.
HandleDmlRunFailure("ANSONNXYOLO::DetectObjects", msg);
return {};
}
@@ -2157,19 +2421,9 @@ namespace ANSCENTER {
}
catch (const std::exception& e) {
const std::string msg = e.what();
if (msg.find("887A0005") != std::string::npos) {
if (!_dmlDeviceLost) {
_dmlDeviceLost = true;
_logger.LogFatal("ANSONNXYOLO::RunInferencesBatch",
"DirectML GPU device lost (887A0005) — attempting CPU fallback",
__FILE__, __LINE__);
try {
m_ortEngine.reset();
if (!InitOrtEngine(ANSCENTER::EngineType::CPU))
_logger.LogFatal("ANSONNXYOLO::RunInferencesBatch",
"CPU fallback session creation failed", __FILE__, __LINE__);
} catch (...) {}
}
if (IsDmlFatalError(msg)) {
// Stay-on-DML retry — see HandleDmlRunFailure.
HandleDmlRunFailure("ANSONNXYOLO::RunInferencesBatch", msg);
return {};
}
_logger.LogFatal("ANSONNXYOLO::RunInferencesBatch",
@@ -2184,11 +2438,15 @@ namespace ANSCENTER {
try {
if (_modelLoading.load()) return {};
// Snapshot config under brief lock
// Snapshot config + engine handle under brief lock.
// B2: shared_ptr copy `engine` keeps the underlying ONNXYOLO
// alive across the lock-free detectBatch() call even if a
// concurrent DML-fatal handler resets m_ortEngine.
float probThresh, nmsThresh;
int numKps;
std::vector<std::string> classes;
bool trackerEnabled, stabilizationEnabled;
std::shared_ptr<ONNXYOLO> engine;
{
auto lk = TryLockWithTimeout("ANSONNXYOLO::DetectObjectsBatch");
if (!lk.owns_lock()) return {};
@@ -2204,13 +2462,20 @@ namespace ANSCENTER {
classes = _classes;
trackerEnabled = _trackerEnabled;
stabilizationEnabled = _stabilizationEnabled;
engine = m_ortEngine; // ref-count++
}
// Heavy work outside lock
auto batchResults = m_ortEngine->detectBatch(
inputImages, classes, probThresh, nmsThresh, numKps);
// Heavy work outside lock — dereference local copy, not member.
// Option A: same DmlInferenceGuard contract as DetectObjects.
std::vector<std::vector<Object>> batchResults;
{
DmlInferenceGuard dmlGuard(engine->getEngineType(),
"ANSONNXYOLO::DetectObjectsBatch");
batchResults = engine->detectBatch(
inputImages, classes, probThresh, nmsThresh, numKps);
}
const bool isClassification = m_ortEngine->lastBatchWasClassification;
const bool isClassification = engine->lastBatchWasClassification;
for (auto& results : batchResults) {
for (auto& obj : results)
@@ -2222,24 +2487,16 @@ namespace ANSCENTER {
}
}
// Clear DML failure budget on success — see DetectObjects
// for the full rationale.
_dmlConsecutiveFailures.store(0, std::memory_order_relaxed);
return batchResults;
}
catch (const std::exception& e) {
const std::string msg = e.what();
if (msg.find("887A0005") != std::string::npos) {
if (!_dmlDeviceLost) {
_dmlDeviceLost = true;
_logger.LogFatal("ANSONNXYOLO::DetectObjectsBatch",
"DirectML GPU device lost (887A0005) — attempting CPU fallback",
__FILE__, __LINE__);
try {
std::lock_guard<std::recursive_mutex> lk(_mutex);
m_ortEngine.reset();
if (!InitOrtEngine(ANSCENTER::EngineType::CPU))
_logger.LogFatal("ANSONNXYOLO::DetectObjectsBatch",
"CPU fallback session creation failed", __FILE__, __LINE__);
} catch (...) {}
}
if (IsDmlFatalError(msg)) {
// Stay-on-DML retry — see HandleDmlRunFailure.
HandleDmlRunFailure("ANSONNXYOLO::DetectObjectsBatch", msg);
return {};
}
_logger.LogFatal("ANSONNXYOLO::DetectObjectsBatch",
+101 -8
View File
@@ -62,8 +62,24 @@ namespace ANSCENTER {
/// Override the input image shape for dynamic-input models.
/// Call after construction when the model config specifies a
/// different resolution than the default 640x640.
///
/// Width/height are snapped UP to the nearest multiple of 32
/// (with a 64-pixel floor). Ultralytics YOLO necks (v8/v11/v12)
/// concat upsampled stride-32 features with backbone stride-16
/// features in /model.12/Concat — that op requires
/// 2*ceil(H/32) == ceil(H/16), which only holds when H is a
/// multiple of 32. Feeding e.g. H=80 fails at runtime with
/// "Axis 2 has mismatched dimensions of 5 and 6". The 64-pixel
/// floor avoids degenerate sub-stride-32 inputs that would
/// produce 1x1 P5 maps.
void setInputShape(int width, int height) {
inputImageShape = cv::Size(width, height);
constexpr int kStride = 32;
constexpr int kMinDim = 64;
auto snap = [](int v) {
if (v < kMinDim) v = kMinDim;
return ((v + kStride - 1) / kStride) * kStride;
};
inputImageShape = cv::Size(snap(width), snap(height));
}
/// True if the ONNX model has dynamic spatial dimensions.
@@ -162,6 +178,23 @@ namespace ANSCENTER {
// Cached model input shape
cv::Size inputImageShape;
bool isDynamicInputShape{ false };
// Per-call preprocess that writes into caller-provided buffer + dims
// instead of the BasicOrtHandler members (input_values_handler /
// input_node_dims). detect() / detectBatch() use these so multiple
// threads sharing one ONNXYOLO instance can preprocess and infer
// concurrently — the Ort input tensor's backing storage is unique
// per call and lives on the caller's stack. Without this, thread B
// could overwrite the shared buffer between thread A's transform()
// and thread A's Run(), causing A's Run() to consume B's pixels and
// return detections that belong to B's frame. Observed on
// Intel/OpenVINO EP where nothing else serializes.
Ort::Value transformInto(const cv::Mat& mat,
std::vector<float>& outBuffer,
std::vector<int64_t>& outDims);
Ort::Value transformBatchInto(const std::vector<cv::Mat>& images,
std::vector<float>& outBuffer,
std::vector<int64_t>& outDims);
};
// ====================================================================
@@ -210,14 +243,74 @@ namespace ANSCENTER {
int NUM_KPS{ 0 };
float KPS_THRESHOLD{ 0.5f };
// ONNX Runtime inference engine
std::unique_ptr<ONNXYOLO> m_ortEngine;
// ONNX Runtime inference engine.
//
// B2: shared_ptr (was unique_ptr) so concurrent inference threads
// can grab a ref-counted local copy under the brief snapshot lock
// and dereference it OUTSIDE the lock without racing with a
// concurrent reset() in the DML-fatal CPU-fallback path. Without
// this, a thread doing `m_ortEngine->detect(...)` lock-free could
// see its underlying object freed mid-call by another thread that
// caught a DML E_FAIL and rebuilt the session — that was the
// 0xc0000005 access violation observed in ANSODEngine.dll under
// AMD/DirectML when multiple AI tasks ran in parallel.
std::shared_ptr<ONNXYOLO> m_ortEngine;
// DML device-lost recovery: when DirectML's GPU device is removed
// (HRESULT 887A0005), the session is permanently broken. We detect
// this once, attempt a CPU-fallback recreation, and suppress further
// error-log flooding.
bool _dmlDeviceLost{ false };
// ────────────────────────────────────────────────────────────────
// DML stay-on-GPU recovery state.
//
// AMD's DirectML EP intermittently throws 80004005 (E_FAIL) from
// DmlGraphFusionHelper / DmlCommandRecorder when the driver
// chokes under concurrent inference pressure. Empirically this
// is NOT a permanent device loss — recreating the ORT session on
// the same DML EP clears the wedged state and resumes GPU
// inference at full speed. Only the truly-fatal 887A0005
// (DXGI_ERROR_DEVICE_REMOVED) requires CPU fallback.
//
// Recovery contract (used by every Run() catch block):
// 1. compare_exchange on _dmlRecreateInProgress elects one
// thread as the recreate owner. Other threads return {}
// and retry on the next call.
// 2. The owner checks the cooldown — if we recreated in the
// last kDmlRecreateCooldownMs ms, skip (driver is in a
// prolonged bad state, don't thrash).
// 3. If error is IsDmlDeviceRemoved (887A0005), or we've hit
// kMaxConsecutiveDmlFailures recreates without a successful
// inference in between, fall back to CPU as last resort.
// 4. Otherwise reset() + InitOrtEngine(<original DML EP>) +
// WarmUpEngine() to rebuild on the same GPU EP.
// 5. On successful inference, _dmlConsecutiveFailures resets
// to 0, so isolated transient hiccups don't accumulate
// toward the CPU-fallback threshold.
//
// This replaces the old single-shot `bool _dmlDeviceLost` flag
// which latched on the first failure and forced CPU forever —
// user requirement is to keep AI tasks running on GPU, not to
// trade GPU performance for stability the moment the driver
// hiccups once.
// ────────────────────────────────────────────────────────────────
std::atomic<bool> _dmlRecreateInProgress{ false };
std::atomic<int> _dmlConsecutiveFailures{ 0 };
std::chrono::steady_clock::time_point _lastDmlRecreateAt{};
// Tunables — tuned for AMD Radeon 780M observed behavior.
// threshold: catches isolated transient AMD DML hiccups. After
// 3 consecutive recreates without a successful inference, the
// driver is in a prolonged bad state and CPU is the only
// forward path.
// cooldown: prevents recreate thrash. AMD DML session creation
// takes 200–500 ms; if E_FAIL fires from many threads at once
// we don't want to recreate N times in a row.
static constexpr int kMaxConsecutiveDmlFailures = 3;
static constexpr int64_t kDmlRecreateCooldownMs = 5000;
// Centralised DML-fatal recovery used by all 3 catch sites.
// Caller must have already determined the exception is DML-related
// (IsDmlFatalError == true). msg is the exception message used
// both to log and to detect the device-removed sub-case. Returns
// immediately without doing anything when another thread already
// owns the recreate (CAS loser path).
void HandleDmlRunFailure(const char* context, const std::string& msg);
// Internal detection pipeline
std::vector<Object> DetectObjects(const cv::Mat& inputImage,
+7 -3
View File
@@ -384,13 +384,17 @@ namespace ANSCENTER {
catch (const std::exception& e) {
const std::string msg = e.what();
// DML device-removal detection (see ANSONNXYOLO.cpp for details)
if (msg.find("887A0005") != std::string::npos) {
// DML fatal-error detection (887A0005 device-removed, 80004005 generic
// DML E_FAIL, branded DmlFusedNode / DmlCommandRecorder / etc.).
// See ANSCENTER::IsDmlFatalError in ANSEngineCommon.h.
if (IsDmlFatalError(msg)) {
if (!_dmlDeviceLost) {
_dmlDeviceLost = true;
_logger.LogFatal("ANSOVFD::RunInference",
"DirectML GPU device lost (887A0005) — will attempt CPU fallback on next call",
std::string("DirectML fatal error — will attempt CPU fallback on next call. msg=")
+ msg.substr(0, 256),
__FILE__, __LINE__);
ANS_DBG("ANSOVFD", "DML fatal msg=%s", msg.c_str());
}
return {};
}
+908
View File
@@ -0,0 +1,908 @@
#include "ANSRTRTMPOSE.h"
#include "Utility.h"
#include "ANSLicense.h" // ANS_DBG macro for DebugView
#include <opencv2/cudaimgproc.hpp>
#include <opencv2/cudawarping.hpp>
#include <algorithm>
#include <cmath>
namespace {
// ── IoU dedup for end-to-end YOLO person detector output ──────────────
//
// Ultralytics E2E ONNX exports bake NMS into the graph, but its IoU
// threshold is set at export time (default ≈ 0.7) and we can't override
// it from outside. For a top-down pose pipeline that's too lax — pairs
// of overlapping bboxes survive and we end up running RTMPose twice on
// the same person.
//
// This filter walks the detector's output (already sorted by confidence
// in the E2E format), keeping each box only if its IoU with every
// already-kept box is below kPersonDedupIoU. O(N²) is fine because N is
// capped at 300 and typically <10 in practice.
constexpr float kPersonDedupIoU = 0.45f;
inline float boxIoU(const cv::Rect& a, const cv::Rect& b) {
const cv::Rect inter = a & b;
if (inter.area() <= 0) return 0.f;
const int uni = a.area() + b.area() - inter.area();
return uni > 0 ? static_cast<float>(inter.area()) / static_cast<float>(uni) : 0.f;
}
std::vector<ANSCENTER::Object> dedupByIoU(
const std::vector<ANSCENTER::Object>& persons, float iouThr) {
std::vector<ANSCENTER::Object> kept;
kept.reserve(persons.size());
for (const auto& p : persons) {
bool isDup = false;
for (const auto& k : kept) {
if (boxIoU(p.box, k.box) > iouThr) { isDup = true; break; }
}
if (!isDup) kept.push_back(p);
}
return kept;
}
} // anonymous namespace
namespace ANSCENTER {
// ====================================================================
// ANSODBase interface — OptimizeModel
// ====================================================================
bool ANSRTRTMPOSE::OptimizeModel(bool fp16, std::string& optimizedModelFolder) {
std::lock_guard<std::recursive_mutex> lock(_mutex);
ANS_DBG("ANSRTRTMPOSE", "OptimizeModel: fp16=%d", fp16 ? 1 : 0);
if (!ANSODBase::OptimizeModel(fp16, optimizedModelFolder)) return false;
if (!FileExist(_modelFilePath)) {
_logger.LogFatal("ANSRTRTMPOSE::OptimizeModel",
"Raw model file path does not exist: " + _modelFilePath, __FILE__, __LINE__);
ANS_DBG("ANSRTRTMPOSE", "OptimizeModel: missing onnx '%s'", _modelFilePath.c_str());
return false;
}
try {
_fp16 = fp16;
optimizedModelFolder = GetParentFolder(_modelFilePath);
if (!m_trtEngine) {
m_options.optBatchSize = _modelConfig.gpuOptBatchSize;
m_options.maxBatchSize = _modelConfig.gpuMaxBatchSize;
m_options.deviceIndex = _modelConfig.gpuDeviceIndex;
m_options.maxInputHeight = _modelConfig.maxInputHeight;
m_options.minInputHeight = _modelConfig.minInputHeight;
m_options.optInputHeight = _modelConfig.optInputHeight;
m_options.maxInputWidth = _modelConfig.maxInputWidth;
m_options.minInputWidth = _modelConfig.minInputWidth;
m_options.optInputWidth = _modelConfig.optInputWidth;
m_options.engineFileDir = optimizedModelFolder;
m_options.precision = (_fp16 ? Precision::FP16 : Precision::FP32);
m_trtEngine = std::make_shared<Engine<float>>(m_options);
}
auto succ = m_trtEngine->buildWithRetry(_modelFilePath, SUB_VALS, DIV_VALS, NORMALIZE);
if (!succ) {
_logger.LogError("ANSRTRTMPOSE::OptimizeModel",
"Error: Unable to build TensorRT engine. " + _modelFilePath, __FILE__, __LINE__);
ANS_DBG("ANSRTRTMPOSE", "OptimizeModel: buildWithRetry FAILED for '%s'", _modelFilePath.c_str());
return false;
}
ANS_DBG("ANSRTRTMPOSE", "OptimizeModel: build OK -> '%s'", optimizedModelFolder.c_str());
// Also build the bundled person-detector engine if present.
// TryLoadPersonDetector triggers ANSRTYOLO::LoadModelFromFolder
// which performs the engine build (= optimization) and caches
// it to disk; the next Initialize/LoadModel will hit the cache.
(void)TryLoadPersonDetector(_modelFolder);
return true;
}
catch (std::exception& e) {
_logger.LogFatal("ANSRTRTMPOSE::OptimizeModel", e.what(), __FILE__, __LINE__);
ANS_DBG("ANSRTRTMPOSE", "OptimizeModel: EXCEPTION '%s'", e.what());
return false;
}
}
// ====================================================================
// ANSODBase interface — LoadModel
// ====================================================================
bool ANSRTRTMPOSE::LoadModel(const std::string& modelZipFilePath,
const std::string& modelZipPassword) {
std::lock_guard<std::recursive_mutex> lock(_mutex);
ModelLoadingGuard mlg(_modelLoading);
ANS_DBG("ANSRTRTMPOSE", "LoadModel: zip='%s'", modelZipFilePath.c_str());
try {
bool result = ANSODBase::LoadModel(modelZipFilePath, modelZipPassword);
if (!result) return false;
_modelConfig.modelType = ModelType::RTRTMPOSE;
// RTMPose has fixed input dims — we still propagate them through ModelConfig
// so the rest of the pipeline (caching, logging) can introspect them.
if (_modelConfig.inpHeight <= 0) _modelConfig.inpHeight = RTM_INPUT_H;
if (_modelConfig.inpWidth <= 0) _modelConfig.inpWidth = RTM_INPUT_W;
if (_modelConfig.numKPS <= 0 || _modelConfig.numKPS > 133)
_modelConfig.numKPS = 26; // default to RTMPose body 2d-26
if (_modelConfig.kpsThreshold <= 0)
_modelConfig.kpsThreshold = KPS_THRESHOLD;
_fp16 = true;
KPS_THRESHOLD = _modelConfig.kpsThreshold;
m_options.optBatchSize = _modelConfig.gpuOptBatchSize;
m_options.maxBatchSize = _modelConfig.gpuMaxBatchSize;
m_options.deviceIndex = _modelConfig.gpuDeviceIndex;
m_options.maxInputHeight = _modelConfig.maxInputHeight;
m_options.minInputHeight = _modelConfig.minInputHeight;
m_options.optInputHeight = _modelConfig.optInputHeight;
m_options.maxInputWidth = _modelConfig.maxInputWidth;
m_options.minInputWidth = _modelConfig.minInputWidth;
m_options.optInputWidth = _modelConfig.optInputWidth;
m_options.engineFileDir = _modelFolder;
m_options.precision = (_fp16 ? Precision::FP16 : Precision::FP32);
_modelFilePath = CreateFilePath(_modelFolder, "train_last.onnx");
if (FileExist(_modelConfigFile)) {
ModelType modelType;
std::vector<int> inputShape;
_classes = ANSUtilityHelper::GetConfigFileContent(_modelConfigFile, modelType, inputShape);
if (inputShape.size() == 2) {
if (inputShape[0] > 0) _modelConfig.inpHeight = inputShape[0];
if (inputShape[1] > 0) _modelConfig.inpWidth = inputShape[1];
}
}
else {
_classFilePath = CreateFilePath(_modelFolder, "classes.names");
std::ifstream isValid(_classFilePath);
if (!isValid) LoadClassesFromString();
else LoadClassesFromFile();
}
if (this->_loadEngineOnCreation) {
if (!m_trtEngine) {
m_poolKey = { _modelFilePath,
static_cast<int>(m_options.precision), m_options.maxBatchSize };
m_trtEngine = EnginePoolManager<float>::instance().acquire(
m_poolKey, m_options, _modelFilePath,
SUB_VALS, DIV_VALS, NORMALIZE, m_maxSlotsPerGpu);
m_usingSharedPool = (m_trtEngine != nullptr);
}
if (!m_trtEngine) {
_logger.LogError("ANSRTRTMPOSE::LoadModel",
"Error: Unable to load TensorRT engine. " + _modelFilePath, __FILE__, __LINE__);
ANS_DBG("ANSRTRTMPOSE", "LoadModel: engine acquire FAILED '%s'", _modelFilePath.c_str());
_modelLoadValid = false;
return false;
}
m_options.maxBatchSize = m_trtEngine->getOptions().maxBatchSize;
m_options.optBatchSize = m_trtEngine->getOptions().optBatchSize;
m_trtEngine->warmUp();
// Pull true input geometry from the engine; RTMPose is fixed-size,
// but reading it back protects us against unexpected ONNX exports.
const auto& inputDims = m_trtEngine->getInputDims();
if (!inputDims.empty()) {
if (inputDims[0].d[1] > 0) RTM_INPUT_H = inputDims[0].d[1];
if (inputDims[0].d[2] > 0) RTM_INPUT_W = inputDims[0].d[2];
}
ANS_DBG("ANSRTRTMPOSE", "LoadModel: engine ready %dx%d kps=%d sharedPool=%d",
RTM_INPUT_W, RTM_INPUT_H, _modelConfig.numKPS, m_usingSharedPool ? 1 : 0);
}
// Top-down pipeline: try to load the bundled person detector.
// Falls back gracefully (whole-image pose) when the files are missing.
(void)TryLoadPersonDetector(_modelFolder);
_modelLoadValid = true;
_isInitialized = true;
return true;
}
catch (std::exception& e) {
_logger.LogFatal("ANSRTRTMPOSE::LoadModel", e.what(), __FILE__, __LINE__);
ANS_DBG("ANSRTRTMPOSE", "LoadModel: EXCEPTION '%s'", e.what());
return false;
}
}
// ====================================================================
// ANSODBase interface — LoadModelFromFolder
// ====================================================================
bool ANSRTRTMPOSE::LoadModelFromFolder(std::string licenseKey, ModelConfig modelConfig,
std::string modelName, std::string className,
const std::string& modelFolder, std::string& labelMap) {
std::lock_guard<std::recursive_mutex> lock(_mutex);
ModelLoadingGuard mlg(_modelLoading);
ANS_DBG("ANSRTRTMPOSE", "LoadModelFromFolder: folder='%s' model='%s'",
modelFolder.c_str(), modelName.c_str());
try {
bool result = ANSODBase::LoadModelFromFolder(licenseKey, modelConfig,
modelName, className, modelFolder, labelMap);
if (!result) return false;
_modelConfig = modelConfig;
_modelConfig.modelType = ModelType::RTRTMPOSE;
if (_modelConfig.inpHeight <= 0) _modelConfig.inpHeight = RTM_INPUT_H;
if (_modelConfig.inpWidth <= 0) _modelConfig.inpWidth = RTM_INPUT_W;
_modelConfig.precisionType = PrecisionType::FP32;
if (_modelConfig.numKPS <= 0 || _modelConfig.numKPS > 133)
_modelConfig.numKPS = 26;
if (_modelConfig.kpsThreshold <= 0)
_modelConfig.kpsThreshold = KPS_THRESHOLD;
_fp16 = true;
KPS_THRESHOLD = _modelConfig.kpsThreshold;
std::string _modelName = modelName;
if (_modelName.empty()) _modelName = "train_last";
std::string modelFullName = _modelName + ".onnx";
m_options.optBatchSize = _modelConfig.gpuOptBatchSize;
m_options.maxBatchSize = _modelConfig.gpuMaxBatchSize;
m_options.deviceIndex = _modelConfig.gpuDeviceIndex;
m_options.maxInputHeight = _modelConfig.maxInputHeight;
m_options.minInputHeight = _modelConfig.minInputHeight;
m_options.optInputHeight = _modelConfig.optInputHeight;
m_options.maxInputWidth = _modelConfig.maxInputWidth;
m_options.minInputWidth = _modelConfig.minInputWidth;
m_options.optInputWidth = _modelConfig.optInputWidth;
m_options.engineFileDir = _modelFolder;
m_options.precision = (_fp16 ? Precision::FP16 : Precision::FP32);
_modelFilePath = CreateFilePath(_modelFolder, modelFullName);
if (FileExist(_modelConfigFile)) {
ModelType modelType;
std::vector<int> inputShape;
_classes = ANSUtilityHelper::GetConfigFileContent(_modelConfigFile, modelType, inputShape);
if (inputShape.size() == 2) {
if (inputShape[0] > 0) _modelConfig.inpHeight = inputShape[0];
if (inputShape[1] > 0) _modelConfig.inpWidth = inputShape[1];
}
}
else {
_classFilePath = CreateFilePath(_modelFolder, className);
std::ifstream isValid(_classFilePath);
if (!isValid) LoadClassesFromString();
else LoadClassesFromFile();
}
labelMap.clear();
if (!_classes.empty())
labelMap = VectorToCommaSeparatedString(_classes);
if (this->_loadEngineOnCreation) {
if (!m_trtEngine) {
m_poolKey = { _modelFilePath,
static_cast<int>(m_options.precision), m_options.maxBatchSize };
m_trtEngine = EnginePoolManager<float>::instance().acquire(
m_poolKey, m_options, _modelFilePath,
SUB_VALS, DIV_VALS, NORMALIZE, m_maxSlotsPerGpu);
m_usingSharedPool = (m_trtEngine != nullptr);
}
if (!m_trtEngine) {
_logger.LogError("ANSRTRTMPOSE::LoadModelFromFolder",
"Error: Unable to load TensorRT engine. " + _modelFilePath, __FILE__, __LINE__);
ANS_DBG("ANSRTRTMPOSE", "LoadModelFromFolder: engine acquire FAILED '%s'",
_modelFilePath.c_str());
_modelLoadValid = false;
return false;
}
m_options.maxBatchSize = m_trtEngine->getOptions().maxBatchSize;
m_options.optBatchSize = m_trtEngine->getOptions().optBatchSize;
m_trtEngine->warmUp();
const auto& inputDims = m_trtEngine->getInputDims();
if (!inputDims.empty()) {
if (inputDims[0].d[1] > 0) RTM_INPUT_H = inputDims[0].d[1];
if (inputDims[0].d[2] > 0) RTM_INPUT_W = inputDims[0].d[2];
}
ANS_DBG("ANSRTRTMPOSE", "LoadModelFromFolder: engine ready %dx%d kps=%d sharedPool=%d",
RTM_INPUT_W, RTM_INPUT_H, _modelConfig.numKPS, m_usingSharedPool ? 1 : 0);
}
(void)TryLoadPersonDetector(_modelFolder);
_modelLoadValid = true;
_isInitialized = true;
return true;
}
catch (std::exception& e) {
_logger.LogFatal("ANSRTRTMPOSE::LoadModelFromFolder", e.what(), __FILE__, __LINE__);
ANS_DBG("ANSRTRTMPOSE", "LoadModelFromFolder: EXCEPTION '%s'", e.what());
return false;
}
}
// ====================================================================
// ANSODBase interface — Initialize
// ====================================================================
bool ANSRTRTMPOSE::Initialize(std::string licenseKey, ModelConfig modelConfig,
const std::string& modelZipFilePath,
const std::string& modelZipPassword,
std::string& labelMap) {
std::lock_guard<std::recursive_mutex> lock(_mutex);
ModelLoadingGuard mlg(_modelLoading);
ANS_DBG("ANSRTRTMPOSE", "Initialize: zip='%s'", modelZipFilePath.c_str());
try {
const bool engineAlreadyLoaded =
_modelLoadValid && _isInitialized && m_trtEngine != nullptr;
_modelLoadValid = false;
bool result = ANSODBase::Initialize(licenseKey, modelConfig,
modelZipFilePath, modelZipPassword, labelMap);
if (!result) return false;
_modelConfig = modelConfig;
_modelConfig.modelType = ModelType::RTRTMPOSE;
if (_modelConfig.inpHeight <= 0) _modelConfig.inpHeight = RTM_INPUT_H;
if (_modelConfig.inpWidth <= 0) _modelConfig.inpWidth = RTM_INPUT_W;
_modelConfig.precisionType = PrecisionType::FP32;
if (_modelConfig.numKPS <= 0 || _modelConfig.numKPS > 133)
_modelConfig.numKPS = 26;
if (_modelConfig.kpsThreshold <= 0)
_modelConfig.kpsThreshold = KPS_THRESHOLD;
_fp16 = true;
KPS_THRESHOLD = _modelConfig.kpsThreshold;
m_options.optBatchSize = _modelConfig.gpuOptBatchSize;
m_options.maxBatchSize = _modelConfig.gpuMaxBatchSize;
m_options.deviceIndex = _modelConfig.gpuDeviceIndex;
m_options.maxInputHeight = _modelConfig.maxInputHeight;
m_options.minInputHeight = _modelConfig.minInputHeight;
m_options.optInputHeight = _modelConfig.optInputHeight;
m_options.maxInputWidth = _modelConfig.maxInputWidth;
m_options.minInputWidth = _modelConfig.minInputWidth;
m_options.optInputWidth = _modelConfig.optInputWidth;
m_options.engineFileDir = _modelFolder;
m_options.precision = (_fp16 ? Precision::FP16 : Precision::FP32);
_modelFilePath = CreateFilePath(_modelFolder, "train_last.onnx");
if (FileExist(_modelConfigFile)) {
ModelType modelType;
std::vector<int> inputShape;
_classes = ANSUtilityHelper::GetConfigFileContent(_modelConfigFile, modelType, inputShape);
if (inputShape.size() == 2) {
if (inputShape[0] > 0) _modelConfig.inpHeight = inputShape[0];
if (inputShape[1] > 0) _modelConfig.inpWidth = inputShape[1];
}
}
else {
_classFilePath = CreateFilePath(_modelFolder, "classes.names");
std::ifstream isValid(_classFilePath);
if (!isValid) LoadClassesFromString();
else LoadClassesFromFile();
}
labelMap.clear();
if (!_classes.empty())
labelMap = VectorToCommaSeparatedString(_classes);
if (this->_loadEngineOnCreation && !engineAlreadyLoaded) {
if (!m_trtEngine) {
m_poolKey = { _modelFilePath,
static_cast<int>(m_options.precision), m_options.maxBatchSize };
m_trtEngine = EnginePoolManager<float>::instance().acquire(
m_poolKey, m_options, _modelFilePath,
SUB_VALS, DIV_VALS, NORMALIZE, m_maxSlotsPerGpu);
m_usingSharedPool = (m_trtEngine != nullptr);
}
if (!m_trtEngine) {
_logger.LogError("ANSRTRTMPOSE::Initialize",
"Error: Unable to load TensorRT engine. " + _modelFilePath, __FILE__, __LINE__);
ANS_DBG("ANSRTRTMPOSE", "Initialize: engine acquire FAILED '%s'", _modelFilePath.c_str());
_modelLoadValid = false;
return false;
}
m_options.maxBatchSize = m_trtEngine->getOptions().maxBatchSize;
m_options.optBatchSize = m_trtEngine->getOptions().optBatchSize;
m_trtEngine->warmUp();
const auto& inputDims = m_trtEngine->getInputDims();
if (!inputDims.empty()) {
if (inputDims[0].d[1] > 0) RTM_INPUT_H = inputDims[0].d[1];
if (inputDims[0].d[2] > 0) RTM_INPUT_W = inputDims[0].d[2];
}
ANS_DBG("ANSRTRTMPOSE", "Initialize: engine ready %dx%d kps=%d sharedPool=%d",
RTM_INPUT_W, RTM_INPUT_H, _modelConfig.numKPS, m_usingSharedPool ? 1 : 0);
}
(void)TryLoadPersonDetector(_modelFolder);
_modelLoadValid = true;
_isInitialized = true;
return true;
}
catch (std::exception& e) {
_logger.LogFatal("ANSRTRTMPOSE::Initialize", e.what(), __FILE__, __LINE__);
ANS_DBG("ANSRTRTMPOSE", "Initialize: EXCEPTION '%s'", e.what());
return false;
}
}
// ====================================================================
// RunInference / Destroy / Destructor
// ====================================================================
std::vector<Object> ANSRTRTMPOSE::RunInference(const cv::Mat& inputImgBGR) {
return RunInference(inputImgBGR, "");
}
std::vector<Object> ANSRTRTMPOSE::RunInference(const cv::Mat& inputImgBGR,
const std::string& camera_id) {
if (!PreInferenceCheck("ANSRTRTMPOSE::RunInference")) return {};
if (inputImgBGR.empty()) {
ANS_DBG("ANSRTRTMPOSE", "RunInference: empty input cam='%s'", camera_id.c_str());
return {};
}
try {
// Two pipelines:
//
// (a) Top-down (preferred) — m_personDetector loaded from
// the bundled person.onnx + person.names. Detect people
// first, then run RTMPose ONCE per person bbox. This is
// what RTMPose is designed for; works on multi-person
// scenes with small / distant subjects.
//
// (b) Standalone fallback — no detector files in the model
// folder. Treat the whole image as a single person crop.
// Only sensible for inputs that are already a person
// (selfies, single-subject fitness clips, unit-test
// crops). Higher-level callers can also use the
// ANSODBase::RunInference(input, vector<Rect>, cam)
// overload which crops each bbox and routes per-crop
// through our Preprocess.
if (m_personDetector) {
std::vector<Object> detections = m_personDetector->RunInference(inputImgBGR, camera_id);
if (detections.empty()) {
ANS_DBG("ANSRTRTMPOSE", "RunInference: detector found 0 detections cam='%s'",
camera_id.c_str());
return {};
}
// person.onnx may be a multi-class detector (e.g. a generic
// COCO YOLO with 80 classes). The COCO 'person' class is id=0
// by convention. Drop everything else BEFORE pose / dedup so
// we don't waste cycles on cars, dogs, signs, etc.
std::vector<Object> persons;
persons.reserve(detections.size());
for (auto& d : detections) {
if (d.classId != 0) continue;
if (d.box.width <= 0 || d.box.height <= 0) continue;
persons.push_back(std::move(d));
}
if (persons.empty()) {
ANS_DBG("ANSRTRTMPOSE",
"RunInference: %zu detections but 0 persons (classId==0) cam='%s'",
detections.size(), camera_id.c_str());
return {};
}
// IoU dedup — Ultralytics E2E NMS in person.onnx is too lax
// (training-time IoU≈0.7 default), letting near-duplicate
// boxes through. Drop duplicates BEFORE running pose so we
// don't pay for two RTMPose inferences on the same subject.
const size_t beforeDedup = persons.size();
persons = dedupByIoU(persons, kPersonDedupIoU);
if (persons.size() != beforeDedup) {
ANS_DBG("ANSRTRTMPOSE", "RunInference: dedup %zu -> %zu (IoU>%g) cam='%s'",
beforeDedup, persons.size(), kPersonDedupIoU, camera_id.c_str());
}
std::vector<Object> results;
results.reserve(persons.size());
for (const auto& person : persons) {
auto poseResults = InferenceOnBox(inputImgBGR, person.box, camera_id);
for (auto& obj : poseResults) {
// Carry the detector's track-id and confidence so
// downstream tracking/visualization stays coherent
// across frames. RTMPose itself has no class /
// tracking concept of its own.
obj.trackId = person.trackId;
obj.confidence = person.confidence;
if (!person.className.empty()) obj.className = person.className;
results.push_back(std::move(obj));
}
}
ANS_DBG("ANSRTRTMPOSE", "RunInference: %zu persons -> %zu poses cam='%s'",
persons.size(), results.size(), camera_id.c_str());
return results;
}
// Fallback — single-person whole-image pose
const cv::Rect wholeImage(0, 0, inputImgBGR.cols, inputImgBGR.rows);
return InferenceOnBox(inputImgBGR, wholeImage, camera_id);
}
catch (const std::exception& e) {
_logger.LogFatal("ANSRTRTMPOSE::RunInference", e.what(), __FILE__, __LINE__);
ANS_DBG("ANSRTRTMPOSE", "RunInference: EXCEPTION '%s'", e.what());
return {};
}
}
ANSRTRTMPOSE::~ANSRTRTMPOSE() {
try { Destroy(); }
catch (std::exception& e) {
_logger.LogError("ANSRTRTMPOSE::~ANSRTRTMPOSE()", e.what(), __FILE__, __LINE__);
}
}
bool ANSRTRTMPOSE::Destroy() {
try {
if (m_personDetector) {
m_personDetector->Destroy();
m_personDetector.reset();
}
if (m_usingSharedPool) {
EnginePoolManager<float>::instance().release(m_poolKey);
m_trtEngine.reset();
m_usingSharedPool = false;
}
else {
m_trtEngine.reset();
}
ANS_DBG("ANSRTRTMPOSE", "Destroy: engine released");
return true;
}
catch (std::exception& e) {
_logger.LogError("ANSRTRTMPOSE::Destroy()", e.what(), __FILE__, __LINE__);
ANS_DBG("ANSRTRTMPOSE", "Destroy: EXCEPTION '%s'", e.what());
return false;
}
}
// ====================================================================
// Per-bbox single-shot inference pipeline
// ====================================================================
std::vector<Object> ANSRTRTMPOSE::InferenceOnBox(const cv::Mat& input,
const cv::Rect& box,
const std::string& camera_id) {
if (!m_trtEngine) {
ANS_DBG("ANSRTRTMPOSE", "InferenceOnBox: engine null");
return {};
}
if (box.width <= 0 || box.height <= 0) {
ANS_DBG("ANSRTRTMPOSE", "InferenceOnBox: invalid box %dx%d", box.width, box.height);
return {};
}
m_trtEngine->setDeviceContext();
// Phase 1 — preprocess under lock (affine warp on CPU, upload to GPU)
ImageMetadata meta;
std::vector<std::vector<cv::cuda::GpuMat>> trtInput;
{
std::lock_guard<std::recursive_mutex> lock(_mutex);
trtInput = Preprocess(input, box, meta);
}
if (trtInput.empty() || trtInput[0].empty()) {
ANS_DBG("ANSRTRTMPOSE", "InferenceOnBox: preprocess failed box=%d,%d %dx%d",
box.x, box.y, box.width, box.height);
return {};
}
// Phase 2 — inference (mutex released; pool dispatches to idle slot)
std::vector<std::vector<std::vector<float>>> featureVectors;
if (!m_trtEngine->runInference(trtInput, featureVectors)) {
_logger.LogError("ANSRTRTMPOSE::InferenceOnBox", "runInference failed", __FILE__, __LINE__);
ANS_DBG("ANSRTRTMPOSE", "InferenceOnBox: runInference FAILED");
return {};
}
if (featureVectors.empty() || featureVectors[0].size() < 2) {
_logger.LogError("ANSRTRTMPOSE::InferenceOnBox",
"Expected 2 SimCC outputs (x,y), got " +
std::to_string(featureVectors.empty() ? 0 : featureVectors[0].size()),
__FILE__, __LINE__);
ANS_DBG("ANSRTRTMPOSE", "InferenceOnBox: bad output count outer=%zu inner=%zu",
featureVectors.size(),
featureVectors.empty() ? size_t{ 0 } : featureVectors[0].size());
return {};
}
// Phase 3 — postprocess. Output ordering in featureVectors matches the
// engine's output binding order; for RTMPose ONNX exports that's
// [simcc_x, simcc_y]. We disambiguate by feature length anyway.
const auto& outputDims = m_trtEngine->getOutputDims();
if (outputDims.size() < 2 ||
outputDims[0].nbDims < 3 || outputDims[1].nbDims < 3) {
_logger.LogError("ANSRTRTMPOSE::InferenceOnBox",
"Unexpected output dims (need 2 outputs of rank>=3)", __FILE__, __LINE__);
ANS_DBG("ANSRTRTMPOSE", "InferenceOnBox: outputDims rank issue count=%zu",
outputDims.size());
return {};
}
const int K0 = outputDims[0].d[1];
const int W0 = outputDims[0].d[2];
const int K1 = outputDims[1].d[1];
const int H1 = outputDims[1].d[2];
if (K0 != K1) {
_logger.LogError("ANSRTRTMPOSE::InferenceOnBox",
"SimCC keypoint count mismatch x=" + std::to_string(K0) +
" y=" + std::to_string(K1), __FILE__, __LINE__);
return {};
}
// Pick which output is X (longer of the model-input dims) vs Y.
// For 192x256 input, ext_w=384 is shorter than ext_h=512, but the order
// of the two SimCC outputs in the ONNX is fixed, so we identify by
// length proportional to RTM_INPUT_W vs RTM_INPUT_H rather than by
// assuming x-first.
const std::vector<float>* simcc_x = &featureVectors[0][0];
const std::vector<float>* simcc_y = &featureVectors[0][1];
int extW = W0;
int extH = H1;
const float ratioFirst = (RTM_INPUT_W > 0)
? static_cast<float>(W0) / static_cast<float>(RTM_INPUT_W) : 0.f;
const float ratioSecond = (RTM_INPUT_H > 0)
? static_cast<float>(W0) / static_cast<float>(RTM_INPUT_H) : 0.f;
// If the first output's last-dim looks like H (proportional to input H,
// not W), swap so simcc_x indexes the W-axis.
if (std::abs(ratioFirst - 2.f) > std::abs(ratioSecond - 2.f)) {
std::swap(simcc_x, simcc_y);
extW = H1;
extH = W0;
}
std::lock_guard<std::recursive_mutex> lock(_mutex);
Object obj = PostprocessSimCC(*simcc_x, *simcc_y, K0, extW, extH, meta, camera_id);
std::vector<Object> out;
if (!obj.kps.empty()) out.push_back(std::move(obj));
ANS_DBG("ANSRTRTMPOSE", "InferenceOnBox: kps=%d cam='%s'",
out.empty() ? 0 : K0, camera_id.c_str());
return out;
}
// ====================================================================
// Preprocess — affine crop -> upload BGR (Engine handles RGB norm)
//
// NOTE on color order: Engine<float>::runInference calls blobFromGpuMats
// with swapRB=false (see EngineRunInference.inl). So the GpuMat we hand
// it is interpreted channel-by-channel as the model input. RTMPose was
// trained on RGB, so we cvtColor BGR -> RGB before upload (mirroring
// ANSTENSORRTPOSE / ANSRTYOLO).
// ====================================================================
std::vector<std::vector<cv::cuda::GpuMat>> ANSRTRTMPOSE::Preprocess(
const cv::Mat& inputImage, const cv::Rect& box, ImageMetadata& outMeta) {
try {
if (!_licenseValid) {
_logger.LogFatal("ANSRTRTMPOSE::Preprocess", "Invalid license", __FILE__, __LINE__);
return {};
}
cv::Mat srcImg = inputImage;
if (srcImg.channels() == 1) {
cv::cvtColor(srcImg, srcImg, cv::COLOR_GRAY2BGR);
}
outMeta.imgHeight = static_cast<float>(srcImg.rows);
outMeta.imgWidth = static_cast<float>(srcImg.cols);
outMeta.personBox = box;
if (outMeta.imgHeight <= 0 || outMeta.imgWidth <= 0) return {};
cv::Mat crop;
CropImageByDetectBox(srcImg, box, crop, outMeta.affine_rev);
if (crop.empty()) {
ANS_DBG("ANSRTRTMPOSE", "Preprocess: warpAffine produced empty crop");
return {};
}
cv::Mat rgb;
cv::cvtColor(crop, rgb, cv::COLOR_BGR2RGB);
cv::cuda::Stream stream;
cv::cuda::GpuMat gpuRGB;
gpuRGB.upload(rgb, stream);
stream.waitForCompletion();
std::vector<cv::cuda::GpuMat> input{ std::move(gpuRGB) };
std::vector<std::vector<cv::cuda::GpuMat>> inputs{ std::move(input) };
return inputs;
}
catch (const std::exception& e) {
_logger.LogWarn("ANSRTRTMPOSE::Preprocess",
std::string("Skipped frame: ") + e.what(), __FILE__, __LINE__);
ANS_DBG("ANSRTRTMPOSE", "Preprocess: EXCEPTION '%s'", e.what());
return {};
}
}
// ====================================================================
// PostprocessSimCC
//
// For each keypoint k:
// px = argmax(simcc_x[k]) ; sx = simcc_x[k][px]
// py = argmax(simcc_y[k]) ; sy = simcc_y[k][py]
// score = max(sx, sy)
// raw_x = px * 0.5 ; raw_y = py * 0.5
// [x, y]^T = affine_rev * [raw_x, raw_y, 1]^T
//
// This matches RTMPose::Inference in rtmpose_tensorrt.cpp.
// ====================================================================
Object ANSRTRTMPOSE::PostprocessSimCC(const std::vector<float>& simcc_x,
const std::vector<float>& simcc_y,
int numKeypoints, int extW, int extH,
const ImageMetadata& meta,
const std::string& camera_id) {
Object obj{};
if (numKeypoints <= 0 || extW <= 0 || extH <= 0) return obj;
if (static_cast<int>(simcc_x.size()) < numKeypoints * extW ||
static_cast<int>(simcc_y.size()) < numKeypoints * extH) {
_logger.LogError("ANSRTRTMPOSE::PostprocessSimCC",
"SimCC buffer underflow: x=" + std::to_string(simcc_x.size()) +
" y=" + std::to_string(simcc_y.size()) +
" expected " + std::to_string(numKeypoints * extW) + "/" +
std::to_string(numKeypoints * extH), __FILE__, __LINE__);
return obj;
}
const double m0 = meta.affine_rev.at<double>(0, 0);
const double m1 = meta.affine_rev.at<double>(0, 1);
const double m2 = meta.affine_rev.at<double>(0, 2);
const double m3 = meta.affine_rev.at<double>(1, 0);
const double m4 = meta.affine_rev.at<double>(1, 1);
const double m5 = meta.affine_rev.at<double>(1, 2);
std::vector<float> kps;
kps.reserve(numKeypoints * 3);
std::vector<cv::Point2f> polygon;
polygon.reserve(numKeypoints);
std::stringstream xss, yss;
float meanScore = 0.f;
int meanCount = 0;
for (int i = 0; i < numKeypoints; ++i) {
const float* xPtr = simcc_x.data() + static_cast<size_t>(i) * extW;
const float* yPtr = simcc_y.data() + static_cast<size_t>(i) * extH;
const int px = static_cast<int>(std::max_element(xPtr, xPtr + extW) - xPtr);
const int py = static_cast<int>(std::max_element(yPtr, yPtr + extH) - yPtr);
const float score = std::max(xPtr[px], yPtr[py]);
// SimCC oversampling factor 2 — same constant as the reference impl.
const double raw_x = px * 0.5;
const double raw_y = py * 0.5;
float worldX = static_cast<float>(m0 * raw_x + m1 * raw_y + m2);
float worldY = static_cast<float>(m3 * raw_x + m4 * raw_y + m5);
// Clamp into image bounds so downstream code can index safely.
worldX = std::clamp(worldX, 0.f, meta.imgWidth - 1.f);
worldY = std::clamp(worldY, 0.f, meta.imgHeight - 1.f);
kps.push_back(worldX);
kps.push_back(worldY);
kps.push_back(score);
polygon.emplace_back(worldX, worldY);
xss << worldX;
yss << worldY;
if (i + 1 < numKeypoints) { xss << ";"; yss << ";"; }
if (score >= KPS_THRESHOLD) { meanScore += score; ++meanCount; }
}
obj.classId = 0;
obj.className = _classes.empty() ? std::string("person") : _classes.front();
obj.confidence = (meanCount > 0) ? meanScore / meanCount : 0.f;
obj.box = meta.personBox;
obj.kps = std::move(kps);
obj.polygon = std::move(polygon);
obj.cameraId = camera_id;
obj.extraInfo = xss.str() + "|" + yss.str();
return obj;
}
// ====================================================================
// Affine helpers — direct port of utils.h / rtmpose_tensorrt.cpp
// ====================================================================
cv::Mat ANSRTRTMPOSE::GetAffineTransform(float center_x, float center_y,
float scale_w, float scale_h,
int output_w, int output_h, bool inverse) {
cv::Point2f src1(center_x, center_y);
cv::Point2f src2(center_x, center_y - scale_w * 0.5f);
cv::Point2f src3(src2.x - (src1.y - src2.y),
src2.y + (src1.x - src2.x));
const float dcx = output_w * 0.5f;
const float dcy = output_h * 0.5f;
cv::Point2f dst1(dcx, dcy);
cv::Point2f dst2(dcx, dcy - output_w * 0.5f);
cv::Point2f dst3(dst2.x - (dst1.y - dst2.y),
dst2.y + (dst1.x - dst2.x));
cv::Point2f srcPts[3] = { src1, src2, src3 };
cv::Point2f dstPts[3] = { dst1, dst2, dst3 };
return inverse ? cv::getAffineTransform(dstPts, srcPts)
: cv::getAffineTransform(srcPts, dstPts);
}
void ANSRTRTMPOSE::CropImageByDetectBox(const cv::Mat& image, const cv::Rect& box,
cv::Mat& crop, cv::Mat& affine_rev) {
// Aspect-fit the model's W:H (e.g. 192:256) over the bbox, then expand
// by 1.2x for context — same heuristic as the reference RTMPose impl.
float w = static_cast<float>(box.width);
float h = static_cast<float>(box.height);
const float cx = box.x + w * 0.5f;
const float cy = box.y + h * 0.5f;
const float aspect = static_cast<float>(RTM_INPUT_W) /
static_cast<float>(RTM_INPUT_H);
if (w > aspect * h) h = w / aspect;
else w = h * aspect;
const float scale_w = w * 1.2f;
const float scale_h = h * 1.2f;
cv::Mat affine = GetAffineTransform(cx, cy, scale_w, scale_h,
RTM_INPUT_W, RTM_INPUT_H, false);
affine_rev = GetAffineTransform(cx, cy, scale_w, scale_h,
RTM_INPUT_W, RTM_INPUT_H, true);
cv::warpAffine(image, crop, affine,
cv::Size(RTM_INPUT_W, RTM_INPUT_H), cv::INTER_LINEAR);
}
// ====================================================================
// TryLoadPersonDetector
//
// Look for person.onnx + person.names alongside the pose model. If
// both exist, instantiate an ANSRTYOLO front-end and load it via
// LoadModelFromFolder (which builds / loads the TRT engine from
// cache). On success m_personDetector is non-null and RunInference
// switches to the proper top-down pipeline.
//
// Missing files are NOT an error — the wrapper is designed to work
// standalone for single-person test inputs. We log a debug line so
// it's clear which path is active and return false.
// ====================================================================
bool ANSRTRTMPOSE::TryLoadPersonDetector(const std::string& modelFolder) {
const std::string personOnnx = CreateFilePath(modelFolder, "person.onnx");
const std::string personNames = CreateFilePath(modelFolder, "person.names");
if (!FileExist(personOnnx) || !FileExist(personNames)) {
ANS_DBG("ANSRTRTMPOSE",
"TryLoadPersonDetector: person.onnx/person.names not in '%s' "
"-- standalone mode (whole-image pose)", modelFolder.c_str());
return false;
}
if (m_personDetector) {
ANS_DBG("ANSRTRTMPOSE", "TryLoadPersonDetector: already loaded, skipping");
return true;
}
try {
auto detector = std::make_unique<ANSRTYOLO>();
detector->SetLoadEngineOnCreation(this->_loadEngineOnCreation);
// SetMaxSlotsPerGpu is declared private in ANSRTYOLO (it's a
// private override of ANSODBase's public virtual). Call it via
// the base-class interface where access is public.
static_cast<ANSODBase*>(detector.get())->SetMaxSlotsPerGpu(m_maxSlotsPerGpu);
ModelConfig personCfg{};
personCfg.detectionType = DetectionType::DETECTION;
personCfg.modelType = ModelType::RTYOLO;
personCfg.precisionType = PrecisionType::FP16;
personCfg.inpHeight = 640;
personCfg.inpWidth = 640;
personCfg.detectionScoreThreshold = 0.5f;
personCfg.modelConfThreshold = 0.5f;
personCfg.modelMNSThreshold = 0.45f;
personCfg.numKPS = 0;
personCfg.gpuOptBatchSize = 1;
personCfg.gpuMaxBatchSize = 1;
personCfg.gpuDeviceIndex = m_options.deviceIndex;
std::string personLabelMap;
const bool ok = detector->LoadModelFromFolder(
/*licenseKey*/ _licenseKey,
personCfg,
/*modelName*/ "person",
/*className*/ "person.names",
/*modelFolder*/ modelFolder,
personLabelMap);
if (!ok) {
_logger.LogWarn("ANSRTRTMPOSE::TryLoadPersonDetector",
"Person detector LoadModelFromFolder FAILED -- falling back to whole-image pose",
__FILE__, __LINE__);
ANS_DBG("ANSRTRTMPOSE", "TryLoadPersonDetector: detector load FAILED");
return false;
}
m_personDetector = std::move(detector);
ANS_DBG("ANSRTRTMPOSE",
"TryLoadPersonDetector: detector loaded OK from '%s' (labels='%s')",
modelFolder.c_str(), personLabelMap.c_str());
return true;
}
catch (const std::exception& e) {
_logger.LogWarn("ANSRTRTMPOSE::TryLoadPersonDetector", e.what(), __FILE__, __LINE__);
ANS_DBG("ANSRTRTMPOSE", "TryLoadPersonDetector: EXCEPTION '%s'", e.what());
m_personDetector.reset();
return false;
}
}
} // namespace ANSCENTER
+141
View File
@@ -0,0 +1,141 @@
#ifndef ANSRTRTMPOSE_H
#define ANSRTRTMPOSE_H
#pragma once
#include "ANSEngineCommon.h"
#include "engine.h"
#include "ANSGpuFrameRegistry.h"
#include "engine/EnginePoolManager.h"
#include "ANSRTYOLO.h" // internal person-detector front-end (top-down pipeline)
namespace ANSCENTER {
// ====================================================================
// ANSRTRTMPOSE — TensorRT-based RTMPose (SimCC) pose estimator
//
// Ported from C:\Projects\AIVISION\RTM_POSE\TensorRT\rtmpose_tensorrt.cpp
// but uses Engine<float> from TensorRTAPI (shared pool / pinned upload /
// multi-GPU dispatch / engine cache) instead of raw nvinfer1.
//
// RTMPose is a top-down single-person pose estimator:
// Input : person crop, 192x256 RGB, ImageNet-normalized
// Output : two SimCC heatmaps
// simcc_x [1, K, ext_w]
// simcc_y [1, K, ext_h]
// K = number of keypoints (e.g. 17 / 26 / 133)
// ext_w / ext_h = SimCC oversampled dims (typ. 2 * input)
//
// RunInference(image) treats the whole input as one person crop. To run
// on multiple people, call ANSODBase::RunInference(image, bboxes, cam) —
// the base class crops each bbox and dispatches to RunInference per crop.
// ====================================================================
class ANSENGINE_API ANSRTRTMPOSE : public ANSODBase {
public:
bool Initialize(std::string licenseKey, ModelConfig modelConfig,
const std::string& modelZipFilePath,
const std::string& modelZipPassword,
std::string& labelMap) override;
bool LoadModel(const std::string& modelZipFilePath,
const std::string& modelZipPassword) override;
bool LoadModelFromFolder(std::string licenseKey, ModelConfig modelConfig,
std::string modelName, std::string className,
const std::string& modelFolder,
std::string& labelMap) override;
bool OptimizeModel(bool fp16, std::string& optimizedModelFolder) override;
std::vector<Object> RunInference(const cv::Mat& input);
std::vector<Object> RunInference(const cv::Mat& input,
const std::string& camera_id);
bool Destroy();
~ANSRTRTMPOSE();
private:
std::string _modelFilePath;
bool _modelLoadValid{ false };
bool _fp16{ false };
int m_maxSlotsPerGpu{ 1 };
void SetMaxSlotsPerGpu(int n) override { m_maxSlotsPerGpu = n; }
// Per-call image geometry — captured before inference, consumed at postprocess.
// affine_rev maps SimCC output coords (in 192x256 model space) back to
// the original image. Carrying this per-call keeps the class re-entrant.
struct ImageMetadata {
cv::Mat affine_rev;
float imgWidth = 0.f;
float imgHeight = 0.f;
cv::Rect personBox; // bbox the affine was built from
};
// Single-bbox pipeline: affine-crop -> upload -> Engine.runInference -> SimCC decode
std::vector<Object> InferenceOnBox(const cv::Mat& input,
const cv::Rect& box,
const std::string& camera_id);
// Build the 192x256 RGB crop around `box` and upload to GPU. outMeta
// captures the inverse-affine and original image size for postprocess.
std::vector<std::vector<cv::cuda::GpuMat>> Preprocess(const cv::Mat& inputImage,
const cv::Rect& box,
ImageMetadata& outMeta);
// Decode SimCC outputs into one Object with kps in original-image coords.
Object PostprocessSimCC(const std::vector<float>& simcc_x,
const std::vector<float>& simcc_y,
int numKeypoints, int extW, int extH,
const ImageMetadata& meta,
const std::string& camera_id);
// Affine helpers — direct port of utils.h::GetAffineTransform and
// rtmpose_tensorrt.cpp::CropImageByDetectBox.
static cv::Mat GetAffineTransform(float center_x, float center_y,
float scale_w, float scale_h,
int output_w, int output_h,
bool inverse = false);
void CropImageByDetectBox(const cv::Mat& image, const cv::Rect& box,
cv::Mat& crop, cv::Mat& affine_rev);
// ── TensorRT engine (shared pool, same pattern as ANSRTYOLO) ─────
std::shared_ptr<Engine<float>> m_trtEngine = nullptr;
EnginePoolManager<float>::PoolKey m_poolKey;
bool m_usingSharedPool = false;
// ── Optional internal person detector (top-down pipeline) ───────
// RTMPose is a top-down single-person estimator — feeding it whole
// scenes produces noise. If the model folder bundles person.onnx +
// person.names alongside the pose model, we auto-load a YOLO person
// detector here and run pose ONCE PER DETECTED PERSON. When those
// files are missing the wrapper falls back to whole-image pose so
// that single-person test inputs (selfies, fitness clips) still work.
std::unique_ptr<ANSRTYOLO> m_personDetector;
// Look for person.onnx + person.names in `modelFolder`. Loads
// m_personDetector and returns true on success; returns false
// (without logging an error) if either file is missing — caller
// should treat that as "no detector, fall back to whole-image".
bool TryLoadPersonDetector(const std::string& modelFolder);
ANSCENTER::Options m_options;
// Fixed RTMPose model geometry (overridden from the engine's input dims
// at LoadModel time once the network is built).
int RTM_INPUT_W{ 192 };
int RTM_INPUT_H{ 256 };
// ImageNet normalization, expressed for Engine<T>::blobFromGpuMats:
// blob = (pixel/255 - subVals) / divVals
// subVals = mean / 255 ; divVals = std / 255
const std::array<float, 3> SUB_VALS{ 0.485f, 0.456f, 0.406f };
const std::array<float, 3> DIV_VALS{ 0.229f, 0.224f, 0.225f };
const bool NORMALIZE = true;
// Keypoint confidence floor for output. Mirrors KPS_THRESHOLD from
// ANSTENSORRTPOSE — keypoints below threshold are still emitted (the
// SimCC max is taken regardless), but caller can read kps[2] (score)
// to decide whether to draw them.
float KPS_THRESHOLD{ 0.3f };
};
} // namespace ANSCENTER
#endif
+5 -3
View File
@@ -518,14 +518,16 @@ namespace ANSCENTER {
}
}
std::vector<Object> YOLO12OD::postprocess(const cv::Size& originalImageSize, const cv::Size& resizedImageShape,
const std::vector<Ort::Value>& outputTensors,
float confThreshold, float iouThreshold)
std::vector<Ort::Value>& outputTensors,
float confThreshold, float iouThreshold)
{
std::lock_guard<std::recursive_mutex> lock(_mutex);
try {
std::vector<Object> detections;
const float* rawOutput = outputTensors[0].GetTensorData<float>(); // Extract raw output data from the first output tensor
// GetTensorMutableData (not GetTensorData) on DML — const variant
// hangs on AMD after ~8 calls. Read-only despite the name.
const float* rawOutput = outputTensors[0].GetTensorMutableData<float>();
const std::vector<int64_t> outputShape = outputTensors[0].GetTensorTypeAndShapeInfo().GetShape();
// Determine the number of features and detections
+4 -1
View File
@@ -49,8 +49,11 @@ namespace ANSCENTER {
std::vector<Object> detect(const cv::Mat& image, float confThreshold = 0.4f, float iouThreshold = 0.45f);
//cv::Mat preprocess(const cv::Mat& image, float*& blob, std::vector<int64_t>& inputTensorShape);
cv::Mat preprocess(const cv::Mat& image, std::vector<float>& blob, std::vector<int64_t>& inputTensorShape);
// outputTensors is non-const because GetTensorMutableData()
// requires a non-const Ort::Value receiver — see ANSONNXCL.h
// for full note.
std::vector<Object> postprocess(const cv::Size& originalImageSize, const cv::Size& resizedImageShape,
const std::vector<Ort::Value>& outputTensors,
std::vector<Ort::Value>& outputTensors,
float confThreshold, float iouThreshold);
private:
+2
View File
@@ -18,6 +18,7 @@ set(ANSOD_SOURCES
ANSONNXCL.cpp
ANSONNXOBB.cpp
ANSONNXPOSE.cpp
ANSONNXRTMPOSE.cpp
ANSONNXSAM3.cpp
ANSONNXSEG.cpp
ANSONNXYOLO.cpp
@@ -27,6 +28,7 @@ set(ANSOD_SOURCES
ANSOVSEG.cpp
ANSPOSE.cpp
ANSRTYOLO.cpp
ANSRTRTMPOSE.cpp
ANSSAM.cpp
ANSSAM3.cpp
ANSTENSORRTCL.cpp
+109 -17
View File
@@ -52,6 +52,8 @@ extern "C" ANSODENGINE_API GpuFrameData** ANSODEngine_GetTlsGpuFrame() {
#include "ANSONNXSAM3.h"
#include "ANSRTYOLO.h"
#include "ANSONNXYOLO.h"
#include "ANSRTRTMPOSE.h"
#include "ANSONNXRTMPOSE.h"
#include <pipelines/metadata.h>
#include <models/input_data.h>
#include "utils/visualizer.hpp"
@@ -441,10 +443,20 @@ extern "C" ANSODENGINE_API std::string CreateANSODHandle(ANSCENTER::ANSODBase**
modelType = onNvidia ? 31 /* RTYOLO */ : 30 /* ONNXYOLO */;
break;
// ── Pose ─────────────────────────────────────────────────────────────
case 21: // ONNXPOSE
case 22: // RTPOSE
modelType = onNvidia ? 22 /* RTPOSE */ : 21 /* ONNXPOSE */;
break;
// Intentionally NOT auto-routed. The user picks the backend explicitly:
// 21 → ANSONNXPOSE (ONNX Runtime)
// 22 → ANSTENSORRTPOSE (TensorRT, falls back to ONNX on non-NVIDIA
// via the case-22 body)
// Removing the rewrite makes 21 vs 22 comparisons meaningful — the
// previous behavior silently upgraded 21 → 22 on any NVIDIA box.
// ── RTMPose (top-down SimCC pose) ────────────────────────────────────
// Intentionally NOT auto-routed (unlike 21/22 ONNXPOSE/RTPOSE).
// The user picks the backend explicitly:
// 32 → ANSONNXRTMPOSE (ONNX Runtime)
// 33 → ANSRTRTMPOSE (TensorRT, falls back to ONNX on non-NVIDIA
// via the intra-case check inside case 33)
// This keeps 32 vs 33 comparisons meaningful — without it, picking
// 32 on an NVIDIA box silently gets rewritten to 33 here.
// ── Segmentation ─────────────────────────────────────────────────────
case 23: // ONNXSEG
case 24: // RTSEG
@@ -699,6 +711,20 @@ extern "C" ANSODENGINE_API std::string CreateANSODHandle(ANSCENTER::ANSODBase**
modelConfig.modelType = ANSCENTER::ModelType::ONNXYOLO;
}
break;
case 32: //ONNXRTMPOSE
(*Handle) = new ANSCENTER::ANSONNXRTMPOSE();
modelConfig.modelType = ANSCENTER::ModelType::ONNXRTMPOSE;
break;
case 33: //RTRTMPOSE
if (engineType == ANSCENTER::EngineType::NVIDIA_GPU) {
(*Handle) = new ANSCENTER::ANSRTRTMPOSE();
modelConfig.modelType = ANSCENTER::ModelType::RTRTMPOSE;
}
else {
(*Handle) = new ANSCENTER::ANSONNXRTMPOSE();
modelConfig.modelType = ANSCENTER::ModelType::ONNXRTMPOSE;
}
break;
default:
(*Handle) = new ANSCENTER::ANSFD();
modelConfig.modelType = ANSCENTER::ModelType::FACEDETECT;
@@ -799,10 +825,20 @@ extern "C" ANSODENGINE_API int CreateANSODHandleEx(ANSCENTER::ANSODBase** Handl
modelType = onNvidia ? 31 /* RTYOLO */ : 30 /* ONNXYOLO */;
break;
// ── Pose ─────────────────────────────────────────────────────────────
case 21: // ONNXPOSE
case 22: // RTPOSE
modelType = onNvidia ? 22 /* RTPOSE */ : 21 /* ONNXPOSE */;
break;
// Intentionally NOT auto-routed. The user picks the backend explicitly:
// 21 → ANSONNXPOSE (ONNX Runtime)
// 22 → ANSTENSORRTPOSE (TensorRT, falls back to ONNX on non-NVIDIA
// via the case-22 body)
// Removing the rewrite makes 21 vs 22 comparisons meaningful — the
// previous behavior silently upgraded 21 → 22 on any NVIDIA box.
// ── RTMPose (top-down SimCC pose) ────────────────────────────────────
// Intentionally NOT auto-routed (unlike 21/22 ONNXPOSE/RTPOSE).
// The user picks the backend explicitly:
// 32 → ANSONNXRTMPOSE (ONNX Runtime)
// 33 → ANSRTRTMPOSE (TensorRT, falls back to ONNX on non-NVIDIA
// via the intra-case check inside case 33)
// This keeps 32 vs 33 comparisons meaningful — without it, picking
// 32 on an NVIDIA box silently gets rewritten to 33 here.
// ── Segmentation ─────────────────────────────────────────────────────
case 23: // ONNXSEG
case 24: // RTSEG
@@ -1066,6 +1102,20 @@ extern "C" ANSODENGINE_API int CreateANSODHandleEx(ANSCENTER::ANSODBase** Handl
modelConfig.modelType = ANSCENTER::ModelType::ONNXYOLO;
}
break;
case 32: //ONNXRTMPOSE
(*Handle) = new ANSCENTER::ANSONNXRTMPOSE();
modelConfig.modelType = ANSCENTER::ModelType::ONNXRTMPOSE;
break;
case 33: //RTRTMPOSE
if (engineType == ANSCENTER::EngineType::NVIDIA_GPU) {
(*Handle) = new ANSCENTER::ANSRTRTMPOSE();
modelConfig.modelType = ANSCENTER::ModelType::RTRTMPOSE;
}
else {
(*Handle) = new ANSCENTER::ANSONNXRTMPOSE();
modelConfig.modelType = ANSCENTER::ModelType::ONNXRTMPOSE;
}
break;
default:
(*Handle) = new ANSCENTER::ANSFD();
modelConfig.modelType = ANSCENTER::ModelType::FACEDETECT;
@@ -1203,10 +1253,19 @@ extern "C" __declspec(dllexport) int LoadModelFromFolder(ANSCENTER::ANSODBase**
modelType = onNvidia ? 31 /* RTYOLO */ : 30 /* ONNXYOLO */;
break;
// ── Pose ─────────────────────────────────────────────────────────
case 21: // ONNXPOSE
case 22: // RTPOSE
modelType = onNvidia ? 22 /* RTPOSE */ : 21 /* ONNXPOSE */;
break;
// Intentionally NOT auto-routed. The user picks the backend explicitly:
// 21 → ANSONNXPOSE (ONNX Runtime)
// 22 → ANSTENSORRTPOSE (TensorRT, falls back to ONNX on
// non-NVIDIA via the case-22 body)
// Removing the rewrite makes 21 vs 22 comparisons meaningful.
// ── RTMPose (top-down SimCC pose) ────────────────────────────────
// Intentionally NOT auto-routed (unlike 21/22 ONNXPOSE/RTPOSE).
// The user picks the backend explicitly:
// 32 → ANSONNXRTMPOSE (ONNX Runtime)
// 33 → ANSRTRTMPOSE (TensorRT, falls back to ONNX on
// non-NVIDIA via the case-33 body)
// This keeps 32 vs 33 comparisons meaningful — without it,
// picking 32 on an NVIDIA box silently gets rewritten to 33.
// ── Segmentation ─────────────────────────────────────────────────
case 23: // ONNXSEG
case 24: // RTSEG
@@ -1459,6 +1518,20 @@ extern "C" __declspec(dllexport) int LoadModelFromFolder(ANSCENTER::ANSODBase**
modelConfig.modelType = ANSCENTER::ModelType::ONNXYOLO;
}
break;
case 32: //ONNXRTMPOSE
(*Handle) = new ANSCENTER::ANSONNXRTMPOSE();
modelConfig.modelType = ANSCENTER::ModelType::ONNXRTMPOSE;
break;
case 33: //RTRTMPOSE
if (engineType == ANSCENTER::EngineType::NVIDIA_GPU) {
(*Handle) = new ANSCENTER::ANSRTRTMPOSE();
modelConfig.modelType = ANSCENTER::ModelType::RTRTMPOSE;
}
else {
(*Handle) = new ANSCENTER::ANSONNXRTMPOSE();
modelConfig.modelType = ANSCENTER::ModelType::ONNXRTMPOSE;
}
break;
default:
(*Handle) = new ANSCENTER::ANSFD();
modelConfig.modelType = ANSCENTER::ModelType::FACEDETECT;
@@ -1526,10 +1599,19 @@ ANSODENGINE_API int OptimizeModelStr(const char* modelFilePath, const char* mode
modelType = onNvidia ? 31 /* RTYOLO */ : 30 /* ONNXYOLO */;
break;
// ── Pose ─────────────────────────────────────────────────────────
case 21: // ONNXPOSE
case 22: // RTPOSE
modelType = onNvidia ? 22 /* RTPOSE */ : 21 /* ONNXPOSE */;
break;
// Intentionally NOT auto-routed. The user picks the backend explicitly:
// 21 → ANSONNXPOSE (ONNX Runtime)
// 22 → ANSTENSORRTPOSE (TensorRT, falls back to ONNX on
// non-NVIDIA via the case-22 body)
// Removing the rewrite makes 21 vs 22 comparisons meaningful.
// ── RTMPose (top-down SimCC pose) ────────────────────────────────
// Intentionally NOT auto-routed (unlike 21/22 ONNXPOSE/RTPOSE).
// The user picks the backend explicitly:
// 32 → ANSONNXRTMPOSE (ONNX Runtime)
// 33 → ANSRTRTMPOSE (TensorRT, falls back to ONNX on
// non-NVIDIA via the case-33 body)
// This keeps 32 vs 33 comparisons meaningful — without it,
// picking 32 on an NVIDIA box silently gets rewritten to 33.
// ── Segmentation ─────────────────────────────────────────────────
case 23: // ONNXSEG
case 24: // RTSEG
@@ -1545,7 +1627,7 @@ ANSODENGINE_API int OptimizeModelStr(const char* modelFilePath, const char* mode
}
}
if (modelType == 31) // If modelType is RTYOLO (31), handle separately.
{
if (engineType == ANSCENTER::EngineType::NVIDIA_GPU) {
@@ -1596,6 +1678,16 @@ ANSODENGINE_API int OptimizeModelStr(const char* modelFilePath, const char* mode
}
break;
}
case 32: Handle = std::make_unique<ANSCENTER::ANSONNXRTMPOSE>(); modelConfig.modelType = ANSCENTER::ModelType::ONNXRTMPOSE; break;
case 33: {
if (engineType == ANSCENTER::EngineType::NVIDIA_GPU) {
Handle = std::make_unique<ANSCENTER::ANSRTRTMPOSE>(); modelConfig.modelType = ANSCENTER::ModelType::RTRTMPOSE;
}
else {
Handle = std::make_unique<ANSCENTER::ANSONNXRTMPOSE>(); modelConfig.modelType = ANSCENTER::ModelType::ONNXRTMPOSE;
}
break;
}
default: {
if (modelDetectionType == 0) // classification
@@ -17,6 +17,9 @@
#include <tlhelp32.h>
#include <unordered_map>
#include <condition_variable>
#include <sstream>
#include <algorithm>
#include <cctype>
// DebugView: filter on "[ANSTRE]" — gated by ANSCORE_DEBUGVIEW in ANSLicense.h.
@@ -164,6 +167,102 @@ namespace ANSCENTER
CloseHandle(snapshot);
return isRunning;
}
bool ANSTRE::GetDeviceList(std::string& deviceList) {
deviceList.clear();
const char* cmd = "nvidia-smi --query-gpu=index --format=csv,noheader";
ANS_DBG("ANSTRE", "GetDeviceList: ENTER, cmd=\"%s\"", cmd);
// Unconditional console trace — visible without enabling the DebugView gate.
// std::endl flushes so output is seen even if the host process crashes shortly after.
std::cout << "[ANSTRE][GetDeviceList] ENTER, cmd=\"" << cmd << "\"" << std::endl;
// `nvidia-smi --query-gpu=index --format=csv,noheader` prints one integer per
// detected GPU. If the binary is missing, cmd.exe writes an error message to
// our captured stderr instead — none of those lines parse as a bare integer,
// so the count below stays at 0 and we return false.
const std::string raw = ExecuteCommand(cmd);
ANS_DBG("ANSTRE", "GetDeviceList: nvidia-smi returned %zu bytes", raw.size());
std::cout << "[ANSTRE][GetDeviceList] nvidia-smi returned " << raw.size() << " bytes" << std::endl;
// Dump the raw output (newlines flattened to '|') so DebugView shows it on one
// line — invaluable for diagnosing missing driver / PATH / permission issues
// on customer machines without round-trips for rebuilds.
if (!raw.empty()) {
std::string oneLine = raw;
std::replace(oneLine.begin(), oneLine.end(), '\r', ' ');
std::replace(oneLine.begin(), oneLine.end(), '\n', '|');
ANS_DBG("ANSTRE", "GetDeviceList: raw output: %s", oneLine.c_str());
std::cout << "[ANSTRE][GetDeviceList] raw output: " << oneLine << std::endl;
}
if (raw.empty()) {
ANS_DBG("ANSTRE", "GetDeviceList: EXIT false (nvidia-smi produced no output -> no NVIDIA GPU)");
std::cout << "[ANSTRE][GetDeviceList] EXIT false (nvidia-smi produced no output -> no NVIDIA GPU)" << std::endl;
return false;
}
std::vector<int> indices;
std::istringstream iss(raw);
std::string line;
int lineNo = 0;
while (std::getline(iss, line)) {
++lineNo;
// Trim whitespace (incl. CR from Windows line endings).
const auto first = line.find_first_not_of(" \t\r\n");
if (first == std::string::npos) continue;
const auto last = line.find_last_not_of(" \t\r\n");
const std::string trimmed = line.substr(first, last - first + 1);
if (trimmed.empty()) continue;
if (!std::all_of(trimmed.begin(), trimmed.end(), [](unsigned char c){ return std::isdigit(c) != 0; })) {
// Non-numeric line — likely an error message from cmd.exe ("'nvidia-smi'
// is not recognized...") or a driver warning. Skip it but log it so the
// reason for a no-GPU verdict is visible.
ANS_DBG("ANSTRE", "GetDeviceList: skip non-numeric line #%d: \"%s\"", lineNo, trimmed.c_str());
std::cout << "[ANSTRE][GetDeviceList] skip non-numeric line #" << lineNo << ": \"" << trimmed << "\"" << std::endl;
continue;
}
try {
int idx = std::stoi(trimmed);
indices.push_back(idx);
ANS_DBG("ANSTRE", "GetDeviceList: parsed GPU index %d (line #%d)", idx, lineNo);
std::cout << "[ANSTRE][GetDeviceList] parsed GPU index " << idx << " (line #" << lineNo << ")" << std::endl;
} catch (const std::exception& e) {
// stoi can throw on overflow — ignore and keep parsing.
ANS_DBG("ANSTRE", "GetDeviceList: stoi failed on line #%d \"%s\": %s", lineNo, trimmed.c_str(), e.what());
std::cout << "[ANSTRE][GetDeviceList] stoi failed on line #" << lineNo << " \"" << trimmed << "\": " << e.what() << std::endl;
}
}
if (indices.empty()) {
ANS_DBG("ANSTRE", "GetDeviceList: EXIT false (nvidia-smi output had no numeric GPU indices -> no NVIDIA GPU)");
std::cout << "[ANSTRE][GetDeviceList] EXIT false (nvidia-smi output had no numeric GPU indices -> no NVIDIA GPU)" << std::endl;
return false;
}
std::ostringstream oss;
oss << "device=";
for (size_t i = 0; i < indices.size(); ++i) {
if (i > 0) oss << ",";
oss << indices[i];
}
deviceList = oss.str();
ANS_DBG("ANSTRE", "GetDeviceList: EXIT true, found %zu GPU(s) -> \"%s\"", indices.size(), deviceList.c_str());
std::cout << "[ANSTRE][GetDeviceList] EXIT true, found " << indices.size() << " GPU(s) -> \"" << deviceList << "\"" << std::endl;
return true;
}
std::string ANSTRE::GetDeviceArgOrDefault() {
std::string list;
if (GetDeviceList(list)) {
const std::string arg = " " + list;
ANS_DBG("ANSTRE", "GetDeviceArgOrDefault: using \"%s\"", arg.c_str());
std::cout << "[ANSTRE][GetDeviceArgOrDefault] using \"" << arg << "\"" << std::endl;
return arg;
}
ANS_DBG("ANSTRE", "GetDeviceArgOrDefault: no NVIDIA GPU, using \" device=-1\"");
std::cout << "[ANSTRE][GetDeviceArgOrDefault] no NVIDIA GPU, using \" device=-1\"" << std::endl;
this->_logger.LogWarn("ANSTRE::GetDeviceArgOrDefault",
"No NVIDIA GPU detected, falling back to device=-1", __FILE__, __LINE__);
return " device=-1";
}
bool ANSTRE::ZipExperiment(std::string projectName, int experimentNumber) {
bool zipModelExist = false;
_projectName = Space2Underscore(projectName);
@@ -35,6 +35,14 @@ namespace ANSCENTER {
[[nodiscard]] std::string GetProjectDirectory() { return _projectDirectory; }
[[nodiscard]] bool ZipExperiment(std::string projectName, int experimentNumber);
[[nodiscard]] bool IsProcessRunning(const char* processName);
// Detect NVIDIA GPUs via nvidia-smi. Returns false if no NVIDIA GPU is present
// (or nvidia-smi is unavailable). On success, populates deviceList with a yolo
// CLI-compatible token, e.g. "device=0", "device=0,1", "device=0,1,2,3,4,5".
[[nodiscard]] bool GetDeviceList(std::string& deviceList);
// Yolo-CLI-ready device argument with a leading space, suitable for direct
// concatenation into a yolo command line. Returns " device=0,1,..." when GPUs
// are detected, or " device=-1" as a fallback when none are.
[[nodiscard]] std::string GetDeviceArgOrDefault();
[[nodiscard]] double ParseObjectDetectionMAP(std::string stEvaluationResult);
[[nodiscard]] std::string GetLatestValidFolderAndDeleteOthers(const std::string& projectName, const std::string& rootPath);
protected:
@@ -425,7 +425,7 @@ namespace ANSCENTER
" data=" + "\"" + projectDataPath + "\"" +
" workers=8" +
" batch=" + batch +
" device=0" +
GetDeviceArgOrDefault() +
" epochs=" + epochs +
" plots=False" +
" patience=0" +
@@ -523,7 +523,7 @@ namespace ANSCENTER
" data=" + "\"" + projectDataPath + "\"" +
" workers=8" +
" batch=" + batch +
" device=0" +
GetDeviceArgOrDefault() +
" epochs=" + epochs +
" plots=False"
" patience=0" +
@@ -596,7 +596,7 @@ namespace ANSCENTER
" split=val" +
" workers=8" +
" batch=8" +
" device=0" +
GetDeviceArgOrDefault() +
" conf=0.25" +
" plots=False" +
" mlflow=False" +
@@ -1124,7 +1124,7 @@ namespace ANSCENTER
" split=val" +
" workers=8" +
" batch=8" +
" device=0" +
GetDeviceArgOrDefault() +
" conf=0.25" +
" plots=False" +
" mlflow=False" +
@@ -433,7 +433,7 @@ namespace ANSCENTER
" imgsz=640" +
" workers=8" +
" batch=" + batch +
" device=0" +
GetDeviceArgOrDefault() +
" epochs=" + epochs +
" patience=0" +
" plots=False" +
@@ -533,7 +533,7 @@ namespace ANSCENTER
" imgsz=640" +
" workers=8" +
" batch=" + batch +
" device=0" +
GetDeviceArgOrDefault() +
" epochs=" + epochs +
" patience=0" +
" plots=False" +
@@ -607,7 +607,7 @@ namespace ANSCENTER
" imgsz=640" +
" workers=8" +
" batch=8" +
" device=0" +
GetDeviceArgOrDefault() +
" conf=0.25" +
" plots=False" +
" save_json=False" +
@@ -1213,7 +1213,7 @@ namespace ANSCENTER
" imgsz=640" +
" workers=8" +
" batch=8" +
" device=0" +
GetDeviceArgOrDefault() +
" conf=0.25" +
" plots=False" +
" save_json=False" +
@@ -432,7 +432,7 @@ namespace ANSCENTER
" imgsz=640" +
" workers=8" +
" batch=" + batch +
" device=0" +
GetDeviceArgOrDefault() +
" epochs=" + epochs +
" patience=0" +
" plots=False" +
@@ -531,7 +531,7 @@ namespace ANSCENTER
" imgsz=640" +
" workers=8" +
" batch=" + batch +
" device=0" +
GetDeviceArgOrDefault() +
" epochs=" + epochs +
" patience=0" +
" plots=False" +
@@ -602,7 +602,7 @@ namespace ANSCENTER
" imgsz=640" +
" workers=8" +
" batch=8" +
" device=0" +
GetDeviceArgOrDefault() +
" conf=0.25" +
" plots=False" +
" save_json=False" +
@@ -1207,7 +1207,7 @@ namespace ANSCENTER
" imgsz=640" +
" workers=8" +
" batch=8" +
" device=0" +
GetDeviceArgOrDefault() +
" conf=0.25" +
" plots=False" +
" save_json=False" +
@@ -457,13 +457,13 @@ namespace ANSCENTER
learningRate = cfgMapData.at("lr0");
batch = cfgMapData.at("batch");
}
std::string trainCommand = trainScript +
std::string trainCommand = trainScript +
" detect train model=" + "\"" + preTrainWeightFile + "\"" +
" data=" + "\"" + projectDataPath + "\"" +
" imgsz=640" +
" workers=8" +
" batch=" + batch +
" device=0" +
GetDeviceArgOrDefault() +
" epochs=" + epochs +
" patience=0" +
//" lr0= 0.01" +
@@ -573,7 +573,7 @@ namespace ANSCENTER
" imgsz=640" +
" workers=8" +
" batch=" + batch +
" device=0" +
GetDeviceArgOrDefault() +
" epochs=" + epochs +
" patience=0" +
//" lr0= 0.01" +
@@ -649,7 +649,7 @@ namespace ANSCENTER
" imgsz=640" +
" workers=8" +
" batch=8" +
" device=0" +
GetDeviceArgOrDefault() +
" conf=0.25" +
" plots=False" +
" save_json=False" +
@@ -1265,7 +1265,7 @@ namespace ANSCENTER
" imgsz=640" +
" workers=8" +
" batch=8" +
" device=0" +
GetDeviceArgOrDefault() +
" conf=0.25" +
" plots=False" +
" save_json=False" +
+176 -22
View File
@@ -4,6 +4,7 @@
#include <CkAuthGoogle.h>
#include <CkSocket.h>
#include <CkGlobal.h>
#include <CkHttp.h>
#include <CkMailMan.h>
#include <CkEmail.h>
#include <CkMht.h>
@@ -264,13 +265,21 @@ namespace ANSCENTER
}
// TryConnect — actual connection attempt (caller must hold _configMutex)
//
// First attempts the AWS virtual-host name (s3.<region>.<base>); for
// S3-compatible servers like MinIO/Ceph that hostname does not exist, so
// we fall back to the raw _baseDomain (path-style). The first attempt is
// therefore *expected* to fail on MinIO — we suppress its error-level log
// (logOnFailure=false) so the operator only sees errors when the fallback
// also fails.
bool ANSAWSS3::TryConnect(bool& awsPath) {
auto testConn = CreateConnection();
const bool fallbackPossible = (_fullAWSURL != _baseDomain);
auto testConn = CreateConnection(/*logOnFailure*/ !fallbackPossible);
if (!testConn) {
// Fallback: connect directly to baseDomain (MinIO, Ceph, etc.)
std::string savedFullURL = _fullAWSURL;
_fullAWSURL = _baseDomain;
testConn = CreateConnection();
testConn = CreateConnection(/*logOnFailure*/ true);
if (!testConn) {
_fullAWSURL = savedFullURL; // restore on failure
return false;
@@ -305,11 +314,18 @@ namespace ANSCENTER
attempt++;
// Quick DNS check — is internet available?
// Quick DNS check — is internet available? Try AWS virtual-host
// form first, then fall back to raw _baseDomain for MinIO/Ceph.
{
CkSocket dnsCheck;
CkString dnsResult;
if (!dnsCheck.DnsLookup(_fullAWSURL.c_str(), 3000, dnsResult)) {
bool resolved = dnsCheck.DnsLookup(_fullAWSURL.c_str(), 3000, dnsResult);
if (!resolved && _fullAWSURL != _baseDomain) {
CkSocket dnsCheck2;
CkString dnsResult2;
resolved = dnsCheck2.DnsLookup(_baseDomain.c_str(), 3000, dnsResult2);
}
if (!resolved) {
_logger.LogDebug("ANSAWSS3::RetryLoop",
"Retry #" + std::to_string(attempt) + " - no internet, retrying in 3s",
__FILE__, __LINE__);
@@ -343,7 +359,11 @@ namespace ANSCENTER
}
// ── Connection pool helpers ──
std::unique_ptr<S3Connection> ANSAWSS3::CreateConnection() {
// logOnFailure=false: caller knows this attempt may legitimately fail
// (e.g. TryConnect's first probe of the AWS virtual-host name on a MinIO
// server). Failures are still emitted to ANS_DBG; just not as error-level
// log entries that would scare ops dashboards.
std::unique_ptr<S3Connection> ANSAWSS3::CreateConnection(bool logOnFailure) {
auto conn = std::make_unique<S3Connection>();
// Apply request timeouts BEFORE Connect() so ConnectTimeoutMs takes
@@ -355,8 +375,11 @@ namespace ANSCENTER
// Connect
if (!conn->rest.Connect(_fullAWSURL.c_str(), _port, _bTls, _bAutoReconnect)) {
const std::string summary = SummarizeChilkatError(conn->rest.lastErrorText());
_logger.LogError("ANSAWSS3::CreateConnection", summary, __FILE__, __LINE__);
ANS_DBG("AWSS3", "Connect failed: %s", summary.c_str());
if (logOnFailure) {
_logger.LogError("ANSAWSS3::CreateConnection", summary, __FILE__, __LINE__);
}
ANS_DBG("AWSS3", "Connect failed%s: %s",
logOnFailure ? "" : " (probe, retry expected)", summary.c_str());
return nullptr;
}
@@ -369,8 +392,11 @@ namespace ANSCENTER
conn->socket.put_HttpProxyForHttp(_bProxy);
if (!conn->rest.UseConnection(conn->socket, true)) {
const std::string summary = SummarizeChilkatError(conn->rest.lastErrorText());
_logger.LogError("ANSAWSS3::CreateConnection - Proxy error", summary, __FILE__, __LINE__);
ANS_DBG("AWSS3", "Proxy error: %s", summary.c_str());
if (logOnFailure) {
_logger.LogError("ANSAWSS3::CreateConnection - Proxy error", summary, __FILE__, __LINE__);
}
ANS_DBG("AWSS3", "Proxy error%s: %s",
logOnFailure ? "" : " (probe, retry expected)", summary.c_str());
return nullptr;
}
}
@@ -385,8 +411,11 @@ namespace ANSCENTER
}
if (!conn->rest.SetAuthAws(conn->authAws)) {
const std::string summary = SummarizeChilkatError(conn->rest.lastErrorText());
_logger.LogError("ANSAWSS3::CreateConnection - Auth error", summary, __FILE__, __LINE__);
ANS_DBG("AWSS3", "Auth error: %s", summary.c_str());
if (logOnFailure) {
_logger.LogError("ANSAWSS3::CreateConnection - Auth error", summary, __FILE__, __LINE__);
}
ANS_DBG("AWSS3", "Auth error%s: %s",
logOnFailure ? "" : " (probe, retry expected)", summary.c_str());
return nullptr;
}
}
@@ -678,14 +707,28 @@ namespace ANSCENTER
_pool.clear();
}
// Check if internet is available (fast DNS check)
// Check if internet is available (fast DNS check). Try the AWS
// virtual-host form first; if that synthetic name doesn't exist
// (MinIO, Ceph, custom S3-compatible servers), fall back to the
// raw _baseDomain — that is the actual host the user gave us.
{
CkSocket dnsCheck;
CkString dnsResult;
if (!dnsCheck.DnsLookup(_fullAWSURL.c_str(), 3000, dnsResult)) {
bool resolved = dnsCheck.DnsLookup(_fullAWSURL.c_str(), 3000, dnsResult);
if (!resolved && _fullAWSURL != _baseDomain) {
CkSocket dnsCheck2;
CkString dnsResult2;
resolved = dnsCheck2.DnsLookup(_baseDomain.c_str(), 3000, dnsResult2);
if (resolved) {
ANS_DBG("AWSS3",
"DNS for '%s' failed but '%s' resolved — proceeding with path-style/MinIO fallback",
_fullAWSURL.c_str(), _baseDomain.c_str());
}
}
if (!resolved) {
// No internet — start background retry, return immediately
_logger.LogDebug("ANSAWSS3::Connect",
"No internet available, starting background retry for '" + _fullAWSURL + "'...",
"No internet available, starting background retry for '" + _fullAWSURL + "' / '" + _baseDomain + "'...",
__FILE__, __LINE__);
_stopRetry = false;
@@ -1705,7 +1748,7 @@ namespace ANSCENTER
int statusCode = conn->rest.get_ResponseStatusCode();
if (statusCode == 200) {
std::string scheme = _bTls ? "https://" : "http://";
uploadedFilePath = scheme + _fullAWSURL + "/" + bucketName + "/" + fileName;
uploadedFilePath = scheme + (_bAwsPath ? _fullAWSURL : _baseDomain) + "/" + bucketName + "/" + fileName;
_logger.LogDebug(kOp,
"Successfully uploaded: " + fileName + " | URL: " + uploadedFilePath,
__FILE__, __LINE__);
@@ -1786,7 +1829,7 @@ namespace ANSCENTER
int statusCode = conn->rest.get_ResponseStatusCode();
if (statusCode == 200) {
std::string scheme = _bTls ? "https://" : "http://";
uploadedFilePath = scheme + _fullAWSURL + "/" + bucketName + "/" + fileName;
uploadedFilePath = scheme + (_bAwsPath ? _fullAWSURL : _baseDomain) + "/" + bucketName + "/" + fileName;
_logger.LogDebug(kOp,
"Successfully uploaded: " + fileName + " | URL: " + uploadedFilePath,
__FILE__, __LINE__);
@@ -1890,7 +1933,7 @@ namespace ANSCENTER
// Build canonical path-style URL from parts — independent
// of the addressing style we actually used.
std::string scheme = _bTls ? "https://" : "http://";
uploadedFilePath = scheme + _fullAWSURL + "/" + bucketName + "/" + objectKey;
uploadedFilePath = scheme + (_bAwsPath ? _fullAWSURL : _baseDomain) + "/" + bucketName + "/" + objectKey;
_logger.LogDebug(kOp,
"Successfully uploaded to: " + objectKey + " | URL: " + uploadedFilePath,
__FILE__, __LINE__);
@@ -1959,7 +2002,7 @@ namespace ANSCENTER
int statusCode = conn->rest.get_ResponseStatusCode();
if (statusCode == 200) {
std::string scheme = _bTls ? "https://" : "http://";
uploadedFilePath = scheme + _fullAWSURL + "/" + bucketName + "/" + fileName;
uploadedFilePath = scheme + (_bAwsPath ? _fullAWSURL : _baseDomain) + "/" + bucketName + "/" + fileName;
_logger.LogDebug(kOp,
"Successfully uploaded file: " + fileName + " | URL: " + uploadedFilePath,
__FILE__, __LINE__);
@@ -2170,7 +2213,7 @@ namespace ANSCENTER
}
std::string scheme = _bTls ? "https://" : "http://";
uploadedFilePath = scheme + _fullAWSURL + "/" + bucketName + "/" + fileName;
uploadedFilePath = scheme + (_bAwsPath ? _fullAWSURL : _baseDomain) + "/" + bucketName + "/" + fileName;
_logger.LogDebug(kOp,
"Multipart upload completed for: " + fileName + " | URL: " + uploadedFilePath,
__FILE__, __LINE__);
@@ -2387,7 +2430,7 @@ namespace ANSCENTER
// Build canonical path-style URL from parts — independent
// of the addressing style we actually used.
std::string scheme = _bTls ? "https://" : "http://";
uploadedFilePath = scheme + _fullAWSURL + "/" + bucketName + "/" + objectKey;
uploadedFilePath = scheme + (_bAwsPath ? _fullAWSURL : _baseDomain) + "/" + bucketName + "/" + objectKey;
_logger.LogDebug(kOp,
"Multipart upload completed. S3 key: " + objectKey + " | URL: " + uploadedFilePath,
__FILE__, __LINE__);
@@ -2451,7 +2494,7 @@ namespace ANSCENTER
int statusCode = conn->rest.get_ResponseStatusCode();
if (statusCode == 200) {
std::string scheme = _bTls ? "https://" : "http://";
uploadedFilePath = scheme + _fullAWSURL + "/" + bucketName + "/" + fileName;
uploadedFilePath = scheme + (_bAwsPath ? _fullAWSURL : _baseDomain) + "/" + bucketName + "/" + fileName;
_logger.LogDebug(kOp,
"Successfully uploaded: " + fileName + " | URL: " + uploadedFilePath,
__FILE__, __LINE__);
@@ -2541,7 +2584,7 @@ namespace ANSCENTER
// Build the returned URL from parts (independent of
// addressing style) — always the canonical path-style URL.
std::string scheme = _bTls ? "https://" : "http://";
uploadedFilePath = scheme + _fullAWSURL + "/" + bucketName + "/" + objectKey;
uploadedFilePath = scheme + (_bAwsPath ? _fullAWSURL : _baseDomain) + "/" + bucketName + "/" + objectKey;
_logger.LogDebug(kOp,
"Successfully uploaded: " + objectKey + " (" + std::to_string(bufferLength) + " bytes) | URL: " + uploadedFilePath,
__FILE__, __LINE__);
@@ -2559,6 +2602,117 @@ namespace ANSCENTER
return retryable ? AttemptResult::Transient : AttemptResult::Permanent;
});
}
// Generates a temporary pre-signed URL (AWS Signature V4) for an object
// already stored in S3. Public users can open the URL — without any AWS
// credentials — until it expires. SigV4 pre-signing is a purely local
// operation: it needs valid credentials and a known endpoint, but performs
// no network I/O, so it succeeds even when not connected.
bool ANSAWSS3::GeneratePresignedURL(const std::string& bucketName,
const std::string& prefix,
const std::string& objectName,
int validDays,
std::string& presignedURL) {
const std::string kOp = "ANSAWSS3::GeneratePresignedURL";
// Permanent failures — do NOT proceed.
if (!_isLicenseValid || !_isUnlockCodeValid) {
_logger.LogError(kOp,
!_isLicenseValid ? "Invalid license" : "Invalid unlock code",
__FILE__, __LINE__);
ANS_DBG("AWSS3", "GeneratePresignedURL: invalid license/unlock code");
return false;
}
if (bucketName.empty() || objectName.empty()) {
_logger.LogError(kOp, "Bucket name or object name is empty", __FILE__, __LINE__);
ANS_DBG("AWSS3", "GeneratePresignedURL: empty bucket/object name");
return false;
}
// AWS SigV4 caps pre-signed URL validity at 7 days.
if (validDays < 1 || validDays > 7) {
_logger.LogError(kOp,
"validDays must be between 1 and 7 (got " + std::to_string(validDays) + ")",
__FILE__, __LINE__);
ANS_DBG("AWSS3", "GeneratePresignedURL: validDays out of range (%d)", validDays);
return false;
}
// Snapshot the config/auth attributes under the config mutex.
std::string accessKey, secretKey, region, endpoint, service;
bool useHttps;
{
std::lock_guard<std::mutex> lock(_configMutex);
accessKey = _accessKey;
secretKey = _secretKey;
region = _bucketRegion;
endpoint = _fullAWSURL;
service = _serviceName;
useHttps = _bTls;
}
if (accessKey.empty() || secretKey.empty()) {
_logger.LogError(kOp,
"AWS credentials not set (call SetAuthentication first)",
__FILE__, __LINE__);
ANS_DBG("AWSS3", "GeneratePresignedURL: missing credentials");
return false;
}
if (endpoint.empty() || region.empty() || service.empty()) {
_logger.LogError(kOp,
"S3 endpoint/region/service not configured (call Connect first)",
__FILE__, __LINE__);
ANS_DBG("AWSS3", "GeneratePresignedURL: endpoint/region/service not configured");
return false;
}
// Build the object key the same way UploadPrefixJpegImage does, so the
// URL is generated for exactly the key that was uploaded.
std::string objectKey;
if (!prefix.empty()) {
std::string normalizedPrefix = prefix;
if (normalizedPrefix.front() == '/') {
normalizedPrefix = normalizedPrefix.substr(1);
}
if (!normalizedPrefix.empty() && normalizedPrefix.back() != '/') {
normalizedPrefix += '/';
}
objectKey = normalizedPrefix;
}
objectKey += objectName;
// Configure a CkHttp with the same credentials/endpoint the upload
// path uses so the signed host matches the stored object's host.
CkHttp http;
http.put_AwsAccessKey(accessKey.c_str());
http.put_AwsSecretKey(secretKey.c_str());
http.put_AwsRegion(region.c_str());
http.put_AwsEndpoint(endpoint.c_str());
const int numSecondsValid = validDays * 86400;
const char* url = http.s3_GenPresignedUrl("GET", useHttps,
bucketName.c_str(), objectKey.c_str(), numSecondsValid, service.c_str());
if (!http.get_LastMethodSuccess()) {
_logger.LogError(kOp,
std::string("Pre-signed URL generation failed: ") + http.lastErrorText(),
__FILE__, __LINE__);
ANS_DBG("AWSS3", "GeneratePresignedURL FAILED: key=%s err=%s",
objectKey.c_str(), http.lastErrorText());
return false;
}
presignedURL = (url != nullptr) ? url : "";
_logger.LogDebug(kOp,
"Generated pre-signed URL for '" + objectKey + "' valid " +
std::to_string(validDays) + " day(s): " + presignedURL,
__FILE__, __LINE__);
ANS_DBG("AWSS3", "GeneratePresignedURL OK: key=%s validDays=%d url=%s",
objectKey.c_str(), validDays, presignedURL.c_str());
return true;
}
// Downloads
bool ANSAWSS3::DownloadFile(const std::string& bucketName,
const std::string& objectName,
+22 -1
View File
@@ -197,7 +197,9 @@ namespace ANSCENTER {
SPDLogger& _logger = SPDLogger::GetInstance("ANSAWSS3", false);
// Pool helpers
std::unique_ptr<S3Connection> CreateConnection();
// logOnFailure=false suppresses error-level logging on failure (still emits ANS_DBG).
// Used by TryConnect's first attempt, which is allowed to fail for path-style/MinIO servers.
std::unique_ptr<S3Connection> CreateConnection(bool logOnFailure = true);
std::unique_ptr<S3Connection> AcquireConnection();
void ReleaseConnection(std::unique_ptr<S3Connection> conn);
@@ -280,6 +282,22 @@ namespace ANSCENTER {
[[nodiscard]] bool UploadJpegImage(const std::string& bucketName, unsigned char* jpeg_string, int32 bufferLength, const std::string& fileName, std::string& uploadedFilePath);
[[nodiscard]] bool DownloadFile(const std::string& bucketName, const std::string& objectName, const std::string& saveFilePath);
// Generate a temporary pre-signed URL (AWS Signature V4) for an
// object already stored in S3, so public users can access it
// without AWS credentials until it expires.
// bucketName, prefix, objectName: the SAME three values passed
// to UploadPrefixJpegImage when the object was uploaded.
// (objectName is what UploadPrefixJpegImage calls its
// `fileName` parameter — the last segment of the S3 key.)
// validDays: URL validity in days, 1..7 (AWS caps pre-signed
// URLs at 7 days).
// The object key is rebuilt as "<prefix>/<objectName>", so the URL
// points at exactly the key that was uploaded. This is a local
// signing operation: it performs no network I/O, but
// Connect()/SetAuthentication() must have been called so the
// credentials and endpoint are known.
[[nodiscard]] bool GeneratePresignedURL(const std::string& bucketName, const std::string& prefix, const std::string& objectName, int validDays, std::string& presignedURL);
};
}
@@ -378,6 +396,8 @@ extern "C" ANSULT_API int UploadPrefixJpegImageANSAWSHandle(ANSCENTER::ANSAWSS3
extern "C" ANSULT_API int DownloadFileStreamANSAWSHandle(ANSCENTER::ANSAWSS3** Handle, const char* bucketName, const char* objectName, const char* savedFilePath);
extern "C" ANSULT_API int GeneratePresignedURLANSAWSHandle(ANSCENTER::ANSAWSS3** Handle, const char* bucketName, const char* prefix, const char* objectName, int validDays, LStrHandle presignedURL);
// C++ test APIs (use std::string& instead of LStrHandle)
extern "C" ANSULT_API int UploadTextDataANSAWSHandle_CPP(ANSCENTER::ANSAWSS3** Handle, const char* bucketName, const char* textDataPath, std::string& uploadedFilePath);
extern "C" ANSULT_API int UploadBinaryDataANSAWSHandle_CPP(ANSCENTER::ANSAWSS3** Handle, const char* bucketName, const char* binaryDataPath, std::string& uploadedFilePath);
@@ -387,6 +407,7 @@ extern "C" ANSULT_API int UploadJpegImageANSAWSHandle_CPP(ANSCENTER::ANSAWSS3**
extern "C" ANSULT_API int UploadPrefixBinaryDataANSAWSHandle_CPP(ANSCENTER::ANSAWSS3** Handle, const char* bucketName, const char* prefix, const char* binaryDataPath, const char* objectName, std::string& uploadedFilePath);
extern "C" ANSULT_API int UploadPrefixMultiPartDataANSAWSHandle_CPP(ANSCENTER::ANSAWSS3** Handle, const char* bucketName, const char* prefix, const char* fileDataPath, const char* objectName, int fileSize, std::string& uploadedFilePath);
extern "C" ANSULT_API int UploadPrefixJpegImageANSAWSHandle_CPP(ANSCENTER::ANSAWSS3** Handle, const char* bucketName, const char* prefix, unsigned char* jpeg_string, int32 bufferLength, const char* fileName, std::string& uploadedFilePath);
extern "C" ANSULT_API int GeneratePresignedURLANSAWSHandle_CPP(ANSCENTER::ANSAWSS3** Handle, const char* bucketName, const char* prefix, const char* objectName, int validDays, std::string& presignedURL);
#endif
+82 -1
View File
@@ -1,6 +1,7 @@
// dllmain.cpp : Defines the entry point for the DLL application.
#include "pch.h"
#include "ANSUtilities.h"
#include "Utility.h"
#include <cstdint>
#include <cstdio>
#include <unordered_map>
@@ -697,7 +698,7 @@ extern "C" ANSULT_API int AESEncryption(const char* inputString, const char* in
}
extern "C" ANSULT_API int MD5HashFile(const char* filePath, LStrHandle decryptionMessage)
{
ANS_DBG("ANSUTIL","MD5HashFile: filePath=%s", filePath ? filePath : "(null)");
ANS_DBG("ANSUTIL","MD5HashFile: filePath=%s", filePath ? path_to_filename(std::string(filePath)).c_str() : "(null)");
if (filePath == nullptr || decryptionMessage == nullptr) return 0;
try {
std::string st = ANSCENTER::ANSUtilities::MD5HashFile(filePath);
@@ -2103,6 +2104,56 @@ extern "C" ANSULT_API int UploadPrefixJpegImageANSAWSHandle(ANSCENTER::ANSAWSS3
return -4;
}
}
extern "C" ANSULT_API int GeneratePresignedURLANSAWSHandle(ANSCENTER::ANSAWSS3** Handle, const char* bucketName, const char* prefix, const char* objectName, int validDays, LStrHandle presignedURL) {
ANS_DBG("ANSAWS", "GeneratePresignedURL called: HandlePtr=%p, *Handle=%p, bucket=%s, prefix=%s, object=%s, validDays=%d",
(void*)Handle,
Handle ? (void*)*Handle : nullptr,
bucketName ? bucketName : "(null)",
prefix ? prefix : "(null)",
objectName ? objectName : "(null)",
validDays);
if (Handle == nullptr || *Handle == nullptr ||
bucketName == nullptr || prefix == nullptr ||
objectName == nullptr || presignedURL == nullptr)
{
ANS_DBG("ANSAWS", "GeneratePresignedURL: returning 0 (null/invalid arg)");
return 0;
}
AWSHandleGuard guard(AcquireAWSHandle(*Handle));
if (!guard) {
ANS_DBG("ANSAWS", "GeneratePresignedURL: returning -1 (handle %p not in registry)", (void*)*Handle);
return -1;
}
try {
std::string outURL;
if (guard.get()->GeneratePresignedURL(bucketName, prefix, objectName, validDays, outURL)) {
int size = (int)outURL.length();
MgErr error = DSSetHandleSize(presignedURL, sizeof(int32) + size * sizeof(uChar));
if (error == noErr) {
(*presignedURL)->cnt = size;
memcpy((*presignedURL)->str, outURL.c_str(), size);
}
ANS_DBG("ANSAWS", "GeneratePresignedURL OK (1): object=%s -> %s",
objectName ? objectName : "(null)", outURL.c_str());
return 1;
}
else {
ANS_DBG("ANSAWS", "GeneratePresignedURL FAILED (-2): object=%s",
objectName ? objectName : "(null)");
return -2;
}
}
catch (std::exception& e) {
ANS_DBG("ANSAWS", "GeneratePresignedURL EXCEPTION (-3): object=%s, what=%s",
objectName ? objectName : "(null)", e.what());
return -3;
}
catch (...) {
ANS_DBG("ANSAWS", "GeneratePresignedURL EXCEPTION (-4): object=%s, unknown exception",
objectName ? objectName : "(null)");
return -4;
}
}
extern "C" ANSULT_API int UploadPrefixMultiPartDataANSAWSHandle(ANSCENTER::ANSAWSS3** Handle, const char* bucketName, const char* prefix, const char* fileDataPath, const char* objectName, int fileSize, LStrHandle uploadedFilePath) {
if (Handle == nullptr || *Handle == nullptr || bucketName == nullptr || prefix == nullptr || fileDataPath == nullptr || fileSize <= 0 ) {
return 0;
@@ -2220,6 +2271,14 @@ extern "C" ANSULT_API int UploadPrefixJpegImageANSAWSHandle_CPP(ANSCENTER::ANSAW
return guard.get()->UploadPrefixJpegImage(bucketName, prefix, jpeg_string, bufferLength, fileName, uploadedFilePath) ? 1 : 0;
} catch (...) { return 0; }
}
extern "C" ANSULT_API int GeneratePresignedURLANSAWSHandle_CPP(ANSCENTER::ANSAWSS3** Handle, const char* bucketName, const char* prefix, const char* objectName, int validDays, std::string& presignedURL) {
if (Handle == nullptr || *Handle == nullptr || bucketName == nullptr || prefix == nullptr || objectName == nullptr) return 0;
AWSHandleGuard guard(AcquireAWSHandle(*Handle));
if (!guard) return 0;
try {
return guard.get()->GeneratePresignedURL(bucketName, prefix, objectName, validDays, presignedURL) ? 1 : 0;
} catch (...) { return 0; }
}
// ════════════════════════════════════════════════════════════════════════════
// V2 API — handle-by-value for LabVIEW (uint64_t instead of Handle**)
@@ -2836,6 +2895,28 @@ extern "C" ANSULT_API int UploadPrefixJpegImageANSAWSHandle_V2(uint64_t handleVa
catch (...) { return 0; }
}
extern "C" ANSULT_API int GeneratePresignedURLANSAWSHandle_V2(uint64_t handleVal, const char* bucketName, const char* prefix, const char* objectName, int validDays, LStrHandle presignedURL) {
auto* _v2h = reinterpret_cast<ANSCENTER::ANSAWSS3*>(handleVal); if (!_v2h) return 0;
if (bucketName == nullptr || prefix == nullptr || objectName == nullptr || presignedURL == nullptr) return 0;
AWSHandleGuard guard(AcquireAWSHandle(_v2h));
if (!guard) return 0;
try {
std::string outURL;
if (guard.get()->GeneratePresignedURL(bucketName, prefix, objectName, validDays, outURL)) {
int size = (int)outURL.length();
MgErr error = DSSetHandleSize(presignedURL, sizeof(int32) + size * sizeof(uChar));
if (error == noErr) {
(*presignedURL)->cnt = size;
memcpy((*presignedURL)->str, outURL.c_str(), size);
}
return 1;
}
else return 0;
}
catch (std::exception& e) { return 0; }
catch (...) { return 0; }
}
extern "C" ANSULT_API int UploadPrefixMultiPartDataANSAWSHandle_V2(uint64_t handleVal, const char* bucketName, const char* prefix, const char* fileDataPath, const char* objectName, int fileSize, LStrHandle uploadedFilePath) {
auto* _v2h = reinterpret_cast<ANSCENTER::ANSAWSS3*>(handleVal); if (!_v2h) return 0;
if (bucketName == nullptr || prefix == nullptr || fileDataPath == nullptr || fileSize <= 0 || uploadedFilePath == nullptr) return 0;
+67 -25
View File
@@ -833,14 +833,36 @@ int RSTPTestCVClient() {
// std::string rtspUrl = "rtsp://admin:admin123@192.168.1.23:554";//"rtsp://alex:admin123@192.168.1.11:554";//
//std::string rtspUrl = "rtsp://anscamuser:SonTay2020@anssontay.tplinkdns.com:554/Streaming/Channels/401";// "rtsp://admin:JA!uV_mx@42.1.117.99:8093/1/1";
std::string rtspUrl = "rtsp://admin:AnsHome16%&@192.168.1.104:554";//"rtsp://alex:admin123@192.168.1.11:554";//112
std::string rtspUrl = "rtsp://admin:Grb%401212@14.224.237.228:15541/Streaming/channels/101";//"rtsp://admin:AnsHome16%&@192.168.1.104:554";//"rtsp://alex:admin123@192.168.1.11:554";//112
CreateANSRTSPHandle(&rtspClient, "", "", "", rtspUrl.c_str());
//void SetRTSPImageQuality(ANSCENTER::ANSRTSPClient** Handle, int mode)
SetRTSPImageQuality(&rtspClient,0);
//SetBBoxRTSP(&rtspClient, 0, 0, 1920, 1080);
//SetCropFlagRTSP(&rtspClient, 1);
// Hardware decode (auto-pick: CUDA if available, otherwise D3D11VA/DXVA2).
// Without this the player uses single-threaded SW decode (default since
// commit e134ebd "Use software decoder by default"), which on 4K HEVC
// can take 80-120ms/frame — too slow for 25/30fps. The decoder falls
// behind, the RTP buffer fills, and frames arrive in bursts (the
// "smooth → 1-2s freeze → catch-up" pattern). VLC uses HW decode by
// default, which is why it plays the same camera smoothly.
// MUST be called before StartRTSP — decoder is constructed inside play().
SetRTSPHWDecoding(&rtspClient, /*HW_DECODING_AUTO=*/0, /*preferredGpu=*/-1);
// Transport: default is RTP-over-RTSP/TCP (interleaved). On a LAN, UDP
// avoids TCP head-of-line stalls. Over WAN/NAT it typically does NOT
// work (port forwarding usually only covers TCP/554), so leave commented
// out for remote cameras. MUST be called before StartRTSP.
//SetRTSPRtpOverUdp(&rtspClient, 1);
StartRTSP(&rtspClient);
// Let ANSRTSP downscale internally (INTER_LINEAR, ~5x faster than the
// INTER_LANCZOS4 the test was doing). Without this, a 4K source plus
// LANCZOS4 resize stretches each loop iteration to ~150ms and the
// 20-slot decoder queue overruns — frames look "skipped" because
// intermediate ones get evicted before getLatestFrame() sees them.
SetRTSPDisplayResolution(&rtspClient, 1920, 1080);
//SetRTSPImageRotation(&rtspClient,0);
cv::namedWindow("Image", cv::WINDOW_NORMAL); // Create a resizable window.
cv::resizeWindow("Image", 1920, 1080); // Set initial size of the window (landscape).
@@ -854,12 +876,13 @@ int RSTPTestCVClient() {
//if ((index == 1800) || (index == 2200) || (index == 2500)) { StartRTSP(&rtspClient); }
//if ((index == 2000) || (index == 2300) || (index == 2700)) { StartRTSP(&rtspClient); }
//if (index > 20000) break;
auto start = std::chrono::system_clock::now();
auto loopStart = std::chrono::steady_clock::now();
auto getStart = loopStart;
cv::Mat* image = nullptr; // ✅ Use a pointer to hold the allocated image
GetRTSPCVImage(&rtspClient, width, height, pts,&image);
auto end1 = std::chrono::system_clock::now();
auto elapsed1 = std::chrono::duration_cast<std::chrono::milliseconds>(end1 - start);
if (elapsed1.count() > 0)std::cout << "Time to get image:" << elapsed1.count() << "ms" << std::endl;
auto getEnd = std::chrono::steady_clock::now();
auto getMs = std::chrono::duration_cast<std::chrono::milliseconds>(getEnd - getStart).count();
if (getMs > 0) std::cout << "Time to get image:" << getMs << "ms" << std::endl;
// ✅ Check if the image is valid BEFORE accessing it
if (!image || image->empty()) {
@@ -868,23 +891,42 @@ int RSTPTestCVClient() {
continue; // Skip processing if the image is empty
}
// High-quality downscale for display: INTER_LANCZOS4 preserves sharpness and edges
cv::Mat displayImage;
if (image->cols > 1920) {
double scale = 1920.0 / image->cols;
cv::resize(*image, displayImage, cv::Size(), scale, scale, cv::INTER_LANCZOS4);
}
else {
displayImage = *image;
}
cv::imshow("Image", displayImage);
// ANSRTSP already returned a 1920x1080 frame — no client-side resize.
// Time the imshow specifically — on Windows it can occasionally block
// for 100s of ms (window message pump, GPU sync, monitor refresh
// alignment), and that would visibly look like a freeze even though
// the producer pulse stays healthy.
auto showStart = std::chrono::steady_clock::now();
cv::imshow("Image", *image);
auto showEnd = std::chrono::steady_clock::now();
auto showMs = std::chrono::duration_cast<std::chrono::milliseconds>(showEnd - showStart).count();
ANSCV_ReleaseImage_S(&image);
//std::cout << "Index="<<index<<". Size: " << width << "x" << height << ". Timestamp: " << pts << std::endl;
if (cv::waitKey(30) == 27) {
// Time waitKey separately — also a Windows GUI pump, can block.
auto waitStart = std::chrono::steady_clock::now();
// 1ms is enough to pump GUI events; 30ms caps the loop at <33fps and
// is the main reason the camera-timer appeared to skip.
bool quit = (cv::waitKey(1) == 27);
auto waitEnd = std::chrono::steady_clock::now();
auto waitMs = std::chrono::duration_cast<std::chrono::milliseconds>(waitEnd - waitStart).count();
// Print only when something is actually slow — keeps the log readable.
// Threshold: 30ms = noticeable stutter, 100ms = visible freeze contributor.
if (showMs >= 30 || waitMs >= 30) {
std::cout << "[SLOW DISPLAY] imshow=" << showMs << "ms waitKey=" << waitMs
<< "ms (frame=" << index << ")" << std::endl;
}
if (quit) {
std::cout << "Break" << std::endl;
break;
}
// Whole-iteration timing — should now be close to 1/cameraFps.
auto loopMs = std::chrono::duration_cast<std::chrono::milliseconds>(
std::chrono::steady_clock::now() - loopStart).count();
if (loopMs >= 50) std::cout << "Loop iteration: " << loopMs << "ms" << std::endl;
}
cv::destroyAllWindows(); // Destroy all OpenCV windows
StopRTSP(&rtspClient);
@@ -1481,7 +1523,7 @@ int VideoPlayerClientTest() {
int64_t pts = 0;
ANSVIDEOPLAYER* videoClient;
std::string testVideoFile = "E:\\Programs\\DemoAssets\\Videos\\classroom.mp4";
std::string testVideoFile = "E:\\Programs\\DemoAssets\\Videos\\classroom.mp4";//"E:\\Programs\\DemoAssets\\ALPR_Test\\cam17.mp4"
CreateANSVideoPlayerHandle(&videoClient, "", testVideoFile.c_str());
StartVideoPlayer(&videoClient);
@@ -1580,7 +1622,7 @@ int FilePlayerClientCVTest() {
if ((index == 1800) || (index == 2200) || (index == 2500)) { StopFilePlayer(&filePlayerClient); }
if ((index == 2000) || (index == 2300) || (index == 2700)) { StartFilePlayer(&filePlayerClient); }
if (index > 20000) break;*/
if (index > 2000) break;
// if (index > 2000) break;
auto start = std::chrono::system_clock::now();
cv::Mat* image = nullptr;
GetFilePlayerCVImage(&filePlayerClient, width, height, pts, &image);
@@ -1647,17 +1689,17 @@ int main()
// resolved inside ANSCV.dll (which is linked against libavcodec etc.),
// so this works without the unit test having to link FFmpeg itself.
//ANSCV_PrintFFmpegLicense_S();
FilePlayerClientDoubleDestroy();
FilePlayerClientCVTest();
VideoPlayerClientDoubleDestroy();
VideoPlayerClientTest();
//FilePlayerClientDoubleDestroy();
//FilePlayerClientCVTest();
//VideoPlayerClientDoubleDestroy();
// VideoPlayerClientTest();
//OpenCVFunctionTest();
//GenerateVideo();
//VideoTestClient();
// TestGetImage();
//PureOpenCV();
// RSTPTestClient();
//RSTPTestCVClient();
//RSTPTestClient();
RSTPTestCVClient();
//TestCreateImageFromJpegStringFile();
//TestCreateImageFromFile();
//for (int i = 0; i < 100; i++) {
File diff suppressed because it is too large Load Diff
+2 -2
View File
@@ -503,9 +503,9 @@ int main()
SetConsoleOutputCP(CP_UTF8);
SetConsoleCP(CP_UTF8);
#endif
TestOCRv5mage();
//TestOCRv5mage();
//ANSOCR_VideoTest();
ANSOCR_VideoTest();
// TestOCRImage();
/* for (int i = 0; i < 20; i++) {
TestOCRImage();
@@ -863,8 +863,11 @@ int ImageDetectionEngine(std::string modelFilePath, std::string imageFilePath,
}
std::string label = (d.class_id >= 0 && d.class_id < static_cast<int>(classes.size()))
? classes[d.class_id] : "?";
cv::putText(frame, cv::format("%s:%d:%.2f", label.c_str(), d.class_id, d.prob),
cv::Point(d.box.x, d.box.y - 5), 0, 0.6, cv::Scalar(0, 0, 255), 1, cv::LINE_AA);
// cv::putText(frame, cv::format("%s:%d:%.2f", label.c_str(), d.class_id, d.prob),
// cv::Point(d.box.x, d.box.y - 5), 0, 0.6, cv::Scalar(0, 0, 255), 1, cv::LINE_AA);
cv::putText(frame, cv::format("%s", label.c_str()),
cv::Point(d.box.x, d.box.y - 5), 0, 0.6, cv::Scalar(0, 0, 255), 1, cv::LINE_AA);
}
}
//resize to fit 1920x1080 window
@@ -944,8 +947,9 @@ int VideoDetectorEngine(std::string modelFilePath, std::string videoFilePath, in
}
unsigned int bufferLength = 0;
//cv::Mat croppedFrame = frame(cv::Rect(0, 0, 1920, 1080));
auto start = std::chrono::system_clock::now();
// ── Phase 1: JPEG encode ───────────────────────────────────────
auto t_enc_start = std::chrono::high_resolution_clock::now();
std::vector<uchar> imageData;
bool success = cv::imencode(".jpg", frame, imageData);
if (!success) {
@@ -953,31 +957,42 @@ int VideoDetectorEngine(std::string modelFilePath, std::string videoFilePath, in
break;
}
std::string jpegImage(imageData.begin(), imageData.end());
/* auto end1 = std::chrono::system_clock::now();
auto elapsed1 = std::chrono::duration_cast<std::chrono::milliseconds>(end1 - start1);
printf("Jpeg conversion Time = %lld ms\n", static_cast<long long int>(elapsed1.count()));*/
auto t_enc_end = std::chrono::high_resolution_clock::now();
int height = frame.rows;
int width = frame.cols;
//auto start = std::chrono::system_clock::now();
/* measured work */
std::string detectionResult = RunInferenceFromJpegString(&infHandle, jpegImage.c_str(), imageData.size(), "ID");// , 100, 100, 0.2);
auto end = std::chrono::system_clock::now();
auto elapsed = std::chrono::duration_cast<std::chrono::milliseconds>(end - start);
// ── Phase 2: inference only (JPEG decode + detector + pose +
// JSON serialization happen inside) ─
auto t_inf_start = std::chrono::high_resolution_clock::now();
std::string detectionResult = RunInferenceFromJpegString(&infHandle, jpegImage.c_str(), imageData.size(), "ID");
auto t_inf_end = std::chrono::high_resolution_clock::now();
auto ms_d = [](auto a, auto b) {
return std::chrono::duration<double, std::milli>(b - a).count();
};
const double encMs = ms_d(t_enc_start, t_enc_end);
const double infMs = ms_d(t_inf_start, t_inf_end);
const double totalMs = encMs + infMs;
frameNum++;
printf("Inference Time = %lld ms\n", static_cast<long long int>(elapsed.count()));
printf("Frame %d: JPEG=%6.2f ms | Inference=%6.2f ms | Total=%6.2f ms\n",
frameNum, encMs, infMs, totalMs);
if (!detectionResult.empty()) {
// COCO pose skeleton connections
const std::vector<std::pair<int, int>> POSE_SKELETON = {
// ── COCO 17-keypoint skeleton (Ultralytics YOLO Pose format) ──
// Indices: 0=nose, 1=Leye, 2=Reye, 3=Lear, 4=Rear,
// 5=Lshoulder, 6=Rshoulder, 7=Lelbow, 8=Relbow,
// 9=Lwrist, 10=Rwrist, 11=Lhip, 12=Rhip,
// 13=Lknee, 14=Rknee, 15=Lankle, 16=Rankle
const std::vector<std::pair<int, int>> COCO17_SKELETON = {
{0,1}, {0,2}, {1,3}, {2,4}, // Face
{3,5}, {4,6}, // Head-to-shoulder
{5,7}, {7,9}, {6,8}, {8,10}, // Arms
{5,6}, {5,11}, {6,12}, {11,12}, // Body
{11,13}, {13,15}, {12,14}, {14,16} // Legs
};
// Colors for different limb groups
const std::vector<cv::Scalar> LIMB_COLORS = {
const std::vector<cv::Scalar> COCO17_COLORS = {
{255,0,0}, {255,0,0}, {255,0,0}, {255,0,0}, // Face - blue
{0,255,255}, {0,255,255}, // Head-shoulder - yellow
{0,255,0}, {0,255,0}, {0,165,255}, {0,165,255}, // Arms - green/orange
@@ -985,6 +1000,68 @@ int VideoDetectorEngine(std::string modelFilePath, std::string videoFilePath, in
{0,255,0}, {0,255,0}, {0,165,255}, {0,165,255} // Legs - green/orange
};
// ── Halpe 26-keypoint skeleton (MMPose body_2d_keypoint, used by RTMPose) ──
// Layout: 0..16 = COCO 17 (same as above);
// 17=Head, 18=Neck, 19=Hip(mid),
// 20=LBigToe, 21=RBigToe,
// 22=LSmallToe, 23=RSmallToe,
// 24=LHeel, 25=RHeel
const std::vector<std::pair<int, int>> HALPE26_SKELETON = {
{0,1}, {0,2}, {1,3}, {2,4}, // face
{17,18}, // head -> neck
{5,18}, {6,18}, // shoulders -> neck
{18,19}, // neck -> mid hip
{19,11}, {19,12}, // mid hip -> L/R hip
{5,7}, {7,9}, // left arm
{6,8}, {8,10}, // right arm
{11,13}, {13,15}, // left leg
{12,14}, {14,16}, // right leg
{15,20}, {15,22}, {15,24}, // left foot (big-toe / small-toe / heel)
{16,21}, {16,23}, {16,25} // right foot
};
const std::vector<cv::Scalar> HALPE26_COLORS = {
{255,0,0}, {255,0,0}, {255,0,0}, {255,0,0}, // face - blue
{0,255,255}, // head-neck - yellow
{0,255,255}, {0,255,255}, // shoulders-neck - yellow
{255,255,0}, // neck-midhip - cyan
{255,255,0}, {255,255,0}, // midhip-hips - cyan
{0,255,0}, {0,255,0}, // left arm - green
{0,165,255}, {0,165,255}, // right arm - orange
{0,255,0}, {0,255,0}, // left leg - green
{0,165,255}, {0,165,255}, // right leg - orange
{200,100,255}, {200,100,255}, {200,100,255}, // left foot - light purple
{255,100,100}, {255,100,100}, {255,100,100} // right foot - light pink
};
// ── OpenPose 18-keypoint skeleton (OpenVINO HumanPose, used by ANSPOSE) ──
// DIFFERENT index layout from COCO-17:
// 0=Nose, 1=Neck, 2=RShoulder, 3=RElbow, 4=RWrist,
// 5=LShoulder, 6=LElbow, 7=LWrist,
// 8=RHip, 9=RKnee, 10=RAnkle,
// 11=LHip, 12=LKnee, 13=LAnkle,
// 14=REye, 15=LEye, 16=REar, 17=LEar
// Note: ANSPOSE writes kps as stride-2 (x,y only — no per-kp score),
// so the drawer detects this layout by buffer size and skips
// the confidence threshold for it.
const std::vector<std::pair<int, int>> OPENPOSE18_SKELETON = {
{1,2}, {2,3}, {3,4}, // right arm (neck->shoulder->elbow->wrist)
{1,5}, {5,6}, {6,7}, // left arm
{1,8}, {8,9}, {9,10}, // right leg (neck->hip->knee->ankle)
{1,11}, {11,12}, {12,13}, // left leg
{1,0}, // neck->nose
{0,14}, {14,16}, // right eye->ear
{0,15}, {15,17} // left eye->ear
};
const std::vector<cv::Scalar> OPENPOSE18_COLORS = {
{0,165,255}, {0,165,255}, {0,165,255}, // right arm - orange
{0,255,0}, {0,255,0}, {0,255,0}, // left arm - green
{0,165,255}, {0,165,255}, {0,165,255}, // right leg - orange
{0,255,0}, {0,255,0}, {0,255,0}, // left leg - green
{255,255,0}, // neck-nose - cyan
{255,0,0}, {255,0,0}, // right face - blue
{255,0,0}, {255,0,0} // left face - blue
};
pt.clear();
std::stringstream ss;
ss.clear();
@@ -1100,29 +1177,88 @@ int VideoDetectorEngine(std::string modelFilePath, std::string videoFilePath, in
}
}
// Draw keypoints if available (format: x1;y1;score1;x2;y2;score2;... stride=3)
// Draw keypoints if available. Three on-the-wire formats are
// auto-detected from the buffer length so the same drawer
// handles all pose models without changing the engines:
//
// 51 floats → 17 kps stride-3 → YOLO Pose (COCO-17)
// 78 floats → 26 kps stride-3 → RTMPose (Halpe-26)
// 36 floats → 18 kps stride-2 → OpenPose/ANSPOSE
//
// For stride-3 formats kps[k*3+2] is a per-keypoint confidence
// and we threshold on it. ANSPOSE doesn't emit confidences
// (kps are stride-2), so we use kp.x>0 && kp.y>0 as the
// visibility proxy (ANSPOSE writes 0 / negatives for missing
// keypoints — see ANSPOSE.cpp:292).
if (!kps_str.empty()) {
std::vector<float> kps = ANSCENTER::ANSUtilityHelper::StringToKeypoints(kps_str);
int numKeypoints = (int)kps.size() / 3; // stride of 3: x, y, confidence
const size_t n = kps.size();
// Collect keypoint positions and confidence
const std::vector<std::pair<int, int>>* skeleton = nullptr;
const std::vector<cv::Scalar>* colors = nullptr;
int stride = 3;
int numKeypoints = 0;
bool hasPerKpScore = true;
if (n == 51) { // YOLO Pose: 17 kps × 3
stride = 3;
numKeypoints = 17;
skeleton = &COCO17_SKELETON;
colors = &COCO17_COLORS;
hasPerKpScore = true;
}
else if (n == 78) { // RTMPose: 26 kps × 3
stride = 3;
numKeypoints = 26;
skeleton = &HALPE26_SKELETON;
colors = &HALPE26_COLORS;
hasPerKpScore = true;
}
else if (n == 36) { // OpenPose/ANSPOSE: 18 kps × 2
stride = 2;
numKeypoints = 18;
skeleton = &OPENPOSE18_SKELETON;
colors = &OPENPOSE18_COLORS;
hasPerKpScore = false;
}
else if (n % 3 == 0) { // Unknown stride-3 — draw dots only
stride = 3;
numKeypoints = static_cast<int>(n) / 3;
hasPerKpScore = true;
}
else if (n % 2 == 0) { // Unknown stride-2 — draw dots only
stride = 2;
numKeypoints = static_cast<int>(n) / 2;
hasPerKpScore = false;
}
// Collect keypoint positions and (optional) confidence.
// For stride-2 we treat any kp with x>0 && y>0 as visible.
std::vector<cv::Point2f> keypointPts(numKeypoints);
std::vector<float> keypointConf(numKeypoints, 0.f);
std::vector<float> keypointConf(numKeypoints, 1.f);
for (int k = 0; k < numKeypoints; k++) {
keypointPts[k] = cv::Point2f(kps[k * 3], kps[k * 3 + 1]);
keypointConf[k] = kps[k * 3 + 2];
const float kx = kps[k * stride];
const float ky = kps[k * stride + 1];
keypointPts[k] = cv::Point2f(kx, ky);
if (hasPerKpScore) {
keypointConf[k] = kps[k * stride + 2];
} else {
keypointConf[k] = (kx > 0.f && ky > 0.f) ? 1.f : 0.f;
}
}
const float kpConfThreshold = 0.3f;
// Draw skeleton lines
for (size_t s = 0; s < POSE_SKELETON.size(); s++) {
int idx1 = POSE_SKELETON[s].first;
int idx2 = POSE_SKELETON[s].second;
if (idx1 < numKeypoints && idx2 < numKeypoints &&
keypointConf[idx1] > kpConfThreshold && keypointConf[idx2] > kpConfThreshold) {
cv::Scalar color = (s < LIMB_COLORS.size()) ? LIMB_COLORS[s] : cv::Scalar(200, 200, 200);
cv::line(frame, keypointPts[idx1], keypointPts[idx2], color, 2, cv::LINE_AA);
// Draw skeleton lines (skipped for unknown layouts).
if (skeleton) {
for (size_t s = 0; s < skeleton->size(); s++) {
int idx1 = (*skeleton)[s].first;
int idx2 = (*skeleton)[s].second;
if (idx1 < numKeypoints && idx2 < numKeypoints &&
keypointConf[idx1] > kpConfThreshold && keypointConf[idx2] > kpConfThreshold) {
cv::Scalar color = (s < colors->size()) ? (*colors)[s] : cv::Scalar(200, 200, 200);
cv::line(frame, keypointPts[idx1], keypointPts[idx2], color, 2, cv::LINE_AA);
}
}
}
@@ -1485,7 +1621,6 @@ int YOLO26POSEYolo11Test() {
int modelType = 31; // ONNX YOLO (30) RT YOLO (31)
std::string onnxclmodelFilePath = "E:\\Programs\\DemoAssets\\ANSAIModels\\PoseEstimationModels\\ANSPOSEN.zip";
std::string videoFile = "E:\\Programs\\DemoAssets\\Videos\\train.mp4";
VideoDetectorEngine(onnxclmodelFilePath, videoFile, modelType);// 22 is RT, 21 is onnx
return 0;
}
@@ -1832,14 +1967,11 @@ int ElasticGPUTest() {
}
int FireNSmokeTest() {
int FireNSmokeGetCofigurationTest() {
int modelType = 16; // Custom model type for helmet detection, adjust as needed based on how the model is set up in the engine
std::string modelFilePath = "C:\\Projects\\ANSVIS\\Models\\ANS_FireSmoke_v2.0.zip";
std::string videoFile = "E:\\Programs\\DemoAssets\\Videos\\Helmet\\HM1.mp4";
ANSCENTER::ANSODBase* infHandle;
std::string licenseKey = "";
std::string modelZipFilePassword = "";
double detectionScoreThreshold = 0.3;
@@ -1848,31 +1980,268 @@ int FireNSmokeTest() {
int loadEngine = 0;
CreateANSODHandle(&infHandle, licenseKey.c_str(), modelFilePath.c_str(), modelZipFilePassword.c_str(),
detectionScoreThreshold, modelConfThreshold, modelNMSThreshold, 1, modelType, 1, loadEngine);
std::string stParam;
GetConfiguredParameters_CPP(&infHandle, stParam);
std::cout << "Configured parameters:\n" << stParam << std::endl;
ReleaseANSODHandle(&infHandle);
return 0;
}
int ALPROD_Test() {
int modelType = 30; // ONNX YOLO (30) RT YOLO (31)
std::string onnmodelFilePath = "E:\\Programs\\DemoAssets\\ALPR_Test\\ANS_ALPR_2204(GPU)_1.zip";
std::string imageFile = "E:\\Programs\\DemoAssets\\ALPR_Test\\Data Test\\pl_ocr_3220.jpg";
ImageDetectionEngine(onnmodelFilePath, imageFile, modelType);//yolov8
return 0;
}
int FallDetectorTest() {
int modelType = 16; // Custom model type for helmet detection, adjust as needed based on how the model is set up in the engine
std::string modelFilePath = "E:\\Programs\\CustomModelDev\\ANSFallDetector_v1.0.zip";
std::string videoFile = "E:\\Programs\\DemoAssets\\Videos\\Fall\\video3.mp4";
VideoDetectorEngine(modelFilePath, videoFile, modelType);
return 0;
}
// ============================================================================
// VideoDetectorEngine_CPP — JPEG-bypass variant of VideoDetectorEngine.
//
// Differences vs. the JPEG version (only 3 actual lines change):
// • No cv::imencode / no JPEG round-trip
// • Calls RunInference_CPP (cv::Mat** in, vector<Object>& out) instead of
// RunInferenceFromJpegString
// • No JSON parsing — iterate vector<Object> directly
//
// Everything else (skeleton tables, drawing, auto-detect format) is identical.
// Trimmed down to focus on the data path; tracker is left enabled like the
// original. Call RTMPOSETest_CPP() from main() to compare timings.
// ============================================================================
int VideoDetectorEngine_CPP(std::string modelFilePath, std::string videoFilePath,
int engineType = 32, int modelDetectionType = 1) {
std::vector<std::string> classes;
ANSCENTER::ANSODBase* infHandle = nullptr;
std::string labelMap;
std::string licenseKey = "";
std::string modelZipFilePassword = "";
float detectionScoreThreshold = 0.3f;
float modelConfThreshold = 0.48f;
float modelNMSThreshold = 0.5f;
int modelType = engineType;
int detectorType = modelDetectionType;
// ── Optimize + create handle (identical to the JPEG variant) ────────────
std::cout << "Optimizing model, please wait...." << std::endl;
std::string optmizedModelFolder;
OptimizeModelStr(modelFilePath.c_str(), modelZipFilePassword.c_str(),
modelType, modelDetectionType, 1, optmizedModelFolder);
std::cout << "Optimized model folder: " << optmizedModelFolder << std::endl;
labelMap = CreateANSODHandle(&infHandle, licenseKey.c_str(), modelFilePath.c_str(),
modelZipFilePassword.c_str(),
detectionScoreThreshold, modelConfThreshold, modelNMSThreshold,
1, modelType, detectorType, 1);
{
std::stringstream ss(labelMap);
std::string token;
while (std::getline(ss, token, ',')) {
if (!token.empty()) classes.push_back(token);
}
}
std::cout << "classes: " << classes.size() << std::endl;
SetTracker(&infHandle, 0, 1); // 0 = BYTETRACK
// ── Skeleton tables (lifted verbatim from VideoDetectorEngine) ──────────
const std::vector<std::pair<int, int>> COCO17_SKELETON = {
{0,1},{0,2},{1,3},{2,4},{3,5},{4,6},
{5,7},{7,9},{6,8},{8,10},
{5,6},{5,11},{6,12},{11,12},
{11,13},{13,15},{12,14},{14,16}
};
const std::vector<cv::Scalar> COCO17_COLORS = {
{255,0,0},{255,0,0},{255,0,0},{255,0,0},
{0,255,255},{0,255,255},
{0,255,0},{0,255,0},{0,165,255},{0,165,255},
{255,255,0},{255,0,255},{255,0,255},{255,255,0},
{0,255,0},{0,255,0},{0,165,255},{0,165,255}
};
const std::vector<std::pair<int, int>> HALPE26_SKELETON = {
{0,1},{0,2},{1,3},{2,4},
{17,18},{5,18},{6,18},
{18,19},{19,11},{19,12},
{5,7},{7,9},{6,8},{8,10},
{11,13},{13,15},{12,14},{14,16},
{15,20},{15,22},{15,24},{16,21},{16,23},{16,25}
};
const std::vector<cv::Scalar> HALPE26_COLORS = {
{255,0,0},{255,0,0},{255,0,0},{255,0,0},
{0,255,255},{0,255,255},{0,255,255},
{255,255,0},{255,255,0},{255,255,0},
{0,255,0},{0,255,0},{0,165,255},{0,165,255},
{0,255,0},{0,255,0},{0,165,255},{0,165,255},
{200,100,255},{200,100,255},{200,100,255},
{255,100,100},{255,100,100},{255,100,100}
};
const std::vector<std::pair<int, int>> OPENPOSE18_SKELETON = {
{1,2},{2,3},{3,4},{1,5},{5,6},{6,7},
{1,8},{8,9},{9,10},{1,11},{11,12},{12,13},
{1,0},{0,14},{14,16},{0,15},{15,17}
};
const std::vector<cv::Scalar> OPENPOSE18_COLORS = {
{0,165,255},{0,165,255},{0,165,255},{0,255,0},{0,255,0},{0,255,0},
{0,165,255},{0,165,255},{0,165,255},{0,255,0},{0,255,0},{0,255,0},
{255,255,0},{255,0,0},{255,0,0},{255,0,0},{255,0,0}
};
cv::VideoCapture capture(videoFilePath);
if (!capture.isOpened()) {
std::cout << "Could not read video file: " << videoFilePath << std::endl;
ReleaseANSODHandle(&infHandle);
return -1;
}
int frameNum = 0;
while (true) {
cv::Mat frame;
if (!capture.read(frame)) {
capture.set(cv::CAP_PROP_POS_FRAMES, 0);
frameNum = 0;
continue;
}
frameNum++;
// ── THE KEY CHANGE — direct cv::Mat path, no JPEG, no JSON ──────────
// Compare the JPEG-version (~5–10 ms encode + ~5 ms decode + JSON
// serialize/parse round-trip) against this single in-memory call.
std::vector<ANSCENTER::Object> detectionObjects;
cv::Mat* framePtr = &frame;
auto t0 = std::chrono::high_resolution_clock::now();
RunInference_CPP(&infHandle, &framePtr, "ID", detectionObjects);
auto t1 = std::chrono::high_resolution_clock::now();
const double infMs = std::chrono::duration<double, std::milli>(t1 - t0).count();
// ────────────────────────────────────────────────────────────────────
printf("Frame %d: Inference=%6.2f ms | %zu detections (cv::Mat direct)\n",
frameNum, infMs, detectionObjects.size());
// ── Render each Object (no JSON parse — fields are already typed) ───
for (const auto& obj : detectionObjects) {
const cv::Rect bbox = obj.box;
cv::rectangle(frame, bbox, cv::Scalar(0, 255, 0), 2);
const std::string label = cv::format("%s:%d:%.2f:%d",
obj.className.c_str(), obj.classId, obj.confidence, obj.trackId);
cv::putText(frame, label, cv::Point(bbox.x, std::max(0, bbox.y - 5)),
cv::FONT_HERSHEY_SIMPLEX, 0.6, cv::Scalar(0, 0, 255), 1, cv::LINE_AA);
// Auto-detect kps layout from buffer length (same logic as the
// JPEG variant — see VideoDetectorEngine for the rationale).
const auto& kps = obj.kps;
if (kps.empty()) continue;
const size_t n = kps.size();
const std::vector<std::pair<int, int>>* skeleton = nullptr;
const std::vector<cv::Scalar>* colors = nullptr;
int stride = 3, numKp = 0;
bool hasScore = true;
if (n == 51) { stride=3; numKp=17; skeleton=&COCO17_SKELETON; colors=&COCO17_COLORS; }
else if (n == 78) { stride=3; numKp=26; skeleton=&HALPE26_SKELETON; colors=&HALPE26_COLORS; }
else if (n == 36) { stride=2; numKp=18; skeleton=&OPENPOSE18_SKELETON; colors=&OPENPOSE18_COLORS;
hasScore=false; }
else if (n % 3 == 0) { stride=3; numKp = static_cast<int>(n)/3; }
else if (n % 2 == 0) { stride=2; numKp = static_cast<int>(n)/2; hasScore=false; }
std::vector<cv::Point2f> pts(numKp);
std::vector<float> conf(numKp, 1.f);
for (int k = 0; k < numKp; ++k) {
const float kx = kps[k * stride];
const float ky = kps[k * stride + 1];
pts[k] = cv::Point2f(kx, ky);
conf[k] = hasScore ? kps[k * stride + 2]
: ((kx > 0.f && ky > 0.f) ? 1.f : 0.f);
}
const float kpThr = 0.3f;
if (skeleton) {
for (size_t s = 0; s < skeleton->size(); ++s) {
int i1 = (*skeleton)[s].first, i2 = (*skeleton)[s].second;
if (i1 < numKp && i2 < numKp && conf[i1] > kpThr && conf[i2] > kpThr) {
cv::Scalar c = (s < colors->size()) ? (*colors)[s] : cv::Scalar(200,200,200);
cv::line(frame, pts[i1], pts[i2], c, 2, cv::LINE_AA);
}
}
}
for (int k = 0; k < numKp; ++k) {
if (conf[k] > kpThr) {
cv::circle(frame, pts[k], 4, cv::Scalar(0, 0, 255), -1, cv::LINE_AA);
cv::circle(frame, pts[k], 4, cv::Scalar(255, 255, 255), 1, cv::LINE_AA);
}
}
}
cv::putText(frame, cv::format("Frame: %d (cv::Mat path)", frameNum),
cv::Point(10, 30), cv::FONT_HERSHEY_SIMPLEX, 0.8,
cv::Scalar(255, 255, 255), 2);
cv::imshow("ANS Pose (cv::Mat direct)", frame);
if (cv::waitKey(1) == 27) break; // ESC to quit
}
ReleaseANSODHandle(&infHandle);
cv::destroyAllWindows();
return 0;
}
int RTMPOSETest_CPP() {
int modelType = 32; // ONNXRTMPOSE (use 33 for RTRTMPOSE)
std::string modelFilePath = "E:\\Programs\\DemoAssets\\ANSAIModels\\PoseEstimationModels\\ANSRTMPOSE.zip";
std::string videoFile = "E:\\Programs\\DemoAssets\\Videos\\ANSOffice.mp4";// "E:\\Programs\\DemoAssets\\Videos\\rtmpose.mp4";
VideoDetectorEngine_CPP(modelFilePath, videoFile, modelType);
return 0;
}
// YOLO OD via the direct cv::Mat / typed-Object path. Use this to verify
// trackIds reach the overlay — the JPEG/JSON variant strips track_id in
// VectorDetectionToJsonString (ANSEngineCommon.cpp:1122-1123), so the
// JSON overlay always shows ":0". This _CPP path reads obj.trackId
// directly so adaptive-padding tracker output is visible.
int YOLO26ODYolo11Test_CPP() {
std::string modelFilePath = "C:\\Projects\\ANSVIS\\Models\\ANS_VehicleDetection_v2.0.zip";
std::string videoFile = "C:\\ProgramData\\ANSCENTER\\Shared\\road.mp4";
int modelType = 3; // auto-detect (matches YOLO26ODYolo11Test)
int modelDetectionType = 1; // object detection
VideoDetectorEngine_CPP(modelFilePath, videoFile, modelType, modelDetectionType);
return 0;
}
int RTMPOSETest() {
int modelType = 33; // ONNX YOLO (30) RT YOLO (31)
std::string onnxclmodelFilePath = "E:\\Programs\\DemoAssets\\ANSAIModels\\PoseEstimationModels\\ANSRTMPOSE.zip";
std::string videoFile ="E:\\Programs\\DemoAssets\\Videos\\rtmpose.mp4";// "E:\\Programs\\DemoAssets\\Videos\\ANSOffice.mp4";//
VideoDetectorEngine(onnxclmodelFilePath, videoFile, modelType);// 22 is RT, 21 is onnx
return 0;
}
int main()
{
//FallDetectorTest();
//ALPROD_Test();
//FireNSmokeTest();
//ElasticGPUTest();
RTMPOSETest_CPP();
//RTMPOSETest();
//YOLO26SEGYolo11Test();
//YOLO26POSEYolo11Test();
//YOLO26CLYolo11Test();
//YOLO26ODYolo12Test();
YOLO26ODYolo11Test();
//YOLO26ODYolo11Test();
//YOLO26ODYolo10Test();
//YOLO26OBBYolo11Test();
//SAM3ONNX_ImageTest(); // ORT reference — runs first, prints decoder input stats
//SAM3TRT_ImageTest(); // TRT under test — compare decoder input stats with above
//CustomModel_StressTest_FilePlayer(); // Multi-task stress test (LabVIEW flow)
//CustomModel_StressTest_FilePlayer(); // Multi-task stress test (LabVIEW flow)
// CustomModel_SingleStream_FilePlayer(); // 1 cam + 1 task — isolates concurrency from per-instance bugs
//SAM3TRT_UnitTest(); // TensorRT SAM3 test (in ANSSAM3-UnitTest.cpp)
//TensorRT10Test();
//FireNSmokeCustomDetection();
+2 -1
View File
@@ -116,4 +116,5 @@ int FaceYoloTest();
int TestYOLOV12();
int PPETest();
int RVATest();
int CustomModel_StressTest_FilePlayer();
int CustomModel_StressTest_FilePlayer();
int CustomModel_SingleStream_FilePlayer(); // 1 camera + 1 task — isolates concurrency from per-instance bugs
@@ -292,15 +292,15 @@ int CustomModel_StressTest_FilePlayer() {
// Video files (one per stream)
const std::string videoFiles[NUM_STREAMS] = {
"E:\\Programs\\DemoAssets\\Videos\\Helmet\\HM1.mp4",
"E:\\Programs\\DemoAssets\\Videos\\Helmet\\HM2.mp4",
"C:\\ProgramData\\ANSCENTER\\Shared\\HM1.mp4",
"C:\\ProgramData\\ANSCENTER\\Shared\\HM2.mp4",
};
// Which stream each task uses
const int taskStreamMap[NUM_TASKS] = { 0, 0, 1, 1 };
// Model config — EDIT for your custom model
const std::string modelFolder = "C:\\Projects\\ANSVIS\\Models\\ANS_Helmet_v2.0.zip";
const std::string modelFolder = "C:\\ProgramData\\ANSCENTER\\ANSVIS Server\\Models\\B-IN_ANS_Helmet_v2.0_102728911.zip";
//const char* modelName = "detector";
//const char* className = "detector.names";
const int modelType = 16; // 16 = CustomDetector, 31 = RTYOLO, 30 = ONNXYOLO
@@ -550,3 +550,237 @@ int CustomModel_StressTest_FilePlayer() {
return 0;
}
// =============================================================================
// CustomModel_SingleStream_FilePlayer
//
// ISOLATION TEST — 1 camera, 1 OD handle, 1 worker thread. No concurrent
// inference whatsoever. Same flow as CustomModel_StressTest_FilePlayer
// (FilePlayer → CloneImage → RunInferenceComplete_CPP → ReleaseImage), just
// without the multi-stream / multi-task fan-out.
//
// Use this to determine whether a hang is per-instance (will still hang here)
// or only triggered by cross-session DML contention (will NOT hang here).
// If THIS test runs cleanly for an extended period but the multi-stream
// stress test hangs after a few inferences, the issue is concurrent DML
// submissions on the AMD iGPU — not a bug in the engine code itself.
//
// Reuses helpers from CustomModel_StressTest_FilePlayer:
// LoadANSCV / UnloadANSCV, ODWorkerThread, GetPerGpuFreeMiB.
// =============================================================================
int CustomModel_SingleStream_FilePlayer() {
printf("\n");
printf("============================================================\n");
printf(" Custom Model SINGLE-STREAM Isolation Test (FilePlayer)\n");
printf(" 1 camera + 1 model + 1 worker thread\n");
printf(" Press ESC to stop\n");
printf("============================================================\n\n");
// --- Load ANSCV.dll at runtime (same helper as stress test) ---
if (!LoadANSCV()) return -1;
if (pInitCameraNetwork) pInitCameraNetwork();
// =====================================================================
// CONFIGURATION — EDIT THESE FOR YOUR TEST
// =====================================================================
const std::string videoFile =
"C:\\ProgramData\\ANSCENTER\\Shared\\HM1.mp4";
const std::string modelFolder =
"C:\\ProgramData\\ANSCENTER\\ANSVIS Server\\Models\\B-IN_ANS_Helmet_v2.0_102728911.zip";
const int modelType = 16; // 16 = CustomDetector (same as stress test)
const int detectorType = 1; // Detection
const float scoreThresh = 0.5f;
const float confThresh = 0.5f;
const float nmsThresh = 0.45f;
// =====================================================================
// Reset shared run flag (it's a static at file scope shared with stress test)
g_stressRunning.store(true);
std::cout << "\n--- Single-stream isolation test (no concurrency) ---\n" << std::endl;
// NOTE: deliberately NOT calling OptimizeModelStr here. OptimizeModelStr
// creates a separate "warmup" ANSCUSTOM instance whose detector and
// classifier sessions stay loaded for the lifetime of the process — even
// though that instance never runs inference, its 2 DML sessions hold AMD
// GPU resources and were suspected of contributing to a hang in the
// active session's GetTensorData<float>. Skipping it here leaves exactly
// 1 ANSCUSTOM = 2 DML sessions (detector + classifier) in the process,
// for the cleanest possible single-session isolation.
(void)detectorType; // unused without the OptimizeModelStr call
// --- Per-task state (just one) ---
StressTaskState taskState;
// --- Create FilePlayer (single stream) ---
void* fpClient = nullptr;
{
printf("[Stream0] Creating FilePlayer: %s\n", videoFile.c_str());
int result = pCreateFilePlayer(&fpClient, "", videoFile.c_str());
if (result != 1 || !fpClient) {
printf("[Stream0] FAILED to create FilePlayer (result=%d)\n", result);
UnloadANSCV();
return -2;
}
if (pSetFilePlayerDisplayRes) {
pSetFilePlayerDisplayRes(&fpClient, 1920, 1080);
}
printf("[Stream0] FilePlayer created (display: 1920x1080)\n");
}
// --- Create OD handle (single instance) ---
ANSCENTER::ANSODBase* odHandle = nullptr;
{
printf("[Task0] Creating OD handle (modelType=%d)...\n", modelType);
auto loadStart = std::chrono::steady_clock::now();
auto vramBefore = GetPerGpuFreeMiB();
std::string labelMap = CreateANSODHandle(
&odHandle,
"", // licenseKey
modelFolder.c_str(), // modelFilePath (zip or folder)
"", // modelZipFilePassword
scoreThresh,
confThresh,
nmsThresh,
1, // autoDetectEngine
modelType,
1, // detectionType (1 = Detection)
1); // loadEngineOnCreation
auto loadEnd = std::chrono::steady_clock::now();
double loadMs = std::chrono::duration<double, std::milli>(loadEnd - loadStart).count();
if (!odHandle) {
printf("[Task0] FAILED to create OD handle\n");
pStopFilePlayer(&fpClient);
pReleaseFilePlayer(&fpClient);
UnloadANSCV();
return -3;
}
auto vramAfter = GetPerGpuFreeMiB();
int bestGpu = 0;
size_t maxDelta = 0;
for (size_t g = 0; g < vramBefore.size() && g < vramAfter.size(); g++) {
size_t delta = (vramBefore[g] > vramAfter[g]) ? vramBefore[g] - vramAfter[g] : 0;
if (delta > maxDelta) { maxDelta = delta; bestGpu = (int)g; }
}
printf("[Task0] Model loaded in %.0f ms | GPU[%d] | VRAM: %zu MiB | Labels: %s\n",
loadMs, bestGpu, maxDelta,
labelMap.empty() ? "(none)" : labelMap.substr(0, 80).c_str());
std::lock_guard<std::mutex> lk(taskState.mtx);
taskState.engineLoaded = true;
taskState.statusMsg = "Running";
taskState.gpuDeviceId = bestGpu;
taskState.vramUsedMiB = maxDelta;
}
// --- Start playback ---
pStartFilePlayer(&fpClient);
printf("[Stream0] Playback started\n");
std::this_thread::sleep_for(std::chrono::milliseconds(500));
// --- Single worker thread (reuse ODWorkerThread from stress test) ---
std::thread worker(ODWorkerThread, /*taskId=*/0, fpClient, odHandle, std::ref(taskState));
// --- Display loop (single cell) ---
const int cellW = 1280, cellH = 720;
const char* windowName = "Custom Model — Single Stream Isolation";
cv::namedWindow(windowName, cv::WINDOW_NORMAL);
cv::resizeWindow(windowName, cellW, cellH + 40);
auto testStart = std::chrono::steady_clock::now();
while (g_stressRunning.load()) {
cv::Mat canvas(cellH + 40, cellW, CV_8UC3, cv::Scalar(30, 30, 30));
cv::Mat cell;
double fps = 0, infMs = 0, grabMs = 0;
int fCount = 0, dCount = 0, gpuId = -1;
std::string statusMsg, lastDet;
bool engineLoaded = false;
{
std::lock_guard<std::mutex> lk(taskState.mtx);
if (!taskState.displayFrame.empty()) {
cv::resize(taskState.displayFrame, cell, cv::Size(cellW, cellH));
}
fps = taskState.fps;
infMs = taskState.inferenceMs;
grabMs = taskState.grabMs;
fCount = taskState.frameCount;
dCount = taskState.detectionCount;
gpuId = taskState.gpuDeviceId;
statusMsg = taskState.statusMsg;
lastDet = taskState.lastDetection;
engineLoaded = taskState.engineLoaded;
}
if (cell.empty()) {
cell = cv::Mat(cellH, cellW, CV_8UC3, cv::Scalar(40, 40, 40));
cv::putText(cell, "Task 0: " + statusMsg,
cv::Point(20, cellH / 2),
cv::FONT_HERSHEY_SIMPLEX, 0.8, cv::Scalar(100, 100, 255), 2);
}
cv::rectangle(cell, cv::Rect(0, cellH - 45, cellW, 45),
cv::Scalar(0, 0, 0), cv::FILLED);
char bar1[256], bar2[128];
snprintf(bar1, sizeof(bar1),
"%.1f FPS | inf:%.0fms grab:%.0fms | Frames:%d | Det:%d",
fps, infMs, grabMs, fCount, dCount);
snprintf(bar2, sizeof(bar2), "GPU[%d] | last:%s",
gpuId, lastDet.empty() ? "-" : lastDet.c_str());
cv::Scalar barColor = engineLoaded ? cv::Scalar(0, 255, 0) : cv::Scalar(0, 100, 255);
cv::putText(cell, bar1, cv::Point(5, cellH - 25),
cv::FONT_HERSHEY_SIMPLEX, 0.5, barColor, 1);
cv::putText(cell, bar2, cv::Point(5, cellH - 5),
cv::FONT_HERSHEY_SIMPLEX, 0.5, cv::Scalar(0, 200, 255), 1);
cell.copyTo(canvas(cv::Rect(0, 0, cellW, cellH)));
double elapsed = std::chrono::duration<double>(
std::chrono::steady_clock::now() - testStart).count();
char bottomBar[256];
snprintf(bottomBar, sizeof(bottomBar),
"Single-stream | Elapsed: %.0fs | %.1f FPS | Press ESC to stop",
elapsed, fps);
cv::putText(canvas, bottomBar, cv::Point(10, cellH + 25),
cv::FONT_HERSHEY_SIMPLEX, 0.5, cv::Scalar(200, 200, 200), 1);
cv::imshow(windowName, canvas);
int key = cv::waitKey(30);
if (key == 27) {
printf("\nESC pressed - stopping...\n");
g_stressRunning.store(false);
}
}
printf("Waiting for worker thread...\n");
if (worker.joinable()) worker.join();
double totalElapsed = std::chrono::duration<double>(
std::chrono::steady_clock::now() - testStart).count();
printf("\n============================================================\n");
printf(" SINGLE-STREAM SUMMARY (runtime: %.0fs)\n", totalElapsed);
printf("============================================================\n");
printf(" GPU[%d] | %d frames | %d detections | %.1f FPS | Inf: %.0fms\n",
taskState.gpuDeviceId, taskState.frameCount, taskState.detectionCount,
taskState.fps, taskState.inferenceMs);
printf("============================================================\n");
if (odHandle) ReleaseANSODHandle(&odHandle);
if (fpClient) {
pStopFilePlayer(&fpClient);
pReleaseFilePlayer(&fpClient);
}
cv::destroyAllWindows();
if (pDeinitCameraNetwork) pDeinitCameraNetwork();
UnloadANSCV();
return 0;
}
+11 -11
View File
@@ -438,47 +438,47 @@ int TestSegmentationEngine() {// For New Yolo version
}
//// Set the working directory
bool setWorkingDirectory = ansEngine.SetWorkingDirectory("C:\\Programs\\TrainingToolTest\\SegmentationEngine\\SegmeTxtYolo");
bool setWorkingDirectory = ansEngine.SetWorkingDirectory("E:\\Programs\\TrainingToolTest\\SegmentationEngine\\SegmeTxtYolo");
if (!setWorkingDirectory)std::cout << "Working Directory Not Set" << std::endl;
////1. Create a project
bool createdProject = ansEngine.CreateProject("SegmentationTest");
bool createdProject = ansEngine.CreateProject("SegmentationTest1");
if (!createdProject)std::cout << "Project Not Created" << std::endl;
////2. Browse to the working directory
projectDirectory = ansEngine.GetProjectDirectory();
std::cout << "Project Directory:" << projectDirectory << std::endl;
bool dataUploaded = ansEngine.UploadTrainingData("SegmentationTest");
bool dataUploaded = ansEngine.UploadTrainingData("SegmentationTest1");
if (!dataUploaded)std::cout << "Cannot upload data" << std::endl;
//////3. Create a training engine
bool createEngine = ansEngine.CreateTrainingEngine("SegmentationTest", 1, 0, 2000, 32, 0.001);
bool createEngine = ansEngine.CreateTrainingEngine("SegmentationTest1", 1, 0, 2000, 32, 0.001);
if (!createEngine)std::cout << "Cannot create engine" << std::endl;
std::string projectExperimentStatus = ansEngine.GetProjectExperimentStatus("SegmentationTest", 1);
std::string projectExperimentStatus = ansEngine.GetProjectExperimentStatus("SegmentationTest1", 1);
std::cout << projectExperimentStatus << std::endl;
std::string trainingCommand = ansEngine.GenerateTrainingCommand("SegmentationTest", 1, 0);
std::string trainingCommand = ansEngine.GenerateTrainingCommand("SegmentationTest1", 1, 0);
std::cout << "Training Command:" << trainingCommand << std::endl;
//// 4. Evaluate the model
std::string evaluateModelCommand = ansEngine.EvaluateModel("SegmentationTest", 1);
std::string evaluateModelCommand = ansEngine.EvaluateModel("SegmentationTest1", 1);
std::cout << "Evaluation Result:" << evaluateModelCommand << std::endl;
//// 5. Download the model
std::string saveModelPath = "C:\\Programs\\TrainingToolTest\\SegmentationEngine\\SegmeTxtYolo\\Models";
ansEngine.DownloadModel("SegmentationTest", 1, saveModelPath, 0);
ansEngine.DownloadModel("SegmentationTest", 1, saveModelPath, 1);
std::string saveModelPath = "E:\\Programs\\TrainingToolTest\\SegmentationEngine\\SegmeTxtYolo\\Models";
ansEngine.DownloadModel("SegmentationTest1", 1, saveModelPath, 0);
ansEngine.DownloadModel("SegmentationTest1", 1, saveModelPath, 1);
return 0;
}
int main()
{
TestSegmentationEngine();
TestSegmentationEngine();
//TestANSYLTRE();
//TestANSYLCLTRE();
//FallDetection();
@@ -267,7 +267,7 @@ int ANSFTechMinIOTest() {
std::string baseDomain = "https://minio.dev.ftech.ai";
std::string bucketRegion = "vn1";
std::string serviceName = "s3";
std::string bucketName = "aicamera";
std::string bucketName = "funzy";
int port = 443;
bool bTls = true;
@@ -281,8 +281,8 @@ int ANSFTechMinIOTest() {
}
// Set authentication
std::string accessKey = "aicamerapush";
std::string secretKey = "88fO8Y3EpMFMG1HFCCcfCE4cWh0k5KKZ";
std::string accessKey = "tuanpm";
std::string secretKey = "BRFbQujGhkKrI3xqSUE9ywnpXHt27c";
if (SetAuthenticationANSAWSHandle(&awsHandle, accessKey.c_str(), secretKey.c_str()) != 1) {
std::cerr << "Failed to set AWS S3 authentication." << std::endl;
ReleaseANSAWSHandle(&awsHandle);
@@ -291,7 +291,7 @@ int ANSFTechMinIOTest() {
// Upload a jpeg image to bucket
std::string jpegFilePath = "E:\\Programs\\DemoAssets\\NV\\Test.jpg";
std::string jpegFilePath = "C:\\ProgramData\\ANSCENTER\\Shared\\Test.jpg";
std::string objectName = "test1.jpg";
@@ -389,9 +389,12 @@ int ANSLLMOllamaTest() {
int main()
{
//ANSLLMOllamaTest();
ANSAWSTest();
//std::cout << "===== MinIO (path-style) =====" << std::endl;
//ANSFTechMinIOTest();
//ANSAWSTest();
std::cout << "\n===== AWS S3 (virtual-host) =====" << std::endl;
ANSAWSTest();
//ANSLLMTest();
//SendEmailExample();
//uploadFileToGoogleStorage();