Files
ANSCustomModels/ANSCustomFireNSmokeDetection/ANSCustomFireNSmoke.h
T
nghianguyenandClaude Fable 5.1 e893be4a75 ANSCustomFireNSmokeDetection: the ANSVIS custom model is judged by ANSMIND's Fire & Smoke verifier
The DLL used to find a detection in one section of the frame, latch a crop
round it and vote on tracked boxes inside that crop. Measured on the field
recordings that was itself the main false-alarm source (the detector needs
context: a floodlight is fake_fire on the whole frame and fire on a tight
crop of the same lamp), and a PTZ tour broke the latch. ANSMIND's native
vision.fire_smoke node replaced it with a whole-frame-first verifier whose
cues are box-relative and carried by global motion, fire-only native zoom
re-checks, a look-alike filter and a ladder of gates measured on real and
false recordings.

The DLL now feeds that verifier, vendored verbatim from ANSMINDCORE dba11ac
(vendor/engine/fire_smoke.hpp, pure C++): ANSLIB runs the package's
detector on the whole frame at a low floor with class-agnostic NMS, the COCO
filter when there is a candidate to explain, and the detector again on the
few native crops the verifier plans; only confirmed tracks come back, as
fire/smoke CustomObjects with fs_seconds/fs_track/evidence in extraInfo.
Parameters: Sensitivity (slider onto low/balanced/high), SmokeSeconds,
DetectFire/DetectSmoke, ZoomChecks, MaxFps (6: evidence is per second of
scene), CandidateScore, MinObjectSize (49a53c3's knob, kept: a candidate
under that fraction of the frame is noise), ExclusiveROIs. Engines load
inside Initialize: a lazily loaded one faulted in ANSLIB's inference entry
on the first frame.

FireNSmokeDetection_Run replays a recording on the video's clock (parity
harness with firesmoke-run). Measured, same package, TensorRT:
  15.9.9.0_9.51_real_smoke   63 alarms, first at 3.0 s (node: 56, 3.0 s)
  15.9.9.51.10.51_real_smoke 113 alarms, first at 3.0 s (node: 126, 3.0 s)
  8 KA false_* clips          0 alarms (node: 0)
Unit tests updated for the new parameter schema; 9/9 pass. The headless
checker from 49a53c3 builds unchanged against the new class.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-01 12:15:33 +11:00

104 lines
4.9 KiB
C++

// ANSCustomFireNSmoke.h — the ANSVIS Fire & Smoke custom model, judged by
// the same verifier the ANSMIND `vision.fire_smoke` node runs.
//
// The first generation of this DLL found a detection in one section of the
// frame, latched a crop round it and voted on tracked boxes inside that crop.
// Measured on the field recordings that was itself the main false-alarm
// source: the detector needs CONTEXT (a floodlight is `fake_fire` on the
// whole frame and `fire` on a tight crop of the same lamp), the active window
// lost the rest of the picture, and a PTZ tour broke the latch. ANSMIND's
// native node replaced that with a whole-frame-first verifier whose every cue
// is box-relative and carried by global motion (PTZ-safe), with native-
// resolution zoom re-checks for fire only, a look-alike filter, and a ladder
// of physical gates measured on real and false recordings (vendor/engine/
// fire_smoke.hpp, pure C++, no OpenCV).
//
// This DLL now feeds that verifier: ANSLIB runs the package's detector on
// the whole frame (low floor, class-agnostic NMS), the filter model when
// there is something to explain, and the detector again on the few native
// crops the verifier asks for; the verifier decides, and only CONFIRMED fire
// / smoke tracks come back as CustomObjects, every frame while they last.
// So ANSVIS (this DLL in its server) and ANSMIND (the node, or this DLL in
// ans-custom-host) judge a scene identically.
#include "ANSLIB.h"
#include "vendor/engine/fire_smoke.hpp"
#include <chrono>
#include <memory>
#include <mutex>
#include <string>
#include <vector>
class CUSTOM_API ANSCustomFS : public IANSCustomClass
{
using ANSLIBPtr = std::unique_ptr<ANSCENTER::ANSLIB, decltype(&ANSCENTER::ANSLIB::Destroy)>;
using Role = ansmind::firesmoke::Role;
using RectF = ansmind::RectF;
/// One detector verdict in normalised frame coordinates.
struct Det {
RectF box;
int classId = -1;
float conf = 0;
};
ANSLIBPtr _detector{ nullptr, &ANSCENTER::ANSLIB::Destroy };
ANSLIBPtr _filter{ nullptr, &ANSCENTER::ANSLIB::Destroy };
std::recursive_mutex _mutex;
// ----- the package's classes -----
std::vector<std::string> _labels; ///< detector classes, in model order
std::vector<Role> _roles; ///< fire / fake_fire / smoke / fake_smoke per class
int _fireClass{ -1 }, _smokeClass{ -1 };
std::string _fireName, _smokeName;
std::vector<std::string> _filterLabels;
std::vector<char> _vehicle, _screen; ///< which filter classes explain a look-alike away
// ----- the verifier and its knobs -----
ansmind::firesmoke::Verifier _verifier;
ansmind::firesmoke::Params _p;
int _zoomBudget{ 2 };
double _maxFps{ 6.0 };
float _minObjectAreaRatio{ 0.0003f }; ///< a candidate box under this fraction of the frame is noise ("MinObjectSize", %)
bool _fsLog{ false };
std::vector<cv::Rect> _exclusiveROIs; ///< pixel rectangles no candidate may touch
std::string _paramSignature; ///< what the last applyParams() saw
// ----- time -----
std::chrono::steady_clock::time_point _t0{};
bool _clockStarted{ false };
double _externalClock{ -1 }; ///< a replay's video time, when set
double _lastRunT{ -1e9 };
std::vector<CustomObject> _lastResults;
uint64_t _frames{ 0 };
// ----- steps of a frame -----
void applyParams();
bool runDetector(const cv::Mat& bgr, const cv::Rect& roi, std::vector<Det>& out, const std::string& camera);
bool runFilter(const cv::Mat& bgr, const cv::Rect& roi, std::vector<Det>& out, const std::string& camera);
static void nmsClassAgnostic(std::vector<Det>& dets, float iouThr);
ansmind::firesmoke::Cues cuesFor(const cv::Mat& bgr, const RectF& box) const;
static cv::Rect pixelRect(const RectF& r, int W, int H);
static RectF normRect(const cv::Rect& r, int W, int H);
static float cover(const RectF& a, const RectF& b);
public:
bool Initialize(const std::string& modelDiretory, float detectionScoreThreshold, std::string& labelMap) override;
bool OptimizeModel(bool fp16) override;
bool ConfigureParameters(CustomParams& param) override;
std::vector<CustomObject> RunInference(const cv::Mat& input) override;
std::vector<CustomObject> RunInference(const cv::Mat& input, const std::string& camera_id) override;
bool Destroy() override;
/// Not part of the plugin contract. A replay harness hands the frame's
/// own time in seconds so the verifier's per-second evidence and gates
/// run on VIDEO time, not on how fast the machine decodes. Live callers
/// (ANSVIS, ans-custom-host) never call it and the steady clock is used.
void SetFrameClock(double seconds) { _externalClock = seconds; }
/// The verifier's per-track debug view after the last frame (replay harness).
std::vector<ansmind::firesmoke::TrackDebug> LastDebug() const { return _verifier.debug(); }
ANSCustomFS();
~ANSCustomFS();
};