The DLL used to find a detection in one section of the frame, latch a crop
round it and vote on tracked boxes inside that crop. Measured on the field
recordings that was itself the main false-alarm source (the detector needs
context: a floodlight is fake_fire on the whole frame and fire on a tight
crop of the same lamp), and a PTZ tour broke the latch. ANSMIND's native
vision.fire_smoke node replaced it with a whole-frame-first verifier whose
cues are box-relative and carried by global motion, fire-only native zoom
re-checks, a look-alike filter and a ladder of gates measured on real and
false recordings.
The DLL now feeds that verifier, vendored verbatim from ANSMINDCORE dba11ac
(vendor/engine/fire_smoke.hpp, pure C++): ANSLIB runs the package's
detector on the whole frame at a low floor with class-agnostic NMS, the COCO
filter when there is a candidate to explain, and the detector again on the
few native crops the verifier plans; only confirmed tracks come back, as
fire/smoke CustomObjects with fs_seconds/fs_track/evidence in extraInfo.
Parameters: Sensitivity (slider onto low/balanced/high), SmokeSeconds,
DetectFire/DetectSmoke, ZoomChecks, MaxFps (6: evidence is per second of
scene), CandidateScore, MinObjectSize (49a53c3's knob, kept: a candidate
under that fraction of the frame is noise), ExclusiveROIs. Engines load
inside Initialize: a lazily loaded one faulted in ANSLIB's inference entry
on the first frame.
FireNSmokeDetection_Run replays a recording on the video's clock (parity
harness with firesmoke-run). Measured, same package, TensorRT:
15.9.9.0_9.51_real_smoke 63 alarms, first at 3.0 s (node: 56, 3.0 s)
15.9.9.51.10.51_real_smoke 113 alarms, first at 3.0 s (node: 126, 3.0 s)
8 KA false_* clips 0 alarms (node: 0)
Unit tests updated for the new parameter schema; 9/9 pass. The headless
checker from 49a53c3 builds unchanged against the new class.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
104 lines
4.9 KiB
C++
104 lines
4.9 KiB
C++
// ANSCustomFireNSmoke.h — the ANSVIS Fire & Smoke custom model, judged by
|
|
// the same verifier the ANSMIND `vision.fire_smoke` node runs.
|
|
//
|
|
// The first generation of this DLL found a detection in one section of the
|
|
// frame, latched a crop round it and voted on tracked boxes inside that crop.
|
|
// Measured on the field recordings that was itself the main false-alarm
|
|
// source: the detector needs CONTEXT (a floodlight is `fake_fire` on the
|
|
// whole frame and `fire` on a tight crop of the same lamp), the active window
|
|
// lost the rest of the picture, and a PTZ tour broke the latch. ANSMIND's
|
|
// native node replaced that with a whole-frame-first verifier whose every cue
|
|
// is box-relative and carried by global motion (PTZ-safe), with native-
|
|
// resolution zoom re-checks for fire only, a look-alike filter, and a ladder
|
|
// of physical gates measured on real and false recordings (vendor/engine/
|
|
// fire_smoke.hpp, pure C++, no OpenCV).
|
|
//
|
|
// This DLL now feeds that verifier: ANSLIB runs the package's detector on
|
|
// the whole frame (low floor, class-agnostic NMS), the filter model when
|
|
// there is something to explain, and the detector again on the few native
|
|
// crops the verifier asks for; the verifier decides, and only CONFIRMED fire
|
|
// / smoke tracks come back as CustomObjects, every frame while they last.
|
|
// So ANSVIS (this DLL in its server) and ANSMIND (the node, or this DLL in
|
|
// ans-custom-host) judge a scene identically.
|
|
#include "ANSLIB.h"
|
|
#include "vendor/engine/fire_smoke.hpp"
|
|
|
|
#include <chrono>
|
|
#include <memory>
|
|
#include <mutex>
|
|
#include <string>
|
|
#include <vector>
|
|
|
|
class CUSTOM_API ANSCustomFS : public IANSCustomClass
|
|
{
|
|
using ANSLIBPtr = std::unique_ptr<ANSCENTER::ANSLIB, decltype(&ANSCENTER::ANSLIB::Destroy)>;
|
|
using Role = ansmind::firesmoke::Role;
|
|
using RectF = ansmind::RectF;
|
|
|
|
/// One detector verdict in normalised frame coordinates.
|
|
struct Det {
|
|
RectF box;
|
|
int classId = -1;
|
|
float conf = 0;
|
|
};
|
|
|
|
ANSLIBPtr _detector{ nullptr, &ANSCENTER::ANSLIB::Destroy };
|
|
ANSLIBPtr _filter{ nullptr, &ANSCENTER::ANSLIB::Destroy };
|
|
std::recursive_mutex _mutex;
|
|
|
|
// ----- the package's classes -----
|
|
std::vector<std::string> _labels; ///< detector classes, in model order
|
|
std::vector<Role> _roles; ///< fire / fake_fire / smoke / fake_smoke per class
|
|
int _fireClass{ -1 }, _smokeClass{ -1 };
|
|
std::string _fireName, _smokeName;
|
|
std::vector<std::string> _filterLabels;
|
|
std::vector<char> _vehicle, _screen; ///< which filter classes explain a look-alike away
|
|
|
|
// ----- the verifier and its knobs -----
|
|
ansmind::firesmoke::Verifier _verifier;
|
|
ansmind::firesmoke::Params _p;
|
|
int _zoomBudget{ 2 };
|
|
double _maxFps{ 6.0 };
|
|
float _minObjectAreaRatio{ 0.0003f }; ///< a candidate box under this fraction of the frame is noise ("MinObjectSize", %)
|
|
bool _fsLog{ false };
|
|
std::vector<cv::Rect> _exclusiveROIs; ///< pixel rectangles no candidate may touch
|
|
std::string _paramSignature; ///< what the last applyParams() saw
|
|
|
|
// ----- time -----
|
|
std::chrono::steady_clock::time_point _t0{};
|
|
bool _clockStarted{ false };
|
|
double _externalClock{ -1 }; ///< a replay's video time, when set
|
|
double _lastRunT{ -1e9 };
|
|
std::vector<CustomObject> _lastResults;
|
|
uint64_t _frames{ 0 };
|
|
|
|
// ----- steps of a frame -----
|
|
void applyParams();
|
|
bool runDetector(const cv::Mat& bgr, const cv::Rect& roi, std::vector<Det>& out, const std::string& camera);
|
|
bool runFilter(const cv::Mat& bgr, const cv::Rect& roi, std::vector<Det>& out, const std::string& camera);
|
|
static void nmsClassAgnostic(std::vector<Det>& dets, float iouThr);
|
|
ansmind::firesmoke::Cues cuesFor(const cv::Mat& bgr, const RectF& box) const;
|
|
static cv::Rect pixelRect(const RectF& r, int W, int H);
|
|
static RectF normRect(const cv::Rect& r, int W, int H);
|
|
static float cover(const RectF& a, const RectF& b);
|
|
|
|
public:
|
|
bool Initialize(const std::string& modelDiretory, float detectionScoreThreshold, std::string& labelMap) override;
|
|
bool OptimizeModel(bool fp16) override;
|
|
bool ConfigureParameters(CustomParams& param) override;
|
|
std::vector<CustomObject> RunInference(const cv::Mat& input) override;
|
|
std::vector<CustomObject> RunInference(const cv::Mat& input, const std::string& camera_id) override;
|
|
bool Destroy() override;
|
|
|
|
/// Not part of the plugin contract. A replay harness hands the frame's
|
|
/// own time in seconds so the verifier's per-second evidence and gates
|
|
/// run on VIDEO time, not on how fast the machine decodes. Live callers
|
|
/// (ANSVIS, ans-custom-host) never call it and the steady clock is used.
|
|
void SetFrameClock(double seconds) { _externalClock = seconds; }
|
|
/// The verifier's per-track debug view after the last frame (replay harness).
|
|
std::vector<ansmind::firesmoke::TrackDebug> LastDebug() const { return _verifier.debug(); }
|
|
|
|
ANSCustomFS();
|
|
~ANSCustomFS();
|
|
};
|