ANSCustomFireNSmokeDetection: the ANSVIS custom model is judged by ANSMIND's Fire & Smoke verifier

The DLL used to find a detection in one section of the frame, latch a crop
round it and vote on tracked boxes inside that crop. Measured on the field
recordings that was itself the main false-alarm source (the detector needs
context: a floodlight is fake_fire on the whole frame and fire on a tight
crop of the same lamp), and a PTZ tour broke the latch. ANSMIND's native
vision.fire_smoke node replaced it with a whole-frame-first verifier whose
cues are box-relative and carried by global motion, fire-only native zoom
re-checks, a look-alike filter and a ladder of gates measured on real and
false recordings.

The DLL now feeds that verifier, vendored verbatim from ANSMINDCORE dba11ac
(vendor/engine/fire_smoke.hpp, pure C++): ANSLIB runs the package's
detector on the whole frame at a low floor with class-agnostic NMS, the COCO
filter when there is a candidate to explain, and the detector again on the
few native crops the verifier plans; only confirmed tracks come back, as
fire/smoke CustomObjects with fs_seconds/fs_track/evidence in extraInfo.
Parameters: Sensitivity (slider onto low/balanced/high), SmokeSeconds,
DetectFire/DetectSmoke, ZoomChecks, MaxFps (6: evidence is per second of
scene), CandidateScore, MinObjectSize (49a53c3's knob, kept: a candidate
under that fraction of the frame is noise), ExclusiveROIs. Engines load
inside Initialize: a lazily loaded one faulted in ANSLIB's inference entry
on the first frame.

FireNSmokeDetection_Run replays a recording on the video's clock (parity
harness with firesmoke-run). Measured, same package, TensorRT:
  15.9.9.0_9.51_real_smoke   63 alarms, first at 3.0 s (node: 56, 3.0 s)
  15.9.9.51.10.51_real_smoke 113 alarms, first at 3.0 s (node: 126, 3.0 s)
  8 KA false_* clips          0 alarms (node: 0)
Unit tests updated for the new parameter schema; 9/9 pass. The headless
checker from 49a53c3 builds unchanged against the new class.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
2026-11-01 12:15:33 +11:00
co-authored by Claude Fable 5.1
parent 49a53c344e
commit e893be4a75
10 changed files with 2113 additions and 1400 deletions
@@ -1,176 +1,86 @@
// ANSCustomFireNSmoke.h — the ANSVIS Fire & Smoke custom model, judged by
// the same verifier the ANSMIND `vision.fire_smoke` node runs.
//
// The first generation of this DLL found a detection in one section of the
// frame, latched a crop round it and voted on tracked boxes inside that crop.
// Measured on the field recordings that was itself the main false-alarm
// source: the detector needs CONTEXT (a floodlight is `fake_fire` on the
// whole frame and `fire` on a tight crop of the same lamp), the active window
// lost the rest of the picture, and a PTZ tour broke the latch. ANSMIND's
// native node replaced that with a whole-frame-first verifier whose every cue
// is box-relative and carried by global motion (PTZ-safe), with native-
// resolution zoom re-checks for fire only, a look-alike filter, and a ladder
// of physical gates measured on real and false recordings (vendor/engine/
// fire_smoke.hpp, pure C++, no OpenCV).
//
// This DLL now feeds that verifier: ANSLIB runs the package's detector on
// the whole frame (low floor, class-agnostic NMS), the filter model when
// there is something to explain, and the detector again on the few native
// crops the verifier asks for; the verifier decides, and only CONFIRMED fire
// / smoke tracks come back as CustomObjects, every frame while they last.
// So ANSVIS (this DLL in its server) and ANSMIND (the node, or this DLL in
// ans-custom-host) judge a scene identically.
#include "ANSLIB.h"
#include <deque>
#include <unordered_set>
#include <unordered_map>
#include "vendor/engine/fire_smoke.hpp"
#include <chrono>
#include <memory>
#include <mutex>
#include <string>
#include <vector>
#define RETAINFRAMES 80
#define FILTERFRAMES 10
//#define FNS_DEBUG
class CUSTOM_API ANSCustomFS : public IANSCustomClass
{
typedef std::pair<cv::Scalar, cv::Scalar> Range;
struct ImageSection {
cv::Rect region;
int priority;
ImageSection(const cv::Rect& r) : region(r), priority(0) {}
};
// Track record for voting-based detection confirmation
struct TrackRecord {
int trackId{ 0 };
int classId{ 0 }; // fire=0, smoke=2
std::deque<cv::Rect> bboxHistory; // bounding box history within window
std::deque<int> highConfHits; // parallel: 1 if >=HIGH_CONF_THRESHOLD, else 0
std::deque<double> vMeanHistory; // parallel: mean HSV V inside bbox (for flicker)
int detectedCount{ 0 }; // high-conf hits in sliding window
int lowConfDetectedCount{ 0 }; // total hits (any conf >= LOW_CONF_FIRE_THRESHOLD)
int totalFrames{ 0 }; // total frames since track appeared
bool confirmed{ false }; // passed high-conf voting threshold
};
// Dual-path voting mechanism constants
static constexpr int VOTE_WINDOW = 15; // sliding window size (frames)
static constexpr int VOTE_THRESHOLD = 8; // high-conf fast path (>=0.5)
static constexpr int LOW_VOTE_THRESHOLD = 6; // low-conf slow path (>=0.25 + gates)
static constexpr float HIGH_CONF_THRESHOLD = 0.5f; // boundary between low/high conf
static constexpr float LOW_CONF_FIRE_THRESHOLD = 0.25f; // detector floor for small-fire path
static constexpr float BBOX_CHANGE_THRESHOLD = 0.05f;
// V-channel flicker gate: real fire oscillates in luma; sunlight does not
static constexpr double FLICKER_V_STD_THRESHOLD = 3.0;
static constexpr int FLICKER_MIN_SAMPLES = 6;
// Minimum fire/smoke box area as a fraction of the full frame (configurable via "MinObjectSize").
// Resolution-independent: rejects tiny boxes (the dominant small false-positive source).
static constexpr float DEFAULT_MIN_OBJECT_RATIO = 0.0003f; // 0.03% of frame area
private:
using ANSLIBPtr = std::unique_ptr<ANSCENTER::ANSLIB, decltype(&ANSCENTER::ANSLIB::Destroy)>;
using Role = ansmind::firesmoke::Role;
using RectF = ansmind::RectF;
/// One detector verdict in normalised frame coordinates.
struct Det {
RectF box;
int classId = -1;
float conf = 0;
};
int engineType{ 0 };
ANSLIBPtr _motiondetector{ nullptr, &ANSCENTER::ANSLIB::Destroy };
ANSLIBPtr _detector{ nullptr, &ANSCENTER::ANSLIB::Destroy };
ANSLIBPtr _filter{ nullptr, &ANSCENTER::ANSLIB::Destroy };
std::recursive_mutex _mutex;
std::recursive_mutex _mutex;
cv::Rect _detectedArea;
cv::Rect _previousDetectedArea;
int _retainDetectedArea{ 0 };
bool _isFireNSmokeDetected{ false };
float _detectionScoreThreshold{ 0.5 };
std::vector<cv::Mat> _frameHistory;
std::deque <cv::Mat> _frameQueue;
std::string _filterLabels;
int _realFireCheck{ 0 };
bool _isRealFireFrame{ false };
double _hsvThreshold;
std::vector<Range> _smoke_colour;
std::vector<Range> _fire_colour;
// ----- the package's classes -----
std::vector<std::string> _labels; ///< detector classes, in model order
std::vector<Role> _roles; ///< fire / fake_fire / smoke / fake_smoke per class
int _fireClass{ -1 }, _smokeClass{ -1 };
std::string _fireName, _smokeName;
std::vector<std::string> _filterLabels;
std::vector<char> _vehicle, _screen; ///< which filter classes explain a look-alike away
// ----- Model config -----
int _detectorModelType;
int _detectorDetectionType;
int _filterModelType;
int _filterDetectionType;
int _motionModelType;
int _motionDetectionType;
// ----- the verifier and its knobs -----
ansmind::firesmoke::Verifier _verifier;
ansmind::firesmoke::Params _p;
int _zoomBudget{ 2 };
double _maxFps{ 6.0 };
float _minObjectAreaRatio{ 0.0003f }; ///< a candidate box under this fraction of the frame is noise ("MinObjectSize", %)
bool _fsLog{ false };
std::vector<cv::Rect> _exclusiveROIs; ///< pixel rectangles no candidate may touch
std::string _paramSignature; ///< what the last applyParams() saw
// Add these constants at class level
int MIN_ROI_SIZE = 50;
float SMOKE_CONFIDENCE_THRESHOLD = 0.7f;
float DETECTION_CONFIDENCE_THRESHOLD = 0.5f;
float MAX_AREA_OVERLAP = 0.5f;
int MAX_MOTION_TRACKING = 10;
int FILTER_VERIFICATION_FRAMES = 10;
int MIN_IMAGE_SIZE = 10;
float INCLUSIVE_IOU_THRESHOLD = 0.001f;
float MOTION_SENSITIVITY = 0.5f;
// ----- time -----
std::chrono::steady_clock::time_point _t0{};
bool _clockStarted{ false };
double _externalClock{ -1 }; ///< a replay's video time, when set
double _lastRunT{ -1e9 };
std::vector<CustomObject> _lastResults;
uint64_t _frames{ 0 };
// Excluded filter class IDs
const std::unordered_set<int> EXCLUDED_FILTER_CLASSES =
{
13, // bench
59, // bed
60, // dining table
68, // microwave
69, // oven
70 // toaster
};
cv::Size previousImageSize = cv::Size(0, 0);
std::vector<ImageSection> cachedSections;
int _currentPriority{ 0 };
bool _readROIs{ false };
std::vector<cv::Rect> _exclusiveROIs;
float _smokeDetetectionThreshold{ 0 };
float _motionSpecificity{ 0 };
// Min fire/smoke box area as a fraction of the full frame; see DEFAULT_MIN_OBJECT_RATIO.
float _minObjectAreaRatio{ DEFAULT_MIN_OBJECT_RATIO };
// Tracker-based voting state
std::unordered_map<int, TrackRecord> _trackHistory;
cv::Rect GenerateMinimumSquareBoundingBox(const std::vector<ANSCENTER::Object>& detectedObjects, int minSize = 640);
void UpdateNoDetectionCondition();
bool detectScreenFlicker(const std::vector<cv::Mat>& frames);
bool detectReflection(const cv::Mat& frame);
bool MajorityColourInFrame(cv::Mat frame, Range range, float area_threshold);
bool MajorityColourInFrame(cv::Mat frame, std::vector<Range> ranges, float area_threshold);
bool DetectFireNSmokeColourInFrame(const cv::Mat& frame, const cv::Rect& bBox, const std::vector<Range>& ranges, float area_threshold);
bool IsFireDetected(const cv::Mat image, const cv::Rect bBox);
bool IsSmokeDetected(const cv::Mat image, const cv::Rect bBox);
float calculateIoU(const cv::Rect& box1, const cv::Rect& box2);
bool IsOverlapping(const ANSCENTER::Object& obj, const std::vector<ANSCENTER::Object>& objectList, float iouThreshold = 0.5);
bool IsOverlapping(const ANSCENTER::Object& obj, const std::vector<cv::Rect>& objectList, float iouThreshold = 0.5);
bool IsROIOverlapping(const cv::Rect& obj, const std::vector<cv::Rect>& objectList, float iouThreshold = 0.5);
bool IsDetectedAreaValid(const cv::Rect& area) const;
bool IsObjectLargeEnough(const cv::Rect& box, const cv::Size& frameSize) const;
bool IsFireOrSmoke(int classId, float confidence) const;
bool ValidateMotionCorrelation(const std::vector<cv::Rect>& fireNSmokeRects) const;
void UpdatePositiveDetection();
void ResetDetectedArea();
void UpdatePriorityRegion(const cv::Mat& frame);
int CalculateOptimalCropSize(const cv::Mat& frame) const;
void RefineDetectedArea(const cv::Mat& frame, const ANSCENTER::Object& detection);
std::vector<CustomObject> ConvertToCustomObjects(const std::vector<ANSCENTER::Object>& objects);
void ResetDetectionState();
void GetModelParameters();
std::vector<ANSCENTER::Object> ProcessExistingDetectedArea(const cv::Mat& frame, const std::string& camera_id, cv::Mat& draw);
void AddConfirmedDetection(ANSCENTER::Object& detectedObj, std::vector<ANSCENTER::Object>& output);
#ifdef FNS_DEBUG
void DisplayDebugFrame(cv::Mat& draw) {
cv::resize(draw, draw, cv::Size(1920, (1920 * draw.rows) / draw.cols));
cv::imshow("Combined Detected Areas", draw);
cv::waitKey(1);
}
#endif
// Function to separate screen size
double calculateDistanceToCenter(const cv::Point& center, const cv::Rect& rect);
std::vector<ImageSection> divideImage(const cv::Mat& image);
int getHighestPriorityRegion();
int getLowestPriorityRegion();
cv::Rect getRegionByPriority(int priority);
// Tracker-based voting methods (dual path: high-conf fast + low-conf slow)
void UpdateTrackHistory(int trackId, int classId, const cv::Rect& bbox, float confidence, double vMean);
bool IsTrackConfirmed(int trackId) const; // high-conf path
bool IsTrackLowConfirmed(int trackId) const; // low-conf path (fire only)
bool HasBboxMovement(int trackId) const;
bool HasFireFlicker(int trackId) const; // V-channel temporal std-dev gate
void AgeTracks(const std::unordered_set<int>& detectedTrackIds);
// Warm-color priority scan: re-rank cached sections by HSV warm-pixel count
void RankCachedSectionsByWarmColor(const cv::Mat& frame);
std::vector<ANSCENTER::Object> FindNewDetectedArea(
const cv::Mat& frame,
const std::string& camera_id, cv::Mat& draw);
std::vector<ANSCENTER::Object> FindMovementObjects(
const cv::Mat& frame,
const std::string& camera_id, cv::Mat& draw);
std::vector<ANSCENTER::Object> FindExcludedObjects(
const cv::Mat& frame,
const std::string& camera_id, cv::Mat& draw);
// ----- steps of a frame -----
void applyParams();
bool runDetector(const cv::Mat& bgr, const cv::Rect& roi, std::vector<Det>& out, const std::string& camera);
bool runFilter(const cv::Mat& bgr, const cv::Rect& roi, std::vector<Det>& out, const std::string& camera);
static void nmsClassAgnostic(std::vector<Det>& dets, float iouThr);
ansmind::firesmoke::Cues cuesFor(const cv::Mat& bgr, const RectF& box) const;
static cv::Rect pixelRect(const RectF& r, int W, int H);
static RectF normRect(const cv::Rect& r, int W, int H);
static float cover(const RectF& a, const RectF& b);
public:
bool Initialize(const std::string& modelDiretory, float detectionScoreThreshold, std::string& labelMap) override;
@@ -179,6 +89,15 @@ public:
std::vector<CustomObject> RunInference(const cv::Mat& input) override;
std::vector<CustomObject> RunInference(const cv::Mat& input, const std::string& camera_id) override;
bool Destroy() override;
/// Not part of the plugin contract. A replay harness hands the frame's
/// own time in seconds so the verifier's per-second evidence and gates
/// run on VIDEO time, not on how fast the machine decodes. Live callers
/// (ANSVIS, ans-custom-host) never call it and the steady clock is used.
void SetFrameClock(double seconds) { _externalClock = seconds; }
/// The verifier's per-track debug view after the last frame (replay harness).
std::vector<ansmind::firesmoke::TrackDebug> LastDebug() const { return _verifier.debug(); }
ANSCustomFS();
~ANSCustomFS();
};