ANSCustomFireNSmokeDetection: the ANSVIS custom model is judged by ANSMIND's Fire & Smoke verifier
The DLL used to find a detection in one section of the frame, latch a crop
round it and vote on tracked boxes inside that crop. Measured on the field
recordings that was itself the main false-alarm source (the detector needs
context: a floodlight is fake_fire on the whole frame and fire on a tight
crop of the same lamp), and a PTZ tour broke the latch. ANSMIND's native
vision.fire_smoke node replaced it with a whole-frame-first verifier whose
cues are box-relative and carried by global motion, fire-only native zoom
re-checks, a look-alike filter and a ladder of gates measured on real and
false recordings.
The DLL now feeds that verifier, vendored verbatim from ANSMINDCORE dba11ac
(vendor/engine/fire_smoke.hpp, pure C++): ANSLIB runs the package's
detector on the whole frame at a low floor with class-agnostic NMS, the COCO
filter when there is a candidate to explain, and the detector again on the
few native crops the verifier plans; only confirmed tracks come back, as
fire/smoke CustomObjects with fs_seconds/fs_track/evidence in extraInfo.
Parameters: Sensitivity (slider onto low/balanced/high), SmokeSeconds,
DetectFire/DetectSmoke, ZoomChecks, MaxFps (6: evidence is per second of
scene), CandidateScore, MinObjectSize (49a53c3's knob, kept: a candidate
under that fraction of the frame is noise), ExclusiveROIs. Engines load
inside Initialize: a lazily loaded one faulted in ANSLIB's inference entry
on the first frame.
FireNSmokeDetection_Run replays a recording on the video's clock (parity
harness with firesmoke-run). Measured, same package, TensorRT:
15.9.9.0_9.51_real_smoke 63 alarms, first at 3.0 s (node: 56, 3.0 s)
15.9.9.51.10.51_real_smoke 113 alarms, first at 3.0 s (node: 126, 3.0 s)
8 KA false_* clips 0 alarms (node: 0)
Unit tests updated for the new parameter schema; 9/9 pass. The headless
checker from 49a53c3 builds unchanged against the new class.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
@@ -1,176 +1,86 @@
|
||||
// ANSCustomFireNSmoke.h — the ANSVIS Fire & Smoke custom model, judged by
|
||||
// the same verifier the ANSMIND `vision.fire_smoke` node runs.
|
||||
//
|
||||
// The first generation of this DLL found a detection in one section of the
|
||||
// frame, latched a crop round it and voted on tracked boxes inside that crop.
|
||||
// Measured on the field recordings that was itself the main false-alarm
|
||||
// source: the detector needs CONTEXT (a floodlight is `fake_fire` on the
|
||||
// whole frame and `fire` on a tight crop of the same lamp), the active window
|
||||
// lost the rest of the picture, and a PTZ tour broke the latch. ANSMIND's
|
||||
// native node replaced that with a whole-frame-first verifier whose every cue
|
||||
// is box-relative and carried by global motion (PTZ-safe), with native-
|
||||
// resolution zoom re-checks for fire only, a look-alike filter, and a ladder
|
||||
// of physical gates measured on real and false recordings (vendor/engine/
|
||||
// fire_smoke.hpp, pure C++, no OpenCV).
|
||||
//
|
||||
// This DLL now feeds that verifier: ANSLIB runs the package's detector on
|
||||
// the whole frame (low floor, class-agnostic NMS), the filter model when
|
||||
// there is something to explain, and the detector again on the few native
|
||||
// crops the verifier asks for; the verifier decides, and only CONFIRMED fire
|
||||
// / smoke tracks come back as CustomObjects, every frame while they last.
|
||||
// So ANSVIS (this DLL in its server) and ANSMIND (the node, or this DLL in
|
||||
// ans-custom-host) judge a scene identically.
|
||||
#include "ANSLIB.h"
|
||||
#include <deque>
|
||||
#include <unordered_set>
|
||||
#include <unordered_map>
|
||||
#include "vendor/engine/fire_smoke.hpp"
|
||||
|
||||
#include <chrono>
|
||||
#include <memory>
|
||||
#include <mutex>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#define RETAINFRAMES 80
|
||||
#define FILTERFRAMES 10
|
||||
//#define FNS_DEBUG
|
||||
class CUSTOM_API ANSCustomFS : public IANSCustomClass
|
||||
{
|
||||
typedef std::pair<cv::Scalar, cv::Scalar> Range;
|
||||
struct ImageSection {
|
||||
cv::Rect region;
|
||||
int priority;
|
||||
ImageSection(const cv::Rect& r) : region(r), priority(0) {}
|
||||
};
|
||||
|
||||
// Track record for voting-based detection confirmation
|
||||
struct TrackRecord {
|
||||
int trackId{ 0 };
|
||||
int classId{ 0 }; // fire=0, smoke=2
|
||||
std::deque<cv::Rect> bboxHistory; // bounding box history within window
|
||||
std::deque<int> highConfHits; // parallel: 1 if >=HIGH_CONF_THRESHOLD, else 0
|
||||
std::deque<double> vMeanHistory; // parallel: mean HSV V inside bbox (for flicker)
|
||||
int detectedCount{ 0 }; // high-conf hits in sliding window
|
||||
int lowConfDetectedCount{ 0 }; // total hits (any conf >= LOW_CONF_FIRE_THRESHOLD)
|
||||
int totalFrames{ 0 }; // total frames since track appeared
|
||||
bool confirmed{ false }; // passed high-conf voting threshold
|
||||
};
|
||||
|
||||
// Dual-path voting mechanism constants
|
||||
static constexpr int VOTE_WINDOW = 15; // sliding window size (frames)
|
||||
static constexpr int VOTE_THRESHOLD = 8; // high-conf fast path (>=0.5)
|
||||
static constexpr int LOW_VOTE_THRESHOLD = 6; // low-conf slow path (>=0.25 + gates)
|
||||
static constexpr float HIGH_CONF_THRESHOLD = 0.5f; // boundary between low/high conf
|
||||
static constexpr float LOW_CONF_FIRE_THRESHOLD = 0.25f; // detector floor for small-fire path
|
||||
static constexpr float BBOX_CHANGE_THRESHOLD = 0.05f;
|
||||
// V-channel flicker gate: real fire oscillates in luma; sunlight does not
|
||||
static constexpr double FLICKER_V_STD_THRESHOLD = 3.0;
|
||||
static constexpr int FLICKER_MIN_SAMPLES = 6;
|
||||
// Minimum fire/smoke box area as a fraction of the full frame (configurable via "MinObjectSize").
|
||||
// Resolution-independent: rejects tiny boxes (the dominant small false-positive source).
|
||||
static constexpr float DEFAULT_MIN_OBJECT_RATIO = 0.0003f; // 0.03% of frame area
|
||||
|
||||
private:
|
||||
using ANSLIBPtr = std::unique_ptr<ANSCENTER::ANSLIB, decltype(&ANSCENTER::ANSLIB::Destroy)>;
|
||||
using Role = ansmind::firesmoke::Role;
|
||||
using RectF = ansmind::RectF;
|
||||
|
||||
/// One detector verdict in normalised frame coordinates.
|
||||
struct Det {
|
||||
RectF box;
|
||||
int classId = -1;
|
||||
float conf = 0;
|
||||
};
|
||||
|
||||
int engineType{ 0 };
|
||||
ANSLIBPtr _motiondetector{ nullptr, &ANSCENTER::ANSLIB::Destroy };
|
||||
ANSLIBPtr _detector{ nullptr, &ANSCENTER::ANSLIB::Destroy };
|
||||
ANSLIBPtr _filter{ nullptr, &ANSCENTER::ANSLIB::Destroy };
|
||||
std::recursive_mutex _mutex;
|
||||
|
||||
std::recursive_mutex _mutex;
|
||||
cv::Rect _detectedArea;
|
||||
cv::Rect _previousDetectedArea;
|
||||
int _retainDetectedArea{ 0 };
|
||||
bool _isFireNSmokeDetected{ false };
|
||||
float _detectionScoreThreshold{ 0.5 };
|
||||
std::vector<cv::Mat> _frameHistory;
|
||||
std::deque <cv::Mat> _frameQueue;
|
||||
std::string _filterLabels;
|
||||
int _realFireCheck{ 0 };
|
||||
bool _isRealFireFrame{ false };
|
||||
double _hsvThreshold;
|
||||
std::vector<Range> _smoke_colour;
|
||||
std::vector<Range> _fire_colour;
|
||||
// ----- the package's classes -----
|
||||
std::vector<std::string> _labels; ///< detector classes, in model order
|
||||
std::vector<Role> _roles; ///< fire / fake_fire / smoke / fake_smoke per class
|
||||
int _fireClass{ -1 }, _smokeClass{ -1 };
|
||||
std::string _fireName, _smokeName;
|
||||
std::vector<std::string> _filterLabels;
|
||||
std::vector<char> _vehicle, _screen; ///< which filter classes explain a look-alike away
|
||||
|
||||
// ----- Model config -----
|
||||
int _detectorModelType;
|
||||
int _detectorDetectionType;
|
||||
int _filterModelType;
|
||||
int _filterDetectionType;
|
||||
int _motionModelType;
|
||||
int _motionDetectionType;
|
||||
// ----- the verifier and its knobs -----
|
||||
ansmind::firesmoke::Verifier _verifier;
|
||||
ansmind::firesmoke::Params _p;
|
||||
int _zoomBudget{ 2 };
|
||||
double _maxFps{ 6.0 };
|
||||
float _minObjectAreaRatio{ 0.0003f }; ///< a candidate box under this fraction of the frame is noise ("MinObjectSize", %)
|
||||
bool _fsLog{ false };
|
||||
std::vector<cv::Rect> _exclusiveROIs; ///< pixel rectangles no candidate may touch
|
||||
std::string _paramSignature; ///< what the last applyParams() saw
|
||||
|
||||
// Add these constants at class level
|
||||
int MIN_ROI_SIZE = 50;
|
||||
float SMOKE_CONFIDENCE_THRESHOLD = 0.7f;
|
||||
float DETECTION_CONFIDENCE_THRESHOLD = 0.5f;
|
||||
float MAX_AREA_OVERLAP = 0.5f;
|
||||
int MAX_MOTION_TRACKING = 10;
|
||||
int FILTER_VERIFICATION_FRAMES = 10;
|
||||
int MIN_IMAGE_SIZE = 10;
|
||||
float INCLUSIVE_IOU_THRESHOLD = 0.001f;
|
||||
float MOTION_SENSITIVITY = 0.5f;
|
||||
// ----- time -----
|
||||
std::chrono::steady_clock::time_point _t0{};
|
||||
bool _clockStarted{ false };
|
||||
double _externalClock{ -1 }; ///< a replay's video time, when set
|
||||
double _lastRunT{ -1e9 };
|
||||
std::vector<CustomObject> _lastResults;
|
||||
uint64_t _frames{ 0 };
|
||||
|
||||
// Excluded filter class IDs
|
||||
const std::unordered_set<int> EXCLUDED_FILTER_CLASSES =
|
||||
{
|
||||
13, // bench
|
||||
59, // bed
|
||||
60, // dining table
|
||||
68, // microwave
|
||||
69, // oven
|
||||
70 // toaster
|
||||
};
|
||||
|
||||
cv::Size previousImageSize = cv::Size(0, 0);
|
||||
std::vector<ImageSection> cachedSections;
|
||||
int _currentPriority{ 0 };
|
||||
bool _readROIs{ false };
|
||||
std::vector<cv::Rect> _exclusiveROIs;
|
||||
float _smokeDetetectionThreshold{ 0 };
|
||||
float _motionSpecificity{ 0 };
|
||||
// Min fire/smoke box area as a fraction of the full frame; see DEFAULT_MIN_OBJECT_RATIO.
|
||||
float _minObjectAreaRatio{ DEFAULT_MIN_OBJECT_RATIO };
|
||||
|
||||
// Tracker-based voting state
|
||||
std::unordered_map<int, TrackRecord> _trackHistory;
|
||||
|
||||
cv::Rect GenerateMinimumSquareBoundingBox(const std::vector<ANSCENTER::Object>& detectedObjects, int minSize = 640);
|
||||
void UpdateNoDetectionCondition();
|
||||
bool detectScreenFlicker(const std::vector<cv::Mat>& frames);
|
||||
bool detectReflection(const cv::Mat& frame);
|
||||
bool MajorityColourInFrame(cv::Mat frame, Range range, float area_threshold);
|
||||
bool MajorityColourInFrame(cv::Mat frame, std::vector<Range> ranges, float area_threshold);
|
||||
bool DetectFireNSmokeColourInFrame(const cv::Mat& frame, const cv::Rect& bBox, const std::vector<Range>& ranges, float area_threshold);
|
||||
bool IsFireDetected(const cv::Mat image, const cv::Rect bBox);
|
||||
bool IsSmokeDetected(const cv::Mat image, const cv::Rect bBox);
|
||||
float calculateIoU(const cv::Rect& box1, const cv::Rect& box2);
|
||||
bool IsOverlapping(const ANSCENTER::Object& obj, const std::vector<ANSCENTER::Object>& objectList, float iouThreshold = 0.5);
|
||||
bool IsOverlapping(const ANSCENTER::Object& obj, const std::vector<cv::Rect>& objectList, float iouThreshold = 0.5);
|
||||
|
||||
bool IsROIOverlapping(const cv::Rect& obj, const std::vector<cv::Rect>& objectList, float iouThreshold = 0.5);
|
||||
bool IsDetectedAreaValid(const cv::Rect& area) const;
|
||||
bool IsObjectLargeEnough(const cv::Rect& box, const cv::Size& frameSize) const;
|
||||
bool IsFireOrSmoke(int classId, float confidence) const;
|
||||
bool ValidateMotionCorrelation(const std::vector<cv::Rect>& fireNSmokeRects) const;
|
||||
void UpdatePositiveDetection();
|
||||
void ResetDetectedArea();
|
||||
void UpdatePriorityRegion(const cv::Mat& frame);
|
||||
int CalculateOptimalCropSize(const cv::Mat& frame) const;
|
||||
void RefineDetectedArea(const cv::Mat& frame, const ANSCENTER::Object& detection);
|
||||
std::vector<CustomObject> ConvertToCustomObjects(const std::vector<ANSCENTER::Object>& objects);
|
||||
void ResetDetectionState();
|
||||
void GetModelParameters();
|
||||
std::vector<ANSCENTER::Object> ProcessExistingDetectedArea(const cv::Mat& frame, const std::string& camera_id, cv::Mat& draw);
|
||||
void AddConfirmedDetection(ANSCENTER::Object& detectedObj, std::vector<ANSCENTER::Object>& output);
|
||||
#ifdef FNS_DEBUG
|
||||
void DisplayDebugFrame(cv::Mat& draw) {
|
||||
cv::resize(draw, draw, cv::Size(1920, (1920 * draw.rows) / draw.cols));
|
||||
cv::imshow("Combined Detected Areas", draw);
|
||||
cv::waitKey(1);
|
||||
}
|
||||
#endif
|
||||
// Function to separate screen size
|
||||
double calculateDistanceToCenter(const cv::Point& center, const cv::Rect& rect);
|
||||
std::vector<ImageSection> divideImage(const cv::Mat& image);
|
||||
int getHighestPriorityRegion();
|
||||
int getLowestPriorityRegion();
|
||||
cv::Rect getRegionByPriority(int priority);
|
||||
|
||||
// Tracker-based voting methods (dual path: high-conf fast + low-conf slow)
|
||||
void UpdateTrackHistory(int trackId, int classId, const cv::Rect& bbox, float confidence, double vMean);
|
||||
bool IsTrackConfirmed(int trackId) const; // high-conf path
|
||||
bool IsTrackLowConfirmed(int trackId) const; // low-conf path (fire only)
|
||||
bool HasBboxMovement(int trackId) const;
|
||||
bool HasFireFlicker(int trackId) const; // V-channel temporal std-dev gate
|
||||
void AgeTracks(const std::unordered_set<int>& detectedTrackIds);
|
||||
|
||||
// Warm-color priority scan: re-rank cached sections by HSV warm-pixel count
|
||||
void RankCachedSectionsByWarmColor(const cv::Mat& frame);
|
||||
std::vector<ANSCENTER::Object> FindNewDetectedArea(
|
||||
const cv::Mat& frame,
|
||||
const std::string& camera_id, cv::Mat& draw);
|
||||
|
||||
std::vector<ANSCENTER::Object> FindMovementObjects(
|
||||
const cv::Mat& frame,
|
||||
const std::string& camera_id, cv::Mat& draw);
|
||||
|
||||
std::vector<ANSCENTER::Object> FindExcludedObjects(
|
||||
const cv::Mat& frame,
|
||||
const std::string& camera_id, cv::Mat& draw);
|
||||
// ----- steps of a frame -----
|
||||
void applyParams();
|
||||
bool runDetector(const cv::Mat& bgr, const cv::Rect& roi, std::vector<Det>& out, const std::string& camera);
|
||||
bool runFilter(const cv::Mat& bgr, const cv::Rect& roi, std::vector<Det>& out, const std::string& camera);
|
||||
static void nmsClassAgnostic(std::vector<Det>& dets, float iouThr);
|
||||
ansmind::firesmoke::Cues cuesFor(const cv::Mat& bgr, const RectF& box) const;
|
||||
static cv::Rect pixelRect(const RectF& r, int W, int H);
|
||||
static RectF normRect(const cv::Rect& r, int W, int H);
|
||||
static float cover(const RectF& a, const RectF& b);
|
||||
|
||||
public:
|
||||
bool Initialize(const std::string& modelDiretory, float detectionScoreThreshold, std::string& labelMap) override;
|
||||
@@ -179,6 +89,15 @@ public:
|
||||
std::vector<CustomObject> RunInference(const cv::Mat& input) override;
|
||||
std::vector<CustomObject> RunInference(const cv::Mat& input, const std::string& camera_id) override;
|
||||
bool Destroy() override;
|
||||
|
||||
/// Not part of the plugin contract. A replay harness hands the frame's
|
||||
/// own time in seconds so the verifier's per-second evidence and gates
|
||||
/// run on VIDEO time, not on how fast the machine decodes. Live callers
|
||||
/// (ANSVIS, ans-custom-host) never call it and the steady clock is used.
|
||||
void SetFrameClock(double seconds) { _externalClock = seconds; }
|
||||
/// The verifier's per-track debug view after the last frame (replay harness).
|
||||
std::vector<ansmind::firesmoke::TrackDebug> LastDebug() const { return _verifier.debug(); }
|
||||
|
||||
ANSCustomFS();
|
||||
~ANSCustomFS();
|
||||
};
|
||||
|
||||
Reference in New Issue
Block a user