feat(ODEngine): open a model package and run it without unpacking it

An engine's model has always been a path to a password-protected zip:
ANSODBase inflates it into a folder, and every engine then picks out
train_last.onnx, classes.names and model_config.json. A .ansmodel
package replaces all four with one sealed file, and this is the part
that turns it back into what the engines already understand.

ANSODBase::LoadModel and ::Initialize now recognise a .ansmodel path and
take a second route: the package is opened with the process keyring, the
model is decrypted into memory, and the classes, task and input geometry
come out of the manifest instead of loose files. Nothing is extracted,
so the unprotected model never touches the disk — which is the reason
the format exists, and it is one function's job to keep.

The engines follow it down. BasicOrtHandler gains a constructor that
takes the bytes rather than a path; it keeps a view of them, not a copy,
because the weights are in RAM either way once a session exists and a
copy would only double what every loaded model costs. The owner of the
plaintext is ANSODBase, which holds it for as long as the engine object
lives — that is what lets the DirectML recovery path rebuild a session
after a driver fault — and wipes it on teardown.

Wired through ANSONNXYOLO (and so the classification facade), ANSONNXOBB,
ANSONNXPOSE, ANSONNXSEG and YOLO12OD. YOLOOD is left alone: it loads
darknet weights and OpenCV DNN graphs, neither of which the writer
produces. A zip still loads exactly as before, on every engine.

Keys come from SetProcessModelKeys, as in ANSMINDCORE, with the same
development fallback from the environment. With none set, a package is
refused with a sentence saying so and every zip still works, so a build
made before the key ceremony runs everything except packages.

The engine layer needs CUDA to compile, so the contract it depends on is
checked where it can be: ANSModelPackage-UnitTest now opens the fixture
ANSMINDCORE wrote through the same glue ANSODBase uses, and covers the
refusal a keyless build gives. 32 checks, no failures.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-09-12 22:21:39 +10:00
co-authored by Claude Opus 5
parent 6c164c3d0f
commit 080910c8ac
13 changed files with 723 additions and 33 deletions
+50
View File
@@ -20,6 +20,7 @@
#include <chrono>
#include "ANSMOT.h"
#include "onnxruntime_cxx_api.h"
#include "ANSModelSource.h"
#define USEONNXOV
#define RETAINDETECTEDFRAMES 80
@@ -797,6 +798,55 @@ namespace ANSCENTER
void CheckLicense();
void LoadClassesFromString();
void LoadClassesFromFile();
// ────────────────────────────────────────────────────────────────
// A model that arrived as an .ansmodel package.
//
// A package replaces the whole inflated folder: the model, the
// classes and the geometry model_config.json used to carry all
// come out of it, and nothing is written to disk. LoadModelPackage
// fills these in; the engine then builds its session from
// _packageModel instead of from _modelFilePath.
//
// The bytes stay for as long as the engine object does, because
// the ONNX handler holds a view of them and rebuilds its session
// from that view when a GPU driver fault forces a recreate.
// WipeModelPackage() clears them on teardown.
// ────────────────────────────────────────────────────────────────
bool _fromPackage{ false };
std::vector<unsigned char> _packageModel;
std::string _packageTask; // detection, classification, …
std::string _packageFamily; // yolo | yolo10 | rfdetr
std::string _packageMarker; // what the zip would have carried
std::string _packageName; // what logs call this model
int _packageInputWidth{ 0 }; // 0: the package does not say
int _packageInputHeight{ 0 };
// Where this engine's session should read its model from: the
// package's bytes when one was opened, the file otherwise. Engines
// keep their own _modelFilePath, so it is passed in.
ModelSource CurrentModelSource(const std::string& filePath) const {
if (_fromPackage && !_packageModel.empty())
return ModelSource::Memory(_packageModel.data(),
_packageModel.size(), _packageName);
return ModelSource::File(filePath);
}
// Engines reset the input geometry to their own default after the
// base class has loaded, which would throw away a package's own
// (a classification model is 224, not 640). This puts it back.
void ApplyPackageInputSize() {
if (_packageInputWidth > 0) _modelConfig.inpWidth = _packageInputWidth;
if (_packageInputHeight > 0) _modelConfig.inpHeight = _packageInputHeight;
}
static bool IsModelPackagePath(const std::string& path);
// Opens one, decrypts the model into _packageModel, and fills
// _classes and _modelConfig's geometry from the manifest. Returns
// false with the reason logged.
bool LoadModelPackage(const std::string& path);
void WipeModelPackage();
std::recursive_mutex _mutex;
// Guard: true while Initialize/LoadModel is in progress on any thread.
// Inference callers can check this to fail-fast instead of blocking.
+105
View File
@@ -1,9 +1,11 @@
#pragma once
#include <set>
#include <cstring>
#include <cmath>
#include <json.hpp>
#include "ANSODEngine.h"
#include "ANSLicense.h" // ANS_DBG macro
#include "ANSModelOpen.h"
#include "OpenVINODeviceConfig.h"
#include "ANSYOLOOD.h"
#include "ANSTENSORRTOD.h"
@@ -61,6 +63,93 @@ namespace ANSCENTER
this->_logger.LogFatal("ANSODBase::CheckLicense. Error:", e.what(), __FILE__, __LINE__);
}
}
// ────────────────────────────────────────────────────────────────────────
// Model packages (.ansmodel)
//
// A package carries what the zip's loose files carried — the model, the
// class names, the task and the input geometry — sealed to a key only
// ANSCORE, ANSMINDCORE and the engines hold. Opening one fills in the
// same members the inflated folder used to fill in, with one difference
// that is the point of the whole format: the model stays in memory.
// ────────────────────────────────────────────────────────────────────────
bool ANSODBase::IsModelPackagePath(const std::string& path) {
return ANSCENTER::modelpkg::IsPackagePath(path);
}
void ANSODBase::WipeModelPackage() {
if (!_packageModel.empty()) {
std::memset(_packageModel.data(), 0, _packageModel.size());
_packageModel.clear();
_packageModel.shrink_to_fit();
}
_fromPackage = false;
_packageInputWidth = 0;
_packageInputHeight = 0;
}
bool ANSODBase::LoadModelPackage(const std::string& path) {
try {
WipeModelPackage();
ANSCENTER::modelpkg::OpenedModel opened;
std::string err;
if (!ANSCENTER::modelpkg::OpenModelPackage(path, opened, err)) {
this->_logger.LogError("ANSODBase::LoadModelPackage", err, __FILE__, __LINE__);
return false;
}
if (!opened.warning.empty()) {
// An expired release certificate warns and loads, by design:
// a model that worked yesterday keeps working on a box that
// has been offline for a year.
this->_logger.LogWarn("ANSODBase::LoadModelPackage", opened.warning, __FILE__, __LINE__);
}
_packageModel = std::move(opened.model);
if (_packageModel.empty()) {
this->_logger.LogError("ANSODBase::LoadModelPackage",
"The package carries no model", __FILE__, __LINE__);
return false;
}
_fromPackage = true;
_packageTask = opened.task;
_packageFamily = opened.family;
_packageMarker = opened.legacyMarker;
_packageName = opened.name;
// The classes come out in order, so there is nothing to parse and
// no classes.names to look for.
if (!opened.classes.empty()) _classes = opened.classes;
_packageInputWidth = opened.inputWidth;
_packageInputHeight = opened.inputHeight;
ApplyPackageInputSize();
// The manifest says in a word what the zip said by the presence of
// a marker file.
if (opened.task == "classification") _modelConfig.detectionType = ANSCENTER::DetectionType::CLASSIFICATION;
else if (opened.task == "segmentation") _modelConfig.detectionType = ANSCENTER::DetectionType::SEGMENTATION;
else if (opened.task == "obb") _modelConfig.detectionType = ANSCENTER::DetectionType::OBB;
else if (opened.task == "pose") _modelConfig.detectionType = ANSCENTER::DetectionType::KEYPOINT;
else _modelConfig.detectionType = ANSCENTER::DetectionType::DETECTION;
// There is no folder and no config file; every engine below tests
// for these, and must find them empty rather than stale.
_modelFolder.clear();
_modelConfigFile.clear();
_classFilePath.clear();
this->_logger.LogDebug("ANSODBase::LoadModelPackage",
"Opened " + path + " (" + opened.task + "/" + opened.family + ", "
+ std::to_string(_classes.size()) + " classes, "
+ std::to_string(_packageModel.size()) + " bytes)", __FILE__, __LINE__);
// Auto-enable tracker + stabilization by default (BYTETRACK)
SetTracker(TrackerType::BYTETRACK, true);
return true;
}
catch (std::exception& e) {
this->_logger.LogFatal("ANSODBase::LoadModelPackage", e.what(), __FILE__, __LINE__);
return false;
}
}
bool ANSODBase::LoadModel(const std::string& modelZipFilePath, const std::string& modelZipPassword)
{
try {
@@ -69,6 +158,7 @@ namespace ANSCENTER
_modelConfigFile = "";
_modelFolder.clear();
_modelConfigFile.clear();
WipeModelPackage(); // whatever was loaded before, package or not
// 0. Check if the modelZipFilePath exist?
if (!FileExist(modelZipFilePath)) {
if (modelZipFilePath.empty()) {// Support motion detector that does not need model file
@@ -82,6 +172,10 @@ namespace ANSCENTER
return false;
}
}
// 0b. A model package replaces the zip entirely: it is opened in
// memory, and no password, no folder and no loose files are involved.
if (IsModelPackagePath(modelZipFilePath))
return LoadModelPackage(modelZipFilePath);
// 1. Unzip model zip file to a special location with folder name as model file (and version)
std::string outputFolder;
std::vector<std::string> passwordArray;
@@ -126,6 +220,7 @@ namespace ANSCENTER
_modelFolder.clear();
_modelConfigFile.clear();
_modelConfig = modelConfig;
WipeModelPackage(); // whatever was loaded before, package or not
CheckLicense();
if (!_licenseValid) {
this->_logger.LogError("ANSODBase::Initialize", "Invalid License", __FILE__, __LINE__);
@@ -145,6 +240,12 @@ namespace ANSCENTER
}
}
// 0b. A model package replaces the zip entirely; see LoadModel.
if (IsModelPackagePath(modelZipFilePath)) {
if (!LoadModelPackage(modelZipFilePath)) return false;
_isInitialized = true;
return true;
}
// 1. Unzip model zip file to a special location with folder name as model file (and version)
std::string outputFolder;
std::vector<std::string> passwordArray;
@@ -952,6 +1053,10 @@ namespace ANSCENTER
ANSODBase::~ANSODBase() {
try {
// A decrypted model is what the package format exists to protect.
// This runs after the derived destructor has dropped the session,
// so nothing is left holding a view of these bytes.
WipeModelPackage();
// Per-camera trackers are released by CameraData::clear()
if (!_cameras.empty())
{
+24
View File
@@ -1394,6 +1394,19 @@ namespace ANSCENTER {
if (_modelConfig.kpsThreshold == 0)_modelConfig.kpsThreshold = 0.5; // If not define
_fp16 = (modelConfig.precisionType == PrecisionType::FP16);
// A package brought its own classes and geometry, and its model never
// touched the disk: the session reads it straight from memory.
if (_fromPackage) {
ApplyPackageInputSize();
labelMap.clear();
if (!_classes.empty()) labelMap = VectorToCommaSeparatedString(_classes);
instanceId_ = instanceCounter_.fetch_add(1);
result = Init(CurrentModelSource(_modelFilePath), true, 0);
_modelLoadValid = result;
_isInitialized = result;
return result;
}
if (FileExist(_modelConfigFile)) {
ModelType modelType;
std::vector<int> inputShape;
@@ -1454,6 +1467,17 @@ namespace ANSCENTER {
// if (_modelConfig.precisionType == PrecisionType::FP16)_fp16 = true;
_fp16 = true; // Load Model from Here
// A package brought its own classes and geometry, and its model never
// touched the disk: the session reads it straight from memory.
if (_fromPackage) {
ApplyPackageInputSize();
instanceId_ = instanceCounter_.fetch_add(1);
result = Init(CurrentModelSource(_modelFilePath), true, 0);
_modelLoadValid = result;
_isInitialized = result;
return result;
}
// 0. Check if the configuration file exist
if (FileExist(_modelConfigFile)) {
ModelType modelType;
+24
View File
@@ -1282,6 +1282,19 @@ namespace ANSCENTER {
if (_modelConfig.kpsThreshold == 0)_modelConfig.kpsThreshold = 0.5; // If not define
_fp16 = (modelConfig.precisionType == PrecisionType::FP16);
// A package brought its own classes and geometry, and its model never
// touched the disk: the session reads it straight from memory.
if (_fromPackage) {
ApplyPackageInputSize();
labelMap.clear();
if (!_classes.empty()) labelMap = VectorToCommaSeparatedString(_classes);
instanceId_ = instanceCounter_.fetch_add(1);
result = Init(CurrentModelSource(_modelFilePath), true, 0);
_modelLoadValid = result;
_isInitialized = result;
return result;
}
if (FileExist(_modelConfigFile)) {
ModelType modelType;
std::vector<int> inputShape;
@@ -1343,6 +1356,17 @@ namespace ANSCENTER {
// if (_modelConfig.precisionType == PrecisionType::FP16)_fp16 = true;
_fp16 = true; // Load Model from Here
// A package brought its own classes and geometry, and its model never
// touched the disk: the session reads it straight from memory.
if (_fromPackage) {
ApplyPackageInputSize();
instanceId_ = instanceCounter_.fetch_add(1);
result = Init(CurrentModelSource(_modelFilePath), true, 0);
_modelLoadValid = result;
_isInitialized = result;
return result;
}
// 0. Check if the configuration file exist
if (FileExist(_modelConfigFile)) {
ModelType modelType;
+24
View File
@@ -1172,6 +1172,19 @@ namespace ANSCENTER {
if (_modelConfig.kpsThreshold == 0)_modelConfig.kpsThreshold = 0.5; // If not define
_fp16 = (modelConfig.precisionType == PrecisionType::FP16);
// A package brought its own classes and geometry, and its model never
// touched the disk: the session reads it straight from memory.
if (_fromPackage) {
ApplyPackageInputSize();
labelMap.clear();
if (!_classes.empty()) labelMap = VectorToCommaSeparatedString(_classes);
instanceId_ = instanceCounter_.fetch_add(1);
result = Init(CurrentModelSource(_modelFilePath), true, 0);
_modelLoadValid = result;
_isInitialized = result;
return result;
}
if (FileExist(_modelConfigFile)) {
ModelType modelType;
std::vector<int> inputShape;
@@ -1233,6 +1246,17 @@ namespace ANSCENTER {
// if (_modelConfig.precisionType == PrecisionType::FP16)_fp16 = true;
_fp16 = true; // Load Model from Here
// A package brought its own classes and geometry, and its model never
// touched the disk: the session reads it straight from memory.
if (_fromPackage) {
ApplyPackageInputSize();
instanceId_ = instanceCounter_.fetch_add(1);
result = Init(CurrentModelSource(_modelFilePath), true, 0);
_modelLoadValid = result;
_isInitialized = result;
return result;
}
// 0. Check if the configuration file exist
if (FileExist(_modelConfigFile)) {
ModelType modelType;
+97 -24
View File
@@ -41,6 +41,37 @@ namespace ANSCENTER {
}
}
ONNXYOLO::ONNXYOLO(const void* modelData, size_t modelSize,
const std::string& logName, unsigned int _num_threads)
: BasicOrtHandler(modelData, modelSize, logName, _num_threads)
{
if (input_node_dims.size() >= 4) {
int h = static_cast<int>(input_node_dims[2]);
int w = static_cast<int>(input_node_dims[3]);
isDynamicInputShape = (h == -1 || w == -1);
inputImageShape = isDynamicInputShape ? cv::Size(640, 640) : cv::Size(w, h);
}
else {
inputImageShape = cv::Size(640, 640);
}
}
ONNXYOLO::ONNXYOLO(const void* modelData, size_t modelSize,
const std::string& logName, EngineType engineType,
unsigned int _num_threads)
: BasicOrtHandler(modelData, modelSize, logName, engineType, _num_threads)
{
if (input_node_dims.size() >= 4) {
int h = static_cast<int>(input_node_dims[2]);
int w = static_cast<int>(input_node_dims[3]);
isDynamicInputShape = (h == -1 || w == -1);
inputImageShape = isDynamicInputShape ? cv::Size(640, 640) : cv::Size(w, h);
}
else {
inputImageShape = cv::Size(640, 640);
}
}
// ------------------------------------------------------------------
// letterBox — Ultralytics-compatible LetterBox transform
// ------------------------------------------------------------------
@@ -1675,6 +1706,20 @@ namespace ANSCENTER {
bool ANSONNXYOLO::InitOrtEngine() {
try {
// A model that came out of an .ansmodel package has no file
// behind it; the handler reads the bytes ANSODBase decrypted.
// Those bytes outlive the handler, which is what lets the DML
// recovery path below rebuild the session from them.
if (_fromPackage) {
if (_packageModel.empty()) {
_logger.LogError("ANSONNXYOLO::InitOrtEngine",
"The package carries no model", __FILE__, __LINE__);
return false;
}
m_ortEngine = std::make_shared<class ONNXYOLO>(
_packageModel.data(), _packageModel.size(), _packageName);
return true;
}
if (!FileExist(_modelFilePath)) {
_logger.LogError("ANSONNXYOLO::InitOrtEngine",
"Model file does not exist: " + _modelFilePath, __FILE__, __LINE__);
@@ -1691,6 +1736,16 @@ namespace ANSCENTER {
bool ANSONNXYOLO::InitOrtEngine(ANSCENTER::EngineType engineType) {
try {
if (_fromPackage) {
if (_packageModel.empty()) {
_logger.LogError("ANSONNXYOLO::InitOrtEngine",
"The package carries no model", __FILE__, __LINE__);
return false;
}
m_ortEngine = std::make_shared<class ONNXYOLO>(
_packageModel.data(), _packageModel.size(), _packageName, engineType);
return true;
}
if (!FileExist(_modelFilePath)) {
_logger.LogError("ANSONNXYOLO::InitOrtEngine",
"Model file does not exist: " + _modelFilePath, __FILE__, __LINE__);
@@ -1846,21 +1901,30 @@ namespace ANSCENTER {
NUM_KPS = _modelConfig.numKPS;
KPS_THRESHOLD = _modelConfig.kpsThreshold;
_modelFilePath = CreateFilePath(_modelFolder, "train_last.onnx");
if (FileExist(_modelConfigFile)) {
ModelType modelType;
std::vector<int> inputShape;
_classes = ANSUtilityHelper::GetConfigFileContent(_modelConfigFile, modelType, inputShape);
if (inputShape.size() == 2) {
if (inputShape[0] > 0) _modelConfig.inpHeight = inputShape[0];
if (inputShape[1] > 0) _modelConfig.inpWidth = inputShape[1];
// A package has already supplied the classes and the geometry;
// there is no folder to look in.
if (!_fromPackage) {
_modelFilePath = CreateFilePath(_modelFolder, "train_last.onnx");
if (FileExist(_modelConfigFile)) {
ModelType modelType;
std::vector<int> inputShape;
_classes = ANSUtilityHelper::GetConfigFileContent(_modelConfigFile, modelType, inputShape);
if (inputShape.size() == 2) {
if (inputShape[0] > 0) _modelConfig.inpHeight = inputShape[0];
if (inputShape[1] > 0) _modelConfig.inpWidth = inputShape[1];
}
}
else {
_classFilePath = CreateFilePath(_modelFolder, "classes.names");
std::ifstream isValid(_classFilePath);
if (!isValid) LoadClassesFromString();
else LoadClassesFromFile();
}
}
else {
_classFilePath = CreateFilePath(_modelFolder, "classes.names");
std::ifstream isValid(_classFilePath);
if (!isValid) LoadClassesFromString();
else LoadClassesFromFile();
// The defaults above have just overwritten what the package
// said its input size was; put it back.
ApplyPackageInputSize();
}
labelMap.clear();
@@ -1920,21 +1984,30 @@ namespace ANSCENTER {
NUM_KPS = _modelConfig.numKPS;
KPS_THRESHOLD = _modelConfig.kpsThreshold;
_modelFilePath = CreateFilePath(_modelFolder, "train_last.onnx");
if (FileExist(_modelConfigFile)) {
ModelType modelType;
std::vector<int> inputShape;
_classes = ANSUtilityHelper::GetConfigFileContent(_modelConfigFile, modelType, inputShape);
if (inputShape.size() == 2) {
if (inputShape[0] > 0) _modelConfig.inpHeight = inputShape[0];
if (inputShape[1] > 0) _modelConfig.inpWidth = inputShape[1];
// A package has already supplied the classes and the geometry;
// there is no folder to look in.
if (!_fromPackage) {
_modelFilePath = CreateFilePath(_modelFolder, "train_last.onnx");
if (FileExist(_modelConfigFile)) {
ModelType modelType;
std::vector<int> inputShape;
_classes = ANSUtilityHelper::GetConfigFileContent(_modelConfigFile, modelType, inputShape);
if (inputShape.size() == 2) {
if (inputShape[0] > 0) _modelConfig.inpHeight = inputShape[0];
if (inputShape[1] > 0) _modelConfig.inpWidth = inputShape[1];
}
}
else {
_classFilePath = CreateFilePath(_modelFolder, "classes.names");
std::ifstream isValid(_classFilePath);
if (!isValid) LoadClassesFromString();
else LoadClassesFromFile();
}
}
else {
_classFilePath = CreateFilePath(_modelFolder, "classes.names");
std::ifstream isValid(_classFilePath);
if (!isValid) LoadClassesFromString();
else LoadClassesFromFile();
// The defaults above have just overwritten what the package
// said its input size was; put it back.
ApplyPackageInputSize();
}
if (this->_loadEngineOnCreation) {
+10
View File
@@ -33,6 +33,16 @@ namespace ANSCENTER {
explicit ONNXYOLO(const std::string& _onnx_path,
EngineType engineType,
unsigned int _num_threads = 1);
// The decrypted contents of an .ansmodel package: the same model,
// with no file behind it. `logName` is what logs call it.
explicit ONNXYOLO(const void* modelData, size_t modelSize,
const std::string& logName,
unsigned int _num_threads = 1);
explicit ONNXYOLO(const void* modelData, size_t modelSize,
const std::string& logName,
EngineType engineType,
unsigned int _num_threads = 1);
~ONNXYOLO() override = default;
/// Run inference on a single image.
+22
View File
@@ -10,6 +10,12 @@ namespace ANSCENTER {
if (!ANSODBase::OptimizeModel(fp16, optimizedModelFolder)) {
return false;
}
if (_fromPackage) {
// A package is already the optimised artefact, and there is no
// folder to point at because nothing was written out.
optimizedModelFolder.clear();
return true;
}
if (FileExist(_modelFilePath)) {
optimizedModelFolder = GetParentFolder(_modelFilePath);
this->_logger.LogDebug("YOLO12OD::OptimizeModel", "This model is optimized. No need other optimization.", __FILE__, __LINE__);
@@ -34,6 +40,13 @@ namespace ANSCENTER {
_modelConfig.modelMNSThreshold = 0.5;
if (_modelConfig.modelConfThreshold < 0.2)
_modelConfig.modelConfThreshold = 0.5;
// A package brought its own classes and geometry, and its model never
// touched the disk: the session reads it straight from memory.
if (_fromPackage) {
ApplyPackageInputSize();
_isInitialized = loadModel(CurrentModelSource(_modelFilePath), true);
return _isInitialized;
}
// 0. Check if the configuration file exist
if (FileExist(_modelConfigFile)) {
ModelType modelType;
@@ -168,6 +181,15 @@ namespace ANSCENTER {
_modelConfig.modelMNSThreshold = 0.5;
if (_modelConfig.modelConfThreshold < 0.2)
_modelConfig.modelConfThreshold = 0.5;
// A package brought its own classes and geometry, and its model never
// touched the disk: the session reads it straight from memory.
if (_fromPackage) {
ApplyPackageInputSize();
labelMap.clear();
if (!_classes.empty()) labelMap = VectorToCommaSeparatedString(_classes);
_isInitialized = loadModel(CurrentModelSource(_modelFilePath), true);
return _isInitialized;
}
// 0. Check if the configuration file exist
if (FileExist(_modelConfigFile)) {
ModelType modelType;
+3
View File
@@ -113,6 +113,9 @@ target_include_directories(ANSODEngine PRIVATE
target_link_libraries(ANSODEngine
PUBLIC ANSLibsLoader
# Reading .ansmodel packages. Header-only apart from the vendored
# curves, and free of CUDA/OpenCV, so it costs this target nothing.
PRIVATE ANSModelPackage
PUBLIC ANSLicensingSystem # PUBLIC: FileExist/SPDLogger symbols must be re-exported
PUBLIC opencv
PRIVATE ANSMOT