feat(ODEngine): open a model package and run it without unpacking it

An engine's model has always been a path to a password-protected zip:
ANSODBase inflates it into a folder, and every engine then picks out
train_last.onnx, classes.names and model_config.json. A .ansmodel
package replaces all four with one sealed file, and this is the part
that turns it back into what the engines already understand.

ANSODBase::LoadModel and ::Initialize now recognise a .ansmodel path and
take a second route: the package is opened with the process keyring, the
model is decrypted into memory, and the classes, task and input geometry
come out of the manifest instead of loose files. Nothing is extracted,
so the unprotected model never touches the disk — which is the reason
the format exists, and it is one function's job to keep.

The engines follow it down. BasicOrtHandler gains a constructor that
takes the bytes rather than a path; it keeps a view of them, not a copy,
because the weights are in RAM either way once a session exists and a
copy would only double what every loaded model costs. The owner of the
plaintext is ANSODBase, which holds it for as long as the engine object
lives — that is what lets the DirectML recovery path rebuild a session
after a driver fault — and wipes it on teardown.

Wired through ANSONNXYOLO (and so the classification facade), ANSONNXOBB,
ANSONNXPOSE, ANSONNXSEG and YOLO12OD. YOLOOD is left alone: it loads
darknet weights and OpenCV DNN graphs, neither of which the writer
produces. A zip still loads exactly as before, on every engine.

Keys come from SetProcessModelKeys, as in ANSMINDCORE, with the same
development fallback from the environment. With none set, a package is
refused with a sentence saying so and every zip still works, so a build
made before the key ceremony runs everything except packages.

The engine layer needs CUDA to compile, so the contract it depends on is
checked where it can be: ANSModelPackage-UnitTest now opens the fixture
ANSMINDCORE wrote through the same glue ANSODBase uses, and covers the
refusal a keyless build gives. 32 checks, no failures.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-09-12 22:21:39 +10:00
co-authored by Claude Opus 5
parent 6c164c3d0f
commit 080910c8ac
13 changed files with 723 additions and 33 deletions
+212
View File
@@ -0,0 +1,212 @@
#ifndef ANSMODELOPEN_H
#define ANSMODELOPEN_H
#pragma once
// ANSModelOpen — the one place that knows how a model package reaches an engine.
//
// An engine's "model" has always been a path: a password-protected zip that
// ANSODBase inflates into a folder, after which the engine picks out
// train_last.onnx, classes.names and model_config.json. A .ansmodel package
// replaces all four with one sealed file, and this header is what turns it
// back into the things the engines already understand — the model's bytes, the
// class list, the task and the input size.
//
// THE PLAINTEXT NEVER REACHES THE DISK. That is the whole reason the format
// exists. OpenModelPackage decrypts into a buffer the caller owns; the caller
// hands that buffer to a session and wipes it. Nothing here writes a file.
//
// WHERE THE KEY COMES FROM. The product sets it once at startup with
// SetProcessModelKeys(): the base PRIVATE key packages are sealed to, and the
// root PUBLIC key that vouches for a release signing key. Nothing here reads a
// key out of the package or off the disk. With no keys set, a package is
// refused with a sentence that says so, and every existing zip still loads — a
// build from before the key ceremony runs everything except packages.
//
// The base private key must never sit in the binary as one contiguous
// constant; assembling it is the job of whatever calls SetProcessModelKeys,
// not of this header, which is why this header has no key material in it.
//
// The development fallback (ANSCORE_MODEL_KEY_ID / ANSCORE_MODEL_BASE_PRIVATE
// / ANSCORE_MODEL_ROOT_ID / ANSCORE_MODEL_ROOT_PUBLIC, all base64) exists so
// the path can be exercised before that ceremony exists. It is read once and
// announced, because a key that came from the environment is not the key a
// shipped product should be using.
#include <cstdio>
#include <cstdlib>
#include <cstring>
#include <mutex>
#include <string>
#include <vector>
#include "ANSModelPackage.h"
namespace ANSCENTER {
namespace modelpkg {
// ── the keys this process opens packages with ─────────────────────────────
namespace detail {
inline std::mutex& keysMutex() {
static std::mutex m;
return m;
}
inline Keyring& keyringStorage() {
static Keyring ring;
return ring;
}
inline bool keyFromB64(const char* b64, Key& out) {
if (!b64 || !*b64) return false;
std::vector<uint8_t> raw;
if (!bits::base64Decode(b64, raw) || raw.size() != out.size()) return false;
std::memcpy(out.data(), raw.data(), out.size());
return true;
}
// Development only: keys from the environment, once, and said out loud.
inline void loadEnvKeysOnce(Keyring& ring) {
static bool done = false;
if (done) return;
done = true;
const char* id = std::getenv("ANSCORE_MODEL_KEY_ID");
const char* priv = std::getenv("ANSCORE_MODEL_BASE_PRIVATE");
Key k{};
if (id && *id && keyFromB64(priv, k)) {
ring.addX25519(id, k);
std::fprintf(stderr,
"[modelpkg] base key %s taken from the environment - for "
"development, not for a shipped build\n",
id);
}
const char* rootId = std::getenv("ANSCORE_MODEL_ROOT_ID");
const char* rootPub = std::getenv("ANSCORE_MODEL_ROOT_PUBLIC");
Key r{};
if (rootId && *rootId && keyFromB64(rootPub, r)) {
ring.addRoot(rootId, r);
std::fprintf(stderr,
"[modelpkg] root key %s taken from the environment\n",
rootId);
}
}
} // namespace detail
inline void SetProcessModelKeys(Keyring ring) {
std::lock_guard<std::mutex> lk(detail::keysMutex());
detail::keyringStorage() = std::move(ring);
}
inline Keyring ProcessModelKeys() {
std::lock_guard<std::mutex> lk(detail::keysMutex());
Keyring& ring = detail::keyringStorage();
detail::loadEnvKeysOnce(ring);
return ring; // by value: the caller may open on another thread
}
// True once this build can open anything at all. A product that has not had
// keys installed should say "this build cannot open model packages" rather
// than let every load fail one at a time.
inline bool HaveProcessModelKeys() {
const Keyring ring = ProcessModelKeys();
return !ring.x25519.empty() || !ring.keys.empty();
}
// ── recognising one ───────────────────────────────────────────────────────
inline bool IsPackagePath(const std::string& path) {
return path.size() > 9 &&
path.compare(path.size() - 9, 9, ".ansmodel") == 0;
}
// ── what an engine needs out of one ───────────────────────────────────────
// Everything the inflated folder used to carry, in the shape the engines read
// it: the model itself, the classes in order, and the geometry that
// model_config.json held.
struct OpenedModel {
std::vector<uint8_t> model; // decrypted; wipe it once loaded
std::vector<std::string> classes;
std::string task; // detection | classification | segmentation | obb | pose
std::string family; // yolo | yolo10 | rfdetr
std::string runtime; // onnx | openvino-ir
std::string legacyMarker; // what the zip would have carried
std::string name; // what to call this model in a log
int inputWidth = 0; // 0 when the package does not say
int inputHeight = 0;
std::string warning; // show it; the model still loaded
void wipe() {
if (!model.empty()) std::memset(model.data(), 0, model.size());
model.clear();
model.shrink_to_fit();
}
};
// Read what a package says about itself without decrypting the model. Cheap
// enough to call on a folder full of packages.
inline bool DescribeModelPackage(const std::string& path, OpenedModel& out,
std::string& err) {
Package pkg;
if (!open(path, ProcessModelKeys(), pkg, err)) return false;
const Json& m = pkg.manifest;
out.task = m.value("task", std::string("detection"));
out.family = m.value("family", std::string());
out.runtime = m.value("runtime", std::string("onnx"));
out.legacyMarker =
m.value("legacy", Json::object()).value("marker", std::string());
out.warning = pkg.warning;
out.classes.clear();
for (const auto& c : m.value("classes", Json::array()))
out.classes.push_back(c.value("name", std::string()));
const Json in = m.value("input", Json::object());
out.inputWidth = in.value("width", 0);
out.inputHeight = in.value("height", 0);
out.name = m.value("project", std::string());
if (out.name.empty()) out.name = path;
return true;
}
// Open one all the way: the description above, plus the model's bytes.
//
// The caller owns those bytes and should wipe them (OpenedModel::wipe) the
// moment the session has copied what it needs.
inline bool OpenModelPackage(const std::string& path, OpenedModel& out,
std::string& err) {
if (!HaveProcessModelKeys()) {
err = "this build has no model-package key, so it cannot open " + path;
return false;
}
Package pkg;
if (!open(path, ProcessModelKeys(), pkg, err)) return false;
const std::string entry = pkg.entryPayloadId();
if (entry.empty()) {
err = "the package names no model to run";
return false;
}
if (!readPayload(pkg, entry, out.model, err)) return false;
const Json& m = pkg.manifest;
out.task = m.value("task", std::string("detection"));
out.family = m.value("family", std::string());
out.runtime = m.value("runtime", std::string("onnx"));
out.legacyMarker =
m.value("legacy", Json::object()).value("marker", std::string());
out.warning = pkg.warning;
out.classes.clear();
for (const auto& c : m.value("classes", Json::array()))
out.classes.push_back(c.value("name", std::string()));
const Json in = m.value("input", Json::object());
out.inputWidth = in.value("width", 0);
out.inputHeight = in.value("height", 0);
out.name = m.value("project", std::string());
if (out.name.empty()) out.name = path;
return true;
}
} // namespace modelpkg
} // namespace ANSCENTER
#endif // ANSMODELOPEN_H
+59 -9
View File
@@ -221,8 +221,45 @@ namespace ANSCENTER {
initialize_handler();
}
// ────────────────────────────────────────────────────────────────────
// A model that is already in memory — the decrypted contents of an
// .ansmodel package. There is no path, so onnx_path stays null and
// every file-shaped step in initialize_handler is skipped: no CWD
// switch, no external-data lookup, no file for ORT to open.
// ────────────────────────────────────────────────────────────────────
BasicOrtHandler::BasicOrtHandler(const void* modelData, size_t modelSize,
const std::string& logName,
unsigned int _num_threads)
: num_threads(_num_threads),
m_engineType(static_cast<EngineType>(-1))
{
log_id_storage = logName;
log_id = log_id_storage.c_str();
model_data = modelData;
model_size = modelSize;
initialize_handler();
}
BasicOrtHandler::BasicOrtHandler(const void* modelData, size_t modelSize,
const std::string& logName,
EngineType engineType,
unsigned int _num_threads)
: num_threads(_num_threads),
m_engineType(engineType)
{
log_id_storage = logName;
log_id = log_id_storage.c_str();
model_data = modelData;
model_size = modelSize;
initialize_handler();
}
BasicOrtHandler::~BasicOrtHandler()
{
// The model bytes, when there are any, belong to whoever handed
// them over; that owner wipes them. Only the view goes here.
model_data = nullptr;
model_size = 0;
if (ort_session) {
delete ort_session;
ort_session = nullptr;
@@ -727,20 +764,26 @@ namespace ANSCENTER {
// AddExternalInitializersFromFilesInMemory() so ORT never
// memory-maps the file itself.
// ----------------------------------------------------------------
std::filesystem::path modelFsPath(onnx_path); // wchar_t*
std::filesystem::path modelDir = modelFsPath.parent_path();
// A model in memory has no directory, no sibling .onnx_data and
// nothing for ORT to open, so all of this is skipped for one.
std::filesystem::path modelFsPath;
std::filesystem::path modelDir;
std::filesystem::path prevCwd = std::filesystem::current_path();
if (!modelDir.empty() && std::filesystem::is_directory(modelDir)) {
std::filesystem::current_path(modelDir);
std::cout << "[ORT] CWD -> " << modelDir.string() << std::endl;
if (!fromMemory()) {
modelFsPath = std::filesystem::path(onnx_path); // wchar_t*
modelDir = modelFsPath.parent_path();
if (!modelDir.empty() && std::filesystem::is_directory(modelDir)) {
std::filesystem::current_path(modelDir);
std::cout << "[ORT] CWD -> " << modelDir.string() << std::endl;
}
}
// --- Pre-load external data files if they exist -----------------
// Keep the buffer alive across session creation (must outlive the
// Ort::Session constructor call).
std::vector<char> extDataBuffer;
{
if (!fromMemory()) {
// Build the expected external-data filename:
// <model_stem>.onnx_data (e.g. anssam3.onnx_data)
std::filesystem::path extDataPath =
@@ -806,7 +849,7 @@ namespace ANSCENTER {
// This avoids ORT opening/mapping ANY files during CreateSession.
std::vector<char> modelBuffer;
bool useModelBuffer = false;
if (!extDataBuffer.empty()) {
if (!extDataBuffer.empty() && !fromMemory()) {
// External data was pre-loaded, so also load the .onnx itself
try {
auto modelFileSize = std::filesystem::file_size(modelFsPath);
@@ -829,7 +872,10 @@ namespace ANSCENTER {
// --- Attempt session creation (with CUDA → CPU fallback) --------
auto createSession = [&](Ort::SessionOptions& opts, const char* label) {
std::cout << "[ORT] Creating session (" << label << ")..." << std::endl;
if (useModelBuffer) {
if (fromMemory()) {
ort_session = new Ort::Session(*ort_env,
model_data, model_size, opts);
} else if (useModelBuffer) {
ort_session = new Ort::Session(*ort_env,
modelBuffer.data(), modelBuffer.size(), opts);
} else {
@@ -838,7 +884,11 @@ namespace ANSCENTER {
std::cout << "[ORT] Session created OK (" << label << ")." << std::endl;
};
ANS_DBG("OrtHandler", "Creating session for model: %ls", onnx_path);
if (fromMemory())
ANS_DBG("OrtHandler", "Creating session for model in memory: %s (%zu bytes)",
log_id ? log_id : "<memory>", model_size);
else
ANS_DBG("OrtHandler", "Creating session for model: %ls", onnx_path);
try {
createSession(session_options, "primary EP");
ANS_DBG("OrtHandler", "Session created OK with primary EP");
+36
View File
@@ -468,6 +468,25 @@ namespace ANSCENTER {
const LITEORT_CHAR* onnx_path = nullptr; // ← points into onnx_path_w
const char* log_id = nullptr;
// ────────────────────────────────────────────────────────────────
// A model handed over as bytes rather than as a path.
//
// An .ansmodel package is decrypted into memory and never written
// out, so there is no file for ORT to open and no directory to
// resolve external data against.
//
// This is a VIEW, not a copy: the owner of the plaintext keeps it
// alive for at least as long as this handler, because
// Reinitialize() rebuilds the session after a driver fault and has
// nothing else to rebuild it from. A copy here would double the
// resident size of every loaded model and protect nothing extra —
// the weights are in RAM either way once a session exists.
// ────────────────────────────────────────────────────────────────
const void* model_data = nullptr; // null for a file model
size_t model_size = 0;
std::string log_id_storage; // owns what log_id points at
bool fromMemory() const { return model_data != nullptr && model_size > 0; }
protected:
const unsigned int num_threads;
@@ -499,6 +518,23 @@ namespace ANSCENTER {
const OrtHandlerOptions& options,
unsigned int _num_threads = 1);
// A model already in memory — the decrypted contents of an
// .ansmodel package. `logName` is what logs call it, since there
// is no file name to fall back on.
//
// The caller KEEPS THE BYTES ALIVE until this handler is destroyed,
// and wipes them then; see model_data above for why a copy is not
// taken here.
explicit BasicOrtHandler(const void* modelData, size_t modelSize,
const std::string& logName,
unsigned int _num_threads = 1);
// The same, with an explicit engine override.
explicit BasicOrtHandler(const void* modelData, size_t modelSize,
const std::string& logName,
EngineType engineType,
unsigned int _num_threads = 1);
virtual ~BasicOrtHandler();
BasicOrtHandler(const BasicOrtHandler&) = delete;
+50
View File
@@ -20,6 +20,7 @@
#include <chrono>
#include "ANSMOT.h"
#include "onnxruntime_cxx_api.h"
#include "ANSModelSource.h"
#define USEONNXOV
#define RETAINDETECTEDFRAMES 80
@@ -797,6 +798,55 @@ namespace ANSCENTER
void CheckLicense();
void LoadClassesFromString();
void LoadClassesFromFile();
// ────────────────────────────────────────────────────────────────
// A model that arrived as an .ansmodel package.
//
// A package replaces the whole inflated folder: the model, the
// classes and the geometry model_config.json used to carry all
// come out of it, and nothing is written to disk. LoadModelPackage
// fills these in; the engine then builds its session from
// _packageModel instead of from _modelFilePath.
//
// The bytes stay for as long as the engine object does, because
// the ONNX handler holds a view of them and rebuilds its session
// from that view when a GPU driver fault forces a recreate.
// WipeModelPackage() clears them on teardown.
// ────────────────────────────────────────────────────────────────
bool _fromPackage{ false };
std::vector<unsigned char> _packageModel;
std::string _packageTask; // detection, classification, …
std::string _packageFamily; // yolo | yolo10 | rfdetr
std::string _packageMarker; // what the zip would have carried
std::string _packageName; // what logs call this model
int _packageInputWidth{ 0 }; // 0: the package does not say
int _packageInputHeight{ 0 };
// Where this engine's session should read its model from: the
// package's bytes when one was opened, the file otherwise. Engines
// keep their own _modelFilePath, so it is passed in.
ModelSource CurrentModelSource(const std::string& filePath) const {
if (_fromPackage && !_packageModel.empty())
return ModelSource::Memory(_packageModel.data(),
_packageModel.size(), _packageName);
return ModelSource::File(filePath);
}
// Engines reset the input geometry to their own default after the
// base class has loaded, which would throw away a package's own
// (a classification model is 224, not 640). This puts it back.
void ApplyPackageInputSize() {
if (_packageInputWidth > 0) _modelConfig.inpWidth = _packageInputWidth;
if (_packageInputHeight > 0) _modelConfig.inpHeight = _packageInputHeight;
}
static bool IsModelPackagePath(const std::string& path);
// Opens one, decrypts the model into _packageModel, and fills
// _classes and _modelConfig's geometry from the manifest. Returns
// false with the reason logged.
bool LoadModelPackage(const std::string& path);
void WipeModelPackage();
std::recursive_mutex _mutex;
// Guard: true while Initialize/LoadModel is in progress on any thread.
// Inference callers can check this to fail-fast instead of blocking.
+105
View File
@@ -1,9 +1,11 @@
#pragma once
#include <set>
#include <cstring>
#include <cmath>
#include <json.hpp>
#include "ANSODEngine.h"
#include "ANSLicense.h" // ANS_DBG macro
#include "ANSModelOpen.h"
#include "OpenVINODeviceConfig.h"
#include "ANSYOLOOD.h"
#include "ANSTENSORRTOD.h"
@@ -61,6 +63,93 @@ namespace ANSCENTER
this->_logger.LogFatal("ANSODBase::CheckLicense. Error:", e.what(), __FILE__, __LINE__);
}
}
// ────────────────────────────────────────────────────────────────────────
// Model packages (.ansmodel)
//
// A package carries what the zip's loose files carried — the model, the
// class names, the task and the input geometry — sealed to a key only
// ANSCORE, ANSMINDCORE and the engines hold. Opening one fills in the
// same members the inflated folder used to fill in, with one difference
// that is the point of the whole format: the model stays in memory.
// ────────────────────────────────────────────────────────────────────────
bool ANSODBase::IsModelPackagePath(const std::string& path) {
return ANSCENTER::modelpkg::IsPackagePath(path);
}
void ANSODBase::WipeModelPackage() {
if (!_packageModel.empty()) {
std::memset(_packageModel.data(), 0, _packageModel.size());
_packageModel.clear();
_packageModel.shrink_to_fit();
}
_fromPackage = false;
_packageInputWidth = 0;
_packageInputHeight = 0;
}
bool ANSODBase::LoadModelPackage(const std::string& path) {
try {
WipeModelPackage();
ANSCENTER::modelpkg::OpenedModel opened;
std::string err;
if (!ANSCENTER::modelpkg::OpenModelPackage(path, opened, err)) {
this->_logger.LogError("ANSODBase::LoadModelPackage", err, __FILE__, __LINE__);
return false;
}
if (!opened.warning.empty()) {
// An expired release certificate warns and loads, by design:
// a model that worked yesterday keeps working on a box that
// has been offline for a year.
this->_logger.LogWarn("ANSODBase::LoadModelPackage", opened.warning, __FILE__, __LINE__);
}
_packageModel = std::move(opened.model);
if (_packageModel.empty()) {
this->_logger.LogError("ANSODBase::LoadModelPackage",
"The package carries no model", __FILE__, __LINE__);
return false;
}
_fromPackage = true;
_packageTask = opened.task;
_packageFamily = opened.family;
_packageMarker = opened.legacyMarker;
_packageName = opened.name;
// The classes come out in order, so there is nothing to parse and
// no classes.names to look for.
if (!opened.classes.empty()) _classes = opened.classes;
_packageInputWidth = opened.inputWidth;
_packageInputHeight = opened.inputHeight;
ApplyPackageInputSize();
// The manifest says in a word what the zip said by the presence of
// a marker file.
if (opened.task == "classification") _modelConfig.detectionType = ANSCENTER::DetectionType::CLASSIFICATION;
else if (opened.task == "segmentation") _modelConfig.detectionType = ANSCENTER::DetectionType::SEGMENTATION;
else if (opened.task == "obb") _modelConfig.detectionType = ANSCENTER::DetectionType::OBB;
else if (opened.task == "pose") _modelConfig.detectionType = ANSCENTER::DetectionType::KEYPOINT;
else _modelConfig.detectionType = ANSCENTER::DetectionType::DETECTION;
// There is no folder and no config file; every engine below tests
// for these, and must find them empty rather than stale.
_modelFolder.clear();
_modelConfigFile.clear();
_classFilePath.clear();
this->_logger.LogDebug("ANSODBase::LoadModelPackage",
"Opened " + path + " (" + opened.task + "/" + opened.family + ", "
+ std::to_string(_classes.size()) + " classes, "
+ std::to_string(_packageModel.size()) + " bytes)", __FILE__, __LINE__);
// Auto-enable tracker + stabilization by default (BYTETRACK)
SetTracker(TrackerType::BYTETRACK, true);
return true;
}
catch (std::exception& e) {
this->_logger.LogFatal("ANSODBase::LoadModelPackage", e.what(), __FILE__, __LINE__);
return false;
}
}
bool ANSODBase::LoadModel(const std::string& modelZipFilePath, const std::string& modelZipPassword)
{
try {
@@ -69,6 +158,7 @@ namespace ANSCENTER
_modelConfigFile = "";
_modelFolder.clear();
_modelConfigFile.clear();
WipeModelPackage(); // whatever was loaded before, package or not
// 0. Check if the modelZipFilePath exist?
if (!FileExist(modelZipFilePath)) {
if (modelZipFilePath.empty()) {// Support motion detector that does not need model file
@@ -82,6 +172,10 @@ namespace ANSCENTER
return false;
}
}
// 0b. A model package replaces the zip entirely: it is opened in
// memory, and no password, no folder and no loose files are involved.
if (IsModelPackagePath(modelZipFilePath))
return LoadModelPackage(modelZipFilePath);
// 1. Unzip model zip file to a special location with folder name as model file (and version)
std::string outputFolder;
std::vector<std::string> passwordArray;
@@ -126,6 +220,7 @@ namespace ANSCENTER
_modelFolder.clear();
_modelConfigFile.clear();
_modelConfig = modelConfig;
WipeModelPackage(); // whatever was loaded before, package or not
CheckLicense();
if (!_licenseValid) {
this->_logger.LogError("ANSODBase::Initialize", "Invalid License", __FILE__, __LINE__);
@@ -145,6 +240,12 @@ namespace ANSCENTER
}
}
// 0b. A model package replaces the zip entirely; see LoadModel.
if (IsModelPackagePath(modelZipFilePath)) {
if (!LoadModelPackage(modelZipFilePath)) return false;
_isInitialized = true;
return true;
}
// 1. Unzip model zip file to a special location with folder name as model file (and version)
std::string outputFolder;
std::vector<std::string> passwordArray;
@@ -952,6 +1053,10 @@ namespace ANSCENTER
ANSODBase::~ANSODBase() {
try {
// A decrypted model is what the package format exists to protect.
// This runs after the derived destructor has dropped the session,
// so nothing is left holding a view of these bytes.
WipeModelPackage();
// Per-camera trackers are released by CameraData::clear()
if (!_cameras.empty())
{
+24
View File
@@ -1394,6 +1394,19 @@ namespace ANSCENTER {
if (_modelConfig.kpsThreshold == 0)_modelConfig.kpsThreshold = 0.5; // If not define
_fp16 = (modelConfig.precisionType == PrecisionType::FP16);
// A package brought its own classes and geometry, and its model never
// touched the disk: the session reads it straight from memory.
if (_fromPackage) {
ApplyPackageInputSize();
labelMap.clear();
if (!_classes.empty()) labelMap = VectorToCommaSeparatedString(_classes);
instanceId_ = instanceCounter_.fetch_add(1);
result = Init(CurrentModelSource(_modelFilePath), true, 0);
_modelLoadValid = result;
_isInitialized = result;
return result;
}
if (FileExist(_modelConfigFile)) {
ModelType modelType;
std::vector<int> inputShape;
@@ -1454,6 +1467,17 @@ namespace ANSCENTER {
// if (_modelConfig.precisionType == PrecisionType::FP16)_fp16 = true;
_fp16 = true; // Load Model from Here
// A package brought its own classes and geometry, and its model never
// touched the disk: the session reads it straight from memory.
if (_fromPackage) {
ApplyPackageInputSize();
instanceId_ = instanceCounter_.fetch_add(1);
result = Init(CurrentModelSource(_modelFilePath), true, 0);
_modelLoadValid = result;
_isInitialized = result;
return result;
}
// 0. Check if the configuration file exist
if (FileExist(_modelConfigFile)) {
ModelType modelType;
+24
View File
@@ -1282,6 +1282,19 @@ namespace ANSCENTER {
if (_modelConfig.kpsThreshold == 0)_modelConfig.kpsThreshold = 0.5; // If not define
_fp16 = (modelConfig.precisionType == PrecisionType::FP16);
// A package brought its own classes and geometry, and its model never
// touched the disk: the session reads it straight from memory.
if (_fromPackage) {
ApplyPackageInputSize();
labelMap.clear();
if (!_classes.empty()) labelMap = VectorToCommaSeparatedString(_classes);
instanceId_ = instanceCounter_.fetch_add(1);
result = Init(CurrentModelSource(_modelFilePath), true, 0);
_modelLoadValid = result;
_isInitialized = result;
return result;
}
if (FileExist(_modelConfigFile)) {
ModelType modelType;
std::vector<int> inputShape;
@@ -1343,6 +1356,17 @@ namespace ANSCENTER {
// if (_modelConfig.precisionType == PrecisionType::FP16)_fp16 = true;
_fp16 = true; // Load Model from Here
// A package brought its own classes and geometry, and its model never
// touched the disk: the session reads it straight from memory.
if (_fromPackage) {
ApplyPackageInputSize();
instanceId_ = instanceCounter_.fetch_add(1);
result = Init(CurrentModelSource(_modelFilePath), true, 0);
_modelLoadValid = result;
_isInitialized = result;
return result;
}
// 0. Check if the configuration file exist
if (FileExist(_modelConfigFile)) {
ModelType modelType;
+24
View File
@@ -1172,6 +1172,19 @@ namespace ANSCENTER {
if (_modelConfig.kpsThreshold == 0)_modelConfig.kpsThreshold = 0.5; // If not define
_fp16 = (modelConfig.precisionType == PrecisionType::FP16);
// A package brought its own classes and geometry, and its model never
// touched the disk: the session reads it straight from memory.
if (_fromPackage) {
ApplyPackageInputSize();
labelMap.clear();
if (!_classes.empty()) labelMap = VectorToCommaSeparatedString(_classes);
instanceId_ = instanceCounter_.fetch_add(1);
result = Init(CurrentModelSource(_modelFilePath), true, 0);
_modelLoadValid = result;
_isInitialized = result;
return result;
}
if (FileExist(_modelConfigFile)) {
ModelType modelType;
std::vector<int> inputShape;
@@ -1233,6 +1246,17 @@ namespace ANSCENTER {
// if (_modelConfig.precisionType == PrecisionType::FP16)_fp16 = true;
_fp16 = true; // Load Model from Here
// A package brought its own classes and geometry, and its model never
// touched the disk: the session reads it straight from memory.
if (_fromPackage) {
ApplyPackageInputSize();
instanceId_ = instanceCounter_.fetch_add(1);
result = Init(CurrentModelSource(_modelFilePath), true, 0);
_modelLoadValid = result;
_isInitialized = result;
return result;
}
// 0. Check if the configuration file exist
if (FileExist(_modelConfigFile)) {
ModelType modelType;
+97 -24
View File
@@ -41,6 +41,37 @@ namespace ANSCENTER {
}
}
ONNXYOLO::ONNXYOLO(const void* modelData, size_t modelSize,
const std::string& logName, unsigned int _num_threads)
: BasicOrtHandler(modelData, modelSize, logName, _num_threads)
{
if (input_node_dims.size() >= 4) {
int h = static_cast<int>(input_node_dims[2]);
int w = static_cast<int>(input_node_dims[3]);
isDynamicInputShape = (h == -1 || w == -1);
inputImageShape = isDynamicInputShape ? cv::Size(640, 640) : cv::Size(w, h);
}
else {
inputImageShape = cv::Size(640, 640);
}
}
ONNXYOLO::ONNXYOLO(const void* modelData, size_t modelSize,
const std::string& logName, EngineType engineType,
unsigned int _num_threads)
: BasicOrtHandler(modelData, modelSize, logName, engineType, _num_threads)
{
if (input_node_dims.size() >= 4) {
int h = static_cast<int>(input_node_dims[2]);
int w = static_cast<int>(input_node_dims[3]);
isDynamicInputShape = (h == -1 || w == -1);
inputImageShape = isDynamicInputShape ? cv::Size(640, 640) : cv::Size(w, h);
}
else {
inputImageShape = cv::Size(640, 640);
}
}
// ------------------------------------------------------------------
// letterBox — Ultralytics-compatible LetterBox transform
// ------------------------------------------------------------------
@@ -1675,6 +1706,20 @@ namespace ANSCENTER {
bool ANSONNXYOLO::InitOrtEngine() {
try {
// A model that came out of an .ansmodel package has no file
// behind it; the handler reads the bytes ANSODBase decrypted.
// Those bytes outlive the handler, which is what lets the DML
// recovery path below rebuild the session from them.
if (_fromPackage) {
if (_packageModel.empty()) {
_logger.LogError("ANSONNXYOLO::InitOrtEngine",
"The package carries no model", __FILE__, __LINE__);
return false;
}
m_ortEngine = std::make_shared<class ONNXYOLO>(
_packageModel.data(), _packageModel.size(), _packageName);
return true;
}
if (!FileExist(_modelFilePath)) {
_logger.LogError("ANSONNXYOLO::InitOrtEngine",
"Model file does not exist: " + _modelFilePath, __FILE__, __LINE__);
@@ -1691,6 +1736,16 @@ namespace ANSCENTER {
bool ANSONNXYOLO::InitOrtEngine(ANSCENTER::EngineType engineType) {
try {
if (_fromPackage) {
if (_packageModel.empty()) {
_logger.LogError("ANSONNXYOLO::InitOrtEngine",
"The package carries no model", __FILE__, __LINE__);
return false;
}
m_ortEngine = std::make_shared<class ONNXYOLO>(
_packageModel.data(), _packageModel.size(), _packageName, engineType);
return true;
}
if (!FileExist(_modelFilePath)) {
_logger.LogError("ANSONNXYOLO::InitOrtEngine",
"Model file does not exist: " + _modelFilePath, __FILE__, __LINE__);
@@ -1846,21 +1901,30 @@ namespace ANSCENTER {
NUM_KPS = _modelConfig.numKPS;
KPS_THRESHOLD = _modelConfig.kpsThreshold;
_modelFilePath = CreateFilePath(_modelFolder, "train_last.onnx");
if (FileExist(_modelConfigFile)) {
ModelType modelType;
std::vector<int> inputShape;
_classes = ANSUtilityHelper::GetConfigFileContent(_modelConfigFile, modelType, inputShape);
if (inputShape.size() == 2) {
if (inputShape[0] > 0) _modelConfig.inpHeight = inputShape[0];
if (inputShape[1] > 0) _modelConfig.inpWidth = inputShape[1];
// A package has already supplied the classes and the geometry;
// there is no folder to look in.
if (!_fromPackage) {
_modelFilePath = CreateFilePath(_modelFolder, "train_last.onnx");
if (FileExist(_modelConfigFile)) {
ModelType modelType;
std::vector<int> inputShape;
_classes = ANSUtilityHelper::GetConfigFileContent(_modelConfigFile, modelType, inputShape);
if (inputShape.size() == 2) {
if (inputShape[0] > 0) _modelConfig.inpHeight = inputShape[0];
if (inputShape[1] > 0) _modelConfig.inpWidth = inputShape[1];
}
}
else {
_classFilePath = CreateFilePath(_modelFolder, "classes.names");
std::ifstream isValid(_classFilePath);
if (!isValid) LoadClassesFromString();
else LoadClassesFromFile();
}
}
else {
_classFilePath = CreateFilePath(_modelFolder, "classes.names");
std::ifstream isValid(_classFilePath);
if (!isValid) LoadClassesFromString();
else LoadClassesFromFile();
// The defaults above have just overwritten what the package
// said its input size was; put it back.
ApplyPackageInputSize();
}
labelMap.clear();
@@ -1920,21 +1984,30 @@ namespace ANSCENTER {
NUM_KPS = _modelConfig.numKPS;
KPS_THRESHOLD = _modelConfig.kpsThreshold;
_modelFilePath = CreateFilePath(_modelFolder, "train_last.onnx");
if (FileExist(_modelConfigFile)) {
ModelType modelType;
std::vector<int> inputShape;
_classes = ANSUtilityHelper::GetConfigFileContent(_modelConfigFile, modelType, inputShape);
if (inputShape.size() == 2) {
if (inputShape[0] > 0) _modelConfig.inpHeight = inputShape[0];
if (inputShape[1] > 0) _modelConfig.inpWidth = inputShape[1];
// A package has already supplied the classes and the geometry;
// there is no folder to look in.
if (!_fromPackage) {
_modelFilePath = CreateFilePath(_modelFolder, "train_last.onnx");
if (FileExist(_modelConfigFile)) {
ModelType modelType;
std::vector<int> inputShape;
_classes = ANSUtilityHelper::GetConfigFileContent(_modelConfigFile, modelType, inputShape);
if (inputShape.size() == 2) {
if (inputShape[0] > 0) _modelConfig.inpHeight = inputShape[0];
if (inputShape[1] > 0) _modelConfig.inpWidth = inputShape[1];
}
}
else {
_classFilePath = CreateFilePath(_modelFolder, "classes.names");
std::ifstream isValid(_classFilePath);
if (!isValid) LoadClassesFromString();
else LoadClassesFromFile();
}
}
else {
_classFilePath = CreateFilePath(_modelFolder, "classes.names");
std::ifstream isValid(_classFilePath);
if (!isValid) LoadClassesFromString();
else LoadClassesFromFile();
// The defaults above have just overwritten what the package
// said its input size was; put it back.
ApplyPackageInputSize();
}
if (this->_loadEngineOnCreation) {
+10
View File
@@ -33,6 +33,16 @@ namespace ANSCENTER {
explicit ONNXYOLO(const std::string& _onnx_path,
EngineType engineType,
unsigned int _num_threads = 1);
// The decrypted contents of an .ansmodel package: the same model,
// with no file behind it. `logName` is what logs call it.
explicit ONNXYOLO(const void* modelData, size_t modelSize,
const std::string& logName,
unsigned int _num_threads = 1);
explicit ONNXYOLO(const void* modelData, size_t modelSize,
const std::string& logName,
EngineType engineType,
unsigned int _num_threads = 1);
~ONNXYOLO() override = default;
/// Run inference on a single image.
+22
View File
@@ -10,6 +10,12 @@ namespace ANSCENTER {
if (!ANSODBase::OptimizeModel(fp16, optimizedModelFolder)) {
return false;
}
if (_fromPackage) {
// A package is already the optimised artefact, and there is no
// folder to point at because nothing was written out.
optimizedModelFolder.clear();
return true;
}
if (FileExist(_modelFilePath)) {
optimizedModelFolder = GetParentFolder(_modelFilePath);
this->_logger.LogDebug("YOLO12OD::OptimizeModel", "This model is optimized. No need other optimization.", __FILE__, __LINE__);
@@ -34,6 +40,13 @@ namespace ANSCENTER {
_modelConfig.modelMNSThreshold = 0.5;
if (_modelConfig.modelConfThreshold < 0.2)
_modelConfig.modelConfThreshold = 0.5;
// A package brought its own classes and geometry, and its model never
// touched the disk: the session reads it straight from memory.
if (_fromPackage) {
ApplyPackageInputSize();
_isInitialized = loadModel(CurrentModelSource(_modelFilePath), true);
return _isInitialized;
}
// 0. Check if the configuration file exist
if (FileExist(_modelConfigFile)) {
ModelType modelType;
@@ -168,6 +181,15 @@ namespace ANSCENTER {
_modelConfig.modelMNSThreshold = 0.5;
if (_modelConfig.modelConfThreshold < 0.2)
_modelConfig.modelConfThreshold = 0.5;
// A package brought its own classes and geometry, and its model never
// touched the disk: the session reads it straight from memory.
if (_fromPackage) {
ApplyPackageInputSize();
labelMap.clear();
if (!_classes.empty()) labelMap = VectorToCommaSeparatedString(_classes);
_isInitialized = loadModel(CurrentModelSource(_modelFilePath), true);
return _isInitialized;
}
// 0. Check if the configuration file exist
if (FileExist(_modelConfigFile)) {
ModelType modelType;
+3
View File
@@ -113,6 +113,9 @@ target_include_directories(ANSODEngine PRIVATE
target_link_libraries(ANSODEngine
PUBLIC ANSLibsLoader
# Reading .ansmodel packages. Header-only apart from the vendored
# curves, and free of CUDA/OpenCV, so it costs this target nothing.
PRIVATE ANSModelPackage
PUBLIC ANSLicensingSystem # PUBLIC: FileExist/SPDLogger symbols must be re-exported
PUBLIC opencv
PRIVATE ANSMOT
@@ -17,6 +17,7 @@
#include <string>
#include <vector>
#include "ANSModelOpen.h"
#include "ANSModelPackage.h"
extern "C" {
@@ -245,6 +246,61 @@ static void testRoundTripAndRefusals(const fs::path& dir) {
}
}
// ── the glue the engines load through ─────────────────────────────────────
//
// ANSODBase asks these three questions and nothing else: is this path a
// package, can this build open one, and what came out of it. The engines
// themselves need CUDA to compile, so this is where that contract is checked.
static void testOpenForAnEngine(const fs::path& dataDir) {
const std::string path = (dataDir / "interop.ansmodel").string();
if (!fs::exists(path)) {
std::printf("SKIP engine glue: %s is not here\n", path.c_str());
return;
}
CHECK(modelpkg::IsPackagePath(path), "a .ansmodel path is recognised as one");
CHECK(!modelpkg::IsPackagePath("/models/ANS_Test_v1.0.zip"),
"and a zip is not");
CHECK(!modelpkg::IsPackagePath(".ansmodel"),
"nor is a bare extension with no name in front of it");
// A build with no keys installed refuses every package with a sentence
// that says why, rather than failing somewhere deeper.
{
modelpkg::SetProcessModelKeys(modelpkg::Keyring{});
CHECK(!modelpkg::HaveProcessModelKeys(),
"a build with no keys knows it cannot open packages");
modelpkg::OpenedModel m;
std::string err;
CHECK(!modelpkg::OpenModelPackage(path, m, err) &&
err.find("no model-package key") != std::string::npos,
"and says so instead of failing at the decryption");
}
const Identity id = fixtureIdentity();
modelpkg::SetProcessModelKeys(fixtureKeyring(id));
CHECK(modelpkg::HaveProcessModelKeys(), "with keys set, it can");
modelpkg::OpenedModel m;
std::string err;
CHECK(modelpkg::OpenModelPackage(path, m, err), ("open: " + err).c_str());
CHECK(m.model.size() == 12000, "the model comes out whole");
CHECK(m.task == "detection" && m.family == "yolo",
"with the task and family the engine branches on");
CHECK(m.classes.size() == 2 && m.classes[0] == "Cavity",
"and the classes in order, so there is no classes.names to read");
CHECK(!m.name.empty(), "and a name for the log line");
// What the engine does the moment its session has the bytes.
m.wipe();
CHECK(m.model.empty(), "wiping leaves the plaintext nowhere");
// Describing one is the same answer without the decryption.
modelpkg::OpenedModel d;
CHECK(modelpkg::DescribeModelPackage(path, d, err) && d.model.empty() &&
d.classes.size() == 2 && d.task == "detection",
"describing a package answers without decrypting the model");
}
int main(int argc, char** argv) {
const fs::path dataDir = argc > 1 ? fs::path(argv[1]) : fs::path("data");
const auto stamp = std::chrono::steady_clock::now().time_since_epoch().count();
@@ -256,6 +312,7 @@ int main(int argc, char** argv) {
testVectors();
testInterop(dataDir);
testRoundTripAndRefusals(tmp);
testOpenForAnEngine(dataDir);
fs::remove_all(tmp, ec);
std::printf("%d checks, %d failures\n", gChecks, gFailures);