feat(ODEngine): open a model package and run it without unpacking it
An engine's model has always been a path to a password-protected zip: ANSODBase inflates it into a folder, and every engine then picks out train_last.onnx, classes.names and model_config.json. A .ansmodel package replaces all four with one sealed file, and this is the part that turns it back into what the engines already understand. ANSODBase::LoadModel and ::Initialize now recognise a .ansmodel path and take a second route: the package is opened with the process keyring, the model is decrypted into memory, and the classes, task and input geometry come out of the manifest instead of loose files. Nothing is extracted, so the unprotected model never touches the disk — which is the reason the format exists, and it is one function's job to keep. The engines follow it down. BasicOrtHandler gains a constructor that takes the bytes rather than a path; it keeps a view of them, not a copy, because the weights are in RAM either way once a session exists and a copy would only double what every loaded model costs. The owner of the plaintext is ANSODBase, which holds it for as long as the engine object lives — that is what lets the DirectML recovery path rebuild a session after a driver fault — and wipes it on teardown. Wired through ANSONNXYOLO (and so the classification facade), ANSONNXOBB, ANSONNXPOSE, ANSONNXSEG and YOLO12OD. YOLOOD is left alone: it loads darknet weights and OpenCV DNN graphs, neither of which the writer produces. A zip still loads exactly as before, on every engine. Keys come from SetProcessModelKeys, as in ANSMINDCORE, with the same development fallback from the environment. With none set, a package is refused with a sentence saying so and every zip still works, so a build made before the key ceremony runs everything except packages. The engine layer needs CUDA to compile, so the contract it depends on is checked where it can be: ANSModelPackage-UnitTest now opens the fixture ANSMINDCORE wrote through the same glue ANSODBase uses, and covers the refusal a keyless build gives. 32 checks, no failures. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -221,8 +221,45 @@ namespace ANSCENTER {
|
||||
initialize_handler();
|
||||
}
|
||||
|
||||
// ────────────────────────────────────────────────────────────────────
|
||||
// A model that is already in memory — the decrypted contents of an
|
||||
// .ansmodel package. There is no path, so onnx_path stays null and
|
||||
// every file-shaped step in initialize_handler is skipped: no CWD
|
||||
// switch, no external-data lookup, no file for ORT to open.
|
||||
// ────────────────────────────────────────────────────────────────────
|
||||
BasicOrtHandler::BasicOrtHandler(const void* modelData, size_t modelSize,
|
||||
const std::string& logName,
|
||||
unsigned int _num_threads)
|
||||
: num_threads(_num_threads),
|
||||
m_engineType(static_cast<EngineType>(-1))
|
||||
{
|
||||
log_id_storage = logName;
|
||||
log_id = log_id_storage.c_str();
|
||||
model_data = modelData;
|
||||
model_size = modelSize;
|
||||
initialize_handler();
|
||||
}
|
||||
|
||||
BasicOrtHandler::BasicOrtHandler(const void* modelData, size_t modelSize,
|
||||
const std::string& logName,
|
||||
EngineType engineType,
|
||||
unsigned int _num_threads)
|
||||
: num_threads(_num_threads),
|
||||
m_engineType(engineType)
|
||||
{
|
||||
log_id_storage = logName;
|
||||
log_id = log_id_storage.c_str();
|
||||
model_data = modelData;
|
||||
model_size = modelSize;
|
||||
initialize_handler();
|
||||
}
|
||||
|
||||
BasicOrtHandler::~BasicOrtHandler()
|
||||
{
|
||||
// The model bytes, when there are any, belong to whoever handed
|
||||
// them over; that owner wipes them. Only the view goes here.
|
||||
model_data = nullptr;
|
||||
model_size = 0;
|
||||
if (ort_session) {
|
||||
delete ort_session;
|
||||
ort_session = nullptr;
|
||||
@@ -727,20 +764,26 @@ namespace ANSCENTER {
|
||||
// AddExternalInitializersFromFilesInMemory() so ORT never
|
||||
// memory-maps the file itself.
|
||||
// ----------------------------------------------------------------
|
||||
std::filesystem::path modelFsPath(onnx_path); // wchar_t*
|
||||
std::filesystem::path modelDir = modelFsPath.parent_path();
|
||||
// A model in memory has no directory, no sibling .onnx_data and
|
||||
// nothing for ORT to open, so all of this is skipped for one.
|
||||
std::filesystem::path modelFsPath;
|
||||
std::filesystem::path modelDir;
|
||||
std::filesystem::path prevCwd = std::filesystem::current_path();
|
||||
|
||||
if (!modelDir.empty() && std::filesystem::is_directory(modelDir)) {
|
||||
std::filesystem::current_path(modelDir);
|
||||
std::cout << "[ORT] CWD -> " << modelDir.string() << std::endl;
|
||||
if (!fromMemory()) {
|
||||
modelFsPath = std::filesystem::path(onnx_path); // wchar_t*
|
||||
modelDir = modelFsPath.parent_path();
|
||||
if (!modelDir.empty() && std::filesystem::is_directory(modelDir)) {
|
||||
std::filesystem::current_path(modelDir);
|
||||
std::cout << "[ORT] CWD -> " << modelDir.string() << std::endl;
|
||||
}
|
||||
}
|
||||
|
||||
// --- Pre-load external data files if they exist -----------------
|
||||
// Keep the buffer alive across session creation (must outlive the
|
||||
// Ort::Session constructor call).
|
||||
std::vector<char> extDataBuffer;
|
||||
{
|
||||
if (!fromMemory()) {
|
||||
// Build the expected external-data filename:
|
||||
// <model_stem>.onnx_data (e.g. anssam3.onnx_data)
|
||||
std::filesystem::path extDataPath =
|
||||
@@ -806,7 +849,7 @@ namespace ANSCENTER {
|
||||
// This avoids ORT opening/mapping ANY files during CreateSession.
|
||||
std::vector<char> modelBuffer;
|
||||
bool useModelBuffer = false;
|
||||
if (!extDataBuffer.empty()) {
|
||||
if (!extDataBuffer.empty() && !fromMemory()) {
|
||||
// External data was pre-loaded, so also load the .onnx itself
|
||||
try {
|
||||
auto modelFileSize = std::filesystem::file_size(modelFsPath);
|
||||
@@ -829,7 +872,10 @@ namespace ANSCENTER {
|
||||
// --- Attempt session creation (with CUDA → CPU fallback) --------
|
||||
auto createSession = [&](Ort::SessionOptions& opts, const char* label) {
|
||||
std::cout << "[ORT] Creating session (" << label << ")..." << std::endl;
|
||||
if (useModelBuffer) {
|
||||
if (fromMemory()) {
|
||||
ort_session = new Ort::Session(*ort_env,
|
||||
model_data, model_size, opts);
|
||||
} else if (useModelBuffer) {
|
||||
ort_session = new Ort::Session(*ort_env,
|
||||
modelBuffer.data(), modelBuffer.size(), opts);
|
||||
} else {
|
||||
@@ -838,7 +884,11 @@ namespace ANSCENTER {
|
||||
std::cout << "[ORT] Session created OK (" << label << ")." << std::endl;
|
||||
};
|
||||
|
||||
ANS_DBG("OrtHandler", "Creating session for model: %ls", onnx_path);
|
||||
if (fromMemory())
|
||||
ANS_DBG("OrtHandler", "Creating session for model in memory: %s (%zu bytes)",
|
||||
log_id ? log_id : "<memory>", model_size);
|
||||
else
|
||||
ANS_DBG("OrtHandler", "Creating session for model: %ls", onnx_path);
|
||||
try {
|
||||
createSession(session_options, "primary EP");
|
||||
ANS_DBG("OrtHandler", "Session created OK with primary EP");
|
||||
|
||||
Reference in New Issue
Block a user