feat(ODEngine): load a model from memory, not only from a path
Every engine built its session from a file path, which is why opening an encrypted model package means inflating it to a directory first: the unprotected model lands on disk, is optimised, and is zipped back into a cache. The new package format cannot avoid that from the outside — the engines have to be able to take bytes. ANSModelSource.h introduces a ModelSource (a path, or bytes with a name for logs) and one MakeSession that builds from either. The six ONNX engines — classification, OBB, pose, segmentation, YOLO12 and YOLO — each gain a load overload taking it, with the existing string form delegating so no caller changes. The Windows wide-path conversion and the from-memory call now live in one place instead of the eleven copies they were spread across; YOLOOD, which had only the Windows branch, becomes portable as a side effect. The factories are named rather than implicit, so a call with a plain string literal still resolves to the path overload without ambiguity. Not compiled here: this layer builds only where CUDA is present, so verification on this machine was the new header compiled and run, plus a structural check that every engine declares and defines both overloads, calls MakeSession exactly once, and has no raw session construction left. It needs a build on a CUDA box before it ships. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,76 @@
|
||||
#ifndef ANSMODELSOURCE_H
|
||||
#define ANSMODELSOURCE_H
|
||||
#pragma once
|
||||
// ANSModelSource — where a model's bytes come from: a file, or memory.
|
||||
//
|
||||
// Every engine here built its session from a file path, which is why loading
|
||||
// an encrypted model package means inflating it to a directory first: the
|
||||
// unprotected model lands on disk, is optimised, and is zipped back into a
|
||||
// cache. Handing the runtime a buffer instead removes both the round trip and
|
||||
// the plaintext on disk, and it is the one change the new .ansmodel package
|
||||
// format cannot work without.
|
||||
//
|
||||
// A model in memory still needs two things a path gave for free. `name` is
|
||||
// what logs call it, since there is no file name to fall back on. And
|
||||
// ONNX Runtime copies what it needs out of the buffer while the session is
|
||||
// being built, so the caller may wipe its plaintext the moment MakeSession
|
||||
// returns — which is exactly what a package reader should do.
|
||||
|
||||
#include <string>
|
||||
|
||||
#include <onnxruntime_cxx_api.h>
|
||||
|
||||
namespace ANSCENTER {
|
||||
|
||||
struct ModelSource {
|
||||
std::string path; // file on disk; empty for a model already in memory
|
||||
std::string name; // what to call it in logs
|
||||
const void* data = nullptr;
|
||||
size_t size = 0;
|
||||
|
||||
bool inMemory() const { return data != nullptr && size > 0; }
|
||||
|
||||
// Named factories, never implicit conversions: an engine that offers
|
||||
// both Init(std::string) and Init(ModelSource) must not become
|
||||
// ambiguous for a caller passing a plain string literal.
|
||||
static ModelSource File(std::string p) {
|
||||
ModelSource s;
|
||||
s.name = p;
|
||||
s.path = std::move(p);
|
||||
return s;
|
||||
}
|
||||
static ModelSource Memory(const void* d, size_t n, std::string name) {
|
||||
ModelSource s;
|
||||
s.data = d;
|
||||
s.size = n;
|
||||
s.name = std::move(name);
|
||||
return s;
|
||||
}
|
||||
};
|
||||
|
||||
// What to call this model in a log line.
|
||||
inline std::string ModelSourceName(const ModelSource& src) {
|
||||
if (!src.name.empty()) return src.name;
|
||||
if (!src.path.empty()) return src.path;
|
||||
return "<memory>";
|
||||
}
|
||||
|
||||
// Build a session from wherever the bytes are. One place, so the Windows
|
||||
// wide-path conversion and the from-memory call are not repeated in every
|
||||
// engine — they were already duplicated eleven times.
|
||||
inline Ort::Session MakeSession(Ort::Env& env, const ModelSource& src,
|
||||
const Ort::SessionOptions& options) {
|
||||
if (src.inMemory())
|
||||
return Ort::Session(env, src.data, src.size, options);
|
||||
#ifdef _WIN32
|
||||
// ORT's Windows API takes a wide (ORTCHAR_T = wchar_t) path.
|
||||
const std::wstring wide(src.path.begin(), src.path.end());
|
||||
return Ort::Session(env, wide.c_str(), options);
|
||||
#else
|
||||
return Ort::Session(env, src.path.c_str(), options);
|
||||
#endif
|
||||
}
|
||||
|
||||
} // namespace ANSCENTER
|
||||
|
||||
#endif // ANSMODELSOURCE_H
|
||||
Reference in New Issue
Block a user