Author SHA1 Message Date
nghianguyenandClaude Opus 5.5 c1c26c400b fix(ANSModelPackage): seal model packages with CSPRNG keys and nonces, not a seed
pack() derived the content key, the ephemeral X25519 private key and every
nonce base from an 8-bit nonceSeed. Anyone holding a package could try the
256 seeds and recover its content key without the base private key.

The content key, ephemeral key, wrap nonce and nonce bases now come from
the OS CSPRNG (BCryptGenRandom / getrandom / arc4random_buf). Nonce bases
are redrawn until distinct within a package. The seeded path survives only
as insecureDeterministicForTests + testSeed, honoured only when built with
ANSMODEL_DETERMINISTIC_PACK_FOR_TESTS; otherwise pack() refuses. The
on-disk format is unchanged. Same change in ANSTSCORE, ANSMINDCORE and
ANSCORE, which keep pack() in step.

Regression test: two packs of identical inputs have different content
keys, ephemeral keys, wrap nonces and ciphertext, neither key is one a
seed could make, and both still open with the base key.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-26 18:15:13 +10:00
nghianguyenandClaude Opus 5 cb964907f0 feat(ANSLS): know ANSMIND as 1009, and stop filing unknown products under it
The gateway's product table already runs to 1009 ANSMIND and 1010 ANSGW.
The licensing library stopped at 1008, so a real ANSMIND licence could not
be named, deactivated or listed here, while every product name it did not
recognise quietly became product 1009. With 1009 now a real product, that
fallback would mint an ANSMIND trial for a typo.

ANSPRODUCTS gains ANSMIND = 1009 and ANSGW = 1010, and both are added
wherever the product table is spelled out: ActivateTrialLicense,
DeactivateLicense, ActivateLicense, ActivateLicenseWithCustomHWID and
ListLicenses, which now reports an ANSMIND and an ANSGW row like the
gateway does.

Unknown names no longer map to any id. ActivateTrialLicense returns 0, its
existing failure value, instead of generating a key. ListLicenses skips
the row. ActivateLicense and ActivateLicenseWithCustomHWID keep passing
the name through as the registration name: they never assigned an id, the
key's own validation data rejects a name it was not issued for, and the
gateway resolves an unknown product the same way.

The ANSVIS master-licence rule in LicenseVerification is untouched.

Separately, the fallback loops in ANSUtilities, ANSAWSS3 and ANSLLM asked
for product 1008 under the registration name "ANSSRT". The product is
ANSTS, so a genuine ANSTS licence could never satisfy them. Only that
string changes; the loops still stop at 1008.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 10:38:31 +10:00
nghianguyenandClaude Opus 5 cd99fcf6d9 feat(ANSODEngine): run a sealed model on TensorRT without unsealing it
A .ansmodel reaching the GPU path went to ModelOptimizer, which knows one
way to open a model — unzip it — and failed on the first byte. Even past
that, the TensorRT engine only ever parsed an ONNX file, so the one thing
the format exists to prevent, a decrypted model on disk, was the only way
it could have run.

The builder now takes its bytes from memory when it has them. The path it
is given still names the plan that gets cached beside it, because a plan
is a built artefact and carries no model: the second run loads it and
never needs the ONNX at all. For a package the plan lands in the folder
the zip flow used, so a model converted from a zip reuses the plan that
zip already built.

OptimizeModelStr recognises a package and skips the unzip-and-optimise
leg entirely: the handle below it opens the package in memory and builds
the plan from the decrypted bytes, which is the same work by a shorter
road.

Still on the file path: the other TensorRT handlers (segmentation, pose,
classification). Only the YOLO detector reads a package today.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 09:38:45 +10:00
nghianguyenandClaude Opus 5 1e369fdfef test(ANSODEngine): let the harness run a model this machine actually has
Every test in that file names a model and a video from whichever
workstation it was written on, so anywhere else the harness prints
"Model zip file is not exist" and exits before it reaches the GPU. That
is what happened on the machine that is meant to be testing CUDA and
TensorRT.

Given a model and a video on the command line it runs those instead,
with an optional engine type and detection type. With no arguments it
behaves exactly as before, so nobody's workflow changes.

The model may be a .zip or a sealed .ansmodel, because the engine now
opens either — and that is the only way to watch a sealed model run on
a GPU, which is the last thing in this format that has never been seen
working end to end.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 08:28:50 +10:00
nghianguyenandClaude Opus 5 b69a0fbcba fix(build): find nlohmann/json where the machine actually keeps it
The package library's Windows branch pointed at vcpkg alone. On a real
Windows build machine the header lives in the ANSLibs tree, so anything
linking this library failed with "Cannot open include file:
'nlohmann/json.hpp'" while the engines compiled fine — they add ANSLibs
to their own include path and never noticed.

Now it is searched for by name, with the ANSLibs tree, vcpkg and the
usual system paths as hints, and the configure says where it found it.
One code path for every platform instead of an assumption per platform.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 08:00:51 +10:00
nghianguyenandClaude Opus 5 e76d3b81e2 fix(cuda): spell the fast-math flag the way nvcc spells it
Every .cu file failed on CUDA 13 with "nvcc fatal: Unknown option
'--use-fast-math'". The flag is --use_fast_math, with underscores; older
toolkits accepted the hyphenated form as well and this one does not, so
the engine layer could not compile a single kernel on a current CUDA.

Found on the Windows machine with CUDA 13.0, where it stopped the build
at the first two kernels.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 07:58:49 +10:00
nghianguyenandClaude Opus 5 eb84d48e5c fix(build): do not demand a policy older CMake has never heard of
Line two set CMP0167 unconditionally. That policy arrived in CMake 3.30
and this project asks for 3.28, so every CMake between the two failed
the configure at line two with "Policy CMP0167 is not known to this
version" — before it had done anything at all. Found on the Windows
machine that builds the CUDA engines, where it stopped the build dead.

Guarded, so a newer CMake still gets the quiet FindBoost the policy was
set for and an older one configures instead of refusing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 07:46:43 +10:00
nghianguyenandClaude Opus 5 3798e3513b fix(YOLO12OD): decode an end-to-end head instead of reading it as a grid
This engine took the feature count from outputShape[1] unconditionally.
For a yolo26 export with end2end on — [1, 300, 6], rows of
(x1 y1 x2 y2 score class) with the suppression already inside the graph
— that means 296 classes, coordinates read as class scores, and boxes
covering the frame. No error, just nonsense, which is exactly what the
Labelling Tool produced from the same model before its decoder was
fixed.

Its sibling ANSONNXYOLO has recognised that head from the tensor shape
for a while; this one had not, and the backend marker cannot tell them
apart because it names the runtime rather than the head.

The branch fires only where the tensor cannot be a grid: the 6 must be
the last axis with a detection count before it, hundreds rather than the
thousands of anchors a two-class [1, 6, 8400] export carries.

The decoder reuses this engine's own scaleCoords rather than repeating
the letterbox arithmetic, so both heads undo the padding the same way
and agree about where a box is, and the rounding into the integral box
type is spelled out instead of left to a narrowing conversion.

NOT COMPILED. The engine layer needs CUDA and the ANSLibs tree, neither
of which exists on this machine, so this is verified by reading: the
shape rule matches the one the other three products now share, every
symbol it uses is declared in this translation unit, and the arithmetic
is delegated to the function the grid path already trusts.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 06:55:45 +10:00
nghianguyenandClaude Opus 5 ea877a61bc fix(keys): decide at configure time whether this build has the key
Whether the model-package key is present was an __has_include test
inside the header, and that test leaves no dependency for the build
system. A key emitted after a build had already compiled the file would
not take effect until someone cleaned, and the resulting engine looks
exactly like one built without a key — which on a release machine means
shipping an engine that silently cannot open a package.

CMake now decides it and says which case it is, so emitting the key and
re-configuring is what installs it, visibly.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 05:56:58 +10:00
nghianguyenandClaude Opus 5 3c9e0b79f5 fix(ANSModelPackage): read the head off the graph, not off the package's word
A package whose family says "yolo" may hold an end-to-end export. Boxes
took the family from the backend marker, and the marker names the
runtime, not the head — so a fast yolo26 export arrived declared as
plain yolo while its graph emits [1, N, 6] with the NMS inside it.
Decoding that as a [1, 4+nc, N] grid reads coordinates as confidences:
detections everywhere, and nothing that looks like a failure.

OpenModelPackage now reads the first output's dims from the model it has
just decrypted, and corrects the family with the reason in the warning
rather than swapping it silently. It costs nothing there: the bytes are
in hand. The legacy marker is left alone, so anything converting back to
a zip still writes what ANSCORE expects.

A two-class v8 export is [1, 6, 8400] and must not be mistaken for one,
which is why the 6 has to be the last axis with a detection count before
it. Both cases are tested, and the package this Mac exported before the
writer was fixed now reads as yolo10 here too. 46 checks, no failures.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 00:19:15 +10:00
nghianguyenandClaude Opus 5 b1196ee8ba chore: ignore the in-tree build/ directory
Only cmake-build-*/ and out/ were ignored, so configuring into build/ —
the directory the ANSCORE_DEBUGVIEW_STRIP usage note in CMakeLists.txt
shows — left an untracked build/ in git status. build-mac/ is untracked
as well and stays that way; this pattern matches build/ alone.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-12 23:39:56 +10:00
nghianguyenandClaude Opus 5 6abe6d6694 fix(build): configure without CUDA instead of failing at generate
tests/CMakeLists.txt added every test directory unconditionally, so a
machine with no CUDA toolkit — every macOS one — died at generate time on
"links to CUDA::cudart but the target was not found", taking the
cross-platform targets down with it.

Gate each test the way its subject is already gated: the five that need
the toolkit behind ANSCORE_HAS_CUDA, the ANSIO one behind WIN32, and the
ANSONVIF one behind both. ANSONVIF itself uses no CUDA; its test names
CUDA::cudart and CUDA::nvjpeg only because it drives capture through
ANSCV and was copied from the ANSCV test.

Two non-test targets had the same problem. ANSCV genuinely needs the
toolkit — ANSGpuFrameOps.h includes <cuda_runtime.h> unconditionally and
calls cudaMemcpy2D/cudaMallocPitch — so it moves into the existing CUDA
block beside the engine modules. ANSLibsLoader binds cudart at runtime
and ships its own stubs, but still compiles against the runtime header
for the types and links CUDA::cudart_static, so it is gated too.

Also call enable_testing(), which the project never did: without it the
add_test() under tests/ reached nothing and ctest found no tests. Plain
enable_testing(), not include(CTest), to avoid its extra targets.

A CUDA machine is unchanged: every gate holds there, ANSCV keeps its
place in the subdirectory order, and the only new output is
CTestTestfile.cmake.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-12 23:38:57 +10:00
25 changed files with 904 additions and 166 deletions
+1
View File
@@ -6,6 +6,7 @@ Release/
out/
# CLion / CMake
build/
.idea/
cmake-build-debug/
cmake-build-release/
+26 -5
View File
@@ -1,5 +1,11 @@
cmake_minimum_required(VERSION 3.28)
cmake_policy(SET CMP0167 NEW) # Suppress FindBoost removal warning
# CMP0167 (FindBoost removed from CMake) only exists from 3.30, and this
# project asks for 3.28 — so setting it unconditionally makes every older
# CMake fail at line two with "policy is not known to this version". Guarded,
# a newer CMake still gets the quiet FindBoost it was added for.
if(POLICY CMP0167)
cmake_policy(SET CMP0167 NEW)
endif()
# ── CUDA is optional (not available on macOS) ───────────────────
include(CheckLanguage)
@@ -17,10 +23,14 @@ if(CMAKE_CUDA_COMPILER)
set(CMAKE_CUDA_ARCHITECTURES
75-real 80-real 86-real 87-real 88-real 89-real 90-real 100-real 120-real
75-virtual)
# --use_fast_math, with underscores: that is nvcc's own spelling. Older
# toolkits also accepted the hyphenated form; CUDA 13 does not, and every
# .cu file fails with "Unknown option '--use-fast-math'" before it compiles
# a line.
if(MSVC)
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} --use-fast-math -Xcompiler=/utf-8")
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} --use_fast_math -Xcompiler=/utf-8")
else()
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} --use-fast-math")
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} --use_fast_math")
endif()
else()
project(ANSCORE LANGUAGES C CXX)
@@ -154,7 +164,11 @@ include(cmake/Dependencies.cmake)
# Core — licensing & infrastructure
add_subdirectory(core/anslicensing)
add_subdirectory(core/ANSLicensingSystem)
add_subdirectory(core/ANSLibsLoader)
# ANSLibsLoader dynamically binds cudart/cuBLAS/nvJPEG at runtime, but it still
# compiles against <cuda_runtime.h> for their types and links CUDA::cudart_static.
if(ANSCORE_HAS_CUDA)
add_subdirectory(core/ANSLibsLoader)
endif()
add_subdirectory(core/ANSModelPackage)
# Modules — feature DLLs (cross-platform)
@@ -178,11 +192,18 @@ if(ANSCORE_HAS_CUDA)
add_subdirectory(modules/ANSOCR)
add_subdirectory(modules/ANSFR)
add_subdirectory(modules/ANSLPR)
# ANSCV hosts NVDEC decode, the NV12 GPU frame pool and the nvJPEG encoder:
# ANSGpuFrameOps.h includes <cuda_runtime.h> unconditionally and calls
# cudaMemcpy2D/cudaMallocPitch directly, so it needs the toolkit like the
# engine modules do.
add_subdirectory(modules/ANSCV)
endif()
add_subdirectory(modules/ANSCV)
# Facade
add_subdirectory(ANSLIB)
# Tests
# enable_testing() (not include(CTest), which would add Nightly/Experimental
# targets) so the add_test() calls under tests/ reach ctest.
enable_testing()
add_subdirectory(tests)
+47 -4
View File
@@ -1777,10 +1777,19 @@ namespace ANSCENTER
registrationName = "ANSTS";
productId = 1008;
}
else {
registrationName = productName;
else if (productName == "ANSMIND") {//1009
registrationName = "ANSMIND";
productId = 1009;
}
else if (productName == "ANSGW") {//1010
registrationName = "ANSGW";
productId = 1010;
}
else {
// Unknown product: refuse rather than mint a trial under some real product's id.
std::cout << "ActivateTrialLicense: unknown product name '" << productName << "'" << std::endl;
return 0;
}
std::string trialLicenseKey = GetTrialLicenseKey(registrationName);
if (trialLicenseKey.empty()) {
@@ -1849,6 +1858,12 @@ namespace ANSCENTER
case ANSCENTER::ANSPRODUCTS::ANS_TS://1008
_productName = "ANSTS";
break;
case ANSCENTER::ANSPRODUCTS::ANSMIND://1009
_productName = "ANSMIND";
break;
case ANSCENTER::ANSPRODUCTS::ANSGW://1010
_productName = "ANSGW";
break;
}
if (_productName == productName) {
// Delete the license file
@@ -1896,6 +1911,12 @@ namespace ANSCENTER
else if (productName == "ANSTS") {//1008
registrationName = "ANSTS";
}
else if (productName == "ANSMIND") {//1009
registrationName = "ANSMIND";
}
else if (productName == "ANSGW") {//1010
registrationName = "ANSGW";
}
else {
registrationName = productName;
}
@@ -1936,6 +1957,12 @@ namespace ANSCENTER
else if (productName == "ANSTS") {//1008
registrationName = "ANSTS";
}
else if (productName == "ANSMIND") {//1009
registrationName = "ANSMIND";
}
else if (productName == "ANSGW") {//1010
registrationName = "ANSGW";
}
else {
registrationName = productName;
}
@@ -2002,6 +2029,8 @@ namespace ANSCENTER
ansLicenseCollection.push_back("ANSALPR");//1006
ansLicenseCollection.push_back("ANSCV");//1007
ansLicenseCollection.push_back("ANSTS");//1008
ansLicenseCollection.push_back("ANSMIND");//1009
ansLicenseCollection.push_back("ANSGW");//1010
for (auto& element : ansLicenseCollection) {
boost::property_tree::ptree dataNode;
@@ -2064,6 +2093,12 @@ namespace ANSCENTER
case ANSCENTER::ANSPRODUCTS::ANS_TS:
_productName = "ANSTS";
break;
case ANSCENTER::ANSPRODUCTS::ANSMIND:
_productName = "ANSMIND";
break;
case ANSCENTER::ANSPRODUCTS::ANSGW:
_productName = "ANSGW";
break;
}
if (_productName == element) {
_registrationName = ANSUtility::DecodeBase64Utility(_validationData.c_str());// Decode
@@ -2141,10 +2176,18 @@ namespace ANSCENTER
registrationName = "ANSTS";
productId = 1008;
}
else {
registrationName = ansLicenseInfo.productName;
else if (ansLicenseInfo.productName == "ANSMIND") {
registrationName = "ANSMIND";
productId = 1009;
}
else if (ansLicenseInfo.productName == "ANSGW") {
registrationName = "ANSGW";
productId = 1010;
}
else {
// Unknown product: skip the row rather than report it under some real product's id.
continue;
}
dataNode.put("product_name", ansLicenseInfo.productName);
std::string trialLicenseKey = GetTrialLicenseKey(registrationName);
if (trialLicenseKey.empty()) {
+3 -1
View File
@@ -259,7 +259,9 @@ namespace ANSCENTER
ANS_OCR = 1005,
ANS_ALPR =1006,
ANS_CV = 1007,
ANS_TS = 1008
ANS_TS = 1008,
ANSMIND = 1009,
ANSGW = 1010
};
enum ANSLICENSESTATUS {
ACTIVE = 0,
+74
View File
@@ -0,0 +1,74 @@
#ifndef ANSCSPRNG_H
#define ANSCSPRNG_H
#pragma once
// ANSCsprng — random bytes from the operating system, for keys, salts and
// nonces.
//
// TAKEN FROM ANSMINDCORE (src/core/csprng.hpp). NOT std::random_device or a
// seeded generator: fine for an id that only needs to be distinct, and wrong
// for anything a guesser must not predict. The OS generator is the only
// source used for key material here.
//
// Windows BCryptGenRandom with the system-preferred RNG (bcrypt.lib)
// Linux getrandom(2), looping over short reads and EINTR
// macOS arc4random_buf, which cannot fail
//
// Returns false when the OS refused — callers treat that as "cannot seal",
// never as "seal with whatever was in the buffer".
#include <cstddef>
#include <cstdint>
#if defined(_WIN32)
#ifndef WIN32_LEAN_AND_MEAN
#define WIN32_LEAN_AND_MEAN
#endif
#ifndef NOMINMAX
#define NOMINMAX
#endif
#include <windows.h>
#include <bcrypt.h>
#if defined(_MSC_VER)
#pragma comment(lib, "bcrypt.lib")
#endif
#elif defined(__APPLE__)
#include <stdlib.h>
#else
#include <cerrno>
#include <sys/random.h>
#endif
namespace ANSCENTER {
inline bool randomBytes(uint8_t* out, size_t n) {
if (n == 0) return true;
if (!out) return false;
#if defined(_WIN32)
while (n > 0) {
const ULONG chunk = n > 0x7fffffffu ? 0x7fffffffu : ULONG(n);
if (BCryptGenRandom(nullptr, out, chunk, BCRYPT_USE_SYSTEM_PREFERRED_RNG) != 0)
return false;
out += chunk;
n -= chunk;
}
return true;
#elif defined(__APPLE__)
arc4random_buf(out, n);
return true;
#else
while (n > 0) {
const ssize_t got = getrandom(out, n, 0);
if (got < 0) {
if (errno == EINTR) continue;
return false;
}
out += size_t(got);
n -= size_t(got);
}
return true;
#endif
}
} // namespace ANSCENTER
#endif // ANSCSPRNG_H
+7 -4
View File
@@ -13,11 +13,14 @@
// package is refused with a sentence that says why. That is what keeps the key
// out of a developer's checkout without keeping developers out of the code.
#if defined(__has_include)
#if __has_include("ANSModelKeysGenerated.h")
// ANSCORE_HAVE_BUILTIN_MODEL_KEYS is set by CMake when the ceremony's
// generated header is there. It is not tested with __has_include on purpose:
// that test leaves no dependency for the build system, so a key emitted after
// a build would not take effect until someone cleaned — and the failure looks
// exactly like a missing key, which on a release machine means shipping an
// engine that silently cannot open a package.
#if defined(ANSCORE_HAVE_BUILTIN_MODEL_KEYS)
#include "ANSModelKeysGenerated.h"
#define ANSCORE_HAVE_BUILTIN_MODEL_KEYS 1
#endif
#endif
#include "ANSModelPackage.h"
+21 -9
View File
@@ -217,16 +217,28 @@ inline bool OpenModelPackage(const std::string& path, OpenedModel& out,
out.name = m.value("project", std::string());
if (out.name.empty()) out.name = path;
// A package written before manifests carried an input size still has to
// answer, and the model in hand is the thing that knows. It costs nothing
// here: the bytes are already decrypted.
if (out.inputWidth <= 0 || out.inputHeight <= 0) {
onnxshape::InputShape shape;
// WHAT AN OLDER PACKAGE CANNOT TELL US, and its model can. The input
// size, when the manifest predates the field; and the head, because a
// package whose family is "yolo" may hold an end-to-end export — the box
// used to take the family from the backend marker, and the marker names
// the runtime rather than the head. It costs nothing to ask here: the
// bytes are already decrypted.
{
onnxshape::GraphShape g;
std::string swhy;
if (onnxshape::fromMemory(out.model.data(), out.model.size(), shape,
swhy)) {
out.inputWidth = shape.width;
out.inputHeight = shape.height;
if (onnxshape::graphFromMemory(out.model.data(), out.model.size(), g,
swhy)) {
if (out.inputWidth <= 0 || out.inputHeight <= 0) {
out.inputWidth = g.input.width;
out.inputHeight = g.input.height;
}
if (g.endToEnd() && out.family == "yolo") {
out.family = "yolo10";
out.warning = out.warning.empty()
? "this package says family yolo, but its head is "
"end-to-end; reading it as yolo10"
: out.warning;
}
}
}
return true;
+62 -18
View File
@@ -75,6 +75,7 @@
#include <nlohmann/json.hpp>
#include "ANSAead.h"
#include "ANSCsprng.h"
#include "ANSPackageBits.h"
extern "C" {
@@ -578,9 +579,15 @@ struct PackRequest {
Json card; // optional, counts only
std::vector<PackModel> models;
uint32_t chunk = kDefaultChunk; // smaller in tests, to exercise chunking
// Deterministic nonces are for TESTS ONLY: a real package draws them from
// the platform CSPRNG, because a repeated (key, nonce) pair is fatal.
uint8_t nonceSeed = 0;
// TESTS ONLY. pack() draws the content key, the ephemeral X25519 key and
// every nonce from the operating system's CSPRNG. Setting this derives them
// from `testSeed` instead, so a fixture can be written byte-for-byte again —
// and so anyone holding the package can recover its content key by trying
// the 256 seeds, without the base key. It is honoured only in a build that
// defines ANSMODEL_DETERMINISTIC_PACK_FOR_TESTS; anywhere else pack() refuses
// rather than write a package that only looks sealed.
bool insecureDeterministicForTests = false;
uint8_t testSeed = 0;
};
inline bool pack(const std::string& path, const PackRequest& req,
@@ -589,11 +596,58 @@ inline bool pack(const std::string& path, const PackRequest& req,
err = "a package with no model describes nothing";
return false;
}
// A content key per package. Tests pass a seed; a real packer must use the
// platform CSPRNG here.
// Everything the package's secrecy rests on, drawn up front: the content
// key, the ephemeral private key it is wrapped with, the wrap nonce and one
// nonce base per sealed part. From the OS CSPRNG — anything a holder of the
// file could re-derive would open it without the base key.
Key contentKey{};
for (size_t i = 0; i < contentKey.size(); ++i)
contentKey[i] = uint8_t(req.nonceSeed * 31 + i * 7 + 11);
Key esk{};
std::array<uint8_t, aead::kNonceBytes> wrapNonce{};
std::array<uint8_t, 4> manifestBase{};
std::vector<std::array<uint8_t, 4>> bases(req.models.size());
struct Wipe {
Key& a;
Key& b;
~Wipe() {
crypto_wipe(a.data(), a.size());
crypto_wipe(b.data(), b.size());
}
} wipe{contentKey, esk};
if (req.insecureDeterministicForTests) {
#if defined(ANSMODEL_DETERMINISTIC_PACK_FOR_TESTS)
for (size_t i = 0; i < contentKey.size(); ++i)
contentKey[i] = uint8_t(req.testSeed * 31 + i * 7 + 11);
for (size_t i = 0; i < esk.size(); ++i)
esk[i] = uint8_t(req.testSeed * 13 + i * 5 + 3);
for (size_t i = 0; i < wrapNonce.size(); ++i)
wrapNonce[i] = uint8_t(req.testSeed + 100 + i);
manifestBase = {uint8_t(req.testSeed + 1), 2, 3, 4};
for (size_t i = 0; i < bases.size(); ++i)
bases[i] = {uint8_t(req.testSeed + 10 + i), 5, 6, 7};
#else
err = "deterministic packing is for tests only, and this build is not one";
return false;
#endif
} else {
bool ok = ANSCENTER::randomBytes(contentKey.data(), contentKey.size()) &&
ANSCENTER::randomBytes(esk.data(), esk.size()) &&
ANSCENTER::randomBytes(wrapNonce.data(), wrapNonce.size()) &&
ANSCENTER::randomBytes(manifestBase.data(), manifestBase.size());
// The parts share the content key, so their bases must differ: two parts
// on one base would reuse every chunk nonce they have in common.
for (size_t i = 0; ok && i < bases.size(); ++i) {
bool clash = true;
while (ok && clash) {
ok = ANSCENTER::randomBytes(bases[i].data(), bases[i].size());
clash = bases[i] == manifestBase;
for (size_t j = 0; j < i; ++j) clash = clash || bases[i] == bases[j];
}
}
if (!ok) {
err = "the operating system gave no random bytes, so nothing was sealed";
return false;
}
}
auto sealChunked = [&](const std::vector<uint8_t>& plain,
const std::string& payloadId,
@@ -616,7 +670,6 @@ inline bool pack(const std::string& path, const PackRequest& req,
const std::string manifestText = req.manifest.dump();
const std::vector<uint8_t> manifestBytes(manifestText.begin(),
manifestText.end());
std::array<uint8_t, 4> manifestBase{uint8_t(req.nonceSeed + 1), 2, 3, 4};
std::string manifestBlob;
{
const auto sealed =
@@ -628,20 +681,14 @@ inline bool pack(const std::string& path, const PackRequest& req,
}
std::vector<std::string> blobs;
std::vector<std::array<uint8_t, 4>> bases;
for (size_t i = 0; i < req.models.size(); ++i) {
std::array<uint8_t, 4> base{uint8_t(req.nonceSeed + 10 + i), 5, 6, 7};
std::string blob;
sealChunked(req.models[i].bytes, req.models[i].id, base, blob);
sealChunked(req.models[i].bytes, req.models[i].id, bases[i], blob);
blobs.push_back(std::move(blob));
bases.push_back(base);
}
// Wrap the content key to the base key. One wrap here; the header's shape
// takes several, which is how a package opens on more than one key.
std::array<uint8_t, aead::kNonceBytes> wrapNonce{};
for (size_t i = 0; i < wrapNonce.size(); ++i)
wrapNonce[i] = uint8_t(req.nonceSeed + 100 + i);
const std::vector<uint8_t> keyPlain(contentKey.begin(), contentKey.end());
Key wrapKey = req.key;
@@ -650,9 +697,6 @@ inline bool pack(const std::string& path, const PackRequest& req,
// An ephemeral key pair per package: its public half travels in the
// header, its private half dies here, and the packer never holds anything
// that opens the result.
Key esk{};
for (size_t i = 0; i < esk.size(); ++i)
esk[i] = uint8_t(req.nonceSeed * 13 + i * 5 + 3);
epk.assign(32, 0);
crypto_x25519_public_key(epk.data(), esk.data());
uint8_t shared[32];
+145 -32
View File
@@ -3,18 +3,16 @@
#pragma once
// ANSOnnxShape — the input geometry a model declares, read off the graph.
//
// WHY A READER NEEDS THIS. An engine has to know what its model expects before
// it can preprocess a single frame, and that answer has always come from
// whatever the bundle happened to carry: model_config.json's input_size, or an
// `input` object in a package manifest. Neither is present on every model —
// most of the bundles in the field are classes.names, a marker file and the
// export, and nothing else. The graph is always there, and the graph is what
// the session is built from, so where the metadata is silent the model itself
// is asked instead of every engine falling back to 640x640.
// WHY A READER NEEDS THIS. A node has to know what its model expects before it
// can preprocess a single frame, and until now that answer came from whatever
// the bundle happened to carry: model_config.json's input_size in a folder, an
// `input` object in a package manifest. Neither is present on every model. The
// graph always is, and the graph is the thing the session is built from — so
// where the metadata is silent, ask the model itself rather than assume 640.
//
// TWO SOURCES, ONE PARSER. A package's model is already in memory, decrypted,
// and must never reach the disk; an inflated bundle's is a file that may be
// hundreds of megabytes. The template below reads either, and the file case
// and must never reach the disk; an imported folder's model is a file that may
// be hundreds of megabytes. The template below reads either, and the file case
// SEEKS past the nodes and the initializers rather than reading them — the
// fields this wants sit after all of the weight on the wire.
//
@@ -23,12 +21,16 @@
// ModelProto.graph field 7, length-delimited
// GraphProto.input field 11, repeated (after node and
// initializer on the wire)
// GraphProto.output field 12, repeated
// ValueInfoProto.type field 2
// TypeProto.tensor_type field 1
// TypeProto.Tensor.shape field 2
// TensorShapeProto.dim field 1, repeated
// Dimension.dim_value field 1, varint (dim_param means the
// axis is dynamic: reported as 0)
// ModelProto.metadata_props field 14, repeated StringStringEntryProto
// StringStringEntryProto.key field 1, string
// StringStringEntryProto.value field 2, string
//
// An axis the graph leaves open is reported as zero, and a file that cannot be
// parsed is reported as a failure. Both mean the same thing to a caller: this
@@ -36,6 +38,7 @@
// model's input size is worse than being silent about it.
#include <cstdint>
#include <cctype>
#include <cstdio>
#include <filesystem>
#include <fstream>
@@ -52,6 +55,32 @@ struct InputShape {
bool known() const { return height > 0 && width > 0; }
};
// What the whole graph says about itself: what goes in, and what shape comes
// out — which is the difference between a head that needs decoding and one
// that has already done it.
struct GraphShape {
InputShape input;
std::vector<int64_t> output; // the FIRST output's dims; empty when unknown
bool exportSaysEndToEnd = false; // the model's own metadata, when it says
// AN END-TO-END HEAD, [1, N, 6]: rows of (x1 y1 x2 y2 score class) with the
// NMS already inside the graph. Telling one apart from an ordinary YOLO head
// matters more than it looks: feeding [1,N,6] to a v8 decoder yields boxes
// everywhere with coordinates read as confidences — plausible-looking
// garbage rather than an error.
//
// The trap is a two-class v8 export, whose [1, 4+nc, N] is [1, 6, 8400] —
// a 6 in the shape, and not end-to-end. So the 6 must be the LAST axis, and
// the axis before it must be a detection count rather than an anchor count;
// an end-to-end head emits max_det rows, which is hundreds, never thousands.
// When the export's own metadata says end2end, that settles it outright.
bool endToEnd() const {
if (exportSaysEndToEnd) return true;
return output.size() == 3 && output[0] == 1 && output[2] == 6 &&
output[1] > 0 && output[1] <= 1000;
}
};
namespace detail {
// A model already in memory: the decrypted contents of a package.
@@ -158,51 +187,105 @@ std::vector<int64_t> dims(Src& s, int64_t end) {
return out;
}
// One ValueInfoProto (a graph input or output), given a stream on its payload:
// its dims, or empty when it is not a shaped tensor.
template <class Src>
bool read(Src& s, int64_t end, InputShape& out, std::string& why) {
std::vector<int64_t> valueDims(Src& s, int64_t valueEnd) {
uint64_t len = 0;
if (!seekField(s, valueEnd, 2, len)) return {}; // ValueInfoProto.type
const int64_t typeEnd = s.tell() + int64_t(len);
if (!seekField(s, typeEnd, 1, len)) return {}; // TypeProto.tensor_type
const int64_t tensorEnd = s.tell() + int64_t(len);
if (!seekField(s, tensorEnd, 2, len)) return {}; // Tensor.shape
return dims(s, s.tell() + int64_t(len));
}
// The graph's first input and first output, and whether the export declared
// itself end-to-end. `end` is the whole file; `graphStart` lets the output be
// found after the input, since both hang off the same graph.
template <class Src>
bool readGraph(Src& s, int64_t end, GraphShape& out, std::string& why) {
uint64_t len = 0;
if (!seekField(s, end, 7, len)) { // ModelProto.graph
why = "this does not look like an ONNX model";
return false;
}
const int64_t graphEnd = s.tell() + int64_t(len);
const int64_t graphStart = s.tell();
const int64_t graphEnd = graphStart + int64_t(len);
if (!seekField(s, graphEnd, 11, len)) { // GraphProto.input
why = "the graph declares no input";
return false;
}
const int64_t inputEnd = s.tell() + int64_t(len);
if (!seekField(s, inputEnd, 2, len)) { // ValueInfoProto.type
why = "the input declares no type";
return false;
const std::vector<int64_t> in = valueDims(s, s.tell() + int64_t(len));
// The output, from the graph's start again: it sits after the input on the
// wire, but seeking from the start costs nothing and does not depend on it.
s.seek(graphStart);
if (seekField(s, graphEnd, 12, len)) // GraphProto.output
out.output = valueDims(s, s.tell() + int64_t(len));
// The export's own word on its head, which beats any shape rule.
s.seek(0);
while (s.ok() && s.tell() >= 0 && s.tell() < end) {
uint32_t field = 0, wire = 0;
if (!tag(s, field, wire)) break;
if (field != 14 || wire != 2) { // ModelProto.metadata_props
if (!skip(s, wire)) break;
continue;
}
uint64_t entryLen = 0;
if (!varint(s, entryLen)) break;
const int64_t entryEnd = s.tell() + int64_t(entryLen);
std::string key, value;
while (s.ok() && s.tell() >= 0 && s.tell() < entryEnd) {
uint32_t ef = 0, ew = 0;
if (!tag(s, ef, ew)) break;
if ((ef == 1 || ef == 2) && ew == 2) {
uint64_t n = 0;
if (!varint(s, n)) break;
std::string text;
for (uint64_t i = 0; i < n; ++i) {
const int c = s.get();
if (c < 0) break;
text.push_back(char(c));
}
(ef == 1 ? key : value) = text;
} else if (!skip(s, ew)) {
break;
}
}
s.seek(entryEnd);
if (key == "end2end") {
for (char& c : value) c = char(::tolower(static_cast<unsigned char>(c)));
out.exportSaysEndToEnd = (value == "true" || value == "1");
}
}
const int64_t typeEnd = s.tell() + int64_t(len);
if (!seekField(s, typeEnd, 1, len)) { // TypeProto.tensor_type
why = "the input is not a tensor";
return false;
}
const int64_t tensorEnd = s.tell() + int64_t(len);
if (!seekField(s, tensorEnd, 2, len)) { // Tensor.shape
why = "the input tensor has no shape";
return false;
}
const std::vector<int64_t> d = dims(s, s.tell() + int64_t(len));
// NCHW is the only layout these engines run.
if (d.size() != 4) {
if (in.size() != 4) {
why = "the input is not a four-dimensional image tensor";
return false;
}
auto axis = [](int64_t v) { return (v > 0 && v < 1000000) ? int(v) : 0; };
out.channels = axis(d[1]);
out.height = axis(d[2]);
out.width = axis(d[3]);
if (!out.known()) {
out.input.channels = axis(in[1]);
out.input.height = axis(in[2]);
out.input.width = axis(in[3]);
if (!out.input.known()) {
why = "the model takes a dynamic input size";
return false;
}
return true;
}
template <class Src>
bool read(Src& s, int64_t end, InputShape& out, std::string& why) {
GraphShape g;
if (!readGraph(s, end, g, why)) return false;
out = g.input;
return true;
}
} // namespace detail
// What a model already in memory declares.
@@ -216,6 +299,17 @@ inline bool fromMemory(const void* data, size_t size, InputShape& out,
return detail::read(s, int64_t(size), out, why);
}
// The whole graph, from a model already in memory.
inline bool graphFromMemory(const void* data, size_t size, GraphShape& out,
std::string& why) {
if (!data || size == 0) {
why = "there is no model to read";
return false;
}
detail::MemSource s{static_cast<const uint8_t*>(data), int64_t(size), 0};
return detail::readGraph(s, int64_t(size), out, why);
}
// What a model on disk declares. The file is walked, not loaded.
inline bool fromFile(const std::string& path, InputShape& out,
std::string& why) {
@@ -235,6 +329,25 @@ inline bool fromFile(const std::string& path, InputShape& out,
return detail::read(s, int64_t(size), out, why);
}
// The whole graph, from a model on disk. Walked, not loaded.
inline bool graphFromFile(const std::string& path, GraphShape& out,
std::string& why) {
std::error_code ec;
const auto size = std::filesystem::file_size(path, ec);
if (ec || size == 0) {
why = "the model is not there to read";
return false;
}
detail::FileSource s;
s.f.open(path, std::ios::binary);
s.size = int64_t(size);
if (!s.f) {
why = "the model could not be opened";
return false;
}
return detail::readGraph(s, int64_t(size), out, why);
}
} // namespace onnxshape
} // namespace ANSCENTER
+28 -11
View File
@@ -14,18 +14,35 @@ target_include_directories(ANSModelPackage PUBLIC
${CMAKE_SOURCE_DIR}/3rdparty/monocypher
)
# nlohmann/json comes from vcpkg on Windows and the system elsewhere, the same
# way ANSFR already takes it.
if(WIN32)
target_include_directories(ANSModelPackage PUBLIC ${VCPKG_INCLUDE_DIR})
# nlohmann/json, wherever this machine keeps it: the ANSLibs tree on Windows,
# vcpkg, or the system elsewhere. Asked for by name rather than assumed — the
# Windows branch used to point at vcpkg alone, and on a real build machine the
# header lives under ANSLibs, so every target that linked this one failed to
# find it while the engines, which add ANSLibs themselves, compiled fine.
find_path(ANSCORE_NLOHMANN_INCLUDE nlohmann/json.hpp
HINTS ${ANSLIBS_DIR} ${VCPKG_INCLUDE_DIR}
/opt/homebrew/include /usr/local/include /usr/include)
if(ANSCORE_NLOHMANN_INCLUDE)
target_include_directories(ANSModelPackage PUBLIC ${ANSCORE_NLOHMANN_INCLUDE})
message(STATUS "ANSModelPackage: nlohmann/json at ${ANSCORE_NLOHMANN_INCLUDE}")
else()
find_path(ANSCORE_NLOHMANN_INCLUDE nlohmann/json.hpp
HINTS /opt/homebrew/include /usr/local/include /usr/include)
if(ANSCORE_NLOHMANN_INCLUDE)
target_include_directories(ANSModelPackage PUBLIC ${ANSCORE_NLOHMANN_INCLUDE})
else()
message(WARNING "nlohmann/json.hpp not found — ANSModelPackage will not build")
endif()
message(WARNING "nlohmann/json.hpp not found — ANSModelPackage will not build")
endif()
# The model-package key, decided at configure time: see ANSModelKeys.h for why
# this is not an __has_include test. PUBLIC so every engine that opens a
# package is compiled with the same answer.
if(EXISTS "${CMAKE_CURRENT_SOURCE_DIR}/ANSModelKeysGenerated.h")
target_compile_definitions(ANSModelPackage PUBLIC ANSCORE_HAVE_BUILTIN_MODEL_KEYS=1)
message(STATUS "ANSModelPackage: key present — packages can be opened")
else()
message(STATUS "ANSModelPackage: no key — packages cannot be opened "
"(run ansmodel-keys emit --product anscore, then re-run cmake)")
endif()
# BCryptGenRandom, behind the content key and nonces pack() draws (ANSCsprng.h).
if(WIN32)
target_link_libraries(ANSModelPackage PUBLIC bcrypt)
endif()
set_target_properties(ANSModelPackage PROPERTIES POSITION_INDEPENDENT_CODE ON)
@@ -316,7 +316,7 @@ bool Engine<T>::buildLoadNetwork(std::string onnxModelPath, const std::array<flo
return false;
}
}
if (!FileExist(onnxModelPath)) {
if (!hasOnnxBytes() && !FileExist(onnxModelPath)) {
// ONNX model does not exist, try to find alternative precision engine
std::cout << "Searching for alternative precision engine..." << std::endl;
@@ -390,12 +390,9 @@ bool Engine<T>::buildLoadNetwork(std::string onnxModelPath, const std::array<flo
auto tempNetwork = std::unique_ptr<nvinfer1::INetworkDefinition>(TRT_CREATE_NETWORK(tempBuilder));
auto tempParser = std::unique_ptr<nvonnxparser::IParser>(nvonnxparser::createParser(*tempNetwork, m_logger));
std::ifstream onnxFile(onnxModelPath, std::ios::binary | std::ios::ate);
std::streamsize onnxSize = onnxFile.tellg();
onnxFile.seekg(0, std::ios::beg);
std::vector<char> onnxBuffer(onnxSize);
if (!onnxFile.read(onnxBuffer.data(), onnxSize)) {
std::cout << "Error: Failed to read ONNX file" << std::endl;
std::vector<char> onnxBuffer;
if (!readOnnxBytes(onnxModelPath, onnxBuffer)) {
std::cout << "Error: Failed to read ONNX model" << std::endl;
return false;
}
@@ -1465,7 +1462,7 @@ bool Engine<T>::build(std::string onnxModelPath, const std::array<float, 3>& sub
std::cout << "Engine file already exists: " << engineName << std::endl;
return true;
}
if (!FileExist(onnxModelPath)) {
if (!hasOnnxBytes() && !FileExist(onnxModelPath)) {
std::cout << "Error: ONNX model file does not exist: " << onnxModelPath << std::endl;
return false;
}
@@ -1502,16 +1499,14 @@ bool Engine<T>::build(std::string onnxModelPath, const std::array<float, 3>& sub
return false;
}
// Read the onnx file into memory
std::ifstream file(onnxModelPath, std::ios::binary | std::ios::ate);
std::streamsize size = file.tellg();
file.seekg(0, std::ios::beg);
std::vector<char> buffer(size);
if (!file.read(buffer.data(), size)) {
std::cout << "Error: Unable to read ONNX file" << std::endl;
// Get the onnx bytes: from the in-memory buffer of a sealed package, or
// from the file on disk.
std::vector<char> buffer;
if (!readOnnxBytes(onnxModelPath, buffer)) {
std::cout << "Error: Unable to read ONNX model" << std::endl;
return false;
}
const std::streamsize size = static_cast<std::streamsize>(buffer.size());
std::cout << "ONNX model size: " << size / (1024 * 1024) << " MiB" << std::endl;
@@ -2471,16 +2466,14 @@ bool Engine<T>::buildWithRetry(std::string onnxModelPath,
auto tempParser = std::unique_ptr<nvonnxparser::IParser>(
nvonnxparser::createParser(*tempNetwork, m_logger));
std::ifstream onnxFile(onnxModelPath, std::ios::binary | std::ios::ate);
if (!onnxFile.good()) {
std::vector<char> onnxBuffer;
const bool onnxReadable = readOnnxBytes(onnxModelPath, onnxBuffer);
if (!onnxReadable) {
ANS_DBG("TRT_Build", "buildWithRetry WARN cannot open ONNX for pre-parse path=%s",
onnxModelPath.c_str());
}
if (onnxFile.good()) {
std::streamsize onnxSize = onnxFile.tellg();
onnxFile.seekg(0, std::ios::beg);
std::vector<char> onnxBuffer(onnxSize);
if (onnxFile.read(onnxBuffer.data(), onnxSize)) {
if (onnxReadable) {
{
unsigned long sehRetryParse = 0;
bool retryParsed = parseOnnxModelSafe(tempParser.get(),
onnxBuffer.data(), onnxBuffer.size(), &sehRetryParse);
@@ -2508,9 +2501,6 @@ bool Engine<T>::buildWithRetry(std::string onnxModelPath,
(int)retryParsed,
retryParsed ? tempNetwork->getNbInputs() : -1);
}
} else {
ANS_DBG("TRT_Build", "buildWithRetry WARN ONNX read failed bytes=%zd",
(long long)onnxSize);
}
}
} else {
@@ -88,8 +88,20 @@ public:
const std::array<float, 3>& subVals,
const std::array<float, 3>& divVals,
bool normalize,
int maxSlotsPerGpu)
int maxSlotsPerGpu,
const void* onnxData = nullptr,
size_t onnxSize = 0)
{
// A sealed .ansmodel is decrypted in memory and never written out, so
// the caller hands the bytes down here: every Engine this call builds
// parses them instead of opening modelPath, which only names the
// TensorRT plan that gets cached.
auto newEngine = [&]() {
auto e = std::make_shared<Engine<T>>(options);
if (onnxData != nullptr && onnxSize > 0) e->setOnnxBytes(onnxData, onnxSize);
return e;
};
// Optimizer / temporary engines: maxSlotsPerGpu==0 means the caller
// only needs a lightweight, non-shared engine (e.g., OptimizeModelStr).
// Bypass the pool cache entirely:
@@ -99,7 +111,7 @@ public:
// Note: maxSlotsPerGpu==1 is now the normal "1 slot per GPU" multi-GPU
// round-robin mode, so it goes through the pool path below.
if (maxSlotsPerGpu == 0) {
auto engine = std::make_shared<Engine<T>>(options);
auto engine = newEngine();
bool ok = engine->buildLoadNetwork(modelPath, subVals, divVals, normalize);
return ok ? engine : nullptr;
}
@@ -141,13 +153,13 @@ public:
}
// Cache miss — create new Engine pool
auto engine = std::make_shared<Engine<T>>(options);
auto engine = newEngine();
bool ok = engine->buildLoadNetwork(modelPath, subVals, divVals, normalize, maxSlotsPerGpu);
if (!ok) {
// Step 1: Force-evict all pools with refcount=0 to reclaim VRAM
int evicted = forceEvictPending();
if (evicted > 0) {
engine = std::make_shared<Engine<T>>(options);
engine = newEngine();
ok = engine->buildLoadNetwork(modelPath, subVals, divVals, normalize, maxSlotsPerGpu);
}
@@ -156,7 +168,7 @@ public:
// consumes ~300-500 MB vs ~50-100 MB for a simple loadNetwork.
// Lightweight mode: tasks queue for a single shared slot — slower but works.
if (!ok) {
engine = std::make_shared<Engine<T>>(options);
engine = newEngine();
ok = engine->buildLoadNetwork(modelPath, subVals, divVals, normalize);
}
@@ -183,7 +195,7 @@ public:
}
// Final retry — try lightweight again after delay
engine = std::make_shared<Engine<T>>(options);
engine = newEngine();
ok = engine->buildLoadNetwork(modelPath, subVals, divVals, normalize);
}
@@ -4,6 +4,48 @@
#include <NvInferVersion.h> // also defines TRT version macros
#include <cudnn_version.h> // CUDNN_MAJOR/MINOR/PATCHLEVEL
#include <cuda_runtime.h> // cudaRuntimeGetVersion
#include <cstring> // memset for wiping decrypted model bytes
#include <fstream>
// ============================================================================
// ONNX bytes in memory
//
// A sealed .ansmodel is decrypted into RAM and stays there: handing the
// parser a buffer is what keeps the plaintext model off the disk. The build
// path still takes a path, because the path names the TensorRT plan that is
// cached beside it — a plan is a built artefact and carries no model.
// ============================================================================
template <typename T>
void Engine<T>::setOnnxBytes(const void* data, size_t size) {
clearOnnxBytes();
if (data == nullptr || size == 0) return;
const char* p = static_cast<const char*>(data);
m_onnxBytes.assign(p, p + size);
}
template <typename T>
void Engine<T>::clearOnnxBytes() {
if (m_onnxBytes.empty()) return;
std::memset(m_onnxBytes.data(), 0, m_onnxBytes.size());
m_onnxBytes.clear();
m_onnxBytes.shrink_to_fit();
}
template <typename T>
bool Engine<T>::readOnnxBytes(const std::string& onnxModelPath,
std::vector<char>& buffer) const {
if (!m_onnxBytes.empty()) {
buffer = m_onnxBytes;
return true;
}
std::ifstream file(onnxModelPath, std::ios::binary | std::ios::ate);
if (!file.good()) return false;
const std::streamsize size = file.tellg();
if (size <= 0) return false;
file.seekg(0, std::ios::beg);
buffer.resize(static_cast<size_t>(size));
return static_cast<bool>(file.read(buffer.data(), size));
}
template <typename T>
void Engine<T>::transformOutput(std::vector<std::vector<std::vector<T>>> &input, std::vector<std::vector<T>> &output) {
+1 -1
View File
@@ -22,7 +22,7 @@ namespace ANSCENTER
{1005, "ANSOCR"},
{1006, "ANSALPR"},
{1007, "ANSCV"},
{1008, "ANSSRT"}
{1008, "ANSTS"}
};
ansllmLicenceValid = false;
for (const auto& [productId, productName] : licenseChecks) {
+84 -29
View File
@@ -11,12 +11,37 @@
namespace ANSCENTER {
// ====================================================================
// Sealed packages (.ansmodel)
//
// TensorRT still writes a plan to disk and still reads it back on the
// next run — that is what makes the second start fast, and a plan is a
// built artefact that carries no model. What must never be written is
// the ONNX itself, so the builder parses the decrypted bytes from
// memory and the path below names nothing that is ever created: it
// only tells the plan where to live and what to be called. The folder
// is the one the zip flow used, so a package converted from a zip
// reuses the plan that zip already built.
// ====================================================================
std::string ANSRTYOLO::PackagePlanBasePath(const std::string& packagePath) {
const std::string folder = GenerateOutputFolder(
packagePath, GetFileNameWithoutExtension(packagePath), false);
return CreateFilePath(folder, "train_last.onnx");
}
// ====================================================================
// ANSODBase interface — OptimizeModel
// ====================================================================
bool ANSRTYOLO::OptimizeModel(bool fp16, std::string& optimizedModelFolder) {
std::lock_guard<std::recursive_mutex> lock(_mutex);
if (!ANSODBase::OptimizeModel(fp16, optimizedModelFolder)) return false;
if (_fromPackage) {
// The plan is built from the decrypted bytes when the engine
// loads; there is no ONNX on disk to pre-optimise. Report the
// folder the plan is cached in so the caller sees a real path.
optimizedModelFolder = GetParentFolder(_modelFilePath);
return true;
}
if (!FileExist(_modelFilePath)) {
_logger.LogFatal("ANSRTYOLO::OptimizeModel", "Raw model file path does not exist", __FILE__, __LINE__);
return false;
@@ -96,21 +121,33 @@ namespace ANSCENTER {
m_options.engineFileDir = _modelFolder;
m_options.precision = (_fp16 ? Precision::FP16 : Precision::FP32);
_modelFilePath = CreateFilePath(_modelFolder, "train_last.onnx");
if (FileExist(_modelConfigFile)) {
ModelType modelType;
std::vector<int> inputShape;
_classes = ANSUtilityHelper::GetConfigFileContent(_modelConfigFile, modelType, inputShape);
if (inputShape.size() == 2) {
if (inputShape[0] > 0) _modelConfig.inpHeight = inputShape[0];
if (inputShape[1] > 0) _modelConfig.inpWidth = inputShape[1];
}
if (_fromPackage) {
// The package already carries the classes and the geometry,
// and there is no folder to look in. All that is needed here
// is where the built plan lives.
_modelFilePath = PackagePlanBasePath(modelZipFilePath);
m_options.engineFileDir = GetParentFolder(_modelFilePath);
// The defaults above have just overwritten the input size the
// package asked for; put it back.
ApplyPackageInputSize();
}
else {
_classFilePath = CreateFilePath(_modelFolder, "classes.names");
std::ifstream isValid(_classFilePath);
if (!isValid) LoadClassesFromString();
else LoadClassesFromFile();
_modelFilePath = CreateFilePath(_modelFolder, "train_last.onnx");
if (FileExist(_modelConfigFile)) {
ModelType modelType;
std::vector<int> inputShape;
_classes = ANSUtilityHelper::GetConfigFileContent(_modelConfigFile, modelType, inputShape);
if (inputShape.size() == 2) {
if (inputShape[0] > 0) _modelConfig.inpHeight = inputShape[0];
if (inputShape[1] > 0) _modelConfig.inpWidth = inputShape[1];
}
}
else {
_classFilePath = CreateFilePath(_modelFolder, "classes.names");
std::ifstream isValid(_classFilePath);
if (!isValid) LoadClassesFromString();
else LoadClassesFromFile();
}
}
if (this->_loadEngineOnCreation) {
@@ -119,7 +156,9 @@ namespace ANSCENTER {
static_cast<int>(m_options.precision), m_options.maxBatchSize };
m_trtEngine = EnginePoolManager<float>::instance().acquire(
m_poolKey, m_options, _modelFilePath,
SUB_VALS, DIV_VALS, NORMALIZE, m_maxSlotsPerGpu);
SUB_VALS, DIV_VALS, NORMALIZE, m_maxSlotsPerGpu,
_fromPackage ? _packageModel.data() : nullptr,
_fromPackage ? _packageModel.size() : 0);
m_usingSharedPool = (m_trtEngine != nullptr);
}
if (!m_trtEngine) {
@@ -221,7 +260,9 @@ namespace ANSCENTER {
static_cast<int>(m_options.precision), m_options.maxBatchSize };
m_trtEngine = EnginePoolManager<float>::instance().acquire(
m_poolKey, m_options, _modelFilePath,
SUB_VALS, DIV_VALS, NORMALIZE, m_maxSlotsPerGpu);
SUB_VALS, DIV_VALS, NORMALIZE, m_maxSlotsPerGpu,
_fromPackage ? _packageModel.data() : nullptr,
_fromPackage ? _packageModel.size() : 0);
m_usingSharedPool = (m_trtEngine != nullptr);
}
if (!m_trtEngine) {
@@ -295,21 +336,33 @@ namespace ANSCENTER {
m_options.engineFileDir = _modelFolder;
m_options.precision = (_fp16 ? Precision::FP16 : Precision::FP32);
_modelFilePath = CreateFilePath(_modelFolder, "train_last.onnx");
if (FileExist(_modelConfigFile)) {
ModelType modelType;
std::vector<int> inputShape;
_classes = ANSUtilityHelper::GetConfigFileContent(_modelConfigFile, modelType, inputShape);
if (inputShape.size() == 2) {
if (inputShape[0] > 0) _modelConfig.inpHeight = inputShape[0];
if (inputShape[1] > 0) _modelConfig.inpWidth = inputShape[1];
}
if (_fromPackage) {
// The package already carries the classes and the geometry,
// and there is no folder to look in. All that is needed here
// is where the built plan lives.
_modelFilePath = PackagePlanBasePath(modelZipFilePath);
m_options.engineFileDir = GetParentFolder(_modelFilePath);
// The defaults above have just overwritten the input size the
// package asked for; put it back.
ApplyPackageInputSize();
}
else {
_classFilePath = CreateFilePath(_modelFolder, "classes.names");
std::ifstream isValid(_classFilePath);
if (!isValid) LoadClassesFromString();
else LoadClassesFromFile();
_modelFilePath = CreateFilePath(_modelFolder, "train_last.onnx");
if (FileExist(_modelConfigFile)) {
ModelType modelType;
std::vector<int> inputShape;
_classes = ANSUtilityHelper::GetConfigFileContent(_modelConfigFile, modelType, inputShape);
if (inputShape.size() == 2) {
if (inputShape[0] > 0) _modelConfig.inpHeight = inputShape[0];
if (inputShape[1] > 0) _modelConfig.inpWidth = inputShape[1];
}
}
else {
_classFilePath = CreateFilePath(_modelFolder, "classes.names");
std::ifstream isValid(_classFilePath);
if (!isValid) LoadClassesFromString();
else LoadClassesFromFile();
}
}
labelMap.clear();
@@ -322,7 +375,9 @@ namespace ANSCENTER {
static_cast<int>(m_options.precision), m_options.maxBatchSize };
m_trtEngine = EnginePoolManager<float>::instance().acquire(
m_poolKey, m_options, _modelFilePath,
SUB_VALS, DIV_VALS, NORMALIZE, m_maxSlotsPerGpu);
SUB_VALS, DIV_VALS, NORMALIZE, m_maxSlotsPerGpu,
_fromPackage ? _packageModel.data() : nullptr,
_fromPackage ? _packageModel.size() : 0);
m_usingSharedPool = (m_trtEngine != nullptr);
}
if (!m_trtEngine) {
+5
View File
@@ -52,6 +52,11 @@ namespace ANSCENTER {
~ANSRTYOLO();
private:
// Where a sealed package's TensorRT plan is cached, and what it is
// called. Nothing but the plan is ever written there — the model
// itself is parsed from memory.
static std::string PackagePlanBasePath(const std::string& packagePath);
std::string _modelFilePath;
bool _modelLoadValid{ false };
bool _fp16{ false };
+94
View File
@@ -552,6 +552,30 @@ namespace ANSCENTER {
const float* rawOutput = outputTensors[0].GetTensorMutableData<float>();
const std::vector<int64_t> outputShape = outputTensors[0].GetTensorTypeAndShapeInfo().GetShape();
if (outputShape.size() < 3) {
return detections;
}
// WHICH HEAD THIS MODEL HAS. An export with end2end on has already
// done its own non-maximum suppression and answers [1, N, 6] —
// rows of (x1 y1 x2 y2 score class) — where an ordinary YOLO
// answers [1, 4+nc, anchors]. Its backend marker is still
// tensorrt.txt either way, because the marker names the runtime
// and not the head, so the tensor is the only thing that knows.
//
// Reading one as the other does not fail: it takes 300 as a
// feature count, decides the model has 296 classes, reads
// coordinates as class scores and returns boxes covering the
// frame. The trap in the other direction is a two-class export,
// whose [1, 4+nc, N] is [1, 6, 8400] — a 6 in the shape and not a
// decoded head — so the 6 must be the LAST axis with a detection
// count before it: hundreds, never thousands of anchors.
if (outputShape[0] == 1 && outputShape[2] == 6 &&
outputShape[1] > 0 && outputShape[1] <= 1000) {
return postprocessEndToEnd(originalImageSize, resizedImageShape,
outputTensors, confThreshold);
}
// Determine the number of features and detections
const size_t num_features = outputShape[1];
const size_t num_detections = outputShape[2];
@@ -675,6 +699,76 @@ namespace ANSCENTER {
}
}
// An end-to-end head has already chosen its detections: the rows are boxes
// in model-pixel corners with a score and a class, and the suppression
// happened inside the graph. None is run here — running one anyway would
// hide a broken export instead of showing it.
std::vector<Object> YOLO12OD::postprocessEndToEnd(const cv::Size& originalImageSize,
const cv::Size& resizedImageShape,
std::vector<Ort::Value>& outputTensors,
float confThreshold)
{
std::vector<Object> detections;
if (outputTensors.empty()) return detections;
// GetTensorMutableData (not GetTensorData) on DML — the const variant
// hangs on AMD after ~8 calls. Read-only despite the name.
const float* rawOutput = outputTensors[0].GetTensorMutableData<float>();
const std::vector<int64_t> outputShape =
outputTensors[0].GetTensorTypeAndShapeInfo().GetShape();
if (outputShape.size() < 3) return detections;
const int numDets = static_cast<int>(outputShape[1]);
const int numFeat = static_cast<int>(outputShape[2]);
const int classNameSize = static_cast<int>(_classes.size());
detections.reserve(static_cast<size_t>(numDets));
for (int i = 0; i < numDets; ++i) {
const float* det = rawOutput + static_cast<size_t>(i) * numFeat;
const float conf = det[4];
// The rows are not obliged to be sorted, so every one is read.
if (conf <= confThreshold) continue;
// Corners in model pixels, handed to the same scaler the grid path
// uses: it undoes this engine's own letterbox and clips to the
// frame, so both heads agree about where a box is.
const float left = det[0];
const float top = det[1];
const float width = det[2] - det[0];
const float height = det[3] - det[1];
if (width <= 0.0f || height <= 0.0f) continue;
// BoundingBox is integral and these corners are not, so the
// rounding is spelled out rather than left to a narrowing
// conversion that would quietly truncate every box by up to a
// pixel.
const BoundingBox modelBox(static_cast<int>(std::lround(left)),
static_cast<int>(std::lround(top)),
static_cast<int>(std::lround(width)),
static_cast<int>(std::lround(height)));
const BoundingBox scaled =
scaleCoords(resizedImageShape, modelBox, originalImageSize, true);
if (scaled.width <= 0 || scaled.height <= 0) continue;
Object detection;
detection.confidence = conf;
detection.classId = static_cast<int>(det[5]);
detection.box = cv::Rect(scaled.x, scaled.y, scaled.width, scaled.height);
if (classNameSize > 0) {
detection.className =
(detection.classId >= 0 && detection.classId < classNameSize)
? _classes[detection.classId]
: _classes[classNameSize - 1];
} else {
detection.className = "Unknown";
}
detection.polygon = ANSUtilityHelper::RectToNormalizedPolygon(
detection.box, m_imgWidth, m_imgHeight);
detections.push_back(std::move(detection));
}
return detections;
}
std::vector<Object> YOLO12OD::detect(const cv::Mat& image, float confThreshold, float iouThreshold) {
std::lock_guard<std::recursive_mutex> lock(_mutex);
try {
+9
View File
@@ -59,6 +59,15 @@ namespace ANSCENTER {
std::vector<Ort::Value>& outputTensors,
float confThreshold, float iouThreshold);
/// The [1, N, 6] head: rows of (x1 y1 x2 y2 score class) whose
/// non-maximum suppression happened inside the graph. postprocess()
/// hands the tensor here when it recognises that shape; reading it
/// as a [1, 4+nc, anchors] grid returns boxes covering the frame.
std::vector<Object> postprocessEndToEnd(const cv::Size& originalImageSize,
const cv::Size& resizedImageShape,
std::vector<Ort::Value>& outputTensors,
float confThreshold);
private:
template <typename T>
typename std::enable_if<std::is_arithmetic<T>::value, T>::type
+13 -2
View File
@@ -8,6 +8,7 @@
#include <climits> // INT_MIN
#include "ANSLicense.h" // ANS_DBG macro for DebugView
#include "ANSODVendorGate.h" // ansod_vendor_gate::IsNvidiaGpuAvailable()
#include "ANSModelOpen.h" // modelpkg::IsPackagePath (sealed .ansmodel)
// Process-wide flag: when true, all engines force single-GPU path (no pool, no idle timers).
// Defined here, declared extern in EngineBuildLoadNetwork.inl.
@@ -1576,6 +1577,13 @@ ANSODENGINE_API int OptimizeModelStr(const char* modelFilePath, const char* mode
modelType, modelDetectionType, fp16, modelFilePath ? modelFilePath : "(null)");
try {
bool optimizedResult = false;
// A sealed .ansmodel is already the artefact: there is no zip to
// inflate and no loose ONNX to point an optimiser at. It still goes
// through the handle below, which opens it in memory and builds the
// TensorRT plan from the decrypted bytes — what it must not go
// through is ModelOptimizer, which only knows how to unzip.
const bool sealedPackage =
ANSCENTER::modelpkg::IsPackagePath(modelFilePath ? modelFilePath : "");
// NOTE: odMutex was removed here. OptimizeModelStr creates its own
// temporary Engine<float> on the stack — no shared state with running
// inference tasks. The global mutex was blocking ALL running tasks'
@@ -1628,7 +1636,7 @@ ANSODENGINE_API int OptimizeModelStr(const char* modelFilePath, const char* mode
}
if (modelType == 31) // If modelType is RTYOLO (31), handle separately.
if (modelType == 31 && !sealedPackage) // If modelType is RTYOLO (31), handle separately.
{
if (engineType == ANSCENTER::EngineType::NVIDIA_GPU) {
if (modelDetectionType == 0) {
@@ -1778,7 +1786,10 @@ ANSODENGINE_API int OptimizeModelStr(const char* modelFilePath, const char* mode
Handle.reset(); // Destroy engines BEFORE guard restores cache
if (optimizedResult && !modelFolder.empty()) return 1;
// A package reports the folder its plan was cached in; when the
// handle declined to name one there is still nothing wrong, because
// nothing was ever meant to be written.
if (optimizedResult && (!modelFolder.empty() || sealedPackage)) return 1;
else return 0;
}
catch (const std::exception& e) {
+17
View File
@@ -188,6 +188,17 @@ public:
/// Full path of the engine file resolved by the last buildLoadNetwork().
[[nodiscard]] const std::string& engineFilePath() const { return m_engineFilePath; }
// ── ONNX bytes held in memory (sealed .ansmodel packages) ───────────────
// A package is decrypted into memory and the plaintext model must never
// reach the disk, so the parser reads these bytes instead of opening
// onnxModelPath. The path is still passed in and still matters: it names
// the TensorRT plan that gets cached beside it, and a plan is a built
// artefact, not the model. Set the bytes before any build call; they are
// wiped by clearOnnxBytes() and by the destructor.
void setOnnxBytes(const void* data, size_t size);
void clearOnnxBytes();
[[nodiscard]] bool hasOnnxBytes() const { return !m_onnxBytes.empty(); }
/// Return the CUDA device index that preprocessing should target.
/// In pool mode this returns the device of the first alive slot;
/// in single-engine mode it returns m_options.deviceIndex.
@@ -244,6 +255,9 @@ public:
// retries until build succeeds or all candidates are exhausted.
// Fixed-spatial models fall through to a single build() call.
bool buildWithRetry(std::string onnxModelPath, const std::array<float, 3>& subVals, const std::array<float, 3>& divVals, bool normalize);
// Where the ONNX bytes come from: the in-memory buffer when one was set,
// otherwise the file at onnxModelPath.
bool readOnnxBytes(const std::string& onnxModelPath, std::vector<char>& buffer) const;
void getDeviceNames(std::vector<std::string>& deviceNames);
void clearGpuBuffers();
int getBindingIndexByName(const std::string& name);
@@ -401,6 +415,7 @@ private:
bool m_usingCachedEngine = false;
bool m_skipEngineCache = false; // true = bypass TRTEngineCache (for OptimizeModelStr)
bool m_skipOnnxRebuild = false; // true = don't delete/rebuild corrupt engine files (elastic growth)
std::vector<char> m_onnxBytes; // model bytes in memory (package); empty = read from disk
bool m_rebuiltFromOnnx = false; // last buildLoadNetwork built from ONNX (no matching cached engine)
std::string m_engineFilePath; // engine file resolved by last buildLoadNetwork
bool m_skipOnnxBuild = false; // true = skip ONNX→TRT build if no cached engine (demand growth)
@@ -575,6 +590,8 @@ template <typename T> Engine<T>::Engine() {
m_lastSetInputDims.clear();
}
template <typename T> Engine<T>::~Engine() {
// A decrypted model must not outlive the engine that parsed it.
try { clearOnnxBytes(); } catch (...) {}
// =========================================================================
// CRITICAL: Destructors must NEVER throw. Every CUDA/TRT call below is
// wrapped in try/catch so that a failure in one step does not skip the
+1 -1
View File
@@ -33,7 +33,7 @@ namespace ANSCENTER
{1005, "ANSOCR"},
{1006, "ANSALPR"},
{1007, "ANSCV"},
{1008, "ANSSRT"}
{1008, "ANSTS"}
};
ansawss3LicenceValid = false;
for (const auto& [productId, productName] : licenseChecks) {
+1 -1
View File
@@ -40,7 +40,7 @@ namespace ANSCENTER
{1005, "ANSOCR"},
{1006, "ANSALPR"},
{1007, "ANSCV"},
{1008, "ANSSRT"}
{1008, "ANSTS"}
};
ansutilityLicenceValid = false;
for (const auto& [productId, productName] : licenseChecks) {
@@ -162,7 +162,6 @@ static void testRoundTripAndRefusals(const fs::path& dir) {
req.useX25519 = true;
req.basePublic = id.basePub;
req.chunk = 2048;
req.nonceSeed = 9;
req.signerId = "anssig-2026-09";
req.signerSecret = id.signerSk;
req.signerPublic = id.signerPk;
@@ -252,6 +251,79 @@ static void testRoundTripAndRefusals(const fs::path& dir) {
// ANSODBase asks these three questions and nothing else: is this path a
// package, can this build open one, and what came out of it. The engines
// themselves need CUDA to compile, so this is where that contract is checked.
// ── every package gets its own key ────────────────────────────────────────
// pack() used to derive the content key, the ephemeral key and the nonces from
// one byte of seed, and the training box passed a seed made of the project's
// name and version: anyone holding a package could try the 256 seeds and read
// it without the base key. Two packs of the very same inputs must now share
// nothing that seals them — and still open with the base key.
static void testEachPackIsSealedAfresh(const fs::path& dir) {
const Identity id = fixtureIdentity();
const modelpkg::Keyring ring = fixtureKeyring(id);
modelpkg::PackRequest req;
req.keyId = "ansk-v1-base";
req.useX25519 = true;
req.basePublic = id.basePub;
req.chunk = 2048;
req.manifest = Json{{"task", "detection"},
{"family", "yolo"},
{"runtime", "onnx"},
{"models", Json::array({Json{{"id", "m0"},
{"payload", "p0"},
{"entry", true}}})}};
const auto model = fakeModel(5000, 11);
req.models.push_back({"p0", model});
req.models.push_back({"p1", model}); // two parts under one key: bases differ
const std::string pa = (dir / "fresh-a.ansmodel").string();
const std::string pb = (dir / "fresh-b.ansmodel").string();
std::string err;
CHECK(modelpkg::pack(pa, req, err), ("pack a: " + err).c_str());
CHECK(modelpkg::pack(pb, req, err), ("pack b: " + err).c_str());
modelpkg::Package a, b;
CHECK(modelpkg::open(pa, ring, a, err) && modelpkg::open(pb, ring, b, err),
("both open with the base key: " + err).c_str());
CHECK(a.contentKey != b.contentKey,
"two packs of identical inputs have different content keys");
bool seeded = false;
for (int seed = 0; seed < 256; ++seed) {
modelpkg::Key guess{};
for (size_t i = 0; i < guess.size(); ++i)
guess[i] = uint8_t(seed * 31 + i * 7 + 11);
seeded = seeded || guess == a.contentKey || guess == b.contentKey;
}
CHECK(!seeded, "and neither is one of the 256 a seed could have made");
const Json& wa = a.header["wraps"][0];
const Json& wb = b.header["wraps"][0];
CHECK(wa.value("epk", std::string()) != wb.value("epk", std::string()),
"each has its own ephemeral key");
CHECK(wa.value("nonce", std::string()) != wb.value("nonce", std::string()) &&
wa.value("ct", std::string()) != wb.value("ct", std::string()),
"and its own wrap nonce and wrapped key");
CHECK(a.header["payloads"][0].value("sha256", std::string()) !=
b.header["payloads"][0].value("sha256", std::string()),
"so the same model seals to different ciphertext");
CHECK(a.header["payloads"][0].value("nonceBase", std::string()) !=
a.header["payloads"][1].value("nonceBase", std::string()) &&
a.header["payloads"][0].value("nonceBase", std::string()) !=
a.header["manifest"].value("nonceBase", std::string()),
"and no two parts of one package share a nonce base");
std::vector<uint8_t> ga, gb;
CHECK(modelpkg::readPayload(a, "p0", ga, err) && ga == model &&
modelpkg::readPayload(b, "p1", gb, err) && gb == model,
"and both still give back the model that went in");
// The seeded path is for tests built as such; this one is not.
req.insecureDeterministicForTests = true;
CHECK(!modelpkg::pack((dir / "seeded.ansmodel").string(), req, err) &&
err.find("tests only") != std::string::npos,
"a build without the test switch refuses to pack from a seed");
}
static void testOpenForAnEngine(const fs::path& dataDir) {
const std::string path = (dataDir / "interop.ansmodel").string();
if (!fs::exists(path)) {
@@ -328,12 +400,15 @@ static std::string pbDim(int64_t v) { // Dimension.dim_value
static std::string pbDynamicDim(const std::string& name) {
return pbField(1, pbField(2, name)); // dim_param: the axis is open
}
static std::string fakeOnnx(const std::string& dims) {
const std::string tensor = pbField(1, pbField(2, dims));
const std::string input = pbField(1, "images") + pbField(2, tensor);
const std::string graph = pbField(1, std::string(4096, 'n')) +
pbField(5, std::string(65536, 'w')) +
pbField(11, input);
static std::string fakeOnnx(const std::string& dims,
const std::string& outDims = std::string()) {
auto value = [](const std::string& name, const std::string& d) {
return pbField(1, name) + pbField(2, pbField(1, pbField(2, d)));
};
std::string graph = pbField(1, std::string(4096, 'n')) +
pbField(5, std::string(65536, 'w')) +
pbField(11, value("images", dims));
if (!outDims.empty()) graph += pbField(12, value("output0", outDims));
return pbField(7, graph);
}
@@ -373,7 +448,6 @@ static void testShapeFromTheGraph(const fs::path& dir) {
req.keyId = "ansk-v1-base";
req.useX25519 = true;
req.basePublic = id.basePub;
req.nonceSeed = 21;
req.signerId = "anssig-2026-09";
req.signerSecret = id.signerSk;
req.signerPublic = id.signerPk;
@@ -405,6 +479,57 @@ static void testShapeFromTheGraph(const fs::path& dir) {
CHECK(modelpkg::OpenModelPackage(path, opened, err), ("open: " + err).c_str());
CHECK(opened.inputWidth == 224 && opened.inputHeight == 224,
"so the engine is told 224 from the graph, not left to assume 640");
// AN END-TO-END HEAD IN A PACKAGE THAT SAYS "yolo". Boxes used to take the
// family from the backend marker, and the marker names the runtime, not the
// head — so a yolo26 export with end2end on arrived declared as plain yolo.
// Decoding [1,N,6] as a v8 grid reads coordinates as confidences: boxes
// everywhere, and no error to notice. The graph settles it.
{
const std::string graph =
fakeOnnx(pbDim(1) + pbDim(3) + pbDim(640) + pbDim(640),
pbDim(1) + pbDim(300) + pbDim(6));
modelpkg::PackRequest r = req;
r.manifest = Json{{"task", "detection"},
{"family", "yolo"},
{"classes", Json::array({Json{{"id", 1}, {"name", "Tooth"}}})},
{"models", Json::array({Json{{"id", "m0"},
{"payload", "p0"},
{"entry", true}}})}};
r.models.clear();
r.models.push_back({"p0", std::vector<uint8_t>(graph.begin(), graph.end())});
const std::string p = (dir / "e2e.ansmodel").string();
CHECK(modelpkg::pack(p, r, err), ("pack e2e: " + err).c_str());
modelpkg::OpenedModel m;
CHECK(modelpkg::OpenModelPackage(p, m, err) && m.family == "yolo10",
"an end-to-end head is read as yolo10 whatever the package says");
CHECK(m.warning.find("end-to-end") != std::string::npos,
"and the reason is said out loud rather than silently swapped");
CHECK(m.legacyMarker.empty() || m.legacyMarker == "tensorrt.txt",
"while the legacy marker is left as it was");
}
// A two-class v8 export is [1, 6, 8400]: a 6 in the shape, not a head that
// has already decoded itself.
{
const std::string graph =
fakeOnnx(pbDim(1) + pbDim(3) + pbDim(640) + pbDim(640),
pbDim(1) + pbDim(6) + pbDim(8400));
modelpkg::PackRequest r = req;
r.manifest = Json{{"task", "detection"},
{"family", "yolo"},
{"classes", Json::array({Json{{"id", 1}, {"name", "Tooth"}}})},
{"models", Json::array({Json{{"id", "m0"},
{"payload", "p0"},
{"entry", true}}})}};
r.models.clear();
r.models.push_back({"p0", std::vector<uint8_t>(graph.begin(), graph.end())});
const std::string p = (dir / "twoclass.ansmodel").string();
CHECK(modelpkg::pack(p, r, err), ("pack v8: " + err).c_str());
modelpkg::OpenedModel m;
CHECK(modelpkg::OpenModelPackage(p, m, err) && m.family == "yolo",
"a 6 on the wrong axis with thousands of anchors stays plain yolo");
}
}
int main(int argc, char** argv) {
@@ -418,6 +543,7 @@ int main(int argc, char** argv) {
testVectors();
testInterop(dataDir);
testRoundTripAndRefusals(tmp);
testEachPackIsSealedAfresh(tmp);
testOpenForAnEngine(dataDir);
testShapeFromTheGraph(tmp);
@@ -2,6 +2,8 @@
#include "ANSCUSTOMPY.h"
#include "ANSMotionDetector.h"
#include "ANSONNXSAM3.h"
#include <cstdlib>
#include <iostream>
#include <thread>
#include <atomic>
#include <chrono>
@@ -2235,8 +2237,31 @@ int FireNSmokeTest() {
return 0;
}
int main()
// RUN A MODEL THIS MACHINE ACTUALLY HAS. Every test below names a path from
// whichever workstation it was written on, so on any other machine the harness
// stops at "Model zip file is not exist" before it ever reaches the GPU. Given
// arguments it runs what it is told instead:
//
// ANSODEngine-UnitTest.exe <model> <video> [engineType] [detectionType]
//
// `model` is a .zip or a sealed .ansmodel — the engine opens either, and a
// package is the only way to see a sealed model run on the GPU. engineType is
// the ModelType enum (3 auto-detects a YOLO export, 30 ONNX YOLO, 16 custom,
// 32 RTMPose); detectionType 1 is object detection.
//
// With no arguments it does exactly what it did before.
int main(int argc, char** argv)
{
if (argc >= 3) {
const std::string model = argv[1];
const std::string video = argv[2];
const int engineType = (argc > 3) ? std::atoi(argv[3]) : 3;
const int detectionType = (argc > 4) ? std::atoi(argv[4]) : 1;
std::cout << "model: " << model << "\nvideo: " << video
<< "\nengineType: " << engineType
<< ", detectionType: " << detectionType << std::endl;
return VideoDetectorEngine_CPP(model, video, engineType, detectionType);
}
//FallDetectorTest();
//ALPROD_Test();
+29 -7
View File
@@ -1,13 +1,35 @@
# ANSCORE Unit Tests
add_subdirectory(ANSCV-UnitTest)
add_subdirectory(ANSODEngine-UnitTest)
add_subdirectory(ANSFR-UnitTest)
add_subdirectory(ANSLPR-UnitTest)
add_subdirectory(ANSOCR-UnitTest)
#
# A test is gated exactly like the code it exercises (see the top-level
# CMakeLists): tests for the CUDA-only modules, or that name CUDA::* targets
# themselves, are added only when ANSCORE_HAS_CUDA; tests for the Windows-only
# integrations only on WIN32. Without these gates a non-CUDA configure (macOS,
# any machine with no toolkit) fails at generate time on "links to CUDA::cudart
# but the target was not found", which blocks the cross-platform targets too.
# Cross-platform
add_subdirectory(ANSUtilities-UnitTest)
add_subdirectory(ANSRabbitMQ-UnitTest)
add_subdirectory(ANSIO-UnitTest)
add_subdirectory(ANSLS-UnitTest)
add_subdirectory(ANSModelPackage-UnitTest)
add_subdirectory(ANSONVIF-UnitTest)
add_subdirectory(ANSODTRE-UnitTest)
# Windows-only — integrations/ANSIO is built on WIN32 only
if(WIN32)
add_subdirectory(ANSIO-UnitTest)
endif()
# CUDA-only — modules/ANSCV, ANSODEngine, ANSOCR, ANSFR and ANSLPR
if(ANSCORE_HAS_CUDA)
add_subdirectory(ANSCV-UnitTest)
add_subdirectory(ANSODEngine-UnitTest)
add_subdirectory(ANSFR-UnitTest)
add_subdirectory(ANSLPR-UnitTest)
add_subdirectory(ANSOCR-UnitTest)
# integrations/ANSONVIF itself uses no CUDA — the test does, because it
# drives capture through ANSCV. Hence both gates.
if(WIN32)
add_subdirectory(ANSONVIF-UnitTest)
endif()
endif()