ANSFR: purge stale ServerOptimised zip when engine rebuilds from ONNX
A ServerOptimised zip is a per-GPU cache of a previous engine build. When the engine profile changes (batch gate, TRT/CUDA upgrade), the cached engine inside it goes stale: every re-extract (e.g. after Temp cleanup) restores the old engine and costs a full ONNX rebuild, forever, because the server only regenerates the zip when the file is missing. Engine now records rebuiltFromOnnx() + engineFilePath() in buildLoadNetwork (set only on the generate-from-ONNX branch; cache hits and alternative-precision loads don't raise it). After a rebuild, ANSFaceRecognizer::Initialize deletes the stale same-GPU engine variants from the model folder and the supplied optimised zip, so the server's next optimize pass recreates it with the fresh engine. Safety: only zips whose parent folder is ServerOptimised/-ized qualify — base model zips can never match. Engine purge is scoped to the "<model>.engine.<GPU>." prefix, leaving other GPUs' engines intact on multi-GPU machines. All deletes are best-effort (error_code); multi-GPU pool paths never set the flag, so the purge is a no-op there. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -265,6 +265,8 @@ bool Engine<T>::buildLoadNetwork(std::string onnxModelPath, const std::array<flo
|
||||
// It is full path
|
||||
std::string engineName = serializeEngineOptions(m_options, onnxModelPath);
|
||||
std::string engineDir = m_options.engineFileDir;
|
||||
m_engineFilePath = engineName;
|
||||
m_rebuiltFromOnnx = false;
|
||||
|
||||
if (FileExist(engineName)) {
|
||||
if (m_verbose) { std::cout << "Engine file found: " << engineName << std::endl; }
|
||||
@@ -335,6 +337,7 @@ bool Engine<T>::buildLoadNetwork(std::string onnxModelPath, const std::array<flo
|
||||
|
||||
if (FileExist(alternativeEngineName)) {
|
||||
std::cout << "Found alternative precision engine: " << alternativeEngineName << std::endl;
|
||||
m_engineFilePath = alternativeEngineName;
|
||||
return loadNetwork(alternativeEngineName, subVals, divVals, normalize);
|
||||
}
|
||||
else {
|
||||
@@ -358,11 +361,13 @@ bool Engine<T>::buildLoadNetwork(std::string onnxModelPath, const std::array<flo
|
||||
std::cout << "FP16 engine not found, but FP32 engine exists: " << fp32EngineName << std::endl;
|
||||
std::cout << "Loading existing FP32 engine..." << std::endl;
|
||||
m_options.precision = ANSCENTER::Precision::FP32;
|
||||
m_engineFilePath = fp32EngineName;
|
||||
return loadNetwork(fp32EngineName, subVals, divVals, normalize);
|
||||
}
|
||||
}
|
||||
|
||||
// ONNX model exists, generate engine
|
||||
m_rebuiltFromOnnx = true;
|
||||
std::cout << "========================================" << std::endl;
|
||||
std::cout << "Engine not found, generating from ONNX model" << std::endl;
|
||||
std::cout << "========================================" << std::endl;
|
||||
|
||||
Reference in New Issue
Block a user