MediaClient: keyframe gating (startup + corruption re-arm), libyuv NV12 path

Restores the previously shelved decoder/player hardening and extends it:

1. Startup keyframe gating (video_decoder): after (re)start the decoder
   joins the stream mid-GOP; frames referencing unseen pictures render as
   green/blocky concealment garbage. Drop everything until the first IDR
   (300-frame safety valve so unmarked streams can't blank forever).

2. NEW - corruption re-arm: on lossy links (Wi-Fi cameras) dropped slices
   surface as "First slice in a frame missing" / "Could not find ref with
   POC n" and FFmpeg marks the output via AVFrame::decode_error_flags.
   Re-arm the keyframe gate and drop the rest of the GOP so the display
   holds the last clean frame instead of showing concealment artifacts;
   corrupt drops count toward the safety valve.

3. libyuv NV12->BGR conversion (video_player): honors NVDEC/D3D11VA padded
   strides (1080->1088 rows) that cv::cvtColorTwoPlane mishandled (green
   cast + striped noise), picks BT.601/709/2020 coefficients from the
   frame's signalled colorspace, and is ~3-5x faster than sws_scale.

Verified live against an RTSP HEVC camera (NVDEC path): steady frame flow,
no regressions on the clean-stream path.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-06 21:49:22 +10:00
co-authored by Claude Fable 5
parent 044fedd344
commit 6344062179
4 changed files with 173 additions and 3 deletions
+76
View File
@@ -761,6 +761,10 @@ void CVideoDecoder::Start() {
if (m_pContext) {
avcodec_flush_buffers(m_pContext);
}
// Re-arm keyframe gating: after a flush the DPB is empty, so wait for the
// next IDR before delivering frames again (avoids post-restart green/blocky).
m_bWaitKeyframe = true;
m_nKeyframeWaitCount = 0;
m_bRunning = TRUE;
log_print(HT_LOG_INFO, "%s, Video decoder started\r\n", __FUNCTION__);
}
@@ -1093,6 +1097,78 @@ BOOL CVideoDecoder::readFrame()
continue;
}
// Mid-stream corruption gating — re-arm the keyframe gate on decode errors.
//
// Packet loss on lossy links (Wi-Fi cameras) drops slices mid-GOP:
// FFmpeg logs "First slice in a frame missing" / "Could not find ref
// with POC n", conceals the damage, and marks the output frame via
// AVFrame::decode_error_flags (missing refs, concealment active, ...).
// Every subsequent frame in the GOP references the concealed one, so
// rather than display green/blocky garbage until the next IDR, drop
// the frame and re-arm the keyframe gate — the display holds the last
// clean frame and resumes on the next clean keyframe.
if (m_pFrame->decode_error_flags != 0) {
if (!m_bWaitKeyframe.load(std::memory_order_relaxed)) {
m_bWaitKeyframe.store(true, std::memory_order_relaxed);
m_nKeyframeWaitCount = 0;
ANS_DBG("MEDIA_Decode",
"corrupt frame (decode_error_flags=0x%x) — re-arming keyframe gate",
m_pFrame->decode_error_flags);
} else {
// Count corrupt drops toward the safety valve so a stream that
// is corrupt for a long stretch eventually shows *something*
// (concealed frames) instead of freezing indefinitely.
++m_nKeyframeWaitCount;
}
if (m_nKeyframeWaitCount < 300) {
av_frame_unref(m_pFrame);
continue;
}
// Safety valve tripped: fall through and let the frame pass
// (concealment artifacts beat a permanently frozen display).
m_bWaitKeyframe.store(false, std::memory_order_relaxed);
}
// Startup keyframe gating — discard everything until the first IDR.
//
// On (re)connect we join the stream mid-GOP. Until a keyframe arrives,
// every frame references pictures we never received; the decoder (and
// NVDEC especially) conceals them into green / blocky garbage and reports
// success, so there is no error flag to test. Waiting for the first
// keyframe gives a clean intra reference, after which inter frames decode
// correctly. This is the standard RTSP "wait for IDR" behaviour.
//
// Keyframe flag is set from the bitstream parse and is reliable for both
// SW and HW decode (it does not depend on the HW surface). pict_type is a
// fallback. The counter is a safety valve: if a stream never advertises a
// keyframe within ~300 frames (~10 s @30fps) we stop gating rather than
// blank the display forever.
if (m_bWaitKeyframe.load(std::memory_order_relaxed)) {
// pict_type==I is version-independent and set for HEVC IDR/I frames
// on both SW and HW decode. AV_FRAME_FLAG_KEY (FFmpeg 5.1+) is added
// when available for robustness; guarded so it also builds against
// the older vendored headers that predate that flag.
bool isKey = (m_pFrame->pict_type == AV_PICTURE_TYPE_I);
#if defined(AV_FRAME_FLAG_KEY)
isKey = isKey || ((m_pFrame->flags & AV_FRAME_FLAG_KEY) != 0);
#endif
if (isKey) {
m_bWaitKeyframe.store(false, std::memory_order_relaxed);
ANS_DBG("MEDIA_Decode",
"keyframe acquired after %d dropped frame(s) — resuming output",
m_nKeyframeWaitCount);
} else if (++m_nKeyframeWaitCount >= 300) {
m_bWaitKeyframe.store(false, std::memory_order_relaxed);
ANS_DBG("MEDIA_Decode",
"no keyframe after %d frames — releasing gate (safety valve)",
m_nKeyframeWaitCount);
} else {
// Pre-IDR frame: drop it (would render as green/blocky garbage).
av_frame_unref(m_pFrame);
continue;
}
}
if (m_pFrame->format == m_hwPixFmt)
{
// CUDA HW accel: clone the HW frame BEFORE transfer so inference