MediaClient: keyframe gating (startup + corruption re-arm), libyuv NV12 path
Restores the previously shelved decoder/player hardening and extends it: 1. Startup keyframe gating (video_decoder): after (re)start the decoder joins the stream mid-GOP; frames referencing unseen pictures render as green/blocky concealment garbage. Drop everything until the first IDR (300-frame safety valve so unmarked streams can't blank forever). 2. NEW - corruption re-arm: on lossy links (Wi-Fi cameras) dropped slices surface as "First slice in a frame missing" / "Could not find ref with POC n" and FFmpeg marks the output via AVFrame::decode_error_flags. Re-arm the keyframe gate and drop the rest of the GOP so the display holds the last clean frame instead of showing concealment artifacts; corrupt drops count toward the safety valve. 3. libyuv NV12->BGR conversion (video_player): honors NVDEC/D3D11VA padded strides (1080->1088 rows) that cv::cvtColorTwoPlane mishandled (green cast + striped noise), picks BT.601/709/2020 coefficients from the frame's signalled colorspace, and is ~3-5x faster than sws_scale. Verified live against an RTSP HEVC camera (NVDEC path): steady frame flow, no regressions on the clean-stream path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -761,6 +761,10 @@ void CVideoDecoder::Start() {
|
||||
if (m_pContext) {
|
||||
avcodec_flush_buffers(m_pContext);
|
||||
}
|
||||
// Re-arm keyframe gating: after a flush the DPB is empty, so wait for the
|
||||
// next IDR before delivering frames again (avoids post-restart green/blocky).
|
||||
m_bWaitKeyframe = true;
|
||||
m_nKeyframeWaitCount = 0;
|
||||
m_bRunning = TRUE;
|
||||
log_print(HT_LOG_INFO, "%s, Video decoder started\r\n", __FUNCTION__);
|
||||
}
|
||||
@@ -1093,6 +1097,78 @@ BOOL CVideoDecoder::readFrame()
|
||||
continue;
|
||||
}
|
||||
|
||||
// Mid-stream corruption gating — re-arm the keyframe gate on decode errors.
|
||||
//
|
||||
// Packet loss on lossy links (Wi-Fi cameras) drops slices mid-GOP:
|
||||
// FFmpeg logs "First slice in a frame missing" / "Could not find ref
|
||||
// with POC n", conceals the damage, and marks the output frame via
|
||||
// AVFrame::decode_error_flags (missing refs, concealment active, ...).
|
||||
// Every subsequent frame in the GOP references the concealed one, so
|
||||
// rather than display green/blocky garbage until the next IDR, drop
|
||||
// the frame and re-arm the keyframe gate — the display holds the last
|
||||
// clean frame and resumes on the next clean keyframe.
|
||||
if (m_pFrame->decode_error_flags != 0) {
|
||||
if (!m_bWaitKeyframe.load(std::memory_order_relaxed)) {
|
||||
m_bWaitKeyframe.store(true, std::memory_order_relaxed);
|
||||
m_nKeyframeWaitCount = 0;
|
||||
ANS_DBG("MEDIA_Decode",
|
||||
"corrupt frame (decode_error_flags=0x%x) — re-arming keyframe gate",
|
||||
m_pFrame->decode_error_flags);
|
||||
} else {
|
||||
// Count corrupt drops toward the safety valve so a stream that
|
||||
// is corrupt for a long stretch eventually shows *something*
|
||||
// (concealed frames) instead of freezing indefinitely.
|
||||
++m_nKeyframeWaitCount;
|
||||
}
|
||||
if (m_nKeyframeWaitCount < 300) {
|
||||
av_frame_unref(m_pFrame);
|
||||
continue;
|
||||
}
|
||||
// Safety valve tripped: fall through and let the frame pass
|
||||
// (concealment artifacts beat a permanently frozen display).
|
||||
m_bWaitKeyframe.store(false, std::memory_order_relaxed);
|
||||
}
|
||||
|
||||
// Startup keyframe gating — discard everything until the first IDR.
|
||||
//
|
||||
// On (re)connect we join the stream mid-GOP. Until a keyframe arrives,
|
||||
// every frame references pictures we never received; the decoder (and
|
||||
// NVDEC especially) conceals them into green / blocky garbage and reports
|
||||
// success, so there is no error flag to test. Waiting for the first
|
||||
// keyframe gives a clean intra reference, after which inter frames decode
|
||||
// correctly. This is the standard RTSP "wait for IDR" behaviour.
|
||||
//
|
||||
// Keyframe flag is set from the bitstream parse and is reliable for both
|
||||
// SW and HW decode (it does not depend on the HW surface). pict_type is a
|
||||
// fallback. The counter is a safety valve: if a stream never advertises a
|
||||
// keyframe within ~300 frames (~10 s @30fps) we stop gating rather than
|
||||
// blank the display forever.
|
||||
if (m_bWaitKeyframe.load(std::memory_order_relaxed)) {
|
||||
// pict_type==I is version-independent and set for HEVC IDR/I frames
|
||||
// on both SW and HW decode. AV_FRAME_FLAG_KEY (FFmpeg 5.1+) is added
|
||||
// when available for robustness; guarded so it also builds against
|
||||
// the older vendored headers that predate that flag.
|
||||
bool isKey = (m_pFrame->pict_type == AV_PICTURE_TYPE_I);
|
||||
#if defined(AV_FRAME_FLAG_KEY)
|
||||
isKey = isKey || ((m_pFrame->flags & AV_FRAME_FLAG_KEY) != 0);
|
||||
#endif
|
||||
if (isKey) {
|
||||
m_bWaitKeyframe.store(false, std::memory_order_relaxed);
|
||||
ANS_DBG("MEDIA_Decode",
|
||||
"keyframe acquired after %d dropped frame(s) — resuming output",
|
||||
m_nKeyframeWaitCount);
|
||||
} else if (++m_nKeyframeWaitCount >= 300) {
|
||||
m_bWaitKeyframe.store(false, std::memory_order_relaxed);
|
||||
ANS_DBG("MEDIA_Decode",
|
||||
"no keyframe after %d frames — releasing gate (safety valve)",
|
||||
m_nKeyframeWaitCount);
|
||||
} else {
|
||||
// Pre-IDR frame: drop it (would render as green/blocky garbage).
|
||||
av_frame_unref(m_pFrame);
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
if (m_pFrame->format == m_hwPixFmt)
|
||||
{
|
||||
// CUDA HW accel: clone the HW frame BEFORE transfer so inference
|
||||
|
||||
Reference in New Issue
Block a user