Commit Graph
19 Commits
Author SHA1 Message Date
nghianguyenandClaude Fable 5 7909c8db2b RTP: resume at access-unit boundary after packet loss, not NAL start
After a sequence gap the H.264/H.265 depacketizers resumed reassembly at
the next NAL/FU start. On multi-slice streams every slice starts a NAL,
so reassembly resumed MID-FRAME and delivered access units missing their
leading slices — manufacturing "First slice in a frame missing" /
"Ignoring POC change between slices" / "Could not find ref" decoder
errors and green/blocky concealment output far beyond the actual loss.

Resume only at a true AU boundary instead:
  H.265 - non-VCL NAL (type >= 32) or VCL slice with
          first_slice_segment_in_pic_flag set (first bit after the
          2-byte NAL header)
  H.264 - non-VCL NAL (type >= 6) or VCL slice with
          first_mb_in_slice == 0 (ue(v): MSB set after 1-byte header)

Also: stderr visibility for RTP sequence gaps (rtp_rx logs went only to
the sys_log file) and for av_log-bridge latches in video_decoder — the
global av_log level filters decoder errors from the console, which made
keyframe-gate arms look causeless during diagnosis.

Live verification, HEVC camera on lossy Wi-Fi (NVDEC decode):
  before: 271 decoder errors / 3 min, delivery collapsed to ~2 fps
  after:  3 errors / 2 min, full ~21 fps, zero corrupt frames displayed
Remaining rare errors show POC ref misses with ZERO RTP sequence gaps,
i.e. pre-decode queue eviction under TCP burst clumps - a separate,
much smaller issue absorbed invisibly by the keyframe gate.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 06:09:37 +10:00
nghianguyenandClaude Fable 5 b341392e0f video_decoder: console visibility for keyframe gate arm/release
log_print writes to the sys_log file (unopened in unit tests / field
debugging), so the gate looked inert next to FFmpeg's stderr error lines.
Emit arm/release one-liners to stderr alongside them.

Live verification during a severe Wi-Fi loss burst (271 HEVC errors /
3 min, NVDEC path): gate armed 19x, released 17x, dropping 29-118
corrupt frames per burst and resuming on the next clean keyframe.
The av_log bridge (273dd44) is confirmed working end-to-end.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 05:40:34 +10:00
nghianguyenandClaude Fable 5 273dd44a6d video_decoder: av_log bridge — corruption detection for hwaccel decode
The corruption re-arm added in 6344062 keyed off AVFrame::decode_error_flags,
but the HEVC decoder on the hwaccel (NVDEC) path never sets it: 120 s of live
Wi-Fi packet loss produced 10+ "First slice in a frame missing" / "Could not
find ref with POC n" log bursts with decode_error_flags == 0 on every frame,
so the keyframe gate never fired.

Bridge: install a process-wide av_log callback (chains to
av_log_default_callback) that maps the logging AVCodecContext back to its
CVideoDecoder and latches a per-decoder m_bCorruptSeen flag on AV_LOG_ERROR
messages. readFrame() consumes the latch alongside decode_error_flags to
re-arm the keyframe gate. With hwaccel the decoder is single-threaded, so
the logging context is exactly our AVCodecContext; SW frame-threaded paths
log from per-thread clones but set decode_error_flags instead, which the
existing check covers.

Registered after avcodec_open2, unregistered in uninit() before the context
is freed. Map lookup only on error-level messages (rare). Also promote the
gate-arm message to log_print(HT_LOG_WARN) for console visibility.

Clean-path verified live: 6105 frames / 5 min NVDEC RTSP with zero errors
and no regressions. Loss-path verification pending a naturally lossy window.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 05:01:22 +10:00
nghianguyenandClaude Fable 5 6344062179 MediaClient: keyframe gating (startup + corruption re-arm), libyuv NV12 path
Restores the previously shelved decoder/player hardening and extends it:

1. Startup keyframe gating (video_decoder): after (re)start the decoder
   joins the stream mid-GOP; frames referencing unseen pictures render as
   green/blocky concealment garbage. Drop everything until the first IDR
   (300-frame safety valve so unmarked streams can't blank forever).

2. NEW - corruption re-arm: on lossy links (Wi-Fi cameras) dropped slices
   surface as "First slice in a frame missing" / "Could not find ref with
   POC n" and FFmpeg marks the output via AVFrame::decode_error_flags.
   Re-arm the keyframe gate and drop the rest of the GOP so the display
   holds the last clean frame instead of showing concealment artifacts;
   corrupt drops count toward the safety valve.

3. libyuv NV12->BGR conversion (video_player): honors NVDEC/D3D11VA padded
   strides (1080->1088 rows) that cv::cvtColorTwoPlane mishandled (green
   cast + striped noise), picks BT.601/709/2020 coefficients from the
   frame's signalled colorspace, and is ~3-5x faster than sws_scale.

Verified live against an RTSP HEVC camera (NVDEC path): steady frame flow,
no regressions on the clean-stream path.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 21:49:22 +10:00
nghianguyen aa323a07a3 Fix debug 2026-05-07 23:35:18 +10:00
nghianguyen 75693cadd9 Add jitter buffer 2026-05-07 22:36:27 +10:00
nghianguyen 2e0fee5e3d Fix VideoPlayer from stop after just few frames 2026-05-07 18:17:33 +10:00
nghianguyen 91efbb9a8f Fix AMD ALPR_OCR and custom model 2026-05-01 07:53:18 +10:00
nghianguyen f068eb716f Add more yuv conversion 2026-04-22 18:25:38 +10:00
nghianguyen fc47a42c6a Use yuv and fix FR to use batch 4 2026-04-22 16:01:44 +10:00
nghianguyen 57cc8e0a56 Fix double stop in ANSVideoPlayer 2026-04-22 10:10:16 +10:00
nghianguyen 97d814936d Enable log information. Disable NPU in U9 2026-04-21 15:48:27 +10:00
nghianguyen 7e772f76bc Improve ANSCV 2026-04-21 09:26:02 +10:00
nghianguyen 9f0a10a4c8 Improve ANSCV with sotfware decoder:
Thread-local staging Mat (video_player.cpp:1400-1407) — single biggest win. Eliminates the 12 MB per-call malloc/free cycle.
Contiguous get_buffer2 allocator (video_decoder.cpp:35-102) — keeps the 3 bulk memcpys cache-friendly. Would also enable FAST/zero-copy for resolutions where visible_h % 64 == 0.
SW-decoder thread config (video_decoder.cpp:528-540) — thread_count=0, thread_type=FRAME|SLICE. FRAME is downgraded to SLICE-only by AV_CODEC_FLAG_LOW_DELAY, but decode throughput is sufficient for your input rate.
SetTargetFPS(100) delivery throttle (already there) — caps onVideoFrame post-decode work at 10 FPS. Keeps the caller path warm-cached.
Instrumentation — [MEDIA_DecInit] / [MEDIA_Convert] / [MEDIA_SWDec] / [MEDIA_Timing] / [MEDIA_JpegTiming] — always-on regression detector, zero cost when ANSCORE_DEBUGVIEW=OFF.
2026-04-20 12:18:43 +10:00
nghianguyenandClaude Opus 4.7 0925aa1d63 Fix RTSP stall by releasing _mutex during BGR conversion
getImage() previously held _mutex across the 4K NV12->BGR sws_scale in
avframeToCVMat, blocking the decoder callback (onVideoFrame) for 100-300ms
per frame. Under multi-camera load this cascaded into 5-21s frame stalls
and STALE PTS events in the log.

- avframeToCVMat: drop outer _mutex. NV12/YUV420P paths touch no shared
  state; avframeAnyToCvmat still locks internally for swsCtx.
- getImage: split into two short locked phases with the BGR conversion
  unlocked between them. Decoder callbacks can push new frames and run
  the CUDA HW capture path in parallel with the reader's conversion.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-20 08:23:28 +10:00
nghianguyen e134ebdf15 Use software decoder by default 2026-04-04 20:19:54 +11:00
nghianguyen 3a21026790 Disable NV12 path for ANSCV by default. Currenly use cv::Mat** directly 2026-04-04 10:09:47 +11:00
nghianguyen 958cab6ae3 Fix NV12 crash issue when recreate camera object 2026-04-02 22:07:27 +11:00
nghianguyen 7b4134133c Initial setup for CLion 2026-03-28 16:54:11 +11:00