har0ke/FFmpeg
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

avcodec/vp9: Fix stack-buffer overflow with VP9 VDPAU available

Browse Source 
ccca62ef991f0a47dfa30c3e822d91294b8afe4c added new VP9 VDPAU profiles
and as a consequence AV_PIX_FMT_VDPAU can now be twice in the list of
pixel formats used for format negotiation by ff_thread_get_format(); yet
there is only one entry in said list reserved for VDPAU, leading to a
stack-buffer overflow. This commit fixes this by making sure that
AV_PIX_FMT_VDPAU will not occur twice in said list.

Fixes Coverity ticket 1468046.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
This commit is contained in:
Andreas Rheinhardt 2020-10-11 01:29:57 +02:00
parent 8227f60fee
commit ddee53e268
1 changed files with 0 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
libavcodec/vp9.c
View File

@ -207,9 +207,6 @@ static int update_size(AVCodecContext *avctx, int w, int h)
switch (s->pix_fmt) { switch (s->pix_fmt) {
case AV_PIX_FMT_YUV420P: case AV_PIX_FMT_YUV420P:
#if CONFIG_VP9_VDPAU_HWACCEL
*fmtp++ = AV_PIX_FMT_VDPAU;
#endif
case AV_PIX_FMT_YUV420P10: case AV_PIX_FMT_YUV420P10:
#if CONFIG_VP9_DXVA2_HWACCEL #if CONFIG_VP9_DXVA2_HWACCEL
*fmtp++ = AV_PIX_FMT_DXVA2_VLD; *fmtp++ = AV_PIX_FMT_DXVA2_VLD;