Fixes: signed integer overflow: 1024 * 13129048 cannot be represented in type 'int'
Fixes: 26378/clusterfuzz-testcase-minimized-ffmpeg_dem_CODEC2RAW_fuzzer-5634018353348608
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Allocating an AVCodecContext's priv_data used to be the first object
allocated in avcodec_open2(), so it was unnecessary to goto free_and_end
(which does the cleanup) upon error here. But this is no longer so since
f3a29b750a5979ae6847879fba758faf1fae88d0.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Don't check for AVCodec.priv_data_size (which is always true if
AVCodec.priv_class is set). Instead check for AVCodecContext.priv_data
to actually exist.
(Note: av_opt_free(NULL) is a no-op.)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The first thing avcodec_open2() allocates is the AVCodecInternal. If
allocating it fails, a jump to end occurs; but if an error happens after
its allocation, a jump to free_and_end happens which frees all
allocations performed so far and then jumps to end. Yet free_and_end
contained a check for AVCodecInternal (after having already dereferenced
it to check whether ff_thread_free() needs to be called) which is of
course always true. So remove it.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
avcodec_open2() also called the AVCodec's close function if an error
happened before init had ever been called if the AVCodec has the
FF_CODEC_CAP_INIT_CLEANUP flag set. This is against the documentation of
said flag: "The codec allows calling the close function for deallocation
even if the init function returned a failure."
E.g. the SVQ3 decoder is not ready to be closed if init has never been
called.
Fixes: NULL dereference
Fixes: 25762/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SVQ3_fuzzer-5716279070294016
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Keeping only the latest packet fed to the decoder works only for decoders that
return a frame immediately after every consumed packet. Decoders that consume
several packets before they return a frame will fill said frame with properties
taken from the last consumed packet instead of the earliest.
Signed-off-by: James Almer <jamrial@gmail.com>
The user has no business modifying the underlying AVCodec.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Reviewed-by: Nicolas George <george@nsup.org>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Following the same logic as 061a0c14bb, this commit turns the old encode API
into a wrapper for the new one.
Signed-off-by: James Almer <jamrial@gmail.com>
This commit follows the same logic as 061a0c14bb, but for the encode API: The
new public encoding API will no longer be a wrapper around the old deprecated
one, and the internal API used by the encoders now consists of a single
receive_packet() callback that pulls frames as required.
amf encoders adapted by James Almer
librav1e encoder adapted by James Almer
nvidia encoders adapted by James Almer
MediaFoundation encoders adapted by James Almer
vaapi encoders adapted by Linjie Fu
v4l2_m2m encoders adapted by Andriy Gelman
Signed-off-by: James Almer <jamrial@gmail.com>
Currently the frame pool used by the default get_buffer2()
implementation is a single struct, allocated when opening the decoder.
A pointer to it is simply copied to each frame thread and we assume that
no thread attempts to modify it at an unexpected time. This is rather
fragile and potentially dangerous.
With this commit, the frame pool is made refcounted, with the reference
being propagated across threads along with other context variables. The
frame pool is now also immutable - when the stream parameters change we
drop the old reference and create a new one.
Fixes: out of array access
Fixes: 18432/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMAV2_fuzzer-5675574936207360
Fixes: 18326/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMAV2_fuzzer-5071752362721280
Fixes: 18384/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMAV1_fuzzer-5769439500304384
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: signed integer overflow: 2 * -1306460384 cannot be represented in type 'int'
Fixes: 17685/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_AAC_fuzzer-5747390337777664
Fixes: 17688/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_INTERPLAY_ACM_fuzzer-5739287210885120
Fixes: 17699/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_INTERPLAY_ACM_fuzzer-5678394531905536
Fixes: 17738/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_TAK_fuzzer-5763415733174272
Fixes: 17746/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_BINKAUDIO_RDFT_fuzzer-5703008159006720
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: James Almer <jamrial@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
4650975 -> 4493240 dezicycles
This optimizes lines 2 and later. Line 1 still uses av_memcpy_backptr()
This change originally fixed ossfuzz 10790 but this is now fixed by other
optimizations already
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: NULL pointer dereference
Fixes: 15733/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IDF_fuzzer-5658616977162240
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
after a successful init if the function fails for another reason close
the codec without requiring FF_CODEC_CAP_INIT_CLEANUP which is meant to
cover init failures themselves. fixes a memory leak in those cases.
BUG=oss-fuzz:15529
Signed-off-by: James Zern <jzern@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This avoids the need for each decoder separately having to handle this case
Fixes: shift exponent -100663046 is negative
Fixes: out of array access
Fixes: 15270/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IFF_ILBM_fuzzer-5727829913763840
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
