har0ke/FFmpeg
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

avutil/mathematics: Do not treat INT64_MIN as positive in av_rescale_rnd

Browse Source 
The code expects actual positive numbers and gives completely wrong
results if INT64_MIN is treated as positive
Instead clip it into the valid range that is add 1 and treat it as
negative

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This commit is contained in:
Michael Niedermayer 2015-12-01 12:44:23 +01:00
parent 3a9cb18855
commit 25e37f5ea9
1 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
libavutil/mathematics.c
View File

@ -71,8 +71,8 @@ int64_t av_rescale_rnd(int64_t a, int64_t b, int64_t c, enum AVRounding rnd)
rnd -= AV_ROUND_PASS_MINMAX;
}
if (a < 0 && a != INT64_MIN)
return -av_rescale_rnd(-a, b, c, rnd ^ ((rnd >> 1) & 1));
if (a < 0)
return -av_rescale_rnd(-FFMAX(a, -INT64_MAX), b, c, rnd ^ ((rnd >> 1) & 1));
if (rnd == AV_ROUND_NEAR_INF)
r = c / 2;