Michael Niedermayer
179ed2d2e0
avcodec/diracdec: check that GetBitContext has not ended in codeblock()
...
Fixes: Timeout (part 2 of 2)
Fixes: 9774/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DIRAC_fuzzer-5748957085958144
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2018-09-28 01:35:12 +02:00
Michael Niedermayer
7274d849cc
avcodec/diracdec: Propagate errors from codeblock()
...
Consider a component to be damaged if more than 50% of its subbands are damaged
Fixes: Timeout (part 1 of 2)
Fixes: 9774/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DIRAC_fuzzer-5748957085958144
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2018-09-28 01:35:09 +02:00
Michael Niedermayer
bed125b710
avcodec/diracdec: Check bytes count in else branch in decode_lowdelay() too
...
Fixes: signed integer overflow: 8 * 340018243 cannot be represented in type 'int'
Fixes: 9441/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DIRAC_fuzzer-5194665207791616
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2018-07-28 14:35:51 +02:00
Michael Niedermayer
f457c0ad7f
avcodec/diracdec: Check slice numbers for overflows in relation to picture dimensions
...
Fixes: signed integer overflow: 88 * 33685506 cannot be represented in type 'int'
Fixes: 9433/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DIRAC_fuzzer-5725943535501312
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2018-07-28 14:35:50 +02:00
Michael Niedermayer
462d1be6de
avcodec/diracdec: Change frame_number to 64bit as its a 32bit from the bitstream and we also have a -1 special case
...
Fixes: signed integer overflow: 2147483647 + 1 cannot be represented in type 'int'
Fixes: 9291/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DIRAC_fuzzer-6324345860259840
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2018-07-28 14:35:50 +02:00
Michael Niedermayer
5129040646
avcodec/diracdec: Prevent integer overflow in intermediate in global_mv()
...
Fixes: signed integer overflow: -393471 * 5460 cannot be represented in type 'int'
Fixes: 8890/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DIRAC_fuzzer-6299775379963904
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2018-07-28 14:35:50 +02:00
Michael Niedermayer
47e65ad63b
avcodec/diracdec: Fix integer overflow in mv computation
...
Fixes: signed integer overflow: -2072 + -2147483646 cannot be represented in type 'int'
Fixes: 6097/clusterfuzz-testcase-minimized-5034145253163008
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2018-02-20 15:27:51 +01:00
Michael Niedermayer
cbcbefdc3b
avcodec/diracdec: Use int64 in global mv to prevent overflow
...
Fixes: runtime error: signed integer overflow: 361 * -6295541 cannot be represented in type 'int'
Fixes: 5911/clusterfuzz-testcase-minimized-6450382197751808
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2018-02-20 15:27:51 +01:00
Michael Niedermayer
eaa9317589
avcodec/diracdec: Fix integer overflow with quant
...
Fixes: signed integer overflow: 2 + 2147483646 cannot be represented in type 'int'
Fixes: 4792/clusterfuzz-testcase-minimized-6322450775146496
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2018-01-08 21:50:12 +01:00
Nicolas Frattaroli
1c06a32cfa
diracdec: fix deprecated API usage
...
avcodec_get_chroma_sub_sample is deprecated and generates a warning
during build, so av_pix_fmt_get_chroma_sub_sample is used
Signed-off-by: Nicolas Frattaroli <ffmpeg@fratti.ch >
Signed-off-by: James Almer <jamrial@gmail.com >
2017-11-03 00:26:26 -03:00
Michael Niedermayer
b5995856a4
avcodec/diracdec: Fix overflow in DC computation
...
Fixes: runtime error: signed integer overflow: 11896 + 2147483646 cannot be represented in type 'int'
Fixes: 3053/clusterfuzz-testcase-minimized-6355082062856192
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2017-09-11 12:23:53 +02:00
Michael Niedermayer
2a0823ae96
avcodec/diracdec: Fix integer overflow in INTRA_DC_PRED()
...
Fixes: runtime error: signed integer overflow: 1168175789 + 1168178473 cannot be represented in type 'int'
Fixes: 3081/clusterfuzz-testcase-minimized-4807564879462400
Fixes: 2844/clusterfuzz-testcase-minimized-5561715838156800
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2017-09-02 23:50:28 +02:00
Michael Niedermayer
92da23093c
avcodec/diracdec: Fixes integer overflow
...
Fixes: runtime error: signed integer overflow: 340018243 * 27 cannot be represented in type 'int'
Fixes: 2861/clusterfuzz-testcase-minimized-5361070510178304
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2017-08-17 00:23:01 +02:00
Michael Niedermayer
1e6cab8745
avcodec/diracdec: Check perspective_exp and zrs_exp.
...
Fixes: undefined shift
Fixes: runtime error: shift exponent 264 is too large for 32-bit type 'int'
Fixes: 2860/clusterfuzz-testcase-minimized-4672811689836544
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2017-08-17 00:23:01 +02:00
Michael Niedermayer
880f5c5913
avcodec/diracdec: Check weight_log2denom
...
Fixes: runtime error: shift exponent -1 is negative
Fixes: 2742/clusterfuzz-testcase-minimized-5724322402402304
Fixes: 2744/clusterfuzz-testcase-minimized-4672435653705728
Fixes: 2749/clusterfuzz-testcase-minimized-5298741273690112
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2017-08-01 13:54:43 +02:00
Michael Niedermayer
c0220c768c
avcodec/diracdec: Fix integer overflow in divide3()
...
Fixes: runtime error: signed integer overflow: -1073746548 * 21845 cannot be represented in type 'int'
Fixes: 2729/clusterfuzz-testcase-minimized-5902915464069120
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2017-07-29 04:05:53 +02:00
Michael Niedermayer
8e275a74b0
avcodec/diracdec: Fix integer overflow in signed multiplication in UNPACK_ARITH()
...
Fixes: runtime error: signed integer overflow: 1073741823 * 4 cannot be represented in type 'int'
Fixes: 2729/clusterfuzz-testcase-minimized-5902915464069120
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2017-07-28 03:35:22 +02:00
Michael Niedermayer
edc88646ee
avcodec/diracdec: Check dimensions which are closer to what is allocated in alloc_sequence_buffers()
...
Fixes OOM
Fixes: 2674/clusterfuzz-testcase-minimized-4999700518273024
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2017-07-21 17:37:05 +02:00
Michael Niedermayer
b946bd8ef2
avcodec/diracdec: Fix off by 1 error in quant check
...
Fixes: out of array read
Fixes: 1781/clusterfuzz-testcase-minimized-4617176877105152
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2017-05-25 19:32:39 +02:00
Michael Niedermayer
6d6fc4105b
avcodec/diracdec: Factor quant matrix reads
...
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2017-05-25 19:32:39 +02:00
Michael Niedermayer
6899e6e560
avcodec/diracdec: Fix Assertion frame->buf[0] failed at libavcodec/decode.c:610
...
Fixes: 1487/clusterfuzz-testcase-minimized-6288036495097856
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2017-05-11 23:55:59 +02:00
Ronald S. Bultje
5ba8c3a0ed
dirac: make initialization of arithmetic coder tables threadsafe.
2017-03-28 11:21:27 -04:00
Andreas Cadhalpun
db79dedb1a
diracdec: check return code of get_buffer_with_edge
...
If it fails, buffers aren't allocated, causing NULL pointer dereferencing.
Reviewed-by: Rostislav Pehlivanov <atomnuker@gmail.com >
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com >
2016-11-04 20:35:23 +01:00
Andreas Cadhalpun
24d20496d2
diracdec: clear slice_params_num_buf on allocation failure
...
Otherwise it can be non-zero next time decode_lowdelay is called, causing
slice_params_buf not to be allocated, leading to a NULL pointer dereference.
The problem was introduced in commit
dcad4677d6
.
Reviewed-by: Rostislav Pehlivanov <atomnuker@gmail.com >
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com >
2016-11-04 20:34:51 +01:00
Andreas Cadhalpun
8a4ea96448
diracdec: use correct buffer for slice_params_buf realloc
...
This fixes a double-free detected by AddressSanitizer.
The problem was introduced in commit
dcad4677d6
.
Reviewed-by: Rostislav Pehlivanov <atomnuker@gmail.com >
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com >
2016-11-04 20:34:38 +01:00
Michael Niedermayer
a31e08fa1a
avcodec/diracdec: Check numx/y
...
Fixes division by 0
Fixes: 60261c4469ba3e11059890fb2832a515/asan_generic_135e694_2790_beb94eaa0aeb7d11c0437375a8964a99.drc
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2016-08-20 19:36:01 +02:00
Rostislav Pehlivanov
000eb01a7d
diracdec: fix unchecked byte length
...
Also drops the start variable since it's redundant.
Found by Coverity, fixes CID1363964
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com >
2016-07-13 23:53:05 +01:00
Rostislav Pehlivanov
b2b12b2d4a
diracdec: fix maximum quantization index checks
...
Found by Coverity, fixes CID1363961 and CID1363962
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com >
2016-07-13 23:40:06 +01:00
Rostislav Pehlivanov
a337cb7361
diracdec: fix #coeffs -> byte conversion
...
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com >
2016-07-12 10:44:24 +01:00
Rostislav Pehlivanov
2094562923
diracdec: do not memset the entire coefficient buffer for HQ pictures
...
This is now handled by the slice decoding function.
Signed-off-by: Rostislav Pehlivanov <rpehlivanov@obe.tv >
2016-07-11 23:41:32 +01:00
Rostislav Pehlivanov
dcad4677d6
diracdec: do not allocate and free slice parameters every frame
...
Signed-off-by: Rostislav Pehlivanov <rpehlivanov@obe.tv >
2016-07-11 23:40:57 +01:00
Rostislav Pehlivanov
0eb0f93109
diracdec: implement a LUT-based Golomb code parser
...
Still much left to optimize, but it provides a significant performance
improvement - 10% for 300Mbps (1080p30), 25% for 1.5Gbps (4k 60fps) in
comparison with the default implementation.
Signed-off-by: Rostislav Pehlivanov <rpehlivanov@obe.tv >
2016-07-11 23:40:28 +01:00
Rostislav Pehlivanov
c43485f707
diracdec: rewrite HQ slice decoding
...
Now coefficients are written to a buffer and are then dequantized by the
new SIMD dequantization functions. For the lower bands without enough
coefficients to fill a register (and hence they overwrite) the C version
of the dequantization function is used.
The buffer is per-thread and will be realloc'd if anything changes.
This prevents regressions and having to limit slice size.
Signed-off-by: Rostislav Pehlivanov <rpehlivanov@obe.tv >
2016-07-11 23:38:45 +01:00
Rostislav Pehlivanov
09d89d9406
diractab: expose the maximum quantization index as a macro
...
Prevents having to have random magic values in the decoder and a
separate macro in the encoder.
Signed-off-by: Rostislav Pehlivanov <rpehlivanov@obe.tv >
2016-07-11 23:38:01 +01:00
Rostislav Pehlivanov
b9c6c5f453
diracdec: decode HQ profile slices in rows
...
Siginificantly improves the performance.
Signed-off-by: Rostislav Pehlivanov <rpehlivanov@obe.tv >
2016-07-11 23:34:54 +01:00
Rostislav Pehlivanov
17caae7202
diracdec: simplify golomb parsing and dequantization
...
In preparation for the following commits, this commit simplifies the
coefficient parsing and dequantization function. It was needlessly
inlined without much performance gain.
Signed-off-by: Rostislav Pehlivanov <rpehlivanov@obe.tv >
2016-07-11 23:34:25 +01:00
Hendrik Leppkes
b20fe650ef
Merge commit '4024b566d664a4b161d677554be52f32e7ad4236'
...
* commit '4024b566d664a4b161d677554be52f32e7ad4236':
golomb: Give svq3_get_se_golomb()/svq3_get_ue_golomb() better names
Merged-by: Hendrik Leppkes <h.leppkes@gmail.com >
2016-06-26 15:12:48 +02:00
Michael Niedermayer
7ecfe4dc36
avcodec/diracdec: Fix potential integer overflow
...
Fixes CID1361948
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2016-05-19 18:13:29 +02:00
Michael Niedermayer
8f2a1990c0
avcodec/diracdec: check bitstream size related fields for overflows
...
Fixes segfault
Fixes Ticket5333
Regression since bfc8a4dabe
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2016-03-28 04:39:57 +02:00
Timothy Gu
671761d713
diracdec: Pass DWTPlane to dwt init
2016-02-07 09:09:13 -08:00
Timothy Gu
e04912c0b6
diracdec: Split DWTPlane struct from Plane
2016-02-07 09:09:13 -08:00
Timothy Gu
58ded09bd1
dirac_dwt: Rename init2 to init
...
The functions are all private.
2016-02-07 09:09:13 -08:00
Rostislav Pehlivanov
f021030365
diradec: split tables away to a separate diractab file
...
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com >
2016-02-03 17:19:33 +00:00
Rostislav Pehlivanov
3bbe7862ec
diracdec: move the MAX_DWT_LEVELS macro to dirac.h
...
Used by the VC-2 encoder.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com >
2016-02-03 17:19:30 +00:00
Rostislav Pehlivanov
5776344a7b
diracdec: fix original Dirac Low Delay profile
...
The version structure in the main decoder context was not (and
apparently has never been) populated since it was added.
Still, having VC-2 break the existing Dirac Low Delay mode was odd and
easily avoidable had the specifications authors noticed/cared.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com >
2016-01-21 16:49:45 +00:00
Rostislav Pehlivanov
8248b51e0b
diracdec: add support for 12 bit videos
...
The DSP lacked a function needed to convert signed to unsigned. This was
ignored when originally adding support and templating for bit depths
greater than 8. The 10 bit function was used for 12 bit pictures and
resulted in an improper conversion.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com >
2016-01-21 14:48:32 +00:00
Michael Niedermayer
5fbd97fc75
avcodec/diracdec: Fix qfactor/offset tables
...
It seems the previous tables where calculated with 32bit integers ignoring
overflows.
Also check for the max qindex, the value is choosen so that the qfactor/offset
fit in int32.
Fixes: 070b7914fd5dfe8f93248bea71363410/asan_static-oob_c8d034_2764_258e20f4a3c79158aecddb61a833d756.drc
Fixes out of array reads
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2016-01-17 17:28:20 +01:00
Michael Niedermayer
39fb3f18c5
avcodec/diracdec: Handle the 0 vlc case at the top of coeff_unpack_golomb()
...
decoding changes from 17 to 20 fps
Reviewed-by; Rostislav Pehlivanov <atomnuker@gmail.com >
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2016-01-13 18:56:49 +01:00
Michael Niedermayer
bbd9771625
avcodec/diracdec: Factor +2 out of the inner loop
...
Reviewed-by: Rostislav Pehlivanov <atomnuker@gmail.com >
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2016-01-13 18:55:46 +01:00
Michael Niedermayer
da144c2ddd
avcodec/diracdec: Inline svq3_get_ue_golomb() and merge the sign bit decoding into it
...
This avoids closing and opening the bit reader
Reviewed-by: Rostislav Pehlivanov <atomnuker@gmail.com >
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2016-01-13 18:54:50 +01:00