94 Commits

Author SHA1 Message Date
Wenbin Chen
83fe28221e hwcontext_vulkan: support creating DRM-tiled images and autodetecting modifiers
When vulkan image exports to drm, the tilling need to be
VK_IMAGE_TILING_DRM_FORMAT_MODIFIER_EXT. Now add code to create vulkan
image using this format.

Now the following command line works:

ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format \
vaapi -i input_1080p.264 -vf "hwmap=derive_device=vulkan,format=vulkan, \
scale_vulkan=1920:1080,hwmap=derive_device=vaapi,format=vaapi" -c:v h264_vaapi output.264

Signed-off-by: Wenbin Chen <wenbin.chen@intel.com>
Further-modifications-by: Lynne <dev@lynne.ee>
2021-12-10 17:03:48 +01:00
Wenbin Chen
0d524b170e hwcontext_vulkan: add support for mapping frames with planes in a single VkDeviceMemory
Add support to map vulkan frames to software frames when
using contiguous_planes flag.

Signed-off-by: Wenbin Chen <wenbin.chen@intel.com>
Further-modifications-by: Lynne <dev@lynne.ee>
2021-12-10 17:03:48 +01:00
Wenbin Chen
bd6ef73399 hwcontext_vulkan: add support for allocating all planes in a single allocation
VAAPI on Intel can import external frame, but the planes of the external
frames should be in the same drm object. A new option "contiguous_planes"
is added to device. This flag tells device to allocate places in one
memory. When device is derived from vaapi this flag will be enabled.
A new flag frame_flag is also added to AVVulkanFramesContext. User
can use this flag to force enable or disable this behaviour.
A new variable "offset "is added to AVVKFrame. It describe describe the
offset from the memory currently bound to the VkImage.

Signed-off-by: Wenbin Chen <wenbin.chen@intel.com>
Further-modifications-by: Lynne <dev@lynne.ee>
2021-12-10 17:03:48 +01:00
Lynne
c90b3661fa
hwcontext_vulkan: use correct return value for allocation failure 2021-11-27 04:46:41 +01:00
Wu Jianhua
b3624069f0 avutil/hwcontext_vulkan: fully support customizable validation layers
Validation layer is an indispensable part of developing on Vulkan.

The following commands is on how to enable validation layers:

ffmpeg -init_hw_device vulkan=0,debug=1,validation_layers=VK_LAYER_LUNARG_monitor+VK_LAYER_LUNARG_api_dump

Signed-off-by: Wu Jianhua <jianhua.wu@intel.com>
2021-11-26 10:36:39 +01:00
Wu Jianhua
7e9e2cf93b avutil/hwcontext_vulkan: check if created before destroying the instance
Signed-off-by: Wu Jianhua <jianhua.wu@intel.com>
2021-11-24 11:09:49 +01:00
Wu Jianhua
c2a356d583 avutil/hwcontext_vulkan: check if created before destroying the device
Signed-off-by: Wu Jianhua <jianhua.wu@intel.com>
2021-11-24 11:09:49 +01:00
Lynne
b159975e80
hwcontext_vulkan: check for non-flagged transfer queue families
"All commands that are allowed on a queue that supports transfer
operations are also allowed on a queue that supports either
graphics or compute operations. Thus, if the capabilities of a
queue family include VK_QUEUE_GRAPHICS_BIT or VK_QUEUE_COMPUTE_BIT,
then reporting the VK_QUEUE_TRANSFER_BIT capability separately for
that queue family is optional."
2021-11-20 02:37:41 +01:00
Lynne
d1133e8c44
lavu/vulkan: move common Vulkan code from libavfilter to libavutil 2021-11-19 13:44:45 +01:00
Timo Rothenpieler
2ece70090d avutil/hwcontext_vulkan: add support for exporting memory via Win32 Handles 2021-11-14 12:50:32 +01:00
Lynne
7f6dc9b386
hwcontext_vaapi: don't use the generic mapping struct for DRM/VAAPI
Avoids a per-frame allocation since we don't need the flag field.
2021-11-13 15:13:03 +01:00
Lynne
f388791ff9
hwcontext_vulkan: fix small memory leak when unmapping 2021-11-13 14:47:12 +01:00
Lynne
9dc544cdb4
hwcontext_vulkan: wait for semaphores when unmapping from VAAPI
We don't really want to do a full all-queue blocking wait here, since
this happens once per frame, and this could delay future frames.
2021-11-13 14:22:11 +01:00
Lynne
6a23a5597c
hwcontext_vulkan: print error information on queue submission failure
Makes it clearer what went wrong.
2021-11-13 14:21:36 +01:00
Lynne
c96d1ee401
hwcontext_vulkan: fix DMABUF import format check call
VkExternalImageFormatProperties is required to be present in the .pNext
chain of VkImageFormatProperties2, or some drivers crash (RADV).
2021-11-13 11:12:50 +01:00
Lynne
f74ceb358c
hwcontext_vulkan: improve CUDA error handling 2021-11-13 04:30:33 +01:00
Lynne
0d1992e025
hwcontext_vulkan: close exported memory FD on CUDA import error
Prevents resource leakage.
2021-11-13 00:40:46 +01:00
Lynne
015b487777
hwcontext_vulkan: do not dup() semaphore FDs for CUDA 2021-11-13 00:32:53 +01:00
Lynne
fa28c1b2f9
hwcontext_vulkan: properly migrate queue families with DRM import/export 2021-11-13 00:03:58 +01:00
Lynne
549d91ae3a
hwcontext_vulkan: properly migrate between queue families on CUDA import/export
It's more correct.
2021-11-13 00:03:56 +01:00
Lynne
8449baf9aa
hwcontext_vulkan: properly error out if timeline semaphores are unsupported
Missing goto.
2021-11-13 00:03:51 +01:00
Lynne
296cb99d46
hwcontext_vulkan: fix CreateSemaphore conflict with synchapi.h
Include windows.h to fix it. Normally, it'd be better to include it in
vulkan_functions.h, but I'm reasonably confident nothing else that uses
the Vulkan code will need to include Windows functions and not windows.h.
2021-11-12 14:45:20 +01:00
Lynne
57e11321ea
hwcontext_vulkan: use vkDeviceWaitIdle instead of vkWaitSemaphores on uninit
To silence a possible validation layer bug, switch the function. It only gets
triggered by vf_libplacebo, which is odd.
2021-11-12 14:45:17 +01:00
Lynne
d05a18cdc7
lavu: move hwcontext_vulkan's function loader into separate files
This allows for the loader to be shared with libavcodec and libavfilter.
2021-11-12 05:23:40 +01:00
Lynne
1ffb59c056
hwcontext_vulkan: clean up extensions code and add additional defaults 2021-11-12 05:23:40 +01:00
Lynne
bde1fc5386
hwcontext_vulkan: host wait on semaphores before freeing frame 2021-11-12 05:23:39 +01:00
Lynne
f7f1613638
hwcontext_vulkan: report device that's used
Not sure why this wasn't done before.
2021-11-12 05:23:39 +01:00
Lynne
6bf9a6539e
vulkan: add support for encode and decode queues and refactor queue code
This simplifies and makes queue family picking simpler and more robust.
The requirements on the device context are relaxed. They made no sense
in the first place.

The video encode/decode extension is still in beta, at least on paper,
but I really doubt they'd change needing a separate queue family.
2021-11-12 05:23:36 +01:00
Lynne
09e4687b5b
hwcontext_vulkan: port CUDA interop to use timeline semaphores 2021-11-12 03:36:44 +01:00
Lynne
0370a580dc
hwcontext_vulkan: fix mapping from/to DRM/VAAPI frames 2021-11-12 03:36:42 +01:00
Lynne
00ef53c3ea
hwcontext_vulkan: switch to using timeline semaphores 2021-11-12 03:36:40 +01:00
Lynne
7f3878828d
hwcontext_vulkan: bump required Vulkan loader version to 1.2 2021-11-12 03:36:35 +01:00
Andreas Rheinhardt
1ea3650823 Replace all occurences of av_mallocz_array() by av_calloc()
They do the same.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2021-09-20 01:03:52 +02:00
Lynne
1c5610824a
hwcontext_vulkan: use GPU memcpy when copying to system RAM
This should speed it up significantly on systems where it matters.
2021-08-14 00:31:28 +02:00
Lynne
08d933bf61
hwcontext_vulkan: fix typo in vulkan_device_init()
load_functions() did not load the device-level functions.
2021-06-10 12:24:04 +02:00
Lynne
cf17e2323f hwcontext_vulkan: dlopen libvulkan
While Vulkan itself went more or less the way it was expected to go,
libvulkan didn't quite solve all of the opengl loader issues. It's multi-vendor,
yes, but unfortunately, the code is Google/Khronos QUALITY, so suffers from
big static linking issues (static linking on anything but OSX is unsupported),
has bugs, and due to the prefix system used, there are 3 or so ways to type out
functions.

Just solve all of those problems by dlopening it. We even have nice emulation
for it on Windows.
2021-04-30 00:08:37 +02:00
Lynne
4a6581e968 hwcontext_vulkan: dynamically load functions
This patch allows for alternative loader implementations.
2021-04-30 00:08:37 +02:00
James Almer
ffeeff4fbc avutil/hwcontext_vulkan: fix format specifiers for some printed variables
VkPhysicalDeviceLimits.optimalBufferCopyRowPitchAlignment and
VkPhysicalDeviceExternalMemoryHostPropertiesEXT.minImportedHostPointerAlignment are of type
VkDeviceSize (a typedef uint64_t).
VkPhysicalDeviceLimits.minMemoryMapAlignment is of type size_t.

Signed-off-by: James Almer <jamrial@gmail.com>
Reviewed-by: Lynne <dev@lynne.ee>
2021-04-29 14:04:02 -03:00
Andreas Rheinhardt
ef6a9e5e31 avutil/buffer: Switch AVBuffer API to size_t
Announced in 14040a1d913794d9a3fd6406a6d8c2f0e37e0062.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Signed-off-by: James Almer <jamrial@gmail.com>
2021-04-27 10:43:13 -03:00
James Almer
e36eb94048 avutil: use the buffer_size_t typedef where required
Signed-off-by: James Almer <jamrial@gmail.com>
2021-03-10 20:26:37 -03:00
Xu Guangxin
ae97f69ce1 avutils/vulkan: hwmap, respect src frame resolution
fixes http://trac.ffmpeg.org/ticket/9055

The hw decoder may allocate a large frame from AVHWFramesContext, and adjust width and height based on bitstream.
We need to use resolution from src frame instead of AVHWFramesContext.

test command:
ffmpeg -loglevel debug -hide_banner -hwaccel vaapi -init_hw_device vaapi=va:/dev/dri/renderD128 -hwaccel_device va -hwaccel_output_format vaapi -init_hw_device vulkan=vulk -filter_hw_device vulk -i 1920x1080.264 -c:v libx264 -r:v 30 -profile:v high -preset veryfast -vf "hwmap,chromaber_vulkan=0:0,hwdownload,format=nv12" -map 0 -y vaapiouts.mkv

expected:
No green bar at bottom.
2021-01-22 04:30:42 +01:00
Lynne
b51b9bbd42
hwcontext_vulkan: wait and signal semaphores when transferring to CUDA
Same as when downloading. Not sure why this isn't done, probably
because the CUDA code predates the sync mechanism we settled on.
2020-12-05 23:53:23 +01:00
Lynne
659e6e9c88
hwcontext_vulkan: reduce priority for PACK32 formats
Due to some endian-dependent overlap, these should be used last.
2020-11-27 02:58:02 +01:00
Niklas Haas
3fbc74582f
hwcontext_vulkan: optionally enable more functionality
These two extensions and two features are both optionally used by
libplacebo to speed up rendering, so it makes sense for libavutil to
automatically enable them as well.
2020-11-25 23:14:47 +01:00
Lynne
2aeb152653
hwcontext_vulkan: support additional pixel formats
We support every single packed format possible now.
There are some fringe leftover mappings which are uninteresting.
2020-11-25 23:14:47 +01:00
Lynne
48b3532183
hwcontext_vulkan: fix incorrect A/0BGR mapping
Vulkan formats with a PACK suffix define native endianess.
Vulkan formats without a PACK suffix are in bytestream order.

Pixel formats with a LE/BE suffix define endianess.
Pixel formats without LE/BE suffix are in bytestream order.
2020-11-25 23:14:46 +01:00
Lynne
3aa8de12ab
hwcontext_vulkan: simplify plane size calculations and support 4-plane formats
Needed to support YUVA.
2020-11-25 23:14:46 +01:00
Lynne
7b274a9b89
hwcontext_vulkan: do not segfault when failing to init a AVHWFramesContext
frames_uninit is always called on failure, and the free_exec_ctx function
did not zero the pool when freeing it, so it resulted in a double free.
2020-11-25 23:14:46 +01:00
Lynne
18a6535b08
hwcontext_vulkan: always attempt to map host memory when transferring
This relies on the fact that host memory is always going to be required
to be aligned to the platform's page size, which means we can adjust
the pointers when we map them to buffers and therefore skip an entire
copy. This has already had extensive testing in libplacebo without
problems, so its safe to use here as well.

Speeds up downloads and uploads on platforms which do not pool their
memory hugely, but less so on platforms that do.

We can pool the buffers ourselves, but that can come as a later patch
if necessary.
2020-11-25 23:14:01 +01:00
Lynne
9cf1811d3d
hwcontext_vulkan: check for memory size before choosing type
It makes allocation a bit more robust in case some weird device with
weird drivers which segments memory in weird ways appears.
2020-11-25 23:06:36 +01:00