enable HW VPU decoding for SoCs that have stateless VPUs

v4l2codecs is the gstreamer plugin for V4L2 stateless video hardware
decoding. The Rockchip SoCs that have a VPU all seems to be based on
Hantro, RKVDEC or RKVDECv2, all stateless encoding/decoding VPUs.

Therefore, let's enable VPU decoding in Gstreamer whenever possible,
when the SoC supports it.

PX30, RK3066, RK3188, RK3288, RK3328, RK3399, RK356x and RK3588(s) all
have at least one Hantro VPU.

RK3328, RK3399, RK356x and RK3588(s) all have at least one
RKVDEC/RKVDECv2 VPU (though not necessarily supported in the upstream
kernel just yet).

=== PX30
Tested on PX30 Ringneck with with Haikou+Haikou Video Demo adapter:

$ gst-launch-1.0 filesrc location=$FILE ! parsebin ! v4l2slh264dec ! waylandsink

with FILE storing the path to any h264 file, e.g.
https://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_720p_h264.mov
https://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_1080p_h264.mov

Needed packages are:
- weston
- gstreamer1.0-plugins-bad (for waylandsink and v4l2slh264dec)
- gstreamer1.0-plugins-base (for parsebin)

A few frames are dropped every other second for 1080p but otherwise
smooth.

=== RK3399
Tested on RK3399 Puma with Haikou:

$ gst-launch-1.0 filesrc location=$FILE ! parsebin ! v4l2slh264dec ! waylandsink

with FILE storing the path to any h264 file, e.g.
https://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_1080p_h264.mov
https://download.blender.org/demo/movies/BBB/bbb_sunflower_2160p_30fps_normal.mp4.zip

Needed packages are:
- weston
- gstreamer1.0-plugins-bad (for waylandsink and v4l2codecs)
- gstreamer1.0-plugins-base (for parsebin)

=== RK3588

Tested on a RK3588 Tiger with Haikou+Haikou Video Demo adapter - on a
downstream v6.6 (upstream-based, not Rockchip BSP-based) with DSI
patches - :

$ gst-launch-1.0 filesrc location=$FILE ! parsebin ! v4l2slav1dec ! fakesink

with FILE storing the path to any AV1 file, e.g.
http://download.opencontent.netflix.com.s3.amazonaws.com/AV1/cmaf/spark-8b-59.94fps/spark_606kbps_432p.mp4
https://woolyss.com/f/av1-nosound-chimera.mp4
https://woolyss.com/f/av1-opus-sita.webm

Needed packages are:
- gstreamer1.0-plugins-bad (for fakesink and v4l2slav1dec)
- gstreamer1.0-plugins-base (for parsebin)

For some reason though, waylandsink is very choppy. Combining
fpsdisplaysink with fakesink shows a ~60fps when decoding the 432p file,
~24fps for the two others.
Note that 10b-depth isn't supported (at least in my setup).

Reviewed-by: Trevor Woerner <twoerner@gmail.com>
Signed-off-by: Quentin Schulz <quentin.schulz@cherry.de>
This commit is contained in:
Quentin Schulz 2024-08-28 18:23:59 +02:00 committed by Trevor Woerner
parent e6789604ab
commit 91a694e250
11 changed files with 41 additions and 0 deletions

22
README
View File

@ -103,6 +103,28 @@ Notes:
this layer, perform the same steps as above except for the step enabling
RK_RAUC_DEMO.
HW video decoding with gstreamer
Most Rockchip SoCs have some integrated VPU, either Hantro, RKVDEC or
RKVDECv2. Those are called stateless VPUs and they require tools to be
adapted so they can be used. Upstream ffmpeg doesn't support them (yet)
but gstreamer does, via the v4l2sl* plugins. However, by default those
aren't built in OE-Core. This layer enables these plugins by default for
all SoCs that have at least one VPU (at the time of writing, all SoCs
supported by this layer to the exception of RK3308).
If you wish to NOT build those plugins anymore, you can set
ENABLE_STATELESS_VPU_GST = "0"
in any appropriate conf file (or in a
gstreamer1.0-plugins-bad_%.bbappend in your own layer).
Note that this is only relevant for upstream-based Linux kernels as
Rockchip have developed their own ABI for their own kernel that isn't
compatible (see Rockchip Media Process Platform (MPP) and
downstream gstreamer-rockchip plugin for those kernels).
U-Boot Environment:
------------------
In order to configure U-Boot to be able to store its environment into the

View File

@ -19,3 +19,5 @@ TFA_BUILD_TARGET = "bl31"
UBOOT_SUFFIX ?= "itb"
UBOOT_ENTRYPOINT ?= "0x06000000"
ENABLE_STATELESS_VPU_GST ?= "1"

View File

@ -12,3 +12,5 @@ SERIAL_CONSOLES = "115200;ttyS2"
KBUILD_DEFCONFIG = "multi_v7_defconfig"
KERNEL_FEATURES:append:rk3066 = " bsp/rockchip/remove-non-rockchip-arch-arm.scc"
KERNEL_IMAGETYPE ?= "zImage"
ENABLE_STATELESS_VPU_GST ?= "1"

View File

@ -12,3 +12,5 @@ SERIAL_CONSOLES = "115200;ttyFIQ0"
KBUILD_DEFCONFIG = "multi_v7_defconfig"
KERNEL_FEATURES:append:rk3188 = " bsp/rockchip/remove-non-rockchip-arch-arm.scc"
KERNEL_IMAGETYPE ?= "zImage"
ENABLE_STATELESS_VPU_GST ?= "1"

View File

@ -14,3 +14,5 @@ KERNEL_FEATURES:append:rk3288 = " bsp/rockchip/remove-non-rockchip-arch-arm.scc"
KERNEL_IMAGETYPE ?= "zImage"
UBOOT_SUFFIX ?= "bin"
ENABLE_STATELESS_VPU_GST ?= "1"

View File

@ -20,3 +20,5 @@ TFA_BUILD_TARGET = "bl31"
UBOOT_SUFFIX ?= "itb"
UBOOT_ENTRYPOINT ?= "0x06000000"
ENABLE_STATELESS_VPU_GST ?= "1"

View File

@ -20,3 +20,5 @@ TFA_BUILD_TARGET = "bl31"
UBOOT_SUFFIX ?= "itb"
UBOOT_ENTRYPOINT ?= "0x06000000"
ENABLE_STATELESS_VPU_GST ?= "1"

View File

@ -19,3 +19,5 @@ PREFERRED_PROVIDER_optee-os = "rockchip-rkbin"
UBOOT_SUFFIX ?= "itb"
UBOOT_ENTRYPOINT ?= "0x06000000"
ENABLE_STATELESS_VPU_GST ?= "1"

View File

@ -19,3 +19,5 @@ PREFERRED_PROVIDER_optee-os = "rockchip-rkbin"
UBOOT_SUFFIX ?= "itb"
UBOOT_ENTRYPOINT ?= "0x06000000"
ENABLE_STATELESS_VPU_GST ?= "1"

View File

@ -18,3 +18,5 @@ PREFERRED_PROVIDER_optee-os = "rockchip-rkbin"
UBOOT_SUFFIX ?= "itb"
UBOOT_ENTRYPOINT ?= "0x06000000"
ENABLE_STATELESS_VPU_GST ?= "1"

View File

@ -0,0 +1 @@
PACKAGECONFIG:append:rockchip = "${@' v4l2codecs' if bb.utils.to_boolean(d.getVar('ENABLE_STATELESS_VPU_GST'), False) else ''}"