Mainline 5.4 kernel with VPU acceleration on iMX6


Fabio Estevam
 

Hi Andreas,

On Tue, Apr 7, 2020 at 5:44 AM Andreas Müller <schnitzeltony@...> wrote:

Have no use case currently so just out of curiosity: Does this also
apply to community kernels - do they support vpu-acceleration?
Not sure what you mean by "community kernel", but any recent kernel
from kernel.org (such as 5.4.x, 5.5.x, 5.6.x) support VPU acceleration
on i.MX6.

Regards,

Fabio Estevam


Andreas Müller
 

On Mon, Apr 6, 2020 at 9:51 PM Fabio Estevam <festevam@...> wrote:

Hi Mark,

On Sun, Apr 5, 2020 at 7:17 PM Mark Farver <mfarver@...> wrote:

On Sun, Apr 5, 2020 at 1:19 AM Carlos Rafael Giani <crg7475@...> wrote:
For using etnaviv, just use glimagesink. For using the VPU, use the v4l2*dec elements.
Definitely doing CPU decoding. I don't see any kernel messages that
indicate the VPU has been detected even though I have
Try grepping for "coda", which is the VPU driver name in mainline:

# dmesg | grep coda
[ 5.483623] coda 2040000.vpu: Direct firmware load for
vpu_fw_imx6q.bin failed with error -2
[ 5.492236] coda 2040000.vpu: Falling back to sysfs fallback for:
vpu_fw_imx6q.bin
[ 69.619815] coda 2040000.vpu: Using fallback firmware vpu/vpu_fw_imx6q.bin
[ 69.632736] coda 2040000.vpu: Firmware code revision: 46076
[ 69.638440] coda 2040000.vpu: Initialized CODA960.
[ 69.643255] coda 2040000.vpu: Firmware version: 3.1.1
[ 69.649875] coda 2040000.vpu: encoder registered as video9
[ 69.656045] coda 2040000.vpu: encoder registered as video10
[ 69.662214] coda 2040000.vpu: decoder registered as video11

CONFIG_VIDEO_IMX_PXP=y in the kernel config. Is that the correct
CONFIG_VIDEO_IMX_PXP=y applies to other i.MX SoCs.

option? Are there others? The device tree entry for the VPU seems to
be included in the kernel configuration by default.

I grabbed OSSystems meta-gstreamer layer to get 1.16.2, but even with
that included I do not appear to get any v4l2 decode plugins:

# gst-inspect-1.0 --version
gst-inspect-1.0 version 1.16.2
GStreamer 1.16.2
Unknown package origi

# gst-inspect-1.0 |grep v4l
imxv4l2video: imxv4l2videosrc: V4L2 CSI Video Source
imxv4l2video: imxv4l2videosink: V4L2 CSI Video Sink
This is wrong. You should not install these imx elements when using a
mainline kernel. These elements are to be used with NXP kernel only.
Have no use case currently so just out of curiosity: Does this also
apply to community kernels - do they support vpu-acceleration?

Andreas


Fabio Estevam
 

Hi Mark,

On Sun, Apr 5, 2020 at 7:17 PM Mark Farver <mfarver@...> wrote:

On Sun, Apr 5, 2020 at 1:19 AM Carlos Rafael Giani <crg7475@...> wrote:
For using etnaviv, just use glimagesink. For using the VPU, use the v4l2*dec elements.
Definitely doing CPU decoding. I don't see any kernel messages that
indicate the VPU has been detected even though I have
Try grepping for "coda", which is the VPU driver name in mainline:

# dmesg | grep coda
[ 5.483623] coda 2040000.vpu: Direct firmware load for
vpu_fw_imx6q.bin failed with error -2
[ 5.492236] coda 2040000.vpu: Falling back to sysfs fallback for:
vpu_fw_imx6q.bin
[ 69.619815] coda 2040000.vpu: Using fallback firmware vpu/vpu_fw_imx6q.bin
[ 69.632736] coda 2040000.vpu: Firmware code revision: 46076
[ 69.638440] coda 2040000.vpu: Initialized CODA960.
[ 69.643255] coda 2040000.vpu: Firmware version: 3.1.1
[ 69.649875] coda 2040000.vpu: encoder registered as video9
[ 69.656045] coda 2040000.vpu: encoder registered as video10
[ 69.662214] coda 2040000.vpu: decoder registered as video11

CONFIG_VIDEO_IMX_PXP=y in the kernel config. Is that the correct
CONFIG_VIDEO_IMX_PXP=y applies to other i.MX SoCs.

option? Are there others? The device tree entry for the VPU seems to
be included in the kernel configuration by default.

I grabbed OSSystems meta-gstreamer layer to get 1.16.2, but even with
that included I do not appear to get any v4l2 decode plugins:

# gst-inspect-1.0 --version
gst-inspect-1.0 version 1.16.2
GStreamer 1.16.2
Unknown package origi

# gst-inspect-1.0 |grep v4l
imxv4l2video: imxv4l2videosrc: V4L2 CSI Video Source
imxv4l2video: imxv4l2videosink: V4L2 CSI Video Sink
This is wrong. You should not install these imx elements when using a
mainline kernel. These elements are to be used with NXP kernel only.

video4linux2: v4l2src: Video (video4linux2) Source
video4linux2: v4l2sink: Video (video4linux2) Sink
video4linux2: v4l2radio: Radio (video4linux2) Tuner
video4linux2: v4l2deviceprovider (GstDeviceProviderFactory)
So first, make sure the coda driver is probed successfully in the kernel.

Then Gstreamer would detect the VPU decoders plugins:

# gst-inspect-1.0 | grep v4l2
video4linux2: v4l2src: Video (video4linux2) Source
video4linux2: v4l2sink: Video (video4linux2) Sink
video4linux2: v4l2radio: Radio (video4linux2) Tuner
video4linux2: v4l2deviceprovider (GstDeviceProviderFactory)
video4linux2: v4l2convert: V4L2 Video Converter
video4linux2: v4l2jpegenc: V4L2 JPEG Encoder
video4linux2: v4l2h264enc: V4L2 H.264 Encoder
video4linux2: v4l2mpeg4enc: V4L2 MPEG4 Encoder
video4linux2: v4l2mpeg4dec: V4L2 MPEG4 Decoder
video4linux2: v4l2mpeg2dec: V4L2 MPEG2 Decoder
video4linux2: v4l2h264dec: V4L2 H264 Decoder


Mark Farver
 

On Sun, Apr 5, 2020 at 1:19 AM Carlos Rafael Giani <crg7475@...> wrote:
For using etnaviv, just use glimagesink. For using the VPU, use the v4l2*dec elements.
Definitely doing CPU decoding. I don't see any kernel messages that
indicate the VPU has been detected even though I have
CONFIG_VIDEO_IMX_PXP=y in the kernel config. Is that the correct
option? Are there others? The device tree entry for the VPU seems to
be included in the kernel configuration by default.

I grabbed OSSystems meta-gstreamer layer to get 1.16.2, but even with
that included I do not appear to get any v4l2 decode plugins:

# gst-inspect-1.0 --version
gst-inspect-1.0 version 1.16.2
GStreamer 1.16.2
Unknown package origi

# gst-inspect-1.0 |grep v4l
imxv4l2video: imxv4l2videosrc: V4L2 CSI Video Source
imxv4l2video: imxv4l2videosink: V4L2 CSI Video Sink
video4linux2: v4l2src: Video (video4linux2) Source
video4linux2: v4l2sink: Video (video4linux2) Sink
video4linux2: v4l2radio: Radio (video4linux2) Tuner
video4linux2: v4l2deviceprovider (GstDeviceProviderFactory)

Thank you
Mark Farver


Carlos Rafael Giani
 

etnaviv takes care of GPU support, which is related to scaling, but unrelated to video decoding - the latter one is handled by the CODA960 VPU.

For using etnaviv, just use glimagesink. For using the VPU, use the v4l2*dec elements.

On 05.04.20 01:31, Mark Farver wrote:
Does anyone know definitively if etnaviv/gstreamer has support for
accelerated h264 decoding/rescaling?  I can play video through both
the fbcondev and wayland sinks, but both consume a great deal of CPU,
and frame dropping gets pretty bad if I attempt to resize the video.

I need to know if I should continue pursuing etnaviv, or if it is a
dead end for now and I need to fall back to using the vivante driver.


    


Mark Farver
 

Does anyone know definitively if etnaviv/gstreamer has support for
accelerated h264 decoding/rescaling? I can play video through both
the fbcondev and wayland sinks, but both consume a great deal of CPU,
and frame dropping gets pretty bad if I attempt to resize the video.

I need to know if I should continue pursuing etnaviv, or if it is a
dead end for now and I need to fall back to using the vivante driver.