IMX6 Video processing build

Terry Barnaby

I wonder if anyone has knowledge of how to build a Yocto image with full iMX video processing support for an iMX6 ? I am going around in circles of failed Yocto builds and/or images that don't appear to support the full video processing abilities.

I am trying to test the video processing abilities of the iMX6, and later the iMX8 SOC's. The video processing pipeline needs to use the MIPI-CSI 1920x1080p30 camera input, the IPU for overlaying a generated image and sending the video stream between processing blocks to reduce memory access, the VPU for H264 encoding and the GPU for rendering the camera video stream with overlay onto a LCD and/or HDMI display with GUI.

It looks like the iMX6DL and some of the newer iMX8's like iMX8M-Plus and iMX8DualXPlus should be able to do most of this in hardware with little CPU usage and without too much memory bandwidth used. But it is difficult to know for sure, hence the testing before we choose a suitable SOC and design our hardware boards.

I am trying to use gstreamer1.0 to perform these tests.

One of our test boards is a Wandboard WB-IMX6U-BW iMX6DL. There was a binary Yocto 2.0 build for this that we have been using, but it is quite old and there appears to be video processing issues so I have been trying to build a more recent Yocto build using 3.1 dunfell. I am still gaining knowledge of the iMX6 with Yocto builds but after many many builds I am still not getting anywhere to obtaining a build with a fully working video processing pipeline. The Yocto 2.0 does allow the use of the iMX IPU/VPU and GPU via gstreamer with commands like:

gst-launch-1.0 -v compositor name=comp ! queue ! vpuenc_h264 ! qtmux ! filesink location=temp.mp4 \

    overlaysink imxv4l2src device=/dev/video0 ! video/x-raw,width=1920,height=1080,framerate=30/1 ! c.sink_0 \

    multifilesrc location=./test1.png caps=image/png,framerate=1/1 ! pngdec ! imxvideoconvert_ipu ! c.sink_1

Which uses around 11% of one of the CPU's.

I have built (amongst many other tries):

BRANCH=dunfel, MACHINE=wandboard, DISTRO=fslc-x11, IMX_DEFAULT_BSP="mainline", kernel-fslc, bitbake core-image-sato

This builds and runs (Need to use a SPL and uboot.img built manually from uboot git master branch) and displays a GUI.

This does have hardware h264 encode via the gstreamer v4l2h264enc element, but the ov56545 camera, although found, does not work with the v4l2src gstreamer element producing a Buffer error messages. Also there are no imxcompositor_ipu, imxcompositor_g2d or other such iMX video elements available.

It appears that with this IMX_DEFAULT_BSP="mainline" build, MACHINEOVERRIDES loses the "imx6dl" and other such compatible "machines" so that I cannot build gstreamer1.0-plugins-imx.

I have tried:

BRANCH=dunfel, MACHINE=wandboard, DISTRO=fslc-x11, IMX_DEFAULT_BSP="nxp", kernel-fslc-imx, bitbake core-image-sato

This failed to build mesa. I added "libdrm" to X11_DEPS to allow this to build, not sure this is correct.

This builds and runs (Need to use a SPL and uboot.img built manually from uboot git master branch) but no GUI. There doesn't seem to be a /dev/fb0 device and no mention in dmesg of the HDMI interface.

I have tried many many other options including the linux-fslc kernel with IMX_DEFAULT_BSP="nxp", fails to build, linux-imx kernel no GUI, DISTRO='fslc-xwayland', bitbake core-image-weston, bitbake fsl-image-multimedia-full etc etc. But am getting nowhere.

1. I presume I do need gstreamer1.0-plugins-imx to support the iMX6 IPU's image conversion/composing and internal data paths ?

2. I am unclear what the IMX_DEFAULT_BSP="nxp" actually does and why "imx6dl" is removed from MACHINEOVERRIDES when building with IMX_DEFAULT_BSP="mainline" ?

3. Any pointers on what I should be doing to get a Yocto build with full video processing support ?

Any ideas gratefully received !