linux-imx-headers and ioctl mismatches


Carlos Rafael Giani
 

The linux-imx-headers recipe gathers certain headers from the kernel source so that other recipes which do access these headers don't have to mark the entire kernel as a dependency.

However, if a platform doesn't actually use linux-imx as its kernel, or at least uses a kernel release version that differs from that of linux-imx, ioctl mismatches can occur.

Current example: linux-imx is at version 4.19 , while linux-boundary is at 4.14. The DMA_BUF_IOCTL_PHYS ioctl is defined in the uapi/dma-buf.h header in 4.14 as:

  #define DMA_BUF_IOCTL_PHYS      _IOW(DMA_BUF_BASE, 1, struct dma_buf_phys)

but in 4.19 as:

  #define DMA_BUF_IOCTL_PHYS      _IOW(DMA_BUF_BASE, 10, struct dma_buf_phys)

This leads to Boundary Devices platforms that can't handle this ioctl properly because of the definition mismatch.

Just mentioning this here to discuss ideas. Perhaps it would make sense to create a virtual/imx-headers package, or place the git repo URL as a configurable variable?


Otavio Salvador
 

On Fri, Apr 10, 2020 at 7:19 AM Carlos Rafael Giani <crg7475@...> wrote:
...
Just mentioning this here to discuss ideas. Perhaps it would make sense
to create a virtual/imx-headers package, or place the git repo URL as a
configurable variable?
This is not an easy problem. The issue is kinda similar of Linux ABI
and we can't use two different versions on same binary feed so we'd
need to mark it all as MACHINE_ARCH.

The easiest solution I can think of is do a backport of this change to
linux-boundary or they bump their kernel version.


--
Otavio Salvador O.S. Systems
http://www.ossystems.com.br http://code.ossystems.com.br
Mobile: +55 (53) 9 9981-7854 Mobile: +1 (347) 903-9750


Carlos Rafael Giani
 

Hm I thought there was some sort of imx-kernel base class. Apparently there isn't. If there were, it could then be merged with the linux-imx-headers recipe, and perhaps create something like a -dev package.

But then again, a -dev package from a kernel recipe ...? Kind of weird.

On 10.04.20 16:12, Otavio Salvador wrote:
On Fri, Apr 10, 2020 at 7:19 AM Carlos Rafael Giani <crg7475@...> wrote:
...
Just mentioning this here to discuss ideas. Perhaps it would make sense
to create a virtual/imx-headers package, or place the git repo URL as a
configurable variable?
This is not an easy problem. The issue is kinda similar of Linux ABI
and we can't use two different versions on same binary feed so we'd
need to mark it all as MACHINE_ARCH.

The easiest solution I can think of is do a backport of this change to
linux-boundary or they bump their kernel version.


Otavio Salvador
 

On Fri, Apr 10, 2020 at 11:20 AM Carlos Rafael Giani
<crg7475@...> wrote:
Hm I thought there was some sort of imx-kernel base class. Apparently
there isn't. If there were, it could then be merged with the
linux-imx-headers recipe, and perhaps create something like a -dev package.

But then again, a -dev package from a kernel recipe ...? Kind of weird.
The problem is that you cannot mix packages / apps with different
versions of this. This ends being part of the binary ABI.

--
Otavio Salvador O.S. Systems
http://www.ossystems.com.br http://code.ossystems.com.br
Mobile: +55 (53) 9 9981-7854 Mobile: +1 (347) 903-9750


Otavio Salvador
 

On Fri, Apr 10, 2020 at 11:46 AM Gary Bisson
<gary.bisson@...> wrote:
On Fri, Apr 10, 2020 at 11:22:45AM -0300, Otavio Salvador wrote:
On Fri, Apr 10, 2020 at 11:20 AM Carlos Rafael Giani
<crg7475@...> wrote:
Hm I thought there was some sort of imx-kernel base class. Apparently
there isn't. If there were, it could then be merged with the
linux-imx-headers recipe, and perhaps create something like a -dev package.

But then again, a -dev package from a kernel recipe ...? Kind of weird.
The problem is that you cannot mix packages / apps with different
versions of this. This ends being part of the binary ABI.
I understand it's a tricky topic as some packages won't build if the
kernel headers are not aligned. But I'm not sure it is better to have
packages building but having runtime issues because of mismatch...

Since our kernel is also used on other OSes we will not update its
headers. It also feels wrong to do so.
Add a patch to the layer for it.

Otavio, do you expect all the vendors to have their kernel matching
exactly the linux-imx-headers version?
Because looking at meta-freescale-3rdparty master branch [1], some
kernels are still based on 3.14...
and it is vendors consideration for their users. If they don't care,
who should care? We are starting removing broken boards and vendors
should pay attention to users.

There are two routes here:

- use mainline BSP
- use NXP BSP

for NXP BSP it comes with the need to keep upgrading it.


I personnaly think that the headers should be installed from the kernel
being built, then it's up to the vendor to check the compatibility but
at least that would fix the issue Carlos is seeing.
This does not help; those headers should not exists in first case but
as we do, a common base need to be used for all distro. Their symbols
goes to other libraries and apps so it cannot change between machines.

FYI, our next kernel release will be 5.4.3_1.0.0 as it is out since last
week. So if master stays with 4.19 we'll still have a mismatch I guess.
Will be going to 5.4 soon and people can help, patches welcome.

--
Otavio Salvador O.S. Systems
http://www.ossystems.com.br http://code.ossystems.com.br
Mobile: +55 (53) 9 9981-7854 Mobile: +1 (347) 903-9750


Kevin Lannen
 

Hi Otavio,

I would be happy to help out some with moving linux-imx to the 5.4 kernel as we are very interested in using it. Is this planned to be in for the Dunfell branch when that is released?

Regards,


Kevin Lannen

Embedded Systems Engineer
kevin@...
970-690-8619

-----Original Message-----
From: meta-freescale@... <meta-freescale@...> On Behalf Of Otavio Salvador via lists.yoctoproject.org
Sent: Friday, April 10, 2020 11:52 AM
To: Gary Bisson <gary.bisson@...>
Cc: Carlos Rafael Giani <crg7475@...>; meta-freescale <meta-freescale@...>; Chris Dimich <chris.dimich@...>
Subject: Re: [meta-freescale] linux-imx-headers and ioctl mismatches

On Fri, Apr 10, 2020 at 11:46 AM Gary Bisson <gary.bisson@...> wrote:
On Fri, Apr 10, 2020 at 11:22:45AM -0300, Otavio Salvador wrote:
On Fri, Apr 10, 2020 at 11:20 AM Carlos Rafael Giani
<crg7475@...> wrote:
Hm I thought there was some sort of imx-kernel base class.
Apparently there isn't. If there were, it could then be merged
with the linux-imx-headers recipe, and perhaps create something like a -dev package.

But then again, a -dev package from a kernel recipe ...? Kind of weird.
The problem is that you cannot mix packages / apps with different
versions of this. This ends being part of the binary ABI.
I understand it's a tricky topic as some packages won't build if the
kernel headers are not aligned. But I'm not sure it is better to have
packages building but having runtime issues because of mismatch...

Since our kernel is also used on other OSes we will not update its
headers. It also feels wrong to do so.
Add a patch to the layer for it.

Otavio, do you expect all the vendors to have their kernel matching
exactly the linux-imx-headers version?
Because looking at meta-freescale-3rdparty master branch [1], some
kernels are still based on 3.14...
and it is vendors consideration for their users. If they don't care, who should care? We are starting removing broken boards and vendors should pay attention to users.

There are two routes here:

- use mainline BSP
- use NXP BSP

for NXP BSP it comes with the need to keep upgrading it.


I personnaly think that the headers should be installed from the
kernel being built, then it's up to the vendor to check the
compatibility but at least that would fix the issue Carlos is seeing.
This does not help; those headers should not exists in first case but as we do, a common base need to be used for all distro. Their symbols goes to other libraries and apps so it cannot change between machines.

FYI, our next kernel release will be 5.4.3_1.0.0 as it is out since
last week. So if master stays with 4.19 we'll still have a mismatch I guess.
Will be going to 5.4 soon and people can help, patches welcome.

--
Otavio Salvador O.S. Systems
http://www.ossystems.com.br http://code.ossystems.com.br
Mobile: +55 (53) 9 9981-7854 Mobile: +1 (347) 903-9750


Otavio Salvador
 

On Fri, Apr 10, 2020 at 3:25 PM Kevin Lannen <kevin@...> wrote:
I would be happy to help out some with moving linux-imx to the 5.4 kernel as we are very interested in using it. Is this planned to be in for the Dunfell branch when that is released?
It all depends how long it takes. Dunfell is an important release and
as such we ought to try to have it using new components.

We need someone to help updating NXP components; I am focusing more on
mainline BSP support since I've been doing this on our free time.

--
Otavio Salvador O.S. Systems
http://www.ossystems.com.br http://code.ossystems.com.br
Mobile: +55 (53) 9 9981-7854 Mobile: +1 (347) 903-9750


Kevin Lannen
 

I would be happy to help out some with moving linux-imx to the 5.4 kernel as we are very interested in using it. Is this planned to be in for the Dunfell branch when that is released?
It all depends how long it takes. Dunfell is an important release and as such we ought to try to have it using new components.
We need someone to help updating NXP components; I am focusing more on mainline BSP support since I've been doing this on our free time.
Is there a github issue or a list somewhere tracking what needs to be done for this?


Kevin Lannen

Embedded Systems Engineer
kevin@...
970-690-8619


Otavio Salvador
 

On Fri, Apr 10, 2020 at 3:43 PM Kevin Lannen <kevin@...> wrote:
I would be happy to help out some with moving linux-imx to the 5.4 kernel as we are very interested in using it. Is this planned to be in for the Dunfell branch when that is released?
It all depends how long it takes. Dunfell is an important release and as such we ought to try to have it using new components.
We need someone to help updating NXP components; I am focusing more on mainline BSP support since I've been doing this on our free time.
Is there a github issue or a list somewhere tracking what needs to be done for this?
No, even though we can open. Generally we need to update:

- linux-imx
- u-boot-imx
- headers
- gpu drivers
- media codecs
- gst

--
Otavio Salvador O.S. Systems
http://www.ossystems.com.br http://code.ossystems.com.br
Mobile: +55 (53) 9 9981-7854 Mobile: +1 (347) 903-9750


Carlos Rafael Giani
 

This issue came up when I noticed that the ION and DWL allocators in libimxdmabuffer failed. (DWL is an API from the Hantro VPU libraries, and it fails, because internally it itself uses ION.)

This is the relevant code: https://github.com/Freescale/libimxdmabuffer/blob/master/imxdmabuffer/imxdmabuffer_ion_allocator.c#L428

I am now wondering if I can add some workaround to also support older kernels that are subject to such a header mismatch. An ugly workaround would be to manually define the IOCTL, like this:

  #if LINUX_VERSION_CODE < KERNEL_VERSION(4, 19, 0)
  #define CUSTOM_DMA_BUF_IOCTL_PHYS      _IOW(DMA_BUF_BASE, 1, struct dma_buf_phys)
  #else
  #define CUSTOM_DMA_BUF_IOCTL_PHYS      DMA_BUF_IOCTL_PHYS
  #endif

And then use the CUSTOM_DMA_BUF_IOCTL_PHYS instead of the regular DMA_BUF_IOCTL_PHYS in the code.

Pro: This would make libimxdmabuffer work with existing setups, and could even be backported to zeus and warrior.

Con: Defining IOCTLs in userspace like this is usually not a good idea.

Comments?


On 10.04.20 16:22, Otavio Salvador wrote:

On Fri, Apr 10, 2020 at 11:20 AM Carlos Rafael Giani
<crg7475@...> wrote:
Hm I thought there was some sort of imx-kernel base class. Apparently
there isn't. If there were, it could then be merged with the
linux-imx-headers recipe, and perhaps create something like a -dev package.

But then again, a -dev package from a kernel recipe ...? Kind of weird.
The problem is that you cannot mix packages / apps with different
versions of this. This ends being part of the binary ABI.



    


Otavio Salvador
 

Carlos,

On Sat, Apr 11, 2020 at 7:32 AM Carlos Rafael Giani <crg7475@...> wrote:
This issue came up when I noticed that the ION and DWL allocators in libimxdmabuffer failed. (DWL is an API from the Hantro VPU libraries, and it fails, because internally it itself uses ION.)

This is the relevant code: https://github.com/Freescale/libimxdmabuffer/blob/master/imxdmabuffer/imxdmabuffer_ion_allocator.c#L428

I am now wondering if I can add some workaround to also support older kernels that are subject to such a header mismatch. An ugly workaround would be to manually define the IOCTL, like this:

#if LINUX_VERSION_CODE < KERNEL_VERSION(4, 19, 0)
#define CUSTOM_DMA_BUF_IOCTL_PHYS _IOW(DMA_BUF_BASE, 1, struct dma_buf_phys)
#else
#define CUSTOM_DMA_BUF_IOCTL_PHYS DMA_BUF_IOCTL_PHYS
#endif

And then use the CUSTOM_DMA_BUF_IOCTL_PHYS instead of the regular DMA_BUF_IOCTL_PHYS in the code.
This does not work as the libimxdmabuffer is shared (and it should be)
across multiple machines of same SoC family. Considering that, the
best choice is to apply a patch to old Linux kernels to make it to use
the newer API/ABI with this backport.

--
Otavio Salvador O.S. Systems
http://www.ossystems.com.br http://code.ossystems.com.br
Mobile: +55 (53) 9 9981-7854 Mobile: +1 (347) 903-9750


Andrey Zhizhikin
 

On Fri, Apr 10, 2020 at 8:46 PM Otavio Salvador
<otavio.salvador@...> wrote:

On Fri, Apr 10, 2020 at 3:43 PM Kevin Lannen <kevin@...> wrote:
I would be happy to help out some with moving linux-imx to the 5.4 kernel as we are very interested in using it. Is this planned to be in for the Dunfell branch when that is released?
It all depends how long it takes. Dunfell is an important release and as such we ought to try to have it using new components.
We need someone to help updating NXP components; I am focusing more on mainline BSP support since I've been doing this on our free time.
Is there a github issue or a list somewhere tracking what needs to be done for this?
No, even though we can open. Generally we need to update:

- linux-imx
- u-boot-imx
- headers
- gpu drivers
- media codecs
- gst
I'm currently working on this one, primarily for imx8mm and imx8mn
machines. So far, I have the following list I've noted for myself:
- linux-imx (with mfgtool)
- u-boot-imx
- linux-imx-headers
- kernel-module-imx-gpu-viv (gpu drivers)
- kernel-module-qca9377 (wlan drivers)
- imx-atf
- imx-mkimage
- imx-test
- optee-imx (client, os, test)
- media codecs
- gst (base, good, bad, gstreamer and imx-gst)

I'm almost through with it for the Mini, then would cross-check if
everything compiles for Nano. I would send a PR once both validations
are done.

Otavio,
One general question: Do we keep the 4.19.35 in the layer, or I can
simply drop it and introduce the 5.4.3? So far, I did just that and it
seems to be going OK.


--
Otavio Salvador O.S. Systems
http://www.ossystems.com.br http://code.ossystems.com.br
Mobile: +55 (53) 9 9981-7854 Mobile: +1 (347) 903-9750


--
Regards,
Andrey.


Otavio Salvador
 

On Sat, Apr 11, 2020 at 7:05 PM Andrey Zhizhikin <andrey.z@...> wrote:
On Fri, Apr 10, 2020 at 8:46 PM Otavio Salvador
<otavio.salvador@...> wrote:

On Fri, Apr 10, 2020 at 3:43 PM Kevin Lannen <kevin@...> wrote:
I would be happy to help out some with moving linux-imx to the 5.4 kernel as we are very interested in using it. Is this planned to be in for the Dunfell branch when that is released?
It all depends how long it takes. Dunfell is an important release and as such we ought to try to have it using new components.
We need someone to help updating NXP components; I am focusing more on mainline BSP support since I've been doing this on our free time.
Is there a github issue or a list somewhere tracking what needs to be done for this?
No, even though we can open. Generally we need to update:

- linux-imx
- u-boot-imx
- headers
- gpu drivers
- media codecs
- gst
I'm currently working on this one, primarily for imx8mm and imx8mn
machines. So far, I have the following list I've noted for myself:
- linux-imx (with mfgtool)
- u-boot-imx
- linux-imx-headers
- kernel-module-imx-gpu-viv (gpu drivers)
- kernel-module-qca9377 (wlan drivers)
- imx-atf
- imx-mkimage
- imx-test
- optee-imx (client, os, test)
- media codecs
- gst (base, good, bad, gstreamer and imx-gst)

I'm almost through with it for the Mini, then would cross-check if
everything compiles for Nano. I would send a PR once both validations
are done.

Otavio,
One general question: Do we keep the 4.19.35 in the layer, or I can
simply drop it and introduce the 5.4.3? So far, I did just that and it
seems to be going OK.
Please drop.


--
Otavio Salvador O.S. Systems
http://www.ossystems.com.br http://code.ossystems.com.br
Mobile: +55 (53) 9 9981-7854 Mobile: +1 (347) 903-9750