Kernel Header UAPI Issue


Karthik Poduval
 

I have a 5.4 kernel where I applied a patch (using a .scc file) from
kernel 5.6 which brings in dmabuf heaps.

The patch exports a uapi header
include/uapi/linux/dma-heap.h.
I am trying to write an application that uses dmabuf with a recipe (no
special DEPENDS).

It is able to find
#include<linux/dma-buf.h>
but not
#include<linux/dma-heap.h> (this file is introduced in the patch)

I looked at the recipe's sysroot
recipe-sysroot/usr/linux/include
and dma-buf.h is there but not dma-heap.h

Why is the header not being imported ?

--
Regards,
Karthik Poduval


Khem Raj
 

On Mon, Feb 22, 2021 at 4:52 PM Karthik Poduval
<karthik.poduval@gmail.com> wrote:

I have a 5.4 kernel where I applied a patch (using a .scc file) from
kernel 5.6 which brings in dmabuf heaps.

The patch exports a uapi header
include/uapi/linux/dma-heap.h.
I am trying to write an application that uses dmabuf with a recipe (no
special DEPENDS).

It is able to find
#include<linux/dma-buf.h>
but not
#include<linux/dma-heap.h> (this file is introduced in the patch)

I looked at the recipe's sysroot
recipe-sysroot/usr/linux/include
and dma-buf.h is there but not dma-heap.h

Why is the header not being imported ?
What does patch look like? just adding the file is not enough, it
should be added to makefiles to be exported as UAPI header

--
Regards,
Karthik Poduval



Bruce Ashfield
 

On Mon, Feb 22, 2021 at 7:52 PM Karthik Poduval
<karthik.poduval@gmail.com> wrote:

I have a 5.4 kernel where I applied a patch (using a .scc file) from
kernel 5.6 which brings in dmabuf heaps.

The patch exports a uapi header
include/uapi/linux/dma-heap.h.
I am trying to write an application that uses dmabuf with a recipe (no
special DEPENDS).

It is able to find
#include<linux/dma-buf.h>
but not
#include<linux/dma-heap.h> (this file is introduced in the patch)

I looked at the recipe's sysroot
recipe-sysroot/usr/linux/include

That's coming from the linux-libc-headers, your new upi header is not
part of the libc-headers package, so it won't show up there. And no,
it shouldn't be part of that recipe and the libc-headers should not be
generated from the current state of whatever kernel is being built.

You need to arrange for it to be exported and staged separately (look
for examples of 'sysroot_stage' variants in the meta-data, you can
bbappend the behaviour to your kernel recipe), or your code can look
for the header in the kernel shared workdir (STAGING_KERNEL_BUILDDIR)
by adding that to your include paths in the recipe.

As khem also mentioned, you'll need to make sure that the uapi header
is not just patched in, but properly exported.

Bruce


and dma-buf.h is there but not dma-heap.h

Why is the header not being imported ?

--
Regards,
Karthik Poduval



--
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II


Karthik Poduval
 

Hi Khem and Bruce,

I got the patch from raspberry pi github and it looked like it put the header into include/uapi/linux/dma-heap.h, for kernel 5.4 it doesn't look like there is a need for a Kbuild file. Are any special steps needed for exporting the header (I thought kernel 5.4 exports everything from include/uapi directory) ? The  linux-libc-headers sound like the issue. Thanks for the pointers, I will dig up further based on the suggestions and revert.

On Mon, Feb 22, 2021 at 5:14 PM Bruce Ashfield <bruce.ashfield@...> wrote:
On Mon, Feb 22, 2021 at 7:52 PM Karthik Poduval
<karthik.poduval@...> wrote:
>
> I have a 5.4 kernel where I applied a patch (using a .scc file) from
> kernel 5.6 which brings in dmabuf heaps.
>
> The patch exports a uapi header
> include/uapi/linux/dma-heap.h.
> I am trying to write an application that uses dmabuf with a recipe (no
> special DEPENDS).
>
> It is able to find
> #include<linux/dma-buf.h>
> but not
> #include<linux/dma-heap.h> (this file is introduced in the patch)
>
> I looked at the recipe's sysroot
> recipe-sysroot/usr/linux/include


That's coming from the linux-libc-headers, your new upi header is not
part of the libc-headers package, so it won't show up there. And no,
it shouldn't be part of that recipe and the libc-headers should not be
generated from the current state of whatever kernel is being built.

You need to arrange for it to be exported and staged separately (look
for examples of 'sysroot_stage' variants in the meta-data, you can
bbappend the behaviour to your kernel recipe), or your code can look
for the header in the kernel shared workdir (STAGING_KERNEL_BUILDDIR)
by adding that to your include paths in the recipe.

As khem also mentioned, you'll need to make sure that the uapi header
is not just patched in, but properly exported.

Bruce

>
> and  dma-buf.h is there but not  dma-heap.h
>
> Why is the header not being imported ?
>
> --
> Regards,
> Karthik Poduval
>
>
>


--
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II


--
Regards,
Karthik Poduval


Mikko Rapeli
 

Hi,

I know it's not the best or recommended approach, but I find it
hard to avoid merging linux-libc-headers recipe with the actual
kernel recipe that a distro is using. At least a static copy
of some version of uapi headers from that kernel can be used
instead of the poky side linux-libc-headers. This helps to get
the actual BSP SW delivery headers into userspace, SDK etc.

Cheers,

-Mikko


Karthik Poduval
 

Hi Mikko,

Do you have an example on how you do that ? Do you bbapend the
linux-libc-headers recipe file ?
I have an application that uses dmabuf heap that potentially extends
across multiple BSP's as its BSP agnostic. I don't want to be patching
individual BSP recipes and generating headers. The issue I am facing
is due to backporting the patch from 5.6 to 5.4 so the required header
isn't a part of the linux-libc-headers.bb recipe. Best would have been
a virtual/kernel-keaders target that applications that require BSP
headers would add to their recipe DEPENDS. Why is this not a solved
issue by yocto project ? Why do individual BSP's need to deal with
this differently when the header install mechanism (make
headers_install) is the same irrespective of the type of BSP ?

On Tue, Feb 23, 2021 at 5:18 AM <Mikko.Rapeli@bmw.de> wrote:

Hi,

I know it's not the best or recommended approach, but I find it
hard to avoid merging linux-libc-headers recipe with the actual
kernel recipe that a distro is using. At least a static copy
of some version of uapi headers from that kernel can be used
instead of the poky side linux-libc-headers. This helps to get
the actual BSP SW delivery headers into userspace, SDK etc.

Cheers,

-Mikko
--
Regards,
Karthik Poduval


Mikko Rapeli
 

Hi,

On Tue, Feb 23, 2021 at 06:56:02AM -0800, Karthik Poduval wrote:
Hi Mikko,

Do you have an example on how you do that ? Do you bbapend the
linux-libc-headers recipe file ?
A kernel recipe will provide linux-libc-headers after a
"make headers_install" call...

So the SRC_URI of linux-libc-headers can be the same as from the
linux kernel recipe, or linux kernel recipe can provide (with some
tricks, possibly) the linux-libc-headers binary packages.

I have an application that uses dmabuf heap that potentially extends
across multiple BSP's as its BSP agnostic. I don't want to be patching
individual BSP recipes and generating headers. The issue I am facing
is due to backporting the patch from 5.6 to 5.4 so the required header
isn't a part of the linux-libc-headers.bb recipe. Best would have been
a virtual/kernel-keaders target that applications that require BSP
headers would add to their recipe DEPENDS. Why is this not a solved
issue by yocto project ? Why do individual BSP's need to deal with
this differently when the header install mechanism (make
headers_install) is the same irrespective of the type of BSP ?
I guess the reason is to split userspace to BSP/board specific and common
parts. It's not good if whole userspace bootstrap depends on kernel
recipe and any kernel changes requires full bootstrap and recompilation
of all userspace.

But there are BSP/board specific recipes which do need the real effective
kernel headers to interface with kernel drivers, and there are SoC
architecture specific ones which just need enough to build gcc and glibc.

Both have their pross and cons, and to me there is no clear winner.

Cheers,

-Mikko


Bruce Ashfield
 

On Tue, Feb 23, 2021 at 9:56 AM Karthik Poduval
<karthik.poduval@gmail.com> wrote:

Hi Mikko,

Do you have an example on how you do that ? Do you bbapend the
linux-libc-headers recipe file ?
I have an application that uses dmabuf heap that potentially extends
across multiple BSP's as its BSP agnostic. I don't want to be patching
individual BSP recipes and generating headers. The issue I am facing
is due to backporting the patch from 5.6 to 5.4 so the required header
isn't a part of the linux-libc-headers.bb recipe. Best would have been
a virtual/kernel-keaders target that applications that require BSP
headers would add to their recipe DEPENDS. Why is this not a solved
issue by yocto project ? Why do individual BSP's need to deal with
this differently when the header install mechanism (make
headers_install) is the same irrespective of the type of BSP ?
Because it's not a simple thing to solve (and there's a bugzilla around for it
already, where the thoughts and issues are captured, but that one seems to
have been closed and I can't find it at the moment). I do have a WIP
class that provides some different modes
(https://bugzilla.yoctoproject.org/show_bug.cgi?id=5305),
but with corner cases and concerns, it keeps slipping. Feel free to try it
out and comment in the bug (I'll try myself to be sure it still applied, it has
been a few months).

What works for your case, doesn't mean it is a general/supporable
solution.

But generally speaking, It's an incredibly bad idea to have your libc-headers
tied to the kernel you are building. Every time that kernel changes, you
basically need to rebuild the entire stack .. hence the bad idea. It is such
a common question, that we actually put a warning in the libc-headers
recipe itself.

We do not really want a parallel set of headers in the shared workdir, that are
from the currently built kernel. You'd end up with all sorts of mismatches
and cross includes and potential different behaviour per-application.

We already have the kernel source installed into the shared workdir,
which is what the tips in the libc-headers recipe suggest using. And it
honestly isn't so common the need for sanitized headers that the need for
something like I did in that bug has made it necessary.

Bruce


On Tue, Feb 23, 2021 at 5:18 AM <Mikko.Rapeli@bmw.de> wrote:

Hi,

I know it's not the best or recommended approach, but I find it
hard to avoid merging linux-libc-headers recipe with the actual
kernel recipe that a distro is using. At least a static copy
of some version of uapi headers from that kernel can be used
instead of the poky side linux-libc-headers. This helps to get
the actual BSP SW delivery headers into userspace, SDK etc.

Cheers,

-Mikko


--
Regards,
Karthik Poduval

--
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II


Karthik Poduval
 

Hi Bruce,

Thanks a lot for your patch (attached in bugzilla
https://bugzilla.yoctoproject.org/show_bug.cgi?id=5305). I am
summarizing all steps once again for the benefit of anyone else who
faces the same issue (including myself in the future) and may use this
email list as a reference. The basic need was to export kernel headers
from kernel source for an application recipe (those headers are not a
part of linux-libc-headers recipe).

I applied the patch pointed by Bruce
https://bugzilla.yoctoproject.org/attachment.cgi?id=4647 to my local
BSP layer. The patch has 2 modes of operation (described in the patch
documentation).

mode1: To make this work I added the following lines to my application recipe.

inherit cmake kernel-alt-headers
DEPENDS = "gtest rsync-native"
EXTRA_OECMAKE =
'-DKERNEL_HEADER_DIR:STRING=${WORKDIR}/${KERNEL_ALT_HEADER_DIR}/include'
Then in my CMakeLists.txt.
include_directories(${KERNEL_HEADER_DIR})

The application compiled fine but there was one side effect that
kernel menuconfig (bitbake virtual/kernel -c menuconfig) wasn't able
to run as it complained that the source tree wasn't clean and make
mrproper was needed. This was resolved by a simple fix to the
do_compile_prepend in the patch (added the mrproper command).
do_compile_prepend() {
if [ "${KERNEL_SOURCE_IS_LOCAL}" = "False" ]; then
# install from the staging kernel directory
oe_runmake -C ${STAGING_KERNEL_DIR} headers_install
INSTALL_HDR_PATH=${WORKDIR}/${KERNEL_ALT_HEADER_DIR}
oe_runmake -C ${STAGING_KERNEL_DIR} mrproper
fi
}

mode2: To make this work I added the following line to my kernel recipe.
inherit kernel-alt-headers

to my application recipe following lines were added.
DEPENDS += "virtual/kernel"
EXTRA_OECMAKE =
'-DKERNEL_HEADER_DIR:STRING=${STAGING_DIR_TARGET}/usr/alt-headers/include'
Then in my CMakeLists.txt.
include_directories(${KERNEL_HEADER_DIR})

--
Regards,
Karthik Poduval

On Tue, Feb 23, 2021 at 10:50 AM Bruce Ashfield
<bruce.ashfield@gmail.com> wrote:

On Tue, Feb 23, 2021 at 9:56 AM Karthik Poduval
<karthik.poduval@gmail.com> wrote:

Hi Mikko,

Do you have an example on how you do that ? Do you bbapend the
linux-libc-headers recipe file ?
I have an application that uses dmabuf heap that potentially extends
across multiple BSP's as its BSP agnostic. I don't want to be patching
individual BSP recipes and generating headers. The issue I am facing
is due to backporting the patch from 5.6 to 5.4 so the required header
isn't a part of the linux-libc-headers.bb recipe. Best would have been
a virtual/kernel-keaders target that applications that require BSP
headers would add to their recipe DEPENDS. Why is this not a solved
issue by yocto project ? Why do individual BSP's need to deal with
this differently when the header install mechanism (make
headers_install) is the same irrespective of the type of BSP ?
Because it's not a simple thing to solve (and there's a bugzilla around for it
already, where the thoughts and issues are captured, but that one seems to
have been closed and I can't find it at the moment). I do have a WIP
class that provides some different modes
(https://bugzilla.yoctoproject.org/show_bug.cgi?id=5305),
but with corner cases and concerns, it keeps slipping. Feel free to try it
out and comment in the bug (I'll try myself to be sure it still applied, it has
been a few months).

What works for your case, doesn't mean it is a general/supporable
solution.

But generally speaking, It's an incredibly bad idea to have your libc-headers
tied to the kernel you are building. Every time that kernel changes, you
basically need to rebuild the entire stack .. hence the bad idea. It is such
a common question, that we actually put a warning in the libc-headers
recipe itself.

We do not really want a parallel set of headers in the shared workdir, that are
from the currently built kernel. You'd end up with all sorts of mismatches
and cross includes and potential different behaviour per-application.

We already have the kernel source installed into the shared workdir,
which is what the tips in the libc-headers recipe suggest using. And it
honestly isn't so common the need for sanitized headers that the need for
something like I did in that bug has made it necessary.

Bruce


On Tue, Feb 23, 2021 at 5:18 AM <Mikko.Rapeli@bmw.de> wrote:

Hi,

I know it's not the best or recommended approach, but I find it
hard to avoid merging linux-libc-headers recipe with the actual
kernel recipe that a distro is using. At least a static copy
of some version of uapi headers from that kernel can be used
instead of the poky side linux-libc-headers. This helps to get
the actual BSP SW delivery headers into userspace, SDK etc.

Cheers,

-Mikko


--
Regards,
Karthik Poduval

--
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II