Building and selecting multiple kernel versions


Ryan Harkin <ryan.harkin@...>
 

Hello all,

I'm looking for advice on how to support multiple kernel versions in my distro with minimal changes, and minimal disruption to my users.

Some background:

I have a legacy Sumo based distro with an image config, and a machine config, with the machine using a 4.9 kernel. Last year, I upgraded everything to Warrior, and moved to a 4.19 kernel.

Some of my users who are migrating from Sumo, wish to keep their 4.9 kernel. So I'm trying to work out how to handle this in the simplest way.

I know that I can add a 4.9 recipes to my Warrior branches, set PREFERRED_VERSION in my distro.conf. But I don't want to change the default preferred version globally. And I don't really want users to update the distro.conf. Ideally, they should be able to take what I give them and "just" build it to get a 4.9 or 4.19 variant.

Ideally, I don't want to *have* to build both kernels and then create two images. I expect that will cause confusion and lead to people flashing the wrong images. So I'd prefer it to be either/or. Of course, I have to test all of this, so I want to be able to build both variants in CI, which makes editing distro.conf even more  unattractive.

I've considered giving them a "setup-environment" script that edits the distro.conf for them, but that seems a bit hacky too. And doesn't really satisfy my CI requirements.

Has anyone done this before? Or do you have some advice about how I could handle this?

Many thanks,
Ryan.


Quentin Schulz
 

Hi Ryan,

On Mon, Feb 03, 2020 at 05:34:23PM +0000, Ryan Harkin wrote:
Hello all,

I'm looking for advice on how to support multiple kernel versions in my
distro with minimal changes, and minimal disruption to my users.

Some background:

I have a legacy Sumo based distro with an image config, and a machine
config, with the machine using a 4.9 kernel. Last year, I upgraded
everything to Warrior, and moved to a 4.19 kernel.

Some of my users who are migrating from Sumo, wish to keep their 4.9
kernel. So I'm trying to work out how to handle this in the simplest way.

I know that I can add a 4.9 recipes to my Warrior branches, set
PREFERRED_VERSION in my distro.conf. But I don't want to change the default
preferred version globally. And I don't really want users to update the
distro.conf. Ideally, they should be able to take what I give them and
"just" build it to get a 4.9 or 4.19 variant.
You can make two machine configuration files. One with
PREFERRED_VERSION_virtual/kernel = "4.9%" and the other with 4.19.

They pick the correct machine when building, they should expect the
correct kernel in output. Only applies to images built for this machine
so I guess that's what you're looking for?

Ideally, I don't want to *have* to build both kernels and then create two
images. I expect that will cause confusion and lead to people flashing the
wrong images. So I'd prefer it to be either/or. Of course, I have to test
all of this, so I want to be able to build both variants in CI, which makes
editing distro.conf even more unattractive.
Same image (POV of bitbake <image>) but different machines, does that
match your requirements?

FWIW, you can pass MACHINE= to the command line just before bitbake
<image> making it rather obvious which machine they pick.

Quentin


Ryan Harkin <ryan.harkin@...>
 

Hi Quentin,

Thanks for your reply.

On Tue, 4 Feb 2020 at 08:51, Quentin Schulz <quentin.schulz@...> wrote:
Hi Ryan,

On Mon, Feb 03, 2020 at 05:34:23PM +0000, Ryan Harkin wrote:
> Hello all,
>
> I'm looking for advice on how to support multiple kernel versions in my
> distro with minimal changes, and minimal disruption to my users.
>
> Some background:
>
> I have a legacy Sumo based distro with an image config, and a machine
> config, with the machine using a 4.9 kernel. Last year, I upgraded
> everything to Warrior, and moved to a 4.19 kernel.
>
> Some of my users who are migrating from Sumo, wish to keep their 4.9
> kernel. So I'm trying to work out how to handle this in the simplest way.
>
> I know that I can add a 4.9 recipes to my Warrior branches, set
> PREFERRED_VERSION in my distro.conf. But I don't want to change the default
> preferred version globally. And I don't really want users to update the
> distro.conf. Ideally, they should be able to take what I give them and
> "just" build it to get a 4.9 or 4.19 variant.
>

You can make two machine configuration files. One with
PREFERRED_VERSION_virtual/kernel = "4.9%" and the other with 4.19.

They pick the correct machine when building, they should expect the
correct kernel in output. Only applies to images built for this machine
so I guess that's what you're looking for?

Yes, this works.

Trying it showed a few small problems. Eg. I have a package that only builds
for the 4.19 kernel, and needs to be excluded for 4.9. That's something I can
handle in the recipes using the machine type, of course.
 

> Ideally, I don't want to *have* to build both kernels and then create two
> images. I expect that will cause confusion and lead to people flashing the
> wrong images. So I'd prefer it to be either/or. Of course, I have to test
> all of this, so I want to be able to build both variants in CI, which makes
> editing distro.conf even more  unattractive.
>

Same image (POV of bitbake <image>) but different machines, does that
match your requirements?

FWIW, you can pass MACHINE= to the command line just before bitbake
<image> making it rather obvious which machine they pick.

Incidentally, that didn't work for me, but that's a symptom of how we setup
the environment, where local.conf sets MACHINE. I changed it to "MACHINE ?= ",
thinking it would let me override it via the shell. But it didn't. Strange, but not a
big problem.

I'd be interested to hear if anyone else has a different approach I could try for
comparison.

Thanks,
Ryan.


Quentin Schulz
 

Hi Ryan,

On Tue, Feb 04, 2020 at 04:00:01PM +0000, Ryan Harkin wrote:
Hi Quentin,

Thanks for your reply.

On Tue, 4 Feb 2020 at 08:51, Quentin Schulz <
quentin.schulz@streamunlimited.com> wrote:

Hi Ryan,

On Mon, Feb 03, 2020 at 05:34:23PM +0000, Ryan Harkin wrote:
Hello all,

I'm looking for advice on how to support multiple kernel versions in my
distro with minimal changes, and minimal disruption to my users.

Some background:

I have a legacy Sumo based distro with an image config, and a machine
config, with the machine using a 4.9 kernel. Last year, I upgraded
everything to Warrior, and moved to a 4.19 kernel.

Some of my users who are migrating from Sumo, wish to keep their 4.9
kernel. So I'm trying to work out how to handle this in the simplest way.

I know that I can add a 4.9 recipes to my Warrior branches, set
PREFERRED_VERSION in my distro.conf. But I don't want to change the
default
preferred version globally. And I don't really want users to update the
distro.conf. Ideally, they should be able to take what I give them and
"just" build it to get a 4.9 or 4.19 variant.
You can make two machine configuration files. One with
PREFERRED_VERSION_virtual/kernel = "4.9%" and the other with 4.19.

They pick the correct machine when building, they should expect the
correct kernel in output. Only applies to images built for this machine
so I guess that's what you're looking for?
Yes, this works.

Trying it showed a few small problems. Eg. I have a package that only builds
for the 4.19 kernel, and needs to be excluded for 4.9. That's something I
can
handle in the recipes using the machine type, of course.
It depends on what exactly does this recipe represent? A kernel module?
In which case, you can put it in your machine configuration file under
MACHINE_ESSENTIAL_EXTRA_RDEPENDS or RRECOMMENDS and omit it for 4.9.

We have a way to specify runtime dependencies on specific versions:
https://www.yoctoproject.org/docs/current/mega-manual/mega-manual.html#var-RDEPENDS

I unfortunately have no idea if Yocto supports such a thing in DEPENDS.



Ideally, I don't want to *have* to build both kernels and then create two
images. I expect that will cause confusion and lead to people flashing
the
wrong images. So I'd prefer it to be either/or. Of course, I have to test
all of this, so I want to be able to build both variants in CI, which
makes
editing distro.conf even more unattractive.
Same image (POV of bitbake <image>) but different machines, does that
match your requirements?

FWIW, you can pass MACHINE= to the command line just before bitbake
<image> making it rather obvious which machine they pick.
Incidentally, that didn't work for me, but that's a symptom of how we setup
the environment, where local.conf sets MACHINE. I changed it to "MACHINE ?=
",
thinking it would let me override it via the shell. But it didn't. Strange,
but not a
big problem.
To check who's setting it, run `bitbake <somesimplerecipe> -e | less` and look
for the line starting with MACHINE=. Then you have the whole bitbake
logic to set it above that line. That could help you find out what's
happening.

Quentin


Ryan Harkin <ryan.harkin@...>
 



On Tue, 4 Feb 2020 at 16:15, Quentin Schulz <quentin.schulz@...> wrote:
Hi Ryan,

On Tue, Feb 04, 2020 at 04:00:01PM +0000, Ryan Harkin wrote:
> Hi Quentin,
>
> Thanks for your reply.
>
> On Tue, 4 Feb 2020 at 08:51, Quentin Schulz <
> quentin.schulz@...> wrote:
>
> > Hi Ryan,
> >
> > On Mon, Feb 03, 2020 at 05:34:23PM +0000, Ryan Harkin wrote:
> > > Hello all,
> > >
> > > I'm looking for advice on how to support multiple kernel versions in my
> > > distro with minimal changes, and minimal disruption to my users.
> > >
> > > Some background:
> > >
> > > I have a legacy Sumo based distro with an image config, and a machine
> > > config, with the machine using a 4.9 kernel. Last year, I upgraded
> > > everything to Warrior, and moved to a 4.19 kernel.
> > >
> > > Some of my users who are migrating from Sumo, wish to keep their 4.9
> > > kernel. So I'm trying to work out how to handle this in the simplest way.
> > >
> > > I know that I can add a 4.9 recipes to my Warrior branches, set
> > > PREFERRED_VERSION in my distro.conf. But I don't want to change the
> > default
> > > preferred version globally. And I don't really want users to update the
> > > distro.conf. Ideally, they should be able to take what I give them and
> > > "just" build it to get a 4.9 or 4.19 variant.
> > >
> >
> > You can make two machine configuration files. One with
> > PREFERRED_VERSION_virtual/kernel = "4.9%" and the other with 4.19.
> >
> > They pick the correct machine when building, they should expect the
> > correct kernel in output. Only applies to images built for this machine
> > so I guess that's what you're looking for?
> >
>
> Yes, this works.
>
> Trying it showed a few small problems. Eg. I have a package that only builds
> for the 4.19 kernel, and needs to be excluded for 4.9. That's something I
> can
> handle in the recipes using the machine type, of course.
>

It depends on what exactly does this recipe represent? A kernel module?
In which case, you can put it in your machine configuration file under
MACHINE_ESSENTIAL_EXTRA_RDEPENDS or RRECOMMENDS and omit it for 4.9.

We have a way to specify runtime dependencies on specific versions:
https://www.yoctoproject.org/docs/current/mega-manual/mega-manual.html#var-RDEPENDS

I unfortunately have no idea if Yocto supports such a thing in DEPENDS.


Good tips, thanks again! I'll have a play with it.
 
>
> >
> > > Ideally, I don't want to *have* to build both kernels and then create two
> > > images. I expect that will cause confusion and lead to people flashing
> > the
> > > wrong images. So I'd prefer it to be either/or. Of course, I have to test
> > > all of this, so I want to be able to build both variants in CI, which
> > makes
> > > editing distro.conf even more  unattractive.
> > >
> >
> > Same image (POV of bitbake <image>) but different machines, does that
> > match your requirements?
> >
> > FWIW, you can pass MACHINE= to the command line just before bitbake
> > <image> making it rather obvious which machine they pick.
> >
>
> Incidentally, that didn't work for me, but that's a symptom of how we setup
> the environment, where local.conf sets MACHINE. I changed it to "MACHINE ?=
> ",
> thinking it would let me override it via the shell. But it didn't. Strange,
> but not a
> big problem.
>

To check who's setting it, run `bitbake <somesimplerecipe> -e | less` and look
for the line starting with MACHINE=. Then you have the whole bitbake
logic to set it above that line. That could help you find out what's
happening.

Actually, it does work like this:

export MACHINE=xyz
bitbake image

But not like this:

MACHINE=xyz bitbake image

No big deal, but I expected the last one to work.


Quentin


Quentin Schulz
 

On Tue, Feb 04, 2020 at 04:32:34PM +0000, Ryan Harkin wrote:
[...]

Actually, it does work like this:

export MACHINE=xyz
bitbake image

But not like this:

MACHINE=xyz bitbake image

No big deal, but I expected the last one to work.
I use it every day. I don't think we're doing something special (well,
for that particular thing :D), so there's at least a way to make it work
for sure :)

Quentin


Ryan Harkin <ryan.harkin@...>
 



On Wed, 5 Feb 2020 at 10:02, Mike Looijmans <mike.looijmans@...> wrote:

Met vriendelijke groet / kind regards,

Mike Looijmans
System Expert


TOPIC Embedded Products B.V.
Materiaalweg 4, 5681 RJ Best
The Netherlands

T: +31 (0) 499 33 69 69
E: mike.looijmans@...
W: www.topicproducts.com

Please consider the environment before printing this e-mail
On 04-02-2020 17:39, Quentin Schulz via Lists.Yoctoproject.Org wrote:
> On Tue, Feb 04, 2020 at 04:32:34PM +0000, Ryan Harkin wrote:
> [...]
>>
>> Actually, it does work like this:
>>
>> export MACHINE=xyz
>> bitbake image
>>
>> But not like this:
>>
>> MACHINE=xyz bitbake image
>>
>> No big deal, but I expected the last one to work.
>>
>
> I use it every day. I don't think we're doing something special (well,
> for that particular thing :D), so there's at least a way to make it work
> for sure :)

Same here, both interactive and in scripts.

Since the first works, you already have done this:
export BB_ENV_EXTRAWHITE=MACHINE

Maybe there's something funky in your shell?
You could check with a command like:
MACHINE=x env | grep MACHINE

 It looks "normal":

$ MACHINE=x env | grep MACHINE
MACHINE=x


Of maybe somebody wrote a bitbake wrapper for you that forks python instead of
exec'ing it?

No, it's the regular bitbake.

I'm not too concerned about this problem anyway, I'm sure if it's a problem for me, I can work out who is overriding it. 

Thanks, 
Ryan