On Wed, 2022-02-09 at 18:41 +0000, Richard Purdie wrote:

The BMW one is about hash equivalence so wouldn't help your ABI output problem
with the LTS. From what I remember, it predates hash equivalence and the project
needed a generic equivalance mechanism complete with server done at the runqueue
level in bitbake. This has now happened so we could revisit the idea of what is
in that layer but translating it to a hash equivalence plugin.

I'd also add that even with hash equivalence done well like we ended up with, we
have only two people interested/able to work on bugs with it, the author and
myself. For a key component of the system, this worries me a lot. Adding more
complexity without maintainer support isn't going to help anyone.

Sorry, I'm getting confused here with earlier work Michael Ho did at BMW. The
links here:

https://lists.yoctoproject.org/g/yocto/message/52650
https://github.com/bmwcarit/meta-abicompat

are the revised version from last year which *does* hook into hash equivalence.
I'm getting two things confused, sorry.

The nice thing about the layer above is that it is a standalone layer, we don't
have to merge it in order to use it. This shows the power of the new hash
equivalence code as it is a plugin to it. We may consider merging it at some
point but there is less of a pressing need and we need time to experiment with
it.

At this point it is a proof of concept and doesn't solve the ABI problem you're
describing in the original email. Sorry about any confusion.

The abidw recipe could be useful to your ABI issue.

Cheers,

Richard

On 2/9/2022 1:41 PM, Richard Purdie wrote:

There are lots of levels it could be implemented at but it is something someone
would need to pick up and drive forward with a long term view to helping with
issues etc.

What would be the minimum acceptable solution with the least investment?
in other words, do we have a list of requirements?

You're after our LTS to maintain ABI. In order to do that we need help, not just
with patches generating some output, but in day to day running of the project,
i.e. help comparing output before and after changes. Whenever a patch is
submitted which breaks this, it will need attention and help some someone to
explain to that submitter what the issue, why it is important and hints on how
to fix it.

That's true, this will require engagement from the community. Tool could
take few iterations to perfect. Until then, I expect tool owner to be
responsible for fixing these bugs. Once stability is reached, it becomes
community maintained.

If the tool owner doesn't maintain and community has no interest, tool
dies and gets reverted. It is as simple as any open source engagement.

When stability is established, each code contributor to LTS would be
subject to addressing issues found before they get merged.

The idea of "least investment" sends shivers down my spine since it sounds like
you want to do the absolute bare minimum to have this happen, rather than a more
community oriented approach.

It depends on your taste. I believe in smaller improvements
as opposed to throwing a big project to you that no-one will use it.

Everyone has different needs. We need to find the common ones.
That's why, I'm asking if there is an existing tool that works for
large part of the community accepting that there will be some folks
that won't have their needs addressed.

I'm interested in revisiting the tooling discussion and have these gaps
addressed for the biggest audience so that we can have something to
build upon.

Anyway, my point is there is more to this than just a patch. We have various
kinds of build output already and reports on test regressions - nobody helps
with them. I need some kind of a sign that ABI would be different and there are
people able to help with review on an ongoing basis, else it will just be
something else I and a small number of others become overloaded with.

Noted. Hopefully, things will be not that volatile for the LTS branch
and tool would actually help the maintainer.

In an ideal world, change needs to be stopped before that happens and
have the patch author address it similar to how you monitor build
pipelines.

Our team has posted a solution. BMW folks posted a solution.
None of them got merged.

Can you remind me of your team's please?

https://lists.yoctoproject.org/g/yocto/topic/85279259?p=,,,20,0,0,0::recentpostdate/sticky,,,20,2,160,85279259

This was an intern project from last summer that we are interested
in expanding coverage.

The BMW one is about hash equivalence so wouldn't help your ABI output problem
with the LTS. From what I remember, it predates hash equivalence and the project
needed a generic equivalance mechanism complete with server done at the runqueue
level in bitbake. This has now happened so we could revisit the idea of what is
in that layer but translating it to a hash equivalence plugin.
I'd also add that even with hash equivalence done well like we ended up with, we
have only two people interested/able to work on bugs with it, the author and
myself. For a key component of the system, this worries me a lot. Adding more
complexity without maintainer support isn't going to help anyone.

OK, I didn't know the story behind the change.

Could we take the version from BMW folks, merge and have the next person
add new features where it doesn't satisfy requirements?

See above on the BMW version. I'm a little worried you're suggesting merging
something which doesn't actually do what you need/want:(.

Got it.

or vice versa? or as Ross said some other work?
or none of the solutions are acceptable?

I have to admit I can't remember what the conclusion was on your team's version
but if you remind me of the patches I can try and remember. This is a bigger
problem than just patches though sadly.

Sure, let's find out what everyone is doing.

On Wed, 2022-02-09 at 18:10 +0000, Bryan Evenson wrote:

Richard,

-----Original Message-----
From: Richard Purdie <richard.purdie@...>
Sent: Wednesday, February 9, 2022 12:25 PM
To: Bryan Evenson <bevenson@...>;
yocto@...
Subject: Re: [yocto] Problems building eSDK: Exception: variable BBFILES
references itself!

On Wed, 2022-02-09 at 16:51 +0000, Bryan Evenson wrote:

All,

I've used the SDK in the past and want to start using the eSDK for my

project.

I'm on the dunfell branch. I'm trying to build the eSDK for my image
by
calling:

bitbake <my_image_name> -c populate_sdk_ext

The sdk generation fails with the error in the subject. I haven't
been able to find any documentation on what this means or how to fix
it. I've tried deleting my tmp directory and running cleansstate on
my image and I still get the same error. I've also tried generating
the eSDK for core-image-minimal and I have the same error. I can
generate the SDK successfully, just not the eSDK. I have copied more
of the build output below if it helps. Note that I have replaced some
text that I'm paranoid about publishing on the internet; the modified text is

enclosed with <>.

Does anyone have any insight as to why the eSDK build is failing?

Could you share what bitbake -e shows BBFILES as being set to? The variable
history from that command for the variable may also be helpful.

There has to be something different to how you're setting it compared to
what we test on the autobuilder.

Cheers,

Richard

Here's what I was able to pull from bitbake -e. I've included details for
both BBFILES and BBFILES_DYNAMIC. Let me know if anything else would be
helpful.

Thanks, I'm not seeing anything too odd there. I guess next we need to see the
bblayers.conf that the code is building into the eSDK and compare it to your
original conf/bblayer.conf.

The generated file should be in the eSDK image directory somewhere, I'm not able
to look up the exact path right now. Let me know if you can't find it. Lets see
if the two bblayers.conf files look different and introduce some issue.

Cheers,

Richard

On Wed, 2022-02-09 at 13:15 -0500, Sinan Kaya wrote:

On 2/9/2022 11:42 AM, Richard Purdie wrote:

There are two reasons people are interested:

a) for release stability as you mention
b) for performance as it could be tied into the hash equivalence mechanism for
artefact reuse - if A hasn't changed ABI, B dependning on it needn't rebuild.

There was a proof of concept of b) here:
https://lists.yoctoproject.org/g/yocto/message/52650

There are lots of levels it could be implemented at but it is something someone
would need to pick up and drive forward with a long term view to helping with
issues etc.

What would be the minimum acceptable solution with the least investment?
in other words, do we have a list of requirements?

You're after our LTS to maintain ABI. In order to do that we need help, not just
with patches generating some output, but in day to day running of the project,
i.e. help comparing output before and after changes. Whenever a patch is
submitted which breaks this, it will need attention and help some someone to
explain to that submitter what the issue, why it is important and hints on how
to fix it.

The idea of "least investment" sends shivers down my spine since it sounds like
you want to do the absolute bare minimum to have this happen, rather than a more
community oriented approach.

Anyway, my point is there is more to this than just a patch. We have various
kinds of build output already and reports on test regressions - nobody helps
with them. I need some kind of a sign that ABI would be different and there are
people able to help with review on an ongoing basis, else it will just be
something else I and a small number of others become overloaded with.

Our team has posted a solution. BMW folks posted a solution.
None of them got merged.

Can you remind me of your team's please?

The BMW one is about hash equivalence so wouldn't help your ABI output problem
with the LTS. From what I remember, it predates hash equivalence and the project
needed a generic equivalance mechanism complete with server done at the runqueue
level in bitbake. This has now happened so we could revisit the idea of what is
in that layer but translating it to a hash equivalence plugin.

I'd also add that even with hash equivalence done well like we ended up with, we
have only two people interested/able to work on bugs with it, the author and
myself. For a key component of the system, this worries me a lot. Adding more
complexity without maintainer support isn't going to help anyone.

Could we take the version from BMW folks, merge and have the next person
add new features where it doesn't satisfy requirements?

See above on the BMW version. I'm a little worried you're suggesting merging
something which doesn't actually do what you need/want :(.

or vice versa? or as Ross said some other work?
or none of the solutions are acceptable?

I have to admit I can't remember what the conclusion was on your team's version
but if you remind me of the patches I can try and remember. This is a bigger
problem than just patches though sadly.

Cheers,

Richard

On 2/9/2022 11:42 AM, Richard Purdie wrote:

There are two reasons people are interested:
a) for release stability as you mention
b) for performance as it could be tied into the hash equivalence mechanism for
artefact reuse - if A hasn't changed ABI, B dependning on it needn't rebuild.
There was a proof of concept of b) here:
https://lists.yoctoproject.org/g/yocto/message/52650
There are lots of levels it could be implemented at but it is something someone
would need to pick up and drive forward with a long term view to helping with
issues etc.

What would be the minimum acceptable solution with the least investment?
in other words, do we have a list of requirements?

Our team has posted a solution. BMW folks posted a solution.
None of them got merged.

Could we take the version from BMW folks, merge and have the next person
add new features where it doesn't satisfy requirements?

or vice versa? or as Ross said some other work?

or none of the solutions are acceptable?

Richard,

On Wed, Feb 9, 2022 at 8:37 AM Steve Sakoman <steve@...> wrote:

On Tue, Feb 8, 2022 at 1:07 PM Richard Purdie
<richard.purdie@...> wrote:

Forwarding to the correct list address and cc the LTS maintainer.

Cheers,

Richard



---------- Forwarded message ----------
From: Sinan Kaya <okaya@...>
To: Richard Purdie <richard.purdie@...>, Paul Eggleton <bluelightning@...>, Yocto list discussion <yocto@...>
Cc:
Bcc:
Date: Sun, 6 Feb 2022 20:14:37 -0500
Subject: Maintaining ABI Compatibility for LTS branch
Hi Everyone,

One of the limitations of Yocto LTS branch is that there is no
guaranteed backwards compatibility. Therefore, any time we move a branch
forward to move to latest dunfell release, we are taking a risk of
breaking our customers.

I'd be interested in hearing about any cases where we've broken things!

In general I don't take version upgrades (other than bug fix/cve fix
only dot releases)

Yocto reserves the right to move a package version forward if a
security fix cannot be applied properly as an example.

These cases should be extremely rare, and the community/technical
steering committee is given an opportunity to review this before it
happens.

I'm certainly open to suggestions on how we can do better.

I'd love to see some tooling to do ABI checking!

I think its not that LTS itself but also upgrade from LTS to LTS or other
newer versions, it wouuld be good to have this tool and prerhaps a list
of API/ABI changes to help migration/upgrade to newer LTS releases.

Steve

This promise is being held true on the kernel by running kernel API
tests etc. and running test benches across different CI environments.

I was curious about how everyone is approaching this problem.
There was some attempt to bring ABI checking functionality in the past
but this has never been merged.

Is everyone rolling their own solution? or never moving forward?

Sinan

I think core-image-swupdate.bb is adding sysvinit explicitly, please
remove it from that.

thanks for patch. Can you create a PR on github please ?

On Wed, 2022-02-09 at 16:51 +0000, Bryan Evenson wrote:

All,

I've used the SDK in the past and want to start using the eSDK for my project.
I'm on the dunfell branch. I'm trying to build the eSDK for my image by
calling:

bitbake <my_image_name> -c populate_sdk_ext

The sdk generation fails with the error in the subject. I haven't been able
to find any documentation on what this means or how to fix it. I've tried
deleting my tmp directory and running cleansstate on my image and I still get
the same error. I've also tried generating the eSDK for core-image-minimal
and I have the same error. I can generate the SDK successfully, just not the
eSDK. I have copied more of the build output below if it helps. Note that I
have replaced some text that I'm paranoid about publishing on the internet;
the modified text is enclosed with <>.

Does anyone have any insight as to why the eSDK build is failing?

Could you share what bitbake -e shows BBFILES as being set to? The variable
history from that command for the variable may also be helpful. 

There has to be something different to how you're setting it compared to what we
test on the autobuilder.

Cheers,

Richard

All,

I've used the SDK in the past and want to start using the eSDK for my project. I'm on the dunfell branch. I'm trying to build the eSDK for my image by calling:

bitbake <my_image_name> -c populate_sdk_ext

The sdk generation fails with the error in the subject. I haven't been able to find any documentation on what this means or how to fix it. I've tried deleting my tmp directory and running cleansstate on my image and I still get the same error. I've also tried generating the eSDK for core-image-minimal and I have the same error. I can generate the SDK successfully, just not the eSDK. I have copied more of the build output below if it helps. Note that I have replaced some text that I'm paranoid about publishing on the internet; the modified text is enclosed with <>.

Does anyone have any insight as to why the eSDK build is failing?

Thanks,
Bryan

NOTE: Running intercept scripts:
NOTE: Executing buildhistory_list_installed_sdk_host ...
DEBUG: Executing python function buildhistory_list_installed_sdk_host
DEBUG: Python function buildhistory_list_installed_sdk_host finished
NOTE: Executing buildhistory_get_sdk_installed_host ...
DEBUG: Executing shell function buildhistory_get_sdk_installed_host
DEBUG: Shell function buildhistory_get_sdk_installed_host finished
NOTE: Executing copy_buildsystem ...
DEBUG: Executing python function copy_buildsystem
NOTE: Excluding local workspace layer <my_home_dir>/poky/poky-build/workspace from extensible SDK
NOTE: Generating sstate task list...
ERROR: Failed to generate filtered task list for extensible SDK:

### Shell environment set up for builds. ###

You can now run 'bitbake <target>'

Common targets are:
core-image-minimal
core-image-sato
meta-toolchain
meta-ide-support

You can also run generated qemu images with a command like 'runqemu qemux86'

Other commonly useful commands are:
- 'devtool' and 'recipetool' handle common recipe tasks
- 'bitbake-layers' handles common layer tasks
- 'oe-pkgdata-util' handles common target package tasks
ERROR: bitbake failed:
ERROR: Command execution failed: Traceback (most recent call last):
File "<my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/bitbake/lib/bb/data_smart.py", line 401, in expandWithRefs
s = __expand_var_regexp__.sub(varparse.var_sub, s)
File "<my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/bitbake/lib/bb/data_smart.py", line 96, in var_sub
raise Exception("variable %s references itself!" % self.varname)
Exception: variable BBFILES references itself!

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "<my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/bitbake/lib/bb/command.py", line 107, in runAsyncCommand
self.cooker.updateCache()
File "<my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/bitbake/lib/bb/cooker.py", line 1558, in updateCache
(filelist, masked, searchdirs) = self.collection.collect_bbfiles(self.data, self.data)
File "<my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/bitbake/lib/bb/cooker.py", line 1729, in collect_bbfiles
files = (config.getVar( "BBFILES") or "").split()
File "<my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/bitbake/lib/bb/data_smart.py", line 589, in getVar
return self.getVarFlag(var, "_content", expand, noweakdefault, parsing)
File "<my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/bitbake/lib/bb/data_smart.py", line 780, in getVarFlag
parser = self.expandWithRefs(value, cachename)
File "<my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/bitbake/lib/bb/data_smart.py", line 418, in expandWithRefs
raise ExpansionError(varname, s, exc).with_traceback(tb) from exc
File "<my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/bitbake/lib/bb/data_smart.py", line 401, in expandWithRefs
s = __expand_var_regexp__.sub(varparse.var_sub, s)
File "<my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/bitbake/lib/bb/data_smart.py", line 96, in var_sub
raise Exception("variable %s references itself!" % self.varname)
bb.data_smart.ExpansionError: Failure expanding variable BBFILES, expression was ${BBFILES} <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/<my_custom_layer1>/recipes-*/*/*.bb <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/<my_custom_layer1>/recipes-*/*/*.bbappend <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/<my_custom_layer2>/recipes*/*/*.bb <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/<my_custom_layer2>/recipes*/*/*.bbappend <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta-bacnet/recipes-*/*/*.bb <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta-bacnet/recipes-*/*/*.bbappend <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta-atmel/recipes*/*/*.bb <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta-atmel/recipes*/*/*.bbappend <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta/recipes-*/*/*.bb <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta-poky/recipes-*/*/*.bb <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta-poky/recipes-*/*/*.bbappend <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta-yocto-bsp/recipes-*/*/*.bb <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta-yocto-bsp/recipes-*/*/*.bbappend <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta-openembedded/meta-oe/recipes-*/*/*.bb <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta-openembedded/meta-oe/recipes-*/*/*.bbappend <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta-openembedded/meta-networking/recipes-*/*/*.bb <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta-openembedded/meta-networking/recipes-*/*/*.bbappend <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta-openembedded/meta-python/recipes*/*/*.bb <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta-openembedded/meta-python/recipes*/*/*.bbappend <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/workspace/recipes/*/*.bb <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/workspace/appends/*.bbappend <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta-openembedded/meta-oe/dynamic-layers/meta-python/recipes-*/*/*.bb which triggered exception Exception: variable BBFILES references itself!


Summary: There was 1 ERROR message shown, returning a non-zero exit code.
Execution of '. layers/poky/oe-init-build-env .; PYTHONDONTWRITEBYTECODE=1 BB_SETSCENE_ENFORCE=1 PSEUDO_DISABLED=1 oe-check-sstate <my_image_name> meta-extsdk-toolchain:do_populate_sysroot -s -o <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/tasklist.txt -l <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/tasklist_bb_log.txt' failed with exit code 1:

### Shell environment set up for builds. ###

You can now run 'bitbake <target>'

Common targets are:
core-image-minimal
core-image-sato
meta-toolchain
meta-ide-support

You can also run generated qemu images with a command like 'runqemu qemux86'

Other commonly useful commands are:
- 'devtool' and 'recipetool' handle common recipe tasks
- 'bitbake-layers' handles common layer tasks
- 'oe-pkgdata-util' handles common target package tasks
ERROR: bitbake failed:
ERROR: Command execution failed: Traceback (most recent call last):
File "<my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/bitbake/lib/bb/data_smart.py", line 401, in expandWithRefs
s = __expand_var_regexp__.sub(varparse.var_sub, s)
File "<my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/bitbake/lib/bb/data_smart.py", line 96, in var_sub
raise Exception("variable %s references itself!" % self.varname)
Exception: variable BBFILES references itself!

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "<my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/bitbake/lib/bb/command.py", line 107, in runAsyncCommand
self.cooker.updateCache()
File "<my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/bitbake/lib/bb/cooker.py", line 1558, in updateCache
(filelist, masked, searchdirs) = self.collection.collect_bbfiles(self.data, self.data)
File "<my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/bitbake/lib/bb/cooker.py", line 1729, in collect_bbfiles
files = (config.getVar( "BBFILES") or "").split()
File "<my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/bitbake/lib/bb/data_smart.py", line 589, in getVar
return self.getVarFlag(var, "_content", expand, noweakdefault, parsing)
File "<my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/bitbake/lib/bb/data_smart.py", line 780, in getVarFlag
parser = self.expandWithRefs(value, cachename)
File "<my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/bitbake/lib/bb/data_smart.py", line 418, in expandWithRefs
raise ExpansionError(varname, s, exc).with_traceback(tb) from exc
File "<my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/bitbake/lib/bb/data_smart.py", line 401, in expandWithRefs
s = __expand_var_regexp__.sub(varparse.var_sub, s)
File "<my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/bitbake/lib/bb/data_smart.py", line 96, in var_sub
raise Exception("variable %s references itself!" % self.varname)
bb.data_smart.ExpansionError: Failure expanding variable BBFILES, expression was ${BBFILES} <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/<my_custom_layer1>/recipes-*/*/*.bb <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/<my_custom_layer1>/recipes-*/*/*.bbappend <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/<my_custom_layer2>/recipes*/*/*.bb <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/<my_custom_layer2>/recipes*/*/*.bbappend <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta-bacnet/recipes-*/*/*.bb <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta-bacnet/recipes-*/*/*.bbappend <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta-atmel/recipes*/*/*.bb <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta-atmel/recipes*/*/*.bbappend <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta/recipes-*/*/*.bb <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta-poky/recipes-*/*/*.bb <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta-poky/recipes-*/*/*.bbappend <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta-yocto-bsp/recipes-*/*/*.bb <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta-yocto-bsp/recipes-*/*/*.bbappend <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta-openembedded/meta-oe/recipes-*/*/*.bb <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta-openembedded/meta-oe/recipes-*/*/*.bbappend <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta-openembedded/meta-networking/recipes-*/*/*.bb <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta-openembedded/meta-networking/recipes-*/*/*.bbappend <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta-openembedded/meta-python/recipes*/*/*.bb <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta-openembedded/meta-python/recipes*/*/*.bbappend <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/workspace/recipes/*/*.bb <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/workspace/appends/*.bbappend <my_home_dir>/poky/poky-build/tmp/work/at91sam9x5ek-poky-linux-gnueabi/<my_image_name>/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta-openembedded/meta-oe/dynamic-layers/meta-python/recipes-*/*/*.bb which triggered exception Exception: variable BBFILES references itself!


Summary: There was 1 ERROR message shown, returning a non-zero exit code.


DEBUG: Python function copy_buildsystem finished
DEBUG: Python function do_populate_sdk_ext finished

On Sun, 2022-02-06 at 20:14 -0500, Sinan Kaya wrote:

One of the limitations of Yocto LTS branch is that there is no
guaranteed backwards compatibility. Therefore, any time we move a branch
forward to move to latest dunfell release, we are taking a risk of
breaking our customers.

Whilst there isn't a guarantee, it is something we're doing our best to ensure.
Where there are security issues and we can't fix them as being backwards
compatible, we reserve the right to break things in preference to the security
issues. We wouldn't do that lightly.

Yocto reserves the right to move a package version forward if a
security fix cannot be applied properly as an example.

It also comes down to resources. If we have a choice between this and not fixing
the issue, we'd have to do this. If there are resources to do a more precision
backport, we'd take that option but we're often resource limited.

This promise is being held true on the kernel by running kernel API
tests etc. and running test benches across different CI environments.

We do run extensive tests on the autobuilder including things like LTP. We
admittedly lack resources for analysing the comparisions in an automated or
manual way sadly. I'd love to see API/ABI reporting added to our builds.

I was curious about how everyone is approaching this problem.
There was some attempt to bring ABI checking functionality in the past
but this has never been merged.

There have been ideas proposed but I've not seen any full solution as yet.

Is everyone rolling their own solution? or never moving forward?

I think most are just accepting some level of change and we've tried to keep it
to a minimal level.

I mentioned this in yesterdays call and these were mentioned:

https://github.com/lvc/abi-compliance-checker 
https://abi-laboratory.pro/index.php?view=tracker

There are two reasons people are interested:

a) for release stability as you mention
b) for performance as it could be tied into the hash equivalence mechanism for
artefact reuse - if A hasn't changed ABI, B dependning on it needn't rebuild.

There was a proof of concept of b) here:
https://lists.yoctoproject.org/g/yocto/message/52650

There are lots of levels it could be implemented at but it is something someone
would need to pick up and drive forward with a long term view to helping with
issues etc.

Cheers,

Richard

On Tue, Feb 8, 2022 at 1:07 PM Richard Purdie
<richard.purdie@...> wrote:

Forwarding to the correct list address and cc the LTS maintainer.

Cheers,

Richard



---------- Forwarded message ----------
From: Sinan Kaya <okaya@...>
To: Richard Purdie <richard.purdie@...>, Paul Eggleton <bluelightning@...>, Yocto list discussion <yocto@...>
Cc:
Bcc:
Date: Sun, 6 Feb 2022 20:14:37 -0500
Subject: Maintaining ABI Compatibility for LTS branch
Hi Everyone,

One of the limitations of Yocto LTS branch is that there is no
guaranteed backwards compatibility. Therefore, any time we move a branch
forward to move to latest dunfell release, we are taking a risk of
breaking our customers.

I'd be interested in hearing about any cases where we've broken things!

In general I don't take version upgrades (other than bug fix/cve fix
only dot releases)

Yocto reserves the right to move a package version forward if a
security fix cannot be applied properly as an example.

These cases should be extremely rare, and the community/technical
steering committee is given an opportunity to review this before it
happens.

I'm certainly open to suggestions on how we can do better.

I'd love to see some tooling to do ABI checking!

Steve

This promise is being held true on the kernel by running kernel API
tests etc. and running test benches across different CI environments.

I was curious about how everyone is approaching this problem.
There was some attempt to bring ABI checking functionality in the past
but this has never been merged.

Is everyone rolling their own solution? or never moving forward?

Sinan

On Tue, 8 Feb 2022 at 23:07, Richard Purdie
<richard.purdie@...> wrote:

I was curious about how everyone is approaching this problem.
There was some attempt to bring ABI checking functionality in the past
but this has never been merged.

Is everyone rolling their own solution? or never moving forward?

Relatedly, somewhere I have a branch that uses libabigail to extract
the library ABIs in an build. The next step was doing the comparison,
but I never got that far. I wouldn't be surprised if someone has done
the same but actually finished the effort.

Ross

From: Aníbal Limón <limon.anibal@...>

Gstreamer upgraded to 1.20 see,

https://git.openembedded.org/openembedded-core/commit/?id=75891f361f3e9df9fc3e97c720a2ae57dda75888

Signed-off-by: Aníbal Limón <anibal@...>
Signed-off-by: Aníbal Limón <limon.anibal@...>
---
..._1.18.%.bbappend => gstreamer1.0-plugins-good_1.20.%.bbappend} | 0
1 file changed, 0 insertions(+), 0 deletions(-)
rename recipes-multimedia/gstreamer/{gstreamer1.0-plugins-good_1.18.%.bbappend => gstreamer1.0-plugins-good_1.20.%.bbappend} (100%)

diff --git a/recipes-multimedia/gstreamer/gstreamer1.0-plugins-good_1.18.%.bbappend b/recipes-multimedia/gstreamer/gstreamer1.0-plugins-good_1.20.%.bbappend
similarity index 100%
rename from recipes-multimedia/gstreamer/gstreamer1.0-plugins-good_1.18.%.bbappend
rename to recipes-multimedia/gstreamer/gstreamer1.0-plugins-good_1.20.%.bbappend
--
2.34.1

Hello All,

While bitbaking an image recipe with systemd configured in "local.conf", I am facing an error while doing do_rootfs task.

In the local.conf :

DISTRO_FEATURES_append = " systemd"
#DISTRO_FEATURES_BACKFILL_CONSIDERED += "sysvinit"
VIRTUAL-RUNTIME_init_manager = "systemd"
VIRTUAL-RUNTIME_initscripts = "systemd-compat-units"

Bitbake error:
.
.
Transaction check succeeded.
Running transaction test
Error: Transaction check error:
  file /sbin/telinit conflicts between attempted installs of sysvinit-2.96-r0.cortexa7t2hf_neon_vfpv4 and systemd-1:244.5-r0.cortexa7t2hf_neon_vfpv4

I was not facing this error before without systemd.

Can anyone please let me know how to resolve this error and what does this error mean?

Your help will be much appreciated.

Thanks in advance

P.S: If I uncomment DISTRO_FEATURES_BACKFILL_CONSIDERED += "sysvinit" in local.conf I get below error:

ERROR: Nothing RPROVIDES 'sysvinit' (but [...]/core-image-swupdate.bb RDEPENDS on or otherwise requires it)
sysvinit was skipped: missing required distro feature 'sysvinit' (not in DISTRO_FEATURES)

HI ,
I am currently working upgrading  recipes using AUH(Auto Upgrade Helper ) tool where in i am able to upgrade recipes,
custom settings such as SRCBRANCH , SRCREV and SRC_URI in a recipe file of interest.
I am able to upgrade recipes in their recipe(.bb)files only i.e., after completion of upgrade process it is able 
to update SRCREV commit hash to latest in the repo.
If i set the above mentioned variables in extended recipe files(.bbappend)  by removing in main recipe instead,
main recipe file still gets modified/added with variables SRCBRANCH and SRCREV(latest hash) after upgrade process
is completed, leaving behind same variables with previous data which was set, how to make the tool to update extended recipes only.


Thanks & Regards,
Manjunath

Hi, I am quite new to Yocto/BitBake and I have a bit of a headache trying to figure out the best way to make use of a native tool in the following context:

I have one recipe that fetch and build third party libs and binaries, let's call it X.
I have another recipe that also fetch and build third party libs and binaries, those depending on X's libs, let's call it Y.
Y needs to run post installation scripts to generate some config files.
Those scripts are calling some of X's binaries to generate those files, therefore I added a dependency on X-native to be able to run those scripts on the building machine.
At the moment, I include those scripts in do_install[postfuncs] of Y-native recipe while adding it as dependency in Y recipe, as I wish those to files to be part of Y package.

Assuming what I am doing until there is alright (correct me if not), here comes my issue:
The way X is configured/compiled involves specifying the path where it will write any file it generates/manage (currently based on the 'datadir' prefix).
This implies that building X-native will also prefix this path with STAGING_DIR_NATIVE in X recipe context.
So, after using X-native binaries from within Y-native recipe I end up generating files in the wrong STAGING_DIR_NATIVE (ie. X recipe context), being unable to easily retrieve those as it's outside of Y recipe context.

Am I missing something here ?
I read about pkg_postinst and deferring them after sysroot creation, but this is not what I need if I wish to package to config files, right ?
If there are no other solution, I may resort not to package them and use this to generate them at image/sdk creation, but this does not seems clean to me.

Also, I wish not to patch X sources to enable some sort of runtime output path configuration.

From: Bartosz Golaszewski <bartosz.golaszewski@...>

Reuse the same code that generates zephyr samples for building tests.
This allows us to generate .bin files in all cases.

Signed-off-by: Bartosz Golaszewski <bartosz.golaszewski@...>
---
.../recipes-kernel/zephyr-kernel/zephyr-image.inc | 9 +--------
1 file changed, 1 insertion(+), 8 deletions(-)

diff --git a/meta-zephyr-core/recipes-kernel/zephyr-kernel/zephyr-image.inc b/meta-zephyr-core/recipes-kernel/zephyr-kernel/zephyr-image.inc
index c77692d..2d4c6ff 100644
--- a/meta-zephyr-core/recipes-kernel/zephyr-kernel/zephyr-image.inc
+++ b/meta-zephyr-core/recipes-kernel/zephyr-kernel/zephyr-image.inc
@@ -1,17 +1,10 @@
-require zephyr-kernel-src.inc
-require zephyr-kernel-common.inc
-
+require zephyr-sample.inc
inherit testimage
-inherit deploy

QEMU_BIN_PATH = "${STAGING_BINDIR_NATIVE}"

ZEPHYR_BASE = "${S}"
OECMAKE_SOURCEPATH = "${S}/${ZEPHYR_SRC_DIR}"

-do_deploy () {
- install -D ${B}/zephyr/${ZEPHYR_MAKE_OUTPUT} ${DEPLOYDIR}/${ZEPHYR_IMAGENAME}
-}
-
addtask deploy after do_compile
do_install[noexec] = "1"
--
2.30.1