Hi,

Sharing this before opening a bug if needed...

I'm building the latest Poky ("core-image-minimal" plus a few extra
packages).

I'm always getting the below setscene errors after upgrading my Poky
sources:

... <more similar errors before>
WARNING: libffi-native-3.4.2-r0 do_populate_sysroot_setscene: Failed to
fetch URL
file://universal/c8/be/sstate:libffi-native:x86_64-linux:3.4.2:r0:x86_64:7:c8be7b784ce8ebbf6d897367bc8e96f6718040858d61f9a4f6aa5325821b0ce5_populate_sysroot.tar.zst.siginfo;downloadfilename=universal/c8/be/sstate:libffi-native:x86_64-linux:3.4.2:r0:x86_64:7:c8be7b784ce8ebbf6d897367bc8e96f6718040858d61f9a4f6aa5325821b0ce5_populate_sysroot.tar.zst.siginfo,
attempting MIRRORS if available
ERROR: libffi-native-3.4.2-r0 do_populate_sysroot_setscene: Fetcher
failure: Unable to find file
file://universal/c8/be/sstate:libffi-native:x86_64-linux:3.4.2:r0:x86_64:7:c8be7b784ce8ebbf6d897367bc8e96f6718040858d61f9a4f6aa5325821b0ce5_populate_sysroot.tar.zst.siginfo;downloadfilename=universal/c8/be/sstate:libffi-native:x86_64-linux:3.4.2:r0:x86_64:7:c8be7b784ce8ebbf6d897367bc8e96f6718040858d61f9a4f6aa5325821b0ce5_populate_sysroot.tar.zst.siginfo
anywhere. The paths that were searched were:
    /media/mike/ssd/yocto/poky/build/sstate-cache
    /media/mike/ssd/yocto/poky/build/sstate-cache
WARNING: libpcre2-native-10.39-r0 do_populate_sysroot_setscene: Failed
to fetch URL
file://universal/bf/03/sstate:libpcre2-native:x86_64-linux:10.39:r0:x86_64:7:bf035286ff06470377fe9c9298ba116222c9d557f2fa44d418beb919aa185db9_populate_sysroot.tar.zst.siginfo;downloadfilename=universal/bf/03/sstate:libpcre2-native:x86_64-linux:10.39:r0:x86_64:7:bf035286ff06470377fe9c9298ba116222c9d557f2fa44d418beb919aa185db9_populate_sysroot.tar.zst.siginfo,
attempting MIRRORS if available
ERROR: libpcre2-native-10.39-r0 do_populate_sysroot_setscene: Fetcher
failure: Unable to find file
file://universal/bf/03/sstate:libpcre2-native:x86_64-linux:10.39:r0:x86_64:7:bf035286ff06470377fe9c9298ba116222c9d557f2fa44d418beb919aa185db9_populate_sysroot.tar.zst.siginfo;downloadfilename=universal/bf/03/sstate:libpcre2-native:x86_64-linux:10.39:r0:x86_64:7:bf035286ff06470377fe9c9298ba116222c9d557f2fa44d418beb919aa185db9_populate_sysroot.tar.zst.siginfo
anywhere. The paths that were searched were:
    /media/mike/ssd/yocto/poky/build/sstate-cache
    /media/mike/ssd/yocto/poky/build/sstate-cache
ERROR: libpcre2-native-10.39-r0 do_populate_sysroot_setscene: No
suitable staging package found
ERROR: Logfile of failure stored in:
/media/mike/ssd/yocto/poky/build/tmp/work/x86_64-linux/libpcre2-native/10.39-r0/temp/log.do_populate_sysroot_setscene.315121
WARNING: Setscene task
(virtual:native:/media/mike/ssd/yocto/poky/meta/recipes-support/libpcre/libpcre2_10.39.bb:do_populate_sysroot_setscene)
failed with exit code '1' - real task will be run instead
Currently  4 running tasks (1838 of 1838/467 of 4630)  10%
|################

As expected, the error messages are highlighted in red, but they are not
critical as BitBake can always build the corresponding recipes from sources.

Two questions:

* Anything wrong with my local sstate cache? Should I erase it?
* Should these issues really be treated as errors, scaring users that
something could be wrong while the resulting build looks correct anyway?

Thanks in advance
Michael.

--
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

--

On Thu, Jan 13, 2022 at 11:23 AM Michael Opdenacker
<michael.opdenacker@...> wrote:

Hi,

Sharing this before opening a bug if needed...

I'm building the latest Poky ("core-image-minimal" plus a few extra
packages).

I'm always getting the below setscene errors after upgrading my Poky
sources:

... <more similar errors before>
WARNING: libffi-native-3.4.2-r0 do_populate_sysroot_setscene: Failed to
fetch URL
file://universal/c8/be/sstate:libffi-native:x86_64-linux:3.4.2:r0:x86_64:7:c8be7b784ce8ebbf6d897367bc8e96f6718040858d61f9a4f6aa5325821b0ce5_populate_sysroot.tar.zst.siginfo;downloadfilename=universal/c8/be/sstate:libffi-native:x86_64-linux:3.4.2:r0:x86_64:7:c8be7b784ce8ebbf6d897367bc8e96f6718040858d61f9a4f6aa5325821b0ce5_populate_sysroot.tar.zst.siginfo,
attempting MIRRORS if available
ERROR: libffi-native-3.4.2-r0 do_populate_sysroot_setscene: Fetcher
failure: Unable to find file
file://universal/c8/be/sstate:libffi-native:x86_64-linux:3.4.2:r0:x86_64:7:c8be7b784ce8ebbf6d897367bc8e96f6718040858d61f9a4f6aa5325821b0ce5_populate_sysroot.tar.zst.siginfo;downloadfilename=universal/c8/be/sstate:libffi-native:x86_64-linux:3.4.2:r0:x86_64:7:c8be7b784ce8ebbf6d897367bc8e96f6718040858d61f9a4f6aa5325821b0ce5_populate_sysroot.tar.zst.siginfo
anywhere. The paths that were searched were:
/media/mike/ssd/yocto/poky/build/sstate-cache
/media/mike/ssd/yocto/poky/build/sstate-cache
WARNING: libpcre2-native-10.39-r0 do_populate_sysroot_setscene: Failed
to fetch URL
file://universal/bf/03/sstate:libpcre2-native:x86_64-linux:10.39:r0:x86_64:7:bf035286ff06470377fe9c9298ba116222c9d557f2fa44d418beb919aa185db9_populate_sysroot.tar.zst.siginfo;downloadfilename=universal/bf/03/sstate:libpcre2-native:x86_64-linux:10.39:r0:x86_64:7:bf035286ff06470377fe9c9298ba116222c9d557f2fa44d418beb919aa185db9_populate_sysroot.tar.zst.siginfo,
attempting MIRRORS if available
ERROR: libpcre2-native-10.39-r0 do_populate_sysroot_setscene: Fetcher
failure: Unable to find file
file://universal/bf/03/sstate:libpcre2-native:x86_64-linux:10.39:r0:x86_64:7:bf035286ff06470377fe9c9298ba116222c9d557f2fa44d418beb919aa185db9_populate_sysroot.tar.zst.siginfo;downloadfilename=universal/bf/03/sstate:libpcre2-native:x86_64-linux:10.39:r0:x86_64:7:bf035286ff06470377fe9c9298ba116222c9d557f2fa44d418beb919aa185db9_populate_sysroot.tar.zst.siginfo
anywhere. The paths that were searched were:
/media/mike/ssd/yocto/poky/build/sstate-cache
/media/mike/ssd/yocto/poky/build/sstate-cache
ERROR: libpcre2-native-10.39-r0 do_populate_sysroot_setscene: No
suitable staging package found
ERROR: Logfile of failure stored in:
/media/mike/ssd/yocto/poky/build/tmp/work/x86_64-linux/libpcre2-native/10.39-r0/temp/log.do_populate_sysroot_setscene.315121
WARNING: Setscene task
(virtual:native:/media/mike/ssd/yocto/poky/meta/recipes-support/libpcre/libpcre2_10.39.bb:do_populate_sysroot_setscene)
failed with exit code '1' - real task will be run instead
Currently 4 running tasks (1838 of 1838/467 of 4630) 10%
|################

As expected, the error messages are highlighted in red, but they are not
critical as BitBake can always build the corresponding recipes from sources.

Two questions:

* Anything wrong with my local sstate cache? Should I erase it?

This usually happens when there is a sstate archive (.tar.zst), but no
corresponding siginfo file (.tar.zst.siginfo). The sstate code assumes
the siginfo file is present if archive is present, and "errors" if
not. In my experience, this usually happens when you try to clean out
the sstate cache using a mechanism that is unaware of the relationship
and deletes the siginfo and not the archive (e.g. some form of "find
-delete ...")

* Should these issues really be treated as errors, scaring users that
something could be wrong while the resulting build looks correct anyway?

As noted by the build, it will actually just execute the missing
tasks; one annoyance (depending on context) is that bitbake will
return a non-zero exit code if any ERROR occurs in the build, even
though in this case it was recovered.

This has been discussed at length before (I can't find a citation
ATM), and I believe it was left this way because this particular case
is considered an actual error that should never happen and needs
investigation on the YP Autobuilder.

Thanks in advance
Michael.

--
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Hi Jose,

Thanks for your advice!

On 1/13/22 6:37 PM, Jose Quaresma wrote:

Hi,

I've seen the same for some time now when building core-image-minimal
with yocto state mirror and hashequiv servers.

After some investigation I think the root cause is some
bandwidth/connection limitation on the http yocto mirror.

Reducing the BB_NUMBER_THREADS="1" to a single core solves many of the
fetch issues for me,
my config is the default one with:

  BB_NUMBER_THREADS="1"
  SSTATE_MIRRORS = "file://.*
http://sstate.yoctoproject.org/dev/PATH;downloadfilename=PATH
<http://sstate.yoctoproject.org/dev/PATH;downloadfilename=PATH>"
  BB_HASHSERVE_UPSTREAM = "typhoon.yocto.io:8687
<http://typhoon.yocto.io:8687>"

Oops, it seems you're right. I have the same SSTATE_MIRRORS and
BB_HASHSERVE_UPSTREAM settings as you do, and adding
BB_NUMBER_THREADS="1" makes the errors disappear.

It seems I should file a bug about that. The error messages just mention
local paths, not remote ones, so it sounds like a local issue while it's
apparently not.

Thanks again
Michael.

--

Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

On Thu, 2022-01-13 at 19:37 +0100, Michael Opdenacker wrote:

Hi Jose,

Thanks for your advice!

On 1/13/22 6:37 PM, Jose Quaresma wrote:

Hi,

I've seen the same for some time now when building core-image-minimal
with yocto state mirror and hashequiv servers.

After some investigation I think the root cause is some
bandwidth/connection limitation on the http yocto mirror.

Reducing the BB_NUMBER_THREADS="1" to a single core solves many of the
fetch issues for me,
my config is the default one with:

  BB_NUMBER_THREADS="1"
  SSTATE_MIRRORS = "file://.*
http://sstate.yoctoproject.org/dev/PATH;downloadfilename=PATH
<http://sstate.yoctoproject.org/dev/PATH;downloadfilename=PATH>"
  BB_HASHSERVE_UPSTREAM = "typhoon.yocto.io:8687
<http://typhoon.yocto.io:8687>"

Oops, it seems you're right. I have the same SSTATE_MIRRORS and
BB_HASHSERVE_UPSTREAM settings as you do, and adding
BB_NUMBER_THREADS="1" makes the errors disappear.

It seems I should file a bug about that. The error messages just mention
local paths, not remote ones, so it sounds like a local issue while it's
apparently not.

That is useful to debug it but suggests there is an issue with concurrent
connections to the YP server.

I think we should have an open bug and get Michael Halstead's input on this.

Cheers,

Richard

On 1/13/22 10:53 PM, Richard Purdie wrote:

On Thu, 2022-01-13 at 19:37 +0100, Michael Opdenacker wrote:

Hi Jose,

Thanks for your advice!

On 1/13/22 6:37 PM, Jose Quaresma wrote:

Hi,

I've seen the same for some time now when building core-image-minimal
with yocto state mirror and hashequiv servers.

After some investigation I think the root cause is some
bandwidth/connection limitation on the http yocto mirror.

Reducing the BB_NUMBER_THREADS="1" to a single core solves many of the
fetch issues for me,
my config is the default one with:

  BB_NUMBER_THREADS="1"
  SSTATE_MIRRORS = "file://.*
http://sstate.yoctoproject.org/dev/PATH;downloadfilename=PATH
<http://sstate.yoctoproject.org/dev/PATH;downloadfilename=PATH>"
  BB_HASHSERVE_UPSTREAM = "typhoon.yocto.io:8687
<http://typhoon.yocto.io:8687>"

Oops, it seems you're right. I have the same SSTATE_MIRRORS and
BB_HASHSERVE_UPSTREAM settings as you do, and adding
BB_NUMBER_THREADS="1" makes the errors disappear.

It seems I should file a bug about that. The error messages just mention
local paths, not remote ones, so it sounds like a local issue while it's
apparently not.

That is useful to debug it but suggests there is an issue with concurrent
connections to the YP server.

I think we should have an open bug and get Michael Halstead's input on this.

Done: https://bugzilla.yoctoproject.org/show_bug.cgi?id=14690
Thanks for your advice.
Cheers
Michael.

--
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

--