Date
1 - 8 of 8
Red alert but apparently harmless setscene errors
|
Michael Opdenacker
Hi,
Sharing this before opening a bug if needed... I'm building the latest Poky ("core-image-minimal" plus a few extra packages). I'm always getting the below setscene errors after upgrading my Poky sources: ... <more similar errors before> WARNING: libffi-native-3.4.2-r0 do_populate_sysroot_setscene: Failed to fetch URL file://universal/c8/be/sstate:libffi-native:x86_64-linux:3.4.2:r0:x86_64:7:c8be7b784ce8ebbf6d897367bc8e96f6718040858d61f9a4f6aa5325821b0ce5_populate_sysroot.tar.zst.siginfo;downloadfilename=universal/c8/be/sstate:libffi-native:x86_64-linux:3.4.2:r0:x86_64:7:c8be7b784ce8ebbf6d897367bc8e96f6718040858d61f9a4f6aa5325821b0ce5_populate_sysroot.tar.zst.siginfo, attempting MIRRORS if available ERROR: libffi-native-3.4.2-r0 do_populate_sysroot_setscene: Fetcher failure: Unable to find file file://universal/c8/be/sstate:libffi-native:x86_64-linux:3.4.2:r0:x86_64:7:c8be7b784ce8ebbf6d897367bc8e96f6718040858d61f9a4f6aa5325821b0ce5_populate_sysroot.tar.zst.siginfo;downloadfilename=universal/c8/be/sstate:libffi-native:x86_64-linux:3.4.2:r0:x86_64:7:c8be7b784ce8ebbf6d897367bc8e96f6718040858d61f9a4f6aa5325821b0ce5_populate_sysroot.tar.zst.siginfo anywhere. The paths that were searched were: /media/mike/ssd/yocto/poky/build/sstate-cache /media/mike/ssd/yocto/poky/build/sstate-cache WARNING: libpcre2-native-10.39-r0 do_populate_sysroot_setscene: Failed to fetch URL file://universal/bf/03/sstate:libpcre2-native:x86_64-linux:10.39:r0:x86_64:7:bf035286ff06470377fe9c9298ba116222c9d557f2fa44d418beb919aa185db9_populate_sysroot.tar.zst.siginfo;downloadfilename=universal/bf/03/sstate:libpcre2-native:x86_64-linux:10.39:r0:x86_64:7:bf035286ff06470377fe9c9298ba116222c9d557f2fa44d418beb919aa185db9_populate_sysroot.tar.zst.siginfo, attempting MIRRORS if available ERROR: libpcre2-native-10.39-r0 do_populate_sysroot_setscene: Fetcher failure: Unable to find file file://universal/bf/03/sstate:libpcre2-native:x86_64-linux:10.39:r0:x86_64:7:bf035286ff06470377fe9c9298ba116222c9d557f2fa44d418beb919aa185db9_populate_sysroot.tar.zst.siginfo;downloadfilename=universal/bf/03/sstate:libpcre2-native:x86_64-linux:10.39:r0:x86_64:7:bf035286ff06470377fe9c9298ba116222c9d557f2fa44d418beb919aa185db9_populate_sysroot.tar.zst.siginfo anywhere. The paths that were searched were: /media/mike/ssd/yocto/poky/build/sstate-cache /media/mike/ssd/yocto/poky/build/sstate-cache ERROR: libpcre2-native-10.39-r0 do_populate_sysroot_setscene: No suitable staging package found ERROR: Logfile of failure stored in: /media/mike/ssd/yocto/poky/build/tmp/work/x86_64-linux/libpcre2-native/10.39-r0/temp/log.do_populate_sysroot_setscene.315121 WARNING: Setscene task (virtual:native:/media/mike/ssd/yocto/poky/meta/recipes-support/libpcre/libpcre2_10.39.bb:do_populate_sysroot_setscene) failed with exit code '1' - real task will be run instead Currently 4 running tasks (1838 of 1838/467 of 4630) 10% |################ As expected, the error messages are highlighted in red, but they are not critical as BitBake can always build the corresponding recipes from sources. Two questions: * Anything wrong with my local sstate cache? Should I erase it? * Should these issues really be treated as errors, scaring users that something could be wrong while the resulting build looks correct anyway? Thanks in advance Michael. -- Michael Opdenacker, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
|
|
|
|
Jose Quaresma
Hi, I've seen the same for some time now when building core-image-minimal with yocto state mirror and hashequiv servers. After some investigation I think the root cause is some bandwidth/connection limitation on the http yocto mirror. Reducing the BB_NUMBER_THREADS="1" to a single core solves many of the fetch issues for me, my config is the default one with: BB_NUMBER_THREADS="1" SSTATE_MIRRORS = "file://.* http://sstate.yoctoproject.org/dev/PATH;downloadfilename=PATH" BB_HASHSERVE_UPSTREAM = "typhoon.yocto.io:8687" Jose Michael Opdenacker <michael.opdenacker@...> escreveu no dia quinta, 13/01/2022 à(s) 17:23:
Hi, --
Best regards, José Quaresma
|
|
|
|
Joshua Watt
On Thu, Jan 13, 2022 at 11:23 AM Michael Opdenacker
<michael.opdenacker@bootlin.com> wrote: This usually happens when there is a sstate archive (.tar.zst), but no corresponding siginfo file (.tar.zst.siginfo). The sstate code assumes the siginfo file is present if archive is present, and "errors" if not. In my experience, this usually happens when you try to clean out the sstate cache using a mechanism that is unaware of the relationship and deletes the siginfo and not the archive (e.g. some form of "find -delete ...") * Should these issues really be treated as errors, scaring users thatAs noted by the build, it will actually just execute the missing tasks; one annoyance (depending on context) is that bitbake will return a non-zero exit code if any ERROR occurs in the build, even though in this case it was recovered. This has been discussed at length before (I can't find a citation ATM), and I believe it was left this way because this particular case is considered an actual error that should never happen and needs investigation on the YP Autobuilder.
|
|
|
|
Michael Opdenacker
Hi Jose,
Thanks for your advice! On 1/13/22 6:37 PM, Jose Quaresma wrote: Hi, Oops, it seems you're right. I have the same SSTATE_MIRRORS and BB_HASHSERVE_UPSTREAM settings as you do, and adding BB_NUMBER_THREADS="1" makes the errors disappear. It seems I should file a bug about that. The error messages just mention local paths, not remote ones, so it sounds like a local issue while it's apparently not. Thanks again Michael. -- Michael Opdenacker, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
|
|
|
|
Richard Purdie
On Thu, 2022-01-13 at 19:37 +0100, Michael Opdenacker wrote:
Hi Jose,That is useful to debug it but suggests there is an issue with concurrent connections to the YP server. I think we should have an open bug and get Michael Halstead's input on this. Cheers, Richard
|
|
|
|
Jose Quaresma
Richard Purdie <richard.purdie@...> escreveu no dia quinta, 13/01/2022 à(s) 21:53: On Thu, 2022-01-13 at 19:37 +0100, Michael Opdenacker wrote: During the last yocto project summit 2021.11 I have submitted a patch to reduce the time needed for handling the sstate mirrors on startup, that without my realizing it actually launches a DoS attack to the mirror. This patch was rejected as expected. After that patch, maybe by coincidence I got the idea that some measures had been taken to mitigate this problem but if it wasn't the case then there is a bug in concurrent connections to the YP server. When the patch was sent there was no problem with the concurrent connections to the YP server. Jose
|
|
|
|
Michael Opdenacker
On 1/13/22 10:53 PM, Richard Purdie wrote:
On Thu, 2022-01-13 at 19:37 +0100, Michael Opdenacker wrote:Hi Jose,That is useful to debug it but suggests there is an issue with concurrent Done: https://bugzilla.yoctoproject.org/show_bug.cgi?id=14690 Thanks for your advice. Cheers Michael. -- Michael Opdenacker, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
|
|
|
|
Jose Quaresma
Hi, There are no problems with the concurrent connections. I'm more sure that the root cause is some bandwidth/connection limitation on the http YP mirror. Jose Michael Opdenacker <michael.opdenacker@...> escreveu no dia sexta, 14/01/2022 à(s) 19:16:
--
Best regards, José Quaresma
|
|
|