Swat is on a holiday break
Ross Burton <ross@...>
Hi,
Because it's the holidays for many of us, Swat is taking a two week break. It will resume on Monday 4th January. With any luck, the autobuilder will be stable over that two week period! Ross |
|
Update to Swat process
Ross Burton <ross@...>
Hi,
I've updated the Swat process wiki page to hopefully make it clearer and explain the process better: https://wiki.yoctoproject.org/wiki/Yocto_Build_Failure_Swat_Team Please do read this and let me know if there's anything that isn't clear! Also, there are still people on this list who are not listed under Members on that page. As the list on the wiki page is how we rotate between people on call, can everyone please ensure that they're present. Thanks, Ross |
|
Autobuilder pending failures
Richard Purdie
The autobuilder doesn't stop running over the holidays so I'm going to
list the failures I haven't been able to log/handle (and I'll probably reply to this to add new ones if I spot them): initramfs-boot reproducibility failure: https://autobuilder.yoctoproject.org/typhoon/#/builders/56/builds/1291/steps/14/logs/stdio qemuppc-alt ping timeout: https://autobuilder.yoctoproject.org/typhoon/#/builders/107/builds/1207 intermittent arm ptest failures: https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1364 https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1359 (the exact ones change a bit each run but we really need open bugs for each one so that someone can ultimately investigate and fix them) qemumips64 shutdown backtrace: https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/2836 If anyone does have some time to file/update bugs or investigate any of the above please feel free and reply to the email. If not, we'll get to things in January unless I get bored! master-next also have something toxic to oe-selftest in it which I've yet to track down. I plan to keep -next running at a slower pace just to keep on top of incoming patches as not everywhere is on holiday right now. Cheers, Richard |
|
SWAT Rotation
Stephen Jolley <sjolley.yp.pm@...>
Naveen,
SWAT will rotate from Paul to you at EOD 12/25/2020. Please RSVP ASAP that you know you are on SWAT!
See: https://wiki.yoctoproject.org/wiki/Yocto_Build_Failure_Swat_Team#Members
Thanks,
Stephen K. Jolley Yocto Project Program Manager ( Cell: (208) 244-4460 * Email: sjolley.yp.pm@...
|
|
Re: SWAT Rotation
Naveen Saini
Acknowledged.
Regards, Naveen
From: swat@... <swat@...>
On Behalf Of Stephen Jolley
Sent: Friday, December 25, 2020 9:56 AM To: Saini, Naveen Kumar <naveen.kumar.saini@...> Cc: swat@...; bluelightning@... Subject: [swat] SWAT Rotation
Naveen,
SWAT will rotate from Paul to you at EOD 12/25/2020. Please RSVP ASAP that you know you are on SWAT!
See: https://wiki.yoctoproject.org/wiki/Yocto_Build_Failure_Swat_Team#Members
Thanks,
Stephen K. Jolley Yocto Project Program Manager ( Cell: (208) 244-4460 * Email: sjolley.yp.pm@...
|
|
Re: SWAT Rotation
Denys Dmytriyenko
FYI, SWAT is on a holiday break until January 4th:
toggle quoted message
Show quoted text
https://lists.yoctoproject.org/g/swat/message/21 -- Denys On Mon, Dec 28, 2020 at 04:02:01AM +0000, Naveen Saini wrote:
Acknowledged. --
Regards, Denys Dmytriyenko <denis@...> PGP: 0x420902729A92C964 - https://denix.org/0x420902729A92C964 Fingerprint: 25FC E4A5 8A72 2F69 1186 6D76 4209 0272 9A92 C964 |
|
Re: Autobuilder pending failures
Naveen Saini
Hi Richard,
toggle quoted message
Show quoted text
-----Original Message----- [Naveen] Cause of failure: device node permission bits are not set (/dev/console). From the recipe do_install(): mknod -m 622 ${D}/dev/console c 5 1 From diff-html: reproducibleA/tmp/deploy/deb/all/initramfs-boot_1.0-r2_all.deb vs reproducibleB/tmp/deploy/deb/all/initramfs-boot_1.0-r2_all.deb 3 crw--w--w-···0·root·········(0)·root·········(0)····5,··1·1970-01-01·00:00:00.000000·./dev/console 3 crw-------···0·root·········(0)·root·········(0)····5,··1·1970-01-01·00:00:00.000000·./dev/console Raised Bug#14162 [https://bugzilla.yoctoproject.org/show_bug.cgi?id=14162] [Naveen] Ping test is failing only with core-image-sato, but passing with core-image-sato-sdk and core-image-full-cmdline. Not sure why it is happening. This issue is observed with master-next. Should I raise bug ? [Naveen] Bug raised. libevent: https://bugzilla.yoctoproject.org/show_bug.cgi?id=14163 libinput: https://bugzilla.yoctoproject.org/show_bug.cgi?id=14164 strace: https://bugzilla.yoctoproject.org/show_bug.cgi?id=14165 [Naveen] Bug raised #14166 [ https://bugzilla.yoctoproject.org/show_bug.cgi?id=14166]
|
|
SWAT Rotation
Stephen Jolley <sjolley.yp.pm@...>
Armin,
SWAT will rotate from Naveen to you at EOD 01/01/2021. Please RSVP ASAP that you know you are on SWAT!
See: https://wiki.yoctoproject.org/wiki/Yocto_Build_Failure_Swat_Team#Members
Thanks,
Stephen K. Jolley Yocto Project Program Manager ( Cell: (208) 244-4460 * Email: sjolley.yp.pm@...
|
|
Re: SWAT Rotation
akuster <akuster808@...>
Stephen,
toggle quoted message
Show quoted text
Thanks for the reminder. If anyone is building on 1/1/2021, they need their head examined. I will do my best over the Holiday weekend. See you next Yr - armin On 12/31/20 12:01 PM, sjolley.yp.pm@... wrote:
|
|
Re: SWAT Rotation
Richard Purdie
On Thu, 2020-12-31 at 13:24 -0800, akuster wrote:
Thanks for the reminder.That sounds like me! :) Thanks Naveen, you've done well at tracking the issues that came up and its much appreciated. There have been quite a few more than expected, despite a fairly low number of builds! Happy New Year everyone! Cheers, Richard |
|
Re: SWAT Rotation
Naveen Saini
Hi Richard,
toggle quoted message
Show quoted text
I have located few more old build failures (master-next): (1) ptest failure on qemux86-64-ptest (Looks intermittent, I will raise Bug) AssertionError: Failed ptests: {'tcl': ['socket.test']} https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1654 (2) test_ltp_groups test failure (Should I raise bug for this ?) Reason: Looks like it's due to ssh connection failure RESULTS - ltp.LtpTest.test_ltp_groups: ERROR (10948.27s) SUMMARY: core-image-sato () - Ran 6 tests in 16385.717s core-image-sato - FAIL - Required tests failed (successes=4, skipped=1, failures=0, errors=1) https://autobuilder.yoctoproject.org/typhoon/#/builders/95/builds/1476 (3) Reproducibility testcase failing due to missing pulseaudio dependency (debian) (need to analysis, which patch causing this failure) pulseaudio was skipped: pulseaudio - pulseaudio-server: system username pulse does not have a static ID defined. Add pulse to one of these files: /home/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-17416/meta-selftest/files/static-passwd ERROR: Required build target 'meta-world-pkgdata' has no buildable providers. Missing or unbuildable dependency chain was: ['meta-world-pkgdata', 'gstreamer1.0-plugins-good', 'mpg123', 'pulseaudio'] https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/1660 Centos: Missing or unbuildable dependency chain was: ['meta-world-pkgdata', 'alsa-plugins', 'pulseaudio'] https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/1678 Fedora: https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/1665 Ubuntu: https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/1690 (4) ptest failure on qemux86-64 ( Seems intermittent, already available for arm machine) AssertionError: Failed ptests: {'strace': ['qual_fault-syscall.test', 'qual_fault.test']} https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/1698 Regards, Naveen -----Original Message----- |
|
Re: SWAT Rotation
Richard Purdie
Hi Naveen,
Thanks for looking at these! On Tue, 2021-01-05 at 02:38 +0000, Saini, Naveen Kumar wrote: Hi Richard,Bug for this one sounds good, thanks! (2) test_ltp_groups test failure (Should I raise bug for this ?)I think there is an open ltp ssh connection issue so we probably need to mention this failure on that bug? (3) Reproducibility testcase failing due to missing pulseaudioThese ones have patches in master which fix this so this one needs no further action. (4) ptest failure on qemux86-64 ( Seems intermittent, alreadyIs this the same strace test failing as the arm failure? If not it should be a separate bug. Cheers, Richard |
|
Re: SWAT Rotation
Naveen Saini
New bug:
toggle quoted message
Show quoted text
Tcl ptest failure: https://bugzilla.yoctoproject.org/show_bug.cgi?id=14177 Updated existing bug: Ltp test failure: https://bugzilla.yoctoproject.org/show_bug.cgi?id=13802 Strace qemux86-64 ptest failure (same as arm): https://bugzilla.yoctoproject.org/show_bug.cgi?id=14165 Regards, Naveen -----Original Message----- |
|
Current autobuilder status
Richard Purdie
I'm not sure what is happening with SWAT but we're in danger of losing
track of issues which is sad after Naveen did a great job and had everything up to date. I'm aware of the following: https://autobuilder.yoctoproject.org/typhoon/#/builders/56/builds/1305 https://autobuilder.yoctoproject.org/typhoon/#/builders/56/builds/1306 both failed in initramfs-boot, already open bug, fix in master-next. https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/2876/steps/14/logs/stdio https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/2890/steps/14/logs/stdio failure running apt-get update. Second time we've seen it, was on debian10-ty-1 here, denian10-ty-3 last time. pseudo.log says: path mismatch [1 link]: ino 143525936 db '/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/apt/lists/partial/_home_pokybuild_yocto-worker_pkgman-non-rpm_build_build_tmp_work_qemux86-poky-linux_core-image-sato_1.0-r0_oe-rootfs-repo_qemux86_._Release.CBFzPd' req '/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/apt/lists/partial/_home_pokybuild_yocto-worker_pkgman-non-rpm_build_build_tmp_work_qemux86-poky-linux_core-image-sato_1.0-r0_oe-rootfs-repo_qemux86_._Release'. similar to the last time. path mismatch [1 link]: ino 27666766 db '/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/apt/lists/partial/_home_pokybuild_yocto-worker_pkgman-non-rpm_build_build_tmp_work_qemux86-poky-linux_core-image-sato_1.0-r0_oe-rootfs-repo_all_._Release.RpRzHw' req '/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/apt/lists/partial/_home_pokybuild_yocto-worker_pkgman-non-rpm_build_build_tmp_work_qemux86-poky-linux_core-image-sato_1.0-r0_oe-rootfs-repo_all_._Release' https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/1720/steps/15/logs/stdio new issue where dnf is complaining checksums are incorrect https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/1716/steps/15/logs/stdio ppp intermittent reproducibility issue https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/1688/steps/14/logs/stdio bash reproducibility issue arm ptest warnings which will need to be added to bugs or bugs opened: https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1403 https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1401 https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1400 https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1399 x86 ptest warnings which will need to be added to bugs or bugs opened: https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1680 https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1676 perl install failure in man pages, think open bug needs updating? https://autobuilder.yoctoproject.org/typhoon/#/builders/47/builds/2909 The issue with new qemu needing python 3.6+ is fixed by changes in autobuilder-helper for debian9. I also resolved the gio postinst execution issue with new qemu. The mingw failure with new qemu is known Joshua is looking into that. Cheers, Richard |
|
arm worker unwell
Richard Purdie
I've also noticed one of the arm workers appears unwell:
https://autobuilder.yoctoproject.org/typhoon/#/workers/39 Michael: Can you take a look when you're up tomorrow please? Thanks, Richard |
|
Re: Current autobuilder status
On 1/7/21 3:07 AM, Richard Purdie wrote:
I'm not sure what is happening with SWAT but we're in danger of losingMy work schedule is such that I have no time to put into this activity. -armin
|
|
SWAT Rotation
Stephen Jolley <sjolley.yp.pm@...>
Christopher,
SWAT will rotate from Armin to you at EOD 01/08/2021. Please RSVP ASAP that you know you are on SWAT!
See: https://wiki.yoctoproject.org/wiki/Yocto_Build_Failure_Swat_Team#Members
Thanks,
Stephen K. Jolley Yocto Project Program Manager ( Cell: (208) 244-4460 * Email: sjolley.yp.pm@...
|
|
Re: Current autobuilder status
Richard Purdie
To try and ensure we don't lose track of issues:
On Thu, 2021-01-07 at 11:07 +0000, Richard Purdie via lists.yoctoproject.org wrote: I'm not sure what is happening with SWAT but we're in danger of losingFix merged, bug closed so resolved. https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/2876/steps/14/logs/stdioStill needs filing. https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/1720/steps/15/logs/stdioStill needs filing. https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/1716/steps/15/logs/stdioIssue resolved, patch sent upstream. https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/1688/steps/14/logs/stdioIssue resolved, patch queued in master-next with the bash upgrade. arm ptest warnings which will need to be added to bugs or bugs opened:Still needs filing. x86 ptest warnings which will need to be added to bugs or bugs opened:Still needs filing. perl install failure in man pages, think open bug needs updating?Added to https://bugzilla.yoctoproject.org/show_bug.cgi?id=14096 so done. The issue with new qemu needing python 3.6+ is fixed by changes inqemu upgrade merged, issues resolved. Still need to upstream the mingw issue. New issues: vulkan-samples-staticdev reproducibility issue: https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20210109-uud35nir/packages/reproducibleA/tmp/deploy/deb/core2-64/ https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/1712 Intermittent ssh failure: https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/2936 qemuarm rcu backtrace and QA failure: https://autobuilder.yoctoproject.org/typhoon/#/builders/53/builds/2921 bitbake pickling failure: https://autobuilder.yoctoproject.org/typhoon/#/builders/101/builds/1800 [have seen this very occasionally before, evidently previous fixes haven't solved everything, bugs were closed for it I think :(] Doing my best to try and keep on top of some of the issues! Cheers, Richard |
|
Re: Current autobuilder status
Ross Burton <ross@...>
https://bugzilla.yoctoproject.org/show_bug.cgi?id=14183https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/2876/steps/14/logs/stdioStill needs filing. https://bugzilla.yoctoproject.org/show_bug.cgi?id=14180https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/1720/steps/15/logs/stdioStill needs filing. These have all been filed already.arm ptest warnings which will need to be added to bugs or bugs opened:Still needs filing. All filed already.x86 ptest warnings which will need to be added to bugs or bugs opened:Still needs filing. vulkan-samples-staticdev reproducibility issue:https://bugzilla.yoctoproject.org/show_bug.cgi?id=14182 Intermittent ssh failure:Nothing useful in the log beyond "cannot connect to host". qemuarm rcu backtrace and QA failure:https://bugzilla.yoctoproject.org/show_bug.cgi?id=14184 bitbake pickling failure:https://bugzilla.yoctoproject.org/show_bug.cgi?id=14181 Ross |
|
SWAT Rotation
Stephen Jolley <sjolley.yp.pm@...>
Lee Chee,
SWAT will rotate from Christopher to you at EOD 01/15/2021. Please RSVP ASAP that you know you are on SWAT!
See: https://wiki.yoctoproject.org/wiki/Yocto_Build_Failure_Swat_Team#Members
Thanks,
Stephen K. Jolley Yocto Project Program Manager ( Cell: (208) 244-4460 * Email: sjolley.yp.pm@...
|
|