Autobuilder pending failures
Richard Purdie
The autobuilder doesn't stop running over the holidays so I'm going to
list the failures I haven't been able to log/handle (and I'll probably reply to this to add new ones if I spot them): initramfs-boot reproducibility failure: https://autobuilder.yoctoproject.org/typhoon/#/builders/56/builds/1291/steps/14/logs/stdio qemuppc-alt ping timeout: https://autobuilder.yoctoproject.org/typhoon/#/builders/107/builds/1207 intermittent arm ptest failures: https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1364 https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1359 (the exact ones change a bit each run but we really need open bugs for each one so that someone can ultimately investigate and fix them) qemumips64 shutdown backtrace: https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/2836 If anyone does have some time to file/update bugs or investigate any of the above please feel free and reply to the email. If not, we'll get to things in January unless I get bored! master-next also have something toxic to oe-selftest in it which I've yet to track down. I plan to keep -next running at a slower pace just to keep on top of incoming patches as not everywhere is on holiday right now. Cheers, Richard |
|
Update to Swat process
Ross Burton <ross@...>
Hi,
I've updated the Swat process wiki page to hopefully make it clearer and explain the process better: https://wiki.yoctoproject.org/wiki/Yocto_Build_Failure_Swat_Team Please do read this and let me know if there's anything that isn't clear! Also, there are still people on this list who are not listed under Members on that page. As the list on the wiki page is how we rotate between people on call, can everyone please ensure that they're present. Thanks, Ross |
|
Swat is on a holiday break
Ross Burton <ross@...>
Hi,
Because it's the holidays for many of us, Swat is taking a two week break. It will resume on Monday 4th January. With any luck, the autobuilder will be stable over that two week period! Ross |
|
Re: Yocto SWAT team
Ross Burton <ross@...>
Hi Oleksiy,
On Tue, 15 Dec 2020 at 17:09, Oleksiy Obitotskyy via lists.yoctoproject.org <oobitots=cisco.com@...> wrote: We have plans to participate in yocto swat team from Cisco (Oleksiy Obitotskyy oobitots@... & Valerii Chernous vchernou@...). Could you provide more information about process (in addition to wiki https://wiki.yoctoproject.org/wiki/Yocto_Build_Failure_Swat_Team)?I wrote that wiki page but am very aware that explaining something to yourself is not the same as to someone else, so I'll expand the page after this to make it clearer. Yes, currently my preferred method is with the yocto-builds mailing list, as the failure mails now include the build notes (comments from the person starting the build), and links to the error reporting service, the build log, and so on. - criteria what to do with failed build is quite unclear for now. master and master-next branches is ok - we have to file bugs and or report to mail lists (?).You're the second to not understand this so this obviously needs to be rewritten! The release branches (gatesgarth, dunfell, etc) and master are critical. Any failure should be in bugzilla, either a new bug or identifying an existing bug. The other branches the policy is a little more fluid. master-next should be monitored by Swat with discussion with RP. I believe Steve does build monitoring himself with stable/dunfell so those can be left to him. Other branches, the build starter should leave a comment in the build notes: if I've started a build to exercise a patchset then I'll often just put "SWAT: ignore" to say that Swat should ignore any errors as I'll handle them myself. I'm looking into how to have this information encoded into the build directly so we can make it clear in the build failure mail. - after pre-triage we should write something to buildlog wiki - this step is unclear;Yes. Every failure in the build log should have a comment from Swat about what the problem was. A link to a bug, or a note that a mail was sent. Just something so that the build owner can look at the log and see what happened. - we have no access to bug tracker and wiki account, but already have access to swat/yocto-builds/yocto-main mail list.Requesting accounts on both of those should be simple enough. If you're having problems (the wiki is definitely request-and-wait-for-approval) then contact Michael Halstead. It would be nice to have some workflow example like:Good idea. I shall update the wiki page today. If anything else is unclear feel free to ask. Cheers, Ross |
|
Re: SWAT Rotation
Paul Eggleton <bluelightning@...>
Acknowledged.
toggle quoted message
Show quoted text
On Friday, 18 December 2020 04:46:29 NZDT sjolley.yp.pm@... wrote:
Paul, |
|
SWAT Rotation
Stephen Jolley <sjolley.yp.pm@...>
Paul,
SWAT will rotate from Alejandro to you at EOD 12/18/2020. Please RSVP ASAP that you know you are on SWAT!
See: https://wiki.yoctoproject.org/wiki/Yocto_Build_Failure_Swat_Team#Members
Thanks,
Stephen K. Jolley Yocto Project Program Manager ( Cell: (208) 244-4460 * Email: sjolley.yp.pm@...
|
|
Yocto SWAT team
Hi,
We have plans to participate in yocto swat team from Cisco (Oleksiy Obitotskyy oobitots@... & Valerii Chernous vchernou@...). Could you provide more information about process (in addition to wiki
https://wiki.yoctoproject.org/wiki/Yocto_Build_Failure_Swat_Team)?
Here is what we have regarding yocto swat:
- we could monitor build status with console or by mail list. It's ok we have mail list or console and we could get error logs and some meta information about build like branch/reason, etc.
- criteria what to do with failed build is quite unclear for now. master and master-next branches is ok - we have to file bugs and or report to mail lists (?).
- after pre-triage we should write something to buildlog wiki - this step is unclear;
- we have no access to bug tracker and wiki account, but already have access to swat/yocto-builds/yocto-main mail list.
It would be nice to have some workflow example like:
- we got bug during build;
- decided on some criterion that we have to:
- send report (to what mail lists?);
- file bug (with some template/tags);
- or just ignore it.
- other steps.
Regards, Oleksiy |
|
Re: [EXTERNAL] Re: [swat] SWAT Rotation
Ross Burton <ross@...>
Yes, according to the list it's Alejandro next week.
toggle quoted message
Show quoted text
Ross On Fri, 11 Dec 2020 at 02:23, Alejandro Hernandez Samaniego via lists.yoctoproject.org <Alejandro.Hernandez=microsoft.com@...> wrote:
|
|
Re: [EXTERNAL] Re: [swat] SWAT Rotation
Alejandro Hernandez Samaniego
I believe its my turn next week guys
Alejandro
From: swat@... <swat@...> on behalf of Anibal Limon via lists.yoctoproject.org <anibal.limon=linaro.org@...>
Sent: Thursday, December 10, 2020 2:37 PM To: swat@... <swat@...>; Denys Dmytriyenko <denis@...> Cc: Alejandro Hernandez <alhe@...>; Saul Wold <Saul.Wold@...> Subject: [EXTERNAL] Re: [swat] SWAT Rotation On Thu, 10 Dec 2020 at 15:35, Denys Dmytriyenko <denis@...> wrote:
Stephen, Hi,
I took the place of Saul about 1 week ago.
Regards,
Anibal
|
|
Re: SWAT Rotation
Anibal Limon
On Thu, 10 Dec 2020 at 15:35, Denys Dmytriyenko <denis@...> wrote: Stephen, Hi, I took the place of Saul about 1 week ago. Regards, Anibal
|
|
Re: SWAT Rotation
Denys Dmytriyenko
Stephen,
toggle quoted message
Show quoted text
FYI, I believe the name and the date below are incorrect. -- Denys On Thu, Dec 10, 2020 at 01:26:28PM -0800, Stephen Jolley wrote:
Saul, |
|
SWAT Rotation
Stephen Jolley <sjolley.yp.pm@...>
Saul,
SWAT will rotate from Leo to you at EOD 12/04/2020. Please RSVP ASAP that you know you are on SWAT!
See: https://wiki.yoctoproject.org/wiki/Yocto_Build_Failure_Swat_Team#Members
Thanks,
Stephen K. Jolley Yocto Project Program Manager ( Cell: (208) 244-4460 * Email: sjolley.yp.pm@...
|
|
SWAT Rotation
Stephen Jolley <sjolley.yp.pm@...>
Saul,
SWAT will rotate from Leo to you at EOD 12/04/2020. Please RSVP ASAP that you know you are on SWAT!
See: https://wiki.yoctoproject.org/wiki/Yocto_Build_Failure_Swat_Team#Members
Thanks,
Stephen K. Jolley Yocto Project Program Manager ( Cell: (208) 244-4460 * Email: sjolley.yp.pm@...
|
|
Swat process and update
Ross Burton <ross@...>
Hi all,
We've had two weeks of Swat being back in action now and I think it's progressing well on the whole. I've made several edits to the wiki page[1] to clarify the process, specifically: * Calling out that the coverage is from the "end of Friday". This is deliberately vague to avoid having to play games with timezones, but basically the person on Swat duty is relieved at the end of day on Friday. This means that the incoming person is responsible for any failures that happened over the weekend. I'll be talking to Stephen shortly so that he can handle the rotation, and we'll send the notification of who is next on call on Thursday so that there is time to arrange an alternative if required. * Reiterating that everyone looking at the builds should be updating the build log on failures. Myself and Richard have been designing a replacement for the buildlog but this hasn't progressed to actual code yet. If anyone is interested in assisting writing a Django webapp then please let us know. Finally, can everyone please ensure that they've added themselves to the wiki page[1]. This is a test that you are reading the list and have a wiki account: both requirements to do Swat! :). If you know anyone else who would be a good fit for the Swat team (just roughly 30 minutes a day, for a week, every few months) then get in touch. Cheers, Ross [1] https://wiki.yoctoproject.org/wiki/Yocto_Build_Failure_Swat_Team |
|
Re: [EXTERNAL] Re: [swat] Restarting SWAT
Richard Purdie
On Tue, 2020-11-10 at 14:28 +0000, Jon Mason wrote:
Feel free to add me on the end of the list. Also, it might beIt *is* on the wiki: https://wiki.yoctoproject.org/wiki/SWAT so you can add yourself somewhere as appropriate! Cheers, Richard |
|
Re: [EXTERNAL] Re: [swat] Restarting SWAT
Jon Mason
Outlook and top posting, hate me!
Feel free to add me on the end of the list. Also, it might be beneficial to put the list on the wiki so that we can trade weeks if there is a vacation, holiday, etc.
Thanks, Jon
From: swat@... <swat@...>
On Behalf Of Naveen Saini via lists.yoctoproject.org
Sent: Monday, November 9, 2020 6:52 PM To: swat@... Cc: Stephen K Jolley (via Google Docs) <sjolley.yp.pm@...> Subject: Re: [EXTERNAL] Re: [swat] Restarting SWAT
Hi all,
I am new to this process, but I can go after Alejandro.
Regards, Naveen
From: swat@... <swat@...>
On Behalf Of Alejandro Hernandez Samaniego via lists.yoctoproject.org
Hello everyone,
I'm glad we're setting this up again, just added myself to the Wiki header, I can go after Anibal for old times sake.
Alejandro From:
swat@... <swat@...> on behalf of Anibal Limon via lists.yoctoproject.org <anibal.limon=linaro.org@...>
On Mon, 9 Nov 2020 at 09:30, Saul Wold <Saul.Wold@...> wrote:
Hi all, I can go after Sau,
Cheers, Anibal
|
|
Re: [EXTERNAL] Re: [swat] Restarting SWAT
Naveen Saini
Hi all,
I am new to this process, but I can go after Alejandro.
Regards, Naveen
From: swat@... <swat@...>
On Behalf Of Alejandro Hernandez Samaniego via lists.yoctoproject.org
Sent: Tuesday, November 10, 2020 2:33 AM To: swat@... Cc: Stephen K Jolley (via Google Docs) <sjolley.yp.pm@...> Subject: Re: [EXTERNAL] Re: [swat] Restarting SWAT
Hello everyone,
I'm glad we're setting this up again, just added myself to the Wiki header, I can go after Anibal for old times sake.
Alejandro From:
swat@... <swat@...> on behalf of Anibal Limon via lists.yoctoproject.org <anibal.limon=linaro.org@...>
On Mon, 9 Nov 2020 at 09:30, Saul Wold <Saul.Wold@...> wrote:
Hi all, I can go after Sau,
Cheers, Anibal
|
|
Re: [EXTERNAL] Re: [swat] Restarting SWAT
Alejandro Hernandez Samaniego
Hello everyone,
I'm glad we're setting this up again, just added myself to the Wiki header, I can go after Anibal for old times sake.
Alejandro
From: swat@... <swat@...> on behalf of Anibal Limon via lists.yoctoproject.org <anibal.limon=linaro.org@...>
Sent: Monday, November 9, 2020 8:50 AM To: swat@... <swat@...> Cc: Stephen K Jolley (via Google Docs) <sjolley.yp.pm@...> Subject: [EXTERNAL] Re: [swat] Restarting SWAT On Mon, 9 Nov 2020 at 09:30, Saul Wold <Saul.Wold@...> wrote:
Hi all, I can go after Sau,
Cheers,
Anibal
|
|
Re: Restarting SWAT
Anibal Limon
On Mon, 9 Nov 2020 at 09:30, Saul Wold <Saul.Wold@...> wrote:
Hi all, I can go after Sau, Cheers, Anibal
|
|
Re: Restarting SWAT
Saul Wold
On 11/9/20 7:15 AM, Leonardo Sandoval wrote:
On Fri, 6 Nov 2020 at 12:27, Ross Burton <ross@... <mailto:ross@...>> wrote:Same here, I will let Leo go after Ross to help shake things out for us old-timers! After three years, I am happy to come back to the YP!+1! Sau! lsg-- Sau! |
|