Date   

Re: Current autobuilder status

Richard Purdie
 

To try and ensure we don't lose track of issues:

On Thu, 2021-01-07 at 11:07 +0000, Richard Purdie via lists.yoctoproject.org wrote:
I'm not sure what is happening with SWAT but we're in danger of losing
track of issues which is sad after Naveen did a great job and had
everything up to date. I'm aware of the following:

https://autobuilder.yoctoproject.org/typhoon/#/builders/56/builds/1305
https://autobuilder.yoctoproject.org/typhoon/#/builders/56/builds/1306
both failed in initramfs-boot, already open bug, fix in master-next.
Fix merged, bug closed so resolved.

https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/2876/steps/14/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/2890/steps/14/logs/stdio
failure running apt-get update. Second time we've seen it, was on
debian10-ty-1 here, denian10-ty-3 last time. pseudo.log says:
path mismatch [1 link]: ino 143525936 db '/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/apt/lists/partial/_home_pokybuild_yocto-worker_pkgman-non-rpm_build_build_tmp_work_qemux86-poky-linux_core-image-sato_1.0-r0_oe-rootfs-repo_qemux86_._Release.CBFzPd' req '/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/apt/lists/partial/_home_pokybuild_yocto-worker_pkgman-non-rpm_build_build_tmp_work_qemux86-poky-linux_core-image-sato_1.0-r0_oe-rootfs-repo_qemux86_._Release'.
similar to the last time.
path mismatch [1 link]: ino 27666766 db '/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/apt/lists/partial/_home_pokybuild_yocto-worker_pkgman-non-rpm_build_build_tmp_work_qemux86-poky-linux_core-image-sato_1.0-r0_oe-rootfs-repo_all_._Release.RpRzHw' req '/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/apt/lists/partial/_home_pokybuild_yocto-worker_pkgman-non-rpm_build_build_tmp_work_qemux86-poky-linux_core-image-sato_1.0-r0_oe-rootfs-repo_all_._Release'
Still needs filing.

https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/1720/steps/15/logs/stdio
new issue where dnf is complaining checksums are incorrect
Still needs filing.

https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/1716/steps/15/logs/stdio
ppp intermittent reproducibility issue
Issue resolved, patch sent upstream.

https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/1688/steps/14/logs/stdio
bash reproducibility issue
Issue resolved, patch queued in master-next with the bash upgrade.

arm ptest warnings which will need to be added to bugs or bugs opened:
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1403
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1401
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1400
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1399
Still needs filing.

x86 ptest warnings which will need to be added to bugs or bugs opened:
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1680
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1676
Still needs filing.

perl install failure in man pages, think open bug needs updating?
https://autobuilder.yoctoproject.org/typhoon/#/builders/47/builds/2909
Added to https://bugzilla.yoctoproject.org/show_bug.cgi?id=14096 so
done.

The issue with new qemu needing python 3.6+ is fixed by changes in
autobuilder-helper for debian9. I also resolved the gio postinst
execution issue with new qemu. The mingw failure with new qemu is known
Joshua is looking into that.
qemu upgrade merged, issues resolved. Still need to upstream the mingw
issue.

New issues:

vulkan-samples-staticdev reproducibility issue:
https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20210109-uud35nir/packages/reproducibleA/tmp/deploy/deb/core2-64/
https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/1712

Intermittent ssh failure:
https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/2936

qemuarm rcu backtrace and QA failure:
https://autobuilder.yoctoproject.org/typhoon/#/builders/53/builds/2921

bitbake pickling failure:
https://autobuilder.yoctoproject.org/typhoon/#/builders/101/builds/1800
[have seen this very occasionally before, evidently previous fixes
haven't solved everything, bugs were closed for it I think :(]

Doing my best to try and keep on top of some of the issues!

Cheers,

Richard


SWAT Rotation

Stephen Jolley <sjolley.yp.pm@...>
 

Christopher,

 

SWAT will rotate from Armin to you at EOD 01/08/2021.   Please RSVP ASAP that you know you are on SWAT!

 

See: https://wiki.yoctoproject.org/wiki/Yocto_Build_Failure_Swat_Team#Members

 

Thanks,

 

Stephen K. Jolley

Yocto Project Program Manager

(    Cell:                (208) 244-4460

* Email:              sjolley.yp.pm@...

 


Re: Current autobuilder status

Armin Kuster
 

On 1/7/21 3:07 AM, Richard Purdie wrote:
I'm not sure what is happening with SWAT but we're in danger of losing
track of issues which is sad after Naveen did a great job and had
everything up to date. I'm aware of the following:
My work schedule is such that I have no time to put into this activity.

-armin

https://autobuilder.yoctoproject.org/typhoon/#/builders/56/builds/1305
https://autobuilder.yoctoproject.org/typhoon/#/builders/56/builds/1306
both failed in initramfs-boot, already open bug, fix in master-next.

https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/2876/steps/14/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/2890/steps/14/logs/stdio
failure running apt-get update. Second time we've seen it, was on
debian10-ty-1 here, denian10-ty-3 last time. pseudo.log says:
path mismatch [1 link]: ino 143525936 db '/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/apt/lists/partial/_home_pokybuild_yocto-worker_pkgman-non-rpm_build_build_tmp_work_qemux86-poky-linux_core-image-sato_1.0-r0_oe-rootfs-repo_qemux86_._Release.CBFzPd' req '/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/apt/lists/partial/_home_pokybuild_yocto-worker_pkgman-non-rpm_build_build_tmp_work_qemux86-poky-linux_core-image-sato_1.0-r0_oe-rootfs-repo_qemux86_._Release'.
similar to the last time.
path mismatch [1 link]: ino 27666766 db '/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/apt/lists/partial/_home_pokybuild_yocto-worker_pkgman-non-rpm_build_build_tmp_work_qemux86-poky-linux_core-image-sato_1.0-r0_oe-rootfs-repo_all_._Release.RpRzHw' req '/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/apt/lists/partial/_home_pokybuild_yocto-worker_pkgman-non-rpm_build_build_tmp_work_qemux86-poky-linux_core-image-sato_1.0-r0_oe-rootfs-repo_all_._Release'

https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/1720/steps/15/logs/stdio
new issue where dnf is complaining checksums are incorrect

https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/1716/steps/15/logs/stdio
ppp intermittent reproducibility issue

https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/1688/steps/14/logs/stdio
bash reproducibility issue

arm ptest warnings which will need to be added to bugs or bugs opened:
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1403
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1401
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1400
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1399

x86 ptest warnings which will need to be added to bugs or bugs opened:
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1680
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1676

perl install failure in man pages, think open bug needs updating?
https://autobuilder.yoctoproject.org/typhoon/#/builders/47/builds/2909

The issue with new qemu needing python 3.6+ is fixed by changes in
autobuilder-helper for debian9. I also resolved the gio postinst
execution issue with new qemu. The mingw failure with new qemu is known
Joshua is looking into that.

Cheers,

Richard








arm worker unwell

Richard Purdie
 

I've also noticed one of the arm workers appears unwell:

https://autobuilder.yoctoproject.org/typhoon/#/workers/39

Michael: Can you take a look when you're up tomorrow please?

Thanks,

Richard


Current autobuilder status

Richard Purdie
 

I'm not sure what is happening with SWAT but we're in danger of losing
track of issues which is sad after Naveen did a great job and had
everything up to date. I'm aware of the following:

https://autobuilder.yoctoproject.org/typhoon/#/builders/56/builds/1305
https://autobuilder.yoctoproject.org/typhoon/#/builders/56/builds/1306
both failed in initramfs-boot, already open bug, fix in master-next.

https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/2876/steps/14/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/2890/steps/14/logs/stdio
failure running apt-get update. Second time we've seen it, was on
debian10-ty-1 here, denian10-ty-3 last time. pseudo.log says:
path mismatch [1 link]: ino 143525936 db '/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/apt/lists/partial/_home_pokybuild_yocto-worker_pkgman-non-rpm_build_build_tmp_work_qemux86-poky-linux_core-image-sato_1.0-r0_oe-rootfs-repo_qemux86_._Release.CBFzPd' req '/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/apt/lists/partial/_home_pokybuild_yocto-worker_pkgman-non-rpm_build_build_tmp_work_qemux86-poky-linux_core-image-sato_1.0-r0_oe-rootfs-repo_qemux86_._Release'.
similar to the last time.
path mismatch [1 link]: ino 27666766 db '/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/apt/lists/partial/_home_pokybuild_yocto-worker_pkgman-non-rpm_build_build_tmp_work_qemux86-poky-linux_core-image-sato_1.0-r0_oe-rootfs-repo_all_._Release.RpRzHw' req '/home/pokybuild/yocto-worker/pkgman-non-rpm/build/build/tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/apt/lists/partial/_home_pokybuild_yocto-worker_pkgman-non-rpm_build_build_tmp_work_qemux86-poky-linux_core-image-sato_1.0-r0_oe-rootfs-repo_all_._Release'

https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/1720/steps/15/logs/stdio
new issue where dnf is complaining checksums are incorrect

https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/1716/steps/15/logs/stdio
ppp intermittent reproducibility issue

https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/1688/steps/14/logs/stdio
bash reproducibility issue

arm ptest warnings which will need to be added to bugs or bugs opened:
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1403
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1401
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1400
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1399

x86 ptest warnings which will need to be added to bugs or bugs opened:
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1680
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1676

perl install failure in man pages, think open bug needs updating?
https://autobuilder.yoctoproject.org/typhoon/#/builders/47/builds/2909

The issue with new qemu needing python 3.6+ is fixed by changes in
autobuilder-helper for debian9. I also resolved the gio postinst
execution issue with new qemu. The mingw failure with new qemu is known
Joshua is looking into that.

Cheers,

Richard


Re: SWAT Rotation

Naveen Saini
 

New bug:
Tcl ptest failure: https://bugzilla.yoctoproject.org/show_bug.cgi?id=14177

Updated existing bug:
Ltp test failure: https://bugzilla.yoctoproject.org/show_bug.cgi?id=13802
Strace qemux86-64 ptest failure (same as arm): https://bugzilla.yoctoproject.org/show_bug.cgi?id=14165

Regards,
Naveen

-----Original Message-----
From: swat@lists.yoctoproject.org <swat@lists.yoctoproject.org> On Behalf
Of Richard Purdie
Sent: Tuesday, January 5, 2021 8:41 PM
To: Saini, Naveen Kumar <naveen.kumar.saini@intel.com>;
swat@lists.yoctoproject.org; akuster@mvista.com
Subject: Re: [swat] SWAT Rotation

Hi Naveen,

Thanks for looking at these!

On Tue, 2021-01-05 at 02:38 +0000, Saini, Naveen Kumar wrote:
Hi Richard,

I have located few more old build failures (master-next):

(1) ptest failure on qemux86-64-ptest (Looks intermittent, I will
raise Bug)

AssertionError: Failed ptests:
{'tcl': ['socket.test']}
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1654
Bug for this one sounds good, thanks!

(2) test_ltp_groups test failure (Should I raise bug for this ?)

Reason: Looks like it's due to ssh connection failure

RESULTS - ltp.LtpTest.test_ltp_groups: ERROR (10948.27s)
SUMMARY:
core-image-sato () - Ran 6 tests in 16385.717s core-image-sato - FAIL
- Required tests failed (successes=4, skipped=1, failures=0, errors=1)


https://autobuilder.yoctoproject.org/typhoon/#/builders/95/builds/1476
I think there is an open ltp ssh connection issue so we probably need to
mention this failure on that bug?


(3) Reproducibility testcase failing due to missing pulseaudio
dependency  (debian) (need to analysis, which patch causing this
failure)

pulseaudio was skipped: pulseaudio - pulseaudio-server: system
username pulse does not have a static ID defined. Add pulse to one of
these files: /home/pokybuild/yocto-worker/oe-selftest-
debian/build/build-st-17416/meta-selftest/files/static-passwd
ERROR: Required build target 'meta-world-pkgdata' has no buildable
providers.
Missing or unbuildable dependency chain was: ['meta-world-pkgdata',
'gstreamer1.0-plugins-good', 'mpg123', 'pulseaudio']


https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/1660
Centos:
Missing or unbuildable dependency chain was: ['meta-world-pkgdata',
'alsa-plugins', 'pulseaudio']

https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/1678
Fedora:

https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/1665
Ubuntu:

https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/1690
These ones have patches in master which fix this so this one needs no
further action.


(4) ptest failure on qemux86-64 ( Seems intermittent, already
available for arm machine)

AssertionError: Failed ptests:
{'strace': ['qual_fault-syscall.test', 'qual_fault.test']}

https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/1698
Is this the same strace test failing as the arm failure? If not it should be a
separate bug.

Cheers,

Richard






Re: SWAT Rotation

Richard Purdie
 

Hi Naveen,

Thanks for looking at these!

On Tue, 2021-01-05 at 02:38 +0000, Saini, Naveen Kumar wrote:
Hi Richard,

I have located few more old build failures (master-next):

(1) ptest failure on qemux86-64-ptest (Looks intermittent, I will
raise Bug)

AssertionError: Failed ptests:
{'tcl': ['socket.test']}
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1654
Bug for this one sounds good, thanks!

(2) test_ltp_groups test failure (Should I raise bug for this ?)

Reason: Looks like it's due to ssh connection failure

RESULTS - ltp.LtpTest.test_ltp_groups: ERROR (10948.27s)
SUMMARY:
core-image-sato () - Ran 6 tests in 16385.717s
core-image-sato - FAIL - Required tests failed (successes=4,
skipped=1, failures=0, errors=1)


https://autobuilder.yoctoproject.org/typhoon/#/builders/95/builds/1476
I think there is an open ltp ssh connection issue so we probably need
to mention this failure on that bug?


(3) Reproducibility testcase failing due to missing pulseaudio
dependency  (debian) (need to analysis, which patch causing this
failure)

pulseaudio was skipped: pulseaudio - pulseaudio-server: system
username pulse does not have a static ID defined. Add pulse to one of
these files: /home/pokybuild/yocto-worker/oe-selftest-
debian/build/build-st-17416/meta-selftest/files/static-passwd
ERROR: Required build target 'meta-world-pkgdata' has no buildable
providers.
Missing or unbuildable dependency chain was: ['meta-world-pkgdata',
'gstreamer1.0-plugins-good', 'mpg123', 'pulseaudio']


https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/1660
Centos:
Missing or unbuildable dependency chain was: ['meta-world-pkgdata',
'alsa-plugins', 'pulseaudio']

https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/1678
Fedora:

https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/1665
Ubuntu:

https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/1690
These ones have patches in master which fix this so this one needs no
further action.


(4) ptest failure on qemux86-64 ( Seems intermittent, already
available for arm machine)

AssertionError: Failed ptests:
{'strace': ['qual_fault-syscall.test', 'qual_fault.test']}

https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/1698
Is this the same strace test failing as the arm failure? If not it
should be a separate bug.

Cheers,

Richard


Re: SWAT Rotation

Naveen Saini
 

Hi Richard,

I have located few more old build failures (master-next):

(1) ptest failure on qemux86-64-ptest (Looks intermittent, I will raise Bug)

AssertionError: Failed ptests:
{'tcl': ['socket.test']}
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1654


(2) test_ltp_groups test failure (Should I raise bug for this ?)

Reason: Looks like it's due to ssh connection failure

RESULTS - ltp.LtpTest.test_ltp_groups: ERROR (10948.27s)
SUMMARY:
core-image-sato () - Ran 6 tests in 16385.717s
core-image-sato - FAIL - Required tests failed (successes=4, skipped=1, failures=0, errors=1)

https://autobuilder.yoctoproject.org/typhoon/#/builders/95/builds/1476



(3) Reproducibility testcase failing due to missing pulseaudio dependency (debian) (need to analysis, which patch causing this failure)

pulseaudio was skipped: pulseaudio - pulseaudio-server: system username pulse does not have a static ID defined. Add pulse to one of these files: /home/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-17416/meta-selftest/files/static-passwd
ERROR: Required build target 'meta-world-pkgdata' has no buildable providers.
Missing or unbuildable dependency chain was: ['meta-world-pkgdata', 'gstreamer1.0-plugins-good', 'mpg123', 'pulseaudio']

https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/1660
Centos:
Missing or unbuildable dependency chain was: ['meta-world-pkgdata', 'alsa-plugins', 'pulseaudio']
https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/1678
Fedora:
https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/1665
Ubuntu:
https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/1690



(4) ptest failure on qemux86-64 ( Seems intermittent, already available for arm machine)

AssertionError: Failed ptests:
{'strace': ['qual_fault-syscall.test', 'qual_fault.test']}
https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/1698

Regards,
Naveen

-----Original Message-----
From: swat@lists.yoctoproject.org <swat@lists.yoctoproject.org> On Behalf
Of Richard Purdie
Sent: Friday, January 1, 2021 7:47 AM
To: swat@lists.yoctoproject.org; sjolley.yp.pm@gmail.com;
akuster@mvista.com
Cc: Saini, Naveen Kumar <naveen.kumar.saini@intel.com>
Subject: Re: [swat] SWAT Rotation

On Thu, 2020-12-31 at 13:24 -0800, akuster wrote:
Thanks for the reminder.

If anyone is building on 1/1/2021, they need their head examined.  I
will do my best over the Holiday weekend. See you next Yr
That sounds like me! :)

Thanks Naveen, you've done well at tracking the issues that came up and its
much appreciated. There have been quite a few more than expected,
despite a fairly low number of builds!

Happy New Year everyone!

Cheers,

Richard






Re: SWAT Rotation

Richard Purdie
 

On Thu, 2020-12-31 at 13:24 -0800, akuster wrote:
Thanks for the reminder.

If anyone is building on 1/1/2021, they need their head examined.  I
will do my best over the Holiday weekend. See you next Yr
That sounds like me! :)

Thanks Naveen, you've done well at tracking the issues that came up and
its much appreciated. There have been quite a few more than expected,
despite a fairly low number of builds!

Happy New Year everyone!

Cheers,

Richard


Re: SWAT Rotation

akuster <akuster808@...>
 

Stephen,

Thanks for the reminder.

If anyone is building on 1/1/2021, they need their head examined.  I
will do my best over the Holiday weekend. See you next Yr

- armin

On 12/31/20 12:01 PM, sjolley.yp.pm@gmail.com wrote:

Armin,

 

SWAT will rotate from Naveen to you at EOD 01/01/2021.   Please RSVP
ASAP that you know you are on SWAT!

 

See:
https://wiki.yoctoproject.org/wiki/Yocto_Build_Failure_Swat_Team#Members

 

Thanks,

 

*/Stephen K. Jolley/**//*

*Yocto Project Program Manager*

(    *Cell*:                (208) 244-4460

* *Email*:              _sjolley.yp.pm@gmail.com
<mailto:sjolley.yp.pm@gmail.com>_

 


SWAT Rotation

Stephen Jolley <sjolley.yp.pm@...>
 

Armin,

 

SWAT will rotate from Naveen to you at EOD 01/01/2021.   Please RSVP ASAP that you know you are on SWAT!

 

See: https://wiki.yoctoproject.org/wiki/Yocto_Build_Failure_Swat_Team#Members

 

Thanks,

 

Stephen K. Jolley

Yocto Project Program Manager

(    Cell:                (208) 244-4460

* Email:              sjolley.yp.pm@...

 


Re: Autobuilder pending failures

Naveen Saini
 

Hi Richard,

-----Original Message-----
From: swat@lists.yoctoproject.org <swat@lists.yoctoproject.org> On Behalf
Of Richard Purdie
Sent: Wednesday, December 23, 2020 8:44 PM
To: swat@lists.yoctoproject.org
Subject: [swat] Autobuilder pending failures

The autobuilder doesn't stop running over the holidays so I'm going to list the
failures I haven't been able to log/handle (and I'll probably reply to this to
add new ones if I spot them):

initramfs-boot reproducibility failure:
https://autobuilder.yoctoproject.org/typhoon/#/builders/56/builds/1291/st
eps/14/logs/stdio

[Naveen] Cause of failure: device node permission bits are not set (/dev/console).
From the recipe do_install():
mknod -m 622 ${D}/dev/console c 5 1

From diff-html:

reproducibleA/tmp/deploy/deb/all/initramfs-boot_1.0-r2_all.deb vs
reproducibleB/tmp/deploy/deb/all/initramfs-boot_1.0-r2_all.deb

3 crw--w--w-···0·root·········(0)·root·········(0)····5,··1·1970-01-01·00:00:00.000000·./dev/console
3 crw-------···0·root·········(0)·root·········(0)····5,··1·1970-01-01·00:00:00.000000·./dev/console

Raised Bug#14162 [https://bugzilla.yoctoproject.org/show_bug.cgi?id=14162]



qemuppc-alt ping timeout:
https://autobuilder.yoctoproject.org/typhoon/#/builders/107/builds/1207
[Naveen] Ping test is failing only with core-image-sato, but passing with core-image-sato-sdk and core-image-full-cmdline. Not sure why it is happening. This issue is observed with master-next. Should I raise bug ?


intermittent arm ptest failures:
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1364
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1359
(the exact ones change a bit each run but we really need open bugs for each
one so that someone can ultimately investigate and fix them)
[Naveen] Bug raised.
libevent: https://bugzilla.yoctoproject.org/show_bug.cgi?id=14163
libinput: https://bugzilla.yoctoproject.org/show_bug.cgi?id=14164
strace: https://bugzilla.yoctoproject.org/show_bug.cgi?id=14165


qemumips64 shutdown backtrace:
https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/2836
[Naveen] Bug raised #14166 [ https://bugzilla.yoctoproject.org/show_bug.cgi?id=14166]


If anyone does have some time to file/update bugs or investigate any of the
above please feel free and reply to the email. If not, we'll get to things in
January unless I get bored!

master-next also have something toxic to oe-selftest in it which I've yet to
track down. I plan to keep -next running at a slower pace just to keep on top
of incoming patches as not everywhere is on holiday right now.

Cheers,

Richard








Re: SWAT Rotation

Denys Dmytriyenko
 

FYI, SWAT is on a holiday break until January 4th:
https://lists.yoctoproject.org/g/swat/message/21

--
Denys

On Mon, Dec 28, 2020 at 04:02:01AM +0000, Naveen Saini wrote:
Acknowledged.

Regards,
Naveen

From: swat@lists.yoctoproject.org <swat@lists.yoctoproject.org> On Behalf Of Stephen Jolley
Sent: Friday, December 25, 2020 9:56 AM
To: Saini, Naveen Kumar <naveen.kumar.saini@intel.com>
Cc: swat@lists.yoctoproject.org; bluelightning@bluelightning.org
Subject: [swat] SWAT Rotation

Naveen,

SWAT will rotate from Paul to you at EOD 12/25/2020. Please RSVP ASAP that you know you are on SWAT!

See: https://wiki.yoctoproject.org/wiki/Yocto_Build_Failure_Swat_Team#Members

Thanks,

Stephen K. Jolley
Yocto Project Program Manager
* Cell: (208) 244-4460
* Email: sjolley.yp.pm@gmail.com<mailto:sjolley.yp.pm@gmail.com>






--
Regards,
Denys Dmytriyenko <denis@denix.org>
PGP: 0x420902729A92C964 - https://denix.org/0x420902729A92C964
Fingerprint: 25FC E4A5 8A72 2F69 1186 6D76 4209 0272 9A92 C964


Re: SWAT Rotation

Naveen Saini
 

Acknowledged.

 

Regards,

Naveen

 

From: swat@... <swat@...> On Behalf Of Stephen Jolley
Sent: Friday, December 25, 2020 9:56 AM
To: Saini, Naveen Kumar <naveen.kumar.saini@...>
Cc: swat@...; bluelightning@...
Subject: [swat] SWAT Rotation

 

Naveen,

 

SWAT will rotate from Paul to you at EOD 12/25/2020.   Please RSVP ASAP that you know you are on SWAT!

 

See: https://wiki.yoctoproject.org/wiki/Yocto_Build_Failure_Swat_Team#Members

 

Thanks,

 

Stephen K. Jolley

Yocto Project Program Manager

(    Cell:                (208) 244-4460

* Email:              sjolley.yp.pm@...

 


SWAT Rotation

Stephen Jolley <sjolley.yp.pm@...>
 

Naveen,

 

SWAT will rotate from Paul to you at EOD 12/25/2020.   Please RSVP ASAP that you know you are on SWAT!

 

See: https://wiki.yoctoproject.org/wiki/Yocto_Build_Failure_Swat_Team#Members

 

Thanks,

 

Stephen K. Jolley

Yocto Project Program Manager

(    Cell:                (208) 244-4460

* Email:              sjolley.yp.pm@...

 


Autobuilder pending failures

Richard Purdie
 

The autobuilder doesn't stop running over the holidays so I'm going to
list the failures I haven't been able to log/handle (and I'll probably
reply to this to add new ones if I spot them):

initramfs-boot reproducibility failure:
https://autobuilder.yoctoproject.org/typhoon/#/builders/56/builds/1291/steps/14/logs/stdio

qemuppc-alt ping timeout:
https://autobuilder.yoctoproject.org/typhoon/#/builders/107/builds/1207

intermittent arm ptest failures:
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1364
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1359
(the exact ones change a bit each run but we really need open bugs for
each one so that someone can ultimately investigate and fix them)

qemumips64 shutdown backtrace:
https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/2836

If anyone does have some time to file/update bugs or investigate any of
the above please feel free and reply to the email. If not, we'll get to
things in January unless I get bored!

master-next also have something toxic to oe-selftest in it which I've
yet to track down. I plan to keep -next running at a slower pace just
to keep on top of incoming patches as not everywhere is on holiday
right now.

Cheers,

Richard


Update to Swat process

Ross Burton
 

Hi,

I've updated the Swat process wiki page to hopefully make it clearer
and explain the process better:

https://wiki.yoctoproject.org/wiki/Yocto_Build_Failure_Swat_Team

Please do read this and let me know if there's anything that isn't clear!

Also, there are still people on this list who are not listed under
Members on that page. As the list on the wiki page is how we rotate
between people on call, can everyone please ensure that they're
present.

Thanks,
Ross


Swat is on a holiday break

Ross Burton
 

Hi,

Because it's the holidays for many of us, Swat is taking a two week
break. It will resume on Monday 4th January.

With any luck, the autobuilder will be stable over that two week period!

Ross


Re: Yocto SWAT team

Ross Burton
 

Hi Oleksiy,

On Tue, 15 Dec 2020 at 17:09, Oleksiy Obitotskyy via
lists.yoctoproject.org <oobitots=cisco.com@lists.yoctoproject.org>
wrote:
We have plans to participate in yocto swat team from Cisco (Oleksiy Obitotskyy oobitots@cisco.com & Valerii Chernous vchernou@cisco.com). Could you provide more information about process (in addition to wiki https://wiki.yoctoproject.org/wiki/Yocto_Build_Failure_Swat_Team)?
I wrote that wiki page but am very aware that explaining something to
yourself is not the same as to someone else, so I'll expand the page
after this to make it clearer.


Here is what we have regarding yocto swat:
- we could monitor build status with console or by mail list. It's ok we have mail list or console and we could get error logs and some meta information about build like branch/reason, etc.
Yes, currently my preferred method is with the yocto-builds mailing
list, as the failure mails now include the build notes (comments from
the person starting the build), and links to the error reporting
service, the build log, and so on.

- criteria what to do with failed build is quite unclear for now. master and master-next branches is ok - we have to file bugs and or report to mail lists (?).
You're the second to not understand this so this obviously needs to be
rewritten!

The release branches (gatesgarth, dunfell, etc) and master are
critical. Any failure should be in bugzilla, either a new bug or
identifying an existing bug.

The other branches the policy is a little more fluid. master-next
should be monitored by Swat with discussion with RP. I believe Steve
does build monitoring himself with stable/dunfell so those can be left
to him. Other branches, the build starter should leave a comment in
the build notes: if I've started a build to exercise a patchset then
I'll often just put "SWAT: ignore" to say that Swat should ignore any
errors as I'll handle them myself.

I'm looking into how to have this information encoded into the build
directly so we can make it clear in the build failure mail.

- after pre-triage we should write something to buildlog wiki - this step is unclear;
Yes. Every failure in the build log should have a comment from Swat
about what the problem was. A link to a bug, or a note that a mail was
sent. Just something so that the build owner can look at the log and
see what happened.

- we have no access to bug tracker and wiki account, but already have access to swat/yocto-builds/yocto-main mail list.
Requesting accounts on both of those should be simple enough. If
you're having problems (the wiki is definitely
request-and-wait-for-approval) then contact Michael Halstead.

It would be nice to have some workflow example like:
- we got bug during build;
- decided on some criterion that we have to:
- send report (to what mail lists?);
- file bug (with some template/tags);
- or just ignore it.
- other steps.
Good idea. I shall update the wiki page today.

If anything else is unclear feel free to ask.

Cheers,
Ross


Re: SWAT Rotation

Paul Eggleton <bluelightning@...>
 

Acknowledged.

On Friday, 18 December 2020 04:46:29 NZDT sjolley.yp.pm@gmail.com wrote:
Paul,



SWAT will rotate from Alejandro to you at EOD 12/18/2020. Please RSVP ASAP
that you know you are on SWAT!



See:
https://wiki.yoctoproject.org/wiki/Yocto_Build_Failure_Swat_Team#Members



Thanks,



Stephen K. Jolley

Yocto Project Program Manager

* Cell: (208) 244-4460

* Email: sjolley.yp.pm@gmail.com
<mailto:sjolley.yp.pm@gmail.com>

161 - 180 of 198