On 2021-06-16 10:17 a.m., Richard Purdie wrote:

On Wed, 2021-06-16 at 08:56 -0400, Paul Gortmaker wrote:

[Re: [swat] ltp failures on autobuilder] On 11/06/2021 (Fri 14:19) Richard Purdie wrote:

On Fri, 2021-06-11 at 12:36 +0100, Richard Purdie via lists.yoctoproject.org wrote:

as a .cfg to the kernel and that still reproduced the crash. However:

CONFIG_DEBUG_KERNEL=y
CONFIG_CGROUP_DEBUG=y
CONFIG_SCHED_DEBUG=y
CONFIG_DEBUG_PREEMPT=y
# CONFIG_RCU_TRACE is not set
# CONFIG_X86_DEBUG_FPU is not set
# CONFIG_CONSOLE_POLL is not set
# CONFIG_DEBUG_INFO is not set
# CONFIG_KGDB is not set
# CONFIG_KGDB_HONOUR_BLOCKLIST is not set
# CONFIG_KGDB_SERIAL_CONSOLE is not set
# CONFIG_KGDB_LOW_LEVEL_TRAP is not set
# CONFIG_KGDB_KDB is not set
# CONFIG_KDB_KEYBOARD is not set
# CONFIG_DEBUG_MISC is not set

Isolated down to CONFIG_SCHED_DEBUG=y being the line which somehow "fixes"
the crash. I can enable all the above apart from that and we can reproduce
it.

Also, I changed gatesgarth to use qemu 5.2.0 copied in from hardknott and that
breaks it. Dropping the 27 CVE patches "fixes" it again. It is possible it
is one of the CVE fixes. Continuing to try and isolate.

For the mail archive trail, and for those not follwing the ongoing
research on IRC, we are hopeful that this fixes it.

https://lore.kernel.org/lkml/20210616125157.438837-1-paul.gortmaker@windriver.com/

Awesome work in tracking that down, much appreciated, thanks!

Curious what upstream will make of it now...

Dropped the wider list but as I said to Richard, I wanted to
confirm that this bug is really gone.

I ran 100 tests overnight and it seems to be dead Jim.


$ bitbake core-image-sato && \
for i in `seq 100`; do \
  echo "--- " $i " ---"; \
  timeout --kill-after=2m 10m bitbake core-image-sato -c testimage \
   && echo GOOD || echo BAD; \
done

$ ls -l tmp/work/qemux86_64-poky-linux/core-image-sato/1.0-r0/testimage/qemu_boot_log.202* | wc -l
100

$ grep -m1 BUG: tmp/work/qemux86_64-poky-linux/core-image-sato/1.0-r0/testimage/qemu_boot_log.2021* | wc -l
0

All the log files are there and roughly the same size
with similar but not idnetical contents due to differing timestamps and
non-deterministic ordering of some of the output.


../Randy

Cheers,

Richard

--
# Randy MacLeod
# Wind River Linux