Date   

Use of SDK for building images?

Leon Woestenberg
 

Hello all,

In some other build systems the generated SDK can be used to also
generate an image. Thus the SDK allows development against the target
sysroot using the prebuilt host and target tools, as well as
(re)generate the target images in quick iteration cycles.

What approaches are recommended with Yocto to achieve the same benefits?

My need is to regenerate an initramfs as well as compose this into an
image using WIC.

I am aware of shared state, and using prebuilt toolchains, but my
question is whether the prebuilt SDK allows to generate images?

Regards,

Leon.


[yocto-autobuilder2][PATCH] reporters/swatbot: sanitize urls

Alexandre Belloni
 

When the log name contains a space, the generated URL is not correct. Thi=
s
later also breaks parsing in swatbot. This was triggered by "property
changes" and the correct URL is indeed logs/property_changes.

Signed-off-by: Alexandre Belloni <alexandre.belloni@...>
---

This is untested because I don't have the setup handy but, I hope, it is
trivial enough to work properly


reporters/swatbot.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/reporters/swatbot.py b/reporters/swatbot.py
index 4a5a04eb1809..41ae762359a0 100644
--- a/reporters/swatbot.py
+++ b/reporters/swatbot.py
@@ -202,7 +202,7 @@ class SwatBotURI(object):
logs =3D list(logs)
urls =3D []
for l in logs:
- urls.append('%s/steps/%s/logs/%s' % (build['url'], step_=
number, l['name']))
+ urls.append('%s/steps/%s/logs/%s' % (build['url'], step_=
number, l['name'].replace(" ", "_")))
if urls:
urls =3D " ".join(urls)
else:
--=20
2.31.1


Re: [qa-build-notification] QA notification for completed autobuilder build (yocto-3.4_M1.rc1)

Sangeeta Jain
 

Hi all,

This is the full report for yocto-3.4_M1.rc1:
https://git.yoctoproject.org/cgit/cgit.cgi/yocto-testresults-contrib/tree/?h=intel-yocto-testresults

======= Summary ========
No high milestone defects.

new issue found

BUG id:14434 - [3.4 M1] dmesg: proc: Bad value for 'hidepid' with poky-altcfg distro

BUG id:14435 - [3.4 M1 beaglebone] Some drm error messages in dmesg


======= Bugs ========
https://bugzilla.yoctoproject.org/show_bug.cgi?14434
https://bugzilla.yoctoproject.org/show_bug.cgi?id=14435

Thanks,
Sangeeta

-----Original Message-----
From: qa-build-notification@... <qa-build-
notification@...> On Behalf Of Pokybuild User
Sent: Saturday, 12 June, 2021 7:49 PM
To: yocto@...
Cc: qa-build-notification@...
Subject: [qa-build-notification] QA notification for completed autobuilder build
(yocto-3.4_M1.rc1)


A build flagged for QA (yocto-3.4_M1.rc1) was completed on the autobuilder
and is available at:


https://autobuilder.yocto.io/pub/releases/yocto-3.4_M1.rc1


Build hash information:

bitbake: 398a1686176c695d103591089a36e25173f9fd6e
meta-arm: 6c3d62c776fc45b4bae47d178899e84b17248b31
meta-gplv2: 1ee1a73070d91e0c727f9d0db11943a61765c8d9
meta-intel: 0937728bcd98dd13d2c6829e1cd604ea2e53e5cd
meta-mingw: bfd22a248c0db4c57d5a72d675979d8341a7e9c1
oecore: 3b2903ccc791d5dedd84c75227f38ae4c8d29251
poky: 59d93693bf24e02ca0f05fe06d96a46f4f0f1bf8



This is an automated message from the Yocto Project Autobuilder
Git: git://git.yoctoproject.org/yocto-autobuilder2
Email: richard.purdie@...







[PATCH] bsps/5.10: update to v5.10.43

Bruce Ashfield
 

From: Bruce Ashfield <bruce.ashfield@...>

Updating linux-yocto/5.10 to the latest korg -stable release, and to
match the qemu BSPs in oe-core

Signed-off-by: Bruce Ashfield <bruce.ashfield@...>
---
.../linux/linux-yocto_5.10.bbappend | 16 ++++++++--------
1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/meta-yocto-bsp/recipes-kernel/linux/linux-yocto_5.10.bbappend b/meta-yocto-bsp/recipes-kernel/linux/linux-yocto_5.10.bbappend
index bc2b3bf576..f8362b6635 100644
--- a/meta-yocto-bsp/recipes-kernel/linux/linux-yocto_5.10.bbappend
+++ b/meta-yocto-bsp/recipes-kernel/linux/linux-yocto_5.10.bbappend
@@ -7,17 +7,17 @@ KMACHINE_genericx86 ?= "common-pc"
KMACHINE_genericx86-64 ?= "common-pc-64"
KMACHINE_beaglebone-yocto ?= "beaglebone"

-SRCREV_machine_genericx86 ?= "8c516ced69f41563404ada0bea315a55bcf1df6f"
-SRCREV_machine_genericx86-64 ?= "8c516ced69f41563404ada0bea315a55bcf1df6f"
-SRCREV_machine_edgerouter ?= "965ab3ab746ae8a1158617b6302d9c218ffbbb66"
-SRCREV_machine_beaglebone-yocto ?= "8c516ced69f41563404ada0bea315a55bcf1df6f"
+SRCREV_machine_genericx86 ?= "ab49d2db98bdee2c8c6e17fb59ded9e5292b0f41"
+SRCREV_machine_genericx86-64 ?= "ab49d2db98bdee2c8c6e17fb59ded9e5292b0f41"
+SRCREV_machine_edgerouter ?= "274d63799465eebfd201b3e8251f16d29e93a978"
+SRCREV_machine_beaglebone-yocto ?= "ab49d2db98bdee2c8c6e17fb59ded9e5292b0f41"

COMPATIBLE_MACHINE_genericx86 = "genericx86"
COMPATIBLE_MACHINE_genericx86-64 = "genericx86-64"
COMPATIBLE_MACHINE_edgerouter = "edgerouter"
COMPATIBLE_MACHINE_beaglebone-yocto = "beaglebone-yocto"

-LINUX_VERSION_genericx86 = "5.10.21"
-LINUX_VERSION_genericx86-64 = "5.10.21"
-LINUX_VERSION_edgerouter = "5.10.21"
-LINUX_VERSION_beaglebone-yocto = "5.10.21"
+LINUX_VERSION_genericx86 = "5.10.43"
+LINUX_VERSION_genericx86-64 = "5.10.43"
+LINUX_VERSION_edgerouter = "5.10.43"
+LINUX_VERSION_beaglebone-yocto = "5.10.43"
--
2.19.1


Re: [PATCH yocto-autobuilder-helper] summarize_top_output.py: add script, use it and publish summary

sakib.sajal@...
 

On 2021-06-16 11:41 a.m., Randy MacLeod wrote:
On 2021-06-16 4:43 a.m., sakib.sajal@... wrote:
summarize_top_output.py is used to summarize the top
output that is captured during autobuilder intermittent
failures.

Use the script to summarize the host top output and
publish the summary that is created instead of
the raw logfile.

Looks good Sakib,

Can you show people what the typical output looks like?
Is the raw top output still published?

../Randy
The script goes over the raw logfile (for example, foo.txt) which consists of multiple top outputs and summarizes each top output and writes them to foo_summary.txt

Some improvements that can be made:

1) create a separate file for each top summary, ie, if foo.txt has n top outputs, create foo_summary_[1...n].txt

2) since python3 is used to run many different kinds of jobs, it is difficult to generalize python3 threads. the script can be modified to log all the different jobs run under python3, right now it only counts the processes that are running python3.

Any feedback is welcome!

Sample output:

NOTE: program names have been shortened for better readability.
Substitutions are as follows:
~/ = /home/pokybuild/yocto-worker/
/.../POKY_32/.../ = /build/build/tmp/work/core2-32-poky-linux/
/.../POKY_64/.../ = /build/build/tmp/work/core2-64-poky-linux/
/...GCC.../ = /recipe-sysroot-native/usr/bin/x86_64-poky-linux/../../libexec/x86_64-poky-linux/gcc/x86_64-poky-linux/

top was invoked 12 times.

top - 16:51:30 up 4 days, 19:26,  1 user,  load average: 93.84, 57.26, 48.58
Tasks: 1084 total,  80 running, 664 sleeping,   0 stopped,   0 zombie
%Cpu(s):  2.1 us,  2.0 sy,  9.8 ni, 83.1 id,  2.9 wa,  0.0 hi, 0.0 si,  0.0 st
KiB Mem : 13192559+total,  5571436 free, 14307780 used, 11204637+buff/cache
KiB Swap:  8388604 total,  8239384 free,   149220 used. 11455193+avail Mem

Summary:
85  /bin/sh
41  python3
39  /bin/bash
30  x86_64-poky-linux-g++
29  make
16 ~/meta-oe/.../POKY_64/.../cpprest/2.10.18-r0/...GCC.../11.1.0/cc1plus
16 ~/meta-oe/.../POKY_64/.../libvpx/1.8.2-r0/...GCC.../11.1.0/cc1plus
16  ~/meta-oe/.../POKY_64/.../libvpx/1.8.2-r0/...GCC.../11.1.0/as
16 ~/meta-oe/.../POKY_64/.../cpprest/2.10.18-r0/recipe-sysroot-native/usr/bin/x86_64-poky-linux/x86_64-poky-linux-g++
16 ~/meta-oe/.../POKY_64/.../cpprest/2.10.18-r0/...GCC.../11.1.0/as
14 ~/meta-oe/.../POKY_64/.../rtorrent/0.9.8-r0/...GCC.../11.1.0/cc1plus
14 ~/meta-oe/.../POKY_64/.../rtorrent/0.9.8-r0/...GCC.../11.1.0/as
13 ~/meta-oe/.../POKY_64/.../vulkan-cts/1.2.6.0-r0/...GCC.../11.1.0/cc1plus
13 ~/meta-oe/.../POKY_64/.../vulkan-cts/1.2.6.0-r0/...GCC.../11.1.0/as
13 ~/meta-oe/.../POKY_64/.../vulkan-cts/1.2.6.0-r0/recipe-sysroot-native/usr/bin/x86_64-poky-linux/x86_64-poky-linux-g++
11  x86_64-poky-linux-gcc
10 ~/meta-oe/.../POKY_64/.../dovecot/2.3.14-r0/...GCC.../11.1.0/cc1
10 ~/meta-oe/.../POKY_64/.../fluentbit/1.3.5-r0/recipe-sysroot-native/usr/bin/x86_64-poky-linux/x86_64-poky-linux-gcc
10 ~/meta-oe/.../POKY_64/.../dovecot/2.3.14-r0/...GCC.../11.1.0/as
7 ~/meta-oe/.../POKY_64/.../fluentbit/1.3.5-r0/...GCC.../11.1.0/cc1
7 ~/meta-oe/.../POKY_64/.../fluentbit/1.3.5-r0/...GCC.../11.1.0/as
4  /usr/bin/python3
3  cmake
2 ~/meta-oe/.../POKY_64/.../vulkan-cts/1.2.6.0-r0/...GCC.../11.1.0/ar
2  sh
2  /lib/systemd/systemd
2  (sd-pam)
2  ninja
2  bitbake-server
2 ~/meta-oe/.../POKY_64/.../vulkan-cts/1.2.6.0-r0/recipe-sysroot-native/usr/bin/x86_64-poky-linux/x86_64-poky-linux-gcc-ar

Kernel Summary:
293  kworker
56  ksoftirqd
56  migration
56  watchdog
56  cpuhp
3  jbd2
3  ext4-rsv-conver
2  kdmflush
2  bioset
....

Sakib



Signed-off-by: Sakib Sajal <sakib.sajal@...>
---
  scripts/collect-results              |   2 +-
  scripts/generate-testresult-index.py |   2 +-
  scripts/run-config                   |   1 +
  scripts/summarize_top_output.py      | 132 +++++++++++++++++++++++++++
  4 files changed, 135 insertions(+), 2 deletions(-)
  create mode 100755 scripts/summarize_top_output.py

diff --git a/scripts/collect-results b/scripts/collect-results
index 7474e36..7178380 100755
--- a/scripts/collect-results
+++ b/scripts/collect-results
@@ -19,7 +19,7 @@ if [ -e $WORKDIR/buildhistory ]; then
  fi
    HSFILE=$WORKDIR/tmp/buildstats/*/host_stats
-d=`date +%Y-%m-%d--%H-%M`
+d="intermittent_failure_host_data"
    mkdir -p $DEST/$target/$d
  diff --git a/scripts/generate-testresult-index.py b/scripts/generate-testresult-index.py
index 7fdc17c..d85d606 100755
--- a/scripts/generate-testresult-index.py
+++ b/scripts/generate-testresult-index.py
@@ -154,7 +154,7 @@ for build in sorted(os.listdir(path), key=keygen, reverse=True):
      hd = []
      counter = 0
      # do we really need the loop?
-    for p in glob.glob(buildpath + "/*/*/host_stats*top.txt"):
+    for p in glob.glob(buildpath + "/*/*/host_stats*top_summary.txt"):
          n_split = p.split(build)
          res = reldir[0:-1] + n_split[1]
          hd.append((res, str(counter)))
diff --git a/scripts/run-config b/scripts/run-config
index 8ed88cf..82de91f 100755
--- a/scripts/run-config
+++ b/scripts/run-config
@@ -327,6 +327,7 @@ elif args.phase == "finish" and args.stepname == "collect-results":
      if args.results_dir:
          hp.printheader("Running results collection")
          runcmd([scriptsdir + "/collect-results", args.builddir, args.results_dir, args.target])
+        runcmd([scriptsdir + "/summarize_top_output.py", args.results_dir, args.target])
      sys.exit(0)
    if jcfg:
diff --git a/scripts/summarize_top_output.py b/scripts/summarize_top_output.py
new file mode 100755
index 0000000..0606a34
--- /dev/null
+++ b/scripts/summarize_top_output.py
@@ -0,0 +1,132 @@
+#!/usr/bin/env python3
+
+import os, sys, glob
+
+# constants
+top_header = 7
+top_summary = 5
+max_cols = 11
+
+# string substitution to make things easier to read
+subs = {
+    "/home/pokybuild/yocto-worker/" : "~/",
+    "/build/build/tmp/work/core2-32-poky-linux/" : "/.../POKY_32/.../",
+    "/build/build/tmp/work/core2-64-poky-linux/" : "/.../POKY_64/.../",
+ "/recipe-sysroot-native/usr/bin/x86_64-poky-linux/../../libexec/x86_64-poky-linux/gcc/x86_64-poky-linux/" : "/...GCC.../"
+}
+
+def usage():
+    print("Usage: " + sys.argv[0] + " <dest> <target>")
+
+def list_top_outputs(logfile):
+    # top delimiter
+    top_start = "start: top output"
+    top_end = "end: top output"
+
+    # list of top outputs
+    top_outputs = []
+
+    # flag
+    collect = False
+    with open(logfile) as log:
+        top_output = []
+        for line in log:
+            lstrip = line.strip()
+            if collect:
+                if lstrip.startswith(top_end):
+                    collect = False
+                    top_outputs.append(top_output)
+                    top_output = []
+                else:
+                    top_output.append(lstrip)
+            if lstrip.startswith(top_start):
+                collect = True
+
+    return top_outputs
+
+def summarize_top(top_outs):
+    summaries = []
+    kernel_summaries = []
+    short_summaries = []
+    for top_out in top_outs:
+        summary = {}
+        kernel_summary = {}
+        short_summary = top_out[:top_summary]
+        for line in top_out[top_header:]:
+            cmd = line.split(maxsplit=max_cols)[-1]
+            # kernel processes
+            if cmd[0] == "[" and cmd[-1] == "]":
+                kproc = cmd[1:-1].split("/")[0]
+                if kproc not in kernel_summary:
+                    kernel_summary[kproc] = 1
+                else:
+                    kernel_summary[kproc] += 1
+                continue
+            cmd_split = cmd.split()
+            prog = cmd_split[0]
+            if prog not in summary:
+                summary[prog] = 1
+            else:
+                summary[prog] += 1
+        summary = dict(sorted(summary.items(), key=lambda item: item[1], reverse=True))
+        kernel_summary = dict(sorted(kernel_summary.items(), key=lambda item: item[1], reverse=True))
+
+        summaries.append(summary)
+        kernel_summaries.append(kernel_summary)
+        short_summaries.append(short_summary)
+
+    return (short_summaries, summaries, kernel_summaries)
+
+def summarize_path(path):
+    p = path
+    for k, v in subs.items():
+        p = p.replace(k, v)
+    return p
+
+def write_summary(short_summary, summary, kernel_summary, logfile):
+    dirname = os.path.dirname(logfile)
+    fname = os.path.basename(logfile)
+    report_name = fname.split(".")[0] + "_summary.txt"
+    outfile = os.path.join(dirname, report_name)
+    out = "NOTE: program names have been shortened for better readability.\nSubstitutions are as follows:\n"
+    for k, v in subs.items():
+        out += (v + " = " + k + "\n")
+    out += "\n"
+
+    out += "top was invoked " + str(len(short_summary)) + " times.\n\n"
+
+    for i in range(len(short_summary)):
+        for l in short_summary[i]:
+            out += (l + "\n")
+
+        out += ("\nSummary: " + "\n")
+        for k, v in summary[i].items():
+            if v > 1:
+                r = summarize_path(k)
+                out += (str(v) + "  " + r + "\n")
+
+        out += ("\nKernel Summary: " + "\n")
+        for k, v in kernel_summary[i].items():
+            if v > 1:
+                r = summarize_path(k)
+                out += (str(v) + "  " + r + "\n")
+        out += ("\n")
+
+    with open(outfile, "w") as of:
+        of.write(out)
+
+def main():
+    if len(sys.argv) != 3:
+        usage()
+        sys.exit()
+
+    dest = sys.argv[1]
+    target = sys.argv[2]
+    host_data_dir = "intermittent_failure_host_data"
+    directory = os.path.join(dest, target, host_data_dir)
+    for f in glob.glob(directory + "/*_top.txt"):
+        outputs = list_top_outputs(f)
+        short_summary, summary, kernel_summary = summarize_top(outputs)
+        write_summary(short_summary, summary, kernel_summary, f)
+
+main()




Re: [meta-intel]: support for NUC11

simon
 

On 2021-06-16 1:24 pm, Alexander Kanavin wrote:

On Wed, 16 Jun 2021 at 16:58, simon <simon@...> wrote:
- There's a warning about not finding the IRIS driver:
MESA_LOADER: failed to open iris: /usr/lib/dri/iris_dri.so: cannot open
shared object file: No such file or directory
failed to load driver: iris

But from what I was able to understand, I was expecting it would be
available with mesa 21.0.3 since I've seen some commit about fixing
issue with it in the change logs.

I've tried to force the driver to both i965 and i915 but I either got a
warning that this GEN cannot use it or still got the same fps.
Am I right to assume that it goes to a default driver that uses the CPU
instead of the GPU?

Hello Simon,
 
I honestly don't know how Intel folks have overlooked this, but mesa in yocto not only does not enable iris, the mesa recipe does not even have an option to enable it.
 
However, it should be pretty easy to add. Please take a look at poky/meta/recipes-graphics/mesa/mesa.inc, specificially any lines that mention GALLIUMDRIVERS and matching PACKAGECONFIG,
and you should be able to add iris there, and rebuild your image. Then you can submit your first patch to yocto via oe-core mailing list ;)
 
Alex


Hello Alex,

I was able to fix the iris driver issue with the following added to mesa.inc

GALLIUMDRIVERS_append = "${@bb.utils.contains('PACKAGECONFIG', 'iris', ',iris', '', d)}"

I will look into sending the patch

thanks a lot for your help

Simon



Re: [PATCH yocto-autobuilder-helper] summarize_top_output.py: add script, use it and publish summary

Richard Purdie
 

On Wed, 2021-06-16 at 04:43 -0400, sakib.sajal@... wrote:
summarize_top_output.py is used to summarize the top
output that is captured during autobuilder intermittent
failures.

Use the script to summarize the host top output and
publish the summary that is created instead of
the raw logfile.


[...]
 if jcfg:
diff --git a/scripts/summarize_top_output.py b/scripts/summarize_top_output.py
new file mode 100755
index 0000000..0606a34
--- /dev/null
+++ b/scripts/summarize_top_output.py
@@ -0,0 +1,132 @@
+#!/usr/bin/env python3
+
+import os, sys, glob
+
+# constants
+top_header = 7
+top_summary = 5
+max_cols = 11
+
+# string substitution to make things easier to read
+subs = {
+ "/home/pokybuild/yocto-worker/" : "~/",
+ "/build/build/tmp/work/core2-32-poky-linux/" : "/.../POKY_32/.../",
+ "/build/build/tmp/work/core2-64-poky-linux/" : "/.../POKY_64/.../",
+ "/recipe-sysroot-native/usr/bin/x86_64-poky-linux/../../libexec/x86_64-poky-linux/gcc/x86_64-poky-linux/" : "/...GCC.../"
+}
One quick question - the above assumes an x86 target machine using those two tunes. 
Should that be wildcarded?

Cheers,

Richard


Re: [meta-intel]: support for NUC11

Alexander Kanavin
 

On Wed, 16 Jun 2021 at 16:58, simon <simon@...> wrote:
- There's a warning about not finding the IRIS driver:
MESA_LOADER: failed to open iris: /usr/lib/dri/iris_dri.so: cannot open
shared object file: No such file or directory
failed to load driver: iris

But from what I was able to understand, I was expecting it would be
available with mesa 21.0.3 since I've seen some commit about fixing
issue with it in the change logs.

I've tried to force the driver to both i965 and i915 but I either got a
warning that this GEN cannot use it or still got the same fps.
Am I right to assume that it goes to a default driver that uses the CPU
instead of the GPU?

Hello Simon,

I honestly don't know how Intel folks have overlooked this, but mesa in yocto not only does not enable iris, the mesa recipe does not even have an option to enable it.

However, it should be pretty easy to add. Please take a look at poky/meta/recipes-graphics/mesa/mesa.inc, specificially any lines that mention GALLIUMDRIVERS and matching PACKAGECONFIG,
and you should be able to add iris there, and rebuild your image. Then you can submit your first patch to yocto via oe-core mailing list ;)

Alex


Re: [PATCH yocto-autobuilder-helper] summarize_top_output.py: add script, use it and publish summary

Randy MacLeod
 

On 2021-06-16 4:43 a.m., sakib.sajal@... wrote:
summarize_top_output.py is used to summarize the top
output that is captured during autobuilder intermittent
failures.
Use the script to summarize the host top output and
publish the summary that is created instead of
the raw logfile.

Looks good Sakib,

Can you show people what the typical output looks like?
Is the raw top output still published?

../Randy


Signed-off-by: Sakib Sajal <sakib.sajal@...>
---
scripts/collect-results | 2 +-
scripts/generate-testresult-index.py | 2 +-
scripts/run-config | 1 +
scripts/summarize_top_output.py | 132 +++++++++++++++++++++++++++
4 files changed, 135 insertions(+), 2 deletions(-)
create mode 100755 scripts/summarize_top_output.py
diff --git a/scripts/collect-results b/scripts/collect-results
index 7474e36..7178380 100755
--- a/scripts/collect-results
+++ b/scripts/collect-results
@@ -19,7 +19,7 @@ if [ -e $WORKDIR/buildhistory ]; then
fi
HSFILE=$WORKDIR/tmp/buildstats/*/host_stats
-d=`date +%Y-%m-%d--%H-%M`
+d="intermittent_failure_host_data"
mkdir -p $DEST/$target/$d
diff --git a/scripts/generate-testresult-index.py b/scripts/generate-testresult-index.py
index 7fdc17c..d85d606 100755
--- a/scripts/generate-testresult-index.py
+++ b/scripts/generate-testresult-index.py
@@ -154,7 +154,7 @@ for build in sorted(os.listdir(path), key=keygen, reverse=True):
hd = []
counter = 0
# do we really need the loop?
- for p in glob.glob(buildpath + "/*/*/host_stats*top.txt"):
+ for p in glob.glob(buildpath + "/*/*/host_stats*top_summary.txt"):
n_split = p.split(build)
res = reldir[0:-1] + n_split[1]
hd.append((res, str(counter)))
diff --git a/scripts/run-config b/scripts/run-config
index 8ed88cf..82de91f 100755
--- a/scripts/run-config
+++ b/scripts/run-config
@@ -327,6 +327,7 @@ elif args.phase == "finish" and args.stepname == "collect-results":
if args.results_dir:
hp.printheader("Running results collection")
runcmd([scriptsdir + "/collect-results", args.builddir, args.results_dir, args.target])
+ runcmd([scriptsdir + "/summarize_top_output.py", args.results_dir, args.target])
sys.exit(0)
if jcfg:
diff --git a/scripts/summarize_top_output.py b/scripts/summarize_top_output.py
new file mode 100755
index 0000000..0606a34
--- /dev/null
+++ b/scripts/summarize_top_output.py
@@ -0,0 +1,132 @@
+#!/usr/bin/env python3
+
+import os, sys, glob
+
+# constants
+top_header = 7
+top_summary = 5
+max_cols = 11
+
+# string substitution to make things easier to read
+subs = {
+ "/home/pokybuild/yocto-worker/" : "~/",
+ "/build/build/tmp/work/core2-32-poky-linux/" : "/.../POKY_32/.../",
+ "/build/build/tmp/work/core2-64-poky-linux/" : "/.../POKY_64/.../",
+ "/recipe-sysroot-native/usr/bin/x86_64-poky-linux/../../libexec/x86_64-poky-linux/gcc/x86_64-poky-linux/" : "/...GCC.../"
+}
+
+def usage():
+ print("Usage: " + sys.argv[0] + " <dest> <target>")
+
+def list_top_outputs(logfile):
+ # top delimiter
+ top_start = "start: top output"
+ top_end = "end: top output"
+
+ # list of top outputs
+ top_outputs = []
+
+ # flag
+ collect = False
+ with open(logfile) as log:
+ top_output = []
+ for line in log:
+ lstrip = line.strip()
+ if collect:
+ if lstrip.startswith(top_end):
+ collect = False
+ top_outputs.append(top_output)
+ top_output = []
+ else:
+ top_output.append(lstrip)
+ if lstrip.startswith(top_start):
+ collect = True
+
+ return top_outputs
+
+def summarize_top(top_outs):
+ summaries = []
+ kernel_summaries = []
+ short_summaries = []
+ for top_out in top_outs:
+ summary = {}
+ kernel_summary = {}
+ short_summary = top_out[:top_summary]
+ for line in top_out[top_header:]:
+ cmd = line.split(maxsplit=max_cols)[-1]
+ # kernel processes
+ if cmd[0] == "[" and cmd[-1] == "]":
+ kproc = cmd[1:-1].split("/")[0]
+ if kproc not in kernel_summary:
+ kernel_summary[kproc] = 1
+ else:
+ kernel_summary[kproc] += 1
+ continue
+ cmd_split = cmd.split()
+ prog = cmd_split[0]
+ if prog not in summary:
+ summary[prog] = 1
+ else:
+ summary[prog] += 1
+ summary = dict(sorted(summary.items(), key=lambda item: item[1], reverse=True))
+ kernel_summary = dict(sorted(kernel_summary.items(), key=lambda item: item[1], reverse=True))
+
+ summaries.append(summary)
+ kernel_summaries.append(kernel_summary)
+ short_summaries.append(short_summary)
+
+ return (short_summaries, summaries, kernel_summaries)
+
+def summarize_path(path):
+ p = path
+ for k, v in subs.items():
+ p = p.replace(k, v)
+ return p
+
+def write_summary(short_summary, summary, kernel_summary, logfile):
+ dirname = os.path.dirname(logfile)
+ fname = os.path.basename(logfile)
+ report_name = fname.split(".")[0] + "_summary.txt"
+ outfile = os.path.join(dirname, report_name)
+ out = "NOTE: program names have been shortened for better readability.\nSubstitutions are as follows:\n"
+ for k, v in subs.items():
+ out += (v + " = " + k + "\n")
+ out += "\n"
+
+ out += "top was invoked " + str(len(short_summary)) + " times.\n\n"
+
+ for i in range(len(short_summary)):
+ for l in short_summary[i]:
+ out += (l + "\n")
+
+ out += ("\nSummary: " + "\n")
+ for k, v in summary[i].items():
+ if v > 1:
+ r = summarize_path(k)
+ out += (str(v) + " " + r + "\n")
+
+ out += ("\nKernel Summary: " + "\n")
+ for k, v in kernel_summary[i].items():
+ if v > 1:
+ r = summarize_path(k)
+ out += (str(v) + " " + r + "\n")
+ out += ("\n")
+
+ with open(outfile, "w") as of:
+ of.write(out)
+
+def main():
+ if len(sys.argv) != 3:
+ usage()
+ sys.exit()
+
+ dest = sys.argv[1]
+ target = sys.argv[2]
+ host_data_dir = "intermittent_failure_host_data"
+ directory = os.path.join(dest, target, host_data_dir)
+ for f in glob.glob(directory + "/*_top.txt"):
+ outputs = list_top_outputs(f)
+ short_summary, summary, kernel_summary = summarize_top(outputs)
+ write_summary(short_summary, summary, kernel_summary, f)
+
+main()

--
# Randy MacLeod
# Wind River Linux


[meta-intel]: support for NUC11

simon
 

Hi all,

I'm having trouble figuring out how to fix my issues with the new intel NUC gen 11 (NUC11TNBi5).

At the moment I'm testing with the following meta-layer
poky (hardknott)
meta-openembedded (hardknott)
meta-intel (master)
meta-qt5 (hardknott)

I'm currently using EGLFS to display our Qt-QML GUI

My first issue is with the our GUI, it is launching as expected but we barely have 1 fps

- There's a warning about not finding the IRIS driver:
MESA_LOADER: failed to open iris: /usr/lib/dri/iris_dri.so: cannot open shared object file: No such file or directory
failed to load driver: iris

But from what I was able to understand, I was expecting it would be available with mesa 21.0.3 since I've seen some commit about fixing issue with it in the change logs.

I've tried to force the driver to both i965 and i915 but I either got a warning that this GEN cannot use it or still got the same fps.
Am I right to assume that it goes to a default driver that uses the CPU instead of the GPU?

My second issue is with the nvme slot, it does not detect the partition on the SSD, but it does when I use a NVME to USB adaptor

- There's a few warning about ACPI bios error AE_NOT_FOUND too, and again with my little knowledge I thought this would have been fixed with the most recent update in master with the new microcode since the Core gen 11 was added as new device.
But again, I'm not entirely sure that I'm doing the right thing to update the microcode. I've installed manually the microcode.cpio in /etc/firmware/intel-ucode/

The ouput of "dmesg | grep microcode":
microcode: sig=0x806c1, pf=0x80, revision=0x72
microcode: Microcode Update Driver: v2.2.

I don't know if I'm too early to be using the latest NUC11, any information about when it should be supported would be very appreciated.

I'm willing to help but my knowledge is pretty limited so I might need assistance.

thanks a lot

Simon


[meta-mingw][PATCH] Disable debuginfod

Joshua Watt
 

Disables debuginfod when using MingGW. This feature brings in
unbuildable dependencies and can't be used.

Signed-off-by: Joshua Watt <JPEWhacker@...>
---
conf/machine-sdk/include/mingw32-common.inc | 2 ++
recipes-devtools/gdb/gdb-cross-canadian_%.bbappend | 1 +
2 files changed, 3 insertions(+)

diff --git a/conf/machine-sdk/include/mingw32-common.inc b/conf/machine-sdk/include/mingw32-common.inc
index 0109e75..3997a26 100644
--- a/conf/machine-sdk/include/mingw32-common.inc
+++ b/conf/machine-sdk/include/mingw32-common.inc
@@ -15,6 +15,8 @@ USE_NLS_mingw32 = "no"
FILES_${PN}-staticdev_append_mingw32 = " ${libdir}/*.lib"
ALLOW_EMPTY_${PN}_mingw32 = "1"

+DISTRO_FEATURES_FILTER_NATIVESDK_remove_mingw32 = "debuginfod"
+
# Do what amounts to a NOOP
SDK_PACKAGING_FUNC = "do_compile"

diff --git a/recipes-devtools/gdb/gdb-cross-canadian_%.bbappend b/recipes-devtools/gdb/gdb-cross-canadian_%.bbappend
index c33a9ce..096fc63 100644
--- a/recipes-devtools/gdb/gdb-cross-canadian_%.bbappend
+++ b/recipes-devtools/gdb/gdb-cross-canadian_%.bbappend
@@ -4,3 +4,4 @@ RDEPENDS_${PN}_remove_sdkmingw32 = "nativesdk-python-core nativesdk-python-lang
EXTRA_OECONF_append_sdkmingw32 = " --without-curses --without-system-readline --with-python=no"
PACKAGECONFIG_remove_sdkmingw32 = "readline"
PACKAGECONFIG_remove_sdkmingw32 = "python"
+PACKAGECONFIG_remove_sdkmingw32 = "debuginfod"
--
2.31.1


devtool build failure first time but builds fine successfully second time. #yocto #devtool

raulpblooper@...
 

Hello All,
I am tyro in yocto ,

I am having a strange issue where I am trying to build the recipe  using    devtool build <recipe>  . It fails first time but builds second time.


 
It flags an error related to some linkage of shared library .
But when I checked the shared library , it is present at the same place,
 
First Error : 
 
    ERROR: <recipe-name>1-1.0-r0 do_compile: Execution of 'Path to build directory/tmp-glibc/work/cortexa7t2hf-neon-vfpv4-ostl-linux-gnueabi/<recipe-name>/1.0-r0/temp/run.do_compile.12944' failed with exit code 2
 
 
 
Second Error : 
 
    -L/<path to build directory>/tmp-glibc/work/cortexa7t2hf-neon-vfpv4-ostl-linux-gnueabi/<my recipe>1/1.0-r0/r1-1.0/<aplication name>/../<shared lib folder>/<sub folder name>  -Wl,-rpath,"\$ORIGIN/../<shared lib folder>/<sub folder name>" -l<shared lib name without lib> -pthread 
 
When i run the same command again **( devtool build recipe 
)** it builds Successfully the application referring this shared library.
 
Does anyone know this peculiar issue.?
Since it builds fine with no change i suspect no mistake in my Cmakefiles.
 
Please assist,
 
 


Re: [poky] [PATCH] local.conf.sample: disable prelink

Alexander Kanavin
 

On Wed, 16 Jun 2021 at 11:11, Richard Purdie <richard.purdie@...> wrote:
What makes me sad is that this is an embedded focused feature which 
the project should be caring about, i.e. memory footprint and speed.
I don't know why the numbers don't show it, Mark and I have theories
but it would take work to explore them and neither of us have that time,
nor are we experts on prelink able to maintain it with the time we have
available.

What I think happened is that glibc simply got better at linking, optimised the runtime bits to the fullest, and any benefits prelink may once have had aren't there anymore. Which is IMO fine - I am terrified of custom address arithmetic that's not really understood by anyone, and very easy to get wrong, with devastating consequences. I spent more or less the whole day stepping assembly in gdb to figure out what exactly is going wrong in that ppc bug.

The value proposition of Yocto is still very strong, and is not at all affected by dropping prelink, in my opinion.

Alex


Re: [poky] [PATCH] local.conf.sample: disable prelink

Richard Purdie
 

On Tue, 2021-06-15 at 17:21 +0200, Alexander Kanavin wrote:
So the memory usage is actually *better* without prelink. And any timing 
benefits are lost in statistical noise, in these tests at least.
The numbers certainly don't look convincing, thanks for running the tests.

So I do not think it is wrong to question the usefulness of this feature. 
I'd like to hear Mark's take on this, as prelink-cross is primarily his 
work, he's put a lot of effort into it, and I would want to know where 
the benefits are. Note that Red Hat abandoned prelink in 2013, and 
prelink-cross likewise hasn't seen any commits for two years.
Mark and I did briefly talk yesterday and whilst Mark can/will speak
up, the summary was "drop it".

What makes me sad is that this is an embedded focused feature which 
the project should be caring about, i.e. memory footprint and speed.
I don't know why the numbers don't show it, Mark and I have theories
but it would take work to explore them and neither of us have that time,
nor are we experts on prelink able to maintain it with the time we have
available.

In my view the project should be caring about things like this, instead 
it looks more like a race to look like 'any other desktop distro' and 
drop anything which looks different.

I've tried to say no. Nobody else would appear to care, I'm hearing 
silence. I can't fight this.

Cheers,

Richard


[PATCH yocto-autobuilder-helper] summarize_top_output.py: add script, use it and publish summary

sakib.sajal@...
 

summarize_top_output.py is used to summarize the top
output that is captured during autobuilder intermittent
failures.

Use the script to summarize the host top output and
publish the summary that is created instead of
the raw logfile.

Signed-off-by: Sakib Sajal <sakib.sajal@...>
---
scripts/collect-results | 2 +-
scripts/generate-testresult-index.py | 2 +-
scripts/run-config | 1 +
scripts/summarize_top_output.py | 132 +++++++++++++++++++++++++++
4 files changed, 135 insertions(+), 2 deletions(-)
create mode 100755 scripts/summarize_top_output.py

diff --git a/scripts/collect-results b/scripts/collect-results
index 7474e36..7178380 100755
--- a/scripts/collect-results
+++ b/scripts/collect-results
@@ -19,7 +19,7 @@ if [ -e $WORKDIR/buildhistory ]; then
fi

HSFILE=$WORKDIR/tmp/buildstats/*/host_stats
-d=`date +%Y-%m-%d--%H-%M`
+d="intermittent_failure_host_data"

mkdir -p $DEST/$target/$d

diff --git a/scripts/generate-testresult-index.py b/scripts/generate-testresult-index.py
index 7fdc17c..d85d606 100755
--- a/scripts/generate-testresult-index.py
+++ b/scripts/generate-testresult-index.py
@@ -154,7 +154,7 @@ for build in sorted(os.listdir(path), key=keygen, reverse=True):
hd = []
counter = 0
# do we really need the loop?
- for p in glob.glob(buildpath + "/*/*/host_stats*top.txt"):
+ for p in glob.glob(buildpath + "/*/*/host_stats*top_summary.txt"):
n_split = p.split(build)
res = reldir[0:-1] + n_split[1]
hd.append((res, str(counter)))
diff --git a/scripts/run-config b/scripts/run-config
index 8ed88cf..82de91f 100755
--- a/scripts/run-config
+++ b/scripts/run-config
@@ -327,6 +327,7 @@ elif args.phase == "finish" and args.stepname == "collect-results":
if args.results_dir:
hp.printheader("Running results collection")
runcmd([scriptsdir + "/collect-results", args.builddir, args.results_dir, args.target])
+ runcmd([scriptsdir + "/summarize_top_output.py", args.results_dir, args.target])
sys.exit(0)

if jcfg:
diff --git a/scripts/summarize_top_output.py b/scripts/summarize_top_output.py
new file mode 100755
index 0000000..0606a34
--- /dev/null
+++ b/scripts/summarize_top_output.py
@@ -0,0 +1,132 @@
+#!/usr/bin/env python3
+
+import os, sys, glob
+
+# constants
+top_header = 7
+top_summary = 5
+max_cols = 11
+
+# string substitution to make things easier to read
+subs = {
+ "/home/pokybuild/yocto-worker/" : "~/",
+ "/build/build/tmp/work/core2-32-poky-linux/" : "/.../POKY_32/.../",
+ "/build/build/tmp/work/core2-64-poky-linux/" : "/.../POKY_64/.../",
+ "/recipe-sysroot-native/usr/bin/x86_64-poky-linux/../../libexec/x86_64-poky-linux/gcc/x86_64-poky-linux/" : "/...GCC.../"
+}
+
+def usage():
+ print("Usage: " + sys.argv[0] + " <dest> <target>")
+
+def list_top_outputs(logfile):
+ # top delimiter
+ top_start = "start: top output"
+ top_end = "end: top output"
+
+ # list of top outputs
+ top_outputs = []
+
+ # flag
+ collect = False
+ with open(logfile) as log:
+ top_output = []
+ for line in log:
+ lstrip = line.strip()
+ if collect:
+ if lstrip.startswith(top_end):
+ collect = False
+ top_outputs.append(top_output)
+ top_output = []
+ else:
+ top_output.append(lstrip)
+ if lstrip.startswith(top_start):
+ collect = True
+
+ return top_outputs
+
+def summarize_top(top_outs):
+ summaries = []
+ kernel_summaries = []
+ short_summaries = []
+ for top_out in top_outs:
+ summary = {}
+ kernel_summary = {}
+ short_summary = top_out[:top_summary]
+ for line in top_out[top_header:]:
+ cmd = line.split(maxsplit=max_cols)[-1]
+ # kernel processes
+ if cmd[0] == "[" and cmd[-1] == "]":
+ kproc = cmd[1:-1].split("/")[0]
+ if kproc not in kernel_summary:
+ kernel_summary[kproc] = 1
+ else:
+ kernel_summary[kproc] += 1
+ continue
+ cmd_split = cmd.split()
+ prog = cmd_split[0]
+ if prog not in summary:
+ summary[prog] = 1
+ else:
+ summary[prog] += 1
+ summary = dict(sorted(summary.items(), key=lambda item: item[1], reverse=True))
+ kernel_summary = dict(sorted(kernel_summary.items(), key=lambda item: item[1], reverse=True))
+
+ summaries.append(summary)
+ kernel_summaries.append(kernel_summary)
+ short_summaries.append(short_summary)
+
+ return (short_summaries, summaries, kernel_summaries)
+
+def summarize_path(path):
+ p = path
+ for k, v in subs.items():
+ p = p.replace(k, v)
+ return p
+
+def write_summary(short_summary, summary, kernel_summary, logfile):
+ dirname = os.path.dirname(logfile)
+ fname = os.path.basename(logfile)
+ report_name = fname.split(".")[0] + "_summary.txt"
+ outfile = os.path.join(dirname, report_name)
+ out = "NOTE: program names have been shortened for better readability.\nSubstitutions are as follows:\n"
+ for k, v in subs.items():
+ out += (v + " = " + k + "\n")
+ out += "\n"
+
+ out += "top was invoked " + str(len(short_summary)) + " times.\n\n"
+
+ for i in range(len(short_summary)):
+ for l in short_summary[i]:
+ out += (l + "\n")
+
+ out += ("\nSummary: " + "\n")
+ for k, v in summary[i].items():
+ if v > 1:
+ r = summarize_path(k)
+ out += (str(v) + " " + r + "\n")
+
+ out += ("\nKernel Summary: " + "\n")
+ for k, v in kernel_summary[i].items():
+ if v > 1:
+ r = summarize_path(k)
+ out += (str(v) + " " + r + "\n")
+ out += ("\n")
+
+ with open(outfile, "w") as of:
+ of.write(out)
+
+def main():
+ if len(sys.argv) != 3:
+ usage()
+ sys.exit()
+
+ dest = sys.argv[1]
+ target = sys.argv[2]
+ host_data_dir = "intermittent_failure_host_data"
+ directory = os.path.join(dest, target, host_data_dir)
+ for f in glob.glob(directory + "/*_top.txt"):
+ outputs = list_top_outputs(f)
+ short_summary, summary, kernel_summary = summarize_top(outputs)
+ write_summary(short_summary, summary, kernel_summary, f)
+
+main()
--
2.25.1


[meta-security][PATCH] sssd: add fix-ldblibdir.patch back

kai
 

From: Kai Kang <kai.kang@...>

The patch fix-ldblibdir.patch has been dropped when update sssd to
2.5.0. But it fails to start sssd without this patch. So add it back.

Signed-off-by: Kai Kang <kai.kang@...>
---
.../sssd/files/fix-ldblibdir.patch | 25 +++++++++++++++++++
recipes-security/sssd/sssd_2.5.0.bb | 1 +
2 files changed, 26 insertions(+)
create mode 100644 recipes-security/sssd/files/fix-ldblibdir.patch

diff --git a/recipes-security/sssd/files/fix-ldblibdir.patch b/recipes-security/sssd/files/fix-ldblibdir.patch
new file mode 100644
index 0000000..e350baf
--- /dev/null
+++ b/recipes-security/sssd/files/fix-ldblibdir.patch
@@ -0,0 +1,25 @@
+When calculate value of ldblibdir, it checks whether the directory of
+$ldblibdir exists. If not, it assigns ldblibdir with ${libdir}/ldb. It is not
+suitable for cross compile. Fix it that only re-assign ldblibdir when its value
+is empty.
+
+Upstream-Status: Inappropriate [cross compile specific]
+
+Signed-off-by: Kai Kang <kai.kang@...>
+---
+ src/external/libldb.m4 | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/src/external/libldb.m4 b/src/external/libldb.m4
+index c400add..5e5f06d 100644
+--- a/src/external/libldb.m4
++++ b/src/external/libldb.m4
+@@ -19,7 +19,7 @@ if test x"$with_ldb_lib_dir" != x; then
+ ldblibdir=$with_ldb_lib_dir
+ else
+ ldblibdir="`$PKG_CONFIG --variable=modulesdir ldb`"
+- if ! test -d $ldblibdir; then
++ if test -z $ldblibdir; then
+ ldblibdir="${libdir}/ldb"
+ fi
+ fi
diff --git a/recipes-security/sssd/sssd_2.5.0.bb b/recipes-security/sssd/sssd_2.5.0.bb
index 4c92519..c5c870c 100644
--- a/recipes-security/sssd/sssd_2.5.0.bb
+++ b/recipes-security/sssd/sssd_2.5.0.bb
@@ -21,6 +21,7 @@ SRC_URI = "https://github.com/SSSD/sssd/releases/download/2.5.0/sssd-2.5.0.tar.g
file://no_gen.patch \
file://fix_gid.patch \
file://drop_ntpdate_chk.patch \
+ file://fix-ldblibdir.patch \
"
SRC_URI[sha256sum] = "afa62d7d8d23fca3aba093abe4ec0d14e7d9346c5b28ceb7c2c624bed98caa06"

--
2.17.1


[meta-security][PATCH] aircrack-ng: update to 1.6

Federico Pellegrin
 

Signed-off-by: Federico Pellegrin <fede@...>
---
.../{aircrack-ng_1.3.bb => aircrack-ng_1.6.bb} | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
rename recipes-security/aircrack-ng/{aircrack-ng_1.3.bb => aircrack-ng_1.6.bb} (82%)

diff --git a/recipes-security/aircrack-ng/aircrack-ng_1.3.bb b/recipes-security/aircrack-ng/aircrack-ng_1.6.bb
similarity index 82%
rename from recipes-security/aircrack-ng/aircrack-ng_1.3.bb
rename to recipes-security/aircrack-ng/aircrack-ng_1.6.bb
index d739227..8d3b531 100644
--- a/recipes-security/aircrack-ng/aircrack-ng_1.3.bb
+++ b/recipes-security/aircrack-ng/aircrack-ng_1.6.bb
@@ -9,8 +9,8 @@ DEPENDS = "libnl openssl sqlite3 libpcre libpcap"

SRC_URI = "http://download.aircrack-ng.org/${BP}.tar.gz"

-SRC_URI[md5sum] = "c7c5b076dee0c25ee580b0f56f455623"
-SRC_URI[sha256sum] = "8ae08a7c28741f6ace2769267112053366550e7f746477081188ad38410383ca"
+SRC_URI[md5sum] = "22ddc85549b51ed0da0931d01ef215e5"
+SRC_URI[sha256sum] = "4f0bfd486efc6ea7229f7fbc54340ff8b2094a0d73e9f617e0a39f878999a247"

inherit autotools-brokensep pkgconfig

@@ -29,6 +29,8 @@ do_install () {
make DESTDIR=${D} ${OEMAKE_EXTRA} ext_scripts=true install
}

-FILES_${PN} += "/usr/local/"
+FILES_${PN} += "${libdir}/*.so"
+FILES_SOLIBSDEV = ""
+INSANE_SKIP_${PN} += "dev-so"

RDEPENDS_${PN} = "libpcap"
--
2.31.1


Re: [qa-build-notification] QA notification for completed autobuilder build (yocto-3.4_M1.rc1)

Sangeeta Jain
 

Hi all,

Intel and WR YP QA is planning for QA execution for YP build yocto-3.4_M1.rc1. We are planning to execute following tests for this cycle:

OEQA-manual tests for following module:
1. OE-Core
2. BSP-hw

Runtime auto test for following platforms:
1. MinnowTurbot 32-bit
2. Coffee Lake
3. NUC 7
4. NUC 6
5. Edgerouter
6. Beaglebone

ETA for completion is next Thursday, June 17.

Thanks,
Sangeeta

-----Original Message-----
From: qa-build-notification@... <qa-build-
notification@...> On Behalf Of Pokybuild User
Sent: Saturday, 12 June, 2021 7:49 PM
To: yocto@...
Cc: qa-build-notification@...
Subject: [qa-build-notification] QA notification for completed autobuilder build
(yocto-3.4_M1.rc1)


A build flagged for QA (yocto-3.4_M1.rc1) was completed on the autobuilder
and is available at:


https://autobuilder.yocto.io/pub/releases/yocto-3.4_M1.rc1


Build hash information:

bitbake: 398a1686176c695d103591089a36e25173f9fd6e
meta-arm: 6c3d62c776fc45b4bae47d178899e84b17248b31
meta-gplv2: 1ee1a73070d91e0c727f9d0db11943a61765c8d9
meta-intel: 0937728bcd98dd13d2c6829e1cd604ea2e53e5cd
meta-mingw: bfd22a248c0db4c57d5a72d675979d8341a7e9c1
oecore: 3b2903ccc791d5dedd84c75227f38ae4c8d29251
poky: 59d93693bf24e02ca0f05fe06d96a46f4f0f1bf8



This is an automated message from the Yocto Project Autobuilder
Git: git://git.yoctoproject.org/yocto-autobuilder2
Email: richard.purdie@...







Re: [poky] [PATCH] local.conf.sample: disable prelink

Khem Raj
 

On 6/15/21 8:21 AM, Alexander Kanavin wrote:
On Tue, 15 Jun 2021 at 10:55, Alexander Kanavin <alex.kanavin@... <mailto:alex.kanavin@...>> wrote:
On Tue, 15 Jun 2021 at 10:48, Richard Purdie
<richard.purdie@...
<mailto:richard.purdie@...>> wrote:
I appreciate the desire simply to delete/disable anything that
causes issues
but in this case I draw the line and my answer is no. It works
fine in the
vast majority of usage.
But where are the benchmarks that show it's actually beneficial? And
commitment from someone to maintain it and address open issues
(there are more than just this one)?
I went ahead and ran some quick benchmarks myself. Specifically:
1. Running 'free' after things have settled down at boot:
a) without prelink
core-image-sato-sdk
               total        used        free      shared  buff/cache available
Mem:          489352      152104      236284         472      100964  323824
core-image-ptest-fast
              total        used        free      shared  buff/cache available
Mem:        1004680       43456      927688         256       33536  941156
b) with prelink
core-image-sato-sdk
               total        used        free      shared  buff/cache available
Mem:          489352      153048      235544         468      100760  322900
core-image-ptest-fast
              total        used        free      shared  buff/cache available
Mem:        1004680       44444      928128         256       32108  940168
2. Running -c testimage
a) without prelink
core-image-sato-sdk () - Ran 66 tests in 96.693s
core-image-sato-sdk () - Ran 66 tests in 96.469s
core-image-sato-sdk () - Ran 66 tests in 94.994s
core-image-ptest-fast () - Ran 66 tests in 583.767s
core-image-ptest-fast () - Ran 66 tests in 576.564s
core-image-ptest-fast () - Ran 66 tests in 576.797s
b) with prelink
core-image-sato-sdk () - Ran 66 tests in 96.390s
core-image-sato-sdk () - Ran 66 tests in 96.615s
core-image-sato-sdk () - Ran 66 tests in 95.596s
core-image-ptest-fast () - Ran 66 tests in 576.248s
core-image-ptest-fast () - Ran 66 tests in 574.618s
core-image-ptest-fast () - Ran 66 tests in 576.760s
I think the advantage is not on high end CPUs but more on less powerful ones, so perhaps trying it on something like rpi0 or lower would be good


So the memory usage is actually *better* without prelink. And any timing benefits are lost in statistical noise, in these tests at least.
So I do not think it is wrong to question the usefulness of this feature. I'd like to hear Mark's take on this, as prelink-cross is primarily his work, he's put a lot of effort into it, and I would want to know where the benefits are. Note that Red Hat abandoned prelink in 2013, and prelink-cross likewise hasn't seen any commits for two years.


Alex


Re: [poky] [PATCH] local.conf.sample: disable prelink

Alexander Kanavin
 



On Tue, 15 Jun 2021 at 10:55, Alexander Kanavin <alex.kanavin@...> wrote:
On Tue, 15 Jun 2021 at 10:48, Richard Purdie <richard.purdie@...> wrote:
I appreciate the desire simply to delete/disable anything that causes issues
but in this case I draw the line and my answer is no. It works fine in the
vast majority of usage.

But where are the benchmarks that show it's actually beneficial? And commitment from someone to maintain it and address open issues (there are more than just this one)?

I went ahead and ran some quick benchmarks myself. Specifically:

1. Running 'free' after things have settled down at boot:

a) without prelink
core-image-sato-sdk
               total        used        free      shared  buff/cache   available
Mem:          489352      152104      236284         472      100964      323824
core-image-ptest-fast
              total        used        free      shared  buff/cache   available
Mem:        1004680       43456      927688         256       33536      941156

b) with prelink
core-image-sato-sdk
               total        used        free      shared  buff/cache   available
Mem:          489352      153048      235544         468      100760      322900
core-image-ptest-fast
              total        used        free      shared  buff/cache   available
Mem:        1004680       44444      928128         256       32108      940168

2. Running -c testimage

a) without prelink
core-image-sato-sdk () - Ran 66 tests in 96.693s
core-image-sato-sdk () - Ran 66 tests in 96.469s
core-image-sato-sdk () - Ran 66 tests in 94.994s
core-image-ptest-fast () - Ran 66 tests in 583.767s
core-image-ptest-fast () - Ran 66 tests in 576.564s
core-image-ptest-fast () - Ran 66 tests in 576.797s

b) with prelink
core-image-sato-sdk () - Ran 66 tests in 96.390s
core-image-sato-sdk () - Ran 66 tests in 96.615s
core-image-sato-sdk () - Ran 66 tests in 95.596s
core-image-ptest-fast () - Ran 66 tests in 576.248s
core-image-ptest-fast () - Ran 66 tests in 574.618s
core-image-ptest-fast () - Ran 66 tests in 576.760s

So the memory usage is actually *better* without prelink. And any timing benefits are lost in statistical noise, in these tests at least.

So I do not think it is wrong to question the usefulness of this feature. I'd like to hear Mark's take on this, as prelink-cross is primarily his work, he's put a lot of effort into it, and I would want to know where the benefits are. Note that Red Hat abandoned prelink in 2013, and prelink-cross likewise hasn't seen any commits for two years.

Alex

3941 - 3960 of 57804