Date   

[PATCH] bsps/5.10: update to v5.10.43

Bruce Ashfield
 

From: Bruce Ashfield <bruce.ashfield@...>

Updating linux-yocto/5.10 to the latest korg -stable release, and to
match the qemu BSPs in oe-core

Signed-off-by: Bruce Ashfield <bruce.ashfield@...>
---
.../linux/linux-yocto_5.10.bbappend | 16 ++++++++--------
1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/meta-yocto-bsp/recipes-kernel/linux/linux-yocto_5.10.bbappend b/meta-yocto-bsp/recipes-kernel/linux/linux-yocto_5.10.bbappend
index bc2b3bf576..f8362b6635 100644
--- a/meta-yocto-bsp/recipes-kernel/linux/linux-yocto_5.10.bbappend
+++ b/meta-yocto-bsp/recipes-kernel/linux/linux-yocto_5.10.bbappend
@@ -7,17 +7,17 @@ KMACHINE_genericx86 ?= "common-pc"
KMACHINE_genericx86-64 ?= "common-pc-64"
KMACHINE_beaglebone-yocto ?= "beaglebone"

-SRCREV_machine_genericx86 ?= "8c516ced69f41563404ada0bea315a55bcf1df6f"
-SRCREV_machine_genericx86-64 ?= "8c516ced69f41563404ada0bea315a55bcf1df6f"
-SRCREV_machine_edgerouter ?= "965ab3ab746ae8a1158617b6302d9c218ffbbb66"
-SRCREV_machine_beaglebone-yocto ?= "8c516ced69f41563404ada0bea315a55bcf1df6f"
+SRCREV_machine_genericx86 ?= "ab49d2db98bdee2c8c6e17fb59ded9e5292b0f41"
+SRCREV_machine_genericx86-64 ?= "ab49d2db98bdee2c8c6e17fb59ded9e5292b0f41"
+SRCREV_machine_edgerouter ?= "274d63799465eebfd201b3e8251f16d29e93a978"
+SRCREV_machine_beaglebone-yocto ?= "ab49d2db98bdee2c8c6e17fb59ded9e5292b0f41"

COMPATIBLE_MACHINE_genericx86 = "genericx86"
COMPATIBLE_MACHINE_genericx86-64 = "genericx86-64"
COMPATIBLE_MACHINE_edgerouter = "edgerouter"
COMPATIBLE_MACHINE_beaglebone-yocto = "beaglebone-yocto"

-LINUX_VERSION_genericx86 = "5.10.21"
-LINUX_VERSION_genericx86-64 = "5.10.21"
-LINUX_VERSION_edgerouter = "5.10.21"
-LINUX_VERSION_beaglebone-yocto = "5.10.21"
+LINUX_VERSION_genericx86 = "5.10.43"
+LINUX_VERSION_genericx86-64 = "5.10.43"
+LINUX_VERSION_edgerouter = "5.10.43"
+LINUX_VERSION_beaglebone-yocto = "5.10.43"
--
2.19.1


Re: [PATCH yocto-autobuilder-helper] summarize_top_output.py: add script, use it and publish summary

sakib.sajal@...
 

On 2021-06-16 11:41 a.m., Randy MacLeod wrote:
On 2021-06-16 4:43 a.m., sakib.sajal@... wrote:
summarize_top_output.py is used to summarize the top
output that is captured during autobuilder intermittent
failures.

Use the script to summarize the host top output and
publish the summary that is created instead of
the raw logfile.

Looks good Sakib,

Can you show people what the typical output looks like?
Is the raw top output still published?

../Randy
The script goes over the raw logfile (for example, foo.txt) which consists of multiple top outputs and summarizes each top output and writes them to foo_summary.txt

Some improvements that can be made:

1) create a separate file for each top summary, ie, if foo.txt has n top outputs, create foo_summary_[1...n].txt

2) since python3 is used to run many different kinds of jobs, it is difficult to generalize python3 threads. the script can be modified to log all the different jobs run under python3, right now it only counts the processes that are running python3.

Any feedback is welcome!

Sample output:

NOTE: program names have been shortened for better readability.
Substitutions are as follows:
~/ = /home/pokybuild/yocto-worker/
/.../POKY_32/.../ = /build/build/tmp/work/core2-32-poky-linux/
/.../POKY_64/.../ = /build/build/tmp/work/core2-64-poky-linux/
/...GCC.../ = /recipe-sysroot-native/usr/bin/x86_64-poky-linux/../../libexec/x86_64-poky-linux/gcc/x86_64-poky-linux/

top was invoked 12 times.

top - 16:51:30 up 4 days, 19:26,  1 user,  load average: 93.84, 57.26, 48.58
Tasks: 1084 total,  80 running, 664 sleeping,   0 stopped,   0 zombie
%Cpu(s):  2.1 us,  2.0 sy,  9.8 ni, 83.1 id,  2.9 wa,  0.0 hi, 0.0 si,  0.0 st
KiB Mem : 13192559+total,  5571436 free, 14307780 used, 11204637+buff/cache
KiB Swap:  8388604 total,  8239384 free,   149220 used. 11455193+avail Mem

Summary:
85  /bin/sh
41  python3
39  /bin/bash
30  x86_64-poky-linux-g++
29  make
16 ~/meta-oe/.../POKY_64/.../cpprest/2.10.18-r0/...GCC.../11.1.0/cc1plus
16 ~/meta-oe/.../POKY_64/.../libvpx/1.8.2-r0/...GCC.../11.1.0/cc1plus
16  ~/meta-oe/.../POKY_64/.../libvpx/1.8.2-r0/...GCC.../11.1.0/as
16 ~/meta-oe/.../POKY_64/.../cpprest/2.10.18-r0/recipe-sysroot-native/usr/bin/x86_64-poky-linux/x86_64-poky-linux-g++
16 ~/meta-oe/.../POKY_64/.../cpprest/2.10.18-r0/...GCC.../11.1.0/as
14 ~/meta-oe/.../POKY_64/.../rtorrent/0.9.8-r0/...GCC.../11.1.0/cc1plus
14 ~/meta-oe/.../POKY_64/.../rtorrent/0.9.8-r0/...GCC.../11.1.0/as
13 ~/meta-oe/.../POKY_64/.../vulkan-cts/1.2.6.0-r0/...GCC.../11.1.0/cc1plus
13 ~/meta-oe/.../POKY_64/.../vulkan-cts/1.2.6.0-r0/...GCC.../11.1.0/as
13 ~/meta-oe/.../POKY_64/.../vulkan-cts/1.2.6.0-r0/recipe-sysroot-native/usr/bin/x86_64-poky-linux/x86_64-poky-linux-g++
11  x86_64-poky-linux-gcc
10 ~/meta-oe/.../POKY_64/.../dovecot/2.3.14-r0/...GCC.../11.1.0/cc1
10 ~/meta-oe/.../POKY_64/.../fluentbit/1.3.5-r0/recipe-sysroot-native/usr/bin/x86_64-poky-linux/x86_64-poky-linux-gcc
10 ~/meta-oe/.../POKY_64/.../dovecot/2.3.14-r0/...GCC.../11.1.0/as
7 ~/meta-oe/.../POKY_64/.../fluentbit/1.3.5-r0/...GCC.../11.1.0/cc1
7 ~/meta-oe/.../POKY_64/.../fluentbit/1.3.5-r0/...GCC.../11.1.0/as
4  /usr/bin/python3
3  cmake
2 ~/meta-oe/.../POKY_64/.../vulkan-cts/1.2.6.0-r0/...GCC.../11.1.0/ar
2  sh
2  /lib/systemd/systemd
2  (sd-pam)
2  ninja
2  bitbake-server
2 ~/meta-oe/.../POKY_64/.../vulkan-cts/1.2.6.0-r0/recipe-sysroot-native/usr/bin/x86_64-poky-linux/x86_64-poky-linux-gcc-ar

Kernel Summary:
293  kworker
56  ksoftirqd
56  migration
56  watchdog
56  cpuhp
3  jbd2
3  ext4-rsv-conver
2  kdmflush
2  bioset
....

Sakib



Signed-off-by: Sakib Sajal <sakib.sajal@...>
---
  scripts/collect-results              |   2 +-
  scripts/generate-testresult-index.py |   2 +-
  scripts/run-config                   |   1 +
  scripts/summarize_top_output.py      | 132 +++++++++++++++++++++++++++
  4 files changed, 135 insertions(+), 2 deletions(-)
  create mode 100755 scripts/summarize_top_output.py

diff --git a/scripts/collect-results b/scripts/collect-results
index 7474e36..7178380 100755
--- a/scripts/collect-results
+++ b/scripts/collect-results
@@ -19,7 +19,7 @@ if [ -e $WORKDIR/buildhistory ]; then
  fi
    HSFILE=$WORKDIR/tmp/buildstats/*/host_stats
-d=`date +%Y-%m-%d--%H-%M`
+d="intermittent_failure_host_data"
    mkdir -p $DEST/$target/$d
  diff --git a/scripts/generate-testresult-index.py b/scripts/generate-testresult-index.py
index 7fdc17c..d85d606 100755
--- a/scripts/generate-testresult-index.py
+++ b/scripts/generate-testresult-index.py
@@ -154,7 +154,7 @@ for build in sorted(os.listdir(path), key=keygen, reverse=True):
      hd = []
      counter = 0
      # do we really need the loop?
-    for p in glob.glob(buildpath + "/*/*/host_stats*top.txt"):
+    for p in glob.glob(buildpath + "/*/*/host_stats*top_summary.txt"):
          n_split = p.split(build)
          res = reldir[0:-1] + n_split[1]
          hd.append((res, str(counter)))
diff --git a/scripts/run-config b/scripts/run-config
index 8ed88cf..82de91f 100755
--- a/scripts/run-config
+++ b/scripts/run-config
@@ -327,6 +327,7 @@ elif args.phase == "finish" and args.stepname == "collect-results":
      if args.results_dir:
          hp.printheader("Running results collection")
          runcmd([scriptsdir + "/collect-results", args.builddir, args.results_dir, args.target])
+        runcmd([scriptsdir + "/summarize_top_output.py", args.results_dir, args.target])
      sys.exit(0)
    if jcfg:
diff --git a/scripts/summarize_top_output.py b/scripts/summarize_top_output.py
new file mode 100755
index 0000000..0606a34
--- /dev/null
+++ b/scripts/summarize_top_output.py
@@ -0,0 +1,132 @@
+#!/usr/bin/env python3
+
+import os, sys, glob
+
+# constants
+top_header = 7
+top_summary = 5
+max_cols = 11
+
+# string substitution to make things easier to read
+subs = {
+    "/home/pokybuild/yocto-worker/" : "~/",
+    "/build/build/tmp/work/core2-32-poky-linux/" : "/.../POKY_32/.../",
+    "/build/build/tmp/work/core2-64-poky-linux/" : "/.../POKY_64/.../",
+ "/recipe-sysroot-native/usr/bin/x86_64-poky-linux/../../libexec/x86_64-poky-linux/gcc/x86_64-poky-linux/" : "/...GCC.../"
+}
+
+def usage():
+    print("Usage: " + sys.argv[0] + " <dest> <target>")
+
+def list_top_outputs(logfile):
+    # top delimiter
+    top_start = "start: top output"
+    top_end = "end: top output"
+
+    # list of top outputs
+    top_outputs = []
+
+    # flag
+    collect = False
+    with open(logfile) as log:
+        top_output = []
+        for line in log:
+            lstrip = line.strip()
+            if collect:
+                if lstrip.startswith(top_end):
+                    collect = False
+                    top_outputs.append(top_output)
+                    top_output = []
+                else:
+                    top_output.append(lstrip)
+            if lstrip.startswith(top_start):
+                collect = True
+
+    return top_outputs
+
+def summarize_top(top_outs):
+    summaries = []
+    kernel_summaries = []
+    short_summaries = []
+    for top_out in top_outs:
+        summary = {}
+        kernel_summary = {}
+        short_summary = top_out[:top_summary]
+        for line in top_out[top_header:]:
+            cmd = line.split(maxsplit=max_cols)[-1]
+            # kernel processes
+            if cmd[0] == "[" and cmd[-1] == "]":
+                kproc = cmd[1:-1].split("/")[0]
+                if kproc not in kernel_summary:
+                    kernel_summary[kproc] = 1
+                else:
+                    kernel_summary[kproc] += 1
+                continue
+            cmd_split = cmd.split()
+            prog = cmd_split[0]
+            if prog not in summary:
+                summary[prog] = 1
+            else:
+                summary[prog] += 1
+        summary = dict(sorted(summary.items(), key=lambda item: item[1], reverse=True))
+        kernel_summary = dict(sorted(kernel_summary.items(), key=lambda item: item[1], reverse=True))
+
+        summaries.append(summary)
+        kernel_summaries.append(kernel_summary)
+        short_summaries.append(short_summary)
+
+    return (short_summaries, summaries, kernel_summaries)
+
+def summarize_path(path):
+    p = path
+    for k, v in subs.items():
+        p = p.replace(k, v)
+    return p
+
+def write_summary(short_summary, summary, kernel_summary, logfile):
+    dirname = os.path.dirname(logfile)
+    fname = os.path.basename(logfile)
+    report_name = fname.split(".")[0] + "_summary.txt"
+    outfile = os.path.join(dirname, report_name)
+    out = "NOTE: program names have been shortened for better readability.\nSubstitutions are as follows:\n"
+    for k, v in subs.items():
+        out += (v + " = " + k + "\n")
+    out += "\n"
+
+    out += "top was invoked " + str(len(short_summary)) + " times.\n\n"
+
+    for i in range(len(short_summary)):
+        for l in short_summary[i]:
+            out += (l + "\n")
+
+        out += ("\nSummary: " + "\n")
+        for k, v in summary[i].items():
+            if v > 1:
+                r = summarize_path(k)
+                out += (str(v) + "  " + r + "\n")
+
+        out += ("\nKernel Summary: " + "\n")
+        for k, v in kernel_summary[i].items():
+            if v > 1:
+                r = summarize_path(k)
+                out += (str(v) + "  " + r + "\n")
+        out += ("\n")
+
+    with open(outfile, "w") as of:
+        of.write(out)
+
+def main():
+    if len(sys.argv) != 3:
+        usage()
+        sys.exit()
+
+    dest = sys.argv[1]
+    target = sys.argv[2]
+    host_data_dir = "intermittent_failure_host_data"
+    directory = os.path.join(dest, target, host_data_dir)
+    for f in glob.glob(directory + "/*_top.txt"):
+        outputs = list_top_outputs(f)
+        short_summary, summary, kernel_summary = summarize_top(outputs)
+        write_summary(short_summary, summary, kernel_summary, f)
+
+main()




Re: [meta-intel]: support for NUC11

simon
 

On 2021-06-16 1:24 pm, Alexander Kanavin wrote:

On Wed, 16 Jun 2021 at 16:58, simon <simon@...> wrote:
- There's a warning about not finding the IRIS driver:
MESA_LOADER: failed to open iris: /usr/lib/dri/iris_dri.so: cannot open
shared object file: No such file or directory
failed to load driver: iris

But from what I was able to understand, I was expecting it would be
available with mesa 21.0.3 since I've seen some commit about fixing
issue with it in the change logs.

I've tried to force the driver to both i965 and i915 but I either got a
warning that this GEN cannot use it or still got the same fps.
Am I right to assume that it goes to a default driver that uses the CPU
instead of the GPU?

Hello Simon,
 
I honestly don't know how Intel folks have overlooked this, but mesa in yocto not only does not enable iris, the mesa recipe does not even have an option to enable it.
 
However, it should be pretty easy to add. Please take a look at poky/meta/recipes-graphics/mesa/mesa.inc, specificially any lines that mention GALLIUMDRIVERS and matching PACKAGECONFIG,
and you should be able to add iris there, and rebuild your image. Then you can submit your first patch to yocto via oe-core mailing list ;)
 
Alex


Hello Alex,

I was able to fix the iris driver issue with the following added to mesa.inc

GALLIUMDRIVERS_append = "${@bb.utils.contains('PACKAGECONFIG', 'iris', ',iris', '', d)}"

I will look into sending the patch

thanks a lot for your help

Simon



Re: [PATCH yocto-autobuilder-helper] summarize_top_output.py: add script, use it and publish summary

Richard Purdie
 

On Wed, 2021-06-16 at 04:43 -0400, sakib.sajal@... wrote:
summarize_top_output.py is used to summarize the top
output that is captured during autobuilder intermittent
failures.

Use the script to summarize the host top output and
publish the summary that is created instead of
the raw logfile.


[...]
 if jcfg:
diff --git a/scripts/summarize_top_output.py b/scripts/summarize_top_output.py
new file mode 100755
index 0000000..0606a34
--- /dev/null
+++ b/scripts/summarize_top_output.py
@@ -0,0 +1,132 @@
+#!/usr/bin/env python3
+
+import os, sys, glob
+
+# constants
+top_header = 7
+top_summary = 5
+max_cols = 11
+
+# string substitution to make things easier to read
+subs = {
+ "/home/pokybuild/yocto-worker/" : "~/",
+ "/build/build/tmp/work/core2-32-poky-linux/" : "/.../POKY_32/.../",
+ "/build/build/tmp/work/core2-64-poky-linux/" : "/.../POKY_64/.../",
+ "/recipe-sysroot-native/usr/bin/x86_64-poky-linux/../../libexec/x86_64-poky-linux/gcc/x86_64-poky-linux/" : "/...GCC.../"
+}
One quick question - the above assumes an x86 target machine using those two tunes. 
Should that be wildcarded?

Cheers,

Richard


Re: [meta-intel]: support for NUC11

Alexander Kanavin
 

On Wed, 16 Jun 2021 at 16:58, simon <simon@...> wrote:
- There's a warning about not finding the IRIS driver:
MESA_LOADER: failed to open iris: /usr/lib/dri/iris_dri.so: cannot open
shared object file: No such file or directory
failed to load driver: iris

But from what I was able to understand, I was expecting it would be
available with mesa 21.0.3 since I've seen some commit about fixing
issue with it in the change logs.

I've tried to force the driver to both i965 and i915 but I either got a
warning that this GEN cannot use it or still got the same fps.
Am I right to assume that it goes to a default driver that uses the CPU
instead of the GPU?

Hello Simon,

I honestly don't know how Intel folks have overlooked this, but mesa in yocto not only does not enable iris, the mesa recipe does not even have an option to enable it.

However, it should be pretty easy to add. Please take a look at poky/meta/recipes-graphics/mesa/mesa.inc, specificially any lines that mention GALLIUMDRIVERS and matching PACKAGECONFIG,
and you should be able to add iris there, and rebuild your image. Then you can submit your first patch to yocto via oe-core mailing list ;)

Alex


Re: [PATCH yocto-autobuilder-helper] summarize_top_output.py: add script, use it and publish summary

Randy MacLeod
 

On 2021-06-16 4:43 a.m., sakib.sajal@... wrote:
summarize_top_output.py is used to summarize the top
output that is captured during autobuilder intermittent
failures.
Use the script to summarize the host top output and
publish the summary that is created instead of
the raw logfile.

Looks good Sakib,

Can you show people what the typical output looks like?
Is the raw top output still published?

../Randy


Signed-off-by: Sakib Sajal <sakib.sajal@...>
---
scripts/collect-results | 2 +-
scripts/generate-testresult-index.py | 2 +-
scripts/run-config | 1 +
scripts/summarize_top_output.py | 132 +++++++++++++++++++++++++++
4 files changed, 135 insertions(+), 2 deletions(-)
create mode 100755 scripts/summarize_top_output.py
diff --git a/scripts/collect-results b/scripts/collect-results
index 7474e36..7178380 100755
--- a/scripts/collect-results
+++ b/scripts/collect-results
@@ -19,7 +19,7 @@ if [ -e $WORKDIR/buildhistory ]; then
fi
HSFILE=$WORKDIR/tmp/buildstats/*/host_stats
-d=`date +%Y-%m-%d--%H-%M`
+d="intermittent_failure_host_data"
mkdir -p $DEST/$target/$d
diff --git a/scripts/generate-testresult-index.py b/scripts/generate-testresult-index.py
index 7fdc17c..d85d606 100755
--- a/scripts/generate-testresult-index.py
+++ b/scripts/generate-testresult-index.py
@@ -154,7 +154,7 @@ for build in sorted(os.listdir(path), key=keygen, reverse=True):
hd = []
counter = 0
# do we really need the loop?
- for p in glob.glob(buildpath + "/*/*/host_stats*top.txt"):
+ for p in glob.glob(buildpath + "/*/*/host_stats*top_summary.txt"):
n_split = p.split(build)
res = reldir[0:-1] + n_split[1]
hd.append((res, str(counter)))
diff --git a/scripts/run-config b/scripts/run-config
index 8ed88cf..82de91f 100755
--- a/scripts/run-config
+++ b/scripts/run-config
@@ -327,6 +327,7 @@ elif args.phase == "finish" and args.stepname == "collect-results":
if args.results_dir:
hp.printheader("Running results collection")
runcmd([scriptsdir + "/collect-results", args.builddir, args.results_dir, args.target])
+ runcmd([scriptsdir + "/summarize_top_output.py", args.results_dir, args.target])
sys.exit(0)
if jcfg:
diff --git a/scripts/summarize_top_output.py b/scripts/summarize_top_output.py
new file mode 100755
index 0000000..0606a34
--- /dev/null
+++ b/scripts/summarize_top_output.py
@@ -0,0 +1,132 @@
+#!/usr/bin/env python3
+
+import os, sys, glob
+
+# constants
+top_header = 7
+top_summary = 5
+max_cols = 11
+
+# string substitution to make things easier to read
+subs = {
+ "/home/pokybuild/yocto-worker/" : "~/",
+ "/build/build/tmp/work/core2-32-poky-linux/" : "/.../POKY_32/.../",
+ "/build/build/tmp/work/core2-64-poky-linux/" : "/.../POKY_64/.../",
+ "/recipe-sysroot-native/usr/bin/x86_64-poky-linux/../../libexec/x86_64-poky-linux/gcc/x86_64-poky-linux/" : "/...GCC.../"
+}
+
+def usage():
+ print("Usage: " + sys.argv[0] + " <dest> <target>")
+
+def list_top_outputs(logfile):
+ # top delimiter
+ top_start = "start: top output"
+ top_end = "end: top output"
+
+ # list of top outputs
+ top_outputs = []
+
+ # flag
+ collect = False
+ with open(logfile) as log:
+ top_output = []
+ for line in log:
+ lstrip = line.strip()
+ if collect:
+ if lstrip.startswith(top_end):
+ collect = False
+ top_outputs.append(top_output)
+ top_output = []
+ else:
+ top_output.append(lstrip)
+ if lstrip.startswith(top_start):
+ collect = True
+
+ return top_outputs
+
+def summarize_top(top_outs):
+ summaries = []
+ kernel_summaries = []
+ short_summaries = []
+ for top_out in top_outs:
+ summary = {}
+ kernel_summary = {}
+ short_summary = top_out[:top_summary]
+ for line in top_out[top_header:]:
+ cmd = line.split(maxsplit=max_cols)[-1]
+ # kernel processes
+ if cmd[0] == "[" and cmd[-1] == "]":
+ kproc = cmd[1:-1].split("/")[0]
+ if kproc not in kernel_summary:
+ kernel_summary[kproc] = 1
+ else:
+ kernel_summary[kproc] += 1
+ continue
+ cmd_split = cmd.split()
+ prog = cmd_split[0]
+ if prog not in summary:
+ summary[prog] = 1
+ else:
+ summary[prog] += 1
+ summary = dict(sorted(summary.items(), key=lambda item: item[1], reverse=True))
+ kernel_summary = dict(sorted(kernel_summary.items(), key=lambda item: item[1], reverse=True))
+
+ summaries.append(summary)
+ kernel_summaries.append(kernel_summary)
+ short_summaries.append(short_summary)
+
+ return (short_summaries, summaries, kernel_summaries)
+
+def summarize_path(path):
+ p = path
+ for k, v in subs.items():
+ p = p.replace(k, v)
+ return p
+
+def write_summary(short_summary, summary, kernel_summary, logfile):
+ dirname = os.path.dirname(logfile)
+ fname = os.path.basename(logfile)
+ report_name = fname.split(".")[0] + "_summary.txt"
+ outfile = os.path.join(dirname, report_name)
+ out = "NOTE: program names have been shortened for better readability.\nSubstitutions are as follows:\n"
+ for k, v in subs.items():
+ out += (v + " = " + k + "\n")
+ out += "\n"
+
+ out += "top was invoked " + str(len(short_summary)) + " times.\n\n"
+
+ for i in range(len(short_summary)):
+ for l in short_summary[i]:
+ out += (l + "\n")
+
+ out += ("\nSummary: " + "\n")
+ for k, v in summary[i].items():
+ if v > 1:
+ r = summarize_path(k)
+ out += (str(v) + " " + r + "\n")
+
+ out += ("\nKernel Summary: " + "\n")
+ for k, v in kernel_summary[i].items():
+ if v > 1:
+ r = summarize_path(k)
+ out += (str(v) + " " + r + "\n")
+ out += ("\n")
+
+ with open(outfile, "w") as of:
+ of.write(out)
+
+def main():
+ if len(sys.argv) != 3:
+ usage()
+ sys.exit()
+
+ dest = sys.argv[1]
+ target = sys.argv[2]
+ host_data_dir = "intermittent_failure_host_data"
+ directory = os.path.join(dest, target, host_data_dir)
+ for f in glob.glob(directory + "/*_top.txt"):
+ outputs = list_top_outputs(f)
+ short_summary, summary, kernel_summary = summarize_top(outputs)
+ write_summary(short_summary, summary, kernel_summary, f)
+
+main()

--
# Randy MacLeod
# Wind River Linux


[meta-intel]: support for NUC11

simon
 

Hi all,

I'm having trouble figuring out how to fix my issues with the new intel NUC gen 11 (NUC11TNBi5).

At the moment I'm testing with the following meta-layer
poky (hardknott)
meta-openembedded (hardknott)
meta-intel (master)
meta-qt5 (hardknott)

I'm currently using EGLFS to display our Qt-QML GUI

My first issue is with the our GUI, it is launching as expected but we barely have 1 fps

- There's a warning about not finding the IRIS driver:
MESA_LOADER: failed to open iris: /usr/lib/dri/iris_dri.so: cannot open shared object file: No such file or directory
failed to load driver: iris

But from what I was able to understand, I was expecting it would be available with mesa 21.0.3 since I've seen some commit about fixing issue with it in the change logs.

I've tried to force the driver to both i965 and i915 but I either got a warning that this GEN cannot use it or still got the same fps.
Am I right to assume that it goes to a default driver that uses the CPU instead of the GPU?

My second issue is with the nvme slot, it does not detect the partition on the SSD, but it does when I use a NVME to USB adaptor

- There's a few warning about ACPI bios error AE_NOT_FOUND too, and again with my little knowledge I thought this would have been fixed with the most recent update in master with the new microcode since the Core gen 11 was added as new device.
But again, I'm not entirely sure that I'm doing the right thing to update the microcode. I've installed manually the microcode.cpio in /etc/firmware/intel-ucode/

The ouput of "dmesg | grep microcode":
microcode: sig=0x806c1, pf=0x80, revision=0x72
microcode: Microcode Update Driver: v2.2.

I don't know if I'm too early to be using the latest NUC11, any information about when it should be supported would be very appreciated.

I'm willing to help but my knowledge is pretty limited so I might need assistance.

thanks a lot

Simon


[meta-mingw][PATCH] Disable debuginfod

Joshua Watt
 

Disables debuginfod when using MingGW. This feature brings in
unbuildable dependencies and can't be used.

Signed-off-by: Joshua Watt <JPEWhacker@...>
---
conf/machine-sdk/include/mingw32-common.inc | 2 ++
recipes-devtools/gdb/gdb-cross-canadian_%.bbappend | 1 +
2 files changed, 3 insertions(+)

diff --git a/conf/machine-sdk/include/mingw32-common.inc b/conf/machine-sdk/include/mingw32-common.inc
index 0109e75..3997a26 100644
--- a/conf/machine-sdk/include/mingw32-common.inc
+++ b/conf/machine-sdk/include/mingw32-common.inc
@@ -15,6 +15,8 @@ USE_NLS_mingw32 = "no"
FILES_${PN}-staticdev_append_mingw32 = " ${libdir}/*.lib"
ALLOW_EMPTY_${PN}_mingw32 = "1"

+DISTRO_FEATURES_FILTER_NATIVESDK_remove_mingw32 = "debuginfod"
+
# Do what amounts to a NOOP
SDK_PACKAGING_FUNC = "do_compile"

diff --git a/recipes-devtools/gdb/gdb-cross-canadian_%.bbappend b/recipes-devtools/gdb/gdb-cross-canadian_%.bbappend
index c33a9ce..096fc63 100644
--- a/recipes-devtools/gdb/gdb-cross-canadian_%.bbappend
+++ b/recipes-devtools/gdb/gdb-cross-canadian_%.bbappend
@@ -4,3 +4,4 @@ RDEPENDS_${PN}_remove_sdkmingw32 = "nativesdk-python-core nativesdk-python-lang
EXTRA_OECONF_append_sdkmingw32 = " --without-curses --without-system-readline --with-python=no"
PACKAGECONFIG_remove_sdkmingw32 = "readline"
PACKAGECONFIG_remove_sdkmingw32 = "python"
+PACKAGECONFIG_remove_sdkmingw32 = "debuginfod"
--
2.31.1


devtool build failure first time but builds fine successfully second time. #yocto #devtool

raulpblooper@...
 

Hello All,
I am tyro in yocto ,

I am having a strange issue where I am trying to build the recipe  using    devtool build <recipe>  . It fails first time but builds second time.


 
It flags an error related to some linkage of shared library .
But when I checked the shared library , it is present at the same place,
 
First Error : 
 
    ERROR: <recipe-name>1-1.0-r0 do_compile: Execution of 'Path to build directory/tmp-glibc/work/cortexa7t2hf-neon-vfpv4-ostl-linux-gnueabi/<recipe-name>/1.0-r0/temp/run.do_compile.12944' failed with exit code 2
 
 
 
Second Error : 
 
    -L/<path to build directory>/tmp-glibc/work/cortexa7t2hf-neon-vfpv4-ostl-linux-gnueabi/<my recipe>1/1.0-r0/r1-1.0/<aplication name>/../<shared lib folder>/<sub folder name>  -Wl,-rpath,"\$ORIGIN/../<shared lib folder>/<sub folder name>" -l<shared lib name without lib> -pthread 
 
When i run the same command again **( devtool build recipe 
)** it builds Successfully the application referring this shared library.
 
Does anyone know this peculiar issue.?
Since it builds fine with no change i suspect no mistake in my Cmakefiles.
 
Please assist,
 
 


Re: [poky] [PATCH] local.conf.sample: disable prelink

Alexander Kanavin
 

On Wed, 16 Jun 2021 at 11:11, Richard Purdie <richard.purdie@...> wrote:
What makes me sad is that this is an embedded focused feature which 
the project should be caring about, i.e. memory footprint and speed.
I don't know why the numbers don't show it, Mark and I have theories
but it would take work to explore them and neither of us have that time,
nor are we experts on prelink able to maintain it with the time we have
available.

What I think happened is that glibc simply got better at linking, optimised the runtime bits to the fullest, and any benefits prelink may once have had aren't there anymore. Which is IMO fine - I am terrified of custom address arithmetic that's not really understood by anyone, and very easy to get wrong, with devastating consequences. I spent more or less the whole day stepping assembly in gdb to figure out what exactly is going wrong in that ppc bug.

The value proposition of Yocto is still very strong, and is not at all affected by dropping prelink, in my opinion.

Alex


Re: [poky] [PATCH] local.conf.sample: disable prelink

Richard Purdie
 

On Tue, 2021-06-15 at 17:21 +0200, Alexander Kanavin wrote:
So the memory usage is actually *better* without prelink. And any timing 
benefits are lost in statistical noise, in these tests at least.
The numbers certainly don't look convincing, thanks for running the tests.

So I do not think it is wrong to question the usefulness of this feature. 
I'd like to hear Mark's take on this, as prelink-cross is primarily his 
work, he's put a lot of effort into it, and I would want to know where 
the benefits are. Note that Red Hat abandoned prelink in 2013, and 
prelink-cross likewise hasn't seen any commits for two years.
Mark and I did briefly talk yesterday and whilst Mark can/will speak
up, the summary was "drop it".

What makes me sad is that this is an embedded focused feature which 
the project should be caring about, i.e. memory footprint and speed.
I don't know why the numbers don't show it, Mark and I have theories
but it would take work to explore them and neither of us have that time,
nor are we experts on prelink able to maintain it with the time we have
available.

In my view the project should be caring about things like this, instead 
it looks more like a race to look like 'any other desktop distro' and 
drop anything which looks different.

I've tried to say no. Nobody else would appear to care, I'm hearing 
silence. I can't fight this.

Cheers,

Richard


[PATCH yocto-autobuilder-helper] summarize_top_output.py: add script, use it and publish summary

sakib.sajal@...
 

summarize_top_output.py is used to summarize the top
output that is captured during autobuilder intermittent
failures.

Use the script to summarize the host top output and
publish the summary that is created instead of
the raw logfile.

Signed-off-by: Sakib Sajal <sakib.sajal@...>
---
scripts/collect-results | 2 +-
scripts/generate-testresult-index.py | 2 +-
scripts/run-config | 1 +
scripts/summarize_top_output.py | 132 +++++++++++++++++++++++++++
4 files changed, 135 insertions(+), 2 deletions(-)
create mode 100755 scripts/summarize_top_output.py

diff --git a/scripts/collect-results b/scripts/collect-results
index 7474e36..7178380 100755
--- a/scripts/collect-results
+++ b/scripts/collect-results
@@ -19,7 +19,7 @@ if [ -e $WORKDIR/buildhistory ]; then
fi

HSFILE=$WORKDIR/tmp/buildstats/*/host_stats
-d=`date +%Y-%m-%d--%H-%M`
+d="intermittent_failure_host_data"

mkdir -p $DEST/$target/$d

diff --git a/scripts/generate-testresult-index.py b/scripts/generate-testresult-index.py
index 7fdc17c..d85d606 100755
--- a/scripts/generate-testresult-index.py
+++ b/scripts/generate-testresult-index.py
@@ -154,7 +154,7 @@ for build in sorted(os.listdir(path), key=keygen, reverse=True):
hd = []
counter = 0
# do we really need the loop?
- for p in glob.glob(buildpath + "/*/*/host_stats*top.txt"):
+ for p in glob.glob(buildpath + "/*/*/host_stats*top_summary.txt"):
n_split = p.split(build)
res = reldir[0:-1] + n_split[1]
hd.append((res, str(counter)))
diff --git a/scripts/run-config b/scripts/run-config
index 8ed88cf..82de91f 100755
--- a/scripts/run-config
+++ b/scripts/run-config
@@ -327,6 +327,7 @@ elif args.phase == "finish" and args.stepname == "collect-results":
if args.results_dir:
hp.printheader("Running results collection")
runcmd([scriptsdir + "/collect-results", args.builddir, args.results_dir, args.target])
+ runcmd([scriptsdir + "/summarize_top_output.py", args.results_dir, args.target])
sys.exit(0)

if jcfg:
diff --git a/scripts/summarize_top_output.py b/scripts/summarize_top_output.py
new file mode 100755
index 0000000..0606a34
--- /dev/null
+++ b/scripts/summarize_top_output.py
@@ -0,0 +1,132 @@
+#!/usr/bin/env python3
+
+import os, sys, glob
+
+# constants
+top_header = 7
+top_summary = 5
+max_cols = 11
+
+# string substitution to make things easier to read
+subs = {
+ "/home/pokybuild/yocto-worker/" : "~/",
+ "/build/build/tmp/work/core2-32-poky-linux/" : "/.../POKY_32/.../",
+ "/build/build/tmp/work/core2-64-poky-linux/" : "/.../POKY_64/.../",
+ "/recipe-sysroot-native/usr/bin/x86_64-poky-linux/../../libexec/x86_64-poky-linux/gcc/x86_64-poky-linux/" : "/...GCC.../"
+}
+
+def usage():
+ print("Usage: " + sys.argv[0] + " <dest> <target>")
+
+def list_top_outputs(logfile):
+ # top delimiter
+ top_start = "start: top output"
+ top_end = "end: top output"
+
+ # list of top outputs
+ top_outputs = []
+
+ # flag
+ collect = False
+ with open(logfile) as log:
+ top_output = []
+ for line in log:
+ lstrip = line.strip()
+ if collect:
+ if lstrip.startswith(top_end):
+ collect = False
+ top_outputs.append(top_output)
+ top_output = []
+ else:
+ top_output.append(lstrip)
+ if lstrip.startswith(top_start):
+ collect = True
+
+ return top_outputs
+
+def summarize_top(top_outs):
+ summaries = []
+ kernel_summaries = []
+ short_summaries = []
+ for top_out in top_outs:
+ summary = {}
+ kernel_summary = {}
+ short_summary = top_out[:top_summary]
+ for line in top_out[top_header:]:
+ cmd = line.split(maxsplit=max_cols)[-1]
+ # kernel processes
+ if cmd[0] == "[" and cmd[-1] == "]":
+ kproc = cmd[1:-1].split("/")[0]
+ if kproc not in kernel_summary:
+ kernel_summary[kproc] = 1
+ else:
+ kernel_summary[kproc] += 1
+ continue
+ cmd_split = cmd.split()
+ prog = cmd_split[0]
+ if prog not in summary:
+ summary[prog] = 1
+ else:
+ summary[prog] += 1
+ summary = dict(sorted(summary.items(), key=lambda item: item[1], reverse=True))
+ kernel_summary = dict(sorted(kernel_summary.items(), key=lambda item: item[1], reverse=True))
+
+ summaries.append(summary)
+ kernel_summaries.append(kernel_summary)
+ short_summaries.append(short_summary)
+
+ return (short_summaries, summaries, kernel_summaries)
+
+def summarize_path(path):
+ p = path
+ for k, v in subs.items():
+ p = p.replace(k, v)
+ return p
+
+def write_summary(short_summary, summary, kernel_summary, logfile):
+ dirname = os.path.dirname(logfile)
+ fname = os.path.basename(logfile)
+ report_name = fname.split(".")[0] + "_summary.txt"
+ outfile = os.path.join(dirname, report_name)
+ out = "NOTE: program names have been shortened for better readability.\nSubstitutions are as follows:\n"
+ for k, v in subs.items():
+ out += (v + " = " + k + "\n")
+ out += "\n"
+
+ out += "top was invoked " + str(len(short_summary)) + " times.\n\n"
+
+ for i in range(len(short_summary)):
+ for l in short_summary[i]:
+ out += (l + "\n")
+
+ out += ("\nSummary: " + "\n")
+ for k, v in summary[i].items():
+ if v > 1:
+ r = summarize_path(k)
+ out += (str(v) + " " + r + "\n")
+
+ out += ("\nKernel Summary: " + "\n")
+ for k, v in kernel_summary[i].items():
+ if v > 1:
+ r = summarize_path(k)
+ out += (str(v) + " " + r + "\n")
+ out += ("\n")
+
+ with open(outfile, "w") as of:
+ of.write(out)
+
+def main():
+ if len(sys.argv) != 3:
+ usage()
+ sys.exit()
+
+ dest = sys.argv[1]
+ target = sys.argv[2]
+ host_data_dir = "intermittent_failure_host_data"
+ directory = os.path.join(dest, target, host_data_dir)
+ for f in glob.glob(directory + "/*_top.txt"):
+ outputs = list_top_outputs(f)
+ short_summary, summary, kernel_summary = summarize_top(outputs)
+ write_summary(short_summary, summary, kernel_summary, f)
+
+main()
--
2.25.1


[meta-security][PATCH] sssd: add fix-ldblibdir.patch back

kai
 

From: Kai Kang <kai.kang@...>

The patch fix-ldblibdir.patch has been dropped when update sssd to
2.5.0. But it fails to start sssd without this patch. So add it back.

Signed-off-by: Kai Kang <kai.kang@...>
---
.../sssd/files/fix-ldblibdir.patch | 25 +++++++++++++++++++
recipes-security/sssd/sssd_2.5.0.bb | 1 +
2 files changed, 26 insertions(+)
create mode 100644 recipes-security/sssd/files/fix-ldblibdir.patch

diff --git a/recipes-security/sssd/files/fix-ldblibdir.patch b/recipes-security/sssd/files/fix-ldblibdir.patch
new file mode 100644
index 0000000..e350baf
--- /dev/null
+++ b/recipes-security/sssd/files/fix-ldblibdir.patch
@@ -0,0 +1,25 @@
+When calculate value of ldblibdir, it checks whether the directory of
+$ldblibdir exists. If not, it assigns ldblibdir with ${libdir}/ldb. It is not
+suitable for cross compile. Fix it that only re-assign ldblibdir when its value
+is empty.
+
+Upstream-Status: Inappropriate [cross compile specific]
+
+Signed-off-by: Kai Kang <kai.kang@...>
+---
+ src/external/libldb.m4 | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/src/external/libldb.m4 b/src/external/libldb.m4
+index c400add..5e5f06d 100644
+--- a/src/external/libldb.m4
++++ b/src/external/libldb.m4
+@@ -19,7 +19,7 @@ if test x"$with_ldb_lib_dir" != x; then
+ ldblibdir=$with_ldb_lib_dir
+ else
+ ldblibdir="`$PKG_CONFIG --variable=modulesdir ldb`"
+- if ! test -d $ldblibdir; then
++ if test -z $ldblibdir; then
+ ldblibdir="${libdir}/ldb"
+ fi
+ fi
diff --git a/recipes-security/sssd/sssd_2.5.0.bb b/recipes-security/sssd/sssd_2.5.0.bb
index 4c92519..c5c870c 100644
--- a/recipes-security/sssd/sssd_2.5.0.bb
+++ b/recipes-security/sssd/sssd_2.5.0.bb
@@ -21,6 +21,7 @@ SRC_URI = "https://github.com/SSSD/sssd/releases/download/2.5.0/sssd-2.5.0.tar.g
file://no_gen.patch \
file://fix_gid.patch \
file://drop_ntpdate_chk.patch \
+ file://fix-ldblibdir.patch \
"
SRC_URI[sha256sum] = "afa62d7d8d23fca3aba093abe4ec0d14e7d9346c5b28ceb7c2c624bed98caa06"

--
2.17.1


[meta-security][PATCH] aircrack-ng: update to 1.6

Federico Pellegrin
 

Signed-off-by: Federico Pellegrin <fede@...>
---
.../{aircrack-ng_1.3.bb => aircrack-ng_1.6.bb} | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
rename recipes-security/aircrack-ng/{aircrack-ng_1.3.bb => aircrack-ng_1.6.bb} (82%)

diff --git a/recipes-security/aircrack-ng/aircrack-ng_1.3.bb b/recipes-security/aircrack-ng/aircrack-ng_1.6.bb
similarity index 82%
rename from recipes-security/aircrack-ng/aircrack-ng_1.3.bb
rename to recipes-security/aircrack-ng/aircrack-ng_1.6.bb
index d739227..8d3b531 100644
--- a/recipes-security/aircrack-ng/aircrack-ng_1.3.bb
+++ b/recipes-security/aircrack-ng/aircrack-ng_1.6.bb
@@ -9,8 +9,8 @@ DEPENDS = "libnl openssl sqlite3 libpcre libpcap"

SRC_URI = "http://download.aircrack-ng.org/${BP}.tar.gz"

-SRC_URI[md5sum] = "c7c5b076dee0c25ee580b0f56f455623"
-SRC_URI[sha256sum] = "8ae08a7c28741f6ace2769267112053366550e7f746477081188ad38410383ca"
+SRC_URI[md5sum] = "22ddc85549b51ed0da0931d01ef215e5"
+SRC_URI[sha256sum] = "4f0bfd486efc6ea7229f7fbc54340ff8b2094a0d73e9f617e0a39f878999a247"

inherit autotools-brokensep pkgconfig

@@ -29,6 +29,8 @@ do_install () {
make DESTDIR=${D} ${OEMAKE_EXTRA} ext_scripts=true install
}

-FILES_${PN} += "/usr/local/"
+FILES_${PN} += "${libdir}/*.so"
+FILES_SOLIBSDEV = ""
+INSANE_SKIP_${PN} += "dev-so"

RDEPENDS_${PN} = "libpcap"
--
2.31.1


Re: [qa-build-notification] QA notification for completed autobuilder build (yocto-3.4_M1.rc1)

Sangeeta Jain
 

Hi all,

Intel and WR YP QA is planning for QA execution for YP build yocto-3.4_M1.rc1. We are planning to execute following tests for this cycle:

OEQA-manual tests for following module:
1. OE-Core
2. BSP-hw

Runtime auto test for following platforms:
1. MinnowTurbot 32-bit
2. Coffee Lake
3. NUC 7
4. NUC 6
5. Edgerouter
6. Beaglebone

ETA for completion is next Thursday, June 17.

Thanks,
Sangeeta

-----Original Message-----
From: qa-build-notification@... <qa-build-
notification@...> On Behalf Of Pokybuild User
Sent: Saturday, 12 June, 2021 7:49 PM
To: yocto@...
Cc: qa-build-notification@...
Subject: [qa-build-notification] QA notification for completed autobuilder build
(yocto-3.4_M1.rc1)


A build flagged for QA (yocto-3.4_M1.rc1) was completed on the autobuilder
and is available at:


https://autobuilder.yocto.io/pub/releases/yocto-3.4_M1.rc1


Build hash information:

bitbake: 398a1686176c695d103591089a36e25173f9fd6e
meta-arm: 6c3d62c776fc45b4bae47d178899e84b17248b31
meta-gplv2: 1ee1a73070d91e0c727f9d0db11943a61765c8d9
meta-intel: 0937728bcd98dd13d2c6829e1cd604ea2e53e5cd
meta-mingw: bfd22a248c0db4c57d5a72d675979d8341a7e9c1
oecore: 3b2903ccc791d5dedd84c75227f38ae4c8d29251
poky: 59d93693bf24e02ca0f05fe06d96a46f4f0f1bf8



This is an automated message from the Yocto Project Autobuilder
Git: git://git.yoctoproject.org/yocto-autobuilder2
Email: richard.purdie@...







Re: [poky] [PATCH] local.conf.sample: disable prelink

Khem Raj
 

On 6/15/21 8:21 AM, Alexander Kanavin wrote:
On Tue, 15 Jun 2021 at 10:55, Alexander Kanavin <alex.kanavin@... <mailto:alex.kanavin@...>> wrote:
On Tue, 15 Jun 2021 at 10:48, Richard Purdie
<richard.purdie@...
<mailto:richard.purdie@...>> wrote:
I appreciate the desire simply to delete/disable anything that
causes issues
but in this case I draw the line and my answer is no. It works
fine in the
vast majority of usage.
But where are the benchmarks that show it's actually beneficial? And
commitment from someone to maintain it and address open issues
(there are more than just this one)?
I went ahead and ran some quick benchmarks myself. Specifically:
1. Running 'free' after things have settled down at boot:
a) without prelink
core-image-sato-sdk
               total        used        free      shared  buff/cache available
Mem:          489352      152104      236284         472      100964  323824
core-image-ptest-fast
              total        used        free      shared  buff/cache available
Mem:        1004680       43456      927688         256       33536  941156
b) with prelink
core-image-sato-sdk
               total        used        free      shared  buff/cache available
Mem:          489352      153048      235544         468      100760  322900
core-image-ptest-fast
              total        used        free      shared  buff/cache available
Mem:        1004680       44444      928128         256       32108  940168
2. Running -c testimage
a) without prelink
core-image-sato-sdk () - Ran 66 tests in 96.693s
core-image-sato-sdk () - Ran 66 tests in 96.469s
core-image-sato-sdk () - Ran 66 tests in 94.994s
core-image-ptest-fast () - Ran 66 tests in 583.767s
core-image-ptest-fast () - Ran 66 tests in 576.564s
core-image-ptest-fast () - Ran 66 tests in 576.797s
b) with prelink
core-image-sato-sdk () - Ran 66 tests in 96.390s
core-image-sato-sdk () - Ran 66 tests in 96.615s
core-image-sato-sdk () - Ran 66 tests in 95.596s
core-image-ptest-fast () - Ran 66 tests in 576.248s
core-image-ptest-fast () - Ran 66 tests in 574.618s
core-image-ptest-fast () - Ran 66 tests in 576.760s
I think the advantage is not on high end CPUs but more on less powerful ones, so perhaps trying it on something like rpi0 or lower would be good


So the memory usage is actually *better* without prelink. And any timing benefits are lost in statistical noise, in these tests at least.
So I do not think it is wrong to question the usefulness of this feature. I'd like to hear Mark's take on this, as prelink-cross is primarily his work, he's put a lot of effort into it, and I would want to know where the benefits are. Note that Red Hat abandoned prelink in 2013, and prelink-cross likewise hasn't seen any commits for two years.


Alex


Re: [poky] [PATCH] local.conf.sample: disable prelink

Alexander Kanavin
 



On Tue, 15 Jun 2021 at 10:55, Alexander Kanavin <alex.kanavin@...> wrote:
On Tue, 15 Jun 2021 at 10:48, Richard Purdie <richard.purdie@...> wrote:
I appreciate the desire simply to delete/disable anything that causes issues
but in this case I draw the line and my answer is no. It works fine in the
vast majority of usage.

But where are the benchmarks that show it's actually beneficial? And commitment from someone to maintain it and address open issues (there are more than just this one)?

I went ahead and ran some quick benchmarks myself. Specifically:

1. Running 'free' after things have settled down at boot:

a) without prelink
core-image-sato-sdk
               total        used        free      shared  buff/cache   available
Mem:          489352      152104      236284         472      100964      323824
core-image-ptest-fast
              total        used        free      shared  buff/cache   available
Mem:        1004680       43456      927688         256       33536      941156

b) with prelink
core-image-sato-sdk
               total        used        free      shared  buff/cache   available
Mem:          489352      153048      235544         468      100760      322900
core-image-ptest-fast
              total        used        free      shared  buff/cache   available
Mem:        1004680       44444      928128         256       32108      940168

2. Running -c testimage

a) without prelink
core-image-sato-sdk () - Ran 66 tests in 96.693s
core-image-sato-sdk () - Ran 66 tests in 96.469s
core-image-sato-sdk () - Ran 66 tests in 94.994s
core-image-ptest-fast () - Ran 66 tests in 583.767s
core-image-ptest-fast () - Ran 66 tests in 576.564s
core-image-ptest-fast () - Ran 66 tests in 576.797s

b) with prelink
core-image-sato-sdk () - Ran 66 tests in 96.390s
core-image-sato-sdk () - Ran 66 tests in 96.615s
core-image-sato-sdk () - Ran 66 tests in 95.596s
core-image-ptest-fast () - Ran 66 tests in 576.248s
core-image-ptest-fast () - Ran 66 tests in 574.618s
core-image-ptest-fast () - Ran 66 tests in 576.760s

So the memory usage is actually *better* without prelink. And any timing benefits are lost in statistical noise, in these tests at least.

So I do not think it is wrong to question the usefulness of this feature. I'd like to hear Mark's take on this, as prelink-cross is primarily his work, he's put a lot of effort into it, and I would want to know where the benefits are. Note that Red Hat abandoned prelink in 2013, and prelink-cross likewise hasn't seen any commits for two years.

Alex


Yocto Project Status WW24`21

Stephen Jolley
 

Current Dev Position: YP 3.4 M2

Next Deadline: 12th July 2021 YP 3.4 M2 build

 

Next Team Meetings:

 

Key Status/Updates:

  • YP 3.4 M1 was built and is in the QA process.
  • The intermittent ltp hanging issue is continuing to be tracked down, currently thought to be a kernel bug introduced around the 5.0 -> 5.1 kernel version window. We ended up unblocking M1 since it is clear the LTP issue will take longer to unravel. Whilst aufs was initially thought to be a possible cause, it wasn’t the issue.
  • We are working around the autobuilder centos8 issue which is an issue with the upstream centos kernel where syscalls were backported in the latest version and don’t work properly.
  • We have a 10th anniversary T-shirt and some other Yocto Project items (hoody, stickers, mugs etc.) now available at https://yoctoproject.org/shop (EU and Americas sources)
  • The multiconfig changes in bitbake continue to cause problems, we still need simpler test cases to reproduce issues rather than huge builds. The existing patches seem to fix some workloads and break others. Richard is trying to fix but trying to fix autobuilder issues and other problems and these are slow builds to debug.
  • Intermittent autobuilder issues continue to occur and continue to be at record high levels. You can see the list of failures we’re continuing to see by searching for the “AB-INT” tag in bugzilla: https://bugzilla.yoctoproject.org/buglist.cgi?quicksearch=AB-INT

We are working to identify the load pattern on the infrastructure that seems to trigger these.

 

Ways to contribute:

 

YP 3.4 Milestone Dates:

  • YP 3.4 M1 is in QA
  • YP 3.4 M1 Release date 2021/06/18
  • YP 3.4 M2 build date 2021/07/12
  • YP 3.4 M2 Release date 2021/07/23
  • YP 3.4 M3 build date 2021/08/23 (Feature Freeze)
  • YP 3.4 M3 Release date 2021/09/03
  • YP 3.4 M4 build date 2021/10/04
  • YP 3.4 M4 Release date 2021/10/29

 

Planned upcoming dot releases:

  • YP 3.1.9 build date 2021/06/21
  • YP 3.1.9 release date 2021/07/02
  • YP 3.3.2 build date 2021/07/19
  • YP 3.3.2 release date 2021/07/30
  • YP 3.1.10 build date 2021/07/26
  • YP 3.1.10 release date 2021/08/06
  • YP 3.1.11 build date 2021/09/13
  • YP 3.1.11 release date 2021/9/24

 

Tracking Metrics:

 

The Yocto Project’s technical governance is through its Technical Steering Committee, more information is available at:

https://wiki.yoctoproject.org/wiki/TSC

 

The Status reports are now stored on the wiki at: https://wiki.yoctoproject.org/wiki/Weekly_Status

 

[If anyone has suggestions for other information you’d like to see on this weekly status update, let us know!]

 

Thanks,

 

Stephen K. Jolley

Yocto Project Program Manager

(    Cell:                (208) 244-4460

* Email:              sjolley.yp.pm@...

 


[meta-rockchip][PATCH v3] Rock64: add machine

Yann Dirson
 

From: Yann Dirson <yann@...>

This is a RK3328 board from Pine64.
Board details at https://wiki.pine64.org/wiki/ROCK64.

Default image is built to boot from SD-card. Building an image for
eMMC requires to set RK_BOOT_DEVICE=3D"mmcblk0".

Signed-off-by: Yann Dirson <yann@...>
---

This is just basic initial support without a kernel BSP. As is the
board boots with a serial console.

Note I had to create the SoC definition for rk3328, and rather than
setting serial at 115200 there just to have the board definition
override it with rockchip-standard 1500000 I've set the latter right
in rk3328.inc.

Changes in v3:
- add board to README
- use rock-pi-e.wks rather than rock-pi-4.wks (identical contents today)
- put copyright info straight

Changes in v2:
- include Ayufan's patch for mmc aliases so presence of eMMC won't
prevent booting from SD

README | 1 +
conf/machine/include/rk3328.inc | 25 +++++++++++++++
conf/machine/rock64.conf | 31 +++++++++++++++++++
...an-dtsi-rk3328-add-mmc0-mmc1-aliases.patch | 27 ++++++++++++++++
recipes-kernel/linux/linux-yocto%.bbappend | 6 ++++
5 files changed, 90 insertions(+)
create mode 100644 conf/machine/include/rk3328.inc
create mode 100644 conf/machine/rock64.conf
create mode 100644 recipes-kernel/linux/files/0001-ayufan-dtsi-rk3328-ad=
d-mmc0-mmc1-aliases.patch

diff --git a/README b/README
index b825eba..cec1b53 100644
--- a/README
+++ b/README
@@ -24,6 +24,7 @@ Status of supported boards:
rock-pi-4a
rock-pi-4b
rock-pi-4c
+ rock64
tinker-board
tinker-board-s
vyasa-rk3288
diff --git a/conf/machine/include/rk3328.inc b/conf/machine/include/rk332=
8.inc
new file mode 100644
index 0000000..a4bbc5d
--- /dev/null
+++ b/conf/machine/include/rk3328.inc
@@ -0,0 +1,25 @@
+# Copyright (C) 2021 Blade SAS
+# Released under the MIT license (see COPYING.MIT for the terms)
+
+SOC_FAMILY =3D "rk3328"
+
+DEFAULTTUNE ?=3D "cortexa53-crypto"
+
+require conf/machine/include/soc-family.inc
+require conf/machine/include/tune-cortexa53.inc
+require conf/machine/include/rockchip-defaults.inc
+
+KBUILD_DEFCONFIG ?=3D "defconfig"
+KERNEL_CLASSES =3D "kernel-fitimage"
+KERNEL_IMAGETYPE =3D "fitImage"
+
+TFA_PLATFORM =3D "rk3328"
+TFA_BUILD_TARGET =3D "bl31"
+
+UBOOT_SUFFIX ?=3D "itb"
+UBOOT_ENTRYPOINT ?=3D "0x06000000"
+
+SERIAL_CONSOLES =3D "1500000;ttyS2"
+
+PREFERRED_PROVIDER_virtual/bootloader ?=3D "u-boot"
+SPL_BINARY ?=3D "idbloader.img"
diff --git a/conf/machine/rock64.conf b/conf/machine/rock64.conf
new file mode 100644
index 0000000..acda018
--- /dev/null
+++ b/conf/machine/rock64.conf
@@ -0,0 +1,31 @@
+# Copyright (C) 2021 Blade SAS
+# Released under the MIT license (see COPYING.MIT for the terms)
+
+#@TYPE: Machine
+#@NAME: Rock64
+#@DESCRIPTION: Rock64 RK3328 board from Pine64
+
+require include/rk3328.inc
+
+MACHINE_FEATURES +=3D "usbhost serial"
+
+UBOOT_MACHINE =3D "rock64-rk3328_defconfig"
+KERNEL_DEVICETREE =3D "rockchip/rk3328-rock64.dtb"
+
+# set to mmcblk0 for booting from optional eMMC
+RK_BOOT_DEVICE ?=3D "mmcblk1"
+
+WKS_FILE ?=3D "rock-pi-e.wks"
+IMAGE_FSTYPES +=3D "wic wic.bmap"
+
+WKS_FILE_DEPENDS ?=3D " \
+ mtools-native \
+ dosfstools-native \
+ virtual/bootloader \
+ virtual/kernel \
+ "
+IMAGE_BOOT_FILES ?=3D "\
+ ${KERNEL_IMAGETYPE} \
+ "
+
+KBUILD_DEFCONFIG =3D "defconfig"
diff --git a/recipes-kernel/linux/files/0001-ayufan-dtsi-rk3328-add-mmc0-=
mmc1-aliases.patch b/recipes-kernel/linux/files/0001-ayufan-dtsi-rk3328-a=
dd-mmc0-mmc1-aliases.patch
new file mode 100644
index 0000000..1ad3b9e
--- /dev/null
+++ b/recipes-kernel/linux/files/0001-ayufan-dtsi-rk3328-add-mmc0-mmc1-al=
iases.patch
@@ -0,0 +1,27 @@
+From f10cfe01f753348d346374008b8e8f5f26ed94ab Mon Sep 17 00:00:00 2001
+From: Kamil Trzcinski <ayufan@...>
+Date: Mon, 28 Aug 2017 11:24:37 +0200
+Subject: [PATCH] ayufan: dtsi: rk3328: add mmc0/mmc1 aliases
+Upstream-Status: Pending [https://github.com/ayufan-rock64/linux-mainlin=
e-kernel/commit/f10cfe01f753348d346374008b8e8f5f26ed94ab]
+
+Change-Id: I82a5394df8a505f7d1496393621c1198895c88b0
+---
+ arch/arm64/boot/dts/rockchip/rk3328.dtsi | 2 ++
+ 1 file changed, 2 insertions(+)
+
+diff --git a/arch/arm64/boot/dts/rockchip/rk3328.dtsi b/arch/arm64/boot/=
dts/rockchip/rk3328.dtsi
+index 0afed15bc7ff..800f1c796882 100644
+--- a/arch/arm64/boot/dts/rockchip/rk3328.dtsi
++++ b/arch/arm64/boot/dts/rockchip/rk3328.dtsi
+@@ -27,6 +27,8 @@
+ i2c1 =3D &i2c1;
+ i2c2 =3D &i2c2;
+ i2c3 =3D &i2c3;
++ mmc0 =3D &emmc;
++ mmc1 =3D &sdmmc;
+ ethernet0 =3D &gmac2io;
+ ethernet1 =3D &gmac2phy;
+ };
+--=20
+2.30.2
+
diff --git a/recipes-kernel/linux/linux-yocto%.bbappend b/recipes-kernel/=
linux/linux-yocto%.bbappend
index 7702e3f..3789c72 100644
--- a/recipes-kernel/linux/linux-yocto%.bbappend
+++ b/recipes-kernel/linux/linux-yocto%.bbappend
@@ -8,3 +8,9 @@ COMPATIBLE_MACHINE_tinker-board-s =3D "tinker-board-s"
COMPATIBLE_MACHINE_rock-pi-4 =3D "rock-pi-4"
COMPATIBLE_MACHINE_nanopi-m4 =3D "nanopi-m4"
COMPATIBLE_MACHINE_nanopi-m4-2gb =3D "nanopi-m4-2gb"
+COMPATIBLE_MACHINE_rock64 =3D "rock64"
+
+FILESEXTRAPATHS_prepend :=3D "${THISDIR}/files:"
+
+# indeed applicable to all rk3328 boards
+SRC_URI_append_rock64 =3D " file://0001-ayufan-dtsi-rk3328-add-mmc0-mmc1=
-aliases.patch"
--=20
2.30.2


Re: [meta-rockchip][PATCH v2] Rock64: add machine

Yann Dirson
 

Le mar. 15 juin 2021 à 10:16, Trevor Woerner <twoerner@...> a écrit :

On Tue 2021-06-15 @ 10:03:31 AM, Yann Dirson wrote:
Le lun. 14 juin 2021 à 18:19, Trevor Woerner <twoerner@...> a écrit :

Hi Yann,

Thanks for the contribution!

On Mon 2021-05-31 @ 04:00:58 PM, yann.dirson@... wrote:
From: Yann Dirson <yann@...>

This is a RK3328 board from Pine64.
Board details at https://wiki.pine64.org/wiki/ROCK64.

Default image is built to boot from SD-card. Building an image for
eMMC requires to set RK_BOOT_DEVICE="mmcblk0".

Signed-off-by: Yann Dirson <yann@...>
---

This is just basic initial support without a kernel BSP. As is the
board boots with a serial console.

Note I had to create the SoC definition for rk3328, and rather than
setting serial at 115200 there just to have the board definition
override it with rockchip-standard 1500000 I've set the latter right
in rk3328.inc.
Sounds good. I prefer the rockchip default of 1,500,000 anyway.


Changes in v2:
- include Ayufan's patch for mmc aliases so presence of eMMC won't
prevent booting from SD

conf/machine/include/rk3328.inc | 25 ++++++++++++++++
conf/machine/rock64.conf | 30 +++++++++++++++++++
...an-dtsi-rk3328-add-mmc0-mmc1-aliases.patch | 27 +++++++++++++++++
recipes-kernel/linux/linux-yocto%.bbappend | 6 ++++
4 files changed, 88 insertions(+)
create mode 100644 conf/machine/include/rk3328.inc
create mode 100644 conf/machine/rock64.conf
create mode 100644 recipes-kernel/linux/files/0001-ayufan-dtsi-rk3328-add-mmc0-mmc1-aliases.patch

diff --git a/conf/machine/include/rk3328.inc b/conf/machine/include/rk3328.inc
new file mode 100644
index 0000000..7d67627
--- /dev/null
+++ b/conf/machine/include/rk3328.inc
@@ -0,0 +1,25 @@
+# Copyright (C) 2020 Garmin Ltd. or its subsidaries
Odd that you'd be assigning the copyright to Garmin ;-)
Oh, right, copypaste rules around here, so Garmin does have a role but
something may be missing :)


+# Released under the MIT license (see COPYING.MIT for the terms)
+
+SOC_FAMILY = "rk3328"
+
+DEFAULTTUNE ?= "cortexa53-crypto"
+
+require conf/machine/include/soc-family.inc
+require conf/machine/include/tune-cortexa53.inc
+require conf/machine/include/rockchip-defaults.inc
+
+KBUILD_DEFCONFIG ?= "defconfig"
+KERNEL_CLASSES = "kernel-fitimage"
+KERNEL_IMAGETYPE = "fitImage"
+
+TFA_PLATFORM = "rk3328"
+TFA_BUILD_TARGET = "bl31"
+
+UBOOT_SUFFIX ?= "itb"
+UBOOT_ENTRYPOINT ?= "0x06000000"
+
+SERIAL_CONSOLES = "1500000;ttyS2"
+
+PREFERRED_PROVIDER_virtual/bootloader ?= "u-boot"
+SPL_BINARY ?= "idbloader.img"
diff --git a/conf/machine/rock64.conf b/conf/machine/rock64.conf
new file mode 100644
index 0000000..38bc9fa
--- /dev/null
+++ b/conf/machine/rock64.conf
@@ -0,0 +1,30 @@
+# Copyright (C) 2021 Blade SAS
Can you also specify an OSS-friendly licence too?

+
+#@TYPE: Machine
+#@NAME: Rock64
+#@DESCRIPTION: Rock64 RK3328 board from Pine64
+
+require include/rk3328.inc
+
+MACHINE_FEATURES += "usbhost serial"
+
+UBOOT_MACHINE = "rock64-rk3328_defconfig"
+KERNEL_DEVICETREE = "rockchip/rk3328-rock64.dtb"
+
+# set to mmcblk0 for booting from optional eMMC
+RK_BOOT_DEVICE ?= "mmcblk1"
+
+WKS_FILE ?= "rock-pi-4.wks"
Personally I'd prefer to see a rock64 wic file which includes an rk3328
default, even if it is a copy of the rock-pi-4 layout.
In fact we already have rock-pi-e.wks doing this.


Right that was something I wondered how to deal with and forgot (and note that
for the nanopi-m4 I used the same file).

My reading of [1] is that all rockchip APs are using the same
partition table. I see
that the existing {rk3288,rk3399}-boot.wks only differ in the choice
of u-boot image,
and I'm wondering if using the SoC to distinguish between them is the
right choice,
as eg. upstream RK is not using the .itb format, and I suspect we
could use it as well
for rk3288 (I'm sure I have one in a drawer and could check that some day). Now
maybe the sate of things is different for 32bit SoCs, and I thought it
could make sense to
distinguish rockchip-32bit-boot.wks and rockchip-64bit-boot.wks, or maybe even
name them using the format, as something like rockchip-legacy-boot.wks
(well there
is probably a more descriptive name for that format) and rockchip-itb-boot.wks ?

Similarly, the .wks for 3288-based boards and for 3399-based ones only
differ by the
console baudrate, and the 3288-based .wks are all identical except for
some whitespace.
And in fact, it is not unthinkable for a given project to use
something else than a
single-partition layout, so those files are indeed closer to a
"default wks". Maybe we
could use more generic filenames there too (until we implement a way to avoid
hardcoding the baudrate here)

[1] http://opensource.rock-chips.com/wiki_Boot_option
True. We could definitely use some cleanup in this area. If you want to
propose something that's going to work, I'll add it.

Also, when adding a new board, please update the top-level README.
Let's do the minimum for now for this to get merged, and we'll improve
incrementally.
Eg. only now I realize, through the presence of rk3328-boot.wks, that
rock-pi-e is
indeed also based on that soc, and its machine def would benefit from
including the
same rk3328.inc.

--
Yann Dirson <yann@...>
Blade / Shadow -- http://shadow.tech

3881 - 3900 of 57741