1. yocto@lists.yoctoproject.org
  2. Messages
   Date   
[yocto-autobuilder2][PATCH] add prioritizeBuilders

Trevor Gamblin
 

This prioritizeBuilders function sorts builders by the length of their
associated worker lists, so that builders that can only be assigned to a
small number of workers are assigned to those workers before other
builds that don't have specific needs when resources are limited. An
example might be when a slot is available on an Ubuntu-based worker, and
"oe-selftest-ubuntu" and "genericx86-64" build requests exist in the
queue. Since oe-selftest-ubuntu requires an Ubuntu-based worker and
genericx86-64 does not, genericx86-64 will be assigned a higher value
(lower priority) so that oe-selftest-ubuntu is assigned to that worker
first.

Signed-off-by: Trevor Gamblin <trevor.gamblin@...>
---
builders.py | 13 +++++++++++++
master.cfg | 6 ++++++
2 files changed, 19 insertions(+)

diff --git a/builders.py b/builders.py
index 53c5f0e..94fb561 100644
--- a/builders.py
+++ b/builders.py
@@ -135,6 +135,19 @@ for builder in config.subbuilders:
workernames=workers, nextWorker=nextWorker, nextBuild=nextBuild,
factory=f, env=extra_env))

+# prioritize assigning builders to available workers based on the length
+# of the worker lists they are associated with. Builders that have fewer
+# valid worker options should always be assigned first
+def prioritizeBuilders(buildmaster, builders):
+ # re-use the builder_to_workers list
+ builder_to_workers = config.builder_to_workers
+
+ # sort builders by the length of their worker lists. Since not all
+ # builders are explicitly listed in builder_to_workers, make sure to
+ # default to the len() of the "default" value
+ builders.sort(key=lambda b: len(builder_to_workers.get(b.name)) if b.name in builder_to_workers.keys() else len(builder_to_workers.get("default")))
+ return builders
+
def create_parent_builder_factory(buildername, waitname):
factory = util.BuildFactory()
# NOTE: Assumes that yocto-autobuilder repo has been cloned to home
diff --git a/master.cfg b/master.cfg
index a7c151f..4f7d74e 100644
--- a/master.cfg
+++ b/master.cfg
@@ -88,6 +88,12 @@ c['www'] = www.www
# These items are specific to an individual AB deployment
c['workers'] = workers.workers

+# This enables our prioritizeBuilders function from builders.py.
+# Builders such as buildperf-centos7, buildperf-ubuntu1604,
+# oe-selftest-*, and reproducible-* will be assigned (if possible)
+# before other builders since their possible worker lists are smaller
+c['prioritizeBuilders'] = builders.prioritizeBuilders
+
c['title'] = "Yocto Autobuilder"
c['titleURL'] = "https://autobuilder.yoctoproject.org/main/"
# visible location for internal web server
--
2.31.1
regd pTest #systemd #yocto

rpaluri@...
 

Hi,
  1. I'm aware that yocto picks up packages from respective open source community. For e.g. dbus package is picked from freedesktop.org etc. In this regard, what kind of tests does yocto runs to ensure that the package that is getting picked-up is stable enough?
  2. Is it Yocto, that writes pTest's to ensure that the package that is getting picked up is stable enough?
  3. I could find pTest report for e.g. on Dunfell @ http://downloads.yoctoproject.org/releases/yocto/yocto-3.1.1/testreport.txt (Refer "qemuarm64 PTest Result Summary" or "qemux86-64 PTest Result Summary" section for details). The test report only details the number of pTest's that passed, failed or skipped for respective package. Is there any report that details the pTest's, which failed, passed or skipped?
Additionally, I'm looking for pTest's for systemd, udev and glibc packages.
Below are my observations on these packages w.r.t pTests.
  1. pTest for systemd package has been removed with http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/meta/recipes-core/systemd?h=dunfell&id=40db00818e3b1cce8aa85ee30f3081ae9a5f8e66 commit. 
  2. Similarly pTest for udev has been removed with http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/meta/recipes-core?h=dunfell&id=3e5e540513665105b963262c2eaf33f197a0a36c.
  3. I don't find ptest's for glibc package either.
As there aren't any pTests for these three packages, are there any other existing tests that can be run to validate these packages?

Thanks,
Ravi
Re: [qa-build-notification] QA notification for completed autobuilder build (yocto-3.4_M3.rc1)

Teoh, Jay Shen
 

Hi All,

This is the full report for yocto-3.4_M3.rc1:
https://git.yoctoproject.org/cgit/cgit.cgi/yocto-testresults-contrib/tree/?h=intel-yocto-testresults

======= Summary ========
No high milestone defects.

No new issue found.


Thanks,
Jay

-----Original Message-----
From: qa-build-notification@... <qa-build-
notification@...> On Behalf Of Richard Purdie
Sent: Saturday, 4 September, 2021 9:27 PM
To: <yocto@...> <yocto@...>
Cc: qa-build-notification <qa-build-notification@...>
Subject: [qa-build-notification] QA notification for completed autobuilder build
(yocto-3.4_M3.rc1)

A build flagged for QA (yocto-3.4_M3.rc1) was completed on the autobuilder
and is available at:


https://autobuilder.yocto.io/pub/releases/yocto-3.4_M3.rc1


Build hash information:

bitbake: 0a11696e0898c3c5108e6d7c5ad28da50e00ea66
meta-agl: 60344efa7a50dc2548fc4b5d68b5ad4d60c4023a
meta-arm: 46e8fc6a67efbcc357cf507b903a3e3e133c78f7
meta-aws: 32ae20566a39454ab0ba4c80c23a32ed7c14dcaf
meta-gplv2: f04e4369bf9dd3385165281b9fa2ed1043b0e400
meta-intel: cb1bf2bdc1b20f76fde8b291a12b361a4bc2511e
meta-mingw: f5d761cbd5c957e4405c5d40b0c236d263c916a8
meta-openembedded: 1511e25cea69b98bf2778984d7a649dad5597878
oecore: ffb886497390d4de2631bda671f2f631bc0bc7be
poky: f2728d3ec8c0589e02e9a3ce7cf8aca902cae0a3



This is an automated message from the Yocto Project Autobuilder
Git: git://git.yoctoproject.org/yocto-autobuilder2
Email: richard.purdie@...
Enable graphic in core-image-minimal for raspberrypi #yocto

yasminebenghozzi6@...
 

Hello, 
I cant find how to add maybe graphic conf for the yocto image I already have for the raspberrypi, 
Any help? 
extra-cmake-modules #yocto

sateesh m
 

Hi All,

I am trying to build an extra-cmake-modules package. Using bitbake command "bitbake extra-cmake-modules" compiled succuss. But using bitbake core-image-plasma-mobile throwing error. So I want to disable native build on the x86 platform is it possible? or else can anybody know how to fix this issue suggest me any corrections required.


ERROR: extra-cmake-modules-native-5.79.0-r0 do_compile: Execution of '/home/sateesh/KDE/sources/fu540-build/tmp-glibc/work/x86_64-linux/extra-cmake-modules-native/5.79.0-r0/temp/run.do_compile.20787' failed with exit code 1:
[1/2] cd /home/sateesh/KDE/sources/fu540-build/tmp-glibc/work/x86_64-linux/extra-cmake-modules-native/5.79.0-r0/build/docs && /usr/local/bin/sphinx-build -c /home/sateesh/KDE/sources/fu540-build/tmp-glibc/work/x86_64-linux/extra-cmake-modules-native/5.79.0-r0/build/docs -d /home/sateesh/KDE/sources/fu540-build/tmp-glibc/work/x86_64-linux/extra-cmake-modules-native/5.79.0-r0/build/docs/doctrees -b html /home/sateesh/KDE/sources/fu540-build/tmp-glibc/work/x86_64-linux/extra-cmake-modules-native/5.79.0-r0/git/docs /home/sateesh/KDE/sources/fu540-build/tmp-glibc/work/x86_64-linux/extra-cmake-modules-native/5.79.0-r0/build/docs/html > build-html.log
FAILED: docs/doc_format_html 
cd /home/sateesh/KDE/sources/fu540-build/tmp-glibc/work/x86_64-linux/extra-cmake-modules-native/5.79.0-r0/build/docs && /usr/local/bin/sphinx-build -c /home/sateesh/KDE/sources/fu540-build/tmp-glibc/work/x86_64-linux/extra-cmake-modules-native/5.79.0-r0/build/docs -d /home/sateesh/KDE/sources/fu540-build/tmp-glibc/work/x86_64-linux/extra-cmake-modules-native/5.79.0-r0/build/docs/doctrees -b html /home/sateesh/KDE/sources/fu540-build/tmp-glibc/work/x86_64-linux/extra-cmake-modules-native/5.79.0-r0/git/docs /home/sateesh/KDE/sources/fu540-build/tmp-glibc/work/x86_64-linux/extra-cmake-modules-native/5.79.0-r0/build/docs/html > build-html.log
 
Extension error:
Could not import extension ecm (exception: cannot import name 'htmlescape')
ninja: build stopped: subcommand failed.
WARNING: exit code 1 from a shell command.
 
ERROR: Logfile of failure stored in: /home/sateesh/KDE/sources/fu540-build/tmp-glibc/work/x86_64-linux/extra-cmake-modules-native/5.79.0-r0/temp/log.do_compile.20787
 
--
Regards,
Sateesh
Re: [meta-security][PATCH 1/2] image-with-hardened-binaries: add class

Maximilian Blenk
 

Hi,

I'm really not used to this e-mail process :-D
forwording my original answer to the mailing list...

BR Max

On 06.09.21 10:18, Maximilian Blenk wrote:

Hi Robert, Hi Armin,

sorry for the late reply.

Regarding  Armins question:
If the config file you've used (e.g. th eon ein the selftest) enables all the tests, then it should be fine indeed. I'm not completely sure what the core-mage-minimal includes though. It if it contains systemd the build should be failing if the rpath check is enabled (you would have to whitelist the binaries in the config file)
Currently there is no test for failing. Should i add one?

Sure i can remove the ".py", i just want to avoid confusion of "checksec.sh" and "checksec.py" (which are completely different tools)

Regarding Roberts question:
In case the tooling finds a violation it fails the build and outputs an error message containing the buildsystem paths to the binaries and the actual check it is failing. The message also contains some reasoning on why the used feature shouldn't be used. The bitbake class basically take the output you pasted (there is a json mode) and checks it against the whitelist (and some other unreasonable things, such as usage of relro in statically linked applications)


Br Max

On 21.08.21 18:59, Robert Berger wrote:
Hi,

On 21/08/2021 18:35, Armin Kuster wrote:

Regarding the selftest, is there test for failure?

I ran this against core-image-minimal and nothing was printed out. Does
that mean its fine?

You may want to remove the ".py" from
python3-checksec.py-native_0.6.1.bb, its not needed.

If you run checksec manually against some binary e.g. ls.coreutils it outputs something like this:

https://pastebin.com/JkeN1h3k

Not sure what this should output.


-armin

Re: Inactive service of hello world #yocto

Nicolas Jeker
 

On Tue, 2021-09-07 at 10:43 -0700, Rudolf J Streif wrote:
What does your 'Hello World' service do? Just print 'Hello World' to
the console and then exit?

If so that would be the reason why it is inactive. Systemd starts it
and then it exits. Typically, services keep running (except for some
special cases) once they are started. That's what makes them a service.
To stop them 'systemctl stop <service>' is used. By default systemctl
sends SIGTERM to tell the service to clean up and terminate itself.
To piggyback on that, it is possible to set Type=oneshot[1] for the
service so that systemd considers the service to be up after it exited.
I find it useful for init or migration scripts that another service can
then depend upon.

[1]:
https://www.freedesktop.org/software/systemd/man/systemd.service.html#Type=


:rjs


On Tue, Sep 7, 2021, 10:20 <yasminebenghozzi6@...> wrote:
Hi everyone, 

I wanted to run the helloworld service, but it keeps telling that its
inactive , I really cant find why, I changed the service many times
but no solution.
I m sorry I ant copy paste from putty
Re: Building Yocto on AWS container getting ERROR: The postinstall intercept hook 'update_icon_cache' failed, #yocto

mail2uvijay@...
 

For the error,  i have tried installed below package in aws container
libgdk-pixbuf2.0-0 in container,
and added DEPENDS += "qemuwrapper-cross" in bb file.

Will try the option building "core-image-minimal" or "core-image-sato"

Regards,
Vijay
Re: Building Yocto on AWS container getting ERROR: The postinstall intercept hook 'update_icon_cache' failed, #yocto

Alexander Kanavin
 

It's right there at the very end:
NOTE: Exit code 127. Output:
/home/wlsom/YoctoCode/poky/build-microchip/tmp/work/sama5d27_wlsom1_ek_sd-poky-linux-gnueabi/utec-mcdgrill-image/1.0-r0/intercept_scripts-7f1ca27092c337bb8d06dfe7837dad99a9fb3e5e5ff266e8ce34762c23025acb/update_icon_cache: 9: /home/wlsom/YoctoCode/poky/build-microchip/tmp/work/sama5d27_wlsom1_ek_sd-poky-linux-gnueabi/utec-mcdgrill-image/1.0-r0/intercept_scripts-7f1ca27092c337bb8d06dfe7837dad99a9fb3e5e5ff266e8ce34762c23025acb/update_icon_cache: /home/wlsom/YoctoCode/poky/build-microchip/tmp/work/sama5d27_wlsom1_ek_sd-poky-linux-gnueabi/utec-mcdgrill-image/1.0-r0/recipe-sysroot-native//gdk-pixbuf-2.0/gdk-pixbuf-query-loaders: not found

This isn't easy to diagnose without having access to your build tree, but you can build plain core-image-sato for example and check that the binary does exist there and gets executed, and then untangle where the difference come from.

Alex


On Wed, 8 Sept 2021 at 06:47, <mail2uvijay@...> wrote:

Raj thanks for the response. Couldnt able to find error from the logs..

Attached detailed logs..


multilib SDK

Arun
 

I have a custom project where I am building for 64-bit kernel and 32-bit userspace using multilib config

# for 32bit userspace build
require conf/multilib.conf
MULTILIBS = "multilib:lib32"
DEFAULTTUNE_virtclass-multilib-lib32 = "armv7athf-neon"

I am successfully able to build the target image as
MACHINE=<custom_machine> bitbake lib32-<custom_target>

However, when I try to build the SDK, I am noticing that the userspace recipes are being built for  both 64-bit and 32-bit. I am running into errors for 64-bit. I have no need for 64-bit, is there any way I can disable 64-bit build for SDK?

note: I am building SDK via this command without lib32- prefix for target, following details from this commit message stating that lib32- prefix is not required
MACHINE=<custom_machine> bitbake -c populate_sdk <custom_target>

thanks
-Arun
Re: Building Yocto on AWS container getting ERROR: The postinstall intercept hook 'update_icon_cache' failed, #yocto

mail2uvijay@...
 


Raj thanks for the response. Couldnt able to find error from the logs..

Attached detailed logs..

?
log.do_rootfs

Re: [poky] Poky:dunfell: SDK install issue (sed: -e expression #1, char 0: no previous regular expression)

Richard Purdie
 

On Tue, 2021-09-07 at 06:10 +0000, Groeting, Claus wrote:
Hi all!

I have a problem with this commit, applied in poky:dunfell on file toolchain-
shar-relocate.sh.
sha1: d6f40be29bf56a835f5825692a22365f04aeb6c3

Author: hongxu <hongxu.jia@...>  2021-08-10 15:33:56
Committer: Richard Purdie <richard.purdie@...>  2021-08-26
09:32:18
Parent: dad6dd12fd6e2512f3bd5591327ee35ee22b913e (toolchain-shar-relocate.sh:
Fix handling files with colons)
Branches: dunfell, remotes/origin/dunfell
Follows: dunfell-23.0.10, yocto-3.1.10
Precedes:

    sdk: fix relocate symlink failed
   
   
    (From OE-Core rev: a155c4d4ca841ef492213a5f0b9121703c62e2b4)
   
    Signed-off-by: Hongxu Jia <hongxu.jia@...>
    Signed-off-by: Alexandre Belloni <alexandre.belloni@...>
    Signed-off-by: Richard Purdie <richard.purdie@...>
    (cherry picked from commit 942c06a7348070b92f722fa5c439c8c4404485b7)
    Signed-off-by: Steve Sakoman <steve@...>
    Signed-off-by: Richard Purdie <richard.purdie@...>

-------------------- meta/files/toolchain-shar-relocate.sh --------------------
index 5433741296..ba873373e2 100644
@@ -72,7 +72,7 @@ fi
 
 # change all symlinks pointing to @SDKPATH@
 for l in $($SUDO_EXEC find $native_sysroot -type l); do
- $SUDO_EXEC ln -sfn $(readlink $l|$SUDO_EXEC sed -e
"s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:") $l
+ $SUDO_EXEC ln -sfn $(readlink $l|$SUDO_EXEC sed -e
"s:$SDK_BUILD_PATH:$target_sdk_dir:") $l
  if [ $? -ne 0 ]; then
  echo "Failed to setup symlinks. Relocate script failed. Abort!"
  exit 1

it looks to me that SDK_BUILD_PATH is not defined in the sed instruction. When
installing the generated SDK shell script, we see a lot of:
sed: -e expression #1, char 0: no previous regular expression
sed: -e expression #1, char 0: no previous regular expression
sed: -e expression #1, char 0: no previous regular expression
sed: -e expression #1, char 0: no previous regular expression
sed: -e expression #1, char 0: no previous regular expression
sed: -e expression #1, char 0: no previous regular expression

Without that commit, everything looks ok. On master-branch SDK_BUILD_PATH is
preset with "@SDKPATH@" but not on dunfell-branch this is missing in the
file toolchain-shar-relocate.sh.
Hi,

I think there is a patch queued in the testing branch for dunfell:

http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=stable/dunfell-nut


http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=stable/dunfell-nut&id=ed0f6236ceb6a127903241f783a526bcdefc66ce

that should address this.

Cheers,

Richard
Re: Inactive service of hello world #yocto

Rudolf J Streif
 

What does your 'Hello World' service do? Just print 'Hello World' to the console and then exit?

If so that would be the reason why it is inactive. Systemd starts it and then it exits. Typically, services keep running (except for some special cases) once they are started. That's what makes them a service. To stop them 'systemctl stop <service>' is used. By default systemctl sends SIGTERM to tell the service to clean up and terminate itself.

:rjs


On Tue, Sep 7, 2021, 10:20 <yasminebenghozzi6@...> wrote:
Hi everyone, 

I wanted to run the helloworld service, but it keeps telling that its inactive , I really cant find why, I changed the service many times but no solution.
I m sorry I ant copy paste from putty
Inactive service of hello world #yocto

yasminebenghozzi6@...
 

Hi everyone, 

I wanted to run the helloworld service, but it keeps telling that its inactive , I really cant find why, I changed the service many times but no solution.
I m sorry I ant copy paste from putty
Re: Debating best tool to use

KL <deco33000@...>
 

That is exactly what I thought and why I insisted that much.

I will step back and constrict to my app, and not the whole OS.

Thanks!

--

17:38, 7 September 2021, Ross Burton <ross@...>:

On Tue, 7 Sept 2021 at 11:22, KL <deco33000@...> wrote:

 Thanks does that mean that I can reshape the whole "everything is a file" under unix?

 I don't want /etc /var /usr (and any other premade structure you find in every linux distro...) but manage it differently via our db.

 You confirm that doable?


Doable? Yes, everything is open source.

Sensible? Not really. I'm not sure you've considered the scale of
what you're asking. For example, /etc/passwd and /etc/hosts are deeply
tied into large amounts of software.

You can make *your* software use whatever configuration database you
want, but you'll have a lot of work to do if you want to move the user
database or DNS resolver information out of /etc.

Ross
Re: Debating best tool to use

Ross Burton <ross@...>
 

On Tue, 7 Sept 2021 at 11:22, KL <deco33000@...> wrote:
Thanks does that mean that I can reshape the whole "everything is a file" under unix?

I don't want /etc /var /usr (and any other premade structure you find in every linux distro...) but manage it differently via our db.

You confirm that doable?
Doable? Yes, everything is open source.

Sensible? Not really. I'm not sure you've considered the scale of
what you're asking. For example, /etc/passwd and /etc/hosts are deeply
tied into large amounts of software.

You can make *your* software use whatever configuration database you
want, but you'll have a lot of work to do if you want to move the user
database or DNS resolver information out of /etc.

Ross
shadow package not installing groupadd commanf in the filesystem #dunfell

cs.raryu <cs.raryu@...>
 

Hi,

The "shadow" package provides the "groupadd" tool. So I added the package to my recipe. Later when I checked in the rootfs, there is no groupadd command added in the filesystem. 

Can anyone please tell me why so?

Thank you.
Yocto Project Status WW36`21

Stephen Jolley
 

Current Dev Position: YP 3.4 M4

Next Deadline: 4th Oct. 2021 YP 3.4 M4 build

 

Next Team Meetings:

 

Key Status/Updates:

  • YP 3.4 M3 is built and now in QA. This means we are now feature complete for 3.4 and will be working on stabilization.
  • YP 3.3.3 has been built and is waiting in the queue for QA after 3.4 M3. This includes the override syntax compatibility changes.
  • SBOM generation work was merged in and will be in the release as preliminary work on this topic, there will likely be some further tweaks during M4.
  • Further rust issues were resolved such as the centos7 build issues.
  • A work around for the glibc 2.34 clone3 syscalls in uninative was added and a new uninative built/released.
  • A fix for some of the multiconfig issues where deadlocked tasks and tasks were showing errors being on both the “covered and not covered lists” is available in master-next for review, hopefully addressing the issues that were being reported. This was possible from a better test case that was submitted enabling debugging.
  • Intermittent issues are ongoing and help is very much welcome on these issues. You can see the list of failures we’re continuing to see by searching for the “AB-INT” tag in bugzilla: https://bugzilla.yoctoproject.org/buglist.cgi?quicksearch=AB-INT

 

Ways to contribute:

 

YP 3.4 Milestone Dates:

  • YP 3.4 M3 built and in QA
  • YP 3.4 M3 Release date 2021/09/03
  • YP 3.4 M4 build date 2021/10/04
  • YP 3.4 M4 Release date 2021/10/29

 

Planned upcoming dot releases:

  • YP 3.3.3 built and in QA
  • YP 3.3.3 release date 2021/10/1
  • YP 3.1.11 build date 2021/09/13
  • YP 3.1.11 release date 2021/9/24

 

Tracking Metrics:

 

The Yocto Project’s technical governance is through its Technical Steering Committee, more information is available at:

https://wiki.yoctoproject.org/wiki/TSC

 

The Status reports are now stored on the wiki at: https://wiki.yoctoproject.org/wiki/Weekly_Status

 

[If anyone has suggestions for other information you’d like to see on this weekly status update, let us know!]

 

Thanks,

 

Stephen K. Jolley

Yocto Project Program Manager

(    Cell:                (208) 244-4460

* Email:              sjolley.yp.pm@...

 
add raspivid to the yocto image for raspberrypi4-64

safouane maaloul <maaloulsafouane@...>
 

Hi, in the first place i succeeded to add raspivid to my image yocto of raspberrypi0-wifi. I did so by adding the userland recipe. But when i try doing so with raspberrypi4-64. Idon't get raspivid when i add the userland recipe to my image. Do you have any idea why it don't work with the raspberrypi4-64?
Thanks,
Safouane
Re: Debating best tool to use

KL <deco33000@...>
 


Thanks does that mean that I can reshape the whole "everything is a file" under unix?

I don't want /etc /var /usr (and any other premade structure you find in every linux distro...) but manage it differently via our db.

You confirm that doable?
--

22:03, 6 September 2021, Khem Raj <raj.khem@...>:

On Mon, Sep 6, 2021 at 9:10 AM KL <deco33000@...> wrote:



 Hi,

 I plan to have a minimal embedded system with BT, wifi, no gui, to host my application with my own webserver.

 My webserver+app are responsible to manage the OS (so I don't want anything else to do it). They receive new programs and load them according to my rules (that is our manager at work).

 I would like to avoid the plague of multiple configuration files put everywhere in files (like now in different profile directories...), but instead have my custom manager handle ALL the different settings for the OS (centralized point).

 We, at the office, compile and maintain the different software needed and send them by wifi.

 I want that OS to be built on top of Linux.

 Would Yocto enable me to do that? Is it the best tool for that?


Yes, it lets you build custom Linux distribution and eases the way to
add needed customization that one might require.


 Thanks,


 --
 MKL


 
4821 - 4840 of 59467