Instead of having a limited set of builders which can deploy artefacts,
let every builder have the ability to deploy. This makes it easier to
experiment with deploy steps.

Signed-off-by: Ross Burton <ross.burton@...>
---
schedulers.py | 11 +++++------
1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/schedulers.py b/schedulers.py
index 86fc6e4..bfd0e60 100644
--- a/schedulers.py
+++ b/schedulers.py
@@ -241,12 +241,11 @@ def props_for_builder(builder):
default=3Dswat_default))
if builder =3D=3D 'build-appliance':
props.append(buildappsrcrev_param())
- if builder in ['build-appliance', 'buildtools', 'eclipse-plugin-neon=
', 'eclipse-plugin-oxygen']:
- props.append(util.BooleanParameter(
- name=3D"deploy_artefacts",
- label=3D"Do we want to deploy artefacts? ",
- default=3DFalse
- ))
+ props.append(util.BooleanParameter(
+ name=3D"deploy_artefacts",
+ label=3D"Do we want to deploy artefacts?",
+ default=3DFalse
+ ))
props =3D props + repos_for_builder(builder)
worker_list =3D config.builder_to_workers.get(builder, config.builde=
r_to_workers['default'])
props.append(util.ChoiceStringParameter(name=3D"worker",
--=20
2.25.1

Signed-off-by: Michael Opdenacker <michael.opdenacker@...>
---
scripts/run-docs-build | 5 +++++
1 file changed, 5 insertions(+)

diff --git a/scripts/run-docs-build b/scripts/run-docs-build
index 3db4a97..4451018 100755
--- a/scripts/run-docs-build
+++ b/scripts/run-docs-build
@@ -15,6 +15,7 @@ mkdir buildtools
$docs_buildtools -y -d $builddir/buildtools
. $builddir/buildtools/environment-setup*

+# Getting the old docbook built docs from an archive. Not rebuilding them.
#wget https://downloads.yoctoproject.org/mirror/docbook-mirror/docbook-archives-20201105.tar.xz
docbookarchive=/srv/autobuilder/autobuilder.yoctoproject.org/pub/docbook-mirror/docbook-archives-20201105.tar.xz
mkdir $outputdir
@@ -35,6 +36,9 @@ mkdir $outputdir/bitbake/next
cp -r ./_build/final/* $outputdir/bitbake/next

# stable branches
+# A decision was made to keep updating all the Sphinx generated docs for the moment,
+# even the ones corresponding to no longer supported releases
+# https://lists.yoctoproject.org/g/docs/message/2193
for branch in 1.46 1.48 1.50 1.52; do
git checkout $branch
make clean
@@ -68,6 +72,7 @@ mkdir $outputdir/next
cp -r ./_build/final/* $outputdir/next

# stable branches
+# Again, keeping even the no longer supported releases (see above comment)
for branch in dunfell gatesgarth hardknott honister; do
cd $ypdocs
git checkout $branch
--
2.25.1

Hi Quentin,

On 11/24/21 7:47 PM, Quentin Schulz wrote:

Hi Michael, Richard,

On Wed, Nov 24, 2021 at 06:10:56PM +0000, Richard Purdie wrote:

On Wed, 2021-11-24 at 18:16 +0100, Michael Opdenacker wrote:

Together with the corresponding Bitbake version, which are no
longer supported.

Signed-off-by: Michael Opdenacker <michael.opdenacker@...>
---
scripts/run-docs-build | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/scripts/run-docs-build b/scripts/run-docs-build
index 3db4a97..5e1d649 100755
--- a/scripts/run-docs-build
+++ b/scripts/run-docs-build
@@ -35,7 +35,7 @@ mkdir $outputdir/bitbake/next
cp -r ./_build/final/* $outputdir/bitbake/next

# stable branches
-for branch in 1.46 1.48 1.50 1.52; do
+for branch in 1.46 1.50 1.52; do
git checkout $branch
make clean
make publish
@@ -68,7 +68,7 @@ mkdir $outputdir/next
cp -r ./_build/final/* $outputdir/next

# stable branches
-for branch in dunfell gatesgarth hardknott honister; do
+for branch in dunfell hardknott honister; do
cd $ypdocs
git checkout $branch
make clean

I'm a bit torn on this. They are no longer officially supported releases now but
it may make sense to rebuild all the sphinx docs in this script rather than some
subset?

I think we want to make sure we have all docs up-to-date, even for the
branches that aren't maintained anymore. Especially since it's not
taking a lot of CPU time to build them, it's fine IMO. We could always
make minor changes to old docs. E.g. the releases.rst might get updates
until we figure something out.

Thanks for casting your vote. It makes sense. I'll send another patch
with this decision in the comments.
Cheers
Michael.

--
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Hi Richard,

Thanks for the review!

On 11/24/21 7:10 PM, Richard Purdie wrote:

On Wed, 2021-11-24 at 18:16 +0100, Michael Opdenacker wrote:

Together with the corresponding Bitbake version, which are no
longer supported.

Signed-off-by: Michael Opdenacker <michael.opdenacker@...>
---
scripts/run-docs-build | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/scripts/run-docs-build b/scripts/run-docs-build
index 3db4a97..5e1d649 100755
--- a/scripts/run-docs-build
+++ b/scripts/run-docs-build
@@ -35,7 +35,7 @@ mkdir $outputdir/bitbake/next
cp -r ./_build/final/* $outputdir/bitbake/next

# stable branches
-for branch in 1.46 1.48 1.50 1.52; do
+for branch in 1.46 1.50 1.52; do
git checkout $branch
make clean
make publish
@@ -68,7 +68,7 @@ mkdir $outputdir/next
cp -r ./_build/final/* $outputdir/next

# stable branches
-for branch in dunfell gatesgarth hardknott honister; do
+for branch in dunfell hardknott honister; do
cd $ypdocs
git checkout $branch
make clean

I'm a bit torn on this. They are no longer officially supported releases now but
it may make sense to rebuild all the sphinx docs in this script rather than some
subset?

I understand. Your decision to make.
I just proposed this change for consistency with the current implementation.

Any other opinions?

Cheers
Michael.

--
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

On Wed, Nov 24, 2021 at 2:18 AM Richard Purdie
<richard.purdie@...> wrote:

On Wed, 2021-11-24 at 09:00 +0100, Alexander Kanavin wrote:

But the AB has not been exhibiting any problems with zstd, and this will
degrade performance. Let's only fix what is broken.

I'm not sure I agree with that.

We have 60+ "intermittent" bugs and some of us are in weekly meetings trying to
do something about working out why these are failing. It feels like we're not
really getting too far with some subset of them and it is using up a lot of the
SWAT and bug triage time.

We've made a few changes to try and reduce the load spikes on the systems and
this fits with the other changes we've made.

From a different data point, we have clipped the parallelism for XZ
and ZSTD internally to very low ( 2 and 4 )
and it has in fact reduced unexpected failures and seen no impact on
build performance. Since parallelism settings
are myopic for these tools, they greedily take every CPU which is not
best for overall build thats doing many
things in parallel, so its best to curtail them to a conservative value.

Cheers,

Richard

On Wed, Nov 24, 2021 at 1:02 AM Bel Hadj Salem Talel <bhstalel@...> wrote:

Hello All,

I created a simple image recipe for initramfs type of image with no IMAGE_FEATURES and simply:

IMAGE_INSTALL = "packagegroup-core-boot busybox"

to get more info, can you try adding

IMAGE_LINGUAS = " "

and see if this changes anything ?

When I bitbake the image I get the following error:

---------------------------
ERROR: menzu-image-initramfs-1.0-r0 do_rootfs: Error executing a python function in exec_python_func() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:do_rootfs(d)
0003:
File: '/home/talel/Desktop/YoctoWork/sources/poky/meta/classes/image.bbclass', lineno: 245, function: do_rootfs
0241: progress_reporter.next_stage()
0242:
0243: # generate rootfs
0244: d.setVarFlag('REPRODUCIBLE_TIMESTAMP_ROOTFS', 'export', '1')
*** 0245: create_rootfs(d, progress_reporter=progress_reporter, logcatcher=logcatcher)
0246:
0247: progress_reporter.finish()
0248:}
0249:do_rootfs[dirs] = "${TOPDIR}"
File: '/home/talel/Desktop/YoctoWork/sources/poky/meta/lib/oe/rootfs.py', lineno: 978, function: create_rootfs
0974: img_type = d.getVar('IMAGE_PKGTYPE')
0975: if img_type == "rpm":
0976: RpmRootfs(d, manifest_dir, progress_reporter, logcatcher).create()
0977: elif img_type == "ipk":
*** 0978: OpkgRootfs(d, manifest_dir, progress_reporter, logcatcher).create()
0979: elif img_type == "deb":
0980: DpkgRootfs(d, manifest_dir, progress_reporter, logcatcher).create()
0981:
0982: os.environ.clear()
File: '/home/talel/Desktop/YoctoWork/sources/poky/meta/lib/oe/rootfs.py', lineno: 204, function: create
0200: if self.progress_reporter:
0201: self.progress_reporter.next_stage()
0202:
0203: # call the package manager dependent create method
*** 0204: self._create()
0205:
0206: sysconfdir = self.image_rootfs + self.d.getVar('sysconfdir')
0207: bb.utils.mkdirhier(sysconfdir)
0208: with open(sysconfdir + "/version", "w+") as ver:
File: '/home/talel/Desktop/YoctoWork/sources/poky/meta/lib/oe/rootfs.py', lineno: 922, function: _create
0918:
0919: if self.progress_reporter:
0920: self.progress_reporter.next_stage()
0921:
*** 0922: self.pm.install_complementary()
0923:
0924: if self.progress_reporter:
0925: self.progress_reporter.next_stage()
0926:
File: '/home/talel/Desktop/YoctoWork/sources/poky/meta/lib/oe/package_manager.py', lineno: 614, function: install_complementary
0610:
0611: target_arch = self.d.getVar('TARGET_ARCH')
0612: localedir = oe.path.join(self.target_rootfs, self.d.getVar("libdir"), "locale")
0613: if os.path.exists(localedir) and os.listdir(localedir):
*** 0614: generate_locale_archive(self.d, self.target_rootfs, target_arch, localedir)
0615: # And now delete the binary locales
0616: self.remove(fnmatch.filter(self.list_installed(), "glibc-binary-localedata-*"), False)
0617:
0618: def deploy_dir_lock(self):
File: '/home/talel/Desktop/YoctoWork/sources/poky/meta/lib/oe/package_manager.py', lineno: 140, function: generate_locale_archive
0136: if os.path.isdir(path):
0137: cmd = ["cross-localedef", "--verbose"]
0138: cmd += arch_options
0139: cmd += ["--add-to-archive", path]
*** 0140: subprocess.check_output(cmd, env=env, stderr=subprocess.STDOUT)
0141:
0142:class Indexer(object, metaclass=ABCMeta):
0143: def __init__(self, d, deploy_dir):
0144: self.d = d
File: '/usr/lib/python3.8/subprocess.py', lineno: 415, function: check_output
0411: else:
0412: empty = b''
0413: kwargs['input'] = empty
0414:
*** 0415: return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
0416: **kwargs).stdout
0417:
0418:
0419:class CompletedProcess(object):
File: '/usr/lib/python3.8/subprocess.py', lineno: 493, function: run
0489: 'with capture_output.')
0490: kwargs['stdout'] = PIPE
0491: kwargs['stderr'] = PIPE
0492:
*** 0493: with Popen(*popenargs, **kwargs) as process:
0494: try:
0495: stdout, stderr = process.communicate(input, timeout=timeout)
0496: except TimeoutExpired as exc:
0497: process.kill()
File: '/usr/lib/python3.8/subprocess.py', lineno: 858, function: __init__
0854: if self.text_mode:
0855: self.stderr = io.TextIOWrapper(self.stderr,
0856: encoding=encoding, errors=errors)
0857:
*** 0858: self._execute_child(args, executable, preexec_fn, close_fds,
0859: pass_fds, cwd, env,
0860: startupinfo, creationflags, shell,
0861: p2cread, p2cwrite,
0862: c2pread, c2pwrite,
File: '/usr/lib/python3.8/subprocess.py', lineno: 1704, function: _execute_child
1700: else:
1701: err_filename = orig_executable
1702: if errno_num != 0:
1703: err_msg = os.strerror(errno_num)
*** 1704: raise child_exception_type(errno_num, err_msg, err_filename)
1705: raise child_exception_type(err_msg)
1706:
1707:
1708: def _handle_exitstatus(self, sts, _WIFSIGNALED=os.WIFSIGNALED,
Exception: FileNotFoundError: [Errno 2] No such file or directory: 'cross-localedef'

ERROR: Logfile of failure stored in: /home/talel/Desktop/YoctoWork/arken/tmp/work/menzu-poky-linux/menzu-image-initramfs/1.0-r0/temp/log.do_rootfs.143822
ERROR: Task (/home/talel/Documents/FinalGit/SelfArkenWork/arken/meta-menzu/recipes-core/images/menzu-image-initramfs.bb:do_rootfs) failed with exit code '1'
---------------------------

I was building the image with success, but now it fails, I don't know why.
The other normal images build successfully.

Thanks,
Talel

On Wed, 2021-11-24 at 18:16 +0100, Michael Opdenacker wrote:

Together with the corresponding Bitbake version, which are no
longer supported.

Signed-off-by: Michael Opdenacker <michael.opdenacker@...>
---
scripts/run-docs-build | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/scripts/run-docs-build b/scripts/run-docs-build
index 3db4a97..5e1d649 100755
--- a/scripts/run-docs-build
+++ b/scripts/run-docs-build
@@ -35,7 +35,7 @@ mkdir $outputdir/bitbake/next
cp -r ./_build/final/* $outputdir/bitbake/next

# stable branches
-for branch in 1.46 1.48 1.50 1.52; do
+for branch in 1.46 1.50 1.52; do
git checkout $branch
make clean
make publish
@@ -68,7 +68,7 @@ mkdir $outputdir/next
cp -r ./_build/final/* $outputdir/next

# stable branches
-for branch in dunfell gatesgarth hardknott honister; do
+for branch in dunfell hardknott honister; do
cd $ypdocs
git checkout $branch
make clean

I'm a bit torn on this. They are no longer officially supported releases now but
it may make sense to rebuild all the sphinx docs in this script rather than some
subset?

Cheers,

Richard

Together with the corresponding Bitbake version, which are no
longer supported.

Signed-off-by: Michael Opdenacker <michael.opdenacker@...>
---
scripts/run-docs-build | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/scripts/run-docs-build b/scripts/run-docs-build
index 3db4a97..5e1d649 100755
--- a/scripts/run-docs-build
+++ b/scripts/run-docs-build
@@ -35,7 +35,7 @@ mkdir $outputdir/bitbake/next
cp -r ./_build/final/* $outputdir/bitbake/next

# stable branches
-for branch in 1.46 1.48 1.50 1.52; do
+for branch in 1.46 1.50 1.52; do
git checkout $branch
make clean
make publish
@@ -68,7 +68,7 @@ mkdir $outputdir/next
cp -r ./_build/final/* $outputdir/next

# stable branches
-for branch in dunfell gatesgarth hardknott honister; do
+for branch in dunfell hardknott honister; do
cd $ypdocs
git checkout $branch
make clean
--
2.25.1

On Thu, Nov 11, 2021 at 03:34 PM, Jeffrey Simons wrote:

Hi Manuel,

Subject: Re: [yocto] User configuration & host contamination

Hi Jeffrey,

Does the recipe which builds the application DEPEND on the recipe which sets up the user? This is what I would try. If I understand
things correctly, Yocto/Bitbake provides every recipe a pristine environment unnaffected from other recipes going into the same image.
For example, if you want to link your application against some libraries provided by other recipes, you need to add them to DEPENDS.
That populates your build environment with that other recipes output. I'm not sure this applies to user accounts as well, but I guess it's worth a try.

Please note I probably used the termins "recipe" and "package" incorrectly.

Hope this helps,
Manuel

Thank you for your reply and suggestion.
I already have a dependency on the user-configuration script, see the below snippet from my recipe.

# Compile-time dependencies for testapp
DEPENDS = "user-configuration"

# Run-time dependencies for testapp
RDEPENDS_${PN} += "rsyslog \

Unfortunately that did not work, I have seen some suggestions on stack-overflow where they added the user multiple times per recipe by using extrauseradd (I believe).
That seems a bit weird to me to add every time the same user, also the drawback is that if the user changes then I have to adjust all recipes that rely on that specific user.
What I did today to circumvent the issue is to assign the user by reference of UID and GID, but I'm not sure if this is the intended Yocto way. As you stated before
Yocto presents you with a pristine environment with all information present, so I would expect that my user is there. Perhaps I did not include the user-configuration
correctly?
I did include the user-configuration by adding it into our distribution description, see the next coding snippet.

#
# Usernames that will be used within the distro.
# Can be changed when desired, each recipe must use this user for the application.
#
TEST_USER = "testuser"
TEST_USER_UID = "1200"

DISTRO_EXTRA_RDEPENDS += "user-configuration"

Can you or any one else clarify if this is the correct way or not?

Thank you in advance.

Jeffrey Simons

Software Engineer
Royal Boon Edam International B.V.

Hi All,

Can anyone elaborate on my fix if this is the correct way, or point me in the correct direction.

With kind regards,
Jeffrey Simons

Software Engineer
Royal Boon Edam International B.V.

On Wed, 2021-11-24 at 09:00 +0100, Alexander Kanavin wrote:

But the AB has not been exhibiting any problems with zstd, and this will
degrade performance. Let's only fix what is broken.

I'm not sure I agree with that. 

We have 60+ "intermittent" bugs and some of us are in weekly meetings trying to
do something about working out why these are failing. It feels like we're not
really getting too far with some subset of them and it is using up a lot of the
SWAT and bug triage time.

We've made a few changes to try and reduce the load spikes on the systems and
this fits with the other changes we've made.

Cheers,

Richard

Hello All,

I created a simple image recipe for initramfs type of image with no IMAGE_FEATURES and simply:

IMAGE_INSTALL = "packagegroup-core-boot busybox"

When I bitbake the image I get the following error:

---------------------------

There was a long thread about this on oe-core recently:
https://lists.openembedded.org/g/openembedded-core/message/157999

//Ernst

My personal build machine has been crashing and dying due to overload
with the default ZSTD_THREADS, and the AB has had problems with xz due
to overload so I was being proactive and ensuring zstd is limited on
the AB the same way xz is.

Ross

On Tue, 2021-11-23 at 11:54 +0000, Teoh, Jay Shen wrote:

This is the full report for yocto-3.1.12.rc1:
https://git.yoctoproject.org/cgit/cgit.cgi/yocto-testresults-contrib/tree/?h=intel-yocto-testresults

======= Summary ========
No high milestone defects.

new issue found

Bug 14622 - bsps-hw.bsps-hw.Test_Seek_bar_and_volume_control manual test case failure

======= Bugs ========
https://bugzilla.yoctoproject.org/show_bug.cgi?id=14622

Thanks,
Jay

Thanks Jay.

The TSC discussed 3.1.12 and are happy for rc1 to be released.

Cheers,

Richard

Signed-off-by: Ross Burton <ross.burton@...>
---
config.json | 1 +
1 file changed, 1 insertion(+)

diff --git a/config.json b/config.json
index 7203783..c0543d9 100644
--- a/config.json
+++ b/config.json
@@ -47,6 +47,7 @@
"PARALLEL_MAKE =3D '-j 16 -l 52'",
"XZ_MEMLIMIT =3D '5%'",
"XZ_THREADS =3D '8'",
+ "ZSTD_THREADS =3D '8'",
"BB_TASK_NICE_LEVEL =3D '5'",
"BB_TASK_NICE_LEVEL:task-testimage =3D '0'",
"BB_TASK_IONICE_LEVEL =3D '2.7'",
--=20
2.25.1

generic-arm64 is the recommeded generic BSP for modern Arm deployments,
so ensure it is exercised in the autobuilder.

Signed-off-by: Ross Burton <ross.burton@...>
---
config.json | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/config.json b/config.json
index f152b47..7203783 100644
--- a/config.json
+++ b/config.json
@@ -330,8 +330,8 @@
"SANITYTARGETS" : "core-image-sato:do_testsdk"
},
"step2": {
- "shortname": "Build for juno",
- "MACHINE": "juno",
+ "shortname": "Build for generic-arm64",
+ "MACHINE": "generic-arm64",
"BBTARGETS": "core-image-minimal core-image-sato core-im=
age-sato:do_populate_sdk",
"SANITYTARGETS" : "core-image-sato:do_testsdk"
}
--=20
2.25.1