having a really funky issue everything builds fine under pyro updated it all to dunfell, and now im hitting this failure below, i also pasted the log and the offending recipe on pastebin logfile: https://pastebin.com/38TzNa5h bb recipe: https://pastebin.com/jhfTBv58 Logfile of failure stored in: /home/dingo/TEST3/Zipato/zipato-yocto-public/build/tmp/work/zipabox2-poky-linux-gnueabi/zipabox2-image-blank/1.0-r0/temp/log.do_image_zipabox2_sdimg.27223 Log data follows: | DEBUG: Executing python function set_image_size | DEBUG: 1032444.400000 = 794188 * 1.300000 | DEBUG: 1032444.400000 = max(1032444.400000, 8192)[1032444.400000] + 0 | DEBUG: 1032445.000000 = int(1032444.400000) | DEBUG: 1032445 = aligned(1032445) | DEBUG: returning 1032445 | DEBUG: Python function set_image_size finished | DEBUG: Executing shell function do_image_zipabox2_sdimg | 0+0 records in | 0+0 records out | 0 bytes copied, 8.9773e-05 s, 0.0 kB/s | /home/dingo/TEST3/Zipato/zipato-yocto-public/build/tmp/work/zipabox2-poky-linux-gnueabi/zipabox2-image-blank/1.0-r0/temp/run.do_image_zipabox2_sdimg.27223: 124: /home/dingo/TEST3/Zipato/zipato-yocto-public/build/tmp/work/zipabox2-poky-linux-gnueabi/zipabox2-image-blank/1.0-r0/temp/run.do_image_zipabox2_sdimg.27223: parted: not found | WARNING: exit code 127 from a shell command. | ERROR: Execution of '/home/dingo/TEST3/Zipato/zipato-yocto-public/build/tmp/work/zipabox2-poky-linux-gnueabi/zipabox2-image-blank/1.0-r0/temp/run.do_image_zipabox2_sdimg.27223' failed with exit code 127: | 0+0 records in | 0+0 records out | 0 bytes copied, 8.9773e-05 s, 0.0 kB/s | /home/dingo/TEST3/Zipato/zipato-yocto-public/build/tmp/work/zipabox2-poky-linux-gnueabi/zipabox2-image-blank/1.0-r0/temp/run.do_image_zipabox2_sdimg.27223: 124: /home/dingo/TEST3/Zipato/zipato-yocto-public/build/tmp/work/zipabox2-poky-linux-gnueabi/zipabox2-image-blank/1.0-r0/temp/run.do_image_zipabox2_sdimg.27223: parted: not found | WARNING: exit code 127 from a shell command. | ERROR: Task (/home/dingo/TEST3/Zipato/zipato-yocto-public/meta-zipato-public/images/zipabox2-image/zipabox2-image-blank_1.0.bb:do_image_zipabox2_sdimg) failed with exit code '1' NOTE: Tasks Summary: Attempted 4542 tasks of which 4540 didn't need to be rerun and 2 failed. Summary: 2 tasks failed: /home/dingo/TEST3/Zipato/zipato-yocto-public/meta-zipato-public/images/zipabox2-image/zipabox2-image-blank_1.0.bb:do_image_zipabox2_mender /home/dingo/TEST3/Zipato/zipato-yocto-public/meta-zipato-public/images/zipabox2-image/zipabox2-image-blank_1.0.bb:do_image_zipabox2_sdimg -- 
Sent with Vivaldi Mail. Download Vivaldi for free at vivaldi.com

Hello,

is anyone here using Zuul along with Yocto Project? We are thinking about integrating Zuul to our project that already uses Yocto, but we are not too familiar with Zuul and we want it to conform to good open source practices, without losing any advantages of both projects and we want to avoid creating some hacky monster.

Could you share some suggestions on how it can be combined together and what is the best approach/structure to do this? Please also share if you have some "what I wish I'd known" about Zuul integration to Yocto. Any hints are appreciated.

I haven't, but it's on my radar to look at as it seems like it could be pretty useful. If you look into it, please share your findings!

Best regards,

Tomasz Dziendzielski

YP AB Intermittent failures meeting
===================================
July 1, 2021, 9 AM ET
https://windriver.zoom.us/j/3696693975

Attendees: Alex, Richard, Saul, Randy


Summary:
========

The autobuilder RCU hang is fixed, build greener.
ptest failures (some glibc-2.34) are the top problem now.

Add Michael Halstead, see questions below in section 4.

If anyone wants to help, we could use more eyes on the logs,
particularly the summary logs and understanding iostat #
when the dd test times out.


Plans for the week:

===================


Richard: glibc upgrade, etc.

Alex: ?

Sakib: pub/non-release link upgrade, script clean-up.

Trevor: make job server test. Try it on YP AB!!! What type of build?

Tony: nothing this week for YP.

Saul: nothing this week for YP.

Randy: vacation!


Meeting Notes:
==============

1. The qemu RCU hang has been fixed to not deadlock anymore!

It still hangs at times but this dramatically reduces the
AB failures.


2. Lots of ptest failures with
- tcl
- valgrind
especially with the upreved glibc-2.34.

3. YP AB INT # bugs reduced: down to ~ 47
Richard went through the list and closed ~ 10 bugs.
- RCU, LTP bugs closed.
- qemu ping bug remains
https://bugzilla.yoctoproject.org/show_bug.cgi?id=14029

4. bitbake server timeout.

"Timeout while waiting for a reply from the bitbake server (60s)"

Randy mentioned that the bitbake server timeouts seen in the
Wind River build cluster have gone away after upgrading to
a newer version of docker.

Old: Docker Version: Docker version 18.09.4, build d14af54266
New: Docker Version: Docker version 20.10.7, build f0df350


Clearly the YP ABs aren't running in docker but what
about firmware and kernel tunings.

Michael,

Is the BIOS/firmware kept up to date on most nodes?

It seems that we are running stock kernels which makes sense but
given that we don't have concerns about privacy since system access
is controlled and the nodes are being used to test open
source software, we might consider optimizing for performance
rather than security.

Alex pointed at: https://make-linux-fast-again.com/
Which just lists a set of kernel boot options:
noibrs noibpb nopti nospectre_v2 nospectre_v1 \
l1tf=off nospec_store_bypass_disable no_stf_barrier \
mds=off tsx=on tsx_async_abort=off mitigations=off

Can we enable some or all of these on a node to see what the
performance difference is?


5. io stalls

Richard said that it would make sense to write an ftrace utility
/ script to monitor io latency and we could install it with sudo
Ch^W mentioned ftrace on IRC.
Sakib and Randy will work on that but not for a week or two.

6. Switch the pub/non-release links from full log to summary.
The host data links on:
https://autobuilder.yocto.io/pub/non-release/
should include links to the summary data. I think we have room to
include both links like this:
0 1 2 3 (Full: 0 1 2 3 )

7. Qemu machine protocol debugging commands

Saul's patch is in.


8. System load: make, ninja jobs

Trevor has been busy with other work but has been playing with
kea and make -l: load average calculation
https://github.com/mirror/make/blob/master/src/job.c#L1947


We're confirmed that this actually works on a build server and
for all versions of make in the cluster.

Randy may write to Paul Smith about the general problem we
are having.


9. iostat

The iostat output is in some of the AB logs:
https://autobuilder.yocto.io/pub/non-release/
for example:

https://autobuilder.yocto.io/pub/non-release/20210623-17/testresults/meta-oe/2021-06-23--21-51/host_stats_0_top.txt

search for "start: iostat"
it looks like the io sub-systems are 100% utilized but
we need more time, data and the summary script to be able to easily
make a general statement about AB INT issues and IO load and
to be able to identify what is typically generating the IO load.




../Randy

Merged. Thanks.

Signed-off-by: Chen Qi <Qi.Chen@...>
---
meta-cgl-common/conf/layer.conf | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta-cgl-common/conf/layer.conf b/meta-cgl-common/conf/layer.conf
index 8100e23..56ddbb9 100644
--- a/meta-cgl-common/conf/layer.conf
+++ b/meta-cgl-common/conf/layer.conf
@@ -11,6 +11,6 @@ BBFILE_PRIORITY_cgl-common = "7"

LAYERDEPENDS_cgl-common = "core openembedded-layer networking-layer perl-layer filesystems-layer security selinux"

-LAYERSERIES_COMPAT_cgl-common = "warrior zeus dunfell gatesgarth"
+LAYERSERIES_COMPAT_cgl-common = "warrior zeus dunfell gatesgarth hardknott"

require conf/distro/include/cgl_common_security_flags.inc
--
2.17.1

On Mon 2021-06-28 @ 11:15:41 AM, Trevor Woerner wrote:

By exporting a couple more variables the wks file for every rockchip device
can be built from one template instead of having separate wks files for each
board and platform.

The following BSP variables were checked before and after this change to make
sure they remained valid/sensible:
- WKS_FILE
- UBOOT_SUFFIX
- SPL_BINARY
- IMAGE_FSTYPES

Built-tested for every MACHINE in this BSP.

Run-tested on the following devices to ensure they continue to boot correctly
to a cmdline (core-image-base):
- tinker-board
- rock-pi-e
- rock-pi-4b
- rock64
- nanopi-m4-2gb

Signed-off-by: Trevor Woerner <twoerner@...>
---
conf/machine/firefly-rk3288.conf | 2 --
conf/machine/include/nanopi-m4.inc | 1 -
conf/machine/include/rk3288.inc | 3 +--
conf/machine/include/rk3328.inc | 1 -
conf/machine/include/rk3399.inc | 2 --
conf/machine/include/rock-pi-4.inc | 1 -
conf/machine/include/rockchip-wic.inc | 5 +++++
conf/machine/include/tinker.inc | 2 --
conf/machine/rock-pi-e.conf | 2 --
conf/machine/rock64.conf | 2 --
conf/machine/vyasa-rk3288.conf | 1 -
wic/firefly-rk3288.wks | 7 -------
wic/rk3288-boot.wks | 24 ------------------------
wic/rk3399-boot.wks | 24 ------------------------
wic/rock-pi-4.wks | 7 -------
wic/rock-pi-e.wks | 4 ----
wic/{rk3328-boot.wks => rockchip.wks} | 9 ++++++---
wic/tinker-board.wks | 8 --------
wic/vyasa-rk3288.wks | 8 --------
19 files changed, 12 insertions(+), 101 deletions(-)
delete mode 100644 wic/firefly-rk3288.wks
delete mode 100644 wic/rk3288-boot.wks
delete mode 100644 wic/rk3399-boot.wks
delete mode 100644 wic/rock-pi-4.wks
delete mode 100644 wic/rock-pi-e.wks
rename wic/{rk3328-boot.wks => rockchip.wks} (64%)
delete mode 100644 wic/tinker-board.wks
delete mode 100644 wic/vyasa-rk3288.wks

diff --git a/conf/machine/firefly-rk3288.conf b/conf/machine/firefly-rk3288.conf
index 138e840..3270bb9 100644

Applied to meta-rockchip master.

On Mon 2021-06-28 @ 11:15:40 AM, Trevor Woerner wrote:

The ext4 IMAGE_FSTYPES does not need to be mentioned explicitly. It will be
automatically generated in cases where it is needed.

Signed-off-by: Trevor Woerner <twoerner@...>
---
conf/machine/include/rockchip-defaults.inc | 1 -
1 file changed, 1 deletion(-)

diff --git a/conf/machine/include/rockchip-defaults.inc b/conf/machine/include/rockchip-defaults.inc
index b41c523..0666119 100644

Applied to meta-rockchip master.

On Mon 2021-06-28 @ 11:15:39 AM, Trevor Woerner wrote:

Specify the full include path to the rk3399.inc file.

Signed-off-by: Trevor Woerner <twoerner@...>
---
conf/machine/include/nanopi-m4.inc | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/conf/machine/include/nanopi-m4.inc b/conf/machine/include/nanopi-m4.inc
index 7ca91db..b5251a1 100644

Applied to meta-rockchip master.

Hi Aashik,

On Wed, Jun 30, 2021 at 4:25 PM Aashik Aswin <thisisaash9698@...> wrote:

Hello Developers,

We are migrating our platforms from Zeus to Gatesgarth, We could see that the native Python2 bb file and core modules have been removed.

As much of our platform code is in Python2, I was wondering if there is some way we can re-enable python2 support in our yocto environment to compile/run python2 scripts ?

You can use meta-python2 [0] layer

Thanks in Advance,

Thanks & Regards
Aashik

[0] - https://git.openembedded.org/meta-python2

marek

On Wed, 2021-06-30 at 10:41 +0000, Cardaillac, Yann wrote:

Hi Richard,

Many thanks for the fast answer.

I’m switching from buildroot to yocto and trying to figure out how to
do the equivalent of a POST_BUILD_SCRIPT.

Indeed I want to format the images built in a specific format for latter use and CI needs.

In OE, the equivalent for that is probably the image types code.
See http://git.yoctoproject.org/cgit.cgi/poky/tree/meta/classes/image_types.bbclass

You can define your own custom image types to extend the system. Some of our defaults are quite simple (tar), some are quite complex (wic has its own bbclass).

I said image, but it was probably not the best word, I just want to make a tarball 
of the generated image + adding some file in it.

Here's the complete file with the job :

FILESEXTRAPATHS_prepend := "${THISDIR}/files:"

SRC_URI += "file://nallino-custom-params.txt"
SRC_URI += "file://configuration_instruction_to_do_on_GUF.txt"

do_release_ycn() {

    rm -rf ${DEPLOY_DIR_IMAGE}/ycn_release
    mkdir -p ${DEPLOY_DIR_IMAGE}/ycn_release/overlays
    cp ${DEPLOY_DIR_IMAGE}/fng-install.sh ${DEPLOY_DIR_IMAGE}/ycn_release
    cp ${DEPLOY_DIR_IMAGE}/ycn-image-imx6ullguf.tar.gz ${DEPLOY_DIR_IMAGE}/ycn_release/guf-image-imx6ullguf.tar.gz
    cp ${DEPLOY_DIR_IMAGE}/*.dtbo ${DEPLOY_DIR_IMAGE}/ycn_release/overlays
    cp ${WORKDIR}/nallino-custom-params.txt ${DEPLOY_DIR_IMAGE}/ycn_release/
    cp ${WORKDIR}/configuration_instruction_to_do_on_GUF.txt ${DEPLOY_DIR_IMAGE}/ycn_release/
    tar -czf ${DEPLOY_DIR_IMAGE}/ycn_release.tar.gz ${DEPLOY_DIR_IMAGE}/ycn_release

}
addtask release_ycn after do_image_complete after do_deploy_kernelmodules

You're used to poking around things in buildroot, OE doesn't quite like you
doing that so much :)

You shouldn't really be poking around DEPLOY_DIR without letting the build
system know what you're doing. Intermediate files like the intermediate directory
you're using to create the tarball above should really be in WORKDIR.
That is itself won't break, it just not good practise. Ideally you'd write
a separate recipe to do this final piece with a do_deploy task, using the
deploy bbclass which would them take care of removing obsolete versions
when you rebuild and so on.

I’ve also tried to use IMAGE_POSTPROCESS_COMMAND without much success
since it wasn’t running at the proper moment.

When is "the right moment"? :)

The right moment, is at the very end of bitbake, indeed I want the job 
to be the overall last job!

You're not really helping us help you find a solution since you've not described
what you'd expect to be happening that isn't happening. How is what you have
failing?

I'm going to take a leap and guess that some of the files you expect there
aren't? Maybe the dtbo files from some other do_deploy tasks? If that is true
you're missing dependencies for your task. You can do things like:

do_release_ycn[depends] += "virtual/kernel:do_deploy"

I'd caution against "last" as it is a fine concept until two things want to 
be "last". It is usually better to describe the dependencies properly.

Cheers,

Richard

Hi Richard,

Many thanks for the fast answer.

I’m switching from buildroot to yocto and trying to figure out how to
do the equivalent of a POST_BUILD_SCRIPT.

Indeed I want to format the images built in a specific format for latter use and CI needs.

In OE, the equivalent for that is probably the image types code.
See http://git.yoctoproject.org/cgit.cgi/poky/tree/meta/classes/image_types.bbclass

You can define your own custom image types to extend the system. Some of our defaults are quite simple (tar), some are quite complex (wic has its own bbclass).

I said image, but it was probably not the best word, I just want to make a tarball of the generated image + adding some file in it.

Here's the complete file with the job :

FILESEXTRAPATHS_prepend := "${THISDIR}/files:"

SRC_URI += "file://nallino-custom-params.txt"
SRC_URI += "file://configuration_instruction_to_do_on_GUF.txt"

do_release_ycn() {

rm -rf ${DEPLOY_DIR_IMAGE}/ycn_release
mkdir -p ${DEPLOY_DIR_IMAGE}/ycn_release/overlays
cp ${DEPLOY_DIR_IMAGE}/fng-install.sh ${DEPLOY_DIR_IMAGE}/ycn_release
cp ${DEPLOY_DIR_IMAGE}/ycn-image-imx6ullguf.tar.gz ${DEPLOY_DIR_IMAGE}/ycn_release/guf-image-imx6ullguf.tar.gz
cp ${DEPLOY_DIR_IMAGE}/*.dtbo ${DEPLOY_DIR_IMAGE}/ycn_release/overlays
cp ${WORKDIR}/nallino-custom-params.txt ${DEPLOY_DIR_IMAGE}/ycn_release/
cp ${WORKDIR}/configuration_instruction_to_do_on_GUF.txt ${DEPLOY_DIR_IMAGE}/ycn_release/
tar -czf ${DEPLOY_DIR_IMAGE}/ycn_release.tar.gz ${DEPLOY_DIR_IMAGE}/ycn_release

}
addtask release_ycn after do_image_complete after do_deploy_kernelmodules

I’ve also tried to use IMAGE_POSTPROCESS_COMMAND without much success
since it wasn’t running at the proper moment.

When is "the right moment"? :)

The right moment, is at the very end of bitbake, indeed I want the job to be the overall last job!

Thanks again,

Yann

On Wed, 2021-06-30 at 09:33 +0000, Cardaillac, Yann wrote:

I’m switching from buildroot to yocto and trying to figure out how to do the equivalent of a
POST_BUILD_SCRIPT.
 
Indeed I want to format the images built in a specific format for latter use and CI needs.

In OE, the equivalent for that is probably the image types code.
See http://git.yoctoproject.org/cgit.cgi/poky/tree/meta/classes/image_types.bbclass

You can define your own custom image types to extend the system. Some of our defaults
are quite simple (tar), some are quite complex (wic has its own bbclass).

I’ve also tried to use IMAGE_POSTPROCESS_COMMAND without much success since it 
wasn’t running at the proper moment.

When is "the right moment"? :)

Cheers,

Richard

Hi Guys,

             I am building core-image-sato image. i am facing issue rootfs creation. can anybody know give suggition how we can solve this issue.


--
Thanks & Regards,
Sateesh