[PATCH yocto-autobuilder-helper 3/4] summarize_top_output.py: summarize data logged on failure
sakib.sajal@...
Signed-off-by: Sakib Sajal <sakib.sajal@...>
--- scripts/summarize_top_output.py | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/scripts/summarize_top_output.py b/scripts/summarize_top_output.py index 50c9b0a..4e96ce9 100755 --- a/scripts/summarize_top_output.py +++ b/scripts/summarize_top_output.py @@ -168,9 +168,11 @@ def main(): target = sys.argv[2] host_data_dir = "intermittent_failure_host_data" directory = os.path.join(dest, target, host_data_dir) - for f in glob.glob(directory + "/*_top.txt"): - outputs = list_top_outputs(f) - short_summary, summary, kernel_summary, zombie_summary, other_build = summarize_top(outputs, target) - write_summary(short_summary, summary, kernel_summary, zombie_summary, other_build, target, f) + regs = (directory + "/*_top.txt", directory + "/*_failure_*.txt") + for exts in regs: + for f in glob.glob(exts): + outputs = list_top_outputs(f) + short_summary, summary, kernel_summary, zombie_summary, other_build = summarize_top(outputs, target) + write_summary(short_summary, summary, kernel_summary, zombie_summary, other_build, target, f) main() -- 2.25.1 |
||||||||
|
||||||||
[PATCH yocto-autobuilder-helper 2/4] collect-results: collect all host_stats files
sakib.sajal@...
data collection on failure is enabled by default.
Collect all host stats files created. Signed-off-by: Sakib Sajal <sakib.sajal@...> --- scripts/collect-results | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/scripts/collect-results b/scripts/collect-results index 7178380..93834d6 100755 --- a/scripts/collect-results +++ b/scripts/collect-results @@ -18,21 +18,27 @@ if [ -e $WORKDIR/buildhistory ]; then fi fi -HSFILE=$WORKDIR/tmp/buildstats/*/host_stats +HSFILE=$WORKDIR/tmp/buildstats/*/host_stats* d="intermittent_failure_host_data" mkdir -p $DEST/$target/$d -step=0 +step_i=1 +step_f=1 for f in $HSFILE; do if [ -e $f ]; then cp $f $DEST/$target/$d + if [[ "$f" == *"failure"* ]] ; then + mv $DEST/$target/$d/`basename $f` $DEST/$target/$d/`basename $f`_${step_f}.txt + step_f=$((step_f+1)) + continue + fi grep -m 1 "^top -" $f if [ $? -eq 0 ]; then - mv $DEST/$target/$d/`basename $f` $DEST/$target/$d/`basename $f`_${step}_top.txt + mv $DEST/$target/$d/`basename $f` $DEST/$target/$d/`basename $f`_${step_i}_top.txt else - mv $DEST/$target/$d/`basename $f` $DEST/$target/$d/`basename $f`_${step}.txt + mv $DEST/$target/$d/`basename $f` $DEST/$target/$d/`basename $f`_${step_i}.txt fi - step=$((step+1)) + step_i=$((step_i+1)) fi done -- 2.25.1 |
||||||||
|
||||||||
[PATCH yocto-autobuilder-helper 1/4] config.json: collect data on failure by default
sakib.sajal@...
Signed-off-by: Sakib Sajal <sakib.sajal@...>
--- config.json | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/config.json b/config.json index f54081b..99593c8 100644 --- a/config.json +++ b/config.json @@ -61,7 +61,9 @@ "RUNQEMU_TMPFS_DIR = '/home/pokybuild/tmp'", "BB_HEARTBEAT_EVENT = '60'", "BB_LOG_HOST_STAT_ON_INTERVAL = '1'", - "BB_LOG_HOST_STAT_CMDS_INTERVAL = 'oe-time-dd-test.sh 100'" + "BB_LOG_HOST_STAT_CMDS_INTERVAL = 'oe-time-dd-test.sh -c 100 -t 15'", + "BB_LOG_HOST_STAT_ON_FAILURE = '1'", + "BB_LOG_HOST_STAT_CMDS_FAILURE = 'oe-time-dd-test.sh -l'" ] }, "templates" : { -- 2.25.1 |
||||||||
|
||||||||
Re: Integration of Docker in Read-only Rootfile system through Yocto
Poornesh <poornesh.g@...>
Greetings ! I have added the docker in yocto (zeus) and it is working with the Rootfile system which is having "Read-write" permission. When I trying to run the "Hello-world" image with the Rootfile system having "Read-only" permission through docker I am facing issue . Please find the attached error logs . Requesting your suggestions to overcome this issue. --
Thanks and Regards Poornesh G |
||||||||
|
||||||||
Re: #archiver Archiver not archiving sources from Mirror
#archiver
On Fri, 9 Jul 2021 10:21:56 +0000
"Seipel, Christoph" <Christoph.Seipel@...> wrote: Hello,If the archives you've been given have names starting with 'gitshallow_' then it's likely that the following variables were set in local.conf or in a distro conf file to generate them: BB_GENERATE_MIRROR_TARBALLS = "1" BB_GENERATE_SHALLOW_TARBALLS = "1" BB_GIT_SHALLOW = "1" Could you make sure those are set in your configuration? If they're not then the archiver may not be looking for the file names you have. If things still don't work after that you could try backporting the archiver class from hardknott and using the mirror archiver mode. That mode is written to pick up the mirror tarballs from the downloads directory so it should be able to handle your situation. Thanks, -- Paul Barker https://pbarker.dev/ |
||||||||
|
||||||||
#archiver Archiver not archiving sources from Mirror
#archiver
Seipel, Christoph
Hello,
I have a problem with the archiver. I want to use it to export the sources used in our image for license compliance (GPL etc.). We have a BSP supplier that supplies the Yocto system and the sources to build it in a mirror directory.
Now, when I use the archiver (mode: original) the exported sources don’t contain the sources from the mirror, but only other files like patches.
To analyse this problem I put some extra logging into the archiver.bbclass to inspect the variable contents, and it seems the “local” variable where the archiver is looking for the sources to export doesn’t fit with the path of the actual sources. This leads to the archiver just skipping that URL and proceeding.
For example for the systemd recipe it has the variable URL=”git://github.com/systemd/systemd.git;protocol=git” and therefore it decides to look at local=”<builddir>/downloads/git2/github.com.systemd.systemd.git” But that file doesn’t exist. I inspected the do_fetch and do_unpack logs to find that the fetcher resolves “git://github.com/systemd/systemd.git;protocol=git” with returning the corresponding file from the mirror file:///<mirrorpath>/gitshallow_github.com.systemd.systemd.git_46659f7-1_master.tar.gz The do_unpack task then unpacks tar -xzf <builddir>/downloads/gitshallow_github.com.systemd.systemd.git_46659f7-1_master.tar.gz Which is possible because there is a symlink, I guess that was created by the fetcher <builddir>/downloads/gitshallow_github.com.systemd.systemd.git_46659f7-1_master.tar.gz -> <mirror>/gitshallow_github.com.systemd.systemd.git_46659f7-1_master.tar.gz
So the problem seems to be that the archiver expects the sources in downloads/git2 but the fetcher creates the symlink in downloads and with additional pre- and postfiexes on the filename.
Sadly we are still on sumo, but at least for the archiver class there isn’t any relevant change to hardknott. At least a dunfell upgrade is incoming in the next weeks.
Best regards,
Christoph _____________________________________________________ Christoph Seipel
System & Software Displays, Operator Panels & Mobile Apps
|
||||||||
|
||||||||
Sebastian Holzgreve
Hi, I found the solution: My problem was that the default project configuration which created by "./configure {CONFIGURE_FLAGS}" doesn't do a build with debug information. So ./configure {CONFIGURE_FLAGS} CXXFLAGS="-g
-O0" has to be called before, and then debug is working.
Sorry for the stupid question
before.
Kind regards, Sebastian |
||||||||
|
||||||||
Re: [error-report-web][PATCH] report-error.bbclass: Add layer and bitbake version info to error report
Milan Shah
Thanks, Richard Closed the Bug as Resolved Fixed Status and commit link as a Comment. Thanks & Regards, Milan Shah
On Fri, Jul 9, 2021 at 2:23 PM Richard Purdie <richard.purdie@...> wrote: On Fri, 2021-07-09 at 14:07 +0530, Milan Shah wrote: |
||||||||
|
||||||||
Re: [error-report-web][PATCH] report-error.bbclass: Add layer and bitbake version info to error report
Richard Purdie
On Fri, 2021-07-09 at 14:07 +0530, Milan Shah wrote:
Well, there was no update on the bug as well as on the patch so I wasWe'd normally expect the bug owner to update the status of the bug since it isn't really possible for me or the triage team to track and update them all. We can close this one with a link to the commit: http://git.yoctoproject.org/cgit.cgi/poky/commit/meta/classes/report-error.bbclass?id=8b299a62c55aa8d1583aff4b8428f16def7231b7 Happy for you to, or I can... Cheers, Richard |
||||||||
|
||||||||
Re: [error-report-web][PATCH] report-error.bbclass: Add layer and bitbake version info to error report
Milan Shah
Well, there was no update on the bug as well as on the patch so I was not aware of it. Thanks, Richard for responding. I'll definitely follow your suggestion next time. Thanks & Regards, Milan Shah
On Fri, Jul 9, 2021 at 2:03 PM Richard Purdie <richard.purdie@...> wrote: On Fri, 2021-07-09 at 10:51 +0530, Milan Shah wrote: |
||||||||
|
||||||||
Re: [error-report-web][PATCH] report-error.bbclass: Add layer and bitbake version info to error report
Richard Purdie
On Fri, 2021-07-09 at 10:51 +0530, Milan Shah wrote:
Hello All,http://git.yoctoproject.org/cgit.cgi/poky/commit/meta/classes/report-error.bbclass?id=8b299a62c55aa8d1583aff4b8428f16def7231b7 It appears to have merged 5 months ago? Also, looking at the patch that inherit base should not be there and needs to be removed since everything always inherits base. In the future please send OE-Core changes to the openembedded-core mailing list as that will speed up review and patch handling (as per the repo's README). Cheers, Richard |
||||||||
|
||||||||
Sebastian Holzgreve
Hi,
we were currently upgrading our yocto system to v3.1. In the older yocto system there was the "Yocto Eclipse Plugin" available. I saw in the changelog that unfortunately that this plugin is not maintained anymore and removed. From the past we got whole bunch of autotools based projects. Is it somehow posible to use eclipse for development and debugging keeping the projects in the autotools world? I tried using the eSDK (Following: https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#sdk-working-projects) Rebuilding the project from eclipse is now possible. It still complains that is can run the executable but I can start debugging. I tried to setup the debugger configuration like it was set up, when the "Yocto Eclipse Plugin" generated it. But when I try to debug the code, eclipse opens up the "Debug Perspective", also attaches to the remotly running gdbserver, but it never stops on breakpoints that were set in the eclipse GUI. Is there any chance to use eclipse without the plugin, without converting all the packages to CMake or plain makefile projects? Any hints or pointings to the right direction is appreciated. |
||||||||
|
||||||||
Re: [error-report-web][PATCH] report-error.bbclass: Add layer and bitbake version info to error report
Milan Shah
Hello All, My Patch has not been reviewed for the past 7 months. Looking forward to see some updates on this. Please find related links below. Thanks & Regards, Milan Shah
|
||||||||
|
||||||||
Re: binman support in u-boot
Tom Rini
On Thu, Jul 08, 2021 at 04:10:32AM +0000, Rebecca Chang Swee Fun wrote:
Hi,Off-hand, I suspect it might be a little tricky to use binman outside of the U-Boot context directly. Simon, any thoughts? -- Tom |
||||||||
|
||||||||
valgrind failed do_compile during bitbake
Michael
I'm using Yocto "warrior" version. |
||||||||
|
||||||||
Installing a .deb produced by npm during build
Matt Bernhard
Hi, I've been struggling to add a debian package to my build. I'm trying to build this app and install it on the target: https://github.com/votingworks/kiosk-browser It's a node app built with yarn, and I've found the instructions in the manual to be a bit off the mark for what I'm trying to do. I want to:
My recipe is below, but as best I can tell all it does right now is package the source directory and put it on the image. DPKG reports that it's installed, but I can't seem to run the app. I'm sure I'm confused a bit about the build process, and perhaps I'm not invoking a task when I should? Any pointers would be greatly appreciated. All the best, Matt P.S. Here's the recipe: # Recipe created by recipetool # This is the basis of a recipe and may need further editing in order to be fully functional. # (Feel free to remove these comments when editing.) SUMMARY = "Generic kiosk-mode browser." # WARNING ommitted for brevity ... LICENSE = "Unknown & MIT & ISC" LIC_FILES_CHKSUM = "file://LICENSE;md5=e49f4652534af377a713df3d9dec60cb \ EOF |
||||||||
|
||||||||
Yocto Autobuilder: Latency Monitor and AB-INT - Meeting notes: July 8, 2021
YP AB Intermittent failures meeting
=================================== July 1, 2021, 9 AM ET https://windriver.zoom.us/j/3696693975 Attendees: Tony, Richard, Trevor, Randy Summary: ======== The autobuilder RCU hang is still fixed ;-), master branch builds greener. ptest failures are the top problem now. Add Michael Halstead, see questions below in section 4. If anyone wants to help, we could use more eyes on the logs, particularly the summary logs and understanding iostat # when the dd test times out. Plans for the week: =================== Richard: glibc upgrade, etc. Alex: ? Sakib: pub/non-release link upgrade, script clean-up. Trevor: make job server test. Try it on YP AB!!! What type of build? Tony: fix/work-around valgrind ptest bug: none/tests/amd64/fb_test_amd64 Saul: nothing this week for YP. Randy: vacation, then email catch-up! Meeting Notes: ============== 1, runqemu Tony having trouble with runqemu on some Wind River machine. Richard has a fix for a race in runqemu in master-next. These might be related but if not Tony should debug the issue/ collect logs. 2. job server - Trevor has conclusive evidence that the 'make' job server is useful. email summary to come. Need to fix some assumptoins in code that parses PARALLEL_MAKE then send patch to yocto-autobuilder-helper. - ninja will have to be done next. 3. AB status generally better but... ptests are having some recurring problems. - parted - only on arm? - valgrind - none/tests/amd64/fb_test_amd64 - gdb test failing again. - Randy! 4. Richard reported - something really flaky going on with serial ports. - particularly bad on qemuppc but also x86. - related to Saul's QMP data dump? 5. Sakib needs to send patch to make testimage failures generate summary logs. 6. Richard says that we may need to redesign the data collection system that Sakib's AB INT tests are based on. Still relevant parts of Previous Meeting Notes: ======================= 1. The qemu RCU hang has been fixed to not deadlock anymore! It still hangs at times but this dramatically reduces the AB failures. 4. bitbake server timeout. "Timeout while waiting for a reply from the bitbake server (60s)" Randy mentioned that the bitbake server timeouts seen in the Wind River build cluster have gone away after upgrading to a newer version of docker. Old: Docker Version: Docker version 18.09.4, build d14af54266 New: Docker Version: Docker version 20.10.7, build f0df350 Clearly the YP ABs aren't running in docker but what about firmware and kernel tunings. Michael, Is the BIOS/firmware kept up to date on most nodes? It seems that we are running stock kernels which makes sense but given that we don't have concerns about privacy since system access is controlled and the nodes are being used to test open source software, we might consider optimizing for performance rather than security. Alex pointed at: https://make-linux-fast-again.com/ Which just lists a set of kernel boot options: noibrs noibpb nopti nospectre_v2 nospectre_v1 \ l1tf=off nospec_store_bypass_disable no_stf_barrier \ mds=off tsx=on tsx_async_abort=off mitigations=off Can we enable some or all of these on a node to see what the performance difference is? 5. io stalls Richard said that it would make sense to write an ftrace utility / script to monitor io latency and we could install it with sudo Ch^W mentioned ftrace on IRC. Sakib and Randy will work on that but not for a week or two. 6. Switch the pub/non-release links from full log to summary. The host data links on: https://autobuilder.yocto.io/pub/non-release/ should include links to the summary data. I think we have room to include both links like this: 0 1 2 3 (Full: 0 1 2 3 ) ../Randy |
||||||||
|
||||||||
Re: Question regarding custom device tree update
Sohil Shah <sohils@...>
Hi Robert, Thank you for your reply. This works and I got my dtb generated in my build directory! Best wishes, Sohil On Wed, Jul 7, 2021 at 10:29 PM Robert Calhoun <rcalhoun@...> wrote: > From: yocto@... <yocto@...> on behalf of Sohil Shah <sohils@...> |
||||||||
|
||||||||
Re: [meta-mingw][PATCH] openssl: support for building nativesdk of mingw
Changqing Li
On 7/7/21 1:23 AM, Joshua Watt wrote:
Thanks. I will check the oe-core commit.
|
||||||||
|
||||||||
[meta-raspberrypi][PATCH] userland: add dtc dependency
massimo toscanelli
When using dtparam, dtoverlay converts /proc/device-tree to a .dtb and
loads it. However, if dtc is not installed, this operation cannot be done and every call to dtparam fails. Therefore, dtc needs to be added to the list of userland dependencies in order to properly call dtparam. Signed-off-by: massimo toscanelli <massimo.toscanelli@...> --- recipes-graphics/userland/userland_git.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/recipes-graphics/userland/userland_git.bb b/recipes-graphics/userland/userland_git.bb index ead81f0..9ec57c9 100644 --- a/recipes-graphics/userland/userland_git.bb +++ b/recipes-graphics/userland/userland_git.bb @@ -104,5 +104,5 @@ FILES_${PN}-dev += "${includedir} \ FILES_${PN}-doc += "${datadir}/install" FILES_${PN}-dbg += "${libdir}/plugins/.debug" -RDEPENDS_${PN} += "bash" +RDEPENDS_${PN} += "bash dtc" RDEPENDS_${PN} += "${@bb.utils.contains("MACHINE_FEATURES", "vc4graphics", "libegl-mesa", "", d)}" -- 2.17.1 |
||||||||
|