|
Re: [ptest-runner][PATCH 3/3] utils.c: add system data collection when a test gets stuck.
Richard Purdie
On Thu, 2021-09-16 at 14:46 +0200, Alexander Kanavin wrote:
Currently, ptest-runner simply kills the offending test without further ado,It is great to see this. I'd suggest dmesg in here since we've seen components of tests segfault before (e.g. lttng-relayd in lttng-tools). Cheers, Richard |
|
|
|
Re: Minutes: Yocto Project Weekly Triage Meeting 9/16/2021
Trevor Gamblin
On 2021-09-16 11:33 a.m., Trevor
Gamblin wrote:
Correction: AB-INT bug count is at 52. |
|
|
|
Minutes: Yocto Project Weekly Triage Meeting 9/16/2021
Trevor Gamblin
Wiki: https://wiki.yoctoproject.org/wiki/Bug_Triage Attendees: Alex, Diane, Jon, Joshua, Michael, Richard,
Ross, Saul, Stephen, Steve, Tim, Trevor ARs: - Trevor to move old AB defects to M4 after call - Trevor to ping WR folks about moving Old Milestone bugs - Richard to add a comment to 7298 Notes:
- (carried over) Steve
encountered build failures such as the one in https://errors.yoctoproject.org/Errors/Details/593109/
when attempting to run dunfell builds with the PARALLEL_MAKE load
averaging added. WR is testing/investigating on internal
Autobuilder instance - Trevor is still planning on looking into
this!
Medium+ 3.4 Unassigned Enhancements/Bugs: 68 (Last week 77) Medium+ 3.5 Unassigned Enhancements/Bugs: 10 (new) AB-INT Bugs: 49
(Last week 48)
|
|
|
|
[ptest-runner][PATCH 3/3] utils.c: add system data collection when a test gets stuck.
Alexander Kanavin
Currently, ptest-runner simply kills the offending test without further ado,
which is not at all helpful when trying to figure out why it happens (especially if such hangs are intermittent and rare). There's now a script that gets executed before killing the test, so ideas on what to have in it are welcome. Signed-off-by: Alexander Kanavin <alex@...> --- Makefile | 2 +- ptest-runner-collect-system-data | 5 +++++ utils.c | 24 ++++++++++++++++++++++++ 3 files changed, 30 insertions(+), 1 deletion(-) create mode 100755 ptest-runner-collect-system-data diff --git a/Makefile b/Makefile index a6372de..168cf5a 100644 --- a/Makefile +++ b/Makefile @@ -43,7 +43,7 @@ $(TEST_EXECUTABLE): $(TEST_OBJECTS) $(CC) $(LDFLAGS) $(TEST_OBJECTS) -o $@ $(TEST_LIBSTATIC) $(TEST_LDFLAGS) check: $(TEST_EXECUTABLE) - ./$(TEST_EXECUTABLE) -d $(TEST_DATA) + PATH=.:$(PATH) ./$(TEST_EXECUTABLE) -d $(TEST_DATA) .c.o: $(CC) $(CFLAGS) -c $< -o $@ diff --git a/ptest-runner-collect-system-data b/ptest-runner-collect-system-data new file mode 100755 index 0000000..5bfeaf3 --- /dev/null +++ b/ptest-runner-collect-system-data @@ -0,0 +1,5 @@ +#!/bin/sh +# Other ideas on what to do when a ptest gets stuck welcome. +pstree -a -l +df +free diff --git a/utils.c b/utils.c index 58c3aa1..a67ac11 100644 --- a/utils.c +++ b/utils.c @@ -281,6 +281,27 @@ close_fds(void) } } +static void +collect_system_state(FILE* fout) +{ + char *cmd = "ptest-runner-collect-system-data"; + + char buf[1024]; + FILE *fp; + + if ((fp = popen(cmd, "r")) == NULL) { + fprintf(fout, "Error opening pipe!\n"); + } + + while (fgets(buf, 1024, fp) != NULL) { + fprintf(fout, "%s", buf); + } + + if(pclose(fp)) { + fprintf(fout, "Command not found or exited with error status\n"); + } +} + static void * read_child(void *arg) { @@ -313,6 +334,9 @@ read_child(void *arg) } } else if (r == 0) { + // no output from the test after a timeout; the test is stuck, so collect + // as much data from the system as possible and kill the test + collect_system_state(_child_reader.fps[0]); _child_reader.timeouted = 1; kill(-_child_reader.pid, SIGKILL); } -- 2.33.0 |
|
|
|
[ptest-runner][PATCH 2/3] utils.c: handle test timeouts directly with poll()
Alexander Kanavin
if poll()'s timeout expires that means the test did not
produce any output, which is exactly what we need to catch. So there's no need to set up separate timeouts with signals and alarms, and this greatly simplifies more sophisticated processing of hanging tests (such as collecting overall system data). Signed-off-by: Alexander Kanavin <alex@...> --- utils.c | 34 ++++++++++------------------------ 1 file changed, 10 insertions(+), 24 deletions(-) diff --git a/utils.c b/utils.c index 128ff61..58c3aa1 100644 --- a/utils.c +++ b/utils.c @@ -51,7 +51,6 @@ #include "utils.h" #define GET_STIME_BUF_SIZE 1024 -#define WAIT_CHILD_POLL_TIMEOUT_MS 200 #define WAIT_CHILD_BUF_MAX_SIZE 1024 #define UNUSED(x) (void)(x) @@ -296,7 +295,7 @@ read_child(void *arg) pfds[1].events = POLLIN; do { - r = poll(pfds, 2, WAIT_CHILD_POLL_TIMEOUT_MS); + r = poll(pfds, 2, _child_reader.timeout*1000); if (r > 0) { char buf[WAIT_CHILD_BUF_MAX_SIZE]; ssize_t n; @@ -313,10 +312,10 @@ read_child(void *arg) fwrite(buf, (size_t)n, 1, _child_reader.fps[1]); } - /* Child output reset alarm */ - alarm(0); - alarm(_child_reader.timeout); - } + } else if (r == 0) { + _child_reader.timeouted = 1; + kill(-_child_reader.pid, SIGKILL); + } fflush(_child_reader.fps[0]); fflush(_child_reader.fps[1]); @@ -344,26 +343,11 @@ run_child(char *run_ptest, int fd_stdout, int fd_stderr) /* exit(1); not needed? */ } -static void -timeout_child_handler(int signo) -{ - UNUSED(signo); - _child_reader.timeouted = 1; - kill(-_child_reader.pid, SIGKILL); -} - static inline int -wait_child(pid_t pid, unsigned int timeout) +wait_child(pid_t pid) { int status = -1; - _child_reader.timeout = timeout; - _child_reader.timeouted = 0; - _child_reader.pid = pid; - - /* setup alarm to timeout based on std{out,err} in the child */ - alarm(timeout); - waitpid(pid, &status, 0); if (WIFEXITED(status)) status = WEXITSTATUS(status); @@ -462,6 +446,8 @@ run_ptests(struct ptest_list *head, const struct ptest_options opts, _child_reader.fds[1] = pipefd_stderr[0]; _child_reader.fps[0] = fp; _child_reader.fps[1] = fp_stderr; + _child_reader.timeout = opts.timeout; + _child_reader.timeouted = 0; rc = pthread_create(&tid, NULL, read_child, NULL); if (rc != 0) { fprintf(fp, "ERROR: Failed to create reader thread, %s\n", strerror(errno)); @@ -469,7 +455,6 @@ run_ptests(struct ptest_list *head, const struct ptest_options opts, close(pipefd_stdout[1]); break; } - signal(SIGALRM, timeout_child_handler); fprintf(fp, "START: %s\n", progname); PTEST_LIST_ITERATE_START(head, p) @@ -511,6 +496,7 @@ run_ptests(struct ptest_list *head, const struct ptest_options opts, } else { int status; + _child_reader.pid = child; if (setpgid(child, pgid) == -1) { fprintf(fp, "ERROR: setpgid() failed, %s\n", strerror(errno)); } @@ -520,7 +506,7 @@ run_ptests(struct ptest_list *head, const struct ptest_options opts, fprintf(fp, "BEGIN: %s\n", ptest_dir); - status = wait_child(child, opts.timeout); + status = wait_child(child); entime = time(NULL); duration = entime - sttime; -- 2.33.0 |
|
|
|
[ptest-runner][PATCH 1/3] tests/utils.c: fix a memory corruption in find_word
Alexander Kanavin
I also took the opportunity to correct a weird API that
returns a result (or not), depending on some internal condition. Signed-off-by: Alexander Kanavin <alex@...> --- tests/utils.c | 35 ++++++++++++++++++----------------- 1 file changed, 18 insertions(+), 17 deletions(-) diff --git a/tests/utils.c b/tests/utils.c index 8fffc18..19657ee 100644 --- a/tests/utils.c +++ b/tests/utils.c @@ -26,6 +26,7 @@ #include <stdlib.h> #include <stdio.h> #include <errno.h> +#include <stdbool.h> #include <check.h> @@ -61,16 +62,13 @@ static char *ptests_not_found[] = { static struct ptest_options EmptyOpts; -static inline void -find_word(int *found, const char *line, const char *word) +static inline bool +find_word(const char *line, const char *word) { - - char *pivot = NULL; - - pivot = strdup(line); - pivot[strlen(word)] = '\0'; - if (strcmp(pivot, word) == 0) { *found = 1; } - free(pivot); + if (strncmp(line, word, strlen(word)) == 0) + return true; + else + return false; } static void test_ptest_expected_failure(struct ptest_list *, const unsigned int, char *, @@ -206,18 +204,19 @@ search_for_timeout_and_duration(const int rp, FILE *fp_stdout) const char *timeout_str = "TIMEOUT"; const char *duration_str = "DURATION"; char line_buf[PRINT_PTEST_BUF_SIZE]; - int found_timeout = 0, found_duration = 0; + bool found_timeout = false, found_duration = false; char *line = NULL; ck_assert(rp != 0); while ((line = fgets(line_buf, PRINT_PTEST_BUF_SIZE, fp_stdout)) != NULL) { - find_word(&found_timeout, line, timeout_str); - find_word(&found_duration, line, duration_str); + // once true, stay true + found_timeout = found_timeout ? found_timeout : find_word(line, timeout_str); + found_duration = found_duration ? found_duration : find_word(line, duration_str); } - ck_assert(found_timeout == 1); - ck_assert(found_duration == 1); + ck_assert(found_timeout == true); + ck_assert(found_duration == true); } START_TEST(test_run_timeout_duration_ptest) @@ -236,16 +235,18 @@ search_for_fail(const int rp, FILE *fp_stdout) { const char *fail_str = "ERROR: Exit status is 10"; char line_buf[PRINT_PTEST_BUF_SIZE]; - int found_fail = 0; + int found_fail = false; char *line = NULL; ck_assert(rp != 0); while ((line = fgets(line_buf, PRINT_PTEST_BUF_SIZE, fp_stdout)) != NULL) { - find_word(&found_fail, line, fail_str); + found_fail = find_word(line, fail_str); + if (found_fail == true) + break; } - ck_assert(found_fail == 1); + ck_assert(found_fail == true); } START_TEST(test_run_fail_ptest) -- 2.33.0 |
|
|
|
[layerindex-web][PATCH] recipeparse.py: Checkout deplayerbranch before parsing
Robert Yang
Fixed:
$ ./update.py -b hardknott,master ERROR: Variable PREMIRRORS_append contains an operation using the old override syntax. Please convert this layer/metadata before attempting to use with a newer bitbake. This is because it doesn't checkout master branch when parse it, this patch fixed the problem. Signed-off-by: Robert Yang <liezhi.yang@...> --- layerindex/recipeparse.py | 1 + 1 file changed, 1 insertion(+) diff --git a/layerindex/recipeparse.py b/layerindex/recipeparse.py index 62c08e91..0870e4f3 100644 --- a/layerindex/recipeparse.py +++ b/layerindex/recipeparse.py @@ -118,6 +118,7 @@ def setup_layer(config_data, fetchdir, layerdir, layer, layerbranch, logger): logger.warning('Recommends %s of layer %s does not have branch record for branch %s - ignoring' % (dep.dependency.name, layer.name, layerbranch.branch.name)) continue deplayerdir = os.path.join(deprepodir, deplayerbranch.vcs_subdir) + utils.checkout_layer_branch(deplayerbranch, deprepodir, logger) utils.parse_layer_conf(deplayerdir, config_data_copy) config_data_copy.delVar('LAYERDIR') return config_data_copy -- 2.17.1 |
|
|
|
QA notification for completed autobuilder build (yocto-3.1.11.rc2)
Richard Purdie
A build flagged for QA (yocto-3.1.11.rc2) was completed on the autobuilder and
is available at: https://autobuilder.yocto.io/pub/releases/yocto-3.1.11.rc2 Build hash information: bitbake: c2a3bda3a29e12472ef7862e424ea1552fab2959 meta-agl: 60344efa7a50dc2548fc4b5d68b5ad4d60c4023a meta-arm: ce535dfb96de4d2529f091d7d85a7172c626001c meta-aws: c5164c1a795c21f7caccc3b68bb2e81a55bddb0e meta-gplv2: 60b251c25ba87e946a0ca4cdc8d17b1cb09292ac meta-intel: 6837552365d3cac5f8044a5ae910aa874435f766 meta-mingw: 524de686205b5d6736661d4532f5f98fee8589b7 meta-openembedded: 2e7e98cd0cb82db214b13224c71134b9335a719b oecore: c7d2281eb6cda9c1637c20b3540b142073bca235 poky: 74b22db6879b388d700f61e08cb3f239cf940d18 This is an automated message from the Yocto Project Autobuilder Git: git://git.yoctoproject.org/yocto-autobuilder2 Email: richard.purdie@... |
|
|
|
Re: Sharing sstate cache across build nodes
On Wed, Sep 15, 2021 at 9:34 PM Rusty Howell <rustyhowell@...> wrote:
yes if you use uninative ( which is default in poky) then it should be able to share across multiple build hosts. if you preserve PR server data then you should be good. sstate can be regenerated. Thanks a bunch. |
|
|
|
Re: Sharing sstate cache across build nodes
Rusty Howell
Thanks for the replies, Richard.
Can SSTATE_DIR be shared across build hosts with different OS's (Ubuntu 18.04, ubuntu 20.04, etc, RHEL)? Our build hosts are somewhat ephemeral. Occasionally we need to swap out a build host for another one. So to bring on a new fresh build host and have it cooperate correctly with the other build hosts and the PR server, what does it need? I understand having the NFS mounted SSTATE_DIR, and using the PRSERV_HOST variable set correctly. But what else? Does the new build host need access to a shared PERSISTENT_DIR or a shared BUILDHISTORY_DIR?
Thanks a bunch. Rusty |
|
|
|
Re: Sharing sstate cache across build nodes
Rusty Howell
Below is an accidental DM between Richard and myself. I am posting it here
for others. > When setting up a shared sstate cache via NFS, do all the build hosts have > read/write access to the sstate cache at the same time? Doesn't that cause > corruption in the sstate cache? If they only have read-only access, is there > anything to consider when selecting which build host will generate the sstate > cache that is shared? Writes to SSTATE_DIR are careful and should use atomic moves into place so sharing read/write via NFS should be safe. We do test this on our autobuilder quite heavily. The main gotcha people run into with sstate is deletion since we can't handle deletion of files from sstate with builds running without the builds potentially showing non-fatal errors. We just don't delete things often on the main AB. > Finally, Is it beneficial to use BUILDHISTORY_PUSH_REPO on all the build hosts > so there is a unified build history? It can be useful, we do this for a subset of our core builds but the repo does get large. The buildhistory codepaths are a lot more complex and likely to have concurrency issues. > Is it problematic to share SSTATE across build hosts > (all Ubuntu 20.04 x86_64) if they build for different MACHINE types (ie > qemux86-64, imx8mq, beaglebone-yocto)? No, sstate is designed to be shared like that. Cheers, Richard |
|
|
|
[ptest-runner] tests/utils.c: fix a memory corruption in find_word
Alexander Kanavin
I also took the opportunity to correct a weird API that
returns a result (or not), depending on some internal condition. Signed-off-by: Alexander Kanavin <alex@...> --- tests/utils.c | 35 ++++++++++++++++++----------------- 1 file changed, 18 insertions(+), 17 deletions(-) diff --git a/tests/utils.c b/tests/utils.c index 8fffc18..19657ee 100644 --- a/tests/utils.c +++ b/tests/utils.c @@ -26,6 +26,7 @@ #include <stdlib.h> #include <stdio.h> #include <errno.h> +#include <stdbool.h> #include <check.h> @@ -61,16 +62,13 @@ static char *ptests_not_found[] = { static struct ptest_options EmptyOpts; -static inline void -find_word(int *found, const char *line, const char *word) +static inline bool +find_word(const char *line, const char *word) { - - char *pivot = NULL; - - pivot = strdup(line); - pivot[strlen(word)] = '\0'; - if (strcmp(pivot, word) == 0) { *found = 1; } - free(pivot); + if (strncmp(line, word, strlen(word)) == 0) + return true; + else + return false; } static void test_ptest_expected_failure(struct ptest_list *, const unsigned int, char *, @@ -206,18 +204,19 @@ search_for_timeout_and_duration(const int rp, FILE *fp_stdout) const char *timeout_str = "TIMEOUT"; const char *duration_str = "DURATION"; char line_buf[PRINT_PTEST_BUF_SIZE]; - int found_timeout = 0, found_duration = 0; + bool found_timeout = false, found_duration = false; char *line = NULL; ck_assert(rp != 0); while ((line = fgets(line_buf, PRINT_PTEST_BUF_SIZE, fp_stdout)) != NULL) { - find_word(&found_timeout, line, timeout_str); - find_word(&found_duration, line, duration_str); + // once true, stay true + found_timeout = found_timeout ? found_timeout : find_word(line, timeout_str); + found_duration = found_duration ? found_duration : find_word(line, duration_str); } - ck_assert(found_timeout == 1); - ck_assert(found_duration == 1); + ck_assert(found_timeout == true); + ck_assert(found_duration == true); } START_TEST(test_run_timeout_duration_ptest) @@ -236,16 +235,18 @@ search_for_fail(const int rp, FILE *fp_stdout) { const char *fail_str = "ERROR: Exit status is 10"; char line_buf[PRINT_PTEST_BUF_SIZE]; - int found_fail = 0; + int found_fail = false; char *line = NULL; ck_assert(rp != 0); while ((line = fgets(line_buf, PRINT_PTEST_BUF_SIZE, fp_stdout)) != NULL) { - find_word(&found_fail, line, fail_str); + found_fail = find_word(line, fail_str); + if (found_fail == true) + break; } - ck_assert(found_fail == 1); + ck_assert(found_fail == true); } START_TEST(test_run_fail_ptest) -- 2.33.0 |
|
|
|
[meta-rockchip][PATCH] rock64: enable lima with rock64
Trevor Woerner
The rock64 has an ARM Mali 450 MP2 GPU, therefore enable mesa's lima for
accelerated, open-source graphics. Signed-off-by: Trevor Woerner <twoerner@...> --- recipes-graphics/mesa/mesa_%.bbappend | 1 + 1 file changed, 1 insertion(+) diff --git a/recipes-graphics/mesa/mesa_%.bbappend b/recipes-graphics/mesa/mesa_%.bbappend index b9089c9..87f4bce 100644 --- a/recipes-graphics/mesa/mesa_%.bbappend +++ b/recipes-graphics/mesa/mesa_%.bbappend @@ -1,2 +1,3 @@ PACKAGECONFIG:append:rk3288 = " kmsro panfrost" PACKAGECONFIG:append:rk3399 = " kmsro panfrost" +PACKAGECONFIG:append:rock64 = " kmsro lima" -- 2.30.0.rc0 |
|
|
|
Re: QA notification for completed autobuilder build (yocto-3.1.11.rc1)
Steve Sakoman
It looks like we'll need to do an rc2 build to pick up a patch for meta-yocto.
Thanks to Denys for noticing the issue! For details see: https://lists.yoctoproject.org/g/poky/message/12559 Steve On Wed, Sep 15, 2021 at 7:12 AM Richard Purdie <richard.purdie@...> wrote:
|
|
|
|
QA notification for completed autobuilder build (yocto-3.1.11.rc1)
Richard Purdie
A build flagged for QA (yocto-3.1.11.rc1) was completed on the autobuilder and
is available at: https://autobuilder.yocto.io/pub/releases/yocto-3.1.11.rc1 Build hash information: bitbake: c2a3bda3a29e12472ef7862e424ea1552fab2959 meta-agl: 60344efa7a50dc2548fc4b5d68b5ad4d60c4023a meta-arm: ce535dfb96de4d2529f091d7d85a7172c626001c meta-aws: c5164c1a795c21f7caccc3b68bb2e81a55bddb0e meta-gplv2: 60b251c25ba87e946a0ca4cdc8d17b1cb09292ac meta-intel: 6837552365d3cac5f8044a5ae910aa874435f766 meta-mingw: 524de686205b5d6736661d4532f5f98fee8589b7 meta-openembedded: 5c347d8ce425dcb4896e6f873810b8bfff5e4e92 oecore: 49ca1f62cc17c951b7737a4ee3c236f732bc8ebe poky: 80b8fc829f809ce07809a89a00cec3ee9dc18795 This is an automated message from the Yocto Project Autobuilder Git: git://git.yoctoproject.org/yocto-autobuilder2 Email: richard.purdie@... |
|
|
|
Re: How to build yocto image with Desktop
#dunfell
On 9/14/21 11:02 PM, prashantsingh@... wrote:
Dear Team,there is core-image-x11 which will be bareminimal and core-image-weston if you want to use wayland/weston based desktop. if you like XFCE for desktop then close meta-openembedded and add meta-oe and meta-xfce to your layers and build core-image-minimal-xfce also see https://git.openembedded.org/meta-openembedded/tree/meta-xfce/README |
|
|
|
Re: [meta-security][PATCH] sssd: 2.5.1 -> 2.5.2
merged
toggle quoted message
Show quoted text
thanks On 9/10/21 1:39 AM, kai wrote:
From: Kai Kang <kai.kang@...> |
|
|
|
Re: [qa-build-notification] QA notification for completed autobuilder build (yocto-3.3.3.rc2)
Teoh, Jay Shen
Hello all,
toggle quoted message
Show quoted text
This is the full report for yocto-3.3.3.rc2: https://git.yoctoproject.org/cgit/cgit.cgi/yocto-testresults-contrib/tree/?h=intel-yocto-testresults ======= Summary ======== No high milestone defects. 1 issue found BUG id:14491 - stap.StapTest.test_stap failure ======= Bugs ======== https://bugzilla.yoctoproject.org/show_bug.cgi?id=14491 Thanks, Jay -----Original Message----- |
|
|
|
How to build yocto image with Desktop
#dunfell
@prashant2314
Dear Team,
I need to build Rpi3 image with yocto which includes Desktop, so how can build the image with desktop feature, so that I can use it for browsing purpose after installing one of the available browser with this image. |
|
|
|
Re: multilib SDK
Arun
I see, are you talking about ${MLPREFIX} variable? The target itself compiles fine, all are 32-bit binaries in userspace. I have this issue only for SDK. On Tue, Sep 14, 2021 at 7:21 PM Khem Raj <raj.khem@...> wrote:
|
|
|