Date   

Re: Minutes: Yocto Project Weekly Triage Meeting 9/16/2021

Trevor Gamblin
 


On 2021-09-16 11:33 a.m., Trevor Gamblin wrote:

Wiki: https://wiki.yoctoproject.org/wiki/Bug_Triage

Attendees: Alex, Diane, Jon, Joshua, Michael, Richard, Ross, Saul, Stephen, Steve, Tim, Trevor

ARs:

- Trevor to move old AB defects to M4 after call

- Trevor to ping WR folks about moving Old Milestone bugs

- Richard to add a comment to 7298


Notes:

- (carried over) Steve encountered build failures such as the one in https://errors.yoctoproject.org/Errors/Details/593109/ when attempting to run dunfell builds with the PARALLEL_MAKE load averaging added. WR is testing/investigating on internal Autobuilder instance - Trevor is still planning on looking into this!

Medium+ 3.4 Unassigned Enhancements/Bugs: 68 (Last week 77)

Medium+ 3.5 Unassigned Enhancements/Bugs: 10 (new)

Medium+ 3.99 Unassigned Enhancements/Bugs: 38 (No change)

AB-INT Bugs: 49 (Last week 48)
Correction: AB-INT bug count is at 52.


Minutes: Yocto Project Weekly Triage Meeting 9/16/2021

Trevor Gamblin
 

Wiki: https://wiki.yoctoproject.org/wiki/Bug_Triage

Attendees: Alex, Diane, Jon, Joshua, Michael, Richard, Ross, Saul, Stephen, Steve, Tim, Trevor

ARs:

- Trevor to move old AB defects to M4 after call

- Trevor to ping WR folks about moving Old Milestone bugs

- Richard to add a comment to 7298


Notes:

- (carried over) Steve encountered build failures such as the one in https://errors.yoctoproject.org/Errors/Details/593109/ when attempting to run dunfell builds with the PARALLEL_MAKE load averaging added. WR is testing/investigating on internal Autobuilder instance - Trevor is still planning on looking into this!

Medium+ 3.4 Unassigned Enhancements/Bugs: 68 (Last week 77)

Medium+ 3.5 Unassigned Enhancements/Bugs: 10 (new)

Medium+ 3.99 Unassigned Enhancements/Bugs: 38 (No change)

AB-INT Bugs: 49 (Last week 48)


[ptest-runner][PATCH 3/3] utils.c: add system data collection when a test gets stuck.

Alexander Kanavin
 

Currently, ptest-runner simply kills the offending test without further ado,
which is not at all helpful when trying to figure out why it happens
(especially if such hangs are intermittent and rare). There's now a script
that gets executed before killing the test, so ideas on what to have in it
are welcome.

Signed-off-by: Alexander Kanavin <alex@...>
---
Makefile | 2 +-
ptest-runner-collect-system-data | 5 +++++
utils.c | 24 ++++++++++++++++++++++++
3 files changed, 30 insertions(+), 1 deletion(-)
create mode 100755 ptest-runner-collect-system-data

diff --git a/Makefile b/Makefile
index a6372de..168cf5a 100644
--- a/Makefile
+++ b/Makefile
@@ -43,7 +43,7 @@ $(TEST_EXECUTABLE): $(TEST_OBJECTS)
$(CC) $(LDFLAGS) $(TEST_OBJECTS) -o $@ $(TEST_LIBSTATIC) $(TEST_LDFLAGS)

check: $(TEST_EXECUTABLE)
- ./$(TEST_EXECUTABLE) -d $(TEST_DATA)
+ PATH=.:$(PATH) ./$(TEST_EXECUTABLE) -d $(TEST_DATA)

.c.o:
$(CC) $(CFLAGS) -c $< -o $@
diff --git a/ptest-runner-collect-system-data b/ptest-runner-collect-system-data
new file mode 100755
index 0000000..5bfeaf3
--- /dev/null
+++ b/ptest-runner-collect-system-data
@@ -0,0 +1,5 @@
+#!/bin/sh
+# Other ideas on what to do when a ptest gets stuck welcome.
+pstree -a -l
+df
+free
diff --git a/utils.c b/utils.c
index 58c3aa1..a67ac11 100644
--- a/utils.c
+++ b/utils.c
@@ -281,6 +281,27 @@ close_fds(void)
}
}

+static void
+collect_system_state(FILE* fout)
+{
+ char *cmd = "ptest-runner-collect-system-data";
+
+ char buf[1024];
+ FILE *fp;
+
+ if ((fp = popen(cmd, "r")) == NULL) {
+ fprintf(fout, "Error opening pipe!\n");
+ }
+
+ while (fgets(buf, 1024, fp) != NULL) {
+ fprintf(fout, "%s", buf);
+ }
+
+ if(pclose(fp)) {
+ fprintf(fout, "Command not found or exited with error status\n");
+ }
+}
+
static void *
read_child(void *arg)
{
@@ -313,6 +334,9 @@ read_child(void *arg)
}

} else if (r == 0) {
+ // no output from the test after a timeout; the test is stuck, so collect
+ // as much data from the system as possible and kill the test
+ collect_system_state(_child_reader.fps[0]);
_child_reader.timeouted = 1;
kill(-_child_reader.pid, SIGKILL);
}
--
2.33.0


[ptest-runner][PATCH 2/3] utils.c: handle test timeouts directly with poll()

Alexander Kanavin
 

if poll()'s timeout expires that means the test did not
produce any output, which is exactly what we need to catch.

So there's no need to set up separate timeouts with signals
and alarms, and this greatly simplifies more sophisticated
processing of hanging tests (such as collecting overall system data).

Signed-off-by: Alexander Kanavin <alex@...>
---
utils.c | 34 ++++++++++------------------------
1 file changed, 10 insertions(+), 24 deletions(-)

diff --git a/utils.c b/utils.c
index 128ff61..58c3aa1 100644
--- a/utils.c
+++ b/utils.c
@@ -51,7 +51,6 @@
#include "utils.h"

#define GET_STIME_BUF_SIZE 1024
-#define WAIT_CHILD_POLL_TIMEOUT_MS 200
#define WAIT_CHILD_BUF_MAX_SIZE 1024

#define UNUSED(x) (void)(x)
@@ -296,7 +295,7 @@ read_child(void *arg)
pfds[1].events = POLLIN;

do {
- r = poll(pfds, 2, WAIT_CHILD_POLL_TIMEOUT_MS);
+ r = poll(pfds, 2, _child_reader.timeout*1000);
if (r > 0) {
char buf[WAIT_CHILD_BUF_MAX_SIZE];
ssize_t n;
@@ -313,10 +312,10 @@ read_child(void *arg)
fwrite(buf, (size_t)n, 1, _child_reader.fps[1]);
}

- /* Child output reset alarm */
- alarm(0);
- alarm(_child_reader.timeout);
- }
+ } else if (r == 0) {
+ _child_reader.timeouted = 1;
+ kill(-_child_reader.pid, SIGKILL);
+ }

fflush(_child_reader.fps[0]);
fflush(_child_reader.fps[1]);
@@ -344,26 +343,11 @@ run_child(char *run_ptest, int fd_stdout, int fd_stderr)
/* exit(1); not needed? */
}

-static void
-timeout_child_handler(int signo)
-{
- UNUSED(signo);
- _child_reader.timeouted = 1;
- kill(-_child_reader.pid, SIGKILL);
-}
-
static inline int
-wait_child(pid_t pid, unsigned int timeout)
+wait_child(pid_t pid)
{
int status = -1;

- _child_reader.timeout = timeout;
- _child_reader.timeouted = 0;
- _child_reader.pid = pid;
-
- /* setup alarm to timeout based on std{out,err} in the child */
- alarm(timeout);
-
waitpid(pid, &status, 0);
if (WIFEXITED(status))
status = WEXITSTATUS(status);
@@ -462,6 +446,8 @@ run_ptests(struct ptest_list *head, const struct ptest_options opts,
_child_reader.fds[1] = pipefd_stderr[0];
_child_reader.fps[0] = fp;
_child_reader.fps[1] = fp_stderr;
+ _child_reader.timeout = opts.timeout;
+ _child_reader.timeouted = 0;
rc = pthread_create(&tid, NULL, read_child, NULL);
if (rc != 0) {
fprintf(fp, "ERROR: Failed to create reader thread, %s\n", strerror(errno));
@@ -469,7 +455,6 @@ run_ptests(struct ptest_list *head, const struct ptest_options opts,
close(pipefd_stdout[1]);
break;
}
- signal(SIGALRM, timeout_child_handler);

fprintf(fp, "START: %s\n", progname);
PTEST_LIST_ITERATE_START(head, p)
@@ -511,6 +496,7 @@ run_ptests(struct ptest_list *head, const struct ptest_options opts,
} else {
int status;

+ _child_reader.pid = child;
if (setpgid(child, pgid) == -1) {
fprintf(fp, "ERROR: setpgid() failed, %s\n", strerror(errno));
}
@@ -520,7 +506,7 @@ run_ptests(struct ptest_list *head, const struct ptest_options opts,
fprintf(fp, "BEGIN: %s\n", ptest_dir);


- status = wait_child(child, opts.timeout);
+ status = wait_child(child);

entime = time(NULL);
duration = entime - sttime;
--
2.33.0


[ptest-runner][PATCH 1/3] tests/utils.c: fix a memory corruption in find_word

Alexander Kanavin
 

I also took the opportunity to correct a weird API that
returns a result (or not), depending on some internal condition.

Signed-off-by: Alexander Kanavin <alex@...>
---
tests/utils.c | 35 ++++++++++++++++++-----------------
1 file changed, 18 insertions(+), 17 deletions(-)

diff --git a/tests/utils.c b/tests/utils.c
index 8fffc18..19657ee 100644
--- a/tests/utils.c
+++ b/tests/utils.c
@@ -26,6 +26,7 @@
#include <stdlib.h>
#include <stdio.h>
#include <errno.h>
+#include <stdbool.h>

#include <check.h>

@@ -61,16 +62,13 @@ static char *ptests_not_found[] = {

static struct ptest_options EmptyOpts;

-static inline void
-find_word(int *found, const char *line, const char *word)
+static inline bool
+find_word(const char *line, const char *word)
{
-
- char *pivot = NULL;
-
- pivot = strdup(line);
- pivot[strlen(word)] = '\0';
- if (strcmp(pivot, word) == 0) { *found = 1; }
- free(pivot);
+ if (strncmp(line, word, strlen(word)) == 0)
+ return true;
+ else
+ return false;
}

static void test_ptest_expected_failure(struct ptest_list *, const unsigned int, char *,
@@ -206,18 +204,19 @@ search_for_timeout_and_duration(const int rp, FILE *fp_stdout)
const char *timeout_str = "TIMEOUT";
const char *duration_str = "DURATION";
char line_buf[PRINT_PTEST_BUF_SIZE];
- int found_timeout = 0, found_duration = 0;
+ bool found_timeout = false, found_duration = false;
char *line = NULL;

ck_assert(rp != 0);

while ((line = fgets(line_buf, PRINT_PTEST_BUF_SIZE, fp_stdout)) != NULL) {
- find_word(&found_timeout, line, timeout_str);
- find_word(&found_duration, line, duration_str);
+ // once true, stay true
+ found_timeout = found_timeout ? found_timeout : find_word(line, timeout_str);
+ found_duration = found_duration ? found_duration : find_word(line, duration_str);
}

- ck_assert(found_timeout == 1);
- ck_assert(found_duration == 1);
+ ck_assert(found_timeout == true);
+ ck_assert(found_duration == true);
}

START_TEST(test_run_timeout_duration_ptest)
@@ -236,16 +235,18 @@ search_for_fail(const int rp, FILE *fp_stdout)
{
const char *fail_str = "ERROR: Exit status is 10";
char line_buf[PRINT_PTEST_BUF_SIZE];
- int found_fail = 0;
+ int found_fail = false;
char *line = NULL;

ck_assert(rp != 0);

while ((line = fgets(line_buf, PRINT_PTEST_BUF_SIZE, fp_stdout)) != NULL) {
- find_word(&found_fail, line, fail_str);
+ found_fail = find_word(line, fail_str);
+ if (found_fail == true)
+ break;
}

- ck_assert(found_fail == 1);
+ ck_assert(found_fail == true);
}

START_TEST(test_run_fail_ptest)
--
2.33.0


[layerindex-web][PATCH] recipeparse.py: Checkout deplayerbranch before parsing

Robert Yang
 

Fixed:
$ ./update.py -b hardknott,master

ERROR: Variable PREMIRRORS_append contains an operation using the old override syntax. Please convert this layer/metadata before attempting to use with a newer bitbake.

This is because it doesn't checkout master branch when parse it, this patch
fixed the problem.

Signed-off-by: Robert Yang <liezhi.yang@...>
---
layerindex/recipeparse.py | 1 +
1 file changed, 1 insertion(+)

diff --git a/layerindex/recipeparse.py b/layerindex/recipeparse.py
index 62c08e91..0870e4f3 100644
--- a/layerindex/recipeparse.py
+++ b/layerindex/recipeparse.py
@@ -118,6 +118,7 @@ def setup_layer(config_data, fetchdir, layerdir, layer, layerbranch, logger):
logger.warning('Recommends %s of layer %s does not have branch record for branch %s - ignoring' % (dep.dependency.name, layer.name, layerbranch.branch.name))
continue
deplayerdir = os.path.join(deprepodir, deplayerbranch.vcs_subdir)
+ utils.checkout_layer_branch(deplayerbranch, deprepodir, logger)
utils.parse_layer_conf(deplayerdir, config_data_copy)
config_data_copy.delVar('LAYERDIR')
return config_data_copy
--
2.17.1


QA notification for completed autobuilder build (yocto-3.1.11.rc2)

Richard Purdie
 

A build flagged for QA (yocto-3.1.11.rc2) was completed on the autobuilder and
is available at:


https://autobuilder.yocto.io/pub/releases/yocto-3.1.11.rc2


Build hash information:

bitbake: c2a3bda3a29e12472ef7862e424ea1552fab2959
meta-agl: 60344efa7a50dc2548fc4b5d68b5ad4d60c4023a
meta-arm: ce535dfb96de4d2529f091d7d85a7172c626001c
meta-aws: c5164c1a795c21f7caccc3b68bb2e81a55bddb0e
meta-gplv2: 60b251c25ba87e946a0ca4cdc8d17b1cb09292ac
meta-intel: 6837552365d3cac5f8044a5ae910aa874435f766
meta-mingw: 524de686205b5d6736661d4532f5f98fee8589b7
meta-openembedded: 2e7e98cd0cb82db214b13224c71134b9335a719b
oecore: c7d2281eb6cda9c1637c20b3540b142073bca235
poky: 74b22db6879b388d700f61e08cb3f239cf940d18



This is an automated message from the Yocto Project Autobuilder
Git: git://git.yoctoproject.org/yocto-autobuilder2
Email: richard.purdie@...


Re: Sharing sstate cache across build nodes

Khem Raj
 

On Wed, Sep 15, 2021 at 9:34 PM Rusty Howell <rustyhowell@...> wrote:

Thanks for the replies, Richard.

Can SSTATE_DIR be shared across build hosts with different OS's (Ubuntu 18.04, ubuntu 20.04, etc, RHEL)?
yes if you use uninative ( which is default in poky) then it should be
able to share across multiple build hosts.


Our build hosts are somewhat ephemeral. Occasionally we need to swap out a build host for another one. So to bring on a new fresh build host and have it cooperate correctly with the other build hosts and the PR server, what does it need? I understand having the NFS mounted SSTATE_DIR, and using the PRSERV_HOST variable set correctly. But what else? Does the new build host need access to a shared PERSISTENT_DIR or a shared BUILDHISTORY_DIR?

What happens if the shared SSTATE cache get corrupted and has to be deleted? Won't that cause all the PR server values to reset? We just want to make sure we know how to recover from a situation like that.
if you preserve PR server data then you should be good. sstate can be
regenerated.

Thanks a bunch.
Rusty


Re: Sharing sstate cache across build nodes

Rusty Howell
 

Thanks for the replies, Richard. 

Can SSTATE_DIR be shared across build hosts with different OS's  (Ubuntu 18.04, ubuntu 20.04, etc, RHEL)?
 
Our build hosts are somewhat ephemeral.  Occasionally we need to swap out a build host for another one. So to bring on a new fresh build host and have it cooperate correctly with the other build hosts and the PR server, what does it need?  I understand having the NFS mounted SSTATE_DIR, and using the PRSERV_HOST variable set correctly. But what else?  Does the new build host need access to a shared PERSISTENT_DIR or a shared BUILDHISTORY_DIR?
 
What happens if the shared SSTATE cache get corrupted and has to be deleted?   Won't that cause all the PR server values to reset?  We just want to make sure we know how to recover from a situation like that.
Thanks a bunch.
Rusty


Re: Sharing sstate cache across build nodes

Rusty Howell
 

Below is an accidental DM between Richard and myself. I am posting it here
for others.

> When setting up a shared sstate cache via NFS, do all the build hosts have
> read/write access to the sstate cache at the same time?  Doesn't that cause
> corruption in the sstate cache?  If they only have read-only access, is there
> anything to consider when selecting which build host will generate the sstate
> cache that is shared? 

Writes to SSTATE_DIR are careful and should use atomic moves into place so
sharing read/write via NFS should be safe. We do test this on our autobuilder
quite heavily.

The main gotcha people run into with sstate is deletion since we can't handle
deletion of files from sstate with builds running without the builds potentially
showing non-fatal errors. We just don't delete things often on the main AB.

> Finally, Is it beneficial to use BUILDHISTORY_PUSH_REPO on all the build hosts
> so there is a unified build history?

It can be useful, we do this for a subset of our core builds but the repo does
get large. The buildhistory codepaths are a lot more complex and likely to have
concurrency issues.

> Is it problematic to share SSTATE across build hosts
> (all Ubuntu 20.04 x86_64) if they build for different MACHINE types (ie
> qemux86-64, imx8mq, beaglebone-yocto)?

No, sstate is designed to be shared like that.

Cheers,

Richard


[ptest-runner] tests/utils.c: fix a memory corruption in find_word

Alexander Kanavin
 

I also took the opportunity to correct a weird API that
returns a result (or not), depending on some internal condition.

Signed-off-by: Alexander Kanavin <alex@...>
---
tests/utils.c | 35 ++++++++++++++++++-----------------
1 file changed, 18 insertions(+), 17 deletions(-)

diff --git a/tests/utils.c b/tests/utils.c
index 8fffc18..19657ee 100644
--- a/tests/utils.c
+++ b/tests/utils.c
@@ -26,6 +26,7 @@
#include <stdlib.h>
#include <stdio.h>
#include <errno.h>
+#include <stdbool.h>

#include <check.h>

@@ -61,16 +62,13 @@ static char *ptests_not_found[] = {

static struct ptest_options EmptyOpts;

-static inline void
-find_word(int *found, const char *line, const char *word)
+static inline bool
+find_word(const char *line, const char *word)
{
-
- char *pivot = NULL;
-
- pivot = strdup(line);
- pivot[strlen(word)] = '\0';
- if (strcmp(pivot, word) == 0) { *found = 1; }
- free(pivot);
+ if (strncmp(line, word, strlen(word)) == 0)
+ return true;
+ else
+ return false;
}

static void test_ptest_expected_failure(struct ptest_list *, const unsigned int, char *,
@@ -206,18 +204,19 @@ search_for_timeout_and_duration(const int rp, FILE *fp_stdout)
const char *timeout_str = "TIMEOUT";
const char *duration_str = "DURATION";
char line_buf[PRINT_PTEST_BUF_SIZE];
- int found_timeout = 0, found_duration = 0;
+ bool found_timeout = false, found_duration = false;
char *line = NULL;

ck_assert(rp != 0);

while ((line = fgets(line_buf, PRINT_PTEST_BUF_SIZE, fp_stdout)) != NULL) {
- find_word(&found_timeout, line, timeout_str);
- find_word(&found_duration, line, duration_str);
+ // once true, stay true
+ found_timeout = found_timeout ? found_timeout : find_word(line, timeout_str);
+ found_duration = found_duration ? found_duration : find_word(line, duration_str);
}

- ck_assert(found_timeout == 1);
- ck_assert(found_duration == 1);
+ ck_assert(found_timeout == true);
+ ck_assert(found_duration == true);
}

START_TEST(test_run_timeout_duration_ptest)
@@ -236,16 +235,18 @@ search_for_fail(const int rp, FILE *fp_stdout)
{
const char *fail_str = "ERROR: Exit status is 10";
char line_buf[PRINT_PTEST_BUF_SIZE];
- int found_fail = 0;
+ int found_fail = false;
char *line = NULL;

ck_assert(rp != 0);

while ((line = fgets(line_buf, PRINT_PTEST_BUF_SIZE, fp_stdout)) != NULL) {
- find_word(&found_fail, line, fail_str);
+ found_fail = find_word(line, fail_str);
+ if (found_fail == true)
+ break;
}

- ck_assert(found_fail == 1);
+ ck_assert(found_fail == true);
}

START_TEST(test_run_fail_ptest)
--
2.33.0


[meta-rockchip][PATCH] rock64: enable lima with rock64

Trevor Woerner
 

The rock64 has an ARM Mali 450 MP2 GPU, therefore enable mesa's lima for
accelerated, open-source graphics.

Signed-off-by: Trevor Woerner <twoerner@...>
---
recipes-graphics/mesa/mesa_%.bbappend | 1 +
1 file changed, 1 insertion(+)

diff --git a/recipes-graphics/mesa/mesa_%.bbappend b/recipes-graphics/mesa/mesa_%.bbappend
index b9089c9..87f4bce 100644
--- a/recipes-graphics/mesa/mesa_%.bbappend
+++ b/recipes-graphics/mesa/mesa_%.bbappend
@@ -1,2 +1,3 @@
PACKAGECONFIG:append:rk3288 = " kmsro panfrost"
PACKAGECONFIG:append:rk3399 = " kmsro panfrost"
+PACKAGECONFIG:append:rock64 = " kmsro lima"
--
2.30.0.rc0


Re: QA notification for completed autobuilder build (yocto-3.1.11.rc1)

Steve Sakoman
 

It looks like we'll need to do an rc2 build to pick up a patch for meta-yocto.

Thanks to Denys for noticing the issue! For details see:
https://lists.yoctoproject.org/g/poky/message/12559

Steve

On Wed, Sep 15, 2021 at 7:12 AM Richard Purdie
<richard.purdie@...> wrote:

A build flagged for QA (yocto-3.1.11.rc1) was completed on the autobuilder and
is available at:


https://autobuilder.yocto.io/pub/releases/yocto-3.1.11.rc1


Build hash information:

bitbake: c2a3bda3a29e12472ef7862e424ea1552fab2959
meta-agl: 60344efa7a50dc2548fc4b5d68b5ad4d60c4023a
meta-arm: ce535dfb96de4d2529f091d7d85a7172c626001c
meta-aws: c5164c1a795c21f7caccc3b68bb2e81a55bddb0e
meta-gplv2: 60b251c25ba87e946a0ca4cdc8d17b1cb09292ac
meta-intel: 6837552365d3cac5f8044a5ae910aa874435f766
meta-mingw: 524de686205b5d6736661d4532f5f98fee8589b7
meta-openembedded: 5c347d8ce425dcb4896e6f873810b8bfff5e4e92
oecore: 49ca1f62cc17c951b7737a4ee3c236f732bc8ebe
poky: 80b8fc829f809ce07809a89a00cec3ee9dc18795



This is an automated message from the Yocto Project Autobuilder
Git: git://git.yoctoproject.org/yocto-autobuilder2
Email: richard.purdie@...






QA notification for completed autobuilder build (yocto-3.1.11.rc1)

Richard Purdie
 

A build flagged for QA (yocto-3.1.11.rc1) was completed on the autobuilder and
is available at:


https://autobuilder.yocto.io/pub/releases/yocto-3.1.11.rc1


Build hash information:

bitbake: c2a3bda3a29e12472ef7862e424ea1552fab2959
meta-agl: 60344efa7a50dc2548fc4b5d68b5ad4d60c4023a
meta-arm: ce535dfb96de4d2529f091d7d85a7172c626001c
meta-aws: c5164c1a795c21f7caccc3b68bb2e81a55bddb0e
meta-gplv2: 60b251c25ba87e946a0ca4cdc8d17b1cb09292ac
meta-intel: 6837552365d3cac5f8044a5ae910aa874435f766
meta-mingw: 524de686205b5d6736661d4532f5f98fee8589b7
meta-openembedded: 5c347d8ce425dcb4896e6f873810b8bfff5e4e92
oecore: 49ca1f62cc17c951b7737a4ee3c236f732bc8ebe
poky: 80b8fc829f809ce07809a89a00cec3ee9dc18795



This is an automated message from the Yocto Project Autobuilder
Git: git://git.yoctoproject.org/yocto-autobuilder2
Email: richard.purdie@...


Re: How to build yocto image with Desktop #dunfell

Khem Raj
 

On 9/14/21 11:02 PM, prashantsingh@... wrote:
Dear Team,
I need to build Rpi3 image  with yocto which includes Desktop, so how can build the image with desktop feature, so that I can use it for browsing purpose after installing one of the available browser with this image.
there is core-image-x11 which will be bareminimal and core-image-weston if you want to use wayland/weston based desktop.

if you like XFCE for desktop then close meta-openembedded and add meta-oe and meta-xfce to your layers and build core-image-minimal-xfce also see https://git.openembedded.org/meta-openembedded/tree/meta-xfce/README


Re: [meta-security][PATCH] sssd: 2.5.1 -> 2.5.2

Armin Kuster
 

merged
thanks

On 9/10/21 1:39 AM, kai wrote:
From: Kai Kang <kai.kang@...>

SSSD 2.5.2 Highlights
* General information
- originalADgidNumber attribute in the SSSD cache is now indexed

* New features
- Debug messages in data provider include a unique request ID that can
be used to track the request from its start to its end (requires
libtevent >= 0.11.0)

* Important fixes
- Update large files in the files provider in batches to avoid timeouts

* Configuration changes
- Add new config option fallback_to_nss

Full release notes:
* https://sssd.io/release-notes/sssd-2.5.2.html

And backport patch to fix CVE-2021-3621.

CVE: CVE-2021-3621

Signed-off-by: Kai Kang <kai.kang@...>
---
.../sssd/files/CVE-2021-3621.patch | 288 ++++++++++++++++++
.../sssd/{sssd_2.5.1.bb => sssd_2.5.2.bb} | 3 +-
2 files changed, 290 insertions(+), 1 deletion(-)
create mode 100644 recipes-security/sssd/files/CVE-2021-3621.patch
rename recipes-security/sssd/{sssd_2.5.1.bb => sssd_2.5.2.bb} (97%)

diff --git a/recipes-security/sssd/files/CVE-2021-3621.patch b/recipes-security/sssd/files/CVE-2021-3621.patch
new file mode 100644
index 0000000..7a59df9
--- /dev/null
+++ b/recipes-security/sssd/files/CVE-2021-3621.patch
@@ -0,0 +1,288 @@
+Backport patch to fix CVE-2021-3621.
+
+Upstream-Status: Backport [https://github.com/SSSD/sssd/commit/7ab83f9]
+CVE: CVE-2021-3621
+
+Signed-off-by: Kai Kang <kai.kang@...>
+
+From 7ab83f97e1cbefb78ece17232185bdd2985f0bbe Mon Sep 17 00:00:00 2001
+From: Alexey Tikhonov <atikhono@...>
+Date: Fri, 18 Jun 2021 13:17:19 +0200
+Subject: [PATCH] TOOLS: replace system() with execvp() to avoid execution of
+ user supplied command
+MIME-Version: 1.0
+Content-Type: text/plain; charset=UTF-8
+Content-Transfer-Encoding: 8bit
+
+:relnote: A flaw was found in SSSD, where the sssctl command was
+vulnerable to shell command injection via the logs-fetch and
+cache-expire subcommands. This flaw allows an attacker to trick
+the root user into running a specially crafted sssctl command,
+such as via sudo, to gain root access. The highest threat from this
+vulnerability is to confidentiality, integrity, as well as system
+availability.
+This patch fixes a flaw by replacing system() with execvp().
+
+:fixes: CVE-2021-3621
+
+Reviewed-by: Pavel Březina <pbrezina@...>
+---
+ src/tools/sssctl/sssctl.c | 39 ++++++++++++++++-------
+ src/tools/sssctl/sssctl.h | 2 +-
+ src/tools/sssctl/sssctl_data.c | 57 +++++++++++-----------------------
+ src/tools/sssctl/sssctl_logs.c | 32 +++++++++++++++----
+ 4 files changed, 73 insertions(+), 57 deletions(-)
+
+diff --git a/src/tools/sssctl/sssctl.c b/src/tools/sssctl/sssctl.c
+index 2997dbf968..8adaf30910 100644
+--- a/src/tools/sssctl/sssctl.c
++++ b/src/tools/sssctl/sssctl.c
+@@ -97,22 +97,36 @@ sssctl_prompt(const char *message,
+ return SSSCTL_PROMPT_ERROR;
+ }
+
+-errno_t sssctl_run_command(const char *command)
++errno_t sssctl_run_command(const char *const argv[])
+ {
+ int ret;
++ int wstatus;
+
+- DEBUG(SSSDBG_TRACE_FUNC, "Running %s\n", command);
++ DEBUG(SSSDBG_TRACE_FUNC, "Running '%s'\n", argv[0]);
+
+- ret = system(command);
++ ret = fork();
+ if (ret == -1) {
+- DEBUG(SSSDBG_CRIT_FAILURE, "Unable to execute %s\n", command);
+ ERROR("Error while executing external command\n");
+ return EFAULT;
+- } else if (WEXITSTATUS(ret) != 0) {
+- DEBUG(SSSDBG_CRIT_FAILURE, "Command %s failed with [%d]\n",
+- command, WEXITSTATUS(ret));
++ }
++
++ if (ret == 0) {
++ /* cast is safe - see
++ https://pubs.opengroup.org/onlinepubs/9699919799/functions/exec.html
++ "The statement about argv[] and envp[] being constants ... "
++ */
++ execvp(argv[0], discard_const_p(char * const, argv));
+ ERROR("Error while executing external command\n");
+- return EIO;
++ _exit(1);
++ } else {
++ if (waitpid(ret, &wstatus, 0) == -1) {
++ ERROR("Error while executing external command '%s'\n", argv[0]);
++ return EFAULT;
++ } else if (WEXITSTATUS(wstatus) != 0) {
++ ERROR("Command '%s' failed with [%d]\n",
++ argv[0], WEXITSTATUS(wstatus));
++ return EIO;
++ }
+ }
+
+ return EOK;
+@@ -132,11 +146,14 @@ static errno_t sssctl_manage_service(enum sssctl_svc_action action)
+ #elif defined(HAVE_SERVICE)
+ switch (action) {
+ case SSSCTL_SVC_START:
+- return sssctl_run_command(SERVICE_PATH" sssd start");
++ return sssctl_run_command(
++ (const char *[]){SERVICE_PATH, "sssd", "start", NULL});
+ case SSSCTL_SVC_STOP:
+- return sssctl_run_command(SERVICE_PATH" sssd stop");
++ return sssctl_run_command(
++ (const char *[]){SERVICE_PATH, "sssd", "stop", NULL});
+ case SSSCTL_SVC_RESTART:
+- return sssctl_run_command(SERVICE_PATH" sssd restart");
++ return sssctl_run_command(
++ (const char *[]){SERVICE_PATH, "sssd", "restart", NULL});
+ }
+ #endif
+
+diff --git a/src/tools/sssctl/sssctl.h b/src/tools/sssctl/sssctl.h
+index 0115b2457c..599ef65196 100644
+--- a/src/tools/sssctl/sssctl.h
++++ b/src/tools/sssctl/sssctl.h
+@@ -47,7 +47,7 @@ enum sssctl_prompt_result
+ sssctl_prompt(const char *message,
+ enum sssctl_prompt_result defval);
+
+-errno_t sssctl_run_command(const char *command);
++errno_t sssctl_run_command(const char *const argv[]); /* argv[0] - command */
+ bool sssctl_start_sssd(bool force);
+ bool sssctl_stop_sssd(bool force);
+ bool sssctl_restart_sssd(bool force);
+diff --git a/src/tools/sssctl/sssctl_data.c b/src/tools/sssctl/sssctl_data.c
+index 8d79b977fd..bf22913416 100644
+--- a/src/tools/sssctl/sssctl_data.c
++++ b/src/tools/sssctl/sssctl_data.c
+@@ -105,15 +105,15 @@ static errno_t sssctl_backup(bool force)
+ }
+ }
+
+- ret = sssctl_run_command("sss_override user-export "
+- SSS_BACKUP_USER_OVERRIDES);
++ ret = sssctl_run_command((const char *[]){"sss_override", "user-export",
++ SSS_BACKUP_USER_OVERRIDES, NULL});
+ if (ret != EOK) {
+ ERROR("Unable to export user overrides\n");
+ return ret;
+ }
+
+- ret = sssctl_run_command("sss_override group-export "
+- SSS_BACKUP_GROUP_OVERRIDES);
++ ret = sssctl_run_command((const char *[]){"sss_override", "group-export",
++ SSS_BACKUP_GROUP_OVERRIDES, NULL});
+ if (ret != EOK) {
+ ERROR("Unable to export group overrides\n");
+ return ret;
+@@ -158,8 +158,8 @@ static errno_t sssctl_restore(bool force_start, bool force_restart)
+ }
+
+ if (sssctl_backup_file_exists(SSS_BACKUP_USER_OVERRIDES)) {
+- ret = sssctl_run_command("sss_override user-import "
+- SSS_BACKUP_USER_OVERRIDES);
++ ret = sssctl_run_command((const char *[]){"sss_override", "user-import",
++ SSS_BACKUP_USER_OVERRIDES, NULL});
+ if (ret != EOK) {
+ ERROR("Unable to import user overrides\n");
+ return ret;
+@@ -167,8 +167,8 @@ static errno_t sssctl_restore(bool force_start, bool force_restart)
+ }
+
+ if (sssctl_backup_file_exists(SSS_BACKUP_USER_OVERRIDES)) {
+- ret = sssctl_run_command("sss_override group-import "
+- SSS_BACKUP_GROUP_OVERRIDES);
++ ret = sssctl_run_command((const char *[]){"sss_override", "group-import",
++ SSS_BACKUP_GROUP_OVERRIDES, NULL});
+ if (ret != EOK) {
+ ERROR("Unable to import group overrides\n");
+ return ret;
+@@ -296,40 +296,19 @@ errno_t sssctl_cache_expire(struct sss_cmdline *cmdline,
+ void *pvt)
+ {
+ errno_t ret;
+- char *cmd_args = NULL;
+- const char *cachecmd = SSS_CACHE;
+- char *cmd = NULL;
+- int i;
+-
+- if (cmdline->argc == 0) {
+- ret = sssctl_run_command(cachecmd);
+- goto done;
+- }
+
+- cmd_args = talloc_strdup(tool_ctx, "");
+- if (cmd_args == NULL) {
+- ret = ENOMEM;
+- goto done;
++ const char **args = talloc_array_size(tool_ctx,
++ sizeof(char *),
++ cmdline->argc + 2);
++ if (!args) {
++ return ENOMEM;
+ }
++ memcpy(&args[1], cmdline->argv, sizeof(char *) * cmdline->argc);
++ args[0] = SSS_CACHE;
++ args[cmdline->argc + 1] = NULL;
+
+- for (i = 0; i < cmdline->argc; i++) {
+- cmd_args = talloc_strdup_append(cmd_args, cmdline->argv[i]);
+- if (i != cmdline->argc - 1) {
+- cmd_args = talloc_strdup_append(cmd_args, " ");
+- }
+- }
+-
+- cmd = talloc_asprintf(tool_ctx, "%s %s", cachecmd, cmd_args);
+- if (cmd == NULL) {
+- ret = ENOMEM;
+- goto done;
+- }
+-
+- ret = sssctl_run_command(cmd);
+-
+-done:
+- talloc_free(cmd_args);
+- talloc_free(cmd);
++ ret = sssctl_run_command(args);
+
++ talloc_free(args);
+ return ret;
+ }
+diff --git a/src/tools/sssctl/sssctl_logs.c b/src/tools/sssctl/sssctl_logs.c
+index 9ff2be05b6..ebb2c4571c 100644
+--- a/src/tools/sssctl/sssctl_logs.c
++++ b/src/tools/sssctl/sssctl_logs.c
+@@ -31,6 +31,7 @@
+ #include <ldb.h>
+ #include <popt.h>
+ #include <stdio.h>
++#include <glob.h>
+
+ #include "util/util.h"
+ #include "tools/common/sss_process.h"
+@@ -230,6 +231,7 @@ errno_t sssctl_logs_remove(struct sss_cmdline *cmdline,
+ {
+ struct sssctl_logs_opts opts = {0};
+ errno_t ret;
++ glob_t globbuf;
+
+ /* Parse command line. */
+ struct poptOption options[] = {
+@@ -253,8 +255,20 @@ errno_t sssctl_logs_remove(struct sss_cmdline *cmdline,
+
+ sss_signal(SIGHUP);
+ } else {
++ globbuf.gl_offs = 4;
++ ret = glob(LOG_PATH"/*.log", GLOB_ERR|GLOB_DOOFFS, NULL, &globbuf);
++ if (ret != 0) {
++ DEBUG(SSSDBG_CRIT_FAILURE, "Unable to expand log files list\n");
++ return ret;
++ }
++ globbuf.gl_pathv[0] = discard_const_p(char, "truncate");
++ globbuf.gl_pathv[1] = discard_const_p(char, "--no-create");
++ globbuf.gl_pathv[2] = discard_const_p(char, "--size");
++ globbuf.gl_pathv[3] = discard_const_p(char, "0");
++
+ PRINT("Truncating log files...\n");
+- ret = sssctl_run_command("truncate --no-create --size 0 " LOG_FILES);
++ ret = sssctl_run_command((const char * const*)globbuf.gl_pathv);
++ globfree(&globbuf);
+ if (ret != EOK) {
+ ERROR("Unable to truncate log files\n");
+ return ret;
+@@ -269,8 +283,8 @@ errno_t sssctl_logs_fetch(struct sss_cmdline *cmdline,
+ void *pvt)
+ {
+ const char *file;
+- const char *cmd;
+ errno_t ret;
++ glob_t globbuf;
+
+ /* Parse command line. */
+ ret = sss_tool_popt_ex(cmdline, NULL, SSS_TOOL_OPT_OPTIONAL, NULL, NULL,
+@@ -280,13 +294,19 @@ errno_t sssctl_logs_fetch(struct sss_cmdline *cmdline,
+ return ret;
+ }
+
+- cmd = talloc_asprintf(tool_ctx, "tar -czf %s %s", file, LOG_FILES);
+- if (cmd == NULL) {
+- ERROR("Out of memory!");
++ globbuf.gl_offs = 3;
++ ret = glob(LOG_PATH"/*.log", GLOB_ERR|GLOB_DOOFFS, NULL, &globbuf);
++ if (ret != 0) {
++ DEBUG(SSSDBG_CRIT_FAILURE, "Unable to expand log files list\n");
++ return ret;
+ }
++ globbuf.gl_pathv[0] = discard_const_p(char, "tar");
++ globbuf.gl_pathv[1] = discard_const_p(char, "-czf");
++ globbuf.gl_pathv[2] = discard_const_p(char, file);
+
+ PRINT("Archiving log files into %s...\n", file);
+- ret = sssctl_run_command(cmd);
++ ret = sssctl_run_command((const char * const*)globbuf.gl_pathv);
++ globfree(&globbuf);
+ if (ret != EOK) {
+ ERROR("Unable to archive log files\n");
+ return ret;
diff --git a/recipes-security/sssd/sssd_2.5.1.bb b/recipes-security/sssd/sssd_2.5.2.bb
similarity index 97%
rename from recipes-security/sssd/sssd_2.5.1.bb
rename to recipes-security/sssd/sssd_2.5.2.bb
index 1c77480..76d6e03 100644
--- a/recipes-security/sssd/sssd_2.5.1.bb
+++ b/recipes-security/sssd/sssd_2.5.2.bb
@@ -23,9 +23,10 @@ SRC_URI = "https://github.com/SSSD/sssd/releases/download/${PV}/sssd-${PV}.tar.g
file://drop_ntpdate_chk.patch \
file://fix-ldblibdir.patch \
file://musl_fixup.patch \
+ file://CVE-2021-3621.patch \
"

-SRC_URI[sha256sum] = "ce2f5d84a3f1750093318afd27f4fd75b1e3e75f7d80fc42d21a40cc54b58ea4"
+SRC_URI[sha256sum] = "5e21b3c7b4a2f1063d0fbdd3216d29886b6eaba153b44fb5961698367f399a0f"

inherit autotools pkgconfig gettext python3-dir features_check systemd




Re: [qa-build-notification] QA notification for completed autobuilder build (yocto-3.3.3.rc2)

Teoh, Jay Shen
 

Hello all,

This is the full report for yocto-3.3.3.rc2:
https://git.yoctoproject.org/cgit/cgit.cgi/yocto-testresults-contrib/tree/?h=intel-yocto-testresults

======= Summary ========
No high milestone defects.

1 issue found

BUG id:14491 - stap.StapTest.test_stap failure


======= Bugs ========
https://bugzilla.yoctoproject.org/show_bug.cgi?id=14491

Thanks,
Jay

-----Original Message-----
From: qa-build-notification@... <qa-build-
notification@...> On Behalf Of Richard Purdie
Sent: Friday, 10 September, 2021 4:00 AM
To: <yocto@...> <yocto@...>
Cc: qa-build-notification <qa-build-notification@...>
Subject: [qa-build-notification] QA notification for completed autobuilder build
(yocto-3.3.3.rc2)

A build flagged for QA (yocto-3.3.3.rc2) was completed on the autobuilder and is
available at:


https://autobuilder.yocto.io/pub/releases/yocto-3.3.3.rc2


Build hash information:

bitbake: c7b506d11df78cfc4610db6578745eaa6220b13a
meta-agl: 60344efa7a50dc2548fc4b5d68b5ad4d60c4023a
meta-arm: ba82ea920a3a43244a0a72bd74817e2f00f4a1af
meta-aws: 171aa2cf4d12ff4877e9104b6ec46be54128e3d8
meta-gplv2: 9e119f333cc8f53bd3cf64326f826dbc6ce3db0f
meta-intel: 5c4a6b02f650a99a5ec55561443fcf880a863d19
meta-mingw: 422b96cb2b6116442be1f40dfb5bd77447d1219e
meta-openembedded: 7bd7e1da9034e72ca4262dba55f70b2b23499aae
oecore: 567dd35d893c5d8969d41f263a24da8fbae3fc2f
poky: 0a2ca9d60f3851515a79d5aa9ddd8b4069b5a206



This is an automated message from the Yocto Project Autobuilder
Git: git://git.yoctoproject.org/yocto-autobuilder2
Email: richard.purdie@...







How to build yocto image with Desktop #dunfell

@prashant2314
 

Dear Team,
I need to build Rpi3 image  with yocto which includes Desktop, so how can build the image with desktop feature, so that I can use it for browsing purpose after installing one of the available browser with this image.


Re: multilib SDK

Arun
 

I see, are you talking about ${MLPREFIX} variable? The target itself compiles fine, all are 32-bit binaries in userspace. I have this issue only for SDK.


On Tue, Sep 14, 2021 at 7:21 PM Khem Raj <raj.khem@...> wrote:


On 9/14/21 6:13 PM, Arun wrote:
> The packages that SDK is trying to build are userspace packages and they
> haven't been ported for 64-bit. There are quite a few of them and short
> of fixing 64-bit compile issues for all of them, I am trying to see if I
> can build SDK without these packages built for. 64-bit. The SDK users
> will only be developing for 32-bit anyway.
>
>

you should check the dependencies and ensure they are multilib safe
sometimes dependencies could be hardcodes and cross the multilib
boundaries accidentally

>
>


Re: multilib SDK

Khem Raj
 

On 9/14/21 6:13 PM, Arun wrote:
The packages that SDK is trying to build are userspace packages and they haven't been ported for 64-bit. There are quite a few of them and short of fixing 64-bit compile issues for all of them, I am trying to see if I can build SDK without these packages built for. 64-bit. The SDK users will only be developing for 32-bit anyway.
you should check the dependencies and ensure they are multilib safe sometimes dependencies could be hardcodes and cross the multilib boundaries accidentally

3101 - 3120 of 57806