From: Alexis Lothoré <alexis.lothore@...>

If we try to run send-qa-email to simulate past releases (for example, for
development or debugging), the execution will very likely fail because the
target revision to examine (ie: the poky revision) is too old, and as a
consequence is not contained in the testresults shallow clone anymore (because
testsresults history keeps moving forward as builds are triggered on
autobuilder). As a consequence, force-fetch the "target" revision, as it is
already done for the "base" revision

Signed-off-by: Alexis Lothoré <alexis.lothore@...>
---
scripts/send_qa_email.py | 1 +
1 file changed, 1 insertion(+)

diff --git a/scripts/send_qa_email.py b/scripts/send_qa_email.py
index 199fe4e..f8ebe34 100755
--- a/scripts/send_qa_email.py
+++ b/scripts/send_qa_email.py
@@ -75,6 +75,7 @@ def generate_regression_report(resulttool, targetrepodir, basebranch, resultdir,
running resulttool
"""
fetch_testresults(resultdir, baserevision)
+ fetch_testresults(resultdir, comparerevision)
regreport = subprocess.check_output([resulttool, "regression-git", "-B", basebranch, "--commit", baserevision, "--commit2", comparerevision, resultdir])
with open(outputdir + "/testresult-regressions-report.txt", "wb") as f:
f.write(regreport)
--
2.39.1

From: Alexis Lothoré <alexis.lothore@...>

Hello,
This is the v2 of minor patch series to ease development/debugging on
send-qa-email. There is no functional change in the patches since v1, those are
resent to fix some mess in signedoff/from fields from my email configuration.
Regards,

Alexis Lothoré (3):
scripts/send-qa-email: force current revision fetch
scripts/send-qa-email: reduce testresults shallow clone depth
scripts/send-qa-email: add target and base revisions in regression
report

scripts/send_qa_email.py | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)

--
2.39.1

On Tue, 2023-02-07 at 15:52 +0100, Alexis Lothoré wrote:

Hello Richard,

On 2/7/23 15:42, Richard Purdie wrote:

On Tue, 2023-02-07 at 15:12 +0100, Alexis Lothoré via
lists.yoctoproject.org wrote:

I assumed that no tool is currently trying to parse the regression reports, so
I chose arbitrarily the metadata format.

The reports have never been useful enough to parse so that should be
fine.

Good

@Richard : on a side note, the broken From field in my previous emails was probably
due to an accent in my name, it should be fixed now. Please let me know if you
still have issues with this, in this case I will force the --from field.

It still shows incorrectly for me. It is an issue with the bootlin mail
setup and DNS, not to do with the character encoding. My changing the
accent, it is now duplicating the signed off by lines so I'd avoid
doing that!

Well, that's a shame, sorry for the annoyance. Once the patches are approved or
lead to a v2, I will resend with "From:" forced, except if you want me to
re-send it right now.

Patches look otherwise ok to me so please resend!

Cheers,

Richard

Hello Richard,

On 2/7/23 15:42, Richard Purdie wrote:

On Tue, 2023-02-07 at 15:12 +0100, Alexis Lothoré via
lists.yoctoproject.org wrote:

I assumed that no tool is currently trying to parse the regression reports, so
I chose arbitrarily the metadata format.

The reports have never been useful enough to parse so that should be
fine.

Good

@Richard : on a side note, the broken From field in my previous emails was probably
due to an accent in my name, it should be fixed now. Please let me know if you
still have issues with this, in this case I will force the --from field.

It still shows incorrectly for me. It is an issue with the bootlin mail
setup and DNS, not to do with the character encoding. My changing the
accent, it is now duplicating the signed off by lines so I'd avoid
doing that!

Well, that's a shame, sorry for the annoyance. Once the patches are approved or
lead to a v2, I will resend with "From:" forced, except if you want me to
re-send it right now.

--
Alexis Lothoré, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

On Tue, 2023-02-07 at 15:12 +0100, Alexis Lothoré via
lists.yoctoproject.org wrote:

Hello,
this minor patch series bring minor fixes into send-qa-email to ease
development/debugging in it:
- fetching current build test results works well in CI builds because we are
sure that the build results will be in the 5 upper commits of testsresults
repository shallow clone (because it has been pushed just before preparing the
regression report). It is not true when trying to run send-qa-email offline on
older revisions (for development, debugging or improvement of scripting), so
we need to force-fetch the target revision as it is done currently for the
base revision. As a consequence, this new behaviour allows to make even more
"shallow" clones of testresults since all needed data will be force-fetched
- When downloading a test regression report on official release webserver, we do
not have information about what base have been used or what is the exact
tested revision (at least, without checking the corresponding build logs on
autobuilder). So I propose to add this information at the top of each
regression report, making it look like this:

========================== Regression report ==============================
=> Target: master (5e249ec855517765f4b99e8039cb888ffa09c211)
=> Base: 4.2_M1 (4d19594b8bdacde6d809d3f2a25cff7c5a42295e)
===========================================================================

Match: oeselftest_almalinux-9.1_qemux86-64_20221208113119
oeselftest_centos-8_qemux86-64_20230126230327
Match: oeselftest_debian-11_qemux86-64_20221208033325
oeselftest_debian-11_qemux86-64_20230126150229
[...]

I assumed that no tool is currently trying to parse the regression reports, so
I chose arbitrarily the metadata format.

The reports have never been useful enough to parse so that should be
fine.

@Richard : on a side note, the broken From field in my previous emails was probably
due to an accent in my name, it should be fixed now. Please let me know if you
still have issues with this, in this case I will force the --from field.

It still shows incorrectly for me. It is an issue with the bootlin mail
setup and DNS, not to do with the character encoding. My changing the
accent, it is now duplicating the signed off by lines so I'd avoid
doing that!

Cheers,

Richard

Current regression reports do not contain information about versions compared
when generating reports. While it is still possible to get the information by
searching the autobuilder log, it is not convenient. Moreover, future
developments will allow to generate multiple reports (with different bases for
comparison) in a single build.
As a consequence, embed target and base revisions in the report header

Signed-off-by: Alexis Lothoré <alexis.lothore@...>
Signed-off-by: Alexis Lothore <alexis.lothore@...>
---
scripts/send_qa_email.py | 4 ++++
1 file changed, 4 insertions(+)

diff --git a/scripts/send_qa_email.py b/scripts/send_qa_email.py
index b87e822..e2d7284 100755
--- a/scripts/send_qa_email.py
+++ b/scripts/send_qa_email.py
@@ -78,6 +78,10 @@ def generate_regression_report(resulttool, targetrepodir, basebranch, resultdir,
fetch_testresults(resultdir, comparerevision)
regreport = subprocess.check_output([resulttool, "regression-git", "-B", basebranch, "--commit", baserevision, "--commit2", comparerevision, resultdir])
with open(outputdir + "/testresult-regressions-report.txt", "wb") as f:
+ f.write(str.encode("========================== Regression report ==============================\n"))
+ f.write(str.encode(f'{"=> Target:": <16}{basebranch: <16}({comparerevision})\n'))
+ f.write(str.encode(f'{"=> Base:": <16}{baseversion: <16}({baserevision})\n'))
+ f.write(str.encode("===========================================================================\n\n"))
f.write(regreport)
except subprocess.CalledProcessError as e:
error = str(e)
--
2.39.1

Since we are now force-fetching base revisions and target revisions for
regression report generation, we can make testresults clone even more "shallow"
to increase clone speed in CI pipelines

Signed-off-by: Alexis Lothoré <alexis.lothore@...>
Signed-off-by: Alexis Lothore <alexis.lothore@...>
---
scripts/send_qa_email.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/scripts/send_qa_email.py b/scripts/send_qa_email.py
index f8ebe34..b87e822 100755
--- a/scripts/send_qa_email.py
+++ b/scripts/send_qa_email.py
@@ -139,10 +139,10 @@ def send_qa_email():
elif basebranch:
cloneopts = ["--branch", basebranch]
try:
- subprocess.check_call(["git", "clone", "git@...:yocto-testresults", tempdir, "--depth", "5"] + cloneopts)
+ subprocess.check_call(["git", "clone", "git@...:yocto-testresults", tempdir, "--depth", "1"] + cloneopts)
except subprocess.CalledProcessError:
print("No comparision branch found, falling back to master")
- subprocess.check_call(["git", "clone", "git@...:yocto-testresults", tempdir, "--depth", "5"])
+ subprocess.check_call(["git", "clone", "git@...:yocto-testresults", tempdir, "--depth", "1"])

# If the base comparision branch isn't present regression comparision won't work
# at least until we can tell the tool to ignore internal branch information
--
2.39.1

If we try to run send-qa-email to simulate past releases (for example, for
development or debugging), the execution will very likely fail because the
target revision to examine (ie: the poky revision) is too old, and as a
consequence is not contained in the testresults shallow clone anymore (because
testsresults history keeps moving forward as builds are triggered on
autobuilder). As a consequence, force-fetch the "target" revision, as it is
already done for the "base" revision

Signed-off-by: Alexis Lothoré <alexis.lothore@...>
Signed-off-by: Alexis Lothore <alexis.lothore@...>
---
scripts/send_qa_email.py | 1 +
1 file changed, 1 insertion(+)

diff --git a/scripts/send_qa_email.py b/scripts/send_qa_email.py
index 199fe4e..f8ebe34 100755
--- a/scripts/send_qa_email.py
+++ b/scripts/send_qa_email.py
@@ -75,6 +75,7 @@ def generate_regression_report(resulttool, targetrepodir, basebranch, resultdir,
running resulttool
"""
fetch_testresults(resultdir, baserevision)
+ fetch_testresults(resultdir, comparerevision)
regreport = subprocess.check_output([resulttool, "regression-git", "-B", basebranch, "--commit", baserevision, "--commit2", comparerevision, resultdir])
with open(outputdir + "/testresult-regressions-report.txt", "wb") as f:
f.write(regreport)
--
2.39.1

Hello,
this minor patch series bring minor fixes into send-qa-email to ease
development/debugging in it:
- fetching current build test results works well in CI builds because we are
sure that the build results will be in the 5 upper commits of testsresults
repository shallow clone (because it has been pushed just before preparing the
regression report). It is not true when trying to run send-qa-email offline on
older revisions (for development, debugging or improvement of scripting), so
we need to force-fetch the target revision as it is done currently for the
base revision. As a consequence, this new behaviour allows to make even more
"shallow" clones of testresults since all needed data will be force-fetched
- When downloading a test regression report on official release webserver, we do
not have information about what base have been used or what is the exact
tested revision (at least, without checking the corresponding build logs on
autobuilder). So I propose to add this information at the top of each
regression report, making it look like this:

========================== Regression report ==============================
=> Target: master (5e249ec855517765f4b99e8039cb888ffa09c211)
=> Base: 4.2_M1 (4d19594b8bdacde6d809d3f2a25cff7c5a42295e)
===========================================================================

Match: oeselftest_almalinux-9.1_qemux86-64_20221208113119
oeselftest_centos-8_qemux86-64_20230126230327
Match: oeselftest_debian-11_qemux86-64_20221208033325
oeselftest_debian-11_qemux86-64_20230126150229
[...]

I assumed that no tool is currently trying to parse the regression reports, so
I chose arbitrarily the metadata format.

@Richard : on a side note, the broken From field in my previous emails was probably
due to an accent in my name, it should be fixed now. Please let me know if you
still have issues with this, in this case I will force the --from field.

Alexis Lothore (3):
scripts/send-qa-email: force current revision fetch
scripts/send-qa-email: reduce testresults shallow clone depth
scripts/send-qa-email: add target and base revisions in regression
report

scripts/send_qa_email.py | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)

--
2.39.1

On some systems, pod2man is not available so add native depends.

Signed-off-by: Armin Kuster <akuster808@...>
(cherry picked from commit 46622a3b26d8f37d3fd4e82d0452bbd1275d5b0c)
[Fixup for Dunfell context]
Signed-off-by: Armin Kuster <akuster808@...>
---
meta-tpm/recipes-tpm/tpm-tools/tpm-tools_1.3.9.1.bb | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta-tpm/recipes-tpm/tpm-tools/tpm-tools_1.3.9.1.bb b/meta-tpm/recipes-tpm/tpm-tools/tpm-tools_1.3.9.1.bb
index 658283f..158fd7f 100644
--- a/meta-tpm/recipes-tpm/tpm-tools/tpm-tools_1.3.9.1.bb
+++ b/meta-tpm/recipes-tpm/tpm-tools/tpm-tools_1.3.9.1.bb
@@ -9,7 +9,7 @@ SECTION = "tpm"
LICENSE = "CPL-1.0"
LIC_FILES_CHKSUM = "file://LICENSE;md5=059e8cd6165cb4c31e351f2b69388fd9"

-DEPENDS = "libtspi openssl perl"
+DEPENDS = "libtspi openssl perl-native"
DEPENDS_class-native = "trousers-native"

SRCREV = "bdf9f1bc8f63cd6fc370c2deb58d03ac55079e84"
--
2.31.1

Many for compile issue now being seen.

rpc/tcstp/.libs/libtspi_la-rpc_cmk.o:/usr/src/debug/trousers/0.3.14+gitAUTOINC+4b9a70d578-r0/build/src/tspi/../../../git/src/include/tcsd.h:169: multiple definition of `tcsd_sa_int'; .libs/libtspi_la-tspi_context.o:/usr/src/debug/trousers/0.3.14+gitAUTOINC+4b9a70d578-r0/build/src/tspi/../../../git/src/include/tcsd.h:169: first defined here
| collect2: error: ld returned 1 exit status

Signed-off-by: Armin Kuster <akuster808@...>
(cherry picked from commit 55cbb636340ed7da08a0ae338b54d72c66d41242)
Signed-off-by: Armin Kuster <akuster808@...>
---
...-security-issues-that-are-present-if.patch | 94 -------------------
meta-tpm/recipes-tpm/trousers/trousers_git.bb | 3 +-
2 files changed, 1 insertion(+), 96 deletions(-)
delete mode 100644 meta-tpm/recipes-tpm/trousers/files/0001-Correct-multiple-security-issues-that-are-present-if.patch

diff --git a/meta-tpm/recipes-tpm/trousers/files/0001-Correct-multiple-security-issues-that-are-present-if.patch b/meta-tpm/recipes-tpm/trousers/files/0001-Correct-multiple-security-issues-that-are-present-if.patch
deleted file mode 100644
index 72c81d1..0000000
--- a/meta-tpm/recipes-tpm/trousers/files/0001-Correct-multiple-security-issues-that-are-present-if.patch
+++ /dev/null
@@ -1,94 +0,0 @@
-From e74dd1d96753b0538192143adf58d04fcd3b242b Mon Sep 17 00:00:00 2001
-From: Matthias Gerstner <mgerstner@...>
-Date: Fri, 14 Aug 2020 22:14:36 -0700
-Subject: [PATCH] Correct multiple security issues that are present if the tcsd
- is started by root instead of the tss user.
-
-Patch fixes the following 3 CVEs:
-
-CVE-2020-24332
-If the tcsd daemon is started with root privileges,
-the creation of the system.data file is prone to symlink attacks
-
-CVE-2020-24330
-If the tcsd daemon is started with root privileges,
-it fails to drop the root gid after it is no longer needed
-
-CVE-2020-24331
-If the tcsd daemon is started with root privileges,
-the tss user has read and write access to the /etc/tcsd.conf file
-
-Authored-by: Matthias Gerstner <mgerstner@...>
-Signed-off-by: Debora Velarde Babb <debora@...>
-
-Upstream-Status: Backport
-CVE: CVE-2020-24332
-CVE: CVE-2020-24330
-CVE: CVE-2020-24331
-
-Signed-off-by: Armin Kuster <akuster@...>
-
----
- src/tcs/ps/tcsps.c | 2 +-
- src/tcsd/svrside.c | 1 +
- src/tcsd/tcsd_conf.c | 10 +++++-----
- 3 files changed, 7 insertions(+), 6 deletions(-)
-
-Index: git/src/tcs/ps/tcsps.c
-===================================================================
---- git.orig/src/tcs/ps/tcsps.c
-+++ git/src/tcs/ps/tcsps.c
-@@ -72,7 +72,7 @@ get_file()
- }
-
- /* open and lock the file */
-- system_ps_fd = open(tcsd_options.system_ps_file, O_CREAT|O_RDWR, 0600);
-+ system_ps_fd = open(tcsd_options.system_ps_file, O_CREAT|O_RDWR|O_NOFOLLOW, 0600);
- if (system_ps_fd < 0) {
- LogError("system PS: open() of %s failed: %s",
- tcsd_options.system_ps_file, strerror(errno));
-Index: git/src/tcsd/svrside.c
-===================================================================
---- git.orig/src/tcsd/svrside.c
-+++ git/src/tcsd/svrside.c
-@@ -473,6 +473,7 @@ main(int argc, char **argv)
- }
- return TCSERR(TSS_E_INTERNAL_ERROR);
- }
-+ setgid(pwd->pw_gid);
- setuid(pwd->pw_uid);
- #endif
- #endif
-Index: git/src/tcsd/tcsd_conf.c
-===================================================================
---- git.orig/src/tcsd/tcsd_conf.c
-+++ git/src/tcsd/tcsd_conf.c
-@@ -743,7 +743,7 @@ conf_file_init(struct tcsd_config *conf)
- #ifndef SOLARIS
- struct group *grp;
- struct passwd *pw;
-- mode_t mode = (S_IRUSR|S_IWUSR);
-+ mode_t mode = (S_IRUSR|S_IWUSR|S_IRGRP);
- #endif /* SOLARIS */
- TSS_RESULT result;
-
-@@ -798,15 +798,15 @@ conf_file_init(struct tcsd_config *conf)
- }
-
- /* make sure user/group TSS owns the conf file */
-- if (pw->pw_uid != stat_buf.st_uid || grp->gr_gid != stat_buf.st_gid) {
-+ if (stat_buf.st_uid != 0 || grp->gr_gid != stat_buf.st_gid) {
- LogError("TCSD config file (%s) must be user/group %s/%s", tcsd_config_file,
-- TSS_USER_NAME, TSS_GROUP_NAME);
-+ "root", TSS_GROUP_NAME);
- return TCSERR(TSS_E_INTERNAL_ERROR);
- }
-
-- /* make sure only the tss user can manipulate the config file */
-+ /* make sure only the tss user can read (but not manipulate) the config file */
- if (((stat_buf.st_mode & 0777) ^ mode) != 0) {
-- LogError("TCSD config file (%s) must be mode 0600", tcsd_config_file);
-+ LogError("TCSD config file (%s) must be mode 0640", tcsd_config_file);
- return TCSERR(TSS_E_INTERNAL_ERROR);
- }
- #endif /* SOLARIS */
diff --git a/meta-tpm/recipes-tpm/trousers/trousers_git.bb b/meta-tpm/recipes-tpm/trousers/trousers_git.bb
index 95e821b..992e7f2 100644
--- a/meta-tpm/recipes-tpm/trousers/trousers_git.bb
+++ b/meta-tpm/recipes-tpm/trousers/trousers_git.bb
@@ -6,7 +6,7 @@ SECTION = "security/tpm"

DEPENDS = "openssl"

-SRCREV = "4b9a70d5789b0b74f43957a6c19ab2156a72d3e0"
+SRCREV = "e74dd1d96753b0538192143adf58d04fcd3b242b"
PV = "0.3.14+git${SRCPV}"

SRC_URI = " \
@@ -16,7 +16,6 @@ SRC_URI = " \
file://tcsd.service \
file://get-user-ps-path-use-POSIX-getpwent-instead-of-getpwe.patch \
file://0001-build-don-t-override-localstatedir-mandir-sysconfdir.patch \
- file://0001-Correct-multiple-security-issues-that-are-present-if.patch \
"

S = "${WORKDIR}/git"
--
2.31.1

Signed-off-by: Armin Kuster <akuster808@...>
(cherry picked from commit 3cd65d96c1e880d378eab070ad6ae164525b78c6)
[Buf fix update only also fixes a build issue
Fixes tpm12/tpm_nvfile.c:347:27: error: ‘%02lx’ directive writing between 2 and 8 bytes into a region of size between 0 and 4095 [-Werror=format-overflow=]
]
Signed-off-by: Armin Kuster <akuster808@...>
---
.../recipes-tpm/libtpm/{libtpm_0.7.0.bb => libtpm_0.7.2.bb} | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
rename meta-tpm/recipes-tpm/libtpm/{libtpm_0.7.0.bb => libtpm_0.7.2.bb} (86%)

diff --git a/meta-tpm/recipes-tpm/libtpm/libtpm_0.7.0.bb b/meta-tpm/recipes-tpm/libtpm/libtpm_0.7.2.bb
similarity index 86%
rename from meta-tpm/recipes-tpm/libtpm/libtpm_0.7.0.bb
rename to meta-tpm/recipes-tpm/libtpm/libtpm_0.7.2.bb
index 4588c8d..0ade01d 100644
--- a/meta-tpm/recipes-tpm/libtpm/libtpm_0.7.0.bb
+++ b/meta-tpm/recipes-tpm/libtpm/libtpm_0.7.2.bb
@@ -2,8 +2,8 @@ SUMMARY = "LIBPM - Software TPM Library"
LICENSE = "BSD-3-Clause"
LIC_FILES_CHKSUM = "file://LICENSE;md5=e73f0786a936da3814896df06ad225a9"

-SRCREV = "c26e8f7b08b19a69cea9e8f1f1e6639c7951fb01"
-SRC_URI = "git://github.com/stefanberger/libtpms.git;branch=stable-${PV}"
+SRCREV = "7325acb4777f70419fe10a1d9621c2666e977e73"
+SRC_URI = "git://github.com/stefanberger/libtpms.git;branch=stable-0.7.0"

PE = "1"

--
2.31.1

Hi NIKHIL.

The name of the license file has been changed to accommodate SPDX.
The part of the recipe that specifies the license file with the warning needs to be replaced with the old name listed in the update history at the URL below.
https://git.yoctoproject.org/poky/commit/meta/files/common-licenses?id=2456f523cfbbae0e509797a0aefa9733f2cb13e3

--
Takayasu Ito
Solution Department, Lineo Solutions, Inc.
https://www.lineo.co.jp/english/
Email: ito@...
Yocto Project Ambassador

Hi Joel,

thanks for creating the issue.

Does the solution from Aleksandr work for you with raspberrypi4-64? The funny thing is for me it doesn't. As I mentioned above if I use his solution:

for MACHINE = "raspberrypi4-64" in local.conf, the .config file has following configs:
CONFIG_HAVE_PREEMPT_LAZY=y
CONFIG_PREEMPT_NONE=y
# CONFIG_PREEMPT_VOLUNTARY is not set
# CONFIG_PREEMPT is not set

If in the local.conf I replace the machine to MACHINE = "raspberrypi4" and I bitbake again, the .config file looks as follow:
CONFIG_HAVE_PREEMPT_LAZY=y
CONFIG_PREEMPT_LAZY=y
# CONFIG_PREEMPT_NONE is not set
# CONFIG_PREEMPT_VOLUNTARY is not set
# CONFIG_PREEMPT is not set
CONFIG_PREEMPT_RT=y
CONFIG_PREEMPT_COUNT=y
CONFIG_PREEMPTION=y

Meaning when the machine is set to raspberrypi4-64 and CONFIG_PREEMPT_RT is requested via the .cfg file in the file .config in work directory CONFIG_PREEMPT_NONE=y is getting set. Whereas with Machine raspberrypi4 it works as expected.

Hello Richard,
Same error occured from /build.sh as well.

Regards
Simon