Date   

Re: On managing debug and production builds

tomzy
 

Thanks Tomasz. I will check kas.
No problem
Yes, for selecting some of the packages I have created prod and debug image
recipes.But this did not work for the kernel as the kernel recipe is picked
as part of PROVIDERin machine conf.

What are the difference there? You want to use different config on prod and debug images?
Maybe add it as config fragments? Then you would need to add some global variable to
distinguish when use given .cfg file.

[1] https://docs.yoctoproject.org/singleindex.html#creating-configuration-fragments

SoI had to use 2 conf to have the
IMAGE_FEATURES (orany other var)set differently for prod and debug. This is for
building the kernelrecipie differently for prodand debug. Setting the
IMAGE_FEATURES in the image recipe (and not inconf) causes2 problems. One is
that kernel and other bootloaders recipes arepicked early via PROVIDER in conf
and not as packages included in image recipe.
Is that a problem?
Secondly,setting the var in the
image recipe breaks this command for e.g.
"bitbake base-image-prod.bbbase-image-debug.bb".

Didn't you want to distinguish this to builds to be able to run `bitbake base-image-prod` or
`bitbake base-image-debug`?

Since the command parses the recipes only once for both image creation.


Nevertheless I would greatly recommend you to use kas. In simple .yml file you
could prepare different `local.conf` per configuration prod/debug.

[2] https://kas.readthedocs.io/en/latest/userguide.html#project-configuration

Regards,
Tomasz Żyjewski
Embedded Systems Engineer
GPG: 5C495EA3EBEECA59
https://3mdeb.com | @3mdeb_com


Re: On managing debug and production builds

Alexander Kanavin
 

If you can't do what you need with only image recipes, then the only
option is to create an additional DISTRO config I'm afraid. Putting
these tweaks into machine config or local.conf is not correct.

Alex

On Tue, 1 Mar 2022 at 09:52, Vinayak Menon <menon.vinayak@...> wrote:

On Tue, Mar 1, 2022 at 12:34 PM tomzy <tomasz.zyjewski@...> wrote:

Hi Vinayak

I believe that the best way to prepare debug and prod images is to create
include file (.inc) with base set of packages and features and then add two
.bb recipes, one for prod and one for debug - every of this recipes will need
to include the `.inc` file. You could name them `base-image-prod.bb` and
`base-image-debug.bb`. This way you could run either prod or debug builds
by running `bitbake base-image-debug` or `bitbake base-image-prod`.

As for the machine, this would need to be set in local.conf every time you want
to switch the machine.To simplify this process, for the management of the
configuration of the entire build (including the selection of debug / prod image
or machine configuration), I recommend the kas[1] project with the very helpful
kas-container tool.

Thanks Tomasz. I will check kas.
Yes, for selecting some of the packages I have created prod and debug
image recipes.
But this did not work for the kernel as the kernel recipe is picked as
part of PROVIDER
in machine conf. SoI had to use 2 conf to have the IMAGE_FEATURES (or
any other var)
set differently for prod and debug. This is for building the kernel
recipie differently for prod
and debug. Setting the IMAGE_FEATURES in the image recipe (and not in
conf) causes
2 problems. One is that kernel and other bootloaders recipes are
picked early via PROVIDER
in conf and not as packages included in image recipe. Secondly,
setting the var in the image
recipe breaks this command for e.g. "bitbake base-image-prod.bb
base-image-debug.bb".
Since the command parses the recipes only once for both image creation.




[1] https://github.com/siemens/kas

Regards,
Tomasz Żyjewski
Embedded Systems Engineer
GPG: 5C495EA3EBEECA59
https://3mdeb.com | @3mdeb_com




--
vinayak



Re: On managing debug and production builds

Vinayak Menon
 

On Tue, Mar 1, 2022 at 12:34 PM tomzy <tomasz.zyjewski@...> wrote:

Hi Vinayak

I believe that the best way to prepare debug and prod images is to create
include file (.inc) with base set of packages and features and then add two
.bb recipes, one for prod and one for debug - every of this recipes will need
to include the `.inc` file. You could name them `base-image-prod.bb` and
`base-image-debug.bb`. This way you could run either prod or debug builds
by running `bitbake base-image-debug` or `bitbake base-image-prod`.

As for the machine, this would need to be set in local.conf every time you want
to switch the machine.To simplify this process, for the management of the
configuration of the entire build (including the selection of debug / prod image
or machine configuration), I recommend the kas[1] project with the very helpful
kas-container tool.

Thanks Tomasz. I will check kas.
Yes, for selecting some of the packages I have created prod and debug
image recipes.
But this did not work for the kernel as the kernel recipe is picked as
part of PROVIDER
in machine conf. SoI had to use 2 conf to have the IMAGE_FEATURES (or
any other var)
set differently for prod and debug. This is for building the kernel
recipie differently for prod
and debug. Setting the IMAGE_FEATURES in the image recipe (and not in
conf) causes
2 problems. One is that kernel and other bootloaders recipes are
picked early via PROVIDER
in conf and not as packages included in image recipe. Secondly,
setting the var in the image
recipe breaks this command for e.g. "bitbake base-image-prod.bb
base-image-debug.bb".
Since the command parses the recipes only once for both image creation.




[1] https://github.com/siemens/kas

Regards,
Tomasz Żyjewski
Embedded Systems Engineer
GPG: 5C495EA3EBEECA59
https://3mdeb.com | @3mdeb_com




--
vinayak


Re: On managing debug and production builds

tomzy
 

Hi Vinayak

I believe that the best way to prepare debug and prod images is to create
include file (.inc) with base set of packages and features and then add two
.bb recipes, one for prod and one for debug - every of this recipes will need
to include the `.inc` file. You could name them `base-image-prod.bb` and
`base-image-debug.bb`. This way you could run either prod or debug builds
by running `bitbake base-image-debug` or `bitbake base-image-prod`.

As for the machine, this would need to be set in local.conf every time you want
to switch the machine.To simplify this process, for the management of the
configuration of the entire build (including the selection of debug / prod image
or machine configuration), I recommend the kas[1] project with the very helpful
kas-container tool.

[1] https://github.com/siemens/kas

Regards,

Tomasz Żyjewski
Embedded Systems Engineer
GPG: 5C495EA3EBEECA59
https://3mdeb.com | @3mdeb_com


Re: python3-smbus no longer builds - Does anyone have an idea?

Matthias Klein
 

Hello Raj,

yes, you are right, {B} works too, and looks more logical.

Thanks for adapting my patch accordingly.

Many greetings,
Matthias


-----Ursprüngliche Nachricht-----
Von: Khem Raj <raj.khem@...>
Gesendet: Montag, 28. Februar 2022 18:08
An: Konrad Weihmann <kweihmann@...>
Cc: Matthias Klein <matthias.klein@...>; yocto@...; Tim Orling <ticotimo@...>; Richard Purdie <richard.purdie@...>
Betreff: Re: [yocto] python3-smbus no longer builds - Does anyone have an idea?

On Mon, Feb 28, 2022 at 5:10 AM Konrad Weihmann <kweihmann@...> wrote:

Hi Matthias,

you're right :(

PYPA_WHEEL = "${S}/py-smbus/dist/smbus-*-*.whl"
this should be B instead I think


finally does it

@Tim @Richard
PYPA_WHEEL doesn't respect SETUPTOOLS_SETUP_PATH, which might be
another issue to fix

BR
Konrad

On 28.02.22 14:04, Matthias Klein wrote:
Hello Konrad,

unfortunately it still does not build:

ERROR: smbus-*-*.whl is not a valid wheel filename.

Best regards,
Matthias

-----Ursprüngliche Nachricht-----
Von: Konrad Weihmann <kweihmann@...>
Gesendet: Montag, 28. Februar 2022 13:59
An: Matthias Klein <matthias.klein@...>;
yocto@...
Betreff: Re: AW: [yocto] python3-smbus no longer builds - Does anyone have an idea?

On 28.02.22 13:56, Matthias Klein wrote:
Hello Konrad,

Thanks for the quick feedback.

Have you been able to build the package with the change?
I get a similar error with it:
Dang - that's the second issue being open in this series...


ERROR: smbus-4.3-*.whl is not a valid wheel filename.
Try

PYPA_WHEEL = "${PIP_INSTALL_DIST_PATH}/smbus-*-*.whl"

instead - and it should really work


Best reagrds,
Matthias

-----Ursprüngliche Nachricht-----
Von: Konrad Weihmann <kweihmann@...>
Gesendet: Montag, 28. Februar 2022 13:51
An: Matthias Klein <matthias.klein@...>;
yocto@...
Betreff: Re: [yocto] python3-smbus no longer builds - Does anyone have an idea?

Hi Matthias,

this is cause by merged PEP-517 changes.
To make it build again just inject

PIP_INSTALL_PACKAGE = "smbus"

into the recipe or a bbappend.
Mainly the python3-prefix of the recipe, makes the name guessing
fail, I suspect patches to be incoming soon

BR
Konrad

On 28.02.22 13:45, Matthias Klein wrote:
Hello,

the python3-smbus package no longer builds.

The install step ends with the following error message:

ERROR: python3_smbus-4.3-*.whl is not a valid wheel filename.

Does anyone have an idea what is the cause?

Many greetings,

Matthias






Re: Custom Image Type

Rudolf J Streif
 

Thanks, Khem.

On 2/28/22 8:54 PM, Khem Raj wrote:
On Mon, Feb 28, 2022 at 8:32 PM Rudolf J Streif
<rudolf.streif@...> wrote:
I ran into a problem with a custom image type class. I called the class
image_types_ota.bbclass.

When I try to use it I am getting these error messages for two
predefined image recipes (but only for these two, for any other image
recipe it is fine):

ERROR:
/develop/projects/tcu/build/../poky/meta/recipes-rt/images/core-image-rt-sdk.bb:
No IMAGE_CMD defined for IMAGE_FSTYPES entry 'ota' - possibly invalid
type name or missing support class
ERROR:
/develop/projects/tcu/build/../meta-openembedded/meta-networking/recipes-core/images/meta-networking-image-base.bb:
No IMAGE_CMD defined for IMAGE_FSTYPES entry 'ota' - possibly invalid
type name or missing support class
ERROR: Failed to parse recipe:
/develop/projects/tcu/build/../poky/meta/recipes-rt/images/core-image-rt-sdk.bb

I have no idea what in these two image recipes could trigger this error
message.
hard to tell without knowing how ota class is being plugged in or what
its content is.
Do you define IMAGE_FSTYPES:ota ... somewhere ?
The image class defines IMAGE_CMD. There is not much to it for now:

OTA_ROOTFS_MD5 ?= "rootfs.md5"

IMAGE_CMD:ota () {
    build_ota="${WORKDIR}/build-ota"

    # rootfs
    md5sum ${IMAGE_ROOTFS} > ${build_ota}/OTA_ROOTFS_MD5
}

What I don't see is why these two are the only image recipes this is occurring with. All the other image recipes do not throw a parser error.

Thanks,
Rudi


--
Rudolf J Streif
CEO/CTO ibeeto
+1.855.442.3386 x700


On managing debug and production builds

Vinayak Menon
 

I have a requirement to create production and release builds for the
same machine. Debug build would use most of the production recipes,
except few images like Linux kernel that will have debug configs
enabled and a separate debug image is created. Also debug images will
have few extra debug packages and image features like debug-tweaks
enabled. What is the standard way of managing this ?
One other requirement is that for e.g. I do not want both debug and
production linux images to be created in one build. That increases the
build time. i.e. just adding debug variant recipes alone does not
suffice.
Also the debug and production variants are not specific to a machine,
i.e. tomorrow if I add another machine, I should be able to generate
both variants for that machine too without any machine conf changes.

What I have tried right now is to create 2 local.conf, one in
meta/conf and other in meta/conf/debug. The debug conf has an
IMAGE_FEATURE indicating that it is a debug variant. I am using the
debug-tweaks image feature as of now. And then I switch the confs with
TEMPLATECONF. Now, for recipes, I have not created separate debug
recipes. For e.g. in the linux recipe I am just looking for the
debug-tweaks feature to pick the right config fragments.

Thanks,
Vinayak


Re: Custom Image Type

Khem Raj
 

On Mon, Feb 28, 2022 at 8:32 PM Rudolf J Streif
<rudolf.streif@...> wrote:

I ran into a problem with a custom image type class. I called the class
image_types_ota.bbclass.

When I try to use it I am getting these error messages for two
predefined image recipes (but only for these two, for any other image
recipe it is fine):

ERROR:
/develop/projects/tcu/build/../poky/meta/recipes-rt/images/core-image-rt-sdk.bb:
No IMAGE_CMD defined for IMAGE_FSTYPES entry 'ota' - possibly invalid
type name or missing support class
ERROR:
/develop/projects/tcu/build/../meta-openembedded/meta-networking/recipes-core/images/meta-networking-image-base.bb:
No IMAGE_CMD defined for IMAGE_FSTYPES entry 'ota' - possibly invalid
type name or missing support class
ERROR: Failed to parse recipe:
/develop/projects/tcu/build/../poky/meta/recipes-rt/images/core-image-rt-sdk.bb

I have no idea what in these two image recipes could trigger this error
message.
hard to tell without knowing how ota class is being plugged in or what
its content is.
Do you define IMAGE_FSTYPES:ota ... somewhere ?

Thanks,
Rudi




Custom Image Type

Rudolf J Streif
 

I ran into a problem with a custom image type class. I called the class image_types_ota.bbclass.

When I try to use it I am getting these error messages for two predefined image recipes (but only for these two, for any other image recipe it is fine):

ERROR: /develop/projects/tcu/build/../poky/meta/recipes-rt/images/core-image-rt-sdk.bb: No IMAGE_CMD defined for IMAGE_FSTYPES entry 'ota' - possibly invalid type name or missing support class
ERROR: /develop/projects/tcu/build/../meta-openembedded/meta-networking/recipes-core/images/meta-networking-image-base.bb: No IMAGE_CMD defined for IMAGE_FSTYPES entry 'ota' - possibly invalid type name or missing support class
ERROR: Failed to parse recipe: /develop/projects/tcu/build/../poky/meta/recipes-rt/images/core-image-rt-sdk.bb

I have no idea what in these two image recipes could trigger this error message.

Thanks,
Rudi


Re: opkg not picking up RPROVIDES alias. #yocto #vivado

Robert Joslyn
 

On Feb 28, 2022, at 3:11 PM, Sam <samdietz2001@...> wrote:

I have a recipe aliased by RPROVIDES which I assume is working due to the project picking it up during the compile.

However, opkg does not seem to pick up the alias, returning this error:
Collected errors:
* opkg_prepare_url_for_install: Couldn't find anything to satisfy 'alias-name'.

This issue seemed to arise after upgrading Vivado to a newer version.

Any ideas?
I encountered a similar issue when updating one of my builds that uses a package feed:
https://lists.yoctoproject.org/g/yocto/topic/88553371#55926

In my case the issue was that bitbake no longer resolves recursive dependencies of the do_build task:
https://git.yoctoproject.org/poky/commit/?id=568f62214bca3ac6d35eef8d9f4562596fb4c9ab

This means if you use a command like “bitbake packagegroup-foo” to build your package feed, you won’t get all of the packages built. I just had to change how my package feed was generated by using "bitbake --runall build packagegroup-foo” to build my package feed from my packagegroup recipe.

I’m not sure if you’re seeing the same issue, but the symptoms are the same as what I saw.

Robert


opkg not picking up RPROVIDES alias. #yocto #vivado

Sam
 

I have a recipe aliased by RPROVIDES which I assume is working due to the project picking it up during the compile.

However, opkg does not seem to pick up the alias, returning this error:
Collected errors:
 * opkg_prepare_url_for_install: Couldn't find anything to satisfy 'alias-name'.

This issue seemed to arise after upgrading Vivado to a newer version.

Any ideas?
 


Re: QA notification for completed autobuilder build (yocto-3.3.5.rc1)

Teoh, Jay Shen
 

Hi Everyone,

This is the full report for yocto-3.3.5.rc1:
https://git.yoctoproject.org/cgit/cgit.cgi/yocto-testresults-contrib/tree/?h=intel-yocto-testresults

======= Summary ========
No high milestone defects.

No new issue found.


Thanks,
Jay

-----Original Message-----
From: yocto@... <yocto@...> On
Behalf Of Richard Purdie
Sent: Tuesday, 22 February, 2022 5:51 PM
To: <yocto@...> <yocto@...>
Cc: qa-build-notification <qa-build-notification@...>
Subject: [yocto] QA notification for completed autobuilder build (yocto-
3.3.5.rc1)

A build flagged for QA (yocto-3.3.5.rc1) was completed on the autobuilder
and is available at:


https://autobuilder.yocto.io/pub/releases/yocto-3.3.5.rc1


Build hash information:

bitbake: aaa7f7af23d5f89fe4a5ed48c57ea3dfca07c79d
meta-agl: 9a50bd62dfac0d6ea1320b2ee083529cb98b9f92
meta-arm: fe35ff5ba809bf4826adfe65899a84e9c99494e8
meta-aws: 6801abf40bb255a31bce5061c5c6b72f5e2a8f58
meta-gplv2: 9e119f333cc8f53bd3cf64326f826dbc6ce3db0f
meta-intel: 36e915402dfe317654568f09f18fb6f7653603bc
meta-mingw: 422b96cb2b6116442be1f40dfb5bd77447d1219e
meta-openembedded: 23598caeafce0af0dde8d1339cf5edff021f6823
oecore: 29cd1d796057ef5599fe17c39b42aa099f7b1c29
poky: 8d3e054f6d432b5ca0fcd613e0c767fab3c85f24



This is an automated message from the Yocto Project Autobuilder
Git: git://git.yoctoproject.org/yocto-autobuilder2
Email: richard.purdie@...


Re: python3-smbus no longer builds - Does anyone have an idea?

Khem Raj
 

On Mon, Feb 28, 2022 at 5:10 AM Konrad Weihmann <kweihmann@...> wrote:

Hi Matthias,

you're right :(

PYPA_WHEEL = "${S}/py-smbus/dist/smbus-*-*.whl"
this should be B instead I think


finally does it

@Tim @Richard
PYPA_WHEEL doesn't respect SETUPTOOLS_SETUP_PATH, which might be another
issue to fix

BR
Konrad

On 28.02.22 14:04, Matthias Klein wrote:
Hello Konrad,

unfortunately it still does not build:

ERROR: smbus-*-*.whl is not a valid wheel filename.

Best regards,
Matthias

-----Ursprüngliche Nachricht-----
Von: Konrad Weihmann <kweihmann@...>
Gesendet: Montag, 28. Februar 2022 13:59
An: Matthias Klein <matthias.klein@...>; yocto@...
Betreff: Re: AW: [yocto] python3-smbus no longer builds - Does anyone have an idea?

On 28.02.22 13:56, Matthias Klein wrote:
Hello Konrad,

Thanks for the quick feedback.

Have you been able to build the package with the change?
I get a similar error with it:
Dang - that's the second issue being open in this series...


ERROR: smbus-4.3-*.whl is not a valid wheel filename.
Try

PYPA_WHEEL = "${PIP_INSTALL_DIST_PATH}/smbus-*-*.whl"

instead - and it should really work


Best reagrds,
Matthias

-----Ursprüngliche Nachricht-----
Von: Konrad Weihmann <kweihmann@...>
Gesendet: Montag, 28. Februar 2022 13:51
An: Matthias Klein <matthias.klein@...>; yocto@...
Betreff: Re: [yocto] python3-smbus no longer builds - Does anyone have an idea?

Hi Matthias,

this is cause by merged PEP-517 changes.
To make it build again just inject

PIP_INSTALL_PACKAGE = "smbus"

into the recipe or a bbappend.
Mainly the python3-prefix of the recipe, makes the name guessing fail, I suspect patches to be incoming soon

BR
Konrad

On 28.02.22 13:45, Matthias Klein wrote:
Hello,

the python3-smbus package no longer builds.

The install step ends with the following error message:

ERROR: python3_smbus-4.3-*.whl is not a valid wheel filename.

Does anyone have an idea what is the cause?

Many greetings,

Matthias






[meta-security][PATCH 2/2] parsec-service: Only enable TPM is layer and DISTRO_FEATURE is defined.

Armin Kuster
 

Signed-off-by: Armin Kuster <akuster808@...>
---
.../recipes-parsec/parsec-service/parsec-service_0.8.1.bb | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/meta-parsec/recipes-parsec/parsec-service/parsec-service_0.8.1.bb b/meta-parsec/recipes-parsec/parsec-service/parsec-service_0.8.1.bb
index 1cbf2bd..3f12139 100644
--- a/meta-parsec/recipes-parsec/parsec-service/parsec-service_0.8.1.bb
+++ b/meta-parsec/recipes-parsec/parsec-service/parsec-service_0.8.1.bb
@@ -12,7 +12,12 @@ SRC_URI += "crate://crates.io/parsec-service/${PV} \

DEPENDS = "clang-native"

-PACKAGECONFIG ??= "TPM PKCS11 MBED-CRYPTO CRYPTOAUTHLIB"
+PACKAGECONFIG ??= "PKCS11 MBED-CRYPTO CRYPTOAUTHLIB"
+
+have_TPM = "${@bb.utils.contains('DISTRO_FEATURES', 'tpm2', 'TPM', '', d)}"
+PACKAGECONFIG:append = " ${@bb.utils.contains('BBFILE_COLLECTIONS', 'tpm-layer', '${have_TPM}', '', d)}"
+
+
PACKAGECONFIG[ALL] = "all-providers cryptoki/generate-bindings tss-esapi/generate-bindings,,tpm2-tss libts,libts"
PACKAGECONFIG[TPM] = "tpm-provider tss-esapi/generate-bindings,,tpm2-tss"
PACKAGECONFIG[PKCS11] = "pkcs11-provider cryptoki/generate-bindings,"
--
2.25.1


[meta-security][PATCH 1/2] layer.conf: enable apparmor for qemu machine

Armin Kuster
 

Signed-off-by: Armin Kuster <akuster808@...>
---
conf/layer.conf | 3 +++
1 file changed, 3 insertions(+)

diff --git a/conf/layer.conf b/conf/layer.conf
index 1f83593..21f03d1 100644
--- a/conf/layer.conf
+++ b/conf/layer.conf
@@ -16,3 +16,6 @@ LAYERDEPENDS_security = "core openembedded-layer perl-layer networking-layer met
# Sanity check for meta-security layer.
# Setting SKIP_META_SECURITY_SANITY_CHECK to "1" would skip the bbappend files check.
INHERIT += "sanity-meta-security"
+
+QB_KERNEL_CMDLINE_APPEND = " ${@bb.utils.contains('DISTRO_FEATURES', 'apparmor', 'apparmor=1 security=apparmor', '', d)}"
+
--
2.25.1


Re: python3-smbus no longer builds - Does anyone have an idea?

Matthias Klein
 

Hello Konrad,

Thank you very much!

The package is now building.
I have sent a corresponding patch.

Many greetings,
Matthias

-----Ursprüngliche Nachricht-----
Von: Konrad Weihmann <kweihmann@...>
Gesendet: Montag, 28. Februar 2022 14:11
An: Matthias Klein <matthias.klein@...>; yocto@...; Tim Orling <ticotimo@...>; Richard Purdie <richard.purdie@...>
Betreff: Re: AW: AW: [yocto] python3-smbus no longer builds - Does anyone have an idea?

Hi Matthias,

you're right :(

PYPA_WHEEL = "${S}/py-smbus/dist/smbus-*-*.whl"

finally does it

@Tim @Richard
PYPA_WHEEL doesn't respect SETUPTOOLS_SETUP_PATH, which might be another issue to fix

BR
Konrad

On 28.02.22 14:04, Matthias Klein wrote:
Hello Konrad,

unfortunately it still does not build:

ERROR: smbus-*-*.whl is not a valid wheel filename.

Best regards,
Matthias

-----Ursprüngliche Nachricht-----
Von: Konrad Weihmann <kweihmann@...>
Gesendet: Montag, 28. Februar 2022 13:59
An: Matthias Klein <matthias.klein@...>;
yocto@...
Betreff: Re: AW: [yocto] python3-smbus no longer builds - Does anyone have an idea?

On 28.02.22 13:56, Matthias Klein wrote:
Hello Konrad,

Thanks for the quick feedback.

Have you been able to build the package with the change?
I get a similar error with it:
Dang - that's the second issue being open in this series...


ERROR: smbus-4.3-*.whl is not a valid wheel filename.
Try

PYPA_WHEEL = "${PIP_INSTALL_DIST_PATH}/smbus-*-*.whl"

instead - and it should really work


Best reagrds,
Matthias

-----Ursprüngliche Nachricht-----
Von: Konrad Weihmann <kweihmann@...>
Gesendet: Montag, 28. Februar 2022 13:51
An: Matthias Klein <matthias.klein@...>;
yocto@...
Betreff: Re: [yocto] python3-smbus no longer builds - Does anyone have an idea?

Hi Matthias,

this is cause by merged PEP-517 changes.
To make it build again just inject

PIP_INSTALL_PACKAGE = "smbus"

into the recipe or a bbappend.
Mainly the python3-prefix of the recipe, makes the name guessing
fail, I suspect patches to be incoming soon

BR
Konrad

On 28.02.22 13:45, Matthias Klein wrote:
Hello,

the python3-smbus package no longer builds.

The install step ends with the following error message:

ERROR: python3_smbus-4.3-*.whl is not a valid wheel filename.

Does anyone have an idea what is the cause?

Many greetings,

Matthias





[yocto-autobuilder-helper][hardknott 3/3] shared-repos: Use tar instead of rsync for speed

Anuj Mittal
 

From: Richard Purdie <richard.purdie@...>

The rysnc of 20,000 files (650MB) onto the nas is slow taking ~3 minutes
at idle and worse at load. This is due to the number of files which
is a pain point for NFS. This piece of the build is also a bottleneck
since the rest of a build depends on it happening.

If we switch to zstd compressed tar, it takes 2.49s. Other compression
methods were much slower but zstd seems 'accptable' and speeds things
up too.

Signed-off-by: Richard Purdie <richard.purdie@...>
(cherry picked from commit aff49e938ee34e1fc5a2954e3e22a4ca1ae9ac7b)
Signed-off-by: Anuj Mittal <anuj.mittal@...>
---
scripts/prepare-shared-repos | 4 ++--
scripts/send-qa-email | 6 ++++--
scripts/shared-repo-unpack | 9 ++++++---
3 files changed, 12 insertions(+), 7 deletions(-)

diff --git a/scripts/prepare-shared-repos b/scripts/prepare-shared-repos
index c221e69..a5bc0da 100755
--- a/scripts/prepare-shared-repos
+++ b/scripts/prepare-shared-repos
@@ -39,5 +39,5 @@ with tempfile.TemporaryDirectory(prefix="shared-repo-temp-", dir="/home/pokybuil
if args.publish_dir:
utils.publishrepo(tempdir, repo, args.publish_dir)

- utils.printheader("Running rsync")
- subprocess.check_call("rsync -a " + tempdir + "/* " + args.sharedsrcdir, shell=True)
+ utils.printheader("Creating shared src tarball")
+ subprocess.check_call("tar -I zstd -cf " + args.sharedsrcdir.rstrip("/") + ".tar.zst ./*", shell=True, cwd=tempdir)
diff --git a/scripts/send-qa-email b/scripts/send-qa-email
index 1b69307..bc594df 100755
--- a/scripts/send-qa-email
+++ b/scripts/send-qa-email
@@ -45,9 +45,11 @@ buildtoolsdir = os.path.dirname(args.repojson) + "/build/buildtools"
if os.path.exists(buildtoolsdir):
utils.enable_buildtools_tarball(buildtoolsdir)

+repodir = os.path.dirname(args.repojson) + "/build/repos"
+
if 'poky' in repos and os.path.exists(resulttool) and args.results_dir:
# Need the finalised revisions (not 'HEAD')
- targetrepodir = "%s/poky" % (args.sharedrepodir)
+ targetrepodir = "%s/poky" % (repodir)
revision = subprocess.check_output(["git", "rev-parse", "HEAD"], cwd=targetrepodir).decode('utf-8').strip()
branch = repos['poky']['branch']
repo = repos['poky']['url']
@@ -116,7 +118,7 @@ if args.send.lower() != 'true' or not args.publish_dir or not args.release:
buildhashes = ""
for repo in sorted(repos.keys()):
# Need the finalised revisions (not 'HEAD')
- targetrepodir = "%s/%s" % (args.sharedrepodir, repo)
+ targetrepodir = "%s/%s" % (repodir, repo)
revision = subprocess.check_output(["git", "rev-parse", "HEAD"], cwd=targetrepodir).decode('utf-8').strip()
buildhashes += "%s: %s\n" % (repo, revision)

diff --git a/scripts/shared-repo-unpack b/scripts/shared-repo-unpack
index f08efa8..f7f87af 100755
--- a/scripts/shared-repo-unpack
+++ b/scripts/shared-repo-unpack
@@ -50,11 +50,14 @@ needrepos_baseddirs = [r.split('/')[0] for r in needrepos]
for repo in sorted(repos.keys()):
if repo not in needrepos_baseddirs:
continue
- targetrepodir = "%s/%s" % (targetsubdir, repo)
if args.cache_dir:
utils.printheader("Copying in repo %s" % repo)
- utils.mkdir(targetrepodir)
- subprocess.check_call(["rsync", "-a", "%s/%s" % (args.cache_dir, repo), targetsubdir])
+ utils.mkdir(targetsubdir)
+ if args.target in ["a-full", "a-quick"]:
+ # full/quick need all repo data due to send-qa-email
+ subprocess.check_call(["tar", "-I", "zstd", "-C", targetsubdir, "-xf", "%s.tar.zst" % args.cache_dir])
+ else:
+ subprocess.check_call(["tar", "-I", "zstd", "-C", targetsubdir, "-xf", "%s.tar.zst" % args.cache_dir, "./" + repo])
else:
utils.printheader("Fetching repo %s" % repo)
utils.fetchgitrepo(targetsubdir, repo, repos[repo], stashdir)
--
2.35.1


[yocto-autobuilder-helper][hardknott 2/3] prepare-shared-repos: Make it clear when rsync starts in logs

Anuj Mittal
 

From: Richard Purdie <richard.purdie@...>

Signed-off-by: Richard Purdie <richard.purdie@...>
(cherry picked from commit 8e996a95a8902b40380dd477ecb606cc969cdee9)
Signed-off-by: Anuj Mittal <anuj.mittal@...>
---
scripts/prepare-shared-repos | 1 +
1 file changed, 1 insertion(+)

diff --git a/scripts/prepare-shared-repos b/scripts/prepare-shared-repos
index 1573f85..c221e69 100755
--- a/scripts/prepare-shared-repos
+++ b/scripts/prepare-shared-repos
@@ -39,4 +39,5 @@ with tempfile.TemporaryDirectory(prefix="shared-repo-temp-", dir="/home/pokybuil
if args.publish_dir:
utils.publishrepo(tempdir, repo, args.publish_dir)

+ utils.printheader("Running rsync")
subprocess.check_call("rsync -a " + tempdir + "/* " + args.sharedsrcdir, shell=True)
--
2.35.1


[yocto-autobuilder-helper][hardknott 1/3] scripts/prepare-shared-repos: Use tmpfs for speed

Anuj Mittal
 

From: Richard Purdie <richard.purdie@...>

Signed-off-by: Richard Purdie <richard.purdie@...>
(cherry picked from commit 298a10575851d501204fe1ee0d1dcbcec37a66cd)
Signed-off-by: Anuj Mittal <anuj.mittal@...>
---
scripts/prepare-shared-repos | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/prepare-shared-repos b/scripts/prepare-shared-repos
index 8400d09..1573f85 100755
--- a/scripts/prepare-shared-repos
+++ b/scripts/prepare-shared-repos
@@ -32,7 +32,7 @@ with open(args.repojson) as f:

stashdir = utils.getconfig("REPO_STASH_DIR", ourconfig)

-with tempfile.TemporaryDirectory(prefix="shared-repo-temp-", dir="/tmp") as tempdir:
+with tempfile.TemporaryDirectory(prefix="shared-repo-temp-", dir="/home/pokybuild/tmp") as tempdir:
for repo in sorted(repos.keys()):
utils.printheader("Intially fetching repo %s" % repo)
utils.fetchgitrepo(tempdir, repo, repos[repo], stashdir)
--
2.35.1


Re: python3-smbus no longer builds - Does anyone have an idea?

Konrad Weihmann <kweihmann@...>
 

Hi Matthias,

you're right :(

PYPA_WHEEL = "${S}/py-smbus/dist/smbus-*-*.whl"

finally does it

@Tim @Richard
PYPA_WHEEL doesn't respect SETUPTOOLS_SETUP_PATH, which might be another issue to fix

BR
Konrad

On 28.02.22 14:04, Matthias Klein wrote:
Hello Konrad,
unfortunately it still does not build:
ERROR: smbus-*-*.whl is not a valid wheel filename.
Best regards,
Matthias
-----Ursprüngliche Nachricht-----
Von: Konrad Weihmann <kweihmann@...>
Gesendet: Montag, 28. Februar 2022 13:59
An: Matthias Klein <matthias.klein@...>; yocto@...
Betreff: Re: AW: [yocto] python3-smbus no longer builds - Does anyone have an idea?
On 28.02.22 13:56, Matthias Klein wrote:
Hello Konrad,

Thanks for the quick feedback.

Have you been able to build the package with the change?
I get a similar error with it:
Dang - that's the second issue being open in this series...


ERROR: smbus-4.3-*.whl is not a valid wheel filename.
Try
PYPA_WHEEL = "${PIP_INSTALL_DIST_PATH}/smbus-*-*.whl"
instead - and it should really work


Best reagrds,
Matthias

-----Ursprüngliche Nachricht-----
Von: Konrad Weihmann <kweihmann@...>
Gesendet: Montag, 28. Februar 2022 13:51
An: Matthias Klein <matthias.klein@...>; yocto@...
Betreff: Re: [yocto] python3-smbus no longer builds - Does anyone have an idea?

Hi Matthias,

this is cause by merged PEP-517 changes.
To make it build again just inject

PIP_INSTALL_PACKAGE = "smbus"

into the recipe or a bbappend.
Mainly the python3-prefix of the recipe, makes the name guessing fail, I suspect patches to be incoming soon

BR
Konrad

On 28.02.22 13:45, Matthias Klein wrote:
Hello,

the python3-smbus package no longer builds.

The install step ends with the following error message:

ERROR: python3_smbus-4.3-*.whl is not a valid wheel filename.

Does anyone have an idea what is the cause?

Many greetings,

Matthias



1501 - 1520 of 57789