Re: meta-aws warnings: SRCREV_FORMAT variable must be set when multiple SCMs are used


Jan Simon Moeller
 

So the main error is:
aws-c-auth-0.6.1-r0 do_packagedata_setscene: ExpansionError('SRCPV',
'${@bb.fetch2.get_srcrev(d)}', FetchError('The SRCREV_FORMAT variable
must be set when multiple SCMs are used.\nThe SCMs
are:\ngit://github.com/awslabs/aws-c-common.git;protocol=https;branch=main;tag=v0.6.8;destsuffix=/home/pokybuild/yocto-worker/meta-aws/build/build/tmp/work/cortexa57-poky-linux/aws-c-auth/0.6.1-r0/git/aws-c-common;name=common\ngit://github.com/awslabs/aws-c-auth.git;protocol=https;branch=main;tag=v0.6.1;destsuffix=/home/pokybuild/yocto-worker/meta-aws/build/build/tmp/work/cortexa57-poky-linux/aws-c-auth/0.6.1-r0/git/aws-c-auth;name=auth',
None))


Let's try to dissect:

The SRCREV_FORMAT variable must be set when multiple SCMs are used.
The SCMs are:
git://github.com/awslabs/aws-c-common.git;protocol=https;branch=main;tag=v0.6.8;destsuffix=/home/pokybuild/yocto-worker/meta-aws/build/build/tmp/work/cortexa57-poky-linux/aws-c-auth/0.6.1-r0/git/aws-c-common;name=common
git://github.com/awslabs/aws-c-auth.git;protocol=https;branch=main;tag=v0.6.1;destsuffix=/home/pokybuild/yocto-worker/meta-aws/build/build/tmp/work/cortexa57-poky-linux/aws-c-auth/0.6.1-r0/git/aws-c-auth;name=auth


I read this like:
- we're using two git:// url's in the SRC_URI
- bitbake needs one single hash value for SRCREV for the recipe to be 'uniq'
- we cannot decide what to use as SRCREV based on this SRC_URI (with
multiple git:// entries)

I found in meta-arm:
SRC_URI = "git://git.trustedfirmware.org/TF-A/trusted-firmware-a.git;protocol=https;name=tfa"
# Use TF-A for version
SRCREV_FORMAT = "tfa"
# ^^^^^^^^^^^^^^^^^^^^^^^^ !!!!!!!! tfa = name from above
# TF-A v2.1
SRCREV_tfa = "e1286bdb968ee74fc52f96cf303a4218e1ae2950"
#
# mbed TLS source
# Those are used in trusted-firmware-a.inc if TFA_MBEDTLS is set to 1
SRC_URI_MBEDTLS =
"git://github.com/ARMmbed/mbedtls.git;name=mbedtls;protocol=https;destsuffix=git/mbedtls;branch=mbedtls-2.16"
SRCREV_mbedtls =
"d81c11b8ab61fd5b2da8133aa73c5fe33a0633eb"


So to apply this to meta-aws, you'd have to pick one which should be
the canonical SRCREV of the two for this recipe.
E.g. (untested)

aws-c-auth_0.6.1.bb :
[...]
TAG ?= "v${PV}"
TAG_COMMON ?= "v0.6.8"
SRC_URI = "git://github.com/awslabs/aws-c-common.git;protocol=https;branch=${BRANCH};tag=${TAG_COMMON};destsuffix=${S}/aws-c-common;name=common
\
git://github.com/awslabs/aws-c-auth.git;protocol=https;branch=${BRANCH};tag=${TAG};destsuffix=${S}/aws-c-auth;name=auth
\
"
S= "${WORKDIR}/git"
BRANCH ?= "main"
# _____________________
SRCREV_FORMAT = "auth"
# ^^^^^^^^^^^^^^^^^^^^



But the main question here is: why do you need aws-c-common in SRC_URI
? . aws-c-common has its own recipe, so if it installs the headers
properly it can be in DEPENDS and will be pulled automagically.

The optimization wrt tag and network / offline that Denis mentioned is
important imho as well ! It is really better to use a SRCREV. But yes,
it might affect workflow and means bumping SRCREV's.

TLDR:
a) If possible make aws-c-common a DEPENDS in the auth recipe
b) if above does not work, try SRCREV_FORMAT = "auth" (other recipes likewise)
c) tag should be optimized to a SRCREV to avoid the enforced network fetch


HTH.
Best,
Jan-Simon

------
Jan-Simon Möller
AGL Release Manager
The Linux Foundation

Visit us at:
www.automotivegradelinux.org
lists.automotivelinux.org
www.linuxfoundation.org

Best regards,
Jan-Simon

------
Jan-Simon Möller
AGL Release Manager
The Linux Foundation

Visit us at:
www.automotivegradelinux.org
lists.automotivelinux.org
www.linuxfoundation.org


On Thu, Feb 3, 2022 at 3:12 PM Alexandre Belloni via
lists.yoctoproject.org
<alexandre.belloni=bootlin.com@...> wrote:


On 03/02/2022 14:02:54+0000, elberger via lists.yoctoproject.org wrote:
The choice to use tags is a deliberate one.

For all AWS repos that meta-aws builds recipes we use release tagging to align with a specific release. While tags can move (aka the old school CVS floating label practice which is very frowned upon except for noting "stable") they do not for these repos. Of course, there is the exception when a release is about to be made and a late breaking change must be added to the release. Then the tag shifts to meet that demand. This happens very rarely and is not a contributor to the general concern of repo maintainers implementing the floating tag/label practice and the recipe using that tag.

Further, I don't think such a check in yocto-check-layer is appropriate. Rather, if a recipe author knows that a specific CM practice is in place for particular repos then have the opportunity to tell the various stages "don't check, I know what I'm doing". If YP/OE really is against using the tag mechanism then I suggest we deprecate it. I have found it very nice for maintainership when we know exactly how the development team handles the CM.
The main drawback is that then you are not able to build with
BB_NO_NETWORK = 1 and so there is no way to ensure you have locally all
the code you need to reproduce the build in case Amazon gets out of
business one day ;)

-----Original Message-----
From: Denys Dmytriyenko <denis@...>
Sent: Wednesday, February 2, 2022 6:17 PM
To: swat@...
Cc: Glimsdale, Nathan <glimsdal@...>; Elberger, Richard <elberger@...>
Subject: RE: [EXTERNAL] [swat] meta-aws warnings: SRCREV_FORMAT variable must be set when multiple SCMs are used

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.



On Wed, Feb 02, 2022 at 11:04:57PM +0000, Richard Purdie wrote:
On Wed, 2022-02-02 at 23:54 +0100, Alexandre Belloni via
lists.yoctoproject.org
wrote:
On 02/02/2022 22:24:26+0000, Glimsdale, Nathan wrote:
I see this intermittently when building "aws-c-iot" directly. 3 of
5 attempts so far. The network restriction is also affected some
of the dependencies consistently.
Ah, interesting. Will you be tracking that on your side or should I
open a bug on bugzilla.yp.org?
I did just have a look at the aws-c-iot recipes:

https://git.yoctoproject.org/meta-aws/tree/recipes-sdk/aws-c-iot/aws-c
-iot_0.0.2.bb
https://git.yoctoproject.org/meta-aws/tree/recipes-sdk/aws-c-iot/aws-c
-iot_0.0.7.bb

and I notice they use git urls with tags.

This does not do what people expect since bitbake resolves the tag
each time over the network since people can change the tags (and do!).
I suspect that is why it is this recipe which is failing.
Ouch, no SRCREVs?

I'm guessing we need to extend yocto-check-layer with another test? :)

--
Denys




--
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com




Join swat@lists.yoctoproject.org to automatically receive all group messages.