Can layer maintainers add yocto-X.Y tags for yocto-3.3 and later?


Randy MacLeod
 

Hi,


I've CCed some of the maintainers of more widely used Yocto layers
to get comments on about tagging. Please add in people who I may
have missed.


For a while now, oe-core has had a yocto-X.Y tag in addition to the
release branch name. This helps users easily find the exact commit
that corresponds to the say 3.3 GA release. There have been some
omissions in tagging but Michael and Richard are adjusting the
release process so that tagging will happen more consistently.

Most yocto layers have not adopted the tagging perhaps because they
weren't aware of it so that's why I'm writing this email. Tagging
will make it easy to find the first commit for a specific release
independent of what the branching policy of a layer is. Layer maintainers sometimes create the release branch in advance of
when oe-core is released and by adding the tag, it would make it
clear when the layer considers content to be officially released.
Of course it's up to users to decide if they are going to follow
the HEAD of a branch or, for some reason, stick with a tagged commit
or private branch off that commit.


Are there any concerns about attempting to do this for yocto-3.3
and later?

Should we make it a requirement for yocto compliance?
Should it be a feature tested by the yocto compliance script?



Here's what's in oe-core and bitbake now:
$ cd .../oe-core.git
$ git tag -l | grep yocto-3
yocto-3.0
yocto-3.1
yocto-3.1.7
yocto-3.2
yocto-3.2.1
yocto-3.3

$ cd bitbake/
$ git tag -l | grep yocto-3
yocto-3.0
yocto-3.1
yocto-3.2

--
# Randy MacLeod
# Wind River Linux


Claude Bing
 

This would be a helpful addition for us when upgrading between named
Yocto releases. Normally, when we are ready to upgrade, we checkout the
HEAD for the named branch and try to get everything working together.

Having a tagged commit for a version would (hopefully) help ensure that
all of the layers are in the same state as when it went through QA
testing. If we are upgrading at some arbitrary point after the release
was made, some layers could have additional commits that may require
changes which have not yet made it into the dependent layers.

Regards,

Claude Bing

On 4/27/21 12:48 PM, Randy MacLeod wrote:
Hi,


I've CCed some of the maintainers of more widely used Yocto layers
to get comments on about tagging. Please add in people who I may
have missed.


For a while now, oe-core has had a yocto-X.Y tag in addition to the
release branch name. This helps users easily find the exact commit
that corresponds to the say 3.3 GA release. There have been some
omissions in tagging but Michael and Richard are adjusting the
release process so that tagging will happen more consistently.

Most yocto layers have not adopted the tagging perhaps because they
weren't aware of it so that's why I'm writing this email. Tagging
will make it easy to find the first commit for a specific release
independent of what the branching policy of a layer is. Layer
maintainers sometimes create the release branch in advance of
when oe-core is released and by adding the tag, it would make it
clear when the layer considers content to be officially released.
Of course it's up to users to decide if they are going to follow
the HEAD of a branch or, for some reason, stick with a tagged commit
or private branch off that commit.


Are there any concerns about attempting to do this for yocto-3.3
and later?

Should we make it a requirement for yocto compliance?
Should it be a feature tested by the yocto compliance script?



Here's what's in oe-core and bitbake now:
$ cd .../oe-core.git
$ git tag -l | grep yocto-3
yocto-3.0
yocto-3.1
yocto-3.1.7
yocto-3.2
yocto-3.2.1
yocto-3.3

$ cd bitbake/
$ git tag -l | grep yocto-3
yocto-3.0
yocto-3.1
yocto-3.2





Khem Raj
 

On Tue, Apr 27, 2021 at 9:48 AM Randy MacLeod
<randy.macleod@...> wrote:

Hi,


I've CCed some of the maintainers of more widely used Yocto layers
to get comments on about tagging. Please add in people who I may
have missed.


For a while now, oe-core has had a yocto-X.Y tag in addition to the
release branch name. This helps users easily find the exact commit
that corresponds to the say 3.3 GA release. There have been some
omissions in tagging but Michael and Richard are adjusting the
release process so that tagging will happen more consistently.

Most yocto layers have not adopted the tagging perhaps because they
weren't aware of it so that's why I'm writing this email. Tagging
will make it easy to find the first commit for a specific release
independent of what the branching policy of a layer is. Layer
maintainers sometimes create the release branch in advance of
when oe-core is released and by adding the tag, it would make it
clear when the layer considers content to be officially released.
Of course it's up to users to decide if they are going to follow
the HEAD of a branch or, for some reason, stick with a tagged commit
or private branch off that commit.
I think this could be a good thing, although it does put the burden on
release maintainers. mostly they
test against the tip of the release branch, So if yocto project
testing is including these layers for wider
testing and can then recommend a validated commit then perhaps this
could be made viable.


Are there any concerns about attempting to do this for yocto-3.3
and later?

Should we make it a requirement for yocto compliance?
Should it be a feature tested by the yocto compliance script?



Here's what's in oe-core and bitbake now:
$ cd .../oe-core.git
$ git tag -l | grep yocto-3
yocto-3.0
yocto-3.1
yocto-3.1.7
yocto-3.2
yocto-3.2.1
yocto-3.3

$ cd bitbake/
$ git tag -l | grep yocto-3
yocto-3.0
yocto-3.1
yocto-3.2

--
# Randy MacLeod
# Wind River Linux


Armin Kuster
 

On 4/27/21 9:48 AM, Randy MacLeod wrote:
Hi,


I've CCed some of the maintainers of more widely used Yocto layers
to get comments on about tagging. Please add in people who I may
have missed.


For a while now, oe-core has had a yocto-X.Y tag in addition to the
release branch name. This helps users easily find the exact commit
that corresponds to the say 3.3 GA release. There have been some
omissions in tagging but Michael and Richard are adjusting the
release process so that tagging will happen more consistently.

Most yocto layers have not adopted the tagging perhaps because they
weren't aware of it so that's why I'm writing this email. Tagging
will make it easy to find the first commit for a specific release
independent of what the branching policy of a layer is. Layer
maintainers sometimes create the release branch in advance of
when oe-core is released and by adding the tag, it would make it
clear when the layer considers content to be officially released.
So the official starting point is what you are looking for? is there any
expectation to tag for dot release alignment?

Of course it's up to users to decide if they are going to follow
the HEAD of a branch or, for some reason, stick with a tagged commit
or private branch off that commit.
What's more important, tag or branch? Many layers hosted on git.yp.org
don't have the 'hardknott' branch.  If the discipline to create a new
branch is not their, I have a hard time believing 'tagging' will be high
on their list.


Are there any concerns about attempting to do this for yocto-3.3
and later?
Tagging in Poky has a meaning of a fully QA set of sources at a given
point of time.  It may be interpreted by users that if a tag showed up
in other layers, those layers are also fully tested.



Should we make it a requirement for yocto compliance?
I think you mean 'Yocto Compatible'.  Branching is already a requirement
IIRC as the program is against a specific branch.

-armin

Should it be a feature tested by the yocto compliance script?



Here's what's in oe-core and bitbake now:
$ cd .../oe-core.git
$ git tag -l | grep yocto-3
yocto-3.0
yocto-3.1
yocto-3.1.7
yocto-3.2
yocto-3.2.1
yocto-3.3

$ cd bitbake/
$ git tag -l | grep yocto-3
yocto-3.0
yocto-3.1
yocto-3.2


Randy MacLeod
 

Adding Robert, Hongxu and Qi who are likely interested in this topic
for future releases.

On 2021-04-27 3:03 p.m., akuster808 wrote:
On 4/27/21 9:48 AM, Randy MacLeod wrote:
Hi,


I've CCed some of the maintainers of more widely used Yocto layers
to get comments on about tagging. Please add in people who I may
have missed.


For a while now, oe-core has had a yocto-X.Y tag in addition to the
release branch name. This helps users easily find the exact commit
that corresponds to the say 3.3 GA release. There have been some
omissions in tagging but Michael and Richard are adjusting the
release process so that tagging will happen more consistently.

Most yocto layers have not adopted the tagging perhaps because they
weren't aware of it so that's why I'm writing this email. Tagging
will make it easy to find the first commit for a specific release
independent of what the branching policy of a layer is. Layer
maintainers sometimes create the release branch in advance of
when oe-core is released and by adding the tag, it would make it
clear when the layer considers content to be officially released.
So the official starting point is what you are looking for?
Yes. It's always bothered me that the tag wasn't there and now
that oe-core/bitbake/... have been doing it, it would be nice to see
other layers add the start-of-release tag. Usually, it's clear
since you can find the first commit that is unique to the branch.
It's often an update to a README or a layer.conf file that you can find
using 'git merge-base' but the tag will make it simple to locate and
in the case of meta-oe, where the branch came well before the oe-core release, the tag may not be the first commit on the 'hardknott' branch.



Is there any
expectation to tag for dot release alignment?
It would be really nice to have but it's less important to me at least.
What do you think of tagging dot upcoming releases for
meta-oe/dunfell Armin?


Of course it's up to users to decide if they are going to follow
the HEAD of a branch or, for some reason, stick with a tagged commit
or private branch off that commit.
What's more important, tag or branch? Many layers hosted on git.yp.org
don't have the 'hardknott' branch.  If the discipline to create a new
branch is not their, I have a hard time believing 'tagging' will be high
on their list.
I don't expect 100% of layers to do this but hopefully maintatiner will
listen users/submitters we'll get some traction. Also for those layers
that don't want to maintain a branch they *might* not mind adding
the tag to at least record where they were when Yocto branched.



Are there any concerns about attempting to do this for yocto-3.3
and later?
Tagging in Poky has a meaning of a fully QA set of sources at a given
point of time.  It may be interpreted by users that if a tag showed up
in other layers, those layers are also fully tested.
I suppose but once people are around for a while, they'll come to
understand how other layers don't go through the same QA cycle.



Should we make it a requirement for yocto compliance?
I think you mean 'Yocto Compatible'.
Right.

Branching is already a requirement
IIRC as the program is against a specific branch.
Yes, this would be an additional requirement or request.
The benefits that I see I've mostly already explained but
I also like having a numerical uniform string that I can us
to remember the pseudo-random branch names! :)


Thanks for the comments Armin.

../Randy

-armin

Should it be a feature tested by the yocto compliance script?



Here's what's in oe-core and bitbake now:
$ cd .../oe-core.git
$ git tag -l | grep yocto-3
yocto-3.0
yocto-3.1
yocto-3.1.7
yocto-3.2
yocto-3.2.1
yocto-3.3

$ cd bitbake/
$ git tag -l | grep yocto-3
yocto-3.0
yocto-3.1
yocto-3.2

--
# Randy MacLeod
# Wind River Linux


Randy MacLeod
 

On 2021-04-27 1:06 p.m., Khem Raj wrote:
On Tue, Apr 27, 2021 at 9:48 AM Randy MacLeod
<randy.macleod@...> wrote:
Hi,


I've CCed some of the maintainers of more widely used Yocto layers
to get comments on about tagging. Please add in people who I may
have missed.


For a while now, oe-core has had a yocto-X.Y tag in addition to the
release branch name. This helps users easily find the exact commit
that corresponds to the say 3.3 GA release. There have been some
omissions in tagging but Michael and Richard are adjusting the
release process so that tagging will happen more consistently.

Most yocto layers have not adopted the tagging perhaps because they
weren't aware of it so that's why I'm writing this email. Tagging
will make it easy to find the first commit for a specific release
independent of what the branching policy of a layer is. Layer
maintainers sometimes create the release branch in advance of
when oe-core is released and by adding the tag, it would make it
clear when the layer considers content to be officially released.
Of course it's up to users to decide if they are going to follow
the HEAD of a branch or, for some reason, stick with a tagged commit
or private branch off that commit.
I think this could be a good thing, although it does put the burden on
release maintainers. mostly they
test against the tip of the release branch, So if yocto project
testing is including these layers for wider
testing and can then recommend a validated commit then perhaps this
could be made viable.

How about:

https://git.openembedded.org/meta-openembedded/commit/?id=71b546ed8595b14d29efc1e8b951f8c845ad10c4

../Randy


Are there any concerns about attempting to do this for yocto-3.3
and later?

Should we make it a requirement for yocto compliance?
Should it be a feature tested by the yocto compliance script?



Here's what's in oe-core and bitbake now:
$ cd .../oe-core.git
$ git tag -l | grep yocto-3
yocto-3.0
yocto-3.1
yocto-3.1.7
yocto-3.2
yocto-3.2.1
yocto-3.3

$ cd bitbake/
$ git tag -l | grep yocto-3
yocto-3.0
yocto-3.1
yocto-3.2

--
# Randy MacLeod
# Wind River Linux

--
# Randy MacLeod
# Wind River Linux


Armin Kuster
 

On 4/27/21 10:06 AM, Khem Raj wrote:
On Tue, Apr 27, 2021 at 9:48 AM Randy MacLeod
<randy.macleod@...> wrote:
Hi,


I've CCed some of the maintainers of more widely used Yocto layers
to get comments on about tagging. Please add in people who I may
have missed.


For a while now, oe-core has had a yocto-X.Y tag in addition to the
release branch name. This helps users easily find the exact commit
that corresponds to the say 3.3 GA release. There have been some
omissions in tagging but Michael and Richard are adjusting the
release process so that tagging will happen more consistently.

Most yocto layers have not adopted the tagging perhaps because they
weren't aware of it so that's why I'm writing this email. Tagging
will make it easy to find the first commit for a specific release
independent of what the branching policy of a layer is. Layer
maintainers sometimes create the release branch in advance of
when oe-core is released and by adding the tag, it would make it
clear when the layer considers content to be officially released.
Of course it's up to users to decide if they are going to follow
the HEAD of a branch or, for some reason, stick with a tagged commit
or private branch off that commit.
I think this could be a good thing, although it does put the burden on
release maintainers. mostly they
test against the tip of the release branch, So if yocto project
testing is including these layers for wider
testing and can then recommend a validated commit then perhaps this
could be made viable.
That could open up a can of worms as who will fix the QA test failures?

-armin



Are there any concerns about attempting to do this for yocto-3.3
and later?

Should we make it a requirement for yocto compliance?
Should it be a feature tested by the yocto compliance script?



Here's what's in oe-core and bitbake now:
$ cd .../oe-core.git
$ git tag -l | grep yocto-3
yocto-3.0
yocto-3.1
yocto-3.1.7
yocto-3.2
yocto-3.2.1
yocto-3.3

$ cd bitbake/
$ git tag -l | grep yocto-3
yocto-3.0
yocto-3.1
yocto-3.2

--
# Randy MacLeod
# Wind River Linux


Randy MacLeod
 

On 2021-04-29 5:22 p.m., akuster808 wrote:
On 4/27/21 10:06 AM, Khem Raj wrote:
On Tue, Apr 27, 2021 at 9:48 AM Randy MacLeod
<randy.macleod@...> wrote:
Hi,


I've CCed some of the maintainers of more widely used Yocto layers
to get comments on about tagging. Please add in people who I may
have missed.


For a while now, oe-core has had a yocto-X.Y tag in addition to the
release branch name. This helps users easily find the exact commit
that corresponds to the say 3.3 GA release. There have been some
omissions in tagging but Michael and Richard are adjusting the
release process so that tagging will happen more consistently.

Most yocto layers have not adopted the tagging perhaps because they
weren't aware of it so that's why I'm writing this email. Tagging
will make it easy to find the first commit for a specific release
independent of what the branching policy of a layer is. Layer
maintainers sometimes create the release branch in advance of
when oe-core is released and by adding the tag, it would make it
clear when the layer considers content to be officially released.
Of course it's up to users to decide if they are going to follow
the HEAD of a branch or, for some reason, stick with a tagged commit
or private branch off that commit.
I think this could be a good thing, although it does put the burden on
release maintainers. mostly they
test against the tip of the release branch, So if yocto project
testing is including these layers for wider
testing and can then recommend a validated commit then perhaps this
could be made viable.
That could open up a can of worms as who will fix the QA test failures?
Hi Armin,

The community will fix bugs on HEAD just like before.
It's a tag indicating where the branch was at GA time.
There's no obligation for he maintainer do do anything in addition
to what they were doing if we didn't have a tag.

I guess that could be made clear in the README if needed.

../Randy

-armin


Are there any concerns about attempting to do this for yocto-3.3
and later?

Should we make it a requirement for yocto compliance?
Should it be a feature tested by the yocto compliance script?



Here's what's in oe-core and bitbake now:
$ cd .../oe-core.git
$ git tag -l | grep yocto-3
yocto-3.0
yocto-3.1
yocto-3.1.7
yocto-3.2
yocto-3.2.1
yocto-3.3

$ cd bitbake/
$ git tag -l | grep yocto-3
yocto-3.0
yocto-3.1
yocto-3.2

--
# Randy MacLeod
# Wind River Linux

--
# Randy MacLeod
# Wind River Linux


Armin Kuster
 

On 4/29/21 12:37 PM, Randy MacLeod wrote:
On 2021-04-27 1:06 p.m., Khem Raj wrote:
On Tue, Apr 27, 2021 at 9:48 AM Randy MacLeod
<randy.macleod@...> wrote:
Hi,


I've CCed some of the maintainers of more widely used Yocto layers
to get comments on about tagging. Please add in people who I may
have missed.


For a while now, oe-core has had a yocto-X.Y tag in addition to the
release branch name. This helps users easily find the exact commit
that corresponds to the say 3.3 GA release. There have been some
omissions in tagging but Michael and Richard are adjusting the
release process so that tagging will happen more consistently.

Most yocto layers have not adopted the tagging perhaps because they
weren't aware of it so that's why I'm writing this email. Tagging
will make it easy to find the first commit for a specific release
independent of what the branching policy of a layer is. Layer
maintainers sometimes create the release branch in advance of
when oe-core is released and by adding the tag, it would make it
clear when the layer considers content to be officially released.
Of course it's up to users to decide if they are going to follow
the HEAD of a branch or, for some reason, stick with a tagged commit
or private branch off that commit.
I think this could be a good thing, although it does put the burden on
release maintainers. mostly they
test against the tip of the release branch,  So if yocto project
testing is including these layers for wider
testing and can then recommend a validated commit then perhaps this
could be made viable.

How about:

https://git.openembedded.org/meta-openembedded/commit/?id=71b546ed8595b14d29efc1e8b951f8c845ad10c4
The implication here is that the Yocto Project has run QA if this is in
response to Khem's statement above, Or am I miss interpreting your
recommendation?


Now regarding meta-security, I would not use a "yocto" named tag.   I am
not  a fan of an upstream Project telling me to use their tagging
scheme. If I do that, then I need to be open to WindRriver, MontaVista,
Petalinux, Mentor, Enea, Arm and  etc tags.  Those Companies send me
patches.  Does RedHat tell the kernel.org to use their tags? No, its the
other way around.

If I would tag meta-security, I would have to write up the meaning of it
and possible a policy/process around it so if a new maintainer came
along they could  continue that process or do something else. This is a
hard sell as I am not seeing the benefit to this layer in adopting a
tagging scheme.

- Armin


../Randy


Are there any concerns about attempting to do this for yocto-3.3
and later?

Should we make it a requirement for yocto compliance?
Should it be a feature tested by the yocto compliance script?



Here's what's in oe-core and bitbake now:
$ cd .../oe-core.git
$ git tag -l | grep yocto-3
yocto-3.0
yocto-3.1
yocto-3.1.7
yocto-3.2
yocto-3.2.1
yocto-3.3

$ cd bitbake/
$ git tag -l | grep yocto-3
yocto-3.0
yocto-3.1
yocto-3.2

--
# Randy MacLeod
# Wind River Linux


Martin Jansa
 

On Thu, Apr 29, 2021 at 06:05:30PM -0400, Randy MacLeod wrote:
Most yocto layers have not adopted the tagging perhaps because they
weren't aware of it so that's why I'm writing this email. Tagging
will make it easy to find the first commit for a specific release
independent of what the branching policy of a layer is. Layer
maintainers sometimes create the release branch in advance of
when oe-core is released and by adding the tag, it would make it
clear when the layer considers content to be officially released.
Of course it's up to users to decide if they are going to follow
the HEAD of a branch or, for some reason, stick with a tagged commit
or private branch off that commit.
I think this could be a good thing, although it does put the burden on
release maintainers. mostly they
test against the tip of the release branch, So if yocto project
testing is including these layers for wider
testing and can then recommend a validated commit then perhaps this
could be made viable.
That could open up a can of worms as who will fix the QA test failures?
Hi Armin,

The community will fix bugs on HEAD just like before.
It's a tag indicating where the branch was at GA time.
There's no obligation for he maintainer do do anything in addition
to what they were doing if we didn't have a tag.

I guess that could be made clear in the README if needed.
To be honest I don't see much value of these tags for layers not
included in the QA testing with oe-core release.

I believe we agreed on promoting latest revisions in the branches for
selected release.

And I think the process for stable branches works reasonably well in the
layers I usually use and only very rarely new regression is introduced
in stable branch so from my experience the latest revision is always better
than GA tag (or any tag for various point releases).

Why should anyone use yocto-3.3 tag of meta-oe if the harknott branch
has few more fixes and tip of hardknott branch was tested exactly the
same as the yocto-3.3 tag?

TLDR: Why should some low traffic layer have these tags which might point to
exactly the same commit for multiple release (when all of them
still parse, build and work correctly). Many layers don't feel the need
to create a separate branch for each release when the same master works
fine for last 3 Yocto releases. And I feel the pain when e.g. with
meta-ros the last 4 branches (dunfell, gatesgarth, hardknott, master)
are 99% identical, so whatever ROS update I merge to one of them gets
distributed across all of them and only rarely with some small
modification added only in some of them. Adding a yocto-3.3 tag there
won't make that commit any better than a tip of hardknott and it might
be actually a lot worse than tip of hardknott.

Regards,


Khem Raj
 

On Thu, Apr 29, 2021 at 3:32 PM Martin Jansa <martin.jansa@...> wrote:

On Thu, Apr 29, 2021 at 06:05:30PM -0400, Randy MacLeod wrote:
Most yocto layers have not adopted the tagging perhaps because they
weren't aware of it so that's why I'm writing this email. Tagging
will make it easy to find the first commit for a specific release
independent of what the branching policy of a layer is. Layer
maintainers sometimes create the release branch in advance of
when oe-core is released and by adding the tag, it would make it
clear when the layer considers content to be officially released.
Of course it's up to users to decide if they are going to follow
the HEAD of a branch or, for some reason, stick with a tagged commit
or private branch off that commit.
I think this could be a good thing, although it does put the burden on
release maintainers. mostly they
test against the tip of the release branch, So if yocto project
testing is including these layers for wider
testing and can then recommend a validated commit then perhaps this
could be made viable.
That could open up a can of worms as who will fix the QA test failures?
Hi Armin,

The community will fix bugs on HEAD just like before.
It's a tag indicating where the branch was at GA time.
There's no obligation for he maintainer do do anything in addition
to what they were doing if we didn't have a tag.

I guess that could be made clear in the README if needed.
To be honest I don't see much value of these tags for layers not
included in the QA testing with oe-core release.

I believe we agreed on promoting latest revisions in the branches for
selected release.

And I think the process for stable branches works reasonably well in the
layers I usually use and only very rarely new regression is introduced
in stable branch so from my experience the latest revision is always better
than GA tag (or any tag for various point releases).

Why should anyone use yocto-3.3 tag of meta-oe if the harknott branch
has few more fixes and tip of hardknott branch was tested exactly the
same as the yocto-3.3 tag?

TLDR: Why should some low traffic layer have these tags which might point to
exactly the same commit for multiple release (when all of them
still parse, build and work correctly). Many layers don't feel the need
to create a separate branch for each release when the same master works
fine for last 3 Yocto releases. And I feel the pain when e.g. with
meta-ros the last 4 branches (dunfell, gatesgarth, hardknott, master)
are 99% identical, so whatever ROS update I merge to one of them gets
distributed across all of them and only rarely with some small
modification added only in some of them. Adding a yocto-3.3 tag there
won't make that commit any better than a tip of hardknott and it might
be actually a lot worse than tip of hardknott.
I think its perhaps also that distros have mechanisms to lock layers at a given
revision whatever tooling they use ( git submod, android repo,
combotools something else)
so distro manifest at that point acts as pseudo tag and seems better
downstream way
to create a uniform tested view of that distro. Something similar is
needed for poky distro perhaps a layer manifest
which tells about revisions of tested layers with poky version X if we
desire to have more layers tested along with what it supports as of
now.
oe-core supports nodistro and its self-sufficient.


Regards,