Re: git fetcher: does not execute git fetch --tags or similar when HEAD has not changed


Matthias Schoepfer
 

Hi Alexander,

thanks for your message. I will try to explain my issue once again, since it still seems not clearly formulated. We build our images on CI (Jenkins). There, we have a shared download dir, so that we do not have to fetch all sources on every build. This gets shared for our "experimental" builds, that use latest git master HEAD (yes, evil AUTOREV) and our "stable" builds, where only tagged (i.e. no AUTOREV versions) are used. Our software uses the git tag to create a version. So, when on experimental, you might get something like:

# ourCoolSoftware --version
This is ourCoolSoftware version 1.2.3-3 (gabcdef1)

What happens is, that people check their changes on the experimental build. After testing, it is considered for the next release, so they attach a version tag to the git repository and update the _<version>.bb accordingly. Now they build the image with the "stable" profile. But the hash "abcdef1" has not changed. Hence, although we pushed a new tag, yocto git fetcher does not update the cached download git, since git hashes match (same with SRCREV). Which means, even tough we have a recipe ourcoolsoftware_1.2.4.bb with SRCREV="abcdef1..." which will build the correct hash, our software checks the local repository, which is missing the last tag and will still be reporting a wrong version 1.2.3-3 for example.

If we were to do a
bitbake -c cleanall ourcoolsoftware && bitbake ourcoolimage

then we get
# ourCoolSoftware --version
This is ourCoolSoftware version 1.2.4 (gabcdef1)

because we artificially force a complete fetch of the repository.

We usually do a clean build (no sstate / tmp / etc.) but we use the download dir, because we do not see why we would need to download gcc etc. every time. A workaround would be to bitbake -c cleanall our packages, but it seems hacky and prone to error (need to keep lists of our used software in at least two places). My question is therefore, if this is a "bug" and if we were to investigate this and propose a patch, if this could this be considered for merge, or if this behaviour is intentional and adding a fetch --tags would be considered harmful in any way.

Hope, I could bring my point across now.

Thanks and Regards,

Matthias

On 3/16/20 9:43 AM, Alexander Kanavin wrote:
'devtool upgrade' updates SRCREV to match the version tag that you specify:
devtool upgrade -V <tag>
If you omit -V, it will upgrade to the latest tag. It updates PV as well. Why is it unsuitable for you?
Alex
On Mon, 16 Mar 2020 at 09:35, Matthias Schöpfer <matthias.schoepfer@... <mailto:matthias.schoepfer@...>> wrote:
Hi Alexander,
does it solve the Problem of a SRCREV, that has been fetched (for
example for a _git testbuild) and now gets a tag applied and build with
the _<version> build. The problem is *not* that the yocto PV package is
wrong *nor* that the wrong version of the software gets build! The
problem is, that the tag does not get fetched and our software gets its
version from the git tags. Since this is very convenient to have the
version information in one place only, we will not change this.
Thanks and Regards,
   Matthias
On 3/15/20 5:02 PM, Alexander Kanavin wrote:
> On Sun, 15 Mar 2020 at 15:04, Matthias Schoepfer via
> Lists.Yoctoproject.Org <http://Lists.Yoctoproject.Org>
<http://Lists.Yoctoproject.Org>
> <matthias.schoepfer=googlemail.com@...
<mailto:googlemail.com@...>
> <mailto:googlemail.com@...
<mailto:googlemail.com@...>>> wrote:
>
>     2) It allows for very convenient bump of versions. We develop a
>     product,
>     so, version bumps are on daily or weekly basis. All that is
needed
>     is to
>     rename the .bb file. No need to edit SRCREV.
>
>
> You can trivially script SRCREV updates ('devtool upgrade'/'devtool
> finish'). The disadvantage of using tags is that you lose
> reproducibility, as tags can be moved or deleted, and so I wouldn't
> encourage this practice.
>
> Alex
--
Dr.-Ing. Matthias Schoepfer
Mobile: +49 175 2062739
email: matthias.schoepfer@...
<mailto:matthias.schoepfer@...>
--

Dr.-Ing. Matthias Schoepfer
Mobile: +49 175 2062739
email: matthias.schoepfer@...

Join {yocto@lists.yoctoproject.org to automatically receive all group messages.