do_patch failing when executed multiple times in the same S=WORKDIR Was: [yocto] Strange sporadic build issues (incremental builds in docker container)


Martin Jansa
 

On Wed, Mar 30, 2022 at 11:29 PM Trevor Woerner <twoerner@...> wrote:
On Wed 2022-03-30 @ 04:08:31 PM, Richard Purdie wrote:
> On Wed, 2022-03-30 at 09:40 -0400, Trevor Woerner wrote:
> > Hi Matthias,
> >
> > On Wed 2022-03-30 @ 06:32:00 AM, Matthias Klein wrote:
> > > Yes, you are right, it is mostly the same recipes that fail. But they also change from time to time.
> > > Today it happened to me even without Jenkins and Docker, normally in the console with the recipe keymaps_1.0.bb.
> >
> > And keymaps follows the exact same pattern as modutils-initscripts and
> > initscripts; namely that their sources are entirely contained in-tree:
> >
> >     keymaps/
> >     ├── files
> >     │   ├── GPLv2.patch
> >     │   └── keymap.sh
> >     └── keymaps_1.0.bb
> >
> >     keymaps/keymaps_1.0.bb
> >      23 SRC_URI = "file://keymap.sh \
> >      24            file://GPLv2.patch"
> >
> > Any recipe that follows this pattern is susceptible, it's probably just a
> > coincidence that most of my failures happened to be with the two recipes I
> > mentioned.
> >
> > This issue has revealed a bug, and fixing that bug would be great. However,
> > the thing is, keymap.sh is a shell program written 12 years ago which hasn't
> > changed since. The GPL/COPYING file is only there for "reasons". The license
> > file doesn't *need* to be moved into the build area for this recipe to get its
> > job done (namely installing keymap.sh into the image's sysvinit).
>
> The "good" news is I did work out how to reproduce this.
>
> bitbake keymaps -c clean
> bitbake keymaps
> bitbake keymaps -c unpack -f
> bitbake keymaps -c patch
> bitbake keymaps -c unpack -f
> bitbake keymaps -c patch

Awesome! That is a very simple and quick reproducer!

> I haven't looked at why but hopefully that helps us more forward with looking at
> the issue.
>
> The complications with S == WORKDIR were one of the reasons I did start work on
> patches to make it work better and maybe move fetching into a dedicated
> direction rather than WORKDIR and then symlink things. I never got that patch to
> work well enough to submit though (and it is too late for a major change like
> that in this release).

As per our conversation I quickly tried the following (not that I expected
this to be a final solution, but just a poking-around kind of thing):

        diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
        index cc81461473..503da61b3d 100644
        --- a/meta/classes/base.bbclass
        +++ b/meta/classes/base.bbclass
        @@ -170,6 +170,7 @@ do_unpack[dirs] = "${WORKDIR}"
         do_unpack[cleandirs] = "${@d.getVar('S') if os.path.normpath(d.getVar('S')) != os.path.normpath(d.getVar('WORKDIR')) else os.path.join('${S}', 'patches')}"

         python base_do_unpack() {
        +    bb.utils.remove(d.getVar('B') + "/.pc", recurse=True)
             src_uri = (d.getVar('SRC_URI') or "").split()
             if not src_uri:
                 return

And it changed the error message from:

        $ bitbake keymaps -c patch
        ...
        ERROR: keymaps-1.0-r31 do_patch: Applying patch 'GPLv2.patch' on target directory '/z/build-master/quilt-fix/qemux86/nodistro/build/tmp-glibc/work/qemux86-oe-linux/keymaps/1.0-r31'
        CmdError('quilt --quiltrc /z/build-master/quilt-fix/qemux86/nodistro/build/tmp-glibc/work/qemux86-oe-linux/keymaps/1.0-r31/recipe-sysroot-native/etc/quiltrc push', 0, 'stdout:
        stderr: File series fully applied, ends at patch GPLv2.patch
        ')

to:

        $ bitbake keymaps -c patch
        ...
        ERROR: keymaps-1.0-r31 do_patch: Applying patch 'GPLv2.patch' on target directory '/z/build-master/quilt-fix/qemux86/nodistro/build/tmp-glibc/work/qemux86-oe-linux/keymaps/1.0-r31'
        CmdError('quilt --quiltrc /z/build-master/quilt-fix/qemux86/nodistro/build/tmp-glibc/work/qemux86-oe-linux/keymaps/1.0-r31/recipe-sysroot-native/etc/quiltrc push', 0, 'stdout: Applying patch GPLv2.patch
        The next patch would create the file COPYING,
        which already exists!  Applying it anyway.
        patching file COPYING
        Hunk #1 FAILED at 1.
        1 out of 1 hunk FAILED -- rejects in file COPYING
        Patch GPLv2.patch can be reverse-applied

        stderr: ')

progress? https://www.reddit.com/r/ProgrammerHumor/comments/8j5qim/progress/

+oe-core ML as it isn't poky/yocto specific

Just small update as multiple people mentioned this (in case I don't send the final fix later today).

There are couple recipes affected by this e.g. keymaps (.patch already removed in oe-core), makedevs (.patch removal sent to ML yesterday https://lists.openembedded.org/g/openembedded-core/message/166172), devmem2 (https://lists.openembedded.org/g/openembedded-devel/message/97270), but there are other recipes with S = "${WORKDIR}" where you can trigger this e.g. by having a .patch file in DISTRO layer .bbappend (e.g. tzdata with webOS https://github.com/webosose/meta-webosose/blob/06e5298d9f5c47679b679081d9930f8d1c776142/meta-webos/recipes-extended/tzdata/tzdata.bbappend#L10)

This do_patch issue is caused by:
introduced in kirkstone with:

I'm still looking how to fix this properly, but the shortest sequence to reproduce this is just
bitbake keymaps -c patch
bitbake keymaps -c unpack -f
bitbake keymaps -c patch

And the change in quilt behavior is causing QuiltTree.Clean (quilt pop -a -f) in:

to fail with "No series file found" before undoing the patches in WORKDIR.

Removing ".pc" as Trevor tried above doesn't help, because we really need quilt's help to undo the patches (in this case to delete COPYING file from WORKDIR before applying the .patch which tries to add it again), because do_unpack cannot just wipe S and start over (because S == WORKDIR) - not selectively removing the files listed in SRC_URI, because COPYING file isn't listed there.

Using skip_series_check in 'quilt pop' (partially reverting the change from upstream) helps a bit, but might be difficult to upstream.

Will send a fix later today or next week.

Cheers,