Switching between multiple DISTROs without "contamination"


Nicolas Jeker
 

Hi all,

I'm currently using an additional layer and image to differentiate
between a release and development build (enabling NFS, SSH, root login,
etc.). To create a development build, I manually add the layer to
bblayers.conf. This works quite well, but feels a bit clumsy to
integrate into a CI/CD pipeline.

Per these past discussions here [1][2], I'm now trying to migrate to
multiple DISTROs, something like "mydistro" and "mydistro-dev".

While migrating some of the changes, I discovered that I run into
caching(?) issues. I have a recipe for an internal application and want
to include additional systemd service files in the development image.

What I did was this:

Added "application-dbg.service" to recipes-internal/application/files

Adapted application.bb recipe:

SRC_URI:append:mydistro-dev = " file://application-dbg.service"

do_install {
# ...snip...
# systemd service
install -d ${D}${systemd_system_unitdir}
install -m 0644 ${WORKDIR}/application.service
${D}${systemd_system_unitdir}
}

do_install:append:mydistro-dev() {
# debug systemd services
install -d ${D}${systemd_system_unitdir}
install -m 0644 ${WORKDIR}/application-dbg.service
${D}${systemd_system_unitdir}
}


When I run "DISTRO=mydistro-dev bitbake application" followed by
"DISTRO=mydistro bitbake application", the debug service file is still
present in the package. This seems to be caused by the "image"
directory in the recipe WORKDIR not being cleaned between subsequent
do_install runs. Is this expected behaviour? What's the best solution?

Kind regards,
Nicolas

[1]:
https://lore.kernel.org/yocto/CAH9dsRiArf_9GgQS4hCg5=J_Jk6cd3eiGaOiQd788+iSLTuU+g@mail.gmail.com/
[2]:
https://lore.kernel.org/yocto/VI1PR0602MB3549F83AC93A53785DE48677D3FD9@VI1PR0602MB3549.eurprd06.prod.outlook.com/


Martin Weber
 

Hi Nicolas,

I stumbled over this issue as well. Think of it this way, the package gets built in one environment and then it's built. Now ask to build it in another environment -- but it's built already, why do It again? Yes, in theory it would contain a different set of files / services / ... but it doesn't differentiate, because it's package A in both environments, not package A and package B. So in a sense, even though your (and my) expectations differed, I believe this works as intended.

I use to solve this issue by creating an A.inc file containing the shared stuff, a A.bb recipe building for the "default" environment, and A-XX.bb recpies for building the "XX" environment(s). Your A-XX package will be distinct from A. Now in your image setup, you can add A-XX to the XX sort of image. But keep in mind that your image is also just another recipe in the end. I believe building both in the same build dir with different distros again asks for surprises.

So, note that sharing the build dir still isn't a good idea. Richard said this is the use-case for the shared sstate so that multiple environments can share build results quickly and efficiently, and thus you wouldn't practically want a shared build dir (once you've setup sstate sharing), because you still get the benefits of not rebuilding unchanged packages, but don't step on your toes if you want to be odd one day and even another.

Best regards,

Martin Weber
Research & Development - Embedded Software Engineer

B&R Industrial Automation GmbH, B&R Straße 1, 5142 Eggelsberg, Austria, www.br-automation.com
Phone +43 7748 6586 - 0

-----Ursprüngliche Nachricht-----
Von: yocto@... <yocto@...> Im Auftrag von Nicolas Jeker via lists.yoctoproject.org
Gesendet: Dienstag, 12. Juli 2022 15:38
An: yocto@...
Betreff: [yocto] Switching between multiple DISTROs without "contamination"

BeSecure! This email comes from outside of ABB. Make sure you verify the sender before clicking any links or downloading/opening attachments.
If this email looks suspicious, report it by clicking 'Report Phishing' button in Outlook or raising a ticket on MyIS.


Hi all,

I'm currently using an additional layer and image to differentiate
between a release and development build (enabling NFS, SSH, root login,
etc.). To create a development build, I manually add the layer to
bblayers.conf. This works quite well, but feels a bit clumsy to
integrate into a CI/CD pipeline.

Per these past discussions here [1][2], I'm now trying to migrate to
multiple DISTROs, something like "mydistro" and "mydistro-dev".

While migrating some of the changes, I discovered that I run into
caching(?) issues. I have a recipe for an internal application and want
to include additional systemd service files in the development image.

What I did was this:

Added "application-dbg.service" to recipes-internal/application/files

Adapted application.bb recipe:

SRC_URI:append:mydistro-dev = " file://application-dbg.service"

do_install {
# ...snip...
# systemd service
install -d ${D}${systemd_system_unitdir}
install -m 0644 ${WORKDIR}/application.service
${D}${systemd_system_unitdir}
}

do_install:append:mydistro-dev() {
# debug systemd services
install -d ${D}${systemd_system_unitdir}
install -m 0644 ${WORKDIR}/application-dbg.service
${D}${systemd_system_unitdir}
}


When I run "DISTRO=mydistro-dev bitbake application" followed by
"DISTRO=mydistro bitbake application", the debug service file is still
present in the package. This seems to be caused by the "image"
directory in the recipe WORKDIR not being cleaned between subsequent
do_install runs. Is this expected behaviour? What's the best solution?

Kind regards,
Nicolas

[1]:
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Fyocto%2FCAH9dsRiArf_9GgQS4hCg5%3DJ_Jk6cd3eiGaOiQd788%2BiSLTuU%2Bg%40mail.gmail.com%2F&;data=05%7C01%7Cmartin.weber%40br-automation.com%7Cb4c2ffc9851847b6d7c208da640bb064%7C372ee9e09ce04033a64ac07073a91ecd%7C0%7C0%7C637932298663056785%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=STA1V1w3ExlYrXkoYb7DDtkAWlkAeKhtFAkjjkUlsoQ%3D&amp;reserved=0
[2]:
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Fyocto%2FVI1PR0602MB3549F83AC93A53785DE48677D3FD9%40VI1PR0602MB3549.eurprd06.prod.outlook.com%2F&;data=05%7C01%7Cmartin.weber%40br-automation.com%7Cb4c2ffc9851847b6d7c208da640bb064%7C372ee9e09ce04033a64ac07073a91ecd%7C0%7C0%7C637932298663056785%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=dT4DkNsm0f0L%2BrCgF7f1Q8XBAN1j2bLYZ0%2FkbKr6nuo%3D&amp;reserved=0


Mike Looijmans
 

Quick answer: Don't build multiple distros in one build directory.

You might get away with setting TMPDIR = "tmp-${DISTRO}" to give each its own.

But I'd rather advice to set up two separate builds and just point the downloads and sstate-cache to the same location. It'll be faster than the TMPDIR option.

Or figure out how to put the difference in the IMAGE only. Then you can just build both images (in parallel, woot), which is faster, more convenient and saves on diskspace.

What I often do is have my-application.bb generate a my-application-utils package that only gets installed in the "dev" image but not in the production one, which only installs "my-application".

You could also create a "my-application-dev.bb" recipe that includes my-application.bb and just changes what it needs to be different.




Met vriendelijke groet / kind regards,

Mike Looijmans
System Expert


TOPIC Embedded Products B.V.
Materiaalweg 4, 5681 RJ Best
The Netherlands

T: +31 (0) 499 33 69 69
E: mike.looijmans@...
W: www.topic.nl

Please consider the environment before printing this e-mail

On 12-07-2022 15:37, Nicolas Jeker via lists.yoctoproject.org wrote:
Hi all,

I'm currently using an additional layer and image to differentiate
between a release and development build (enabling NFS, SSH, root login,
etc.). To create a development build, I manually add the layer to
bblayers.conf. This works quite well, but feels a bit clumsy to
integrate into a CI/CD pipeline.

Per these past discussions here [1][2], I'm now trying to migrate to
multiple DISTROs, something like "mydistro" and "mydistro-dev".

While migrating some of the changes, I discovered that I run into
caching(?) issues. I have a recipe for an internal application and want
to include additional systemd service files in the development image.

What I did was this:

Added "application-dbg.service" to recipes-internal/application/files

Adapted application.bb recipe:

SRC_URI:append:mydistro-dev = " file://application-dbg.service"

do_install {
# ...snip...
# systemd service
install -d ${D}${systemd_system_unitdir}
install -m 0644 ${WORKDIR}/application.service
${D}${systemd_system_unitdir}
}

do_install:append:mydistro-dev() {
# debug systemd services
install -d ${D}${systemd_system_unitdir}
install -m 0644 ${WORKDIR}/application-dbg.service
${D}${systemd_system_unitdir}
}


When I run "DISTRO=mydistro-dev bitbake application" followed by
"DISTRO=mydistro bitbake application", the debug service file is still
present in the package. This seems to be caused by the "image"
directory in the recipe WORKDIR not being cleaned between subsequent
do_install runs. Is this expected behaviour? What's the best solution?

Kind regards,
Nicolas

[1]:
https://lore.kernel.org/yocto/CAH9dsRiArf_9GgQS4hCg5=J_Jk6cd3eiGaOiQd788+iSLTuU+g@mail.gmail.com/
[2]:
https://lore.kernel.org/yocto/VI1PR0602MB3549F83AC93A53785DE48677D3FD9@VI1PR0602MB3549.eurprd06.prod.outlook.com/

--
Mike Looijmans


Khoi Dinh Trinh
 

Thank you Nicolas for asking this question since I will probably run into this issue soon if not for this email thread. The answers so far have been very helpful but I just want to clarify a bit more on why doesn't the package get rebuilt? From my understanding, Yocto should rerun a task when the signature of the task changes and since do_install has an override on mydistro-dev, shouldn't the content and thus the signature of do_install change when switching distro and so Yocto should rerun it? I have a lot of tasks with override not just on DISTRO but other things like MACHINE or custom variables so I want to understand the rebuild mechanism as best I can.

Best,
Khoi Trinh


On Tue, Jul 12, 2022 at 8:05 AM Mike Looijmans <mike.looijmans@...> wrote:
Quick answer: Don't build multiple distros in one build directory.

You might get away with setting TMPDIR = "tmp-${DISTRO}" to give each
its own.

But I'd rather advice to set up two separate builds and just point the
downloads and sstate-cache to the same location. It'll be faster than
the TMPDIR option.

Or figure out how to put the difference in the IMAGE only. Then you can
just build both images (in parallel, woot), which is faster, more
convenient and saves on diskspace.

What I often do is have my-application.bb generate a
my-application-utils package that only gets installed in the "dev" image
but not in the production one, which only installs "my-application".

You could also create a "my-application-dev.bb" recipe that includes
my-application.bb and just changes what it needs to be different.




Met vriendelijke groet / kind regards,

Mike Looijmans
System Expert


TOPIC Embedded Products B.V.
Materiaalweg 4, 5681 RJ Best
The Netherlands

T: +31 (0) 499 33 69 69
E: mike.looijmans@...
W: www.topic.nl

Please consider the environment before printing this e-mail
On 12-07-2022 15:37, Nicolas Jeker via lists.yoctoproject.org wrote:
> Hi all,
>
> I'm currently using an additional layer and image to differentiate
> between a release and development build (enabling NFS, SSH, root login,
> etc.). To create a development build, I manually add the layer to
> bblayers.conf. This works quite well, but feels a bit clumsy to
> integrate into a CI/CD pipeline.
>
> Per these past discussions here [1][2], I'm now trying to migrate to
> multiple DISTROs, something like "mydistro" and "mydistro-dev".
>
> While migrating some of the changes, I discovered that I run into
> caching(?) issues. I have a recipe for an internal application and want
> to include additional systemd service files in the development image.
>
> What I did was this:
>
> Added "application-dbg.service" to recipes-internal/application/files
>
> Adapted application.bb recipe:
>
> SRC_URI:append:mydistro-dev = " file://application-dbg.service"
>
> do_install {
>      # ...snip...
>      # systemd service
>      install -d ${D}${systemd_system_unitdir}
>      install -m 0644 ${WORKDIR}/application.service
> ${D}${systemd_system_unitdir}
> }
>
> do_install:append:mydistro-dev() {
>      # debug systemd services
>      install -d ${D}${systemd_system_unitdir}
>      install -m 0644 ${WORKDIR}/application-dbg.service
> ${D}${systemd_system_unitdir}
> }
>
>
> When I run "DISTRO=mydistro-dev bitbake application" followed by
> "DISTRO=mydistro bitbake application", the debug service file is still
> present in the package. This seems to be caused by the "image"
> directory in the recipe WORKDIR not being cleaned between subsequent
> do_install runs. Is this expected behaviour? What's the best solution?
>
> Kind regards,
> Nicolas
>
> [1]:
> https://lore.kernel.org/yocto/CAH9dsRiArf_9GgQS4hCg5=J_Jk6cd3eiGaOiQd788+iSLTuU+g@.../
> [2]:
> https://lore.kernel.org/yocto/VI1PR0602MB3549F83AC93A53785DE48677D3FD9@.../
>
>
>

--
Mike Looijmans






--
Best,
Khoi Trinh


Nicolas Jeker
 

Thanks Martin and Mike for your explanations and tips.

So, I've done a lot of testing today and it seems I simplified the
example in my first email a bit too much. The example as-is works fine
when switching DISTROs as far as I can tell. The problem only arises
when wildcards are used.

Changing my initial example like this should trigger the behaviour I've
initially described:

SRC_URI:append:mydistro-dev = " file://application-dbg.service"

do_install {
# ...snip...
# systemd service
install -d ${D}${systemd_system_unitdir}
install -m 0644 ${WORKDIR}/*.service ${D}${systemd_system_unitdir}
}

do_install:append:mydistro-dev() {
# debug systemd services
install -d ${D}${systemd_system_unitdir}
install -m 0644 ${WORKDIR}/application-dbg.service
${D}${systemd_system_unitdir}
}

Notice the *.service in do_install.

From my testing, this is how contamination happens:

1) Build with 'DISTRO=mydistro bitbake application'. All tasks for the
recipe are run and the directories in WORKDIR are populated, including
the "application.service" file.
2) Build with 'DISTRO=mydistro-dev bitbake application'. do_unpack is
rerun and places the additional "application-dbg.service" file in
WORKDIR.
3) Switching back to 'mydistro' will get the recipe from sstate cache,
which works fine.
4) Changing application.bb and rebuilding with 'DISTRO=mydistro bitbake
application' reruns do_install (as expected). This leads to the
packages do_install picking up the additional "application-dbg.service"
file left behind by the invocation in step 2).

Mike, Martin: Do you remember in which cases you encountered problems
when sharing the build directory?

On Tue, 2022-07-12 at 09:15 -0700, Khoi Dinh Trinh wrote:
Thank you Nicolas for asking this question since I will probably run
into this issue soon if not for this email thread. The answers so far
have been very helpful but I just want to clarify a bit more on why
doesn't the package get rebuilt? From my understanding, Yocto should
rerun a task when the signature of the task changes and since
do_install has an override on mydistro-dev, shouldn't the content and
thus the signature of do_install change when switching distro and so
Yocto should rerun it? 
As far as I can tell, all the relevant tasks are rerun correctly when
something is changed. Relevant tasks meaning only the tasks that are
actually different.

The specific issue I experienced is due to the WORKDIR not being
cleaned between different task invocations and the recipe (probably
wrongfully) relying on wildcards to gather files, see example above.

I have a lot of tasks with override not just on DISTRO but other
things like MACHINE or custom variables so I want to understand the
rebuild mechanism as best I can.
There's surely someone more knowledgeable here that could clarify the
inner workings of this mechanism a lot better than me.

Best,
Khoi Trinh

On Tue, Jul 12, 2022 at 8:05 AM Mike Looijmans
<mike.looijmans@...> wrote:
Quick answer: Don't build multiple distros in one build directory.
It's really a bummer that it's not reliably possible to switch between
DISTROs inside one build directory.

You might get away with setting TMPDIR = "tmp-${DISTRO}" to give
each
its own.

But I'd rather advice to set up two separate builds and just point
the
downloads and sstate-cache to the same location. It'll be faster
than
the TMPDIR option.

Or figure out how to put the difference in the IMAGE only. Then you
can
just build both images (in parallel, woot), which is faster, more
convenient and saves on diskspace.
As I'm currently working with something close to what you describe, I
think I'll try to stay away from multiple DISTROs if possible and
improve on what I'm already doing.


What I often do is have my-application.bb generate a
my-application-utils package that only gets installed in the "dev"
image
but not in the production one, which only installs "my-
application".
This is probably what Martin meant with his A.bb and A-xx.bb example.
It's so far one of the best approaches I've seen, thanks.


You could also create a "my-application-dev.bb" recipe that
includes
my-application.bb and just changes what it needs to be different.




Met vriendelijke groet / kind regards,

Mike Looijmans
System Expert


TOPIC Embedded Products B.V.
Materiaalweg 4, 5681 RJ Best
The Netherlands

T: +31 (0) 499 33 69 69
E: mike.looijmans@...
W: www.topic.nl

Please consider the environment before printing this e-mail
On 12-07-2022 15:37, Nicolas Jeker via lists.yoctoproject.org
wrote:
Hi all,

I'm currently using an additional layer and image to
differentiate
between a release and development build (enabling NFS, SSH, root
login,
etc.). To create a development build, I manually add the layer to
bblayers.conf. This works quite well, but feels a bit clumsy to
integrate into a CI/CD pipeline.

Per these past discussions here [1][2], I'm now trying to migrate
to
multiple DISTROs, something like "mydistro" and "mydistro-dev".

While migrating some of the changes, I discovered that I run into
caching(?) issues. I have a recipe for an internal application
and want
to include additional systemd service files in the development
image.

What I did was this:

Added "application-dbg.service" to recipes-
internal/application/files

Adapted application.bb recipe:

SRC_URI:append:mydistro-dev = " file://application-dbg.service"

do_install {
      # ...snip...
      # systemd service
      install -d ${D}${systemd_system_unitdir}
      install -m 0644 ${WORKDIR}/application.service
${D}${systemd_system_unitdir}
}

do_install:append:mydistro-dev() {
      # debug systemd services
      install -d ${D}${systemd_system_unitdir}
      install -m 0644 ${WORKDIR}/application-dbg.service
${D}${systemd_system_unitdir}
}


When I run "DISTRO=mydistro-dev bitbake application" followed by
"DISTRO=mydistro bitbake application", the debug service file is
still
present in the package. This seems to be caused by the "image"
directory in the recipe WORKDIR not being cleaned between
subsequent
do_install runs. Is this expected behaviour? What's the best
solution?

Kind regards,
Nicolas

[1]:
https://lore.kernel.org/yocto/CAH9dsRiArf_9GgQS4hCg5=J_Jk6cd3eiGaOiQd788+iSLTuU+g@mail.gmail.com/
[2]:
https://lore.kernel.org/yocto/VI1PR0602MB3549F83AC93A53785DE48677D3FD9@VI1PR0602MB3549.eurprd06.prod.outlook.com/


--
Mike Looijmans




Mike Looijmans
 

Hi Nicolas,

I guess part of your problem is that DISTRO does not end up in any package grouping, unlike MACHINE and ARCH and so.

So if OE/Yocto does The Right Thing, this means that it will have to remove a bunch of packages from your feed and sysroot, and add them again (from sstate-cache or by building) every time you switch DISTRO (which, on a build server, would be each and every build).
Hence my advice to use a separate build directory for each, otherwise Yocto will have to juggle with several packages on each and every build. With accidents waiting to happen.

Hence my advice remains, you can have "my-image-dev" and even "my-machine-dev" in the same build directory, but don't share "my-distro-dev" that way, it's an invitation to trouble, even though It Should Work In Theory...

Mike.


Met vriendelijke groet / kind regards,

Mike Looijmans
System Expert


TOPIC Embedded Products B.V.
Materiaalweg 4, 5681 RJ Best
The Netherlands

T: +31 (0) 499 33 69 69
E: mike.looijmans@...
W: www.topic.nl

Please consider the environment before printing this e-mail

On 13-07-2022 15:32, Nicolas Jeker wrote:
Thanks Martin and Mike for your explanations and tips.

So, I've done a lot of testing today and it seems I simplified the
example in my first email a bit too much. The example as-is works fine
when switching DISTROs as far as I can tell. The problem only arises
when wildcards are used.

Changing my initial example like this should trigger the behaviour I've
initially described:

SRC_URI:append:mydistro-dev = " file://application-dbg.service"

do_install {
# ...snip...
# systemd service
install -d ${D}${systemd_system_unitdir}
install -m 0644 ${WORKDIR}/*.service ${D}${systemd_system_unitdir}
}

do_install:append:mydistro-dev() {
# debug systemd services
install -d ${D}${systemd_system_unitdir}
install -m 0644 ${WORKDIR}/application-dbg.service
${D}${systemd_system_unitdir}
}

Notice the *.service in do_install.

From my testing, this is how contamination happens:

1) Build with 'DISTRO=mydistro bitbake application'. All tasks for the
recipe are run and the directories in WORKDIR are populated, including
the "application.service" file.
2) Build with 'DISTRO=mydistro-dev bitbake application'. do_unpack is
rerun and places the additional "application-dbg.service" file in
WORKDIR.
3) Switching back to 'mydistro' will get the recipe from sstate cache,
which works fine.
4) Changing application.bb and rebuilding with 'DISTRO=mydistro bitbake
application' reruns do_install (as expected). This leads to the
packages do_install picking up the additional "application-dbg.service"
file left behind by the invocation in step 2).

Mike, Martin: Do you remember in which cases you encountered problems
when sharing the build directory?

On Tue, 2022-07-12 at 09:15 -0700, Khoi Dinh Trinh wrote:
Thank you Nicolas for asking this question since I will probably run
into this issue soon if not for this email thread. The answers so far
have been very helpful but I just want to clarify a bit more on why
doesn't the package get rebuilt? From my understanding, Yocto should
rerun a task when the signature of the task changes and since
do_install has an override on mydistro-dev, shouldn't the content and
thus the signature of do_install change when switching distro and so
Yocto should rerun it?
As far as I can tell, all the relevant tasks are rerun correctly when
something is changed. Relevant tasks meaning only the tasks that are
actually different.

The specific issue I experienced is due to the WORKDIR not being
cleaned between different task invocations and the recipe (probably
wrongfully) relying on wildcards to gather files, see example above.

I have a lot of tasks with override not just on DISTRO but other
things like MACHINE or custom variables so I want to understand the
rebuild mechanism as best I can.
There's surely someone more knowledgeable here that could clarify the
inner workings of this mechanism a lot better than me.

Best,
Khoi Trinh

On Tue, Jul 12, 2022 at 8:05 AM Mike Looijmans
<mike.looijmans@...> wrote:
Quick answer: Don't build multiple distros in one build directory.
It's really a bummer that it's not reliably possible to switch between
DISTROs inside one build directory.

You might get away with setting TMPDIR = "tmp-${DISTRO}" to give
each
its own.

But I'd rather advice to set up two separate builds and just point
the
downloads and sstate-cache to the same location. It'll be faster
than
the TMPDIR option.

Or figure out how to put the difference in the IMAGE only. Then you
can
just build both images (in parallel, woot), which is faster, more
convenient and saves on diskspace.
As I'm currently working with something close to what you describe, I
think I'll try to stay away from multiple DISTROs if possible and
improve on what I'm already doing.

What I often do is have my-application.bb generate a
my-application-utils package that only gets installed in the "dev"
image
but not in the production one, which only installs "my-
application".
This is probably what Martin meant with his A.bb and A-xx.bb example.
It's so far one of the best approaches I've seen, thanks.

You could also create a "my-application-dev.bb" recipe that
includes
my-application.bb and just changes what it needs to be different.




Met vriendelijke groet / kind regards,

Mike Looijmans
System Expert


TOPIC Embedded Products B.V.
Materiaalweg 4, 5681 RJ Best
The Netherlands

T: +31 (0) 499 33 69 69
E: mike.looijmans@...
W: www.topic.nl

Please consider the environment before printing this e-mail
On 12-07-2022 15:37, Nicolas Jeker via lists.yoctoproject.org
wrote:
Hi all,

I'm currently using an additional layer and image to
differentiate
between a release and development build (enabling NFS, SSH, root
login,
etc.). To create a development build, I manually add the layer to
bblayers.conf. This works quite well, but feels a bit clumsy to
integrate into a CI/CD pipeline.

Per these past discussions here [1][2], I'm now trying to migrate
to
multiple DISTROs, something like "mydistro" and "mydistro-dev".

While migrating some of the changes, I discovered that I run into
caching(?) issues. I have a recipe for an internal application
and want
to include additional systemd service files in the development
image.

What I did was this:

Added "application-dbg.service" to recipes-
internal/application/files

Adapted application.bb recipe:

SRC_URI:append:mydistro-dev = " file://application-dbg.service"

do_install {
      # ...snip...
      # systemd service
      install -d ${D}${systemd_system_unitdir}
      install -m 0644 ${WORKDIR}/application.service
${D}${systemd_system_unitdir}
}

do_install:append:mydistro-dev() {
      # debug systemd services
      install -d ${D}${systemd_system_unitdir}
      install -m 0644 ${WORKDIR}/application-dbg.service
${D}${systemd_system_unitdir}
}


When I run "DISTRO=mydistro-dev bitbake application" followed by
"DISTRO=mydistro bitbake application", the debug service file is
still
present in the package. This seems to be caused by the "image"
directory in the recipe WORKDIR not being cleaned between
subsequent
do_install runs. Is this expected behaviour? What's the best
solution?

Kind regards,
Nicolas

[1]:
https://lore.kernel.org/yocto/CAH9dsRiArf_9GgQS4hCg5=J_Jk6cd3eiGaOiQd788+iSLTuU+g@mail.gmail.com/
[2]:
https://lore.kernel.org/yocto/VI1PR0602MB3549F83AC93A53785DE48677D3FD9@VI1PR0602MB3549.eurprd06.prod.outlook.com/


--
Mike Looijmans



--
Mike Looijmans


Richard Purdie
 

On Wed, 2022-07-13 at 15:32 +0200, Nicolas Jeker wrote:
Thanks Martin and Mike for your explanations and tips.

So, I've done a lot of testing today and it seems I simplified the
example in my first email a bit too much. The example as-is works fine
when switching DISTROs as far as I can tell. The problem only arises
when wildcards are used.

Changing my initial example like this should trigger the behaviour I've
initially described:

SRC_URI:append:mydistro-dev = " file://application-dbg.service"

do_install {
# ...snip...
# systemd service
install -d ${D}${systemd_system_unitdir}
install -m 0644 ${WORKDIR}/*.service ${D}${systemd_system_unitdir}
}

do_install:append:mydistro-dev() {
# debug systemd services
install -d ${D}${systemd_system_unitdir}
install -m 0644 ${WORKDIR}/application-dbg.service
${D}${systemd_system_unitdir}
}

Notice the *.service in do_install.

From my testing, this is how contamination happens:

1) Build with 'DISTRO=mydistro bitbake application'. All tasks for the
recipe are run and the directories in WORKDIR are populated, including
the "application.service" file.
2) Build with 'DISTRO=mydistro-dev bitbake application'. do_unpack is
rerun and places the additional "application-dbg.service" file in
WORKDIR.
3) Switching back to 'mydistro' will get the recipe from sstate cache,
which works fine.
4) Changing application.bb and rebuilding with 'DISTRO=mydistro bitbake
application' reruns do_install (as expected). This leads to the
packages do_install picking up the additional "application-dbg.service"
file left behind by the invocation in step 2).

Mike, Martin: Do you remember in which cases you encountered problems
when sharing the build directory?
This is unfortunately a known issue and is one reason I'd like to stop
recipes downloading files to WORKDIR. Sadly changing that to fix this
bug is very invasive and painful but I think we do need to do so
somehow.

Cheers,

Richard