Re: [linux-yocto] how can be improved the yocto build speed using network computer resources ?

Randy MacLeod

Oops, I just noticed that this was posted to the linux-yocto not
the yocto list. I'm BCCing linux-yocto and redirecting the
discussion to the yocto list where it's more appropriate.

for the scope of each list.


On 2022-01-09 16:10, Randy MacLeod wrote:
On 2022-01-07 07:44, Weingart, Mircea wrote:
Hello ,

How can be improved the yocto build  speed using network computer resources ?

Suppose that the virtual linux machine where the yocto build is run , is in a network with other computers that may share their resources

with the yocto build virtual machine , what solution may work to improve significantly the speed of the yocto build run.


Hi Mircea,
Building images using bitbake (or any source-based build system), does take
significant computational resources. If you're using a VM, you'd best be sure
that it is well-tuned and doesn't have a significant overhead compared to a
bare metal distro. Containers tend to be somewhat better here but can also
introduce IO overhead.
Have you tried using icecream? Are you sharing sstate-cache across the nodes on the network?
Are you caching and sharing the downloads directory?
You could also buy a system with > ~100 cores and lots of RAM, SSDs but that does cost
quite a bit. Depending on your environment it may be a good decision.
At work, I almost never build on my laptop but rather use a hefty server with 192 cores!
$ time bitbake core-image-minimal
NOTE: Tasks Summary: Attempted 3104 tasks of which 0 didn't need to be rerun and all succeeded.
real    20m8.536s
user    0m20.574s
sys     0m3.759s
You can see that even on a large machine the build takes ~20 minutes due ot Amhahl's law,
i.e. there are some serialization points that prevent bitbake from continuously using
all of the resources on a machine.
Shameless plug:
There are some vendors, such as Wind River, who provide a cloud-based build environment to
try to help people who face this problem.
Good luck,
# Randy MacLeod
# Wind River Linux

Join to automatically receive all group messages.