Re: What are the key factors for yocto build speed?


Yann Dirson
 



Le jeu. 19 mars 2020 à 18:04, Richard Purdie <richard.purdie@...> a écrit :
On Thu, 2020-03-19 at 17:29 +0100, Yann Dirson wrote:
>
>
> Le jeu. 19 mars 2020 à 17:07, Mike Looijmans <mike.looijmans@...> a écrit :
> > On 19-03-2020 12:04, Richard Purdie via Lists.Yoctoproject.Org wrote:
> > >> , fetch, configure, package and rootfs tasks.
> > >
> > > Sadly these tasks are much harder.
> >
> > It would be really great if some sort of "weight" could be attached to a
> > task. This relates to memory usage.
> >
> > My system has 16 cores but only 8GB RAM. With both parallelization
> > options to "16", I might end up with 16 compile tasks running 16 compile
> > threads each, i.e. 256 running processes. In practice this doesn't
> > actually happen, but the memory load gets high sometimes, so I reduce
> > the TASKS to 8 at most. That has kept my system out of swap trouble for
> > the time being.
>
> This could be neatly handled by using the GNU-make job-server mechanism.
> If bitbake itself would provide a jub-server, all make-based recipes would
> automatically get their jobs properly limited.  There is a (sadly not merged yet)
> MR [1] for ninja tu gain job-server support as well, through which we should have
> a pretty good coverage of the recipes set (as a backend for cmake, meson, and more).
>
> [1] https://github.com/ninja-build/ninja/issues/1139

You mean like:

http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/wipqueue4&id=d66a327fb6189db5de8bc489859235dcba306237

? :)


Awesome :)

Sadly we never fixed all the issues to let that merge.

The problems listed in the commit message don't amount to much at first glance. Any other issues
that got visible only later ?

--
Yann Dirson <yann@...>
Blade / Shadow -- http://shadow.tech

Join yocto@lists.yoctoproject.org to automatically receive all group messages.