4 hours seems extremely long to me for a 8 core system, but I have not tried this in a while. Were you removing all the sources and re-downloading them for every build?

 

From: <yocto@...> on behalf of Srini <rsrinivasan@...>
Date: Wednesday, March 18, 2020 at 10:13 AM
To: "mikko.rapeli@..." <mikko.rapeli@...>, "mike.looijmans@..." <mike.looijmans@...>
Cc: "yocto@..." <yocto@...>
Subject: Re: <EXT> Re: [yocto] What are the key factors for yocto build speed?

 

My own experience (pardon me if already discussed)

Fought the build times for several months - ending up eventually at 8 cores (but specifying 16 threads in poky builds). Best times for my build about 4 hours. Clearly impractical during engineering.

Generated an sdk and used it for app development. Each build is now a minute or 2.

Using a homegrown utility, updated the image file with applications in a jiffy - to produce burnable sdcard image.

Complete build required only for the final release -- or made major changes like python2 to python3!

YMMV.

srini

-----Original Message-----
From: yocto@... <yocto@...> On Behalf Of Mikko Rapeli via Lists.Yoctoproject.Org
Sent: Wednesday, March 18, 2020 11:52 AM
To: mike.looijmans@...
Cc: yocto@...
Subject: <EXT> Re: [yocto] What are the key factors for yocto build speed?

On Wed, Mar 18, 2020 at 04:09:39PM +0100, Mike Looijmans wrote:
> On 18-03-2020 15:49, Adrian Bunk via Lists.Yoctoproject.Org wrote:
> > On Wed, Mar 18, 2020 at 10:12:26AM -0400, Jean-Marie Lemetayer wrote:
> > > ...
> > > For example one of our build servers is using:
> > > - AMD Ryzen 9 3900X
> > > ...
> > > - 32Go DDR4 3200 MHZ CL14
> > > ...
> > > It is a really good price / build time ratio configuration.
> >
> > Depends on what you are building.
> >
> > Building non-trivial C++ code (e.g. webkitgtk) with 24 cores but
> > only 32 GB RAM will not work, for such code you need more than 2
> > GB/core.
>
> Seems a bit excessive to buy hardware just to handle a particular
> corner case. Most of OE/Yocto code is plain C, not even C++.
>
> My rig only has 8GB but doesn't run into memory issues during big GUI
> builds. The only thing that made it swap was the populate_sdk task
> that created a 1.1GB fiel and needed 20GB of RAM to compress that.
> Took a few minutes more due to swapping.
> I submitted a patch today to fix that in OE.
>
> Your mileage may vary. But RAM is easy to add.

Well, I can't build with under 2 gigs per core or I run out of physical memory and kernel oom-killer kicks in to kill the build. Also can't run with yocto default parallel settings which only take into account the number of cores and thus have a custom script which does caps the threads so that 2 gigs of RAM for each are available.

Though I'm sure plain C and plain poky projects have less requirements for RAM.

-Mikko

________________________________

CONFIDENTIALITY NOTICE: This email message and any attachments are confidential and may be privileged and are meant to be read by the intended recipient only. If you are not the intended recipient, please notify sender immediately and destroy all copies of this message and any attachments without reading or disclosing their contents. Thank you