On 19-03-2020 18:21, Adrian Bunk wrote:

On Thu, Mar 19, 2020 at 05:07:17PM +0100, Mike Looijmans wrote:

...
With both parallelization options
to "16", I might end up with 16 compile tasks running 16 compile threads
each, i.e. 256 running processes.
...

This is a bug:
http://bugzilla.yoctoproject.org/show_bug.cgi?id=13306
I sometimes wonder whether something basic like "no more than one
compile task at a time" would be sufficient in practice to avoid
overloading all cores.
It would also help with RAM usage, there are some combinations of
recipes where the build gets aborted by the oom killer on my laptop
(8 cores, 32 GB RAM) when bitbake runs the compile tasks in parallel.

I tried compiling octave on an ARM board with 1GB of RAM (because octave is virtually impossible to cross-compile). There was one C++ in there that triggered a huge memory load of about 1GB, thus failing to run fully in RAM. Had to create an extra 1GB swap on the SD card to get the build to succeed. The actual swap usage wasn't much, and I could finish the build with two threads even (a dual core ARM yay).

So the memory usage is dependent on what's in the C++ file. Especially heavy template programming can put a huge load on memory. There's no way to predict that beforehand.


--
Mike Looijmans