Re: Update on build time performance - not good

Richard Purdie

Just for interest, for the first time ever I did a dump out of profile
information on the python side of bitbake's task execution for all tasks
of a core-image-sato build.

I enabled profiling of each forked off task with this patch:

and then combined the results together in the attached profile report.
Most of it isn't a big surprise but there are some interesting numbers
such as the 278 million calls to _keys() in data_smart. For that
specific issue I have a patch which I'm considering:

as experience shows that function calls have overhead in python at that
number of iterations and reworking the code likely will reduce overhead.
I've yet to profile the result on this scale though.

Overall, a build makes over 10 billion python function calls. We access
data store variable flags 150 million times.

Not shown here but interestingly, early in the build process, the number
of calls to time.sleep() was low, at the end of this build its rather
significantly higher showing bitbake was hitting its idle handler a lot
more. Time spend there is an illusion as the system should have yielded
to other processes.

There are probably some other interesting things to be noticed somewhere
in that data :)



Join to automatically receive all group messages.