Date 1 - 2 of 2
Some data collection and analysis on poky performance
|1 - 2 of 2|
Qing He <qing.he@...>
As we know, many of us have experienced slow builds of recent poky,
and it also takes larger disk space. This affects user exprience thus
is one of our directions in 1.0.
To find the problems leading to performance issues, I tried some
profiling on poky builds, below is a very brief summary of the data.
I profiled poky-image-minimal of both the current master branch and
green release, with similar parameters (4 CPUs) on NHM. Note that
both rpm and ipk packages are built for current master branch, while
oonly ipk packages are built for green release
I. some stats
1. recipes (including -native)
recipes built: 76
tasks run: 998
recipes built: 133
tasks run: 1600
3. space (haven't analyze though)
green release: 7.8G
I tried a brief profile by collecting the time used for every task, so
we can scrutinize the result from a microscopic point of view. I'm still
looking into the full result, but there's something of immediate
In master, hardly any task consume less than 1.3s, this is quite
surprising, since many tasks like do_patch virtually do nothing, while
in the green release, these tasks may simply consume 0.1~0.2s. A deeper
investigation shows that this large overhead goes to bitbake-runtask,
the bitbake config and cache mechanism is executed for every task,
considerably increased the time. The overhead introduced solely by
this is around 1600*1.3=2080s, approximately 35 minutes (user+sys).
Also, we should count the additional rpm packaging system,
do_package_write_rpm costs around 1400s, (excluding 1.3s per task,
btw, this is about 50% slower than do_package_write_ipk, in average),
that's around 23 minutes.
Roughly considering that the build time is proportional to recipes
count, we can try to estimate master build time from green release:
67 * (76 / 133) + 35 + 23 = 175
very close to the real time consuming (although somewhat too closed...),
so possibly the above two are the most significant time consumers
in the slowness of current poky
Tian, Kevin <kevin.tian@...>
From: Qing Hethanks Qing, that's a great start.
could you get a compare list which recipes have been newly added? I thought that
minimal image was seldom changed...
I recall that RP mentioned debug symbol is enabled in current master, but I'm not sure
whether green has done the same. You may double-confirm that part first.
It's said this change comes from pseudo integration. Now it's time for us to
revisit this implementation then. RP can have more insights here.
above are all good findings!
|1 - 2 of 2|