FW: [oe] BitBake parallel parsing testing


Xu, Dongxiao <dongxiao.xu@...>
 

FYI, bitbake parallel parsing is enabled, which can greatly reduce the file parsing time.
There are some data test result in OE's following mails.

Bitbake 1.10: 3m2.185s
parallel-parsing (BB_NUM_PARSE_THREADS=num_cpu): 1m48.232s parallel-parsing (BB_NUM_PARSE_THREADS=2*num_cpu): 1m1.869s

Thanks,
Dongxiao

Chris Larson wrote:

Greetings all,
This is a request for further testers for the parallel parsing branch
of bitbake, as well as for review of the code.  I'm fairly certain
that the last real issue with it was resolved yesterday, and I
haven't gotten any reports of problems from those testing it since
last week, so I'm opening it up to more public testing, so we can be
absolutely certain that we can merge it to master in the near future
without risking problems.

For code review:
Either follow the below steps, then git log master..parallel-parsing
or git diff master..parallel-parsing, or use the web interface at
https://github.com/kergoth/bitbake/commits/parallel-parsing.

For testing from scratch:

git clone git://github.com/kergoth/bitbake cd bitbake git checkout -b
parallel-parsing origin/parallel-parsing

From an existing bitbake repository:

git remote add kergoth git://github.com/kergoth/bitbake git remote
update git checkout -b parallel-parsing kergoth/parallel-parsing

Thanks for your time, and do let me know what you think, either of
the code, performance in general, or the new progress bar (shows an
ETA), or anything else :)


Gary Thomas
 

On 11/24/2010 01:45 AM, Xu, Dongxiao wrote:
FYI, bitbake parallel parsing is enabled, which can greatly reduce the file parsing time.
There are some data test result in OE's following mails.

Bitbake 1.10: 3m2.185s
parallel-parsing (BB_NUM_PARSE_THREADS=num_cpu): 1m48.232s parallel-parsing (BB_NUM_PARSE_THREADS=2*num_cpu): 1m1.869s
To be clear - this is in the Poky main tree? since what commit?

Thanks

Chris Larson wrote:
Greetings all,
This is a request for further testers for the parallel parsing branch
of bitbake, as well as for review of the code. I'm fairly certain
that the last real issue with it was resolved yesterday, and I
haven't gotten any reports of problems from those testing it since
last week, so I'm opening it up to more public testing, so we can be
absolutely certain that we can merge it to master in the near future
without risking problems.

For code review:
Either follow the below steps, then git log master..parallel-parsing
or git diff master..parallel-parsing, or use the web interface at
https://github.com/kergoth/bitbake/commits/parallel-parsing.

For testing from scratch:

git clone git://github.com/kergoth/bitbake cd bitbake git checkout -b
parallel-parsing origin/parallel-parsing

From an existing bitbake repository:

git remote add kergoth git://github.com/kergoth/bitbake git remote
update git checkout -b parallel-parsing kergoth/parallel-parsing

Thanks for your time, and do let me know what you think, either of
the code, performance in general, or the new progress bar (shows an
ETA), or anything else :)
_______________________________________________
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
--
------------------------------------------------------------
Gary Thomas | Consulting for the
MLB Associates | Embedded world
------------------------------------------------------------


Joshua Lock <josh@...>
 

On Wed, 2010-11-24 at 02:50 -0700, Gary Thomas wrote:
On 11/24/2010 01:45 AM, Xu, Dongxiao wrote:
FYI, bitbake parallel parsing is enabled, which can greatly reduce the file parsing time.
There are some data test result in OE's following mails.

Bitbake 1.10: 3m2.185s
parallel-parsing (BB_NUM_PARSE_THREADS=num_cpu): 1m48.232s parallel-parsing (BB_NUM_PARSE_THREADS=2*num_cpu): 1m1.869s
To be clear - this is in the Poky main tree? since what commit?
No, this is an RFC from one of the bitbake developers and currently only
resides in his personal branch of bitbake.

Cheers,
Joshua
--
Joshua Lock
Intel Open Source Technology Centre