From: Tom Zanussi <tom.zanussi@...>

To configure crownbay, crownbay-noemgd, fishriver, jasperforest, n450,
and sugarbay as compatible machines for systemtap.

Signed-off-by: Tom Zanussi <tom.zanussi@...>
---
.../recipes-core/tasks/task-core-tools.bbappend | 3 +++
.../systemtap/systemtap_git.bbappend | 1 +
.../recipes-core/tasks/task-core-tools.bbappend | 2 ++
.../systemtap/systemtap_git.bbappend | 1 +
.../recipes-core/tasks/task-core-tools.bbappend | 2 ++
.../systemtap/systemtap_git.bbappend | 1 +
.../recipes-core/tasks/task-core-tools.bbappend | 2 ++
.../systemtap/systemtap_git.bbappend | 1 +
.../recipes-core/tasks/task-core-tools.bbappend | 2 ++
.../systemtap/systemtap_git.bbappend | 1 +
10 files changed, 16 insertions(+), 0 deletions(-)
create mode 100644 meta-crownbay/recipes-core/tasks/task-core-tools.bbappend
create mode 100644 meta-crownbay/recipes-kernel/systemtap/systemtap_git.bbappend
create mode 100644 meta-fishriver/recipes-core/tasks/task-core-tools.bbappend
create mode 100644 meta-fishriver/recipes-kernel/systemtap/systemtap_git.bbappend
create mode 100644 meta-jasperforest/recipes-core/tasks/task-core-tools.bbappend
create mode 100644 meta-jasperforest/recipes-kernel/systemtap/systemtap_git.bbappend
create mode 100644 meta-n450/recipes-core/tasks/task-core-tools.bbappend
create mode 100644 meta-n450/recipes-kernel/systemtap/systemtap_git.bbappend
create mode 100644 meta-sugarbay/recipes-core/tasks/task-core-tools.bbappend
create mode 100644 meta-sugarbay/recipes-kernel/systemtap/systemtap_git.bbappend

diff --git a/meta-crownbay/recipes-core/tasks/task-core-tools.bbappend b/meta-crownbay/recipes-core/tasks/task-core-tools.bbappend
new file mode 100644
index 0000000..3dfeadc
--- /dev/null
+++ b/meta-crownbay/recipes-core/tasks/task-core-tools.bbappend
@@ -0,0 +1,3 @@
+RDEPENDS_task-core-tools-profile_append_crownbay = " systemtap"
+RDEPENDS_task-core-tools-profile_append_crownbay-noemgd = " systemtap"
+
diff --git a/meta-crownbay/recipes-kernel/systemtap/systemtap_git.bbappend b/meta-crownbay/recipes-kernel/systemtap/systemtap_git.bbappend
new file mode 100644
index 0000000..41c2ff2
--- /dev/null
+++ b/meta-crownbay/recipes-kernel/systemtap/systemtap_git.bbappend
@@ -0,0 +1 @@
+COMPATIBLE_MACHINE = ${MACHINE}
diff --git a/meta-fishriver/recipes-core/tasks/task-core-tools.bbappend b/meta-fishriver/recipes-core/tasks/task-core-tools.bbappend
new file mode 100644
index 0000000..acdf5b1
--- /dev/null
+++ b/meta-fishriver/recipes-core/tasks/task-core-tools.bbappend
@@ -0,0 +1,2 @@
+RDEPENDS_task-core-tools-profile_append_fishriver = " systemtap"
+
diff --git a/meta-fishriver/recipes-kernel/systemtap/systemtap_git.bbappend b/meta-fishriver/recipes-kernel/systemtap/systemtap_git.bbappend
new file mode 100644
index 0000000..41c2ff2
--- /dev/null
+++ b/meta-fishriver/recipes-kernel/systemtap/systemtap_git.bbappend
@@ -0,0 +1 @@
+COMPATIBLE_MACHINE = ${MACHINE}
diff --git a/meta-jasperforest/recipes-core/tasks/task-core-tools.bbappend b/meta-jasperforest/recipes-core/tasks/task-core-tools.bbappend
new file mode 100644
index 0000000..12fa1c2
--- /dev/null
+++ b/meta-jasperforest/recipes-core/tasks/task-core-tools.bbappend
@@ -0,0 +1,2 @@
+RDEPENDS_task-core-tools-profile_append_jasperforest = " systemtap"
+
diff --git a/meta-jasperforest/recipes-kernel/systemtap/systemtap_git.bbappend b/meta-jasperforest/recipes-kernel/systemtap/systemtap_git.bbappend
new file mode 100644
index 0000000..41c2ff2
--- /dev/null
+++ b/meta-jasperforest/recipes-kernel/systemtap/systemtap_git.bbappend
@@ -0,0 +1 @@
+COMPATIBLE_MACHINE = ${MACHINE}
diff --git a/meta-n450/recipes-core/tasks/task-core-tools.bbappend b/meta-n450/recipes-core/tasks/task-core-tools.bbappend
new file mode 100644
index 0000000..3ab49a3
--- /dev/null
+++ b/meta-n450/recipes-core/tasks/task-core-tools.bbappend
@@ -0,0 +1,2 @@
+RDEPENDS_task-core-tools-profile_append_n450 = " systemtap"
+
diff --git a/meta-n450/recipes-kernel/systemtap/systemtap_git.bbappend b/meta-n450/recipes-kernel/systemtap/systemtap_git.bbappend
new file mode 100644
index 0000000..41c2ff2
--- /dev/null
+++ b/meta-n450/recipes-kernel/systemtap/systemtap_git.bbappend
@@ -0,0 +1 @@
+COMPATIBLE_MACHINE = ${MACHINE}
diff --git a/meta-sugarbay/recipes-core/tasks/task-core-tools.bbappend b/meta-sugarbay/recipes-core/tasks/task-core-tools.bbappend
new file mode 100644
index 0000000..23c30a1
--- /dev/null
+++ b/meta-sugarbay/recipes-core/tasks/task-core-tools.bbappend
@@ -0,0 +1,2 @@
+RDEPENDS_task-core-tools-profile_append_sugarbay = " systemtap"
+
diff --git a/meta-sugarbay/recipes-kernel/systemtap/systemtap_git.bbappend b/meta-sugarbay/recipes-kernel/systemtap/systemtap_git.bbappend
new file mode 100644
index 0000000..41c2ff2
--- /dev/null
+++ b/meta-sugarbay/recipes-kernel/systemtap/systemtap_git.bbappend
@@ -0,0 +1 @@
+COMPATIBLE_MACHINE = ${MACHINE}
--
1.7.0.4

From: Tom Zanussi <tom.zanussi@...>

To configure this is a compatible machine for systemtap.

Signed-off-by: Tom Zanussi <tom.zanussi@...>
---
.../systemtap/systemtap_git.bbappend | 1 +
1 files changed, 1 insertions(+), 0 deletions(-)
create mode 100644 meta-emenlow/recipes-kernel/systemtap/systemtap_git.bbappend

diff --git a/meta-emenlow/recipes-kernel/systemtap/systemtap_git.bbappend b/meta-emenlow/recipes-kernel/systemtap/systemtap_git.bbappend
new file mode 100644
index 0000000..41c2ff2
--- /dev/null
+++ b/meta-emenlow/recipes-kernel/systemtap/systemtap_git.bbappend
@@ -0,0 +1 @@
+COMPATIBLE_MACHINE = ${MACHINE}
--
1.7.0.4

From: Tom Zanussi <tom.zanussi@...>

Add a 'utrace feature' that turns on the kernel options required
for utrace, used mainly to enable userspace tracing for systemtap.

Signed-off-by: Tom Zanussi <tom.zanussi@...>
---
meta/cfg/kernel-cache/features/utrace/utrace.cfg | 1 +
meta/cfg/kernel-cache/features/utrace/utrace.scc | 2 ++
meta/cfg/kernel-cache/ktypes/standard/standard.scc | 3 +++
3 files changed, 6 insertions(+), 0 deletions(-)
create mode 100644 meta/cfg/kernel-cache/features/utrace/utrace.cfg
create mode 100644 meta/cfg/kernel-cache/features/utrace/utrace.scc

diff --git a/meta/cfg/kernel-cache/features/utrace/utrace.cfg b/meta/cfg/kernel-cache/features/utrace/utrace.cfg
new file mode 100644
index 0000000..bf85aa2
--- /dev/null
+++ b/meta/cfg/kernel-cache/features/utrace/utrace.cfg
@@ -0,0 +1 @@
+CONFIG_UTRACE=y
diff --git a/meta/cfg/kernel-cache/features/utrace/utrace.scc b/meta/cfg/kernel-cache/features/utrace/utrace.scc
new file mode 100644
index 0000000..0cab7d9
--- /dev/null
+++ b/meta/cfg/kernel-cache/features/utrace/utrace.scc
@@ -0,0 +1,2 @@
+kconf non-hardware utrace.cfg
+
diff --git a/meta/cfg/kernel-cache/ktypes/standard/standard.scc b/meta/cfg/kernel-cache/ktypes/standard/standard.scc
index 290c50a..ac80038 100644
--- a/meta/cfg/kernel-cache/ktypes/standard/standard.scc
+++ b/meta/cfg/kernel-cache/ktypes/standard/standard.scc
@@ -17,6 +17,9 @@ tag lttng
include features/systemtap/systemtap.scc
# tag systemtap

+include features/utrace/utrace.scc
+# tag utrace
+
include arch/arm/arm.scc
tag arm

--
1.7.0.4

From: Tom Zanussi <tom.zanussi@...>

This adds a utrace feature, in support of userspace tracing for systemtap.

It's been tested on actual x86 and x86-64 hardware, as well as qemux86-64
and qemuppc (utrace doesn't currently have support for arm or mips).

A couple examples with output have been added to the systemtap entry in
the tracing and profiling section of the Yocto wiki:

https://wiki.yoctoproject.org/wiki/Tracing_and_Profiling#systemtap

Please pull into the meta branch of linux-yocto.

Tom Zanussi (1):
meta: add utrace feature for linux-yocto

meta/cfg/kernel-cache/features/utrace/utrace.cfg | 1 +
meta/cfg/kernel-cache/features/utrace/utrace.scc | 2 ++
meta/cfg/kernel-cache/ktypes/standard/standard.scc | 3 +++
3 files changed, 6 insertions(+), 0 deletions(-)
create mode 100644 meta/cfg/kernel-cache/features/utrace/utrace.cfg
create mode 100644 meta/cfg/kernel-cache/features/utrace/utrace.scc

From: Tom Zanussi <tom.zanussi@...>

Used by SystemTap.

Patch generated by diff of 2.6.37 base and utrace branches of the
official utrace git repo at:

git://git.kernel.org/pub/scm/linux/kernel/git/frob/linux-2.6-utrace.git

Integrated-by: Tom Zanussi <tom.zanussi@...>
---
Documentation/DocBook/Makefile | 2 +-
Documentation/DocBook/utrace.tmpl | 589 +++++++++
fs/proc/array.c | 3 +
include/linux/ptrace.h | 1 +
include/linux/sched.h | 6 +
include/linux/tracehook.h | 97 ++-
include/linux/utrace.h | 692 +++++++++++
init/Kconfig | 9 +
kernel/Makefile | 1 +
kernel/fork.c | 3 +
kernel/ptrace.c | 16 +-
kernel/signal.c | 4 +-
kernel/utrace.c | 2440 +++++++++++++++++++++++++++++++++++++
13 files changed, 3853 insertions(+), 10 deletions(-)
create mode 100644 Documentation/DocBook/utrace.tmpl
create mode 100644 include/linux/utrace.h
create mode 100644 kernel/utrace.c

Patch too large to post, see git branch for details.

From: Tom Zanussi <tom.zanussi@...>

This adds support for utrace to linux-yocto, for userspace tracing
in systemtap.

Please pull into the yocto/base branch linux-yocto.

Pull URL: git://git.pokylinux.org/linux-yocto-2.6.37-contrib
Branch: tzanussi/yocto/base-utrace
Browse: http://git.pokylinux.org/cgit.cgi/linux-yocto-2.6.37-contrib/log/?h=tzanussi/yocto/base-utrace

Tom Zanussi (1):
utrace: support for user space probing/tracing

Documentation/DocBook/Makefile | 2 +-
Documentation/DocBook/utrace.tmpl | 589 +++++++++
fs/proc/array.c | 3 +
include/linux/ptrace.h | 1 +
include/linux/sched.h | 6 +
include/linux/tracehook.h | 97 ++-
include/linux/utrace.h | 692 +++++++++++
init/Kconfig | 9 +
kernel/Makefile | 1 +
kernel/fork.c | 3 +
kernel/ptrace.c | 16 +-
kernel/signal.c | 4 +-
kernel/utrace.c | 2440 +++++++++++++++++++++++++++++++++++++
13 files changed, 3853 insertions(+), 10 deletions(-)
create mode 100644 Documentation/DocBook/utrace.tmpl
create mode 100644 include/linux/utrace.h
create mode 100644 kernel/utrace.c

From: Tom Zanussi <tom.zanussi@...>

Removing any old references to systemtap from meta means we need to add
it here. It also adds systemtap support for the other, previously
uncovered meta-intel bsps.

The following changes since commit 01d1a545477cbc1b2a1da5b66edb52fd9e783206:
Tom Zanussi (1):
meta-intel: update kernel SRCREVs

are available in the git repository at:

git://git.yoctoproject.org/meta-intel.git tzanussi/systemtap-update
http://git.yoctoproject.org/cgit.cgi//log/?h=tzanussi/systemtap-update

Tom Zanussi (2):
meta-emenlow: add systemtap bbappend
meta-intel: add systemtap bbappends

.../recipes-core/tasks/task-core-tools.bbappend | 3 +++
.../systemtap/systemtap_git.bbappend | 1 +
.../systemtap/systemtap_git.bbappend | 1 +
.../recipes-core/tasks/task-core-tools.bbappend | 2 ++
.../systemtap/systemtap_git.bbappend | 1 +
.../recipes-core/tasks/task-core-tools.bbappend | 2 ++
.../systemtap/systemtap_git.bbappend | 1 +
.../recipes-core/tasks/task-core-tools.bbappend | 2 ++
.../systemtap/systemtap_git.bbappend | 1 +
.../recipes-core/tasks/task-core-tools.bbappend | 2 ++
.../systemtap/systemtap_git.bbappend | 1 +
11 files changed, 17 insertions(+), 0 deletions(-)
create mode 100644 meta-crownbay/recipes-core/tasks/task-core-tools.bbappend
create mode 100644 meta-crownbay/recipes-kernel/systemtap/systemtap_git.bbappend
create mode 100644 meta-emenlow/recipes-kernel/systemtap/systemtap_git.bbappend
create mode 100644 meta-fishriver/recipes-core/tasks/task-core-tools.bbappend
create mode 100644 meta-fishriver/recipes-kernel/systemtap/systemtap_git.bbappend
create mode 100644 meta-jasperforest/recipes-core/tasks/task-core-tools.bbappend
create mode 100644 meta-jasperforest/recipes-kernel/systemtap/systemtap_git.bbappend
create mode 100644 meta-n450/recipes-core/tasks/task-core-tools.bbappend
create mode 100644 meta-n450/recipes-kernel/systemtap/systemtap_git.bbappend
create mode 100644 meta-sugarbay/recipes-core/tasks/task-core-tools.bbappend
create mode 100644 meta-sugarbay/recipes-kernel/systemtap/systemtap_git.bbappend

Hi all,
The open bug number of last week increased to 210. QA finished Yocto 1.1 M1 testing with no critical bugs open now. Bug status of WW24 could be found on https://wiki.pokylinux.org/wiki/Yocto_Bug_Trend.

Best Regards,
Jiajun

On Fri, 2011-06-10 at 14:43 -0700, Daryl Spitzer wrote:

Thanks Joshua. I assume that
http://www.yoctoproject.org/docs/1.1/yocto-project-qs/yocto-project-qs.html
won't be updated until the next release (if this makes it into the
next release)? And the patch should then be updated to match the new
version numbers?

It's my intention to get the document updated ASAP so that we don't
confuse anyone else as to which version to download.

Regards,
Joshua
--
Joshua Lock
Yocto Build System Monkey
Intel Open Source Technology Centre

On 06/10/2011 11:21 AM, Scott Garman wrote:

Hello,

As a data point for where we stand in ensuring our packages are
producing documentation, I have some scripts which build all recipes
(using the output of bitbake -s, not world), and then check that a -doc
package is generated which is populated with files.

A summary of the results using yesterday's master are as follows:

584 recipes in total
307 recipes are building documentation
277 recipes are not building documentation

20 recipes did not build, and are counted above as "not building
documentation"

It's the impression of Saul and I that some of the recipes which do not
build are failing due to dependency issues. In this case, it's generally
not from a missing dependency, but having another recipe built first
which then gets picked up (e.g, during do_configure) and causes a build
failure.

For comparison, here were the results from last month (May 6th):

591 recipes in total
308 recipes are building documentation
283 recipes are not building documentation

31 recipes did not build

The lists of recipes are attached to this email. We'd like to improve
the percentage of recipes that produce documentation (separated into
-doc packages, of course) for our next major release in October.

Our userspace recipe maintainers should look into setting aside some
time to know which of their recipes are in the "not building
documentation" list and work to improve them. A lot of this is likely to
be low-hanging fruit.

I have pending fixes for a number of the build_errors list, many where dependency issues that build on a full build, I am going to research those.

We have bugs for alsa-tools, gobject-interospection and clutter-box2d.

So the recipe maintainers really just need to look at the not building docs list, not the errors list at this time.

Sau!

Thanks,

Scott



_______________________________________________
yocto mailing list
yocto@...
https://lists.yoctoproject.org/listinfo/yocto

Thanks Joshua. I assume that
http://www.yoctoproject.org/docs/1.1/yocto-project-qs/yocto-project-qs.html
won't be updated until the next release (if this makes it into the
next release)? And the patch should then be updated to match the new
version numbers?

--
Daryl

Please let me know if you would like to add anything else to the Agenda. - Song

Agenda:

- Final check on Yocto 1.1 M1 release - 5 min (Song)
- Review Yocto 1.1 M2 schedule and Release Criteria - 20 min (Song)
See details at: https://wiki.yoctoproject.org/wiki/Yocto_Project_v1.1_Release_Criteria
- Design check (Sprint B, C features) - 20 min (Song)
- Opens - 10 min



Tuesday, June 14, 2011, 08:00 AM US Pacific Time
916-356-2663, 8-356-2663, Bridge: 94, Passcode: 5995524
Speed dialer: inteldialer://94,5995524 | Learn more

Hello,

As a data point for where we stand in ensuring our packages are producing documentation, I have some scripts which build all recipes (using the output of bitbake -s, not world), and then check that a -doc package is generated which is populated with files.

A summary of the results using yesterday's master are as follows:

584 recipes in total
307 recipes are building documentation
277 recipes are not building documentation

20 recipes did not build, and are counted above as "not building documentation"

It's the impression of Saul and I that some of the recipes which do not build are failing due to dependency issues. In this case, it's generally not from a missing dependency, but having another recipe built first which then gets picked up (e.g, during do_configure) and causes a build failure.

For comparison, here were the results from last month (May 6th):

591 recipes in total
308 recipes are building documentation
283 recipes are not building documentation

31 recipes did not build

The lists of recipes are attached to this email. We'd like to improve the percentage of recipes that produce documentation (separated into -doc packages, of course) for our next major release in October.

Our userspace recipe maintainers should look into setting aside some time to know which of their recipes are in the "not building documentation" list and work to improve them. A lot of this is likely to be low-hanging fruit.

Thanks,

Scott

--
Scott Garman
Embedded Linux Engineer - Yocto Project
Intel Open Source Technology Center

This allows meta-intel sub-layers to coexist with meta-yocto when building
formfactor for different machines with the same layer setup.

Signed-off-by: Paul Eggleton <paul.eggleton@...>
---
.../recipes-bsp/formfactor/formfactor_0.0.bbappend | 2 +-
.../recipes-bsp/formfactor/formfactor_0.0.bbappend | 2 +-
.../recipes-bsp/formfactor/formfactor_0.0.bbappend | 2 +-
.../recipes-bsp/formfactor/formfactor_0.0.bbappend | 2 +-
.../recipes-bsp/formfactor/formfactor_0.0.bbappend | 2 +-
.../recipes-bsp/formfactor/formfactor_0.0.bbappend | 2 +-
6 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/meta-crownbay/recipes-bsp/formfactor/formfactor_0.0.bbappend b/meta-crownbay/recipes-bsp/formfactor/formfactor_0.0.bbappend
index 4a41d48..54da0ff 100644
--- a/meta-crownbay/recipes-bsp/formfactor/formfactor_0.0.bbappend
+++ b/meta-crownbay/recipes-bsp/formfactor/formfactor_0.0.bbappend
@@ -1,3 +1,3 @@
-FILESEXTRAPATHS := "${THISDIR}/${PN}"
+FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"

PRINC = "1"
diff --git a/meta-emenlow/recipes-bsp/formfactor/formfactor_0.0.bbappend b/meta-emenlow/recipes-bsp/formfactor/formfactor_0.0.bbappend
index 4a41d48..54da0ff 100644
--- a/meta-emenlow/recipes-bsp/formfactor/formfactor_0.0.bbappend
+++ b/meta-emenlow/recipes-bsp/formfactor/formfactor_0.0.bbappend
@@ -1,3 +1,3 @@
-FILESEXTRAPATHS := "${THISDIR}/${PN}"
+FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"

PRINC = "1"
diff --git a/meta-fishriver/recipes-bsp/formfactor/formfactor_0.0.bbappend b/meta-fishriver/recipes-bsp/formfactor/formfactor_0.0.bbappend
index 4a41d48..54da0ff 100644
--- a/meta-fishriver/recipes-bsp/formfactor/formfactor_0.0.bbappend
+++ b/meta-fishriver/recipes-bsp/formfactor/formfactor_0.0.bbappend
@@ -1,3 +1,3 @@
-FILESEXTRAPATHS := "${THISDIR}/${PN}"
+FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"

PRINC = "1"
diff --git a/meta-jasperforest/recipes-bsp/formfactor/formfactor_0.0.bbappend b/meta-jasperforest/recipes-bsp/formfactor/formfactor_0.0.bbappend
index 4a41d48..54da0ff 100644
--- a/meta-jasperforest/recipes-bsp/formfactor/formfactor_0.0.bbappend
+++ b/meta-jasperforest/recipes-bsp/formfactor/formfactor_0.0.bbappend
@@ -1,3 +1,3 @@
-FILESEXTRAPATHS := "${THISDIR}/${PN}"
+FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"

PRINC = "1"
diff --git a/meta-n450/recipes-bsp/formfactor/formfactor_0.0.bbappend b/meta-n450/recipes-bsp/formfactor/formfactor_0.0.bbappend
index 4a41d48..54da0ff 100644
--- a/meta-n450/recipes-bsp/formfactor/formfactor_0.0.bbappend
+++ b/meta-n450/recipes-bsp/formfactor/formfactor_0.0.bbappend
@@ -1,3 +1,3 @@
-FILESEXTRAPATHS := "${THISDIR}/${PN}"
+FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"

PRINC = "1"
diff --git a/meta-sugarbay/recipes-bsp/formfactor/formfactor_0.0.bbappend b/meta-sugarbay/recipes-bsp/formfactor/formfactor_0.0.bbappend
index 4a41d48..54da0ff 100644
--- a/meta-sugarbay/recipes-bsp/formfactor/formfactor_0.0.bbappend
+++ b/meta-sugarbay/recipes-bsp/formfactor/formfactor_0.0.bbappend
@@ -1,3 +1,3 @@
-FILESEXTRAPATHS := "${THISDIR}/${PN}"
+FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"

PRINC = "1"
--
1.7.4.1

The following changes since commit 01d1a545477cbc1b2a1da5b66edb52fd9e783206:

meta-intel: update kernel SRCREVs (2011-06-06 16:13:01 -0500)

are available in the git repository at:
git://git.yoctoproject.org/poky-contrib paule/meta-intel-paths
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=paule/meta-intel-paths

Paul Eggleton (1):
meta-intel: fix formfactor bbappends to use FILESEXTRAPATHS_prepend

.../recipes-bsp/formfactor/formfactor_0.0.bbappend | 2 +-
.../recipes-bsp/formfactor/formfactor_0.0.bbappend | 2 +-
.../recipes-bsp/formfactor/formfactor_0.0.bbappend | 2 +-
.../recipes-bsp/formfactor/formfactor_0.0.bbappend | 2 +-
.../recipes-bsp/formfactor/formfactor_0.0.bbappend | 2 +-
.../recipes-bsp/formfactor/formfactor_0.0.bbappend | 2 +-
6 files changed, 6 insertions(+), 6 deletions(-)

--
1.7.4.1

Just for interest, for the first time ever I did a dump out of profile
information on the python side of bitbake's task execution for all tasks
of a core-image-sato build.

I enabled profiling of each forked off task with this patch:

http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/temp5&id=c221b4174627ac24f4c9cae687b4f015a5b6523a

and then combined the results together in the attached profile report.
Most of it isn't a big surprise but there are some interesting numbers
such as the 278 million calls to _keys() in data_smart. For that
specific issue I have a patch which I'm considering:

http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/temp5&id=f549077bbd529dfa5ff6ef75ef9432a823b3a3f5

as experience shows that function calls have overhead in python at that
number of iterations and reworking the code likely will reduce overhead.
I've yet to profile the result on this scale though.

Overall, a build makes over 10 billion python function calls. We access
data store variable flags 150 million times.

Not shown here but interestingly, early in the build process, the number
of calls to time.sleep() was low, at the end of this build its rather
significantly higher showing bitbake was hitting its idle handler a lot
more. Time spend there is an illusion as the system should have yielded
to other processes.

There are probably some other interesting things to be noticed somewhere
in that data :)

Cheers,

Richard

Op 10 jun 2011, om 10:44 heeft Richard Purdie het volgende geschreven:

We've merged a number of things which should have helped performance
recently. Changes in my test included:

* Splitting out the locale generation [not merged yet]
* Not building tar-replacement-native when we don't need it
* Cleaned up dependencies for native/nativesdk
* Cleaned up dependencies for all-arch packages
* Reduced dbus-native dependencies

The time for the benchmark timing test against master with the locale
split patch applied was:

real 105m5.813s
user 386m39.170s
sys 59m40.250s

so not that much different from where we have been :(.

I did observe things through the build process:

* The first part of the build was *fast* getting through the first 2000
of 4295 tasks in about 10 minutes.
* The CPU usage was pretty much full for that time
* The next 5 minutes covered the next 150 tasks
* gettext-native didn't seem to get built until surprisingly late in
the process
* There was a highly serialised chain of gettext-native, libelf-native,
binutils-native, binutils-cross etc.
* Once eglibc/libgcc completed, processor usage was high again for the
rest of the build

I'm hoping my BB_NUMBER_THREADS (12 on a quad core machine) was too high
causing too much context switching and as the build process might now be
more efficient, lowering it might help. I am a bit disappointed in those
numbers though.

Do you have pretty bootchart graphs for that?

regards,

Koen

We've merged a number of things which should have helped performance
recently. Changes in my test included:

* Splitting out the locale generation [not merged yet]
* Not building tar-replacement-native when we don't need it
* Cleaned up dependencies for native/nativesdk
* Cleaned up dependencies for all-arch packages
* Reduced dbus-native dependencies

The time for the benchmark timing test against master with the locale
split patch applied was:

real 105m5.813s
user 386m39.170s
sys 59m40.250s

so not that much different from where we have been :(.

I did observe things through the build process:

* The first part of the build was *fast* getting through the first 2000
of 4295 tasks in about 10 minutes.
* The CPU usage was pretty much full for that time
* The next 5 minutes covered the next 150 tasks
* gettext-native didn't seem to get built until surprisingly late in
the process
* There was a highly serialised chain of gettext-native, libelf-native,
binutils-native, binutils-cross etc.
* Once eglibc/libgcc completed, processor usage was high again for the
rest of the build

I'm hoping my BB_NUMBER_THREADS (12 on a quad core machine) was too high
causing too much context switching and as the build process might now be
more efficient, lowering it might help. I am a bit disappointed in those
numbers though.

Cheers,

Richard

On Thu, 2011-06-09 at 11:17 -0700, Daryl Spitzer wrote:

Looks like we fixed this issue in our recent point release. Could you
try http://www.yoctoproject.org/downloads/poky/poky-bernard-5.0.1.tar.bz2 ?

I tried it, and am getting what I think is an unrelated error. I'll
start a new thread for that.

Shouldn't documentation references to
"http://www.yoctoproject.org/downloads/poky/poky-bernard-5.0.tar.bz2"
be changed to "http://www.yoctoproject.org/downloads/poky/poky-bernard-5.0.1.tar.bz"
(such as in http://www.yoctoproject.org/docs/1.1/yocto-project-qs/yocto-project-qs.html)?

You are right Daryl, apologies for any inconvenience. I've filed a bug
with a patch attached for the Quick Start doc.

http://bugzilla.yoctoproject.org/show_bug.cgi?id=1153

Regards,
Joshua
--
Joshua Lock
Yocto Build System Monkey
Intel Open Source Technology Centre

On Thu, 2011-06-09 at 15:17 -0700, Daryl Spitzer wrote:

How do I find out where DL_DIR is pointing?

I found this line, commented out in conf/local.conf:

#DL_DIR ?= "${TOPDIR}/downloads"

It's not listed after I run `bitbake -k poky-image-sato`:

OE Build Configuration:
BB_VERSION = "1.11.0"
METADATA_BRANCH = "<unknown>"
METADATA_REVISION = "<unknown>"
TARGET_ARCH = "i586"
TARGET_OS = "linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "1.0.1"
TARGET_FPU = ""

And there is no DL_DIR environment variable defined (after I run
`source poky-bernard-5.0.1/poky-init-build-env poky-5.0.1-build`).

There is a poky-5.0.1-build/downloads directory. I'll assume that's
it. But I'd like to learn how to find out without guessing.

You can dump the bitbake datastore and find it like this:

bitbake -e | grep ^DL_DIR

Also, I think I have a patch for your problem:

diff --git a/bitbake/lib/bb/fetch2/__init__.py b/bitbake/lib/bb/fetch2/__init__.py
index 27fcc3c..02a36b5 100644
--- a/bitbake/lib/bb/fetch2/__init__.py
+++ b/bitbake/lib/bb/fetch2/__init__.py
@@ -203,7 +203,10 @@ def uri_replace(ud, uri_find, uri_replace, d):
result_decoded[loc] = uri_decoded[loc]
if isinstance(i, basestring):
if (re.match(i, uri_decoded[loc])):
- result_decoded[loc] = re.sub(i, uri_replace_decoded[loc], uri_decoded[loc])
+ if not uri_replace_decoded[loc]:
+ result_decoded[loc] = ""
+ else:
+ result_decoded[loc] = re.sub(i, uri_replace_decoded[loc], uri_decoded[loc])
if uri_find_decoded.index(i) == 2:
if ud.mirrortarball:
result_decoded[loc] = os.path.join(os.path.dirname(result_decoded[loc]), os.path.basename(ud.mirrortarball))

although I'm running some further tests on this to confirm it...

Cheers,

Richard