Date   

Re: bitbake controlling memory use

Ferry Toth
 

Hi,

On 10-01-2023 02:57, Randy MacLeod wrote:
Add Zheng.
Switch to Trevor's gmail since he might still be interested in this.

On 2023-01-09 05:49, Ferry Toth via lists.yoctoproject.org wrote:
Hi

On 09-01-2023 11:39, Alexander Kanavin wrote:
On Sun, 8 Jan 2023 at 23:13, Ferry Toth <fntoth@...> wrote:

Now it works I'm not sure what to do. Richard marked the original patch
as WIP. Maybe it's not appropriate for including into poky? If so I
wouldn't mind carrying it in meta-intel-edison.

Or may be with a little work (from me) and a run on the CI servers we
could make it go in?
I'd rather teach make/ninja upstream to watch memory consumption
and/or host pressure directly (e.g. similar to how it handles -l). And
offer any resulting patches to upstream first.
Zheng and I *started* on that for make over the Xmas holiday.
See the (poorly formatted) thread:
  Add support for limiting CPU pressure, contrib, 2022/12/20
in:
https://lists.gnu.org/archive/html/bug-make/2022-12/threads.html

There were mixed reviews upstream with the maintainer, Paul Smith,
seeming to prefer that we investigate the job server approach and the current
load averaging. I'll happily try to find time to play with Ferry's job server work.
The work is actually Richard's. I only fixed what broke over time.
I've been thinking about the various workflows and as Richard said, it seems
that for many people who only do one build at a time, the job server and maybe
a little linker restraint, would be all that's needed. For activities such
as YP AB, we could teach the main job server to also look at /proc/pressure
as a way to limit the pool size we could make a meta-jobserver for those who don't
want/need to worry about non-compile tasks such as tests and build clean-up.


Note that cargo has the notion of a job server:
   https://github.com/rust-lang/cargo/issues/1744
   https://github.com/rust-lang/cargo/pull/4110


and ninja has an open issue:
   https://github.com/ninja-build/ninja/issues/1139
and an initial implementation:
   https://github.com/stefanb2/ninja/tree/topic-jobserver-fifo

What other build tools are in need of regulation and/or job server patches?
What I read, gcc has already -flto=jobserver.

Other (single threaded CPU intensive) might just be started from jobclient?

../Randy



Alex
Yeah, I'd rather teach the kernel to consider thrashing when scheduling jobs. As is now any user process can slow down any other users process and even the kernel itself to a standstill.

But kernel developers consider those issues "orthogonal" (i.e. they don't want to make the scheduler aware of io).




Re: bitbake controlling memory use

Randy MacLeod
 

Add Zheng.
Switch to Trevor's gmail since he might still be interested in this.

On 2023-01-09 05:49, Ferry Toth via lists.yoctoproject.org wrote:
Hi
On 09-01-2023 11:39, Alexander Kanavin wrote:
On Sun, 8 Jan 2023 at 23:13, Ferry Toth <fntoth@...> wrote:

Now it works I'm not sure what to do. Richard marked the original patch
as WIP. Maybe it's not appropriate for including into poky? If so I
wouldn't mind carrying it in meta-intel-edison.

Or may be with a little work (from me) and a run on the CI servers we
could make it go in?
I'd rather teach make/ninja upstream to watch memory consumption
and/or host pressure directly (e.g. similar to how it handles -l). And
offer any resulting patches to upstream first.
Zheng and I *started* on that for make over the Xmas holiday.
See the (poorly formatted) thread:
Add support for limiting CPU pressure, contrib, 2022/12/20
in:
https://lists.gnu.org/archive/html/bug-make/2022-12/threads.html

There were mixed reviews upstream with the maintainer, Paul Smith,
seeming to prefer that we investigate the job server approach and the current
load averaging. I'll happily try to find time to play with Ferry's job server work.

I've been thinking about the various workflows and as Richard said, it seems
that for many people who only do one build at a time, the job server and maybe
a little linker restraint, would be all that's needed. For activities such
as YP AB, we could teach the main job server to also look at /proc/pressure
as a way to limit the pool size we could make a meta-jobserver for those who don't
want/need to worry about non-compile tasks such as tests and build clean-up.


Note that cargo has the notion of a job server:
https://github.com/rust-lang/cargo/issues/1744
https://github.com/rust-lang/cargo/pull/4110


and ninja has an open issue:
https://github.com/ninja-build/ninja/issues/1139
and an initial implementation:
https://github.com/stefanb2/ninja/tree/topic-jobserver-fifo

What other build tools are in need of regulation and/or job server patches?

../Randy



Alex
Yeah, I'd rather teach the kernel to consider thrashing when scheduling jobs. As is now any user process can slow down any other users process and even the kernel itself to a standstill.
But kernel developers consider those issues "orthogonal" (i.e. they don't want to make the scheduler aware of io).
--
# Randy MacLeod
# Wind River Linux


Enhancements/Bugs closed WW01!

Stephen Jolley
 

All,

The below were the owners of enhancements or bugs closed during the last week!

Who

Count

richard.purdie@...

2

randy.macleod@...

1

michael.opdenacker@...

1

alexandre.belloni@...

1

Grand Total

5

Thanks,

 

Stephen K. Jolley

Yocto Project Program Manager

(    Cell:                (208) 244-4460

* Email:              sjolley.yp.pm@...

 


Current high bug count owners for Yocto Project 4.2

Stephen Jolley
 

All,

Below is the list as of top 35 bug owners as of the end of WW01 of who have open medium or higher bugs and enhancements against YP 4.2.   There are 76 possible work days left until the final release candidates for YP 4.2 needs to be released.

Who

Count

michael.opdenacker@...

34

ross.burton@...

30

richard.purdie@...

25

bruce.ashfield@...

25

randy.macleod@...

25

david.reyna@...

23

JPEWhacker@...

10

sakib.sajal@...

8

saul.wold@...

5

pavel@...

5

pidge@...

4

tim.orling@...

4

Zheng.Qiu@...

3

alexis.lothore@...

2

bluelightning@...

2

rybczynska@...

2

sgw@...

2

Naveen.Gowda@...

2

jon.mason@...

2

sundeep.kokkonda@...

2

sundeep.kokkonda@...

1

mathew.prokos@...

1

aehs29@...

1

mhalstead@...

1

yoann.congal@...

1

akuster808@...

1

Anton.Antonov@...

1

alexandre.belloni@...

1

yashinde145@...

1

throos@...

1

bst@...

1

hongxu.jia@...

1

Martin.Jansa@...

1

thomas.perrot@...

1

tvgamblin@...

1

Grand Total

230

Thanks,

 

Stephen K. Jolley

Yocto Project Program Manager

(    Cell:                (208) 244-4460

* Email:              sjolley.yp.pm@...

 


Yocto Project Newcomer & Unassigned Bugs - Help Needed

Stephen Jolley
 

All,

 

The triage team is starting to try and collect up and classify bugs which a newcomer to the project would be able to work on in a way which means people can find them. They're being listed on the triage page under the appropriate heading:

https://wiki.yoctoproject.org/wiki/Bug_Triage#Newcomer_Bugs  Also please review: https://www.openembedded.org/wiki/How_to_submit_a_patch_to_OpenEmbedded and how to create a bugzilla account at: https://bugzilla.yoctoproject.org/createaccount.cgi

The idea is these bugs should be straight forward for a person to help work on who doesn't have deep experience with the project.  If anyone can help, please take ownership of the bug and send patches!  If anyone needs help/advice there are people on irc who can likely do so, or some of the more experienced contributors will likely be happy to help too.

 

Also, the triage team meets weekly and does its best to handle the bugs reported into the Bugzilla. The number of people attending that meeting has fallen, as have the number of people available to help fix bugs. One of the things we hear users report is they don't know how to help. We (the triage team) are therefore going to start reporting out the currently 417 unassigned or newcomer bugs.

 

We're hoping people may be able to spare some time now and again to help out with these.  Bugs are split into two types, "true bugs" where things don't work as they should and "enhancements" which are features we'd want to add to the system.  There are also roughly four different "priority" classes right now,  “4.2”, “4.3”, "4.99" and "Future", the more pressing/urgent issues being in "4.2" and then “4.3”.

 

Please review this link and if a bug is something you would be able to help with either take ownership of the bug, or send me (sjolley.yp.pm@...) an e-mail with the bug number you would like and I will assign it to you (please make sure you have a Bugzilla account).  The list is at: https://wiki.yoctoproject.org/wiki/Bug_Triage_Archive#Unassigned_or_Newcomer_Bugs

 

Thanks,

 

Stephen K. Jolley

Yocto Project Program Manager

(    Cell:                (208) 244-4460

* Email:              sjolley.yp.pm@...

 


Re: Git repo setup for Yocto upstream contribution

Alexandre Belloni
 

On 09/01/2023 19:30:36+0100, Alexander Kanavin wrote:
On Mon, 9 Jan 2023 at 19:23, Alexandre Belloni
<alexandre.belloni@...> wrote:
No, please send patches against the correct git tree, else you are
putting more load on the maintainers than necessary.
This was the cause of:
https://lore.kernel.org/all/Y5Sr9HSWIc5LVzl9@mail.local/

This can silently fail and then you need to wait for 7hours for the AB
to fail and this delays testing for one more day.
We never got to the bottom of what happened to that one. Part of the
patch applied correctly, the other part did not.
Just try to apply your patch on a bitbake repository, it will reproduce.


--
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com


Re: Git repo setup for Yocto upstream contribution

Alexander Kanavin
 

On Mon, 9 Jan 2023 at 19:23, Alexandre Belloni
<alexandre.belloni@...> wrote:
No, please send patches against the correct git tree, else you are
putting more load on the maintainers than necessary.
This was the cause of:
https://lore.kernel.org/all/Y5Sr9HSWIc5LVzl9@mail.local/

This can silently fail and then you need to wait for 7hours for the AB
to fail and this delays testing for one more day.
We never got to the bottom of what happened to that one. Part of the
patch applied correctly, the other part did not.

Alex


Re: Git repo setup for Yocto upstream contribution

Alexandre Belloni
 

On 09/01/2023 10:29:13+0100, Alexander Kanavin wrote:
I've been making patches against an integrated poky checkout for many
years, including bitbake patches. They're usually correctly picked up
and applied; you only need to ensure you route them to correct mailing
lists.
No, please send patches against the correct git tree, else you are
putting more load on the maintainers than necessary.
This was the cause of:
https://lore.kernel.org/all/Y5Sr9HSWIc5LVzl9@mail.local/

This can silently fail and then you need to wait for 7hours for the AB
to fail and this delays testing for one more day.

Alex

On Sat, 7 Jan 2023 at 14:05, Yoann Congal <yoann.congal@...> wrote:

Hi,

Since I plan to work regularly on upstream Yocto and contributing, I
wonder about the best way to best setup my git repos to do that...

I've started with the upstream poky repo but when I make a commit on,
for example, bitbake, my patch applies on a bitbake/some-path file which
does not exist in the bitbake repo (only "some-path" exists in the
bitbake repo).

Is this something I should address ? I found some patches for bitbake in
the mailing list that are for "bitbake/..." paths... So, maybe, basing
my patches on the poky repo is totally OK for now...?

To the Yocto maintainers, how do you manage this (poky repo, combo-layer
and individual bitbake/doc/meta-yocto repo) ?

Note: I have found the "combo-layer splitpatch" tool but it does not
look like a full solution... Maybe I'm missing something?

Thanks!
--
Yoann Congal ("yocton" on IRC)
Smile ECS - Tech Expert





--
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com


Re: [qa-build-notification] QA notification for completed autobuilder build (yocto-4.1.2.rc1)

Jing Hui Tham
 

Hi all,

Intel and WR YP QA is planning for QA execution for YP build yocto-4.1.2.rc1. We are planning to execute following tests for this cycle:

OEQA-manual tests for following module:
1. OE-Core
2. BSP-hw

Runtime auto test for following platforms:
1. MinnowTurbot 32-bit
2. NUC 7
3. ADL
4. TGL NUC 11
5. Edgerouter
6. Beaglebone

ETA for completion Thursday, 12th Jan 2023.

Best regards,
Jing Hui

-----Original Message-----
From: qa-build-notification@... <qa-build-
notification@...> On Behalf Of Pokybuild User
Sent: Saturday, 7 January, 2023 9:12 PM
To: yocto@...
Cc: qa-build-notification@...
Subject: [qa-build-notification] QA notification for completed autobuilder build
(yocto-4.1.2.rc1)


A build flagged for QA (yocto-4.1.2.rc1) was completed on the autobuilder and is
available at:


https://autobuilder.yocto.io/pub/releases/yocto-4.1.2.rc1


Build hash information:

bitbake: f0f166aee766b4bb1f8cf8b35dfc7d406c75e6a4
meta-agl: 6fe6a57bffdded98347c2eab53463179b643904c
meta-arm: 898a4aa99bd7c7a30a0613c56bb3cc6607c5b020
meta-aws: 173b444a1e49b2497e4840ae70319cc868c1bae4
meta-intel: 09e30d85e2e405e2f4a0e35c5d72aa285a2d8cbc
meta-mingw: b0067202db8573df3d23d199f82987cebe1bee2c
meta-openembedded: c354f92778c1d4bcd3680af7e0fb0d1414de2344
meta-virtualization: 8857b36ebfec3d548755755b009adc491ef320ab
oecore: 670f4f103b25897524d115c1f290ecae441fe4bd
poky: 74c92e38c701e268406bb656b45ccd68471c217e



This is an automated message from the Yocto Project Autobuilder
Git: git://git.yoctoproject.org/yocto-autobuilder2
Email: richard.purdie@...







Re: Git repo setup for Yocto upstream contribution

Yoann Congal
 

On 1/9/23 10:29, Alexander Kanavin wrote:
I've been making patches against an integrated poky checkout for many
years, including bitbake patches. They're usually correctly picked up
and applied; you only need to ensure you route them to correct mailing
lists.
Ok, I'll keep it that way.

Thanks!

Alex
On Sat, 7 Jan 2023 at 14:05, Yoann Congal <yoann.congal@...> wrote:

Hi,

Since I plan to work regularly on upstream Yocto and contributing, I
wonder about the best way to best setup my git repos to do that...

I've started with the upstream poky repo but when I make a commit on,
for example, bitbake, my patch applies on a bitbake/some-path file which
does not exist in the bitbake repo (only "some-path" exists in the
bitbake repo).

Is this something I should address ? I found some patches for bitbake in
the mailing list that are for "bitbake/..." paths... So, maybe, basing
my patches on the poky repo is totally OK for now...?

To the Yocto maintainers, how do you manage this (poky repo, combo-layer
and individual bitbake/doc/meta-yocto repo) ?

Note: I have found the "combo-layer splitpatch" tool but it does not
look like a full solution... Maybe I'm missing something?

Thanks!
--
Yoann Congal ("yocton" on IRC)
Smile ECS - Tech Expert
--
Yoann Congal
Smile ECS - Tech Expert


Re: bitbake controlling memory use

Ferry Toth
 

Hi

On 09-01-2023 11:43, Richard Purdie wrote:
On Mon, 2023-01-09 at 11:39 +0100, Alexander Kanavin wrote:
On Sun, 8 Jan 2023 at 23:13, Ferry Toth <fntoth@...> wrote:

Now it works I'm not sure what to do. Richard marked the original patch
as WIP. Maybe it's not appropriate for including into poky? If so I
wouldn't mind carrying it in meta-intel-edison.

Or may be with a little work (from me) and a run on the CI servers we
could make it go in?
I'd rather teach make/ninja upstream to watch memory consumption
and/or host pressure directly (e.g. similar to how it handles -l). And
offer any resulting patches to upstream first.
Whilst teaching make/ninja about pressure is probably a nice idea, I
think there is a use case for sharing the job pool between bitbake
tasks rather than having a job pool per task since that scales badly.

One issue that comes to mind with the patch is that it is currently
writing into /tmp/ and we likely need to work out a better location for
the job server fifo. Currently that would break things on the
autobuilder as there are multiple builds on a given host.

Ha, I didn't consider multiple bitbakes running simultaneously.

Maybe the location /tmp is fine but we would need a unique (random?) filename instead of makefifo?

Cheers,

Richard



kirkstone: issue with fetching sources using tag

Marek Belisko
 

Hi,

I'm moving my layers from dunfell to kirkstone and for some recipes
I'm hitting an issue with following fetch issue:
Bitbake Fetcher Error: FetchError("Recipe uses a floating tag/branch
without a fixed SRCREV yet doesn't call bb.fetch2.get_srcrev() (use
SRCPV in PV for OE).", None)

My SRC_URI looks like this:

SRC_URI = "git://mysupersecretgitproject.git;protocol=https;branch=master;tag=v${PV}"
and PV is properly set to tag name.

By adding this to beginning of recipe it seems it start working:
python() {
bb.warn("SRCPV:%s"%d.getVar("SRCPV"))
}

Any ideas what am I missing?

Thanks and regards,

marek

--
as simple and primitive as possible
-------------------------------------------------
Marek Belisko - OPEN-NANDRA
Freelance Developer

Ruska Nova Ves 219 | Presov, 08005 Slovak Republic
Tel: +421 915 052 184
skype: marekwhite
twitter: #opennandra
web: http://open-nandra.com


Re: bitbake controlling memory use

Ferry Toth
 

Hi

On 09-01-2023 11:39, Alexander Kanavin wrote:
On Sun, 8 Jan 2023 at 23:13, Ferry Toth <fntoth@...> wrote:

Now it works I'm not sure what to do. Richard marked the original patch
as WIP. Maybe it's not appropriate for including into poky? If so I
wouldn't mind carrying it in meta-intel-edison.

Or may be with a little work (from me) and a run on the CI servers we
could make it go in?
I'd rather teach make/ninja upstream to watch memory consumption
and/or host pressure directly (e.g. similar to how it handles -l). And
offer any resulting patches to upstream first.

Alex
Yeah, I'd rather teach the kernel to consider thrashing when scheduling jobs. As is now any user process can slow down any other users process and even the kernel itself to a standstill.

But kernel developers consider those issues "orthogonal" (i.e. they don't want to make the scheduler aware of io).


Re: bitbake controlling memory use

Richard Purdie
 

On Mon, 2023-01-09 at 11:39 +0100, Alexander Kanavin wrote:
On Sun, 8 Jan 2023 at 23:13, Ferry Toth <fntoth@...> wrote:

Now it works I'm not sure what to do. Richard marked the original patch
as WIP. Maybe it's not appropriate for including into poky? If so I
wouldn't mind carrying it in meta-intel-edison.

Or may be with a little work (from me) and a run on the CI servers we
could make it go in?
I'd rather teach make/ninja upstream to watch memory consumption
and/or host pressure directly (e.g. similar to how it handles -l). And
offer any resulting patches to upstream first.
Whilst teaching make/ninja about pressure is probably a nice idea, I
think there is a use case for sharing the job pool between bitbake
tasks rather than having a job pool per task since that scales badly.

One issue that comes to mind with the patch is that it is currently
writing into /tmp/ and we likely need to work out a better location for
the job server fifo. Currently that would break things on the
autobuilder as there are multiple builds on a given host.

Cheers,

Richard


Re: bitbake controlling memory use

Alexander Kanavin
 

On Sun, 8 Jan 2023 at 23:13, Ferry Toth <fntoth@...> wrote:

Now it works I'm not sure what to do. Richard marked the original patch
as WIP. Maybe it's not appropriate for including into poky? If so I
wouldn't mind carrying it in meta-intel-edison.

Or may be with a little work (from me) and a run on the CI servers we
could make it go in?
I'd rather teach make/ninja upstream to watch memory consumption
and/or host pressure directly (e.g. similar to how it handles -l). And
offer any resulting patches to upstream first.

Alex


Re: Git repo setup for Yocto upstream contribution

Alexander Kanavin
 

I've been making patches against an integrated poky checkout for many
years, including bitbake patches. They're usually correctly picked up
and applied; you only need to ensure you route them to correct mailing
lists.

Alex

On Sat, 7 Jan 2023 at 14:05, Yoann Congal <yoann.congal@...> wrote:

Hi,

Since I plan to work regularly on upstream Yocto and contributing, I
wonder about the best way to best setup my git repos to do that...

I've started with the upstream poky repo but when I make a commit on,
for example, bitbake, my patch applies on a bitbake/some-path file which
does not exist in the bitbake repo (only "some-path" exists in the
bitbake repo).

Is this something I should address ? I found some patches for bitbake in
the mailing list that are for "bitbake/..." paths... So, maybe, basing
my patches on the poky repo is totally OK for now...?

To the Yocto maintainers, how do you manage this (poky repo, combo-layer
and individual bitbake/doc/meta-yocto repo) ?

Note: I have found the "combo-layer splitpatch" tool but it does not
look like a full solution... Maybe I'm missing something?

Thanks!
--
Yoann Congal ("yocton" on IRC)
Smile ECS - Tech Expert



Re: roundup feature for Wic partitions?

Alexander Kanavin
 

There is; the source code for wic and its tests are fully available to
you, so you are welcome to implement the missing bits and submit them
for review and inclusion.

Also, we do not have a wic maintainer.

Alex

On Sat, 7 Jan 2023 at 00:29, Benjamin Mordaunt
<crawford.benjamin15@...> wrote:


Hi,

I am trying to configure an image to conform to the VHD size alignment
requirements of Azure. Azure requires that the filesize of a VHD be a
multiple of 1 MiB. The '--align' option works to align the _start_ of a
partition to a particular multiple, but I need the _end_ to be aligned.
As a workaround, I tried:

part /fake --fixed-size=0 --no-table --align=1024

I figured this would create a zero-sized partition which would be
appropriately aligned at the end of the image, but wic complains that
zero-sized partitions are not allowed.

Is there a way to achieve this, or extend the capabilities of wic to
handle this use case?

Best,
Ben



Re: How to make root file system writable?

Alexander Kanavin
 

It helps if you show your build configuration, e.g. the output of
'bitbake -e core-image-full-cmdline'.

Alex

On Mon, 9 Jan 2023 at 09:01, <jovanbosic95@...> wrote:

After bitbaking core-image-full-cmdline, my root file system is readable only. How to make it writable? Is there something I have to change in local.conf?


How to make root file system writable?

jovanbosic95@...
 

After bitbaking core-image-full-cmdline, my root file system is readable only. How to make it writable? Is there something I have to change in local.conf?


Re: bitbake controlling memory use

Ferry Toth
 

Hi,

Just in case you are interested in the result.

Op 03-01-2023 om 18:33 schreef Alexander Kanavin:
On Tue, 3 Jan 2023 at 16:58, Ferry Toth <fntoth@...> wrote:

I know you don't want to spend time on this. So don't. Others may have
an interest or a tip to improve Yocto in this area.

I'm just trying to get an old patch from Richard Purdie to work
(http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/wipqueue4&id=d66a327fb6189db5de8bc489859235dcba306237)
with later fixes by Trevor Gamblin.
I got Richards patch "Add shared make jobserver support <https://github.com/htot/poky/commit/28fd4b9e6e60bdd07352827ec713510ae86a39c6>" working again. It was a bit more work then last time, it was broken in 3 places (poky, make and python). Find it here: https://github.com/htot/poky/commits/kirkstone

It works by making each make believe it is a submake, while the named pipe holding the execution tokens is created by bitbake.

Of course as is it only limits number of threads started by make, not ninja. On my machine (16 threads) while compiling (linux, nodejs, nodejs-native etc.) simultaneously this limits memory consumption to ~14GB RAM. A little less then 1GB/thread.

Linking nodejs is another story. Each thread uses 4-5GB, and the recipe allows 5 threads simultaneously. With a nodejs_%.bbappend:

EXTRA_OEMAKE:prepend = "\
LINK='flock /tmp ${CXX}' \
"
only one thread will be linked simultaneously.

This brought my image build time from 3h > 1u47m (that is after removing out and sstate, not bbcache so excluding download time for sources).

Or alternatively something based on https://github.com/olsner/jobclient.
In principle we could use jobclient to start other programs then make, but for meta-intel-edison I didn't see any that would benefit from this.

It would be nice if ninja could use the same named pipe for the execution tokens as make, that might be for another holiday.

By all means please do. Everyone will benefit from teaching make to
abstain from starting new processes if RAM is tight, even people with
xeon/epyc grade hardware. It's not that I don't want to spend time on
this, it's simply that I do not have the time.
Ah, yes, that is a known problem. Like I said, this was for me just a little holiday project.

Now it works I'm not sure what to do. Richard marked the original patch as WIP. Maybe it's not appropriate for including into poky? If so I wouldn't mind carrying it in meta-intel-edison.

Or may be with a little work (from me) and a run on the CI servers we could make it go in?

Alex