We finally did it!

After getting some final patches yesterday, we made it to 100% with patch Upsteam-Status.

Total Patches Files: 1243
All Upstream-Status: 1243
Fix Upstream-Status: 0
Need Upstream-Status: 0
Pending Upstream-Status: 461

This means we have 461 patches to now work their way into the Upstream
communities.

Let's work to maintain this, I will be watching incoming patches and using a check script to verify that patches have Upstream-Status.

Thanks to all those people who worked to get Upstream-Status into their patches.

Sau!

Saul Wold wrote:

After getting some final patches yesterday, we made it to 100% with
patch Upsteam-Status.

Who sets the Upstream-Status? Are there guidelines how to do it?

I spoke to the author of curl and mentioned the two patches in Yocto against it, both of which are marked as "Upstream-Status: Inappropriate". He said those patches were never submitted to him.

Are we dismissing patches without even giving upstream a chance to comment?

--
Björn

From: Saul Wold [mailto:saul.wold@...]
Sent: Wednesday, February 08, 2012 1:12 AM

We finally did it!

After getting some final patches yesterday, we made it to 100% with patch
Upsteam-Status.

Total Patches Files: 1243
All Upstream-Status: 1243
Fix Upstream-Status: 0
Need Upstream-Status: 0
Pending Upstream-Status: 461

This means we have 461 patches to now work their way into the Upstream
communities.

Let's work to maintain this, I will be watching incoming patches and using a
check script to verify that patches have Upstream-Status.

Thanks to all those people who worked to get Upstream-Status into their
patches.

Nicely done, all! I'm hoping we have can a reputation as a strong supporter of upstream projects, giving back wherever we can.

This sounds fantastic, and I'd love to create a page on the website reflecting this. Just so I am clear, what exactly is this 100% of? Do we have no local patches to upstream projects at all? 

--
Jeff Osier-Mixon http://jefro.net/blog

On Wednesday 08 February 2012 09:34:56 Osier-mixon, Jeffrey wrote:

This sounds fantastic, and I'd love to create a page on the website
reflecting this. Just so I am clear, what exactly is this 100% of? Do we
have no local patches to upstream projects at all?

Not quite - we still have most of the patches, it's just we've now declared on
each and every one whether it can be/has been upstreamed.

Cheers,
Paul

--

Paul Eggleton
Intel Open Source Technology Centre

On Wed, Feb 8, 2012 at 9:34 AM, Osier-mixon, Jeffrey
<jeffrey.osier-mixon@...> wrote:

This sounds fantastic, and I'd love to create a page on the website
reflecting this. Just so I am clear, what exactly is this 100% of? Do we
have no local patches to upstream projects at all?

it means that all patches have a field 'Upstream-Status'
and for most of them it reflects the status of patch w.r.t. upstream
of given package

Ah, documentation :)  excellent

--
Jeff Osier-Mixon http://jefro.net/blog

On 02/08/2012 10:04 AM, Osier-mixon, Jeffrey wrote:

Ah, documentation :) excellent

Jefro:

You can get more info about this from Mark's OE page:
http://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines

The Key thing to note on my numbers is that we have 461 patches that could potentially be up-streamed to other communities, depending the status of those communities, from active communities accepting patches to upstream source that is just download-able with no activity or maintainers.

We have 1243 patches overall, which include OE Specific configuration patches and Embedded specific tweaks to various upstreams that may not be appropriate or acceptable to the upstream community.

Sau!

On Wed, Feb 8, 2012 at 9:45 AM, Khem Raj <raj.khem@...
<mailto:raj.khem@...>> wrote:

On Wed, Feb 8, 2012 at 9:34 AM, Osier-mixon, Jeffrey
<jeffrey.osier-mixon@...
<mailto:jeffrey.osier-mixon@...>> wrote:
> This sounds fantastic, and I'd love to create a page on the website
> reflecting this. Just so I am clear, what exactly is this 100%
of? Do we
> have no local patches to upstream projects at all?

it means that all patches have a field 'Upstream-Status'
and for most of them it reflects the status of patch w.r.t. upstream
of given package




--
Jeff Osier-Mixon http://jefro.net/blog
Yocto Project Community Manager @Intel http://yoctoproject.org
<http://yoctoproject.org/>

On 02/08/2012 02:07 AM, Björn Stenberg wrote:

Saul Wold wrote:

After getting some final patches yesterday, we made it to 100% with
patch Upsteam-Status.

Who sets the Upstream-Status? Are there guidelines how to do it?

The developer of the patch submitted to any OE branch (oe-core, meta-oe, ...) should add the appropriate Upstream-Status entry.

See: http://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines

We had a big push within the OE-Core team to try and determine if patches are appropriate for upstream or not.

I spoke to the author of curl and mentioned the two patches in Yocto against it, both of which are marked as "Upstream-Status: Inappropriate". He said those patches were never submitted to him.

Are we dismissing patches without even giving upstream a chance to comment?

In some cases yes we might be doing that, particularly patches that seemed to be specific to the OE cross compilation environment or to deal with packaging with in the embedded space where marked as Inappropriate. Once we get though round one of the obvious potential upstream-able patches we can revisit other.

If the author of curl would like to review and/or implement modification for OE that would be awesome, feel free to share the patches with them.

Sau!

On Wed, Feb 8, 2012 at 2:07 AM, Björn Stenberg <bjst@...> wrote:

Who sets the Upstream-Status? Are there guidelines how to do it?

patch author importer whoever brings this patch in into oe. Sometimes
there might be judgement error on patches
thats why I said "for most of them it reflects the status of patch
w.r.t. upstream"

I spoke to the author of curl and mentioned the two patches in Yocto against it, both of which are marked as "Upstream-Status: Inappropriate". He said those patches were never submitted to him.

Are we dismissing patches without even giving upstream a chance to comment?

Thats not the intention at all. All patches should go upstream from
OE's POV it would be cool to have 0 patches locally
If someone had better insights into patches and submit more
appropriate analysis of patches thats welcome
all the time.

On Wed, 8 Feb 2012, Saul Wold wrote:

If the author of curl would like to review and/or implement modification for OE that would be awesome, feel free to share the patches with them.

I am the maintainer of curl.

The curl patches Björn mentioned are clearly not written in way intended to be "upstreamable" so they cannot be accepted by the curl project and nobody has tried to.

This said, at least one of the patches fixes a problem that still exists upstream but the yocto patch [*] is made in such a hard-coded way it'd have to be seriously edited to get accepted. The flaw has not even been discussed with or mentioned to the curl project AFAICR...

So, room for improvements!

[*] = http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/recipes-support/curl/curl/noldlibpath.patch

--

/ daniel.haxx.se

On Wed, Feb 8, 2012 at 1:26 PM, Daniel Stenberg <daniel@...> wrote:

On Wed, 8 Feb 2012, Saul Wold wrote:

If the author of curl would like to review and/or implement modification
for OE that would be awesome, feel free to share the patches with them.

I am the maintainer of curl.

The curl patches Björn mentioned are clearly not written in way intended to
be "upstreamable" so they cannot be accepted by the curl project and nobody
has tried to.

This said, at least one of the patches fixes a problem that still exists
upstream but the yocto patch [*] is made in such a hard-coded way it'd have
to be seriously edited to get accepted. The flaw has not even been discussed
with or mentioned to the curl project AFAICR...

So, room for improvements!

We understand the lack of upstreaming hence the process of documenting
the patches with Upstream-Status
was put in place. So step by step we are getting there where we will
be able to interact with upstream projects
on the patches we carry. The first step it to account for them which
we did. Next step
would be to interact with respective upstream projects and propose the
patches or describe
the problems if they are to be fixed differently.

-Khem

On 02/08/2012 01:26 PM, Daniel Stenberg wrote:

On Wed, 8 Feb 2012, Saul Wold wrote:

If the author of curl would like to review and/or implement
modification for OE that would be awesome, feel free to share the
patches with them.

I am the maintainer of curl.

The curl patches Björn mentioned are clearly not written in way intended
to be "upstreamable" so they cannot be accepted by the curl project and
nobody has tried to.

I am sure there are many patches like that in OE, they are written, tested and then forgotten about, our goal here is to not let them get forgotten.

This said, at least one of the patches fixes a problem that still exists
upstream but the yocto patch [*] is made in such a hard-coded way it'd
have to be seriously edited to get accepted. The flaw has not even been
discussed with or mentioned to the curl project AFAICR...

So, room for improvements!

[*] =
http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/recipes-support/curl/curl/noldlibpath.patch

Daniel,

I think Khem has already said that we are taking incremental steps here, as I mentioned in my prior email, we have over 1200 patches lurking around in OE currently, initially we have about 460 as marked as pending.

If you can fix those issues, since we can't address all of them initially or be experts in all upstreams, we would be very grateful to remove 1 or 2 more patches.

Thanks for your understanding.

Sau!

On Wed, 8 Feb 2012, Saul Wold wrote:

If you can fix those issues, since we can't address all of them initially or be experts in all upstreams, we would be very grateful to remove 1 or 2 more patches.

Yes, I started looking into that.

--

/ daniel.haxx.se

Op 9 feb. 2012, om 00:18 heeft Saul Wold het volgende geschreven:

On 02/08/2012 01:26 PM, Daniel Stenberg wrote:

On Wed, 8 Feb 2012, Saul Wold wrote:

If the author of curl would like to review and/or implement
modification for OE that would be awesome, feel free to share the
patches with them.

I am the maintainer of curl.

The curl patches Björn mentioned are clearly not written in way intended
to be "upstreamable" so they cannot be accepted by the curl project and
nobody has tried to.

I am sure there are many patches like that in OE, they are written, tested and then forgotten about, our goal here is to not let them get forgotten.

This said, at least one of the patches fixes a problem that still exists
upstream but the yocto patch [*] is made in such a hard-coded way it'd
have to be seriously edited to get accepted. The flaw has not even been
discussed with or mentioned to the curl project AFAICR...

So, room for improvements!

[*] =
http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/recipes-support/curl/curl/noldlibpath.patch

Daniel,

I think Khem has already said that we are taking incremental steps here, as I mentioned in my prior email, we have over 1200 patches lurking around in OE currently, initially we have about 460 as marked as pending.

I find the 'pending' confusing, is it 'pending submission' or 'pending approval'? I'm marking patches in meta-oe with 'Upstream-status: Unknown' as default instead of 'Pending' to make it a bit clearer. And patches marked 'inappropriate' won't go in, it's 'rejected', 'unknown' or 'needs work' in those cases. I'm not going to guess what upstream might think of it, since I can't speak for them.

All patches in OE-core now have an Upstream-status, but how many have an *incorrect* Upstream-status? I suspect only a small percentage, but I think it's worth rethinking the lingo used to be clearer to non-native speakers like me.

regards,

Koen

On Thursday 09 February 2012 13:22:10 Koen Kooi wrote:

I find the 'pending' confusing, is it 'pending submission' or 'pending
approval'? I'm marking patches in meta-oe with 'Upstream-status: Unknown'
as default instead of 'Pending' to make it a bit clearer. And patches
marked 'inappropriate' won't go in, it's 'rejected', 'unknown' or 'needs
work' in those cases. I'm not going to guess what upstream might think of
it, since I can't speak for them.

I think the distinction between Pending and Unknown is important. The status
is not completely unknown - the person who set it made an assessment that the
patch should be appropriate for sending upstream, even if it would need
further cleanup beforehand. Maybe "Pending" isn't the best word, I'm not sure,
but "Unknown" is not right either.

All patches in OE-core now have an Upstream-status, but how many have an
*incorrect* Upstream-status?

The status ought to be correct with regard to the patch author's assessment of
whether or not the patch can go upstream. That's what matters - it's a tool
you can use in the separate exercise of going through the patches we do have
and trying to get them merged upstream.

Cheers,
Paul

--

Paul Eggleton
Intel Open Source Technology Centre

On Thu, Feb 9, 2012 at 4:22 AM, Koen Kooi <koen@...> wrote:

I find the 'pending' confusing, is it 'pending submission' or 'pending approval'? I'm marking patches in meta-oe with 'Upstream-status: Unknown' as default instead of 'Pending' to make it a bit clearer. And patches marked 'inappropriate' won't go in, it's 'rejected', 'unknown' or 'needs work' in those cases. I'm not going to guess what upstream might think of it, since I can't speak for them.

there I have seen 'Pending' and 'Submitted' and 'Accepted' which may
be not best wording but explains the status of where the patch is.
Here I think Pending means Pending submission. Umknown would be too
vague.

All patches in OE-core now have an Upstream-status, but how many have an *incorrect* Upstream-status? I suspect only a small percentage, but I think it's worth rethinking the lingo used to be clearer to non-native speakers like me.

Yes.

Op 9 feb. 2012, om 13:30 heeft Paul Eggleton het volgende geschreven:

On Thursday 09 February 2012 13:22:10 Koen Kooi wrote:

I find the 'pending' confusing, is it 'pending submission' or 'pending
approval'? I'm marking patches in meta-oe with 'Upstream-status: Unknown'
as default instead of 'Pending' to make it a bit clearer. And patches
marked 'inappropriate' won't go in, it's 'rejected', 'unknown' or 'needs
work' in those cases. I'm not going to guess what upstream might think of
it, since I can't speak for them.

I think the distinction between Pending and Unknown is important. The status
is not completely unknown - the person who set it made an assessment that the
patch should be appropriate for sending upstream, even if it would need
further cleanup beforehand. Maybe "Pending" isn't the best word, I'm not sure,
but "Unknown" is not right either.

All patches in OE-core now have an Upstream-status, but how many have an
*incorrect* Upstream-status?

The status ought to be correct with regard to the patch author's assessment of
whether or not the patch can go upstream.

That's where I disagree, it's called 'Upstream-status', not 'Perceived-upstream-status'. The field should reflect the status from an upstream perspective, not from the OE perspective. So both 'Pending' and 'Inappropriate' boil down to 'Not submitted' currently. Maybe I'm overthinking all this :)

regards,

Koen

On Thursday 09 February 2012 15:51:11 Koen Kooi wrote:

The status ought to be correct with regard to the patch author's
assessment of whether or not the patch can go upstream.

That's where I disagree, it's called 'Upstream-status', not
'Perceived-upstream-status'. The field should reflect the status from an
upstream perspective, not from the OE perspective. So both 'Pending' and
'Inappropriate' boil down to 'Not submitted' currently. Maybe I'm
overthinking all this :)

Well unless I'm mistaken, the purpose of the field for which it was originally
introduced is as I stated it, to track where we (layer maintainers) are in
sending things upstream since the expectation is that we will be the ones
doing the work required to do that. Whether or not the label(s) that get used
accurately communicate that is another matter.

Cheers,
Paul

--

Paul Eggleton
Intel Open Source Technology Centre

On 02/09/2012 07:31 AM, Paul Eggleton wrote:

On Thursday 09 February 2012 15:51:11 Koen Kooi wrote:

The status ought to be correct with regard to the patch author's
assessment of whether or not the patch can go upstream.

That's where I disagree, it's called 'Upstream-status', not
'Perceived-upstream-status'. The field should reflect the status from an
upstream perspective, not from the OE perspective. So both 'Pending' and
'Inappropriate' boil down to 'Not submitted' currently. Maybe I'm
overthinking all this :)

Well unless I'm mistaken, the purpose of the field for which it was originally
introduced is as I stated it, to track where we (layer maintainers) are in
sending things upstream since the expectation is that we will be the ones
doing the work required to do that. Whether or not the label(s) that get used
accurately communicate that is another matter.

Paul is correct here, a number of people made various proposals for what to put into this field from the perspective of the maintainers. This was then documented by Mark Hatle and reviewed in the TSC at somepoint. It is posted at:

http://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines

There is never a good set of words because everyone can translate then differently. I think everyone is doing there best. For the existing set of Pending, we are working to get those upstream, they would then be marked Submitted, after that we can get more accurate response from the upstream and mark the patch as such. I think that the Submitted step is getting missed and we go from Pending -> updated upstream status.

Once we get through the "Pending" batch we can revisit the remaining 800 or so patches.

We are working on it, every little step makes things better.

Sau!

Cheers,
Paul