Date   

Re: who are using archlinux?

niqingliang
 

thanks your reply, indeed, this is not the first email thread about the
gconf compile problem. it in there only because of no echo.

I have installed python2, and made a symlink in the build directory, and
exported to the env var PATH.

this is the script I used to build (located in the build directory):
#!/bin/bash
loc_dir="`pwd`"
. ${loc_dir}/../optimus/oe-init-build-env ${loc_dir}
export PATH="${loc_dir}:$PATH"
bitbake $@

OS: x86-64 arch / desktop env: gnome 3.2
local.conf ( modified section ):
MACHINE ??= "qemux86-64"
DISTRO ?= "poky-lsb"
SOURCE_MIRROR_URL = "http://10.10.20.149/optimussources/"
INHERIT += "own-mirrors"
BB_GENERATE_MIRROR_TARBALLS = "0"
INHERIT += "rm_work"
CCACHE = ""

when build gconf, got error:
/usr/lib/../lib/libXrandr.so.2: undefined reference to
`memcpy@...'
when 'do_compile' at sanity-check step.

the log of do_compile is attached, here is the last cmd (only reserve
rpath):
../x86_64-poky-linux-libtool ... -Wl,-rpath -Wl,/usr/lib/../lib

the libXrandr.so.2 (depended by the libgtk/libgdk) not appear in the
command explicitly.

I have add --verbose in the command, and found the ld find it in the
host's rootfs, but it should search it in the sysroot arg.

If remove the rpath arg or with the rpath arg prefixed by sysroot arg, I
can build success. (wich will search the libXrandr in the sysroot arg
correctly)

so I think the error is focused on the `ld' which should search in the
sysroot arg at first for implicit dependended libs, but not host's
rootfs.

On Tue, 2012-03-27 at 16:49 +0800, Jack Mitchell wrote:
On 27/03/12 03:33, Ni Qingliang wrote:
who are using archlinux? I want to talk about gconf compile fail
problem.

I use Archlinux for OE/Yocto, could you post your error so we can get an
idea of the problem you are having?

One note for Archlinux is that it ships Python 3 as default, to you must
install Python2 from the repositories then symlink python2 to python in
/usb/bin.

Regards,
--
Yi Qingliang
niqingliang@...
https://niqingliang2003.wordpress.com


Re: Support of meta-kde for Poky

Paul Eggleton
 

On Tuesday 27 March 2012 11:07:55 Samuel Stirtzel wrote:
2012/3/26 Paul Eggleton <paul.eggleton@...>:
On Monday 26 March 2012 14:55:40 Samuel Stirtzel wrote:
meta-kde has no mailing list, so I'm looking for a list to host the
discussion, any suggestions?
Probably just the oe-devel list - if the volume of commits gets too high
then we can look at splitting off a separate one.
Do I need an official statement, and from whom can I get it in case?
I don't think any is needed (at least I don't think anyone asked for any of
the other layers).

My assumption is that the initial request, that started this
discussion, was made with the idea in mind not to include any
"overhead" from meta-oe.

Too bad it isn't possible to include only a subset of recipes from a layer..
But it just looks like layers don't work like that.
That's right, the layer is supposed to be used as a unit; which means those
units are supposed to be kept reasonably small and focused where possible.
meta-oe is the exception here, but we should continue to try to improve it.

As an aside it would be interesting to know why KDE requires additional
gif support from giflib when gif loading is already built into Qt (and we
do enable it in our Qt recipes).
The KDE gifloader code is from 2004.
It looks like this was a possible solution and it worked, so they kept it.
Fair enough.

But since it is KHTML depending on giflib, KDE could replace it with
Webkit in the "future" (this is really old news).
Right, I won't be holding my breath on that one ;)

Cheers,
Paul

--

Paul Eggleton
Intel Open Source Technology Centre


Re: Support of meta-kde for Poky

Samuel Stirtzel <s.stirtzel@...>
 

2012/3/26 Paul Eggleton <paul.eggleton@...>:
On Monday 26 March 2012 14:55:40 Samuel Stirtzel wrote:
the development of meta-kde has come to a point where kde-applications
can be tested (and maybe used).

Status of meta-kde:
Most common applications work, for example:
Calligra (not all of the toolkit), Kate, KCalc, Konqueror, Kwin &
Kwin_gles (KDE window manager), Okular and the applications from
KDE-baseapps.

Plasma Active (and Desktop) do not work correctly, but I'm working on it.
Plasma Netbook seems to work, but it needs more testing.


Involving the community:
As the development of meta-kde advanced to a point where developers
can use the layer to port KDE applications.
I'd like to invite others to test the layer, port their favorite KDE
applications and express their opinions about the current status.
Feel free to ask questions and suggest applications to be ported.
Great work! I will give it a try over the next few days and let you know how I
get on. (Sorry I know I promised to do this some weeks ago.)

meta-kde has no mailing list, so I'm looking for a list to host the
discussion, any suggestions?
Probably just the oe-devel list - if the volume of commits gets too high then
we can look at splitting off a separate one.
Do I need an official statement, and from whom can I get it in case?


Current issues with Poky:
The giflib recipe is not contained in Poky.
Is this based on a design decision, or just because no one needs it?
It's based upon nothing in OE-Core needing it. (The poky repository is just
OE-Core + bitbake + meta-yocto + yocto-docs).

As a meta-kde developer I don't want to limit the testing of meta-kde.
It should be OK to use meta-oe with Poky, and therefore meta-kde can depend
upon it. However we are aware that meta-oe overlays a bunch of recipes that
ideally it should not. We're working (albeit slowly) to address these.

An alternative in the meantime would be to add giflib to meta-kde, this will do
no significant harm if it is already available in another layer (and giflib
doesn't look like it has really changed since 2008, so I don't think it will
be much of a maintenance burden).
My assumption is that the initial request, that started this
discussion, was made with the idea in mind not to include any
"overhead" from meta-oe.
But TBH I don't know exactly.
Too bad it isn't possible to include only a subset of recipes from a layer..
But it just looks like layers don't work like that.


As an aside it would be interesting to know why KDE requires additional gif
support from giflib when gif loading is already built into Qt (and we do enable
it in our Qt recipes).
The KDE gifloader code is from 2004.
It looks like this was a possible solution and it worked, so they kept it.
But since it is KHTML depending on giflib, KDE could replace it with
Webkit in the "future" (this is really old news).


Cheers,
Paul

--

Paul Eggleton
Intel Open Source Technology Centre


--
Regards
Samuel


Re: who are using archlinux?

Jack Mitchell <ml@...>
 

On 27/03/12 03:33, Ni Qingliang wrote:
who are using archlinux? I want to talk about gconf compile fail
problem.

I use Archlinux for OE/Yocto, could you post your error so we can get an idea of the problem you are having?

One note for Archlinux is that it ships Python 3 as default, to you must install Python2 from the repositories then symlink python2 to python in /usb/bin.

Regards,

--

Jack Mitchell (jack@...)
Embedded Systems Engineer
http://www.embed.me.uk

--


Re: Deleting layers in Hob

Lu, Lianhao <lianhao.lu@...>
 

-----Original Message-----
From: yocto-bounces@... [mailto:yocto-bounces@...] On Behalf Of Xu, Dongxiao
Sent: Tuesday, March 27, 2012 2:49 PM
To: yocto
Subject: [yocto] Deleting layers in Hob

When using Hob in Yocto Project, I found a issue when deleting layers. I
think I ever raised this problem before.

Let me briefly introduce how layer removal works in Hob. When user
changes a layer, it will following the below steps
1) init the cooker.
2) set new layers to cooker.
3) parse configuration files.
4) get available machines, distros, SDKs, etc.

As we know, if we source oe-init-build-env in Yocto project environment,
we will have DISTRO="poky" set in local.conf by default, where the
"poky" DISTRO comes from the meta-yocto layer. If user deletes
meta-yocto in Hob, and then error will happen when bitbake parsing the
local.conf, since it could not find where the "poky" DISTRO is defined.

Even if we are able to successfully removed the meta-yocto layer by
removing the DISTRO definition in local.conf, system will report another
issue that:

Your configuration is using stamp files including the sstate hash but
your build directory was built with stamp files that do not include
this.
To continue, either rebuild or switch back to the OEBasic signature
handler with BB_SIGNATURE_HANDLER = 'OEBasic'.

This is because BB_SIGNATURE_HANDLER = "OEBasic" is also defined in
meta-yocto layer (poky.conf).
Meta-yocto is using OEBasicHash as default signature handler(ABI_VERSION=8, see ${TMPDIR}/abi_version), while oe-core is still using the OEBasic(ABI_VERSION=7). This means the oe-core can not reuse the stamp files generated by meta-yocto.


So it seems that in certain environment (e.x, Yocto Project), certain
layer (e.x, meta-yocto) should not be removed?

Or any idea on how to solve this problem?

Thanks,
Dongxiao


_______________________________________________
yocto mailing list
yocto@...
https://lists.yoctoproject.org/listinfo/yocto


Deleting layers in Hob

Xu, Dongxiao <dongxiao.xu@...>
 

When using Hob in Yocto Project, I found a issue when deleting layers. I
think I ever raised this problem before.

Let me briefly introduce how layer removal works in Hob. When user
changes a layer, it will following the below steps
1) init the cooker.
2) set new layers to cooker.
3) parse configuration files.
4) get available machines, distros, SDKs, etc.

As we know, if we source oe-init-build-env in Yocto project environment,
we will have DISTRO="poky" set in local.conf by default, where the
"poky" DISTRO comes from the meta-yocto layer. If user deletes
meta-yocto in Hob, and then error will happen when bitbake parsing the
local.conf, since it could not find where the "poky" DISTRO is defined.

Even if we are able to successfully removed the meta-yocto layer by
removing the DISTRO definition in local.conf, system will report another
issue that:

Your configuration is using stamp files including the sstate hash but
your build directory was built with stamp files that do not include
this.
To continue, either rebuild or switch back to the OEBasic signature
handler with BB_SIGNATURE_HANDLER = 'OEBasic'.

This is because BB_SIGNATURE_HANDLER = "OEBasic" is also defined in
meta-yocto layer (poky.conf).


So it seems that in certain environment (e.x, Yocto Project), certain
layer (e.x, meta-yocto) should not be removed?

Or any idea on how to solve this problem?

Thanks,
Dongxiao


Re: pseudo interaction issue

Peter Seebach <peter.seebach@...>
 

On Mon, 26 Mar 2012 22:45:16 +0100
Richard Purdie <richard.purdie@...> wrote:

I'm still a little surprised we don't make any system() calls though.
I just tried putting a os.system("true") call into the "breakit"
class and it doesn't trigger the warnings. Could that be down to the
lack of a popen wrapper?
I think that part could be, but this doesn't explain why adding the
popen() wrapper doesn't fix it. Oh, wait. Yes, it does. I think I am
now happy with this, although I have a loose end or two.

So, here's what I've figured out.

We start bitbake with PSEUDO_PREFIX set. This then gets stored in the
internal stash. Thus, on any call we catch, we should be restoring it.
We then unset it, because it's not part of the whitelisted environment.

Now, what this means is that when we spawn child processes, they should
be getting the environment, but the parent bitbake is running with
PSEUDO_DISABLED. Which, in turn, sets antimagic. So most calls run
through their non-wrapped form.

The problem: I wrote my popen() wrapper wrong. See, I carefully
removed the check for pseudo_disabled from the top of it. But! That
code path is not actually the only way in which pseudo_disabled affects
behavior. That's just an *OPTIMIZATION*. The pseudo_disabled flag
also means that antimagic is set, and I copied that part of the wrapper
in unmodified:

if (antimagic > 0) {
/* call the real syscall */
rc = (*real_popen)(command, mode);
} else {
/* exec*() use this to restore the sig mask */
pseudo_saved_sigmask = saved;
rc = wrap_popen(command, mode);
}

And antimagic is 1, so I call real popen. Which forks. And even
though pseudo isn't in the LD_PRELOAD environment variable, it's still
in the process's address space, but PSEUDO_PREFIX isn't set, and for
some reason, the stashed value is missing. Not sure I can explain that
part yet; maybe we do have a path where we wipe the stashed value.

But the underlying problem is that my popen() wrapper was never
actually doing the setupenv/dropenv, or just a setupenv.

And the other underlying problem is that calling os.popen() directly is
probably something we should discourage, because we really do want to
know, for each subprocess we plan to spawn, whether it is running in
the pseudo environment or not. So if you call os.popen(), you get
"whatever state bitbake is in", which may not be at all what you wanted
or intended.

If I fix the popen() patch, it may actually start working again,
although I'm still not totally sure why the prefix is getting wiped out.

So... I think...

1. We should probably whitelist PSEUDO_PREFIX because life is a heck
of a lot easier if we aren't trying to set it and unset it all the time.
2. I need to fix my popen patch.
3. Once I've fixed that, I can probably do a much better job of
articulating what's happening to pseudo_prefix that's causing us to end
up with a child process where both the internal stash and the
environment variable are unset.

The big thing I was missing was that PSEUDO_DISABLED implies that
everything will always have antimagic >= 1.

-s
--
Listen, get this. Nobody with a good compiler needs to be justified.


Re: who are using archlinux?

James W. <wjn740@...>
 

my sutdent.


On Tue, Mar 27, 2012 at 10:33 AM, Ni Qingliang <niqingliang@...> wrote:
who are using archlinux? I want to talk about gconf compile fail
problem.


--
Yi Qingliang
niqingliang@...
https://niqingliang2003.wordpress.com

_______________________________________________
yocto mailing list
yocto@...
https://lists.yoctoproject.org/listinfo/yocto


who are using archlinux?

niqingliang
 

who are using archlinux? I want to talk about gconf compile fail
problem.


--
Yi Qingliang
niqingliang@...
https://niqingliang2003.wordpress.com


Re: tentative list of vars to be dropped from variable glossary

Rudolf Streif <rudolf.streif@...>
 

Thanks, Richard. Clearer now but not entirely.


>> BBMASK removes recipes from being parsed. It does not remove them from
>> images although that would I guess be an indirect result since you could
>>no longer build them.

Yes, it removes it from being parsed. I used, for test although not sensible, BBMASK = "base-passwd". If I then tried building core-image-minimal it fails with "Nothing provides base-passwd". If one could use BBMASK to customize images as suggested in http://www.yoctoproject.org/docs/latest/dev-manual/dev-manual.html#usingpoky-extend-customimage-localconf then one would expect the package to be removed indirectly from the image. However, that does not happen, the build just fails.
 

>> Its designed so that if you for example didn't care out any *gpe*
>> recipe, you could exclude them entirely and not even have the parse
>> overhead.

That use case makes sense. If I know that I don't include certain packages like *gpe* in the images I am building then I can speed up the build/parsing process by masking the recipes and therefore the regular expression syntax makes perfectly sense.

Thanks,
Rudi


Re: tentative list of vars to be dropped from variable glossary

Richard Purdie
 

On Mon, 2012-03-26 at 12:42 -0700, Rudolf Streif wrote:
Another variable that seems to be dropped from the latest 1.1.1
revision of the Reference Manual is BBMASK. In previous versions of
the manual it was explained that this variable can be used to remove
packages from images. However, that did not work for me and it did not
surprise me either.
BBMASK removes recipes from being parsed. It does not remove them from
images although that would I guess be an indirect result since you could
no longer build them.

To include a package with an image you will have to make it a
dependency of that image. If you then hide the recipe of the package
using BBMASK then you are breaking the dependency and the Bitbake will
complain with "Nothing provides <package>" because the dependency is
still there.

I could not think of any other use case for BBMASK in the way it is
implemented. Are there any?
Its designed so that if you for example didn't care out any *gpe*
recipe, you could exclude them entirely and not even have the parse
overhead.

Its less useful now things are split into layers but nonetheless still
valid syntax.

Cheers,

Richard


Re: can all of the licensing discussion be centralized in an appendix?

Rifenbark, Scott M <scott.m.rifenbark@...>
 

Rudi,

 

Great comments!  Thanks very much.  I will immediately update the revision history tables to show the 1.2 release as WIP.  Not having it there did cause you confusion.  Good suggestion.

 

Regarding your other observations.  If you have the time I would appreciate you looking at the YP Development manual, particularly Chapter 4.  Much of the “how-to” stuff that was in the YP reference manual has been moved to there.  “Adding a Package” section that you noted was now missing is in fact in that area.  Let me give you some background on the strategy of the two manuals….

 

I am trying to eventually turn the YP reference guide into a “reference” guide with minimal “how-to” information.  That is the long-term goal.  The YP Development Manual has been getting some how-to stuff added to it.  Chapter 4 (Common Tasks) tells how to create layers, add a package, customize images, etc.). 

 

Here is the link to the “latest” YP Development Manual:  http://www.yoctoproject.org/docs/latest/dev-manual/dev-manual.html.  With this new context, maybe you can augment or comment on your previous comments J

 

I appreciate your help,

Scott

 

From: rstreif@... [mailto:rstreif@...] On Behalf Of Rudolf Streif
Sent: Monday, March 26, 2012 4:02 PM
To: Rifenbark, Scott M
Cc: yocto@...
Subject: Re: [yocto] can all of the licensing discussion be centralized in an appendix?

 

Scott,

 

Yes, I looked at that one but I now noticed that I referenced it incorrectly. This is the latest version, however, in the revision history it shows "Revision 1.1, October 6, 2011, Released with the Yocto Project Release 1.1".

 

Now, this one http://www.yoctoproject.org/docs/current/poky-ref-manual/poky-ref-manual.html is the current one but its revision history shows "Revision 1.1.1, March 15, 2012, Released with the Yocto Project Release 1.1.1".

 

Ok, now I get how I confused myself successfully. I presume that the first is the newer one although its revision history suggests otherwise? Would you mind updating the revision history with something like "Revision 1.2, WIP, To be released with Yocto Project Release 1.2"?

 

The latest one does not seem to have a section about writing recipes anymore at all. Are you planning on putting it back? While the previous one had a section on writing recipes it did so in a "vacuum". It told you how to write a recipe but not really where to put it and how to include it with an image. The latter it explained in the next section about customizing an image.

 

The info about the licensing is great and dead-nuts on, however, a reader new to the Yocto Project would be missing the context. A good add-on to that paragraph would be that if you omit the md5 parameter in the LIC_FILES_CHKSUM variable Bitbake will actually spit it out for you which is in particular useful if the checksum is calculated over a subset of a license file specified by startline and endline because md5sum won't easily do that for you.

 

 

These are my suggestions:

 

* Remove 3.3 x32 and 3.4 Licenses from section 3. Section 3 currently looks a little bit like a kitchen sink. The first two paragraphs deal with build system components and architecture, x32 then mixes a very specific technology into it for particular targets, and then Licenses tops it off with package licensing details. I would dedicate Section 3 to Yocto Project Build System Architecture only.

 

* Then I would dedicate a section 4 to Build Customization:

** The first subsection could deal with the most trivial customization through local.conf: EXTRA_IMAGE_FEATURES, IMAGE_INSTALL_append and CORE_IMAGE_EXTRA_INSTALL (formerly known as POKY_EXTRA_INSTALL)

** The second subsection should then explain how to create your own layer within the build environment because before adding any recipe you need to have a layer to add it to. Mixing your own recipes in with meta or meta-yocto works but is not sustainable when upgrading to a newer release of the Yocto Project. And it's good practice too to keep your own stuff separate.

** The third subsection should then explain how to extend images writing your own recipes:

*** writing a recipe that includes a base image and then adds more packages:

    require recipes-core/images/core-imabg-base.bb

    IMAGE_INSTALL += "strace"

*** writing an image that inherits from core-image

    IMAGE_INSTALL = "task-core-boot task-core-extended"

    IMAGE_FEATURES += "blabla"

    inherit core-image

** the fourth subsection should then explain how to add your own packages with your own recipes. Now you already have everything in place: a layer to add to and a ways of including your recipes with an image. And here we then also need to explain the licensing stuff because recipes for building packages won't fly without.

** the fifth section should then just reference the BSP manual for BSPs.

 

Finally for my concept, we need to find a home for x32. I would put that under an "Advanced Topics" section which could act as a container for other stuff too such as multi-lib etc.

 

Rudi

 

 

From: yocto-bounces@... [mailto:yocto-bounces@...] On Behalf Of Rudolf Streif
Sent: Monday, March 26, 2012 2:52 PM
To: yocto@...
Subject: Re: [yocto] can all of the licensing discussion be centralized in an appendix?

 

>> there's *way* too much coverage of licensing sprinkled throughout.
>> most people are not going to care about licensing until the time comes
>> to maybe start distributing their BSP.

The ability of collecting licensing information, tracking license changes and providing the license information automatically with the images and packages for deployment is in my opinion a major feature of Yocto albeit underestimated. Most people will only appreciate it once they have to deliver that information with a product for the end user. However, it starts much earlier, with the recipe.

 

How to include licensing tracking information with your recipe needs to be part of the section explaining how to write recipes of the reference manual. In both, the current 1.1 and the upcoming 1.1.1, versions of the manual the license tracking section is a little disjoint from the sections explaining how to add packages/recipes in my opinion. The examples include the license tracking info, of course because otherwise the sanity checker will not allow building the recipe, but they don't explain that you actually need it (unless you use LICENSE = "CLOSED").

 

Since the license tracking and deployment feature follows the rule "garbage in, garbage out" the manual cannot stress enough providing the correct info in particular when writing recipes for upstream projects.

 

Side note: And nice features for a future release are SPDX info (already worked on as far as I know) and providing the license info in ${TMPDIR}/deploy/licenses according to the images in ${TMPDIR}/images so that one knows what licenses are in use for what image. That would be very convenient when building multiple images with different package content.

 

Rudi

 


Re: can all of the licensing discussion be centralized in an appendix?

Rudolf Streif <rudolf.streif@...>
 

Scott,

Yes, I looked at that one but I now noticed that I referenced it incorrectly. This is the latest version, however, in the revision history it shows "Revision 1.1, October 6, 2011, Released with the Yocto Project Release 1.1".

Now, this one http://www.yoctoproject.org/docs/current/poky-ref-manual/poky-ref-manual.html is the current one but its revision history shows "Revision 1.1.1, March 15, 2012, Released with the Yocto Project Release 1.1.1".

Ok, now I get how I confused myself successfully. I presume that the first is the newer one although its revision history suggests otherwise? Would you mind updating the revision history with something like "Revision 1.2, WIP, To be released with Yocto Project Release 1.2"?

The latest one does not seem to have a section about writing recipes anymore at all. Are you planning on putting it back? While the previous one had a section on writing recipes it did so in a "vacuum". It told you how to write a recipe but not really where to put it and how to include it with an image. The latter it explained in the next section about customizing an image.

The info about the licensing is great and dead-nuts on, however, a reader new to the Yocto Project would be missing the context. A good add-on to that paragraph would be that if you omit the md5 parameter in the LIC_FILES_CHKSUM variable Bitbake will actually spit it out for you which is in particular useful if the checksum is calculated over a subset of a license file specified by startline and endline because md5sum won't easily do that for you.


These are my suggestions:

* Remove 3.3 x32 and 3.4 Licenses from section 3. Section 3 currently looks a little bit like a kitchen sink. The first two paragraphs deal with build system components and architecture, x32 then mixes a very specific technology into it for particular targets, and then Licenses tops it off with package licensing details. I would dedicate Section 3 to Yocto Project Build System Architecture only.

* Then I would dedicate a section 4 to Build Customization:
** The first subsection could deal with the most trivial customization through local.conf: EXTRA_IMAGE_FEATURES, IMAGE_INSTALL_append and CORE_IMAGE_EXTRA_INSTALL (formerly known as POKY_EXTRA_INSTALL)
** The second subsection should then explain how to create your own layer within the build environment because before adding any recipe you need to have a layer to add it to. Mixing your own recipes in with meta or meta-yocto works but is not sustainable when upgrading to a newer release of the Yocto Project. And it's good practice too to keep your own stuff separate.
** The third subsection should then explain how to extend images writing your own recipes:
*** writing a recipe that includes a base image and then adds more packages:
    require recipes-core/images/core-imabg-base.bb
    IMAGE_INSTALL += "strace"
*** writing an image that inherits from core-image
    IMAGE_INSTALL = "task-core-boot task-core-extended"
    IMAGE_FEATURES += "blabla"
    inherit core-image
** the fourth subsection should then explain how to add your own packages with your own recipes. Now you already have everything in place: a layer to add to and a ways of including your recipes with an image. And here we then also need to explain the licensing stuff because recipes for building packages won't fly without.
** the fifth section should then just reference the BSP manual for BSPs.

Finally for my concept, we need to find a home for x32. I would put that under an "Advanced Topics" section which could act as a container for other stuff too such as multi-lib etc.

Rudi
 

 

From: yocto-bounces@... [mailto:yocto-bounces@...] On Behalf Of Rudolf Streif
Sent: Monday, March 26, 2012 2:52 PM
To: yocto@...
Subject: Re: [yocto] can all of the licensing discussion be centralized in an appendix?

 

>> there's *way* too much coverage of licensing sprinkled throughout.
>> most people are not going to care about licensing until the time comes
>> to maybe start distributing their BSP.

The ability of collecting licensing information, tracking license changes and providing the license information automatically with the images and packages for deployment is in my opinion a major feature of Yocto albeit underestimated. Most people will only appreciate it once they have to deliver that information with a product for the end user. However, it starts much earlier, with the recipe.

 

How to include licensing tracking information with your recipe needs to be part of the section explaining how to write recipes of the reference manual. In both, the current 1.1 and the upcoming 1.1.1, versions of the manual the license tracking section is a little disjoint from the sections explaining how to add packages/recipes in my opinion. The examples include the license tracking info, of course because otherwise the sanity checker will not allow building the recipe, but they don't explain that you actually need it (unless you use LICENSE = "CLOSED").

 

Since the license tracking and deployment feature follows the rule "garbage in, garbage out" the manual cannot stress enough providing the correct info in particular when writing recipes for upstream projects.

 

Side note: And nice features for a future release are SPDX info (already worked on as far as I know) and providing the license info in ${TMPDIR}/deploy/licenses according to the images in ${TMPDIR}/images so that one knows what licenses are in use for what image. That would be very convenient when building multiple images with different package content.

 

Rudi



Re: pseudo interaction issue

Richard Purdie
 

On Mon, 2012-03-26 at 12:18 -0500, Peter Seebach wrote:
On Mon, 26 Mar 2012 17:47:29 +0100
Richard Purdie <richard.purdie@...> wrote:

This is pretty much what we do at the moment, it gets unset after we
load. Pseudo is of course disabled at this point.

I guess we just got lucky to this point and avoided "Bad Things"?
I suspect so. What's weird to me is that PSEUDO_PREFIX wasn't in the
environment before, either. So I still don't quite get this. I am
still missing something which will make this all make sense.

... at this point, I am leaning towards viewing this as a bug where it
is not enough to simply correct the behavior, I will not feel confident
in it until I have understood how it could have happened, but worked in
many other cases.
This is why I'm not saying lets just set PSEUDO_PREFIX. Its bothering me
too.

What puzzles me is we get this value from envbackup[key] =
os.environ.get("PSEUDO_PREFIX") so its already not in the
environment.

So basically if we read "PSEUDO_PREFIX" from the environment we get
nothing. If we unset the value back to being "nothing", things
break.
Yes. This is, of course, obviously impossible.
Obviously :). Except the code does this and I've watched it happen. I'm
not claiming to understand it...

Hmm. Well, hmm. When we start up, we should pick up PSEUDO_PREFIX
from our environment, and during some of the initial client setup, we
should be stashing that value in our stashed values table. At this
point, so far as I can tell, nothing should ever unset that stashed
value.

On fork(), we don't change anything until we're in the client side of
the fork, but that setup should happen in the same address space, with
the values still stashed.
When we poke new values into the environment, will it corrupt the
internal stash?

Oh, nevermind, I just realized: We use antimagic as the
implementation
goo for PSEUDO_DISABLED.

So a call to os.popen() from a program which has PSEUDO_DISABLED set
is going to think it's in antimagic mode.

And suddenly, the trick is revealed:

os.popen() is bypassing all the runqueue stuff which is trying to
ensure that the environment is in a valid state. So if bitbake code
calls os.popen(), it may behave weirdly, for the same reason that any
other direct invocation of fork() or system() or whatnot would behave
weirdly -- because bitbake is running with pseudo in a strange state.
I'd be very surprised if we don't make some other system() call
somewhere in bitbake's parent context.

If this were a trigger, it could go a long way to explaining some errors
people have reported though.

So I think the thing is:

Because bitbake is running with PSEUDO_DISABLED, any child process
that
is not explicitly set to either enable or unload pseudo is going to be
running under pseudo, with PSEUDO_DISABLED. And that means we need to
ensure that PSEUDO_PREFIX stays set, because we need that even when
pseudo is disabled. (It's used during some of the initial setup
sanity checks.)
This is the answer I was expecting we'd come to.

I'm still a little surprised we don't make any system() calls though. I
just tried putting a os.system("true") call into the "breakit" class and
it doesn't trigger the warnings. Could that be down to the lack of a
popen wrapper?

Cheers,

Richard


[PATCH 2/2] sato: Remove questioned ISO image

Yang Shi <yang.shi@...>
 

[YOCTO #1487]

For the liveCD image, interactive bootup is needed, but psplash prevents from
booting interactively. In such case ISO image is not usable, so remove ISO image
and the corresponding link and throw error info to warn outside to enable
unionfs in kenrel.

Signed-off-by: Yang Shi <yang.shi@...>
---
meta/recipes-sato/images/core-image-sato.bb | 16 ++++++++++++++++
1 files changed, 16 insertions(+), 0 deletions(-)

diff --git a/meta/recipes-sato/images/core-image-sato.bb b/meta/recipes-sato/images/core-image-sato.bb
index 11c3318..10e2239 100644
--- a/meta/recipes-sato/images/core-image-sato.bb
+++ b/meta/recipes-sato/images/core-image-sato.bb
@@ -9,3 +9,19 @@ IMAGE_FEATURES += "apps-console-core ${SATO_IMAGE_FEATURES}"
LICENSE = "MIT"

inherit core-image
+
+LIVE = "${@base_contains('IMAGE_FSTYPES', 'live', 'yes', 'no', d)}"
+
+do_check_unionfs() {
+ if [ "${NOISO}" = "1" ]; then
+ return
+ fi
+
+ if [ "${LIVE}" = "yes" ] && ! grep -q "CONFIG_UNION_FS=y" ${STAGING_KERNEL_DIR}/.config; then
+ rm -f ${DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME}.iso
+ rm -f ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.iso
+ bbfatal "Building LIVE CD without UNION FS enabled in kernel"
+ fi
+}
+
+addtask check_unionfs before do_build after do_bootimg
--
1.7.5.4


[PATCH 1/2] initrdscripts: fix init-live.sh and use unionfs

Yang Shi <yang.shi@...>
 

[YOCTO #1487]

Use unionfs to mount rootfs and make root file system can be writen when using liveCD to boot up.
Set UNION_FS variable depending on kenrel config, so that it can work with kernel which doesn't
have unionfs feature.

Signed-off-by: Yang Shi <yang.shi@...>
---
meta/recipes-core/initrdscripts/files/init-live.sh | 21 +++++++++++++++++--
.../initrdscripts/initramfs-live-boot_1.0.bb | 9 +++++++-
2 files changed, 26 insertions(+), 4 deletions(-)

diff --git a/meta/recipes-core/initrdscripts/files/init-live.sh b/meta/recipes-core/initrdscripts/files/init-live.sh
index eb5ab5b..abaf16c 100644
--- a/meta/recipes-core/initrdscripts/files/init-live.sh
+++ b/meta/recipes-core/initrdscripts/files/init-live.sh
@@ -7,6 +7,7 @@ ROOT_IMAGE="rootfs.img"
MOUNT="/bin/mount"
UMOUNT="/bin/umount"
ISOLINUX=""
+UNIONFS="no"

early_setup() {
mkdir /proc
@@ -89,10 +90,24 @@ case $label in
mkdir $ROOT_MOUNT
mknod /dev/loop0 b 7 0 2>/dev/null

- if ! $MOUNT -o rw,loop,noatime,nodiratime /media/$i/$ISOLINUX/$ROOT_IMAGE $ROOT_MOUNT ; then
- fatal "Couldnt mount rootfs image"
+
+ if [ "$UNIONFS" = "yes" ]; then
+ mkdir /rootfs-tmp
+
+ if ! $MOUNT -o rw,loop,noatime,nodiratime /media/$i/$ISOLINUX/$ROOT_IMAGE /rootfs-tmp ; then
+ fatal "Couldnt mount rootfs image"
+ else
+ mkdir /cow
+ mount -t tmpfs -o rw,noatime,mode=755 tmpfs /cow
+ mount -t unionfs -o dirs=/cow:/rootfs-tmp=ro unionfs $ROOT_MOUNT
+ boot_live_root
+ fi
else
- boot_live_root
+ if ! $MOUNT -o rw,loop,noatime,nodiratime /media/$i/$ISOLINUX/$ROOT_IMAGE $ROOT_MOUNT ; then
+ fatal "Couldnt mount rootfs image"
+ else
+ boot_live_root
+ fi
fi
;;
install)
diff --git a/meta/recipes-core/initrdscripts/initramfs-live-boot_1.0.bb b/meta/recipes-core/initrdscripts/initramfs-live-boot_1.0.bb
index e85a0e1..f7f0c9d 100644
--- a/meta/recipes-core/initrdscripts/initramfs-live-boot_1.0.bb
+++ b/meta/recipes-core/initrdscripts/initramfs-live-boot_1.0.bb
@@ -2,10 +2,17 @@ DESCRIPTION = "A live image init script"
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
RDEPENDS = "udev"
+DEPENDS = "virtual/kernel"
SRC_URI = "file://init-live.sh"

-PR = "r7"
+PR = "r8"

+do_compile() {
+ if grep -q "CONFIG_UNION_FS=y" ${STAGING_KERNEL_DIR}/.config; then
+ sed -i 's/UNIONFS="no"/UNIONFS="yes"/g' ${WORKDIR}/init-live.sh
+ fi
+}
+
do_install() {
install -m 0755 ${WORKDIR}/init-live.sh ${D}/init
}
--
1.7.5.4


[PATCH 0/2] Fix init-live.sh to use unionfs for live CD and remove questioned ISO image

Yang Shi <yang.shi@...>
 

Use unionfs to mount rootfs and make root file system can be writen when using liveCD to boot up.
Set UNION_FS variable depending on kenrel config, so that it can work with kernel which doesn't
have unionfs feature.

Without UNION FS enabled in kernel, minimal liveCD image can be bootup to ignore the failure of mouting
read-only filesystem. For the sato liveCD image, interactive bootup is needed, but psplash prevents from
booting interactively. In such case ISO image is not usable, so remove ISO image and the corresponding
link and throw error info to warn outside to enable unionfs in kenrel.

We tried a lot of different ways to achieve this:

1. Check if ISO image is built in initramfs-live-boot.bb or not, then prevent from generating ISO image.
But, we can't distinguish minimal image or sato image.

2. Check if x11 feature is included in initramfs-live-boot.bb or not, then prevent from generating ISO image.
But, x11 feature is contained in minimal image as well even though the package is not built.

3. Tried to append some check code in build_iso() in core-image-sato.bb, but poky can't support append code
in build_xxx functions like what we do for do_xxx funcitons.

So, we came up with current solution. Any new suggestion is appreciated.

One more thing is that if IMAGE_FSTYPE += "live" is not set in conf/local.conf with this fix, we will get below
error when building sato image:

ERROR: Running idle function
Traceback (most recent call last):
File "/home/yshi/yocto/poky/bitbake/lib/bb/server/process.py", line
122, in ProcessServer.idle_commands(delay=0.1):
try:
> retval = function(self, data, False)
if retval is False:
File "/home/yshi/yocto/poky/bitbake/lib/bb/cooker.py", line 1130, in
buildTargetsIdle(server=<ProcessServer(ProcessServer-1, started)>,
rq=<bb.runqueue.RunQueue instance at 0x74e4638>, abort=False):
try:
> retval = rq.execute_runqueue()
except runqueue.TaskFailure as exc:
File "/home/yshi/yocto/poky/bitbake/lib/bb/runqueue.py", line 947,
in RunQueue.execute_runqueue():
self.rqexe = RunQueueExecuteDummy(self)
> if self.rqdata.prepare() == 0:
self.state = runQueueComplete
File "/home/yshi/yocto/poky/bitbake/lib/bb/runqueue.py", line 719,
in RunQueueData.prepare():

procdep.append(self.taskData.fn_index[self.runq_fnid[dep]] + "." +
self.runq_task[dep])
> self.runq_hash[task] =
bb.parse.siggen.get_taskhash(self.taskData.fn_index[self.runq_fnid[task]],
self.runq_task[task], procdep, self.dataCache)

File "/home/yshi/yocto/poky/bitbake/lib/bb/siggen.py", line 153, in
SignatureGeneratorOEBasicHash.get_taskhash(fn='/home/yshi/yocto/poky/meta/recipes-sato/images/core-image-sato.bb',
task='do_bootimg', deps=[], dataCache=<bb.cache.CacheData object at
0x2148490>):
k = fn + "." + task
> data = dataCache.basetaskhash[k]
self.runtaskdeps[k] = []
KeyError: '/home/yshi/yocto/poky/meta/recipes-sato/images/core-image-sato.bb.do_bootimg'

NOTE: Preparing runqueue

Bruce pointed out there was a large discussion about this on hte list last week, so this should be a known issue in poky.

The following changes since commit f81b0593e74a31cb2d992df0583948ff57e3ed98:

gdbm: Activate -enable-libgdbm-compat and add symlinks to headers in include/gdbm (2012-03-23 17:56:29 +0200)

are available in the git repository at:
git://git.yoctoproject.org/poky-contrib yshi/1487
http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=yshi/1487

Yang Shi (2):
initrdscripts: fix init-live.sh and use unionfs
sato: Remove questioned ISO image

meta/recipes-core/initrdscripts/files/init-live.sh | 21 +++++++++++++++++--
.../initrdscripts/initramfs-live-boot_1.0.bb | 9 +++++++-
meta/recipes-sato/images/core-image-sato.bb | 16 +++++++++++++++
3 files changed, 42 insertions(+), 4 deletions(-)


Re: can all of the licensing discussion be centralized in an appendix?

Rifenbark, Scott M <scott.m.rifenbark@...>
 

Rudi,

 

I would like you to examine the “latest” documents (the ones under development).  I have added some new licensing information and would like your feedback for this upcoming version.  There are licensing discussions in both the latest versions of the BSP Guide and the reference manual.  Since, at the moment, the BSP Guide is included as a chapter in the reference manual, I’ll point you to the latest version of the reference manual:

 

http://www.yoctoproject.org/docs/latest/poky-ref-manual/poky-ref-manual.html.

 

 

Thanks,

Scott

 

From: yocto-bounces@... [mailto:yocto-bounces@...] On Behalf Of Rudolf Streif
Sent: Monday, March 26, 2012 2:52 PM
To: yocto@...
Subject: Re: [yocto] can all of the licensing discussion be centralized in an appendix?

 

>> there's *way* too much coverage of licensing sprinkled throughout.
>> most people are not going to care about licensing until the time comes
>> to maybe start distributing their BSP.

The ability of collecting licensing information, tracking license changes and providing the license information automatically with the images and packages for deployment is in my opinion a major feature of Yocto albeit underestimated. Most people will only appreciate it once they have to deliver that information with a product for the end user. However, it starts much earlier, with the recipe.

 

How to include licensing tracking information with your recipe needs to be part of the section explaining how to write recipes of the reference manual. In both, the current 1.1 and the upcoming 1.1.1, versions of the manual the license tracking section is a little disjoint from the sections explaining how to add packages/recipes in my opinion. The examples include the license tracking info, of course because otherwise the sanity checker will not allow building the recipe, but they don't explain that you actually need it (unless you use LICENSE = "CLOSED").

 

Since the license tracking and deployment feature follows the rule "garbage in, garbage out" the manual cannot stress enough providing the correct info in particular when writing recipes for upstream projects.

 

Side note: And nice features for a future release are SPDX info (already worked on as far as I know) and providing the license info in ${TMPDIR}/deploy/licenses according to the images in ${TMPDIR}/images so that one knows what licenses are in use for what image. That would be very convenient when building multiple images with different package content.

 

Rudi


Re: can all of the licensing discussion be centralized in an appendix?

Rudolf Streif <rudolf.streif@...>
 

>> there's *way* too much coverage of licensing sprinkled throughout.
>> most people are not going to care about licensing until the time comes
>> to maybe start distributing their BSP.

The ability of collecting licensing information, tracking license changes and providing the license information automatically with the images and packages for deployment is in my opinion a major feature of Yocto albeit underestimated. Most people will only appreciate it once they have to deliver that information with a product for the end user. However, it starts much earlier, with the recipe.

How to include licensing tracking information with your recipe needs to be part of the section explaining how to write recipes of the reference manual. In both, the current 1.1 and the upcoming 1.1.1, versions of the manual the license tracking section is a little disjoint from the sections explaining how to add packages/recipes in my opinion. The examples include the license tracking info, of course because otherwise the sanity checker will not allow building the recipe, but they don't explain that you actually need it (unless you use LICENSE = "CLOSED").

Since the license tracking and deployment feature follows the rule "garbage in, garbage out" the manual cannot stress enough providing the correct info in particular when writing recipes for upstream projects.

Side note: And nice features for a future release are SPDX info (already worked on as far as I know) and providing the license info in ${TMPDIR}/deploy/licenses according to the images in ${TMPDIR}/images so that one knows what licenses are in use for what image. That would be very convenient when building multiple images with different package content.

Rudi


Re: pseudo interaction issue

Peter Seebach <peter.seebach@...>
 

Oh, nevermind, I just realized: We use antimagic as the implementation
goo for PSEUDO_DISABLED.

So a call to os.popen() from a program which has PSEUDO_DISABLED set is
going to think it's in antimagic mode.

And suddenly, the trick is revealed:

os.popen() is bypassing all the runqueue stuff which is trying to
ensure that the environment is in a valid state. So if bitbake code
calls os.popen(), it may behave weirdly, for the same reason that any
other direct invocation of fork() or system() or whatnot would behave
weirdly -- because bitbake is running with pseudo in a strange state.

So I think the thing is:

Because bitbake is running with PSEUDO_DISABLED, any child process that
is not explicitly set to either enable or unload pseudo is going to be
running under pseudo, with PSEUDO_DISABLED. And that means we need to
ensure that PSEUDO_PREFIX stays set, because we need that even when
pseudo is disabled. (It's used during some of the initial setup
sanity checks.)

-s
--
Listen, get this. Nobody with a good compiler needs to be justified.