Date   

Re: project that builds target and host

Joel Winarske
 


Putting files in the target sysroot that are in fact native binaries.
Don't do that.

Yeah i don't want to disable stripping the Target files.


> The major surgery would be less maintainable.

Can you not work with upstream to integrate the changes?
Alternatively, if the native parts are relatively simple, you can just
build them directly.

Well actually, I'll take a look at patching the tree with a custom CMakeLists.txt file.  It may not be that bad.

Thanks Ross


Re: project that builds target and host

Ross Burton
 

On Wed, 13 May 2020 at 21:05, Joel Winarske <joel.winarske@gmail.com> wrote:
What are some nasty hack options?
Putting files in the target sysroot that are in fact native binaries.
Don't do that.

The major surgery would be less maintainable.
Can you not work with upstream to integrate the changes?
Alternatively, if the native parts are relatively simple, you can just
build them directly.

Ross


Re: project that builds target and host

Joel Winarske
 

What are some nasty hack options?  

The major surgery would be less maintainable.

On Wed, May 13, 2020, 12:51 PM Ross Burton <ross@...> wrote:
On Wed, 13 May 2020 at 20:28, Joel Winarske <joel.winarske@...> wrote:
> In this case I already have a Target and Native recipe.  The Project generates a Native artifact when building the Target.  Not resolvable without major surgery.  This artifact is then required in other Target recipes.

A target recipe can't drop files into the native sysroot, so it's
either nasty hacks or major surgery.

Ross


Re: project that builds target and host

Ross Burton
 

On Wed, 13 May 2020 at 20:28, Joel Winarske <joel.winarske@gmail.com> wrote:
In this case I already have a Target and Native recipe. The Project generates a Native artifact when building the Target. Not resolvable without major surgery. This artifact is then required in other Target recipes.
A target recipe can't drop files into the native sysroot, so it's
either nasty hacks or major surgery.

Ross


Re: project that builds target and host

Joel Winarske
 

Hi Ross,

Yes I've done what you mention before.  

In this case I already have a Target and Native recipe.  The Project generates a Native artifact when building the Target.  Not resolvable without major surgery.  This artifact is then required in other Target recipes.


Joel

On Wed, May 13, 2020, 7:13 AM Ross Burton <ross@...> wrote:
On Wed, 13 May 2020 at 06:03, Joel Winarske <joel.winarske@...> wrote:
> I have a project that generates a native artifact during the target build.  Both the native and target artifacts are needed for other target recipes.
>
> What's the recommended pattern for handling this?

Either use BBCLASSEXTEND or write a -native recipe, build just the
native parts in a foo-native recipe and depend on that to build foo
and other recipes.

For example, glib-2.0 DEPENDS on glib-2.0-native.

Ross


Re: project that builds target and host

Ross Burton
 

On Wed, 13 May 2020 at 06:03, Joel Winarske <joel.winarske@gmail.com> wrote:
I have a project that generates a native artifact during the target build. Both the native and target artifacts are needed for other target recipes.

What's the recommended pattern for handling this?
Either use BBCLASSEXTEND or write a -native recipe, build just the
native parts in a foo-native recipe and depend on that to build foo
and other recipes.

For example, glib-2.0 DEPENDS on glib-2.0-native.

Ross


Re: what is the proper treatment of the "Unlicense" license?

Quentin Schulz
 

On Wed, May 13, 2020 at 09:53:12AM -0400, Robert P. J. Day wrote:
[...]

it gets weirder ... the project i'm working with is based on morty
so that variable *would* still be relevant, but even adding
"Unlicense" to that variable didn't stop the offending recipe
from still generating a warning. so i thought, "i wonder if there are
any other recipes in the layers i'm working with that have
'Unlicense," and sure enough, there's one: pyelftools (created
in-house).

so i added pyelftools to the image i'm building, but *that* recipe
*didn't* generate a warning, so now i'm thoroughly baffled. and,
finally, i decided to check the current state of pyelftools to see
what its licensing is, and in meta-python, we have the recipe
python3-pyelftools_0.25.bb, wherein we read:

LICENSE = "PD"

argh ... and if one checks OE/meta/files/common-licenses, there is
indeed a license file named "PD" whose contents are simply:

This is a placeholder for the Public Domain License

so now i'm not sure if a "Unlicense" license file is redundant or
what.
If Unlicense and Public Domain (PD) were actual synonyms we could put
them into the SPDXLICENSEMAP in
openembedded-core/meta/conf/licenses.conf, but according to
https://unlicense.org/ lists there should be a difference between both.

It looks like it's decently used though as GitHub reported that 2% of
the projects they host are Unlicense-licensed[1] which is more than
BSD-2 or [AL]GPLv3!

[1] https://github.blog/2015-03-09-open-source-license-usage-on-github-com/

i'm confused.
Since we have [AL]GPLv3 "support", I'd say Unlicense has its place as
well.

Quentin


Re: what is the proper treatment of the "Unlicense" license?

Robert P. J. Day
 

On Wed, 13 May 2020, Quentin Schulz wrote:

On Wed, May 13, 2020 at 08:40:28AM -0400, Robert P. J. Day wrote:
On Wed, 13 May 2020, Quentin Schulz wrote:

Hi Robert,

On Wed, May 13, 2020 at 07:19:59AM -0400, Robert P. J. Day wrote:
On Wed, 13 May 2020, Richard Leitner wrote:

On Wed, May 13, 2020 at 12:39:44PM +0200, Quentin Schulz wrote:
On Wed, May 13, 2020 at 06:25:01AM -0400, Robert P. J. Day wrote:
...

If it's really widely used, maybe something to add to
openembedded-core/files/common-licenses/ ? So that you don't need any of
the suggested ways?
+1 for adding Unlicense to openembedded-core's common-licenses
as long as this requires only adding an Unlicense file to that
directory, i can do that shortly.
I'm not sure, but there might be a need to add it to
SRC_DISTRIBUTE_LICENSES in
openembedded-core/meta/conf/licenses.conf.
that variable is gone:

commit 64daaf29e2c12c8b587bafdebf9409433187ddf7
Author: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Date: Wed Dec 11 17:48:14 2019 +0100

licenses.conf: Remove the SRC_DISTRIBUTE_LICENSES variable

The SRC_DISTRIBUTE_LICENSES variable and its static list of licenses
has been replaced by AVAILABLE_LICENSES, which automatically contains
all available licenses.

That'll teach me to check in master instead of my release of Yocto :)
it gets weirder ... the project i'm working with is based on morty
so that variable *would* still be relevant, but even adding
"Unlicense" to that variable didn't stop the offending recipe
from still generating a warning. so i thought, "i wonder if there are
any other recipes in the layers i'm working with that have
'Unlicense," and sure enough, there's one: pyelftools (created
in-house).

so i added pyelftools to the image i'm building, but *that* recipe
*didn't* generate a warning, so now i'm thoroughly baffled. and,
finally, i decided to check the current state of pyelftools to see
what its licensing is, and in meta-python, we have the recipe
python3-pyelftools_0.25.bb, wherein we read:

LICENSE = "PD"

argh ... and if one checks OE/meta/files/common-licenses, there is
indeed a license file named "PD" whose contents are simply:

This is a placeholder for the Public Domain License

so now i'm not sure if a "Unlicense" license file is redundant or
what.

i'm confused.

rday


Re: what is the proper treatment of the "Unlicense" license?

Quentin Schulz
 

On Wed, May 13, 2020 at 08:40:28AM -0400, Robert P. J. Day wrote:
On Wed, 13 May 2020, Quentin Schulz wrote:

Hi Robert,

On Wed, May 13, 2020 at 07:19:59AM -0400, Robert P. J. Day wrote:
On Wed, 13 May 2020, Richard Leitner wrote:

On Wed, May 13, 2020 at 12:39:44PM +0200, Quentin Schulz wrote:
On Wed, May 13, 2020 at 06:25:01AM -0400, Robert P. J. Day wrote:
...

If it's really widely used, maybe something to add to
openembedded-core/files/common-licenses/ ? So that you don't need any of
the suggested ways?
+1 for adding Unlicense to openembedded-core's common-licenses
as long as this requires only adding an Unlicense file to that
directory, i can do that shortly.
I'm not sure, but there might be a need to add it to
SRC_DISTRIBUTE_LICENSES in
openembedded-core/meta/conf/licenses.conf.
that variable is gone:

commit 64daaf29e2c12c8b587bafdebf9409433187ddf7
Author: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Date: Wed Dec 11 17:48:14 2019 +0100

licenses.conf: Remove the SRC_DISTRIBUTE_LICENSES variable

The SRC_DISTRIBUTE_LICENSES variable and its static list of licenses
has been replaced by AVAILABLE_LICENSES, which automatically contains
all available licenses.

That'll teach me to check in master instead of my release of Yocto :)

Quentin


Re: what is the proper treatment of the "Unlicense" license?

Robert P. J. Day
 

On Wed, 13 May 2020, Quentin Schulz wrote:

Hi Robert,

On Wed, May 13, 2020 at 07:19:59AM -0400, Robert P. J. Day wrote:
On Wed, 13 May 2020, Richard Leitner wrote:

On Wed, May 13, 2020 at 12:39:44PM +0200, Quentin Schulz wrote:
On Wed, May 13, 2020 at 06:25:01AM -0400, Robert P. J. Day wrote:
...

If it's really widely used, maybe something to add to
openembedded-core/files/common-licenses/ ? So that you don't need any of
the suggested ways?
+1 for adding Unlicense to openembedded-core's common-licenses
as long as this requires only adding an Unlicense file to that
directory, i can do that shortly.
I'm not sure, but there might be a need to add it to
SRC_DISTRIBUTE_LICENSES in
openembedded-core/meta/conf/licenses.conf.
that variable is gone:

commit 64daaf29e2c12c8b587bafdebf9409433187ddf7
Author: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Date: Wed Dec 11 17:48:14 2019 +0100

licenses.conf: Remove the SRC_DISTRIBUTE_LICENSES variable

The SRC_DISTRIBUTE_LICENSES variable and its static list of licenses
has been replaced by AVAILABLE_LICENSES, which automatically contains
all available licenses.



rday


Re: what is the proper treatment of the "Unlicense" license?

Quentin Schulz
 

Hi Robert,

On Wed, May 13, 2020 at 07:19:59AM -0400, Robert P. J. Day wrote:
On Wed, 13 May 2020, Richard Leitner wrote:

On Wed, May 13, 2020 at 12:39:44PM +0200, Quentin Schulz wrote:
On Wed, May 13, 2020 at 06:25:01AM -0400, Robert P. J. Day wrote:
...

If it's really widely used, maybe something to add to
openembedded-core/files/common-licenses/ ? So that you don't need any of
the suggested ways?
+1 for adding Unlicense to openembedded-core's common-licenses
as long as this requires only adding an Unlicense file to that
directory, i can do that shortly.
I'm not sure, but there might be a need to add it to SRC_DISTRIBUTE_LICENSES
in openembedded-core/meta/conf/licenses.conf.

Quentin


Re: what is the proper treatment of the "Unlicense" license?

Robert P. J. Day
 

On Wed, 13 May 2020, Richard Leitner wrote:

On Wed, May 13, 2020 at 12:39:44PM +0200, Quentin Schulz wrote:
On Wed, May 13, 2020 at 06:25:01AM -0400, Robert P. J. Day wrote:
...

If it's really widely used, maybe something to add to
openembedded-core/files/common-licenses/ ? So that you don't need any of
the suggested ways?
+1 for adding Unlicense to openembedded-core's common-licenses
as long as this requires only adding an Unlicense file to that
directory, i can do that shortly.

rday


Re: what is the proper treatment of the "Unlicense" license?

Robert P. J. Day
 

On Wed, 13 May 2020, Quentin Schulz wrote:

... snip ...

If it's really widely used, maybe something to add to
openembedded-core/files/common-licenses/ ? So that you don't need
any of the suggested ways?
that was one of the first things that came to mind ... if there are
enough packages with this license, why not just make it official? i
just did a quick search, and this is the only layer i see that makes
reference to it:

busybox/networking/tls_aes.c

/*
* Copyright (C) 2017 Denys Vlasenko
*
* Licensed under GPLv2, see file LICENSE in this source tree.
*/

/* This AES implementation is derived from tiny-AES128-C code,
* which was put by its author into public domain:
*
* tiny-AES128-C/unlicense.txt, Dec 8, 2014
* """
* This is free and unencumbered software released into the public domain.

more info on that license:

https://choosealicense.com/licenses/unlicense/

open to suggestions.

rday


Re: what is the proper treatment of the "Unlicense" license?

Richard Leitner
 

On Wed, May 13, 2020 at 12:39:44PM +0200, Quentin Schulz wrote:
On Wed, May 13, 2020 at 06:25:01AM -0400, Robert P. J. Day wrote:
...

If it's really widely used, maybe something to add to
openembedded-core/files/common-licenses/ ? So that you don't need any of
the suggested ways?
+1 for adding Unlicense to openembedded-core's common-licenses

regards;rl


Quentin


Re: what is the proper treatment of the "Unlicense" license?

Quentin Schulz
 

Hi Robert,

On Wed, May 13, 2020 at 06:25:01AM -0400, Robert P. J. Day wrote:

colleague added a new recipe to a build, got a warning "The license
listed Unlicense was not in the licenses collected for recipe
python-filelock" and, sure enough, that source was released under the
"Unlicense" which i had never heard of:

https://github.com/benediktschmitt/py-filelock/blob/master/LICENSE

what is the standard strategy for dealing with that? should i just
whitelist that license? has anyone else run into this "Unlicense"
license?
You can add a license to your layer by doing the following in
conf/layer.conf:
LICENSE_PATH += "${LAYERDIR}/licenses"

and in there you put the Unlicense file named exactly the same with the
correct content. That's one way.

The other is to use NO_GENERIC_LICENSE[Unlicense] = "/path/to/Unlicense"
in the recipe.

If it's really widely used, maybe something to add to
openembedded-core/files/common-licenses/ ? So that you don't need any of
the suggested ways?

Quentin


what is the proper treatment of the "Unlicense" license?

Robert P. J. Day
 

colleague added a new recipe to a build, got a warning "The license
listed Unlicense was not in the licenses collected for recipe
python-filelock" and, sure enough, that source was released under the
"Unlicense" which i had never heard of:

https://github.com/benediktschmitt/py-filelock/blob/master/LICENSE

what is the standard strategy for dealing with that? should i just
whitelist that license? has anyone else run into this "Unlicense"
license?

rday


Re: pkg_postinst_ontarget not executed

Damien LEFEVRE
 

OK I found.

Yes "opkg configure" will call /var/lib/opkg/info/*.postinst for packages marked as "unpacked" in /var/lib/opkg/status ex:

Package: test-deployment-lic
Version: 1.0-r0
Status: install ok unpacked
Architecture: aarch64
Installed-Time: 1589349316
Auto-Installed: yes
 
After update 
Package: test-deployment-lic
Version: 1.0-r0
Status: install ok unpacked
Architecture: aarch64
Installed-Time: 1589349316
Auto-Installed: yes
 
Should be easy to merge.

Thanks!
-Damien

On Wed, May 13, 2020 at 12:04 PM Alexander Kanavin <alex.kanavin@...> wrote:
Reading recipes-devtools/run-postinsts/run-postinsts/run-postinsts and_save_postinsts_common (in rootfs.py) once more, it seems /etc/ipk-postinsts is only used if there is no package manager on the target.

If there is, then 'opkg configure' is run directly, and so postinsts come from some internal database. There is some additional magic in rootfs.py to mark packages with first-boot postinsts as unpacked, so they'll get picked up by opkg.

Alex

On Wed, 13 May 2020 at 09:33, Damien LEFEVRE <lefevre.da@...> wrote:
Thanks Alex,

When I do a factory reset, the system detects as a first boot and the script is executed.

> cat /var/log/postinstall.log
Configuring test-deployment.

One thing which puzzles me: the /etc/ipk-postinsts directory do not exist. Not in the image recipe folder, not in the image file which I mount to check the content before flashing and of course it's deleted at the end of run-postinsts 

rootfs.py:
    def _save_postinsts_common(self, dst_postinst_dir, src_postinst_dir):
        if bb.utils.contains("IMAGE_FEATURES", "package-management",
                         True, False, self.d):
            return
        num = 0
        for p in self._get_delayed_postinsts():
            bb.utils.mkdirhier(dst_postinst_dir)

            if os.path.exists(os.path.join(src_postinst_dir, p + ".postinst")):
                shutil.copy(os.path.join(src_postinst_dir, p + ".postinst"),
                            os.path.join(dst_postinst_dir, "%03d-%s" % (num, p)))

            num += 1


package-management is in my image features so I understand nothing get written.

So who temporarily creates that /etc/ipk-postinsts?

Thanks,
-Damien

On Fri, May 8, 2020 at 6:15 PM Alexander Kanavin <alex.kanavin@...> wrote:
On Fri, 8 May 2020 at 11:32, Damien LEFEVRE <lefevre.da@...> wrote:

If can find the postinst script in /var/lib/opkg/info/test-deployment.postinst and execute it.

Since test-deployment is a new package, I would have expected pkg_postinst_ontarget to be executed

How is the first boot detected on a poky image? Is there a way to detect if .postinst scripts haven't been executed?

The scripts to be executed  are written into /etc/rpm|deb|ipk-postinsts/ and executed by
meta/recipes-devtools/run-postinsts/run-postinsts/run-postinsts script (which is started on first boot as a service).
Then they get deleted.

Alex


Re: pkg_postinst_ontarget not executed

Alexander Kanavin
 

Reading recipes-devtools/run-postinsts/run-postinsts/run-postinsts and_save_postinsts_common (in rootfs.py) once more, it seems /etc/ipk-postinsts is only used if there is no package manager on the target.

If there is, then 'opkg configure' is run directly, and so postinsts come from some internal database. There is some additional magic in rootfs.py to mark packages with first-boot postinsts as unpacked, so they'll get picked up by opkg.

Alex


On Wed, 13 May 2020 at 09:33, Damien LEFEVRE <lefevre.da@...> wrote:
Thanks Alex,

When I do a factory reset, the system detects as a first boot and the script is executed.

> cat /var/log/postinstall.log
Configuring test-deployment.

One thing which puzzles me: the /etc/ipk-postinsts directory do not exist. Not in the image recipe folder, not in the image file which I mount to check the content before flashing and of course it's deleted at the end of run-postinsts 

rootfs.py:
    def _save_postinsts_common(self, dst_postinst_dir, src_postinst_dir):
        if bb.utils.contains("IMAGE_FEATURES", "package-management",
                         True, False, self.d):
            return
        num = 0
        for p in self._get_delayed_postinsts():
            bb.utils.mkdirhier(dst_postinst_dir)

            if os.path.exists(os.path.join(src_postinst_dir, p + ".postinst")):
                shutil.copy(os.path.join(src_postinst_dir, p + ".postinst"),
                            os.path.join(dst_postinst_dir, "%03d-%s" % (num, p)))

            num += 1


package-management is in my image features so I understand nothing get written.

So who temporarily creates that /etc/ipk-postinsts?

Thanks,
-Damien

On Fri, May 8, 2020 at 6:15 PM Alexander Kanavin <alex.kanavin@...> wrote:
On Fri, 8 May 2020 at 11:32, Damien LEFEVRE <lefevre.da@...> wrote:

If can find the postinst script in /var/lib/opkg/info/test-deployment.postinst and execute it.

Since test-deployment is a new package, I would have expected pkg_postinst_ontarget to be executed

How is the first boot detected on a poky image? Is there a way to detect if .postinst scripts haven't been executed?

The scripts to be executed  are written into /etc/rpm|deb|ipk-postinsts/ and executed by
meta/recipes-devtools/run-postinsts/run-postinsts/run-postinsts script (which is started on first boot as a service).
Then they get deleted.

Alex


Re: Is there a relationship between the sstate and the machine?

Mikko Rapeli
 

Hi,

On Wed, May 13, 2020 at 10:36:49AM +0200, Mans Zigher wrote:
I am using a build environment based on the yocto project from one of
the big HW suppliers in the mobile industries. They are continuously
breaking the principles behind the yocto project and at one point they
managed to break the sstate cache because they are doing things in
there own way instead of using what already exists. They managed to
fix the sstate cache but it now looks to only work when using there
machines when I am defining my own machine even though it is a copy of
one of theirs the sstate cache it is not working. The sstate cache is
detect and the SSTATE_DIR is correct but it finds only a handful of
matches and builds the rest from scratch. Is there a logical
explanation for this? Or is this yet again some custom crap that this
supplier have done. FYI if you know what supplier I am talking about I
would very much recommend staying the hell away from them this is the
worst linux distro I have ever worked with both on the build and on
the system it is some kind of freak combination of a normal linux
system and android worst combination ever.
This is a common problem. It is likely that I know which SoC supplier you
are talking about and have had the exact same issues. We have managed to
fix or workaround all of them. In most cases by reducing what the SoC layers
do by BBMASK'ing all non-essential BSP recipes, reverse engineering the
machine configuration and bbclasses to only use bare minimal ones.

I can only hope that our feedback in terms of patches and workarounds also
going back the supplier chain(s) and you at some point see the same patches.
For example we updated the yocto open source layers from 2.5 sumo to 3.0 zeus
with very minimal support from the SoC and other suppliers and their layers
are still only compatible with 2.5 sumo, officially. Also switched from
using a custom Android based binary toolchain(s there were multiple
at some point) to using meta-clang.

If you have detailed question or problems, it would be nice to discuss them
here, but I know that some details may be closed. At least some general
aspects how to debug issues may be discussed without exposing too much
details or, well, company names.

Cheers,

-Mikko


Is there a relationship between the sstate and the machine?

Mans Zigher <mans.zigher@...>
 

Hi,

I am using a build environment based on the yocto project from one of
the big HW suppliers in the mobile industries. They are continuously
breaking the principles behind the yocto project and at one point they
managed to break the sstate cache because they are doing things in
there own way instead of using what already exists. They managed to
fix the sstate cache but it now looks to only work when using there
machines when I am defining my own machine even though it is a copy of
one of theirs the sstate cache it is not working. The sstate cache is
detect and the SSTATE_DIR is correct but it finds only a handful of
matches and builds the rest from scratch. Is there a logical
explanation for this? Or is this yet again some custom crap that this
supplier have done. FYI if you know what supplier I am talking about I
would very much recommend staying the hell away from them this is the
worst linux distro I have ever worked with both on the build and on
the system it is some kind of freak combination of a normal linux
system and android worst combination ever.

Thanks

3181 - 3200 of 52576