Thank you for the replies, Alex and Michael.
Have joined the docs list.
Couple of related threads for reference:
https://lists.yoctoproject.org/g/yocto/topic/question_how_to_handle/90285507
https://lists.yoctoproject.org/g/yocto/topic/incompatible_license_how_to/75210517

Thank you
-- Peter

Thank you for the replies, Alex and Michael.
Have joined the docs list.
Couple of related threads for reference:
https://lists.yoctoproject.org/g/yocto/topic/question_how_to_handle/90285507
https://lists.yoctoproject.org/g/yocto/topic/incompatible_license_how_to/75210517

Thank you
-- Peter

On Thu, 6 Oct 2022 at 15:27, Peter via lists.yoctoproject.org
<poberauer=yahoo.co.uk@...> wrote:

"It is over. There are no excuses left if you are still using meta-gplv2."
-- https://twitter.com/yoctoproject/status/1552209990148145153
"There are other ways of removing GPLv3 components from modern OE/YP builds and we'd like to focus people's attention onto those."
-- https://git.yoctoproject.org/meta-gplv2/commit/?id=43bf0e8d5985945d19d01f94bfbbda420c4435f3

Where can we find more details re the best ways of removing GPLv3 components please?

Or, starting with a clean slate and adding back only what we actually need (for i.MX 8X and Qt Commercial).

Have tried searching, but going round in circles, as all the results so far still point back to meta-gplv2.
Including the "How do I":
"If you use INCOMPATIBLE_LICENSE to exclude GPLv3 or set PREFERRED_VERSION to substitute a GPLv2 version of a GPLv3 recipe, then you must add the meta-gplv2 layer to your configuration."
-- https://wiki.yoctoproject.org/wiki/How_do_I#Q:_How_do_I_build_an_image_without_GPLv3_Licensed_packages_.3F

The context is that some of our customers require Secure Boot and Chain of Trust (in an embedded environment).
If we keep GPLv3 components, then we need to provide "any methods, procedures, authorization keys, or other information required to install and execute modified versions of a covered work".

The suggested way nowadays is to set INCOMPATIBLE_LICENSE only for the
image that is actually going to ship in a product, then work your way
throgh specific gpl3 dependencies that get pulled in, and eliminate
them one by one - precisely how is impossible to tell beforehands, but
typical problems are bash scripts (which you need to package
separately and not include into the product, or rewrite in posix shell
and use #!/bin/sh), or optional dependencies on things like readline,
which can be switched off via PACKAGECONFIG tweaks.

"It is over. There are no excuses left if you are still using meta-gplv2."
-- https://twitter.com/yoctoproject/status/1552209990148145153
"There are other ways of removing GPLv3 components from modern OE/YP builds and we'd like to focus people's attention onto those."
-- https://git.yoctoproject.org/meta-gplv2/commit/?id=43bf0e8d5985945d19d01f94bfbbda420c4435f3

Where can we find more details re the best ways of removing GPLv3 components please?

Or, starting with a clean slate and adding back only what we actually need (for i.MX 8X and Qt Commercial).

Have tried searching, but going round in circles, as all the results so far still point back to meta-gplv2.
Including the "How do I":
"If you use INCOMPATIBLE_LICENSE to exclude GPLv3 or set PREFERRED_VERSION to substitute a GPLv2 version of a GPLv3 recipe, then you must add the meta-gplv2 layer to your configuration."
-- https://wiki.yoctoproject.org/wiki/How_do_I#Q:_How_do_I_build_an_image_without_GPLv3_Licensed_packages_.3F

The context is that some of our customers require Secure Boot and Chain of Trust (in an embedded environment).
If we keep GPLv3 components, then we need to provide "any methods, procedures, authorization keys, or other information required to install and execute modified versions of a covered work".

Hi

"It is over. There are no excuses left if you are still using meta-gplv2."
-- https://twitter.com/yoctoproject/status/1552209990148145153
"There are other ways of removing GPLv3 components from modern OE/YP builds and we'd like to focus people's attention onto those."
-- https://git.yoctoproject.org/meta-gplv2/commit/?id=43bf0e8d5985945d19d01f94bfbbda420c4435f3

Where can we find more details re the best ways of removing GPLv3 components please?

Or, starting with a clean slate and adding back only what we actually need (for i.MX 8X and Qt Commercial).

Have tried searching, but going round in circles, as all the results so far still point back to meta-gplv2.
Including the "How do I":
"If you use INCOMPATIBLE_LICENSE to exclude GPLv3 or set PREFERRED_VERSION to substitute a GPLv2 version of a GPLv3 recipe, then you must add the meta-gplv2 layer to your configuration."
-- https://wiki.yoctoproject.org/wiki/How_do_I#Q:_How_do_I_build_an_image_without_GPLv3_Licensed_packages_.3F

The context is that some of our customers require Secure Boot and Chain of Trust (in an embedded environment).
If we keep GPLv3 components, then we need to provide "any methods, procedures, authorization keys, or other information required to install and execute modified versions of a covered work".

Thank you
-- Peter

Hi all.
I'd like to explod a tar file into subdirectory of source file.
The recipe fetch the original source from a git repos.
I make a tar of folder I'd like to add to the original sources.
SRC_URI := "\
    git://git.myserver.com/project.git \
    file://added_folder.tar \
"
# S is ${WORKDIR}/git
Now added_folder.tar is exploded in ${WORKDIR} but I'd like to explod it in ${WORKDIR}/git
There is some parameter for file:// fetcher?

Thank you for the proposed documentation.
Do you want me to look for an appropriate place in the documentation
sources where it could be included, and then propose a patch?

TL;DR version
=============

If you want to get everything needed to run a yocto build:

- clone the bootstrap repository (ask your distribution maintainer where it is):
$ git clone bootstrap_repo_uri

- run 'setup-layers' from that repo:
$ /path/to/bootstrap_repo/setup-layers

If you want to save the layers setup for replication elsewhere:

- save the config file and the replication script into a 'bootstrap repository')
(this can be an organization-specific yocto layer, or a standalone repository):
$ bitbake-layers create-layers-setup /path/to/bootstrap_repo

- document where the bootstrap repository can be found for your users

What is this layer setup tooling? (the long version)
====================================================

Once you have a working build with the correct set of layers, it is beneficial
to capture the layer setup --- what they are, which repositories they come from
and which SCM revisions they're at --- into a configuration file, so that this
setup can be easily replicated later, perhaps on a different machine. Here's
how to do this::

$ bitbake-layers create-layers-setup /srv/work/alex/meta-alex/
NOTE: Starting bitbake server...
NOTE: Created /srv/work/alex/meta-alex/setup-layers.json
NOTE: Created /srv/work/alex/meta-alex/setup-layers

The tool needs a single argument which tells where to place the output, consisting
of a json formatted layer configuration, and a ``setup-layers`` script that can use that configuration
to restore the layers in a different location, or on a different host machine. The argument
can point to a custom layer (which is then deemed a "bootstrap" layer that needs to be
checked out first), or into a completely independent location.

The replication of the layers is performed by running the ``setup-layers`` script provided
above:

1. Clone the bootstrap layer or some other repository to obtain
the json config and the setup script that can use it.

2. Run the script directly with no options::

alex@Zen2:/srv/work/alex/my-build$ meta-alex/setup-layers
Note: not checking out source meta-alex, use --force-bootstraplayer-checkout to override.

Setting up source meta-intel, revision 15.0-hardknott-3.3-310-g0a96edae, branch master
Running 'git init -q /srv/work/alex/my-build/meta-intel'
Running 'git remote remove origin > /dev/null 2>&1; git remote add origin git://git.yoctoproject.org/meta-intel' in /srv/work/alex/my-build/meta-intel
Running 'git fetch -q origin || true' in /srv/work/alex/my-build/meta-intel
Running 'git checkout -q 0a96edae609a3f48befac36af82cf1eed6786b4a' in /srv/work/alex/my-build/meta-intel

Setting up source poky, revision 4.1_M1-372-g55483d28f2, branch akanavin/setup-layers
Running 'git init -q /srv/work/alex/my-build/poky'
Running 'git remote remove origin > /dev/null 2>&1; git remote add origin git://git.yoctoproject.org/poky' in /srv/work/alex/my-build/poky
Running 'git fetch -q origin || true' in /srv/work/alex/my-build/poky
Running 'git remote remove poky-contrib > /dev/null 2>&1; git remote add poky-contrib ssh://git@.../poky-contrib' in /srv/work/alex/my-build/poky
Running 'git fetch -q poky-contrib || true' in /srv/work/alex/my-build/poky
Running 'git checkout -q 11db0390b02acac1324e0f827beb0e2e3d0d1d63' in /srv/work/alex/my-build/poky

.. note::

This will work to update an existing checkout as well.

.. note::
The script is self-sufficient and requires only python3
and git on the build machine.

.. note::
Both the ``create-layers-setup`` and the ``setup-layers`` provided several additional options
that customize their behavior - you are welcome to study them via ``--help`` command line parameter.

Questions and answers
=====================

1. Why JSON, and not YAML?

JSON is a part of the python standard library, while YAML is not. This means you can bootstrap a build on just about any
machine that has python and its core library, without first having to install any dependencies. JSON not quite as easy
on the eye, but with decent indentation it's totally ok.

Before anyone asks, XML is appallingly difficult to read by modern standards of readability. No thanks.

2. I don't want the script in the bootstrap repo, just the config. I will provide my own tools. How?

Use --json-only to generate only the config, and not the script:

$ bitbake-layers create-layers-setup -json-only /srv/work/alex/meta-alex/

3. I want to generate the compatible json with my own tools. Is there a schema to validate it?

There is:
https://git.yoctoproject.org/poky/tree/meta/files/layers.schema.json

You can use python3-jsonschema package to check the validity.

4. I don't want the json or the script, and would want to use an entirely different format and tools. Is
'create-layers-setup' hardcoded to the OE json format?

Actually not! Other formats can be added through a plugin mechanism. OE json writer is just
the default plugin but others can be implemented and selected:

--writer {oe-setup-layers}, -w {oe-setup-layers}
Choose the output format (defaults to oe-setup-layers).

5. I want to use something else than hardcoded revisions in the json (e.g. refer to branch
names or tags).

No problem! Edit the json to say 'main' or 'release-x.y'. Anything that 'git checkout' can take will
work. Just be careful that these modification are not overwritten later by 'create-layers-setup' by
e.g. renaming the json file first.

6. I already have an existing checkout and want to update it to the latest greatest. How?

First, update the bootstrap repository to the latest (or otherwise desired) revision to
obtain the latest versions of the script, and the json config. Ask the distro maintainers
for the recommended practice to do so.

Then, run the script again - it should be able to handle a layers checkout that already
exists, and update everything accoriding to the latest json, obtained in the first step.

7. Ok, I have checked out all the layers. Now what?

The next step is to actually set up a yocto build from an existing configuration template,
and start a bitbake session. This is made separate from handling the layer repositories and
will be described separately.

Hi Quentin,
On Tue, 4 Oct 2022 10:15:20 +0200
Quentin Schulz <quentin.schulz@...> wrote:

Hi Luca,

On 10/3/22 23:15, Luca Ceresoli wrote:

Hi Quentin,

On Mon, 3 Oct 2022 19:04:01 +0200
"Quentin Schulz" <foss@...> wrote:

From: Quentin Schulz <quentin.schulz@...>

It happened that the git repositories were dirty and resulted in
incorrect files being used. Let's use git clean -ffdx to force a
completely clean git repositories before and after checking out a branch
so that nothing is left from or to another branch build

Cc: Quentin Schulz <foss+yocto@...>
Signed-off-by: Quentin Schulz <quentin.schulz@...>
---
scripts/run-docs-build | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/scripts/run-docs-build b/scripts/run-docs-build
index c6b3965..69e3257 100755
--- a/scripts/run-docs-build
+++ b/scripts/run-docs-build
@@ -61,6 +61,7 @@ for branch in 1.46 $(git branch --remote --contains "$first_sphinx_commit" --for
echo Building bitbake $branch branch
git checkout $branch
+ git clean -ffdx
git checkout origin/master releases.rst
make clean
SPHINXOPTS="-j auto" make publish
@@ -80,7 +81,7 @@ for branch in 1.46 $(git branch --remote --contains "$first_sphinx_commit" --for
fi
cp -r ./_build/final/* $outputdir/bitbake/$branch
- git reset --hard
+ git clean -ffdx

Sure this is correct? 'git clean -ffdx' does not revert changes to
tracked files, be them staged or not.

Nope, not sure this is correct. I misread git clean manpage, we should
have a git reset --hard and git clean -ffdx. Now the question is when
those are necessary because with this patch we do it twice, before and
after the git checkout. I did this because I remember doing checkouts
between branches of U-Boot/kernel and while the pre-checkout branch was
not dirty, the after-checkout branch was dirty. I assume this might have
something to do with build artifacts of the pre-checkout build that
weren't .gitignored in the afer-checkout branch? Something that git
clean -ffdx should tackle I think.

Sooo, I guess only having git reset --hard and git clean -ffdx before a
checkout should be enough and we don't need them both before and after
the checkout like I did in this patch?

I think 'reset --hard' + 'clean -ffdx' only before the checkout should
be enough. However I'm not sure whether there are corner cases such as
a file that is .gitignored in commit A and versioned in commit B or
similar. Perhaps worth trying with reset+clean only before, and see

what happens. However I don't know exactly the initial problem you're
trying to fix.

Hi Luca,

On 10/3/22 23:15, Luca Ceresoli wrote:

Hi Quentin,

On Mon, 3 Oct 2022 19:04:01 +0200
"Quentin Schulz" <foss@...> wrote:

From: Quentin Schulz <quentin.schulz@...>

It happened that the git repositories were dirty and resulted in
incorrect files being used. Let's use git clean -ffdx to force a
completely clean git repositories before and after checking out a branch
so that nothing is left from or to another branch build

Cc: Quentin Schulz <foss+yocto@...>
Signed-off-by: Quentin Schulz <quentin.schulz@...>
---
scripts/run-docs-build | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/scripts/run-docs-build b/scripts/run-docs-build
index c6b3965..69e3257 100755
--- a/scripts/run-docs-build
+++ b/scripts/run-docs-build
@@ -61,6 +61,7 @@ for branch in 1.46 $(git branch --remote --contains "$first_sphinx_commit" --for

echo Building bitbake $branch branch
git checkout $branch
+ git clean -ffdx
git checkout origin/master releases.rst
make clean
SPHINXOPTS="-j auto" make publish
@@ -80,7 +81,7 @@ for branch in 1.46 $(git branch --remote --contains "$first_sphinx_commit" --for
fi

cp -r ./_build/final/* $outputdir/bitbake/$branch
- git reset --hard
+ git clean -ffdx

Sure this is correct? 'git clean -ffdx' does not revert changes to
tracked files, be them staged or not.

Nope, not sure this is correct. I misread git clean manpage, we should
have a git reset --hard and git clean -ffdx. Now the question is when
those are necessary because with this patch we do it twice, before and
after the git checkout. I did this because I remember doing checkouts
between branches of U-Boot/kernel and while the pre-checkout branch was
not dirty, the after-checkout branch was dirty. I assume this might have
something to do with build artifacts of the pre-checkout build that
weren't .gitignored in the afer-checkout branch? Something that git
clean -ffdx should tackle I think.

Sooo, I guess only having git reset --hard and git clean -ffdx before a
checkout should be enough and we don't need them both before and after
the checkout like I did in this patch?