Navigating the layer labyrinth


Bernd <prof7bit@...>
 

I am a new user for a few weeks now, trying to make a customized image
for a toradex colibri-vf module, so far I have succeeded in the
following disciplines:

* adding the 3rd party layers that I need
* making my own layers
* using a .bbappend to patch the device tree
* using a .bbappend to workaround a bug(?) in one of the freescale layers
* writing my own recipe to install a python script
* writing recipes for pulling additional python packages with pypi and
setuptools3
* writing my own image recipe
* making it boot and run on the target platform

During this learning experience I have made the following observations
of circumstances that made it especially hard for me to get things
done, I'm not yet really sure if this is a documentation issue or if
it is really a missing feature but I feel I could have had a much
*much* easier time learning and understanding the concepts and
relationships and the inner workings of existing layers upon which I
want to build my system if the following things were possible (and/or
if they are already possible they should be documented in the very
first chapter of the documentation):

* Finding the *file path* of an existing recipe (or append file or
class) *by its name* and also all existing .bbappends for it, i
imagine something simple like bitbake --show-paths foo-bar would
output me the small list of absolute paths of recipe files by the name
foo-bar and all matching .bbappend files in the order in which they
would be applied, it would show me only this small list of paths and
not dump 100kb of unrelated information along with it. This would be
incredibly helpful when I need to inspect an existing recipe in order
to understand how I can bbappend it or even just to see and understand
what it actually does.

* A simple way to track the assignment of a certain variable, to
inspect its contents and if it refers to other variables then
recursively show their contents too (and also the path of the bb file
where this happens), and also show which other recipes will directly
and indirectrly depend on this variable further down the line, I
imagine this should output two tree-like structures where one can see
at one glance how and where all the contents of that variable come
from and where they are going to be used. Again this should be a
simple command that formats and outputs that (and only that)
information in a well formatted and compact tree-like representation.

* The absolute killer application would be an IDE or an editor plugin
where I open any .bb file and can then just CTRL-click on any include,
require, inherit, depend, rdepend, or any variable name and it would
open another editor containing that recipe file where it is defined
and/or populate a sidebar with a list or a tree of direct and indirect
references to that name, backward and forward, and I could just click
on any node of that tree an and it would open the file in the editor
and jump to that line of code. Such a thing would be an incredibly
helpful tool, it would make even the most complex and tangled
labyrinth of recipes navigable with ease.

Please tell me that such a thing already exists and I just have not
found it yet.

Bernd


Aaron Schwartz <aaron.schwartz@...>
 

 Hello,

I am not sure if you've found the OpenEmbedded Layer Index [0] yet, but that's a good resource and an example of what can be done.  I believe the source code is available [1] and I've toyed with the idea of getting it working locally (although I've not had the time to do so).

That could be a part of what you're looking for, at least.

Hope that helps,
Aaron





On Thu, Oct 12, 2017 at 5:34 AM, Bernd <prof7bit@...> wrote:
I am a new user for a few weeks now, trying to make a customized image
for a toradex colibri-vf module, so far I have succeeded in the
following disciplines:

* adding the 3rd party layers that I need
* making my own layers
* using a .bbappend to patch the device tree
* using a .bbappend to workaround a bug(?) in one of the freescale layers
* writing my own recipe to install a python script
* writing recipes for pulling additional python packages with pypi and
setuptools3
* writing my own image recipe
* making it boot and run on the target platform

During this learning experience I have made the following observations
of circumstances that made it especially hard for me to get things
done, I'm not yet really sure if this is a documentation issue or if
it is really a missing feature but I feel I could have had a much
*much* easier time learning and understanding the concepts and
relationships and the inner workings of existing layers upon which I
want to build my system if the following things were possible (and/or
if they are already possible they should be documented in the very
first chapter of the documentation):

* Finding the *file path* of an existing recipe (or append file or
class) *by its name* and also all existing .bbappends for it, i
imagine something simple like bitbake --show-paths foo-bar would
output me the small list of absolute paths of recipe files by the name
foo-bar and all matching .bbappend files in the order in which they
would be applied, it would show me only this small list of paths and
not dump 100kb of unrelated information along with it. This would be
incredibly helpful when I need to inspect an existing recipe in order
to understand how I can bbappend it or even just to see and understand
what it actually does.

* A simple way to track the assignment of a certain variable, to
inspect its contents and if it refers to other variables then
recursively show their contents too (and also the path of the bb file
where this happens), and also show which other recipes will directly
and indirectrly depend on this variable further down the line, I
imagine this should output two tree-like structures where one can see
at one glance how and where all the contents of that variable come
from and where they are going to be used. Again this should be a
simple command that formats and outputs that (and only that)
information in a well formatted and compact tree-like representation.

* The absolute killer application would be an IDE or an editor plugin
where I open any .bb file and can then just CTRL-click on any include,
require, inherit, depend, rdepend, or any variable name and it would
open another editor containing that recipe file where it is defined
and/or populate a sidebar with a list or a tree of direct and indirect
references to that name, backward and forward, and I could just click
on any node of that tree an and it would open the file in the editor
and jump to that line of code. Such a thing would be an incredibly
helpful tool, it would make even the most complex and tangled
labyrinth of recipes navigable with ease.

Please tell me that such a thing already exists and I just have not
found it yet.

Bernd
--
_______________________________________________
yocto mailing list
yocto@...
https://lists.yoctoproject.org/listinfo/yocto



--

Aaron Schwartz
Production
Logic Supply
Direct: +1 802 861 2300 Ext. 530
Main: +1 802 861 2300
www.logicsupply.com

Google+ | Twitter | LinkedIn | YouTube


Alexander Kanavin <alexander.kanavin@...>
 

On 10/12/2017 12:34 PM, Bernd wrote:
* Finding the *file path* of an existing recipe (or append file or
class) *by its name* and also all existing .bbappends for it, i
imagine something simple like bitbake --show-paths foo-bar would
output me the small list of absolute paths of recipe files by the name
foo-bar and all matching .bbappend files in the order in which they
would be applied, it would show me only this small list of paths and
not dump 100kb of unrelated information along with it. This would be
incredibly helpful when I need to inspect an existing recipe in order
to understand how I can bbappend it or even just to see and understand
what it actually does.
* A simple way to track the assignment of a certain variable, to
inspect its contents and if it refers to other variables then
recursively show their contents too (and also the path of the bb file
where this happens), and also show which other recipes will directly
and indirectrly depend on this variable further down the line, I
imagine this should output two tree-like structures where one can see
at one glance how and where all the contents of that variable come
from and where they are going to be used. Again this should be a
simple command that formats and outputs that (and only that)
information in a well formatted and compact tree-like representation.
I think bitbake -e could be of some help with these two things. It's not particularly compact, but it tells you the full story.

Alex


Joshua Lock <joshua.g.lock@...>
 

On 12/10/17 10:34, Bernd wrote:
I am a new user for a few weeks now, trying to make a customized image
for a toradex colibri-vf module, so far I have succeeded in the
following disciplines:
* adding the 3rd party layers that I need
* making my own layers
* using a .bbappend to patch the device tree
* using a .bbappend to workaround a bug(?) in one of the freescale layers
* writing my own recipe to install a python script
* writing recipes for pulling additional python packages with pypi and
setuptools3
* writing my own image recipe
* making it boot and run on the target platform
During this learning experience I have made the following observations
of circumstances that made it especially hard for me to get things
done, I'm not yet really sure if this is a documentation issue or if
it is really a missing feature but I feel I could have had a much
*much* easier time learning and understanding the concepts and
relationships and the inner workings of existing layers upon which I
want to build my system if the following things were possible (and/or
if they are already possible they should be documented in the very
first chapter of the documentation):
* Finding the *file path* of an existing recipe (or append file or
class) *by its name* and also all existing .bbappends for it, i
imagine something simple like bitbake --show-paths foo-bar would
output me the small list of absolute paths of recipe files by the name
foo-bar and all matching .bbappend files in the order in which they
would be applied, it would show me only this small list of paths and
not dump 100kb of unrelated information along with it. This would be
incredibly helpful when I need to inspect an existing recipe in order
to understand how I can bbappend it or even just to see and understand
what it actually does.
bitbake-layers supports at least some of this functionality:

$ bitbake-layers -h
NOTE: Starting bitbake server...
usage: bitbake-layers [-d] [-q] [-F] [--color COLOR] [-h] <subcommand> ...

BitBake layers utility

optional arguments:
-d, --debug Enable debug output
-q, --quiet Print only errors
-F, --force Force add without recipe parse verification
--color COLOR Colorize output (where COLOR is auto, always, never)
-h, --help show this help message and exit

subcommands:
<subcommand>
add-layer Add a layer to bblayers.conf.
remove-layer Remove a layer from bblayers.conf.
flatten flatten layer configuration into a separate output
directory.
layerindex-fetch Fetches a layer from a layer index along with its
dependent layers, and adds them to conf/bblayers.conf.
layerindex-show-depends
Find layer dependencies from layer index.
show-layers show current configured layers.
show-overlayed list overlayed recipes (where the same recipe exists
in another layer)
show-recipes list available recipes, showing the layer they are
provided by
show-appends list bbappend files and recipe files they apply to
show-cross-depends Show dependencies between recipes that cross layer
boundaries.
create-layer Create a basic layer

Use bitbake-layers <subcommand> --help to get help on a specific command

Thanks,

Joshua