Seems like a shame not to add this feature for everyone to use... 

Yishai Jaffe

On Thu, Nov 17, 2022, 2:34 PM Alexander Kanavin <alex.kanavin@...> wrote:

 Making your own! You can have it added to the layer index for better publicity.

Alex

On Thu, 17 Nov 2022 at 13:27, Yishai Jaffe <yishai1999@...> wrote:
>
> What layer would you suggest?
>
> Yishai Jaffe
>
> On Thu, Nov 17, 2022, 2:21 PM Alexander Kanavin <alex.kanavin@...> wrote:
>>
>> You can start by placing it in a separate layer?
>>
>> Alex
>>
>> On Thu, 17 Nov 2022 at 13:04, Yishai Jaffe <yishai1999@...> wrote:
>> >
>> > I assume by core you mean openembedded-core/poky, yes?
>> >
>> > So where would you add it to?
>> >
>> > Seems to me like a very useful feature that many people would use if they knew the option exists.
>> >
>> > Also, I noticed that in the recipe for python there is a variable named INCLUDE_PYC which does exactly what you think it would. This looks to me like someone did think about the whole python size subject but didn't exactly go through with it all the way. You can decide if to include pyc files but not the opposite. Also, this feature which does exist only applies to the python recipe but not to all other non-base python module recipes. Maybe this can be added to setuptools3 bbclass?
>> >
>> > Yishai Jaffe
>> >
>> > On Thu, Nov 17, 2022, 1:24 PM Alexander Kanavin <alex.kanavin@...> wrote:
>> >>
>> >> I just wonder if this should really be in core. The standards for core
>> >> are high: it needs to be both testable and tested, and there's only so
>> >> many possible options and tweaks where things can regress that we can
>> >> take. On the other hand, there has not been much demand for it.
>> >>
>> >> Alex
>> >>
>> >> On Thu, 17 Nov 2022 at 12:13, Yishai Jaffe <yishai1999@...> wrote:
>> >> >
>> >> > Hi Alexander,
>> >> >
>> >> > After some research, these are the numbers I came up with for compiling core-image-minimal with python3-core:
>> >> >
>> >> > Normal compilation:
>> >> > tar.bz2 - 6.6MB
>> >> > squashfs-xz - 6.1MB
>> >> >
>> >> > With my pyc-only patch:
>> >> > tar.bz2 - 5.8MB
>> >> > squashfs-xz - 5.2MB
>> >> >
>> >> > So that's about a 15% decrease in size.
>> >> > Again, this is for an image only with python3-core. Logically that would mean that if you had an image with more python packages it would be even a bigger percentage.
>> >> >
>> >> >
>> >> > Yishai Jaffe
>> >> >
>> >> >
>> >> > On Sun, Nov 13, 2022 at 5:12 PM Alexander Kanavin <alex.kanavin@...> wrote:
>> >> >>
>> >> >> Generally we slim down python installations by not installing all of
>> >> >> the standard library, and rather having precise dependencies for
>> >> >> specific modules. Can you illustrate the kind of space savings that
>> >> >> can be gained in actual numbers?
>> >> >>
>> >> >> Another issue is that this should be supported upstream and in the
>> >> >> wider python community. Is it?
>> >> >>
>> >> >> Alex
>> >> >>
>> >> >> On Sun, 13 Nov 2022 at 14:25, Federico Pellegrin <fede@...> wrote:
>> >> >> >
>> >> >> >
>> >> >> > Hello,
>> >> >> > Just as a small reference since I raised some doubts and questions in Buildroot community on this: there has been also some troubles to understand the correctness or not there (as I found some packages with problems due to this source-less management) and this then sparked, besides discussions in the Buildroot mailing list (roughly end of July / beginning of August if someone interested searching there), also an issue to the Python community, which albeit some discussion I think never arrived to a concrete conclusion. This is the issue: https://github.com/python/cpython/issues/95827 (see also the linked one maybe)
>> >> >> >
>> >> >> > These are just my 2 cents to this discussion, then of course the Python experts will probably chime in and describe the official/discussed position of Yocto regarding to this, but just felt like mentioning that although Buildroot is somehow offering this right now (and likely mostly works as well!), it's not really a so clean and/or agreed solution.
>> >> >> >
>> >> >> > Cheers,
>> >> >> > Federico
>> >> >> >
>> >> >> > Il giorno dom 13 nov 2022 alle ore 13:55 Yishai Jaffe <yishai1999@...> ha scritto:
>> >> >> >>
>> >> >> >> Hi,
>> >> >> >> I was wondering if there has been talk about support for source-less python on an image. Installing py and pyc files doubles the size of python on the rootfs. I can imagine this being implemented as an image feature.
>> >> >> >> I know that in buildroot it is supported.
>> >> >> >> Was this discussed and decided against? Is this an open issue?
>> >> >> >> I have a working patch that implements this. I can submit it for review.
>> >> >> >>
>> >> >> >> Thanks,
>> >> >> >> Yishai Jaffe
>> >> >> >> Yishai Jaffe
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >
>> >> >> >
>> >> >> >

Hi,

Depending on the image that's being built, e.g. development vs. production, we would like to be able to include different user passwords and firewall settings.

We know this can be achieved by just having two different recipes that do the same thing, but with different variables or included files. However, we were wondering if there is a neater way?

We saw this post:

https://www.yoctoproject.org/pipermail/yocto/2018-June/041378.html

which seems to do what we want, because one recipe can install recipe-development, and the other can install recipe-production, while the recipe itself can then implement what needs to happen for either.

However, other recipes included in the images can also depend on recipe, and they shouldn't depend on one specifically. They should accept that either recipe-development or recipe-production is included. Currently we see that both the generic recipe and the specific recipe used by the image are built and overwrite each other.

What would be a neat way to achieve two variants of a recipe, and having different contents and settings in them?

Or, can we solve that other included recipes depend on one of the variants, instead of on the generic one.

Kind regards,

Maik Vermeulen

Embedded Software Engineer — Lightyear

+31 6 16 82 73 79

Automotive Campus 70 —5708 JZ Helmond, the Netherlands

www.lightyear.one

This email may contain information which is privileged and/or confidential. If you received this e-mail in error, please notify us immediately by e-mail and delete the email without copying or disclosing its contents to any other person. Lightyear is a trade name of Atlas Technologies B.V. and is registered at the Dutch Chamber of Commerce under number 67264298. 

 Making your own! You can have it added to the layer index for better publicity.

Alex

On Thu, 17 Nov 2022 at 13:27, Yishai Jaffe <yishai1999@...> wrote:
>
> What layer would you suggest?
>
> Yishai Jaffe
>
> On Thu, Nov 17, 2022, 2:21 PM Alexander Kanavin <alex.kanavin@...> wrote:
>>
>> You can start by placing it in a separate layer?
>>
>> Alex
>>
>> On Thu, 17 Nov 2022 at 13:04, Yishai Jaffe <yishai1999@...> wrote:
>> >
>> > I assume by core you mean openembedded-core/poky, yes?
>> >
>> > So where would you add it to?
>> >
>> > Seems to me like a very useful feature that many people would use if they knew the option exists.
>> >
>> > Also, I noticed that in the recipe for python there is a variable named INCLUDE_PYC which does exactly what you think it would. This looks to me like someone did think about the whole python size subject but didn't exactly go through with it all the way. You can decide if to include pyc files but not the opposite. Also, this feature which does exist only applies to the python recipe but not to all other non-base python module recipes. Maybe this can be added to setuptools3 bbclass?
>> >
>> > Yishai Jaffe
>> >
>> > On Thu, Nov 17, 2022, 1:24 PM Alexander Kanavin <alex.kanavin@...> wrote:
>> >>
>> >> I just wonder if this should really be in core. The standards for core
>> >> are high: it needs to be both testable and tested, and there's only so
>> >> many possible options and tweaks where things can regress that we can
>> >> take. On the other hand, there has not been much demand for it.
>> >>
>> >> Alex
>> >>
>> >> On Thu, 17 Nov 2022 at 12:13, Yishai Jaffe <yishai1999@...> wrote:
>> >> >
>> >> > Hi Alexander,
>> >> >
>> >> > After some research, these are the numbers I came up with for compiling core-image-minimal with python3-core:
>> >> >
>> >> > Normal compilation:
>> >> > tar.bz2 - 6.6MB
>> >> > squashfs-xz - 6.1MB
>> >> >
>> >> > With my pyc-only patch:
>> >> > tar.bz2 - 5.8MB
>> >> > squashfs-xz - 5.2MB
>> >> >
>> >> > So that's about a 15% decrease in size.
>> >> > Again, this is for an image only with python3-core. Logically that would mean that if you had an image with more python packages it would be even a bigger percentage.
>> >> >
>> >> >
>> >> > Yishai Jaffe
>> >> >
>> >> >
>> >> > On Sun, Nov 13, 2022 at 5:12 PM Alexander Kanavin <alex.kanavin@...> wrote:
>> >> >>
>> >> >> Generally we slim down python installations by not installing all of
>> >> >> the standard library, and rather having precise dependencies for
>> >> >> specific modules. Can you illustrate the kind of space savings that
>> >> >> can be gained in actual numbers?
>> >> >>
>> >> >> Another issue is that this should be supported upstream and in the
>> >> >> wider python community. Is it?
>> >> >>
>> >> >> Alex
>> >> >>
>> >> >> On Sun, 13 Nov 2022 at 14:25, Federico Pellegrin <fede@...> wrote:
>> >> >> >
>> >> >> >
>> >> >> > Hello,
>> >> >> > Just as a small reference since I raised some doubts and questions in Buildroot community on this: there has been also some troubles to understand the correctness or not there (as I found some packages with problems due to this source-less management) and this then sparked, besides discussions in the Buildroot mailing list (roughly end of July / beginning of August if someone interested searching there), also an issue to the Python community, which albeit some discussion I think never arrived to a concrete conclusion. This is the issue: https://github.com/python/cpython/issues/95827 (see also the linked one maybe)
>> >> >> >
>> >> >> > These are just my 2 cents to this discussion, then of course the Python experts will probably chime in and describe the official/discussed position of Yocto regarding to this, but just felt like mentioning that although Buildroot is somehow offering this right now (and likely mostly works as well!), it's not really a so clean and/or agreed solution.
>> >> >> >
>> >> >> > Cheers,
>> >> >> > Federico
>> >> >> >
>> >> >> > Il giorno dom 13 nov 2022 alle ore 13:55 Yishai Jaffe <yishai1999@...> ha scritto:
>> >> >> >>
>> >> >> >> Hi,
>> >> >> >> I was wondering if there has been talk about support for source-less python on an image. Installing py and pyc files doubles the size of python on the rootfs. I can imagine this being implemented as an image feature.
>> >> >> >> I know that in buildroot it is supported.
>> >> >> >> Was this discussed and decided against? Is this an open issue?
>> >> >> >> I have a working patch that implements this. I can submit it for review.
>> >> >> >>
>> >> >> >> Thanks,
>> >> >> >> Yishai Jaffe
>> >> >> >> Yishai Jaffe
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >
>> >> >> >
>> >> >> >

You can start by placing it in a separate layer?

Alex

On Thu, 17 Nov 2022 at 13:04, Yishai Jaffe <yishai1999@...> wrote:
>
> I assume by core you mean openembedded-core/poky, yes?
>
> So where would you add it to?
>
> Seems to me like a very useful feature that many people would use if they knew the option exists.
>
> Also, I noticed that in the recipe for python there is a variable named INCLUDE_PYC which does exactly what you think it would. This looks to me like someone did think about the whole python size subject but didn't exactly go through with it all the way. You can decide if to include pyc files but not the opposite. Also, this feature which does exist only applies to the python recipe but not to all other non-base python module recipes. Maybe this can be added to setuptools3 bbclass?
>
> Yishai Jaffe
>
> On Thu, Nov 17, 2022, 1:24 PM Alexander Kanavin <alex.kanavin@...> wrote:
>>
>> I just wonder if this should really be in core. The standards for core
>> are high: it needs to be both testable and tested, and there's only so
>> many possible options and tweaks where things can regress that we can
>> take. On the other hand, there has not been much demand for it.
>>
>> Alex
>>
>> On Thu, 17 Nov 2022 at 12:13, Yishai Jaffe <yishai1999@...> wrote:
>> >
>> > Hi Alexander,
>> >
>> > After some research, these are the numbers I came up with for compiling core-image-minimal with python3-core:
>> >
>> > Normal compilation:
>> > tar.bz2 - 6.6MB
>> > squashfs-xz - 6.1MB
>> >
>> > With my pyc-only patch:
>> > tar.bz2 - 5.8MB
>> > squashfs-xz - 5.2MB
>> >
>> > So that's about a 15% decrease in size.
>> > Again, this is for an image only with python3-core. Logically that would mean that if you had an image with more python packages it would be even a bigger percentage.
>> >
>> >
>> > Yishai Jaffe
>> >
>> >
>> > On Sun, Nov 13, 2022 at 5:12 PM Alexander Kanavin <alex.kanavin@...> wrote:
>> >>
>> >> Generally we slim down python installations by not installing all of
>> >> the standard library, and rather having precise dependencies for
>> >> specific modules. Can you illustrate the kind of space savings that
>> >> can be gained in actual numbers?
>> >>
>> >> Another issue is that this should be supported upstream and in the
>> >> wider python community. Is it?
>> >>
>> >> Alex
>> >>
>> >> On Sun, 13 Nov 2022 at 14:25, Federico Pellegrin <fede@...> wrote:
>> >> >
>> >> >
>> >> > Hello,
>> >> > Just as a small reference since I raised some doubts and questions in Buildroot community on this: there has been also some troubles to understand the correctness or not there (as I found some packages with problems due to this source-less management) and this then sparked, besides discussions in the Buildroot mailing list (roughly end of July / beginning of August if someone interested searching there), also an issue to the Python community, which albeit some discussion I think never arrived to a concrete conclusion. This is the issue: https://github.com/python/cpython/issues/95827 (see also the linked one maybe)
>> >> >
>> >> > These are just my 2 cents to this discussion, then of course the Python experts will probably chime in and describe the official/discussed position of Yocto regarding to this, but just felt like mentioning that although Buildroot is somehow offering this right now (and likely mostly works as well!), it's not really a so clean and/or agreed solution.
>> >> >
>> >> > Cheers,
>> >> > Federico
>> >> >
>> >> > Il giorno dom 13 nov 2022 alle ore 13:55 Yishai Jaffe <yishai1999@...> ha scritto:
>> >> >>
>> >> >> Hi,
>> >> >> I was wondering if there has been talk about support for source-less python on an image. Installing py and pyc files doubles the size of python on the rootfs. I can imagine this being implemented as an image feature.
>> >> >> I know that in buildroot it is supported.
>> >> >> Was this discussed and decided against? Is this an open issue?
>> >> >> I have a working patch that implements this. I can submit it for review.
>> >> >>
>> >> >> Thanks,
>> >> >> Yishai Jaffe
>> >> >> Yishai Jaffe
>> >> >>
>> >> >>
>> >> >>
>> >> >
>> >> >
>> >> >

I just wonder if this should really be in core. The standards for core
are high: it needs to be both testable and tested, and there's only so
many possible options and tweaks where things can regress that we can
take. On the other hand, there has not been much demand for it.

Alex

On Thu, 17 Nov 2022 at 12:13, Yishai Jaffe <yishai1999@...> wrote:
>
> Hi Alexander,
>
> After some research, these are the numbers I came up with for compiling core-image-minimal with python3-core:
>
> Normal compilation:
> tar.bz2 - 6.6MB
> squashfs-xz - 6.1MB
>
> With my pyc-only patch:
> tar.bz2 - 5.8MB
> squashfs-xz - 5.2MB
>
> So that's about a 15% decrease in size.
> Again, this is for an image only with python3-core. Logically that would mean that if you had an image with more python packages it would be even a bigger percentage.
>
>
> Yishai Jaffe
>
>
> On Sun, Nov 13, 2022 at 5:12 PM Alexander Kanavin <alex.kanavin@...> wrote:
>>
>> Generally we slim down python installations by not installing all of
>> the standard library, and rather having precise dependencies for
>> specific modules. Can you illustrate the kind of space savings that
>> can be gained in actual numbers?
>>
>> Another issue is that this should be supported upstream and in the
>> wider python community. Is it?
>>
>> Alex
>>
>> On Sun, 13 Nov 2022 at 14:25, Federico Pellegrin <fede@...> wrote:
>> >
>> >
>> > Hello,
>> > Just as a small reference since I raised some doubts and questions in Buildroot community on this: there has been also some troubles to understand the correctness or not there (as I found some packages with problems due to this source-less management) and this then sparked, besides discussions in the Buildroot mailing list (roughly end of July / beginning of August if someone interested searching there), also an issue to the Python community, which albeit some discussion I think never arrived to a concrete conclusion. This is the issue: https://github.com/python/cpython/issues/95827 (see also the linked one maybe)
>> >
>> > These are just my 2 cents to this discussion, then of course the Python experts will probably chime in and describe the official/discussed position of Yocto regarding to this, but just felt like mentioning that although Buildroot is somehow offering this right now (and likely mostly works as well!), it's not really a so clean and/or agreed solution.
>> >
>> > Cheers,
>> > Federico
>> >
>> > Il giorno dom 13 nov 2022 alle ore 13:55 Yishai Jaffe <yishai1999@...> ha scritto:
>> >>
>> >> Hi,
>> >> I was wondering if there has been talk about support for source-less python on an image. Installing py and pyc files doubles the size of python on the rootfs. I can imagine this being implemented as an image feature.
>> >> I know that in buildroot it is supported.
>> >> Was this discussed and decided against? Is this an open issue?
>> >> I have a working patch that implements this. I can submit it for review.
>> >>
>> >> Thanks,
>> >> Yishai Jaffe
>> >> Yishai Jaffe
>> >>
>> >>
>> >>
>> >
>> >
>> >

Generally we slim down python installations by not installing all of
the standard library, and rather having precise dependencies for
specific modules. Can you illustrate the kind of space savings that
can be gained in actual numbers?

Another issue is that this should be supported upstream and in the
wider python community. Is it?

Alex

On Sun, 13 Nov 2022 at 14:25, Federico Pellegrin <fede@...> wrote:
>
>
> Hello,
> Just as a small reference since I raised some doubts and questions in Buildroot community on this: there has been also some troubles to understand the correctness or not there (as I found some packages with problems due to this source-less management) and this then sparked, besides discussions in the Buildroot mailing list (roughly end of July / beginning of August if someone interested searching there), also an issue to the Python community, which albeit some discussion I think never arrived to a concrete conclusion. This is the issue: https://github.com/python/cpython/issues/95827 (see also the linked one maybe)
>
> These are just my 2 cents to this discussion, then of course the Python experts will probably chime in and describe the official/discussed position of Yocto regarding to this, but just felt like mentioning that although Buildroot is somehow offering this right now (and likely mostly works as well!), it's not really a so clean and/or agreed solution.
>
> Cheers,
> Federico
>
> Il giorno dom 13 nov 2022 alle ore 13:55 Yishai Jaffe <yishai1999@...> ha scritto:
>>
>> Hi,
>> I was wondering if there has been talk about support for source-less python on an image. Installing py and pyc files doubles the size of python on the rootfs. I can imagine this being implemented as an image feature.
>> I know that in buildroot it is supported.
>> Was this discussed and decided against? Is this an open issue?
>> I have a working patch that implements this. I can submit it for review.
>>
>> Thanks,
>> Yishai Jaffe
>> Yishai Jaffe
>>
>>
>>
>
>
>

On Mon, Nov 14, 2022 at 9:46 PM Shruti gandotra <shruti10gandotra@...> wrote:

I am trying to build rcar-image-minimal for RCar S4, while bitbake I'm getting do_rootfs: The postinstall intercept hook 'update_pixbuf_cache' failed error. I tried using cleanall and cleansstate but I'm still getting the same error. 
Can anyone help me with issue?

Look into the logs in the rootfs build area and it might reveal the underlying problem perhaps some missing library or qemu user mode crash could be the reason