Reducing the perl footprint on my image


Rusty Howell
 

I have an image that is using debian package management  (PACKAGE_CLASSES = "package_deb").  Because apt and dpkg require perl, perl is being installed in the image.   No problem.  Except that the entire perl stack is 669 packages. Most of the packages are super tiny and are probably not needed by apt or dpkg.  Is there a way to strip down the large number of perl packages in the image to just what is needed by apt/dpkg?
I haven't found anything in the MegaManual yet.
Thanks in advance.


Steve Sakoman
 

On Tue, Mar 2, 2021 at 6:26 AM <rustyhowell@...> wrote:

I have an image that is using debian package management (PACKAGE_CLASSES = "package_deb"). Because apt and dpkg require perl, perl is being installed in the image. No problem. Except that the entire perl stack is 669 packages.
I just took a look at the manifest for one of my images that includes
PACKAGE_CLASSES = "package_deb". I see the perl package plus 43
perl-module packages. Are you sure that something else in your images
isn't pulling in all of those other perl-module packages?

Steve

Most of the packages are super tiny and are probably not needed by apt or dpkg. Is there a way to strip down the large number of perl packages in the image to just what is needed by apt/dpkg?
I haven't found anything in the MegaManual yet.
Thanks in advance.


Steve Sakoman
 

On Tue, Mar 2, 2021 at 10:01 AM Steve Sakoman via
lists.yoctoproject.org <steve=sakoman.com@...>
wrote:

On Tue, Mar 2, 2021 at 6:26 AM <rustyhowell@...> wrote:

I have an image that is using debian package management (PACKAGE_CLASSES = "package_deb"). Because apt and dpkg require perl, perl is being installed in the image. No problem. Except that the entire perl stack is 669 packages.
I just took a look at the manifest for one of my images that includes
PACKAGE_CLASSES = "package_deb". I see the perl package plus 43
perl-module packages. Are you sure that something else in your images
isn't pulling in all of those other perl-module packages?
It just occurred to me to make sure you are looking in the image
manifest to see which packages are actually installed in your image.
The perl recipe does generate 676 packages (in dunfell) so perhaps you
might be looking at the generated packages rather than the installed
packages??

Steve


Most of the packages are super tiny and are probably not needed by apt or dpkg. Is there a way to strip down the large number of perl packages in the image to just what is needed by apt/dpkg?
I haven't found anything in the MegaManual yet.
Thanks in advance.



Diego Santa Cruz
 

-----Original Message-----
From: yocto@... <yocto@...> On
Behalf Of Steve Sakoman via lists.yoctoproject.org
Sent: 02 March 2021 21:06
To: Steve Sakoman <steve@...>
Cc: rustyhowell@...; Yocto (yocto@...)
<yocto@...>
Subject: Re: [yocto] Reducing the perl footprint on my image

On Tue, Mar 2, 2021 at 10:01 AM Steve Sakoman via
lists.yoctoproject.org <steve=sakoman.com@...>
wrote:

On Tue, Mar 2, 2021 at 6:26 AM <rustyhowell@...> wrote:

I have an image that is using debian package management
(PACKAGE_CLASSES = "package_deb"). Because apt and dpkg require perl,
perl is being installed in the image. No problem. Except that the entire perl
stack is 669 packages.

I just took a look at the manifest for one of my images that includes
PACKAGE_CLASSES = "package_deb". I see the perl package plus 43
perl-module packages. Are you sure that something else in your images
isn't pulling in all of those other perl-module packages?
It just occurred to me to make sure you are looking in the image
manifest to see which packages are actually installed in your image.
The perl recipe does generate 676 packages (in dunfell) so perhaps you
might be looking at the generated packages rather than the installed
packages??
I encountered a similar problem with package management enabled and rpm as package format, where I also just install rpm for package management and not all dnf stack. I get quite a lot of perl and python packages into the image which are pulled by the rpm package, but they are only needed for things like rpm-build, rpm-sign, etc., not for the bare rpm command, which is the only one I need in the image.

So I locally extended the rpm recipe to split those tools into rpm-build, rpm-sign and rpm-archive and skip those packages in the image. I should probably send patches for that to oe-core. Is that something that could be accepted?

--
Diego Santa Cruz, PhD
Technology Architect
spinetix.com


Richard Purdie
 

On Tue, 2021-03-02 at 20:42 +0000, Diego Santa Cruz via lists.yoctoproject.org wrote:
-----Original Message-----
From: yocto@... <yocto@...> On
Behalf Of Steve Sakoman via lists.yoctoproject.org
Sent: 02 March 2021 21:06
To: Steve Sakoman <steve@...>
Cc: rustyhowell@...; Yocto (yocto@...)
<yocto@...>
Subject: Re: [yocto] Reducing the perl footprint on my image

On Tue, Mar 2, 2021 at 10:01 AM Steve Sakoman via
lists.yoctoproject.org <steve=sakoman.com@...>
wrote:

On Tue, Mar 2, 2021 at 6:26 AM <rustyhowell@...> wrote:

I have an image that is using debian package management
(PACKAGE_CLASSES = "package_deb"). Because apt and dpkg require perl,
perl is being installed in the image. No problem. Except that the entire perl
stack is 669 packages.

I just took a look at the manifest for one of my images that includes
PACKAGE_CLASSES = "package_deb". I see the perl package plus 43
perl-module packages. Are you sure that something else in your images
isn't pulling in all of those other perl-module packages?
It just occurred to me to make sure you are looking in the image
manifest to see which packages are actually installed in your image.
The perl recipe does generate 676 packages (in dunfell) so perhaps you
might be looking at the generated packages rather than the installed
packages??
I encountered a similar problem with package management enabled and rpm 
as package format, where I also just install rpm for package management 
and not all dnf stack. I get quite a lot of perl and python packages 
into the image which are pulled by the rpm package, but they are only
needed for things like rpm-build, rpm-sign, etc., not for the bare 
rpm command, which is the only one I need in the image.

So I locally extended the rpm recipe to split those tools into rpm-build, 
rpm-sign and rpm-archive and skip those packages in the image. I should 
probably send patches for that to oe-core. Is that something that could
be accepted?
Not sure they need to go to separate packages but moving those 
three to some kind of "build" package would make a lot of sense
to me at least.

Cheers,

Richard


Rusty Howell
 

Steve, you're right.  My image.bb is adding various lmsensors packages to the IMAGE_INSTALL, some of which depend on perl-modules.   perl-modules seems to be the big monster, not perl.  We need lmsensors.  So I guess my question is really, how can I (or can I at all) reduce the footprint of perl-modules?  perl-modules doesn't have any Depends, rather it has a long list (~ 670) of Recommends. 


On Tue, Mar 2, 2021 at 1:49 PM Richard Purdie <richard.purdie@...> wrote:
On Tue, 2021-03-02 at 20:42 +0000, Diego Santa Cruz via lists.yoctoproject.org wrote:
> > -----Original Message-----
> > From: yocto@... <yocto@...> On
> > Behalf Of Steve Sakoman via lists.yoctoproject.org
> > Sent: 02 March 2021 21:06
> > To: Steve Sakoman <steve@...>
> > Cc: rustyhowell@...; Yocto (yocto@...)
> > <yocto@...>
> > Subject: Re: [yocto] Reducing the perl footprint on my image
> >
> > On Tue, Mar 2, 2021 at 10:01 AM Steve Sakoman via
> > lists.yoctoproject.org <steve=sakoman.com@...>
> > wrote:
> > >
> > > On Tue, Mar 2, 2021 at 6:26 AM <rustyhowell@...> wrote:
> > > >
> > > > I have an image that is using debian package management
> > (PACKAGE_CLASSES = "package_deb").  Because apt and dpkg require perl,
> > perl is being installed in the image.   No problem.  Except that the entire perl
> > stack is 669 packages.
> > >
> > > I just took a look at the manifest for one of my images that includes
> > > PACKAGE_CLASSES = "package_deb".  I see the perl package plus 43
> > > perl-module packages.  Are you sure that something else in your images
> > > isn't pulling in all of those other perl-module packages?
> >
> > It just occurred to me to make sure you are looking in the image
> > manifest to see which packages are actually installed in your image.
> > The perl recipe does generate 676 packages (in dunfell) so perhaps you
> > might be looking at the generated packages rather than the installed
> > packages??
> >
>
> I encountered a similar problem with package management enabled and rpm 
> as package format, where I also just install rpm for package management 
> and not all dnf stack. I get quite a lot of perl and python packages 
> into the image which are pulled by the rpm package, but they are only
> needed for things like rpm-build, rpm-sign, etc., not for the bare 
> rpm command, which is the only one I need in the image.
>
> So I locally extended the rpm recipe to split those tools into rpm-build, 
> rpm-sign and rpm-archive and skip those packages in the image. I should 
> probably send patches for that to oe-core. Is that something that could
> be accepted?

Not sure they need to go to separate packages but moving those 
three to some kind of "build" package would make a lot of sense
to me at least.

Cheers,

Richard


Diego Santa Cruz
 

-----Original Message-----
From: Richard Purdie <richard.purdie@...>
Sent: 02 March 2021 21:50
To: Diego Santa Cruz <Diego.SantaCruz@...>;
steve@...
Cc: rustyhowell@...; Yocto (yocto@...)
<yocto@...>
Subject: Re: [yocto] Reducing the perl footprint on my image
[...]
So I locally extended the rpm recipe to split those tools into rpm-build,
rpm-sign and rpm-archive and skip those packages in the image. I should
probably send patches for that to oe-core. Is that something that could
be accepted?
Not sure they need to go to separate packages but moving those
three to some kind of "build" package would make a lot of sense
to me at least.
I'll be sending a patch to split it then. Having the sign and archive in separate packages allows to skip their dependencies in images too, and they are not really related to the rpm build part and the complexity delta is minimal; usually there is no need to have rpm-sign in a target image nor rpm2archive if the archive PACKAGECONFIG is enabled.

Best,
Diego

--
Diego Santa Cruz, PhD
Technology Architect
spinetix.com


Tim Orling
 



On Wed, Mar 3, 2021 at 7:06 AM Rusty Howell <rustyhowell@...> wrote:
Steve, you're right.  My image.bb is adding various lmsensors packages to the IMAGE_INSTALL, some of which depend on perl-modules.   perl-modules seems to be the big monster, not perl.  We need lmsensors.  So I guess my question is really, how can I (or can I at all) reduce the footprint of perl-modules?  perl-modules doesn't have any Depends, rather it has a long list (~ 670) of Recommends. 


If you look at the lmsensors recipe, the lmsensors-sensorsdetect [1] and lmsensors-sensorsconfconvert [2] subpackages both have perl-modules in RDEPENDS.
One way to determine the ACTUAL dependencies is to run the perl.req script from rpm [3].

$ perl.req sensors-detect
perl >= 0:5.004
perl(Fcntl)
perl(File::Basename)
perl(constant)
perl(strict)
perl(vars)

$ perl.req sensors-conf-convert
perl(strict)
perl(vars)

You then need to translate those into the perl-modules-* subpackages, which is easiest to see in perl-rdepends.txt [4].

I have not built nor tested this, but using the described approach I have a patch for lmsensors [5].

Please cherry-pick, test and if it works, add your signed-off-by and submit to the mailing list.


On Tue, Mar 2, 2021 at 1:49 PM Richard Purdie <richard.purdie@...> wrote:
On Tue, 2021-03-02 at 20:42 +0000, Diego Santa Cruz via lists.yoctoproject.org wrote:
> > -----Original Message-----
> > From: yocto@... <yocto@...> On
> > Behalf Of Steve Sakoman via lists.yoctoproject.org
> > Sent: 02 March 2021 21:06
> > To: Steve Sakoman <steve@...>
> > Cc: rustyhowell@...; Yocto (yocto@...)
> > <yocto@...>
> > Subject: Re: [yocto] Reducing the perl footprint on my image
> >
> > On Tue, Mar 2, 2021 at 10:01 AM Steve Sakoman via
> > lists.yoctoproject.org <steve=sakoman.com@...>
> > wrote:
> > >
> > > On Tue, Mar 2, 2021 at 6:26 AM <rustyhowell@...> wrote:
> > > >
> > > > I have an image that is using debian package management
> > (PACKAGE_CLASSES = "package_deb").  Because apt and dpkg require perl,
> > perl is being installed in the image.   No problem.  Except that the entire perl
> > stack is 669 packages.
> > >
> > > I just took a look at the manifest for one of my images that includes
> > > PACKAGE_CLASSES = "package_deb".  I see the perl package plus 43
> > > perl-module packages.  Are you sure that something else in your images
> > > isn't pulling in all of those other perl-module packages?
> >
> > It just occurred to me to make sure you are looking in the image
> > manifest to see which packages are actually installed in your image.
> > The perl recipe does generate 676 packages (in dunfell) so perhaps you
> > might be looking at the generated packages rather than the installed
> > packages??
> >
>
> I encountered a similar problem with package management enabled and rpm 
> as package format, where I also just install rpm for package management 
> and not all dnf stack. I get quite a lot of perl and python packages 
> into the image which are pulled by the rpm package, but they are only
> needed for things like rpm-build, rpm-sign, etc., not for the bare 
> rpm command, which is the only one I need in the image.
>
> So I locally extended the rpm recipe to split those tools into rpm-build, 
> rpm-sign and rpm-archive and skip those packages in the image. I should 
> probably send patches for that to oe-core. Is that something that could
> be accepted?

Not sure they need to go to separate packages but moving those 
three to some kind of "build" package would make a lot of sense
to me at least.

Cheers,

Richard





Tim Orling
 



On Wed, Mar 3, 2021 at 12:28 PM Tim Orling via lists.yoctoproject.org <ticotimo=gmail.com@...> wrote:


On Wed, Mar 3, 2021 at 7:06 AM Rusty Howell <rustyhowell@...> wrote:
Steve, you're right.  My image.bb is adding various lmsensors packages to the IMAGE_INSTALL, some of which depend on perl-modules.   perl-modules seems to be the big monster, not perl.  We need lmsensors.  So I guess my question is really, how can I (or can I at all) reduce the footprint of perl-modules?  perl-modules doesn't have any Depends, rather it has a long list (~ 670) of Recommends. 


If you look at the lmsensors recipe, the lmsensors-sensorsdetect [1] and lmsensors-sensorsconfconvert [2] subpackages both have perl-modules in RDEPENDS.
One way to determine the ACTUAL dependencies is to run the perl.req script from rpm [3].

Related bug [6]

$ perl.req sensors-detect
perl >= 0:5.004
perl(Fcntl)
perl(File::Basename)
perl(constant)
perl(strict)
perl(vars)

$ perl.req sensors-conf-convert
perl(strict)
perl(vars)

You then need to translate those into the perl-modules-* subpackages, which is easiest to see in perl-rdepends.txt [4].

I have not built nor tested this, but using the described approach I have a patch for lmsensors [5].

Please cherry-pick, test and if it works, add your signed-off-by and submit to the mailing list.


On Tue, Mar 2, 2021 at 1:49 PM Richard Purdie <richard.purdie@...> wrote:
On Tue, 2021-03-02 at 20:42 +0000, Diego Santa Cruz via lists.yoctoproject.org wrote:
> > -----Original Message-----
> > From: yocto@... <yocto@...> On
> > Behalf Of Steve Sakoman via lists.yoctoproject.org
> > Sent: 02 March 2021 21:06
> > To: Steve Sakoman <steve@...>
> > Cc: rustyhowell@...; Yocto (yocto@...)
> > <yocto@...>
> > Subject: Re: [yocto] Reducing the perl footprint on my image
> >
> > On Tue, Mar 2, 2021 at 10:01 AM Steve Sakoman via
> > lists.yoctoproject.org <steve=sakoman.com@...>
> > wrote:
> > >
> > > On Tue, Mar 2, 2021 at 6:26 AM <rustyhowell@...> wrote:
> > > >
> > > > I have an image that is using debian package management
> > (PACKAGE_CLASSES = "package_deb").  Because apt and dpkg require perl,
> > perl is being installed in the image.   No problem.  Except that the entire perl
> > stack is 669 packages.
> > >
> > > I just took a look at the manifest for one of my images that includes
> > > PACKAGE_CLASSES = "package_deb".  I see the perl package plus 43
> > > perl-module packages.  Are you sure that something else in your images
> > > isn't pulling in all of those other perl-module packages?
> >
> > It just occurred to me to make sure you are looking in the image
> > manifest to see which packages are actually installed in your image.
> > The perl recipe does generate 676 packages (in dunfell) so perhaps you
> > might be looking at the generated packages rather than the installed
> > packages??
> >
>
> I encountered a similar problem with package management enabled and rpm 
> as package format, where I also just install rpm for package management 
> and not all dnf stack. I get quite a lot of perl and python packages 
> into the image which are pulled by the rpm package, but they are only
> needed for things like rpm-build, rpm-sign, etc., not for the bare 
> rpm command, which is the only one I need in the image.
>
> So I locally extended the rpm recipe to split those tools into rpm-build, 
> rpm-sign and rpm-archive and skip those packages in the image. I should 
> probably send patches for that to oe-core. Is that something that could
> be accepted?

Not sure they need to go to separate packages but moving those 
three to some kind of "build" package would make a lot of sense
to me at least.

Cheers,

Richard