Fw: Reducing rootfs size in yocto


Ajam Ali
 




 
Hi All,

I am trying to reduce the image size.

how can i  reduce the size of following packages so that i could get a small image size after integrating these packages in yocto?


1) boost-dev 2) docker 3) containerd-opencontainers 4) nodejs 5) nodejs-npm 6) python3-numpy 7) cpp 8) postgresql



Thanks in advance,

Ajam Ali


::DISCLAIMER::

The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects.


Konrad Weihmann
 

Pretty general questions, but I try to answer your question.


I don't know what the purpose of your image is, but installing development tools like boost-dev or gcc (I guess that is what you mean with cpp) makes your image automatically a lot bigger, I would suggest to put devtools into a SDK, which could be shipped separately.


Next thing you can do is to analyze the what is actually so big in your image, e.g. by creating a tar.gz of the image, unpacking it and browsing by 'du -sh *'.
You can get the package/recipe to a file by using 'oe-pkgdata-util find-path <path>'.


In the identified recipe you can look for PACKAGECONFIG or OE_EXTRACONF switches that you don't need and disable them.


As said, I don't know what the purpose/environment of your image will be like, but you could also consider to use a different fs-format like squash.


In general if you want small images preferring musl over glibc and systemV over systemd is something that you could consider as well.


TL;DR


1) move to SDK

2) check PACKAGECONFIG switches

3) same as 2 or consider something "smaller" like lxc

4) same as 2

5) same as 2

6) same as 2

7) move to SDK

8) same as 2


On 28.03.20 14:25, Ajam Ali wrote:



 
Hi All,

I am trying to reduce the image size.

how can i  reduce the size of following packages so that i could get a small image size after integrating these packages in yocto?


1) boost-dev 2) docker 3) containerd-opencontainers 4) nodejs 5) nodejs-npm 6) python3-numpy 7) cpp 8) postgresql



Thanks in advance,

Ajam Ali


::DISCLAIMER::

The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects.



    


Ajam Ali
 

Hi Konrad,

Actually my current image size is 3.9GB because of heavy size packages required by my project and without project required packages my image size in Yocto is 800MB.

I want to reduce the image size as maximum as possible.


Thanks a lot in advance,
Ajam Ali


From: Konrad Weihmann <kweihmann@...>
Sent: Saturday, March 28, 2020 7:38 PM
To: Ajam Ali <AjamA@...>
Cc: yocto@... <yocto@...>
Subject: Re: [yocto] Fw: Reducing rootfs size in yocto
 

[CAUTION: This Email is from outside the Organization. Do not click links or open attachments unless you trust the sender.]

Pretty general questions, but I try to answer your question.


I don't know what the purpose of your image is, but installing development tools like boost-dev or gcc (I guess that is what you mean with cpp) makes your image automatically a lot bigger, I would suggest to put devtools into a SDK, which could be shipped separately.


Next thing you can do is to analyze the what is actually so big in your image, e.g. by creating a tar.gz of the image, unpacking it and browsing by 'du -sh *'.
You can get the package/recipe to a file by using 'oe-pkgdata-util find-path <path>'.


In the identified recipe you can look for PACKAGECONFIG or OE_EXTRACONF switches that you don't need and disable them.


As said, I don't know what the purpose/environment of your image will be like, but you could also consider to use a different fs-format like squash.


In general if you want small images preferring musl over glibc and systemV over systemd is something that you could consider as well.


TL;DR


1) move to SDK

2) check PACKAGECONFIG switches

3) same as 2 or consider something "smaller" like lxc

4) same as 2

5) same as 2

6) same as 2

7) move to SDK

8) same as 2


On 28.03.20 14:25, Ajam Ali wrote:



 
Hi All,

I am trying to reduce the image size.

how can i  reduce the size of following packages so that i could get a small image size after integrating these packages in yocto?


1) boost-dev 2) docker 3) containerd-opencontainers 4) nodejs 5) nodejs-npm 6) python3-numpy 7) cpp 8) postgresql



Thanks in advance,

Ajam Ali


::DISCLAIMER::

The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects.




Konrad Weihmann
 

HI,


800MB sounds fairly much for just the base system - maybe there is some potential for you to slim it down.
If your project requires all these packages, I' afraid there isn't an easy magical switch to reduce the size.

What you could do is take the image manifest (https://www.yoctoproject.org/docs/current/mega-manual/mega-manual.html#var-IMAGE_MANIFEST) and browse manually though it, if there

is any obvious package that shouldn't be installed, which could be removed - but to be honest 3.9GB for all that stuff required according to your list sounds like a reasonable size for the rootfs.

If you already know what the real "big ones" are in your image you could try to configure them to include only things really needed - if not you could also take the

image manifest feed the information into "oe-pkgdata-util package-info <pkgname>" to get the size of the package.

I guess from here on you can take it.


In the end 3.9GB for a fully fledged development environment doesn't sounds too much - just remember a vanilla ubuntu for example takes at least 15GB.


BR

Konrad


On 28.03.20 19:36, Ajam Ali wrote:
Hi Konrad,

Actually my current image size is 3.9GB because of heavy size packages required by my project and without project required packages my image size in Yocto is 800MB.

I want to reduce the image size as maximum as possible.


Thanks a lot in advance,
Ajam Ali

From: Konrad Weihmann <kweihmann@...>
Sent: Saturday, March 28, 2020 7:38 PM
To: Ajam Ali <AjamA@...>
Cc: yocto@... <yocto@...>
Subject: Re: [yocto] Fw: Reducing rootfs size in yocto
 

[CAUTION: This Email is from outside the Organization. Do not click links or open attachments unless you trust the sender.]

Pretty general questions, but I try to answer your question.


I don't know what the purpose of your image is, but installing development tools like boost-dev or gcc (I guess that is what you mean with cpp) makes your image automatically a lot bigger, I would suggest to put devtools into a SDK, which could be shipped separately.


Next thing you can do is to analyze the what is actually so big in your image, e.g. by creating a tar.gz of the image, unpacking it and browsing by 'du -sh *'.
You can get the package/recipe to a file by using 'oe-pkgdata-util find-path <path>'.


In the identified recipe you can look for PACKAGECONFIG or OE_EXTRACONF switches that you don't need and disable them.


As said, I don't know what the purpose/environment of your image will be like, but you could also consider to use a different fs-format like squash.


In general if you want small images preferring musl over glibc and systemV over systemd is something that you could consider as well.


TL;DR


1) move to SDK

2) check PACKAGECONFIG switches

3) same as 2 or consider something "smaller" like lxc

4) same as 2

5) same as 2

6) same as 2

7) move to SDK

8) same as 2


On 28.03.20 14:25, Ajam Ali wrote:



 
Hi All,

I am trying to reduce the image size.

how can i  reduce the size of following packages so that i could get a small image size after integrating these packages in yocto?


1) boost-dev 2) docker 3) containerd-opencontainers 4) nodejs 5) nodejs-npm 6) python3-numpy 7) cpp 8) postgresql



Thanks in advance,

Ajam Ali


::DISCLAIMER::

The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects.



          


Oliver Westermann
 

Do you need the -dev version of boost on the device?


Ajam Ali
 

Hi,

-dev version of boost is a supporting package which is getting installed by default
not by me using local.conf.

Thanks,
Ajam Ali


From: yocto@... <yocto@...> on behalf of Oliver Westermann via Lists.Yoctoproject.Org <oliver.westermann=cognex.com@...>
Sent: Sunday, March 29, 2020 8:16 PM
To: yocto@... <yocto@...>
Cc: yocto@... <yocto@...>
Subject: Re: [yocto] Fw: Reducing rootfs size in yocto
 

[CAUTION: This Email is from outside the Organization. Do not click links or open attachments unless you trust the sender.]

Do you need the -dev version of boost on the device?
::DISCLAIMER::

The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects.


Mikko Rapeli
 

Hi,

Generic approaches for reducing yocto target image size where buildhistory
is an important source of information:

* review and minimize distro features, enable only what you need
https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#ref-features-distro

* review and minimize image recipe, install only those image features and packages
which you need
https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#ref-features-image

* review the needed binary packages: traverse the tree of dependencies what you need,
review every single recipe and change PACKAGECONFIG or build flags to remove
features which you do not need. This it may be possible to remove large set
of dependencies from images, e.g. language bindings, debug tools. In some cases
one just needs a single binary executable or shared library but it is bundled
in a larger binary package with more complex dependencies. For these cases
add a bbappend to custom layers to change the packaging to only include what
you need.
https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#var-PACKAGECONFIG

* change compiler flags from default O2 to Os optimization. This can work for
some recipes which are not prerformance critical or to the whole build. Note
that many SW components break this in their own build scripts by adding
O2 or even O3 by default. Check from buildhistory that binary sizes go smaller
after switch from O2 to Os and review every important recipe which did not.
You may be able to save up to 15% in rootfs size this way, but all depends
on the details of SW components and image contents.
see poky/meta/conf/bitbake.conf

I've followed this for multiple products and found out that poky is nice to work with
but BSP layers add or depend on all kinds of extra things that products do not need.
Thus I have ended by BBMASK'ing away large parts of BSP layer recipes in distro
configs. This with multiple x86 and ARM BSP layers from various SoC vendors.

Also, RTFM :)

https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#building-a-tiny-system

Hope this helps,

-Mikko


Randy MacLeod
 

On 2020-03-30 2:57 a.m., Mikko Rapeli wrote:
Hi,
Generic approaches for reducing yocto target image size where buildhistory
is an important source of information:
* review and minimize distro features, enable only what you need
https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#ref-features-distro
* review and minimize image recipe, install only those image features and packages
which you need
https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#ref-features-image
* review the needed binary packages: traverse the tree of dependencies what you need,
review every single recipe and change PACKAGECONFIG or build flags to remove
features which you do not need. This it may be possible to remove large set
of dependencies from images, e.g. language bindings, debug tools. In some cases
one just needs a single binary executable or shared library but it is bundled
in a larger binary package with more complex dependencies. For these cases
add a bbappend to custom layers to change the packaging to only include what
you need.
https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#var-PACKAGECONFIG
* change compiler flags from default O2 to Os optimization. This can work for
some recipes which are not prerformance critical or to the whole build. Note
that many SW components break this in their own build scripts by adding
O2 or even O3 by default. Check from buildhistory that binary sizes go smaller
after switch from O2 to Os and review every important recipe which did not.
You may be able to save up to 15% in rootfs size this way, but all depends
on the details of SW components and image contents.
see poky/meta/conf/bitbake.conf
I've followed this for multiple products and found out that poky is nice to work with
but BSP layers add or depend on all kinds of extra things that products do not need.
Thus I have ended by BBMASK'ing away large parts of BSP layer recipes in distro
configs. This with multiple x86 and ARM BSP layers from various SoC vendors.
I've also seen this problem and was frustrated by it.

The BSP vendors/developers want to test the full feature set of the
hardware but they often don't bother stripping out packages after
confirming that everything works. This results in bloat that can
be a problem for some users.

Using BBMASK is a good coping mechanism! :)

It would also be good if BSP developers split the BSP into the
essential elements and extras. Where to draw the line between
two such categories can tricky. One could also have finer-grained
packagegroups but splitting things into essential and extras is
a good start.

In WR Linux, we have a setup tool that let's you add 'features'
This comes from a layer that improves ease of use for some people:

https://github.com/WindRiver-Labs/wr-template/blob/WRLINUX_10_19_BASE/README

and is always used in the WR Linux distro.

And as can be seen here:
https://github.com/WindRiver-Labs/wrlinux/tree/WRLINUX_10_19_BASE/wrlinux-distro/templates/feature/bsp-extras

There is a generic BSP independent feature that you can add to
a setup:

feature/bsp-extras

Adds some useful tools defined by the BSP_EXTRAS_PACKAGES option
in the BSP layer.

To see the list of available tools, run the bitbake -e command in
a sourced terminal, and look for the BSP_EXTRAS_PACKAGES.

Maybe this description will help get BSP developers to agree on
doing something similar regardless of the 'setup/wr-template' layer.

Good luck and keep your BSPs minimal!

../Randy

Also, RTFM :)
https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#building-a-tiny-system
Hope this helps,
-Mikko

--
# Randy MacLeod
# Wind River Linux