yocto@lists.yoctoproject.org | On managing debug and production builds

Home Messages Hashtags Subgroups

Date Date 1 - 9 of 9

On managing debug and production builds

Vinayak Menon #56329 I have a requirement to create production and release builds for the same machine. Debug build would use most of the production recipes, except few images like Linux kernel that will have debug configs enabled and a separate debug image is created. Also debug images will have few extra debug packages and image features like debug-tweaks enabled. What is the standard way of managing this ? One other requirement is that for e.g. I do not want both debug and production linux images to be created in one build. That increases the build time. i.e. just adding debug variant recipes alone does not suffice. Also the debug and production variants are not specific to a machine, i.e. tomorrow if I add another machine, I should be able to generate both variants for that machine too without any machine conf changes. What I have tried right now is to create 2 local.conf, one in meta/conf and other in meta/conf/debug. The debug conf has an IMAGE_FEATURE indicating that it is a debug variant. I am using the debug-tweaks image feature as of now. And then I switch the confs with TEMPLATECONF. Now, for recipes, I have not created separate debug recipes. For e.g. in the linux recipe I am just looking for the debug-tweaks feature to pick the right config fragments. Thanks, Vinayak
More All Messages By This Member
tomzy #56332 Hi Vinayak I believe that the best way to prepare debug and prod images is to create include file (.inc) with base set of packages and features and then add two .bb recipes, one for prod and one for debug - every of this recipes will need to include the `.inc` file. You could name them `base-image-prod.bb` and `base-image-debug.bb`. This way you could run either prod or debug builds by running `bitbake base-image-debug` or `bitbake base-image-prod`. As for the machine, this would need to be set in local.conf every time you want to switch the machine.To simplify this process, for the management of the configuration of the entire build (including the selection of debug / prod image or machine configuration), I recommend the kas[1] project with the very helpful kas-container tool. [1] https://github.com/siemens/kas Regards, Tomasz Żyjewski Embedded Systems Engineer GPG: 5C495EA3EBEECA59 https://3mdeb.com \| @3mdeb_com
More All Messages By This Member
Vinayak Menon #56333 On Tue, Mar 1, 2022 at 12:34 PM tomzy <tomasz.zyjewski@...> wrote: Hi Vinayak I believe that the best way to prepare debug and prod images is to create include file (.inc) with base set of packages and features and then add two .bb recipes, one for prod and one for debug - every of this recipes will need to include the `.inc` file. You could name them `base-image-prod.bb` and `base-image-debug.bb`. This way you could run either prod or debug builds by running `bitbake base-image-debug` or `bitbake base-image-prod`. As for the machine, this would need to be set in local.conf every time you want to switch the machine.To simplify this process, for the management of the configuration of the entire build (including the selection of debug / prod image or machine configuration), I recommend the kas[1] project with the very helpful kas-container tool. Thanks Tomasz. I will check kas. Yes, for selecting some of the packages I have created prod and debug image recipes. But this did not work for the kernel as the kernel recipe is picked as part of PROVIDER in machine conf. SoI had to use 2 conf to have the IMAGE_FEATURES (or any other var) set differently for prod and debug. This is for building the kernel recipie differently for prod and debug. Setting the IMAGE_FEATURES in the image recipe (and not in conf) causes 2 problems. One is that kernel and other bootloaders recipes are picked early via PROVIDER in conf and not as packages included in image recipe. Secondly, setting the var in the image recipe breaks this command for e.g. "bitbake base-image-prod.bb base-image-debug.bb". Since the command parses the recipes only once for both image creation. [1] https://github.com/siemens/kas Regards, Tomasz Żyjewski Embedded Systems Engineer GPG: 5C495EA3EBEECA59 https://3mdeb.com \| @3mdeb_com -- vinayak
More All Messages By This Member
Alexander Kanavin #56334 If you can't do what you need with only image recipes, then the only option is to create an additional DISTRO config I'm afraid. Putting these tweaks into machine config or local.conf is not correct. Alex toggle quoted message Show quoted text On Tue, 1 Mar 2022 at 09:52, Vinayak Menon <menon.vinayak@...> wrote: On Tue, Mar 1, 2022 at 12:34 PM tomzy <tomasz.zyjewski@...> wrote: Hi Vinayak I believe that the best way to prepare debug and prod images is to create include file (.inc) with base set of packages and features and then add two .bb recipes, one for prod and one for debug - every of this recipes will need to include the `.inc` file. You could name them `base-image-prod.bb` and `base-image-debug.bb`. This way you could run either prod or debug builds by running `bitbake base-image-debug` or `bitbake base-image-prod`. As for the machine, this would need to be set in local.conf every time you want to switch the machine.To simplify this process, for the management of the configuration of the entire build (including the selection of debug / prod image or machine configuration), I recommend the kas[1] project with the very helpful kas-container tool. Thanks Tomasz. I will check kas. Yes, for selecting some of the packages I have created prod and debug image recipes. But this did not work for the kernel as the kernel recipe is picked as part of PROVIDER in machine conf. SoI had to use 2 conf to have the IMAGE_FEATURES (or any other var) set differently for prod and debug. This is for building the kernel recipie differently for prod and debug. Setting the IMAGE_FEATURES in the image recipe (and not in conf) causes 2 problems. One is that kernel and other bootloaders recipes are picked early via PROVIDER in conf and not as packages included in image recipe. Secondly, setting the var in the image recipe breaks this command for e.g. "bitbake base-image-prod.bb base-image-debug.bb". Since the command parses the recipes only once for both image creation. [1] https://github.com/siemens/kas Regards, Tomasz Żyjewski Embedded Systems Engineer GPG: 5C495EA3EBEECA59 https://3mdeb.com \| @3mdeb_com -- vinayak
More All Messages By This Member
tomzy #56335 Thanks Tomasz. I will check kas. No problem Yes, for selecting some of the packages I have created prod and debug image recipes.But this did not work for the kernel as the kernel recipe is picked as part of PROVIDERin machine conf. What are the difference there? You want to use different config on prod and debug images? Maybe add it as config fragments? Then you would need to add some global variable to distinguish when use given .cfg file. [1] https://docs.yoctoproject.org/singleindex.html#creating-configuration-fragments SoI had to use 2 conf to have the IMAGE_FEATURES (orany other var)set differently for prod and debug. This is for building the kernelrecipie differently for prodand debug. Setting the IMAGE_FEATURES in the image recipe (and not inconf) causes2 problems. One is that kernel and other bootloaders recipes arepicked early via PROVIDER in conf and not as packages included in image recipe. Is that a problem? Secondly,setting the var in the image recipe breaks this command for e.g. "bitbake base-image-prod.bbbase-image-debug.bb". Didn't you want to distinguish this to builds to be able to run `bitbake base-image-prod` or `bitbake base-image-debug`? Since the command parses the recipes only once for both image creation. Nevertheless I would greatly recommend you to use kas. In simple .yml file you could prepare different `local.conf` per configuration prod/debug. [2] https://kas.readthedocs.io/en/latest/userguide.html#project-configuration Regards, Tomasz Żyjewski Embedded Systems Engineer GPG: 5C495EA3EBEECA59 https://3mdeb.com \| @3mdeb_com
More All Messages By This Member
Karthik Poduval #56346 This concept of user vs debug builds is there in Android and any Android developer who gets introduced to yocto may look for these options. There seem to be many implementation choices here, trying to summarize some of the options here. 1. use different image recipes example-image-user.bb vs example-image-debug.bb but this still poses a problem for kernel recipes as kernel needs to have different config fragments or different defconfig for debug vs user variants. One possible option is to use KTYPE to select tiny vs standard or define own for custom BSP layers. What about other recipes like u-boot or firmware for other remote processors, how to percolate the debug vs user options to those recipes via just an image recipe ? 2. use different configs using multiconfig. Base config (which is debug) also selects the user config hence always building both build variants in different tmp directories. Kernel, remote firmware and image recipes use variables from the multiconfig cof files to decide whether to build debug or user variants. 3. use different distros. DSITRO=user bitbake example-image, DISTRO=debug example-image. Kernel and remote firmware recipes use variables from the distro to decide whether to build debug or user variants. Which is the best method ? All these options seem very BSP layer specific, is there something more generic and better than the above options ? If not, should this be a feature request to the Yocto project ? NOTE: debug variants may include the following. - more debug related kernel configs and security loosened and UART ports disabled - image recipe debug variant may include debug utilities absent from the user variant There could be more than just debug and user variants. Android has engineering, tests, user, tiny and user-debug. -- Regards, Karthik Poduval toggle quoted message Show quoted text On Tue, Mar 1, 2022 at 1:26 AM tomzy <tomasz.zyjewski@...> wrote: Thanks Tomasz. I will check kas. No problem Yes, for selecting some of the packages I have created prod and debug image recipes.But this did not work for the kernel as the kernel recipe is picked as part of PROVIDERin machine conf. What are the difference there? You want to use different config on prod and debug images? Maybe add it as config fragments? Then you would need to add some global variable to distinguish when use given .cfg file. [1] https://docs.yoctoproject.org/singleindex.html#creating-configuration-fragments SoI had to use 2 conf to have the IMAGE_FEATURES (orany other var)set differently for prod and debug. This is for building the kernelrecipie differently for prodand debug. Setting the IMAGE_FEATURES in the image recipe (and not inconf) causes2 problems. One is that kernel and other bootloaders recipes arepicked early via PROVIDER in conf and not as packages included in image recipe. Is that a problem? Secondly,setting the var in the image recipe breaks this command for e.g. "bitbake base-image-prod.bbbase-image-debug.bb". Didn't you want to distinguish this to builds to be able to run `bitbake base-image-prod` or `bitbake base-image-debug`? Since the command parses the recipes only once for both image creation. Nevertheless I would greatly recommend you to use kas. In simple .yml file you could prepare different `local.conf` per configuration prod/debug. [2] https://kas.readthedocs.io/en/latest/userguide.html#project-configuration Regards, Tomasz Żyjewski Embedded Systems Engineer GPG: 5C495EA3EBEECA59 https://3mdeb.com \| @3mdeb_com
More All Messages By This Member
Vinayak Menon #56347 Add list. missed earlier. toggle quoted message Show quoted text On Wed, Mar 2, 2022 at 8:11 AM Vinayak Menon <menon.vinayak@...> wrote: On Tue, Mar 1, 2022 at 2:56 PM tomzy <tomasz.zyjewski@...> wrote: Thanks Tomasz. I will check kas. No problem Yes, for selecting some of the packages I have created prod and debug image recipes.But this did not work for the kernel as the kernel recipe is picked as part of PROVIDERin machine conf. What are the difference there? You want to use different config on prod and debug images? Maybe add it as config fragments? Then you would need to add some global variable to distinguish when use given .cfg file. Ya, and I am using exisitng IMAGE_FEATURE instead of the global var. I can use global var too. The only thing is that I am setting the global var from a conf and not image recipe. [1] https://docs.yoctoproject.org/singleindex.html#creating-configuration-fragments SoI had to use 2 conf to have the IMAGE_FEATURES (orany other var)set differently for prod and debug. This is for building the kernelrecipie differently for prodand debug. Setting the IMAGE_FEATURES in the image recipe (and not inconf) causes2 problems. One is that kernel and other bootloaders recipes are picked early via PROVIDER in conf and not as packages included in image recipe. Is that a problem? Ya you are right, that is not a problem as such. But I have seen in the yocto documentation that IMAGE_FEATURES are recommended to be set from conf. Is there any reason for such a recommendation ? Secondly,setting the var in the image recipe breaks this command for e.g. "bitbake base-image-prod.bbbase-image-debug.bb". Didn't you want to distinguish this to builds to be able to run `bitbake base-image-prod` or `bitbake base-image-debug`? Yes I want to distinguish, and as a developer I can give separate bitbake commands. But the thought was that later the user can issue commands the way they want. And if they use it in combined manner described above, expecting 2 images with different kernels, it does not work. Since the command parses the recipes only once for both image creation. Nevertheless I would greatly recommend you to use kas. In simple .yml file you could prepare different `local.conf` per configuration prod/debug. Sure, I will explore this. [2] https://kas.readthedocs.io/en/latest/userguide.html#project-configuration Regards, Tomasz Żyjewski Embedded Systems Engineer GPG: 5C495EA3EBEECA59 https://3mdeb.com \| @3mdeb_com -- vinayak -- vinayak
More All Messages By This Member
Josef Holzmayr #56348 Howdy! Karthik Poduval <karthik.poduval@...> schrieb am Mi. 2. März 2022 um 04:23: This concept of user vs debug builds is there in Android and any Android developer who gets introduced to yocto may look for these options. There seem to be many implementation choices here, trying to summarize some of the options here. 1. use different image recipes example-image-user.bb vs example-image-debug.bb but this still poses a problem for kernel recipes as kernel needs to have different config fragments or different defconfig for debug vs user variants. One possible option is to use KTYPE to select tiny vs standard or define own for custom BSP layers. What about other recipes like u-boot or firmware for other remote processors, how to percolate the debug vs user options to those recipes via just an image recipe ? As a recipe cannot affect another recipe, and image recipes are obviously recipes too, this is usually not useful. 2. use different configs using multiconfig. Base config (which is debug) also selects the user config hence always building both build variants in different tmp directories. Kernel, remote firmware and image recipes use variables from the multiconfig cof files to decide whether to build debug or user variants. The whole point of multiconfig is builds depending on each other. I don’t think it applies here. 3. use different distros. DSITRO=user bitbake example-image, DISTRO=debug example-image. Kernel and remote firmware recipes use variables from the distro to decide whether to build debug or user variants. This is clearly the standard way. Which is the best method ? All these options seem very BSP layer specific, is there something more generic and better than the above options ? If not, should this be a feature request to the Yocto project ? NOTE: debug variants may include the following. - more debug related kernel configs and security loosened and UART ports disabled - image recipe debug variant may include debug utilities absent from the user variant Easily archived with 3. There could be more than just debug and user variants. Android has engineering, tests, user, tiny and user-debug. See 3. You can have an arbitrary number of distros, interrelated however you wish. Greetz -- Regards, Karthik Poduval On Tue, Mar 1, 2022 at 1:26 AM tomzy <tomasz.zyjewski@...> wrote: > > Thanks Tomasz. I will check kas. > > No problem > > Yes, for selecting some of the packages I have created prod and debug image > recipes.But this did not work for the kernel as the kernel recipe is picked > as part of PROVIDERin machine conf. > > What are the difference there? You want to use different config on prod and debug images? > Maybe add it as config fragments? Then you would need to add some global variable to > distinguish when use given .cfg file. > > [1] https://docs.yoctoproject.org/singleindex.html#creating-configuration-fragments > > SoI had to use 2 conf to have the > IMAGE_FEATURES (orany other var)set differently for prod and debug. This is for > building the kernelrecipie differently for prodand debug. Setting the > IMAGE_FEATURES in the image recipe (and not inconf) causes2 problems. One is > that kernel and other bootloaders recipes arepicked early via PROVIDER in conf > and not as packages included in image recipe. > > Is that a problem? > > Secondly,setting the var in the > image recipe breaks this command for e.g. > "bitbake base-image-prod.bbbase-image-debug.bb". > > Didn't you want to distinguish this to builds to be able to run `bitbake base-image-prod` or > `bitbake base-image-debug`? > > Since the command parses the recipes only once for both image creation. > > > Nevertheless I would greatly recommend you to use kas. In simple .yml file you > could prepare different `local.conf` per configuration prod/debug. > > [2] https://kas.readthedocs.io/en/latest/userguide.html#project-configuration > > Regards, > Tomasz Żyjewski > Embedded Systems Engineer > GPG: 5C495EA3EBEECA59 > https://3mdeb.com \| @3mdeb_com > > > >
More All Messages By This Member
csimmonds #56349 Hi Karthik, On 02/03/2022 03:22, Karthik Poduval wrote: This concept of user vs debug builds is there in Android and any Android developer who gets introduced to yocto may look for these options. There seem to be many implementation choices here, trying to summarize some of the options here. For clarity, I just want to point out that the Android build system does NOT build the kernel. Linux is built separately and then binary kernel image and modules are copied into the Android images. So, you get the same kernel whether you have an eng, userdebug or user build I mention this because building different kernels for release/debug seems to be complicating things quite a lot. Maybe that is a separate issue 1. use different image recipes example-image-user.bb vs example-image-debug.bb but this still poses a problem for kernel recipes as kernel needs to have different config fragments or different defconfig for debug vs user variants. One possible option is to use KTYPE to select tiny vs standard or define own for custom BSP layers. What about other recipes like u-boot or firmware for other remote processors, how to percolate the debug vs user options to those recipes via just an image recipe ? 2. use different configs using multiconfig. Base config (which is debug) also selects the user config hence always building both build variants in different tmp directories. Kernel, remote firmware and image recipes use variables from the multiconfig cof files to decide whether to build debug or user variants. 3. use different distros. DSITRO=user bitbake example-image, DISTRO=debug example-image. Kernel and remote firmware recipes use variables from the distro to decide whether to build debug or user variants. Which is the best method ? All these options seem very BSP layer specific, is there something more generic and better than the above options ? If not, should this be a feature request to the Yocto project ? NOTE: debug variants may include the following. - more debug related kernel configs and security loosened and UART ports disabled - image recipe debug variant may include debug utilities absent from the user variant There could be more than just debug and user variants. Android has engineering, tests, user, tiny and user-debug. -- Regards, Karthik Poduval On Tue, Mar 1, 2022 at 1:26 AM tomzy <tomasz.zyjewski@...> wrote: Thanks Tomasz. I will check kas. No problem Yes, for selecting some of the packages I have created prod and debug image recipes.But this did not work for the kernel as the kernel recipe is picked as part of PROVIDERin machine conf. What are the difference there? You want to use different config on prod and debug images? Maybe add it as config fragments? Then you would need to add some global variable to distinguish when use given .cfg file. [1] https://docs.yoctoproject.org/singleindex.html#creating-configuration-fragments SoI had to use 2 conf to have the IMAGE_FEATURES (orany other var)set differently for prod and debug. This is for building the kernelrecipie differently for prodand debug. Setting the IMAGE_FEATURES in the image recipe (and not inconf) causes2 problems. One is that kernel and other bootloaders recipes arepicked early via PROVIDER in conf and not as packages included in image recipe. Is that a problem? Secondly,setting the var in the image recipe breaks this command for e.g. "bitbake base-image-prod.bbbase-image-debug.bb". Didn't you want to distinguish this to builds to be able to run `bitbake base-image-prod` or `bitbake base-image-debug`? Since the command parses the recipes only once for both image creation. Nevertheless I would greatly recommend you to use kas. In simple .yml file you could prepare different `local.conf` per configuration prod/debug. [2] https://kas.readthedocs.io/en/latest/userguide.html#project-configuration Regards, Tomasz Żyjewski Embedded Systems Engineer GPG: 5C495EA3EBEECA59 https://3mdeb.com \| @3mdeb_com Cheers Chris
More All Messages By This Member

1 - 9 of 9

1

Previous Topic Next Topic

Home Hashtags Subgroups Terms