fw_printenv works but fw_setenv does not after switching to linux-fslc-imx 5.10.69 from linux-fslc-imx 5.4.114


Brian Hutchinson
 

Hi,

I'm using u-boot U-Boot 2019.04-4.19.35-1.1.0+g4d377539a1 in pre-programmed NOR flash and was using linux-fslc-imx 5.4.114 kernel and fw_printenv/fw_setenv were working fine.

After upgrading to linux-fslc-imx 5.10.69, fw_setenv will not change any u-boot variable.  fw_printenv looks like it is working ok.  I've verified all my .config settings are the same with regard to MTD etc as my older 5.4.114 kernel.

Any ideas what could be broken?

I'm using a Dunfell based rootfs for our main system.  I did a hardknott build of core-image-base in order to build linux-fslc-imx 5.10.69 and when I realized fw_setenv wasn't working with Dunfell rootfs and 5.10.69 kernel, I built hardknott core-image-base rootfs with libubootenv package added to CORE_IMAGE_EXTRA_INSTALL thinking I'd copy over fw_printenv/fs_setenv and libubootenv and that would probably solve my issue ... but it doesn't include fw_printenv/fw_setenv (but libubootenv is there) in the rootfs so I don't understand what all is going on here.

Been using fw_printenv/fw_setenv since like ... forever and haven't run into something like this before that I can remember.

Regards,

Brian


Peter Bergin
 

Hi Brian,

On 2021-11-08 05:01, Brian Hutchinson wrote:
Hi,

I'm using u-boot U-Boot 2019.04-4.19.35-1.1.0+g4d377539a1 in pre-programmed NOR flash and was using linux-fslc-imx 5.4.114 kernel and fw_printenv/fw_setenv were working fine.

After upgrading to linux-fslc-imx 5.10.69, fw_setenv will not change any u-boot variable.  fw_printenv looks like it is working ok.  I've verified all my .config settings are the same with regard to MTD etc as my older 5.4.114 kernel.

Any ideas what could be broken?

One fundamental part of the libubootenv is the file configuration file, default /etc/fw_env.config. This file must match what is configured in u-boot. If you only have changed kernel I guess this file is untouched?

I don't know what debugging tools you have in the target, but one idea could be to use strace with working and non-working setup and compare.

I'm using a Dunfell based rootfs for our main system.  I did a hardknott build of core-image-base in order to build linux-fslc-imx 5.10.69 and when I realized fw_setenv wasn't working with Dunfell rootfs and 5.10.69 kernel, I built hardknott core-image-base rootfs with libubootenv package added to CORE_IMAGE_EXTRA_INSTALL thinking I'd copy over fw_printenv/fs_setenv and libubootenv and that would probably solve my issue ... but it doesn't include fw_printenv/fw_setenv (but libubootenv is there) in the rootfs so I don't understand what all is going on here.

Been using fw_printenv/fw_setenv since like ... forever and haven't run into something like this before that I can remember.

The binaries fw_*env is located in the package libubootenv-bin. Add that one to IMAGE_INSTALL and I hope you  get them in your rootfs.

/Peter


Fabio Estevam
 

Hi Brian,

On Mon, Nov 8, 2021 at 1:01 AM Brian Hutchinson <b.hutchman@...> wrote:

Hi,

I'm using u-boot U-Boot 2019.04-4.19.35-1.1.0+g4d377539a1 in pre-programmed NOR flash and was using linux-fslc-imx 5.4.114 kernel and fw_printenv/fw_setenv were working fine.

After upgrading to linux-fslc-imx 5.10.69, fw_setenv will not change any u-boot variable. fw_printenv looks like it is working ok. I've verified all my .config settings are the same with regard to MTD etc as my older 5.4.114 kernel.

Any ideas what could be broken?
Could it be that on 5.10 the MTD devices are registered in a different
order, which causes a different mtd numbering of the devices?

Does fw_setenv give any error?


Brian Hutchinson
 



On Mon, Nov 8, 2021 at 3:03 AM Peter Bergin <peter@...> wrote:

Hi Peter,

Hi Brian,

On 2021-11-08 05:01, Brian Hutchinson wrote:
Hi,

I'm using u-boot U-Boot 2019.04-4.19.35-1.1.0+g4d377539a1 in pre-programmed NOR flash and was using linux-fslc-imx 5.4.114 kernel and fw_printenv/fw_setenv were working fine.

After upgrading to linux-fslc-imx 5.10.69, fw_setenv will not change any u-boot variable.  fw_printenv looks like it is working ok.  I've verified all my .config settings are the same with regard to MTD etc as my older 5.4.114 kernel.

Any ideas what could be broken?

One fundamental part of the libubootenv is the file configuration file, default /etc/fw_env.config. This file must match what is configured in u-boot. If you only have changed kernel I guess this file is untouched?

I don't know what debugging tools you have in the target, but one idea could be to use strace with working and non-working setup and compare.


My /etc/fw_env.config is correct.  I have my mtd partitions defined in my .dts and cat /proc/mtd is showing the partitions correctly so I don't see any problems there.

I'm using a Dunfell based rootfs for our main system.  I did a hardknott build of core-image-base in order to build linux-fslc-imx 5.10.69 and when I realized fw_setenv wasn't working with Dunfell rootfs and 5.10.69 kernel, I built hardknott core-image-base rootfs with libubootenv package added to CORE_IMAGE_EXTRA_INSTALL thinking I'd copy over fw_printenv/fs_setenv and libubootenv and that would probably solve my issue ... but it doesn't include fw_printenv/fw_setenv (but libubootenv is there) in the rootfs so I don't understand what all is going on here.

Been using fw_printenv/fw_setenv since like ... forever and haven't run into something like this before that I can remember.

The binaries fw_*env is located in the package libubootenv-bin. Add that one to IMAGE_INSTALL and I hope you  get them in your rootfs.

Why wouldn't  CORE_IMAGE_EXTRA_INSTALL work to put these in rootfs like previous builds did?

Thanks,

Brian


Brian Hutchinson
 

Hi Fabio,

On Mon, Nov 8, 2021 at 5:49 AM Fabio Estevam <festevam@...> wrote:
Hi Brian,

On Mon, Nov 8, 2021 at 1:01 AM Brian Hutchinson <b.hutchman@...> wrote:
>
> Hi,
>
> I'm using u-boot U-Boot 2019.04-4.19.35-1.1.0+g4d377539a1 in pre-programmed NOR flash and was using linux-fslc-imx 5.4.114 kernel and fw_printenv/fw_setenv were working fine.
>
> After upgrading to linux-fslc-imx 5.10.69, fw_setenv will not change any u-boot variable.  fw_printenv looks like it is working ok.  I've verified all my .config settings are the same with regard to MTD etc as my older 5.4.114 kernel.
>
> Any ideas what could be broken?

Could it be that on 5.10 the MTD devices are registered in a different
order, which causes a different mtd numbering of the devices?

Nope, cat /proc/mtd looks good.  Same order as in previous kernels.


Does fw_setenv give any error?

It does not.  It shows no signs of a problem, returns no errors or warnings at all.  It "looks" like it worked normal but does not change the u-boot environment variable.

Regards,

Brian


Brian Hutchinson
 



On Mon, Nov 8, 2021 at 7:55 AM Brian Hutchinson via lists.yoctoproject.org <b.hutchman=gmail.com@...> wrote:
Hi Fabio,

On Mon, Nov 8, 2021 at 5:49 AM Fabio Estevam <festevam@...> wrote:
Hi Brian,

On Mon, Nov 8, 2021 at 1:01 AM Brian Hutchinson <b.hutchman@...> wrote:
>
> Hi,
>
> I'm using u-boot U-Boot 2019.04-4.19.35-1.1.0+g4d377539a1 in pre-programmed NOR flash and was using linux-fslc-imx 5.4.114 kernel and fw_printenv/fw_setenv were working fine.
>
> After upgrading to linux-fslc-imx 5.10.69, fw_setenv will not change any u-boot variable.  fw_printenv looks like it is working ok.  I've verified all my .config settings are the same with regard to MTD etc as my older 5.4.114 kernel.
>
> Any ideas what could be broken?

Could it be that on 5.10 the MTD devices are registered in a different
order, which causes a different mtd numbering of the devices?

Nope, cat /proc/mtd looks good.  Same order as in previous kernels.


Does fw_setenv give any error?

It does not.  It shows no signs of a problem, returns no errors or warnings at all.  It "looks" like it worked normal but does not change the u-boot environment variable.

Regards,

Brian



Update.  I booted my board via NFS using core-image-base from the hardknot build I got 5.10.69 from and libubootenv.  I copied in fw_printenv/fs_setenv from  tmp/work/imx8mm_ddr4_evk-poky-linux/libubootenv/0.3.1-r0/image/usr/bin ... and got the same result.  This eliminates any possibility that it was a libc issue with me copying fw_printenv/fw_setenv to my Dunfell rootfs.  So something else must be going on here.  fw_setenv not liking the format of older u-boot env???  It will be a royal pain if I have to update our u-boot just to get fw_setenv to work again.

I may take this hardknott wic image and flash it to sdcard and boot from sdcard so both u-boot and 5.10.69 are of same release to see if that works ... if it does then it points to being a u-boot issue with my older u-boot ;(

Regards,

Brian


Peter Bergin
 

Hi Brian,

On 2021-11-08 13:51, Brian Hutchinson wrote:
I'm using a Dunfell based rootfs for our main system.  I did a hardknott build of core-image-base in order to build linux-fslc-imx 5.10.69 and when I realized fw_setenv wasn't working with Dunfell rootfs and 5.10.69 kernel, I built hardknott core-image-base rootfs with libubootenv package added to CORE_IMAGE_EXTRA_INSTALL thinking I'd copy over fw_printenv/fs_setenv and libubootenv and that would probably solve my issue ... but it doesn't include fw_printenv/fw_setenv (but libubootenv is there) in the rootfs so I don't understand what all is going on here.

Been using fw_printenv/fw_setenv since like ... forever and haven't run into something like this before that I can remember.

The binaries fw_*env is located in the package libubootenv-bin. Add that one to IMAGE_INSTALL and I hope you  get them in your rootfs.

Why wouldn't  CORE_IMAGE_EXTRA_INSTALL work to put these in rootfs like previous builds did?


CORE_IMAGE_EXTRA_INSTALL can be used as well and it will end up in IMAGE_INSTALL variable handled by core-image.bbclass. My point was that the binaries are in the package libubootenv-bin not in libubootenv. So you have to make sure you have libubootenv-bin in your IMAGE_INSTALL or another package that RDEPENDS on libubootenv-bin in order to get the binaries in your rootfs.

Best regards,
/Peter


Peter Bergin
 

Hi Brian,

On 2021-11-08 13:55, Brian Hutchinson wrote:
It does not.  It shows no signs of a problem, returns no errors or warnings at all.  It "looks" like it worked normal but does not change the u-boot environment variable.
libubootenv in hardknott seems to be 0.3.1. Looked in the libubootenv repo and found this commit that is added after which indicates that it could be silent failure of fw_setenv. https://github.com/sbabic/libubootenv/commit/42f919e29c7b5e86a25877a9c492017e62f8827c I don't have a clue if this helps but just to help you with ideas one thing could be to test this and see if the function libuboot_set_env returns error code or not.

Can you write to the memory with some other application than fw_setenv? Just to make sure it is possible to write to the NOR memory with the new kernel and avoid problems related to drivers, dtb etc.

Best regards,

/Peter


Brian Hutchinson
 

Hi Peter,

On Mon, Nov 8, 2021 at 3:54 PM Peter Bergin <peter@...> wrote:

Hi Brian,

On 2021-11-08 13:51, Brian Hutchinson wrote:
I'm using a Dunfell based rootfs for our main system.  I did a hardknott build of core-image-base in order to build linux-fslc-imx 5.10.69 and when I realized fw_setenv wasn't working with Dunfell rootfs and 5.10.69 kernel, I built hardknott core-image-base rootfs with libubootenv package added to CORE_IMAGE_EXTRA_INSTALL thinking I'd copy over fw_printenv/fs_setenv and libubootenv and that would probably solve my issue ... but it doesn't include fw_printenv/fw_setenv (but libubootenv is there) in the rootfs so I don't understand what all is going on here.

Been using fw_printenv/fw_setenv since like ... forever and haven't run into something like this before that I can remember.

The binaries fw_*env is located in the package libubootenv-bin. Add that one to IMAGE_INSTALL and I hope you  get them in your rootfs.

Why wouldn't  CORE_IMAGE_EXTRA_INSTALL work to put these in rootfs like previous builds did?


CORE_IMAGE_EXTRA_INSTALL can be used as well and it will end up in IMAGE_INSTALL variable handled by core-image.bbclass. My point was that the binaries are in the package libubootenv-bin not in libubootenv. So you have to make sure you have libubootenv-bin in your IMAGE_INSTALL or another package that RDEPENDS on libubootenv-bin in order to get the binaries in your rootfs.

You got me!  In my previous build I just had libubootenv and that appeared to work for my dunfell build somehow even that release shows fw_printenv/fw_setenv to be in the libubootenv-bin package too.  Weird. 

But putting libubootenv-bin in my build did make the bins included in my rootfs now.  So one mystery solved.

Thank you!

Regards,

Brian


Brian Hutchinson
 

Hi Peter,

Update below.

On Mon, Nov 8, 2021 at 4:14 PM Peter Bergin <peter@...> wrote:
Hi Brian,

On 2021-11-08 13:55, Brian Hutchinson wrote:
> It does not.  It shows no signs of a problem, returns no errors or
> warnings at all.  It "looks" like it worked normal but does not change
> the u-boot environment variable.
>
libubootenv in hardknott seems to be 0.3.1. Looked in the libubootenv
repo and found this commit that is added after which indicates that it
could be silent failure of fw_setenv.
https://github.com/sbabic/libubootenv/commit/42f919e29c7b5e86a25877a9c492017e62f8827c
I don't have a clue if this helps but just to help you with ideas one
thing could be to test this and see if the function libuboot_set_env
returns error code or not.

Can you write to the memory with some other application than fw_setenv?
 
No.  Apparently not.  I've compared kernel .config and device tree source from 5.4 series kernel and 5.10 series kernel and I cannot find an explanation for this but the following test fails:

root@imx8mmevk:~# flashcp -v ./imx8mm-evk.dtb /dev/mtd7                                                                                                                                                                                                          
Erasing blocks: 11/11 (100%)
Writing data: 40k/40k (100%)
Verifying data: 10k/40k (24%)File does not seem to match flash data. First mismatch at 0x00000000-0x00002800

I simply tried writing a file to an area of my NOR that I didn't care if it got overwritten.  So it does look like there is a problem with kernel flexspi driver or mtd drivers?

Regards,

Brian


Brian Hutchinson
 

Hi everyone and thank you for the comments/ideas etc.

I found my problem.  In the new imx8mm-evk.dts for 5.10.69 the flexspi node has  spi-tx-bus-width = <1>; and in all the previous kernel versions it was spi-tx-bus-width = <4>;  so when moving our customized .dts over to new kernel that was missed.

flashcp to our NOR and fw_setenv both work now.

Sorry for the noise.

Regards,

Brian

On Tue, Nov 9, 2021 at 9:07 AM Brian Hutchinson via lists.yoctoproject.org <b.hutchman=gmail.com@...> wrote:
Hi Peter,

Update below.

On Mon, Nov 8, 2021 at 4:14 PM Peter Bergin <peter@...> wrote:
Hi Brian,

On 2021-11-08 13:55, Brian Hutchinson wrote:
> It does not.  It shows no signs of a problem, returns no errors or
> warnings at all.  It "looks" like it worked normal but does not change
> the u-boot environment variable.
>
libubootenv in hardknott seems to be 0.3.1. Looked in the libubootenv
repo and found this commit that is added after which indicates that it
could be silent failure of fw_setenv.
https://github.com/sbabic/libubootenv/commit/42f919e29c7b5e86a25877a9c492017e62f8827c
I don't have a clue if this helps but just to help you with ideas one
thing could be to test this and see if the function libuboot_set_env
returns error code or not.

Can you write to the memory with some other application than fw_setenv?
 
No.  Apparently not.  I've compared kernel .config and device tree source from 5.4 series kernel and 5.10 series kernel and I cannot find an explanation for this but the following test fails:

root@imx8mmevk:~# flashcp -v ./imx8mm-evk.dtb /dev/mtd7                                                                                                                                                                                                          
Erasing blocks: 11/11 (100%)
Writing data: 40k/40k (100%)
Verifying data: 10k/40k (24%)File does not seem to match flash data. First mismatch at 0x00000000-0x00002800

I simply tried writing a file to an area of my NOR that I didn't care if it got overwritten.  So it does look like there is a problem with kernel flexspi driver or mtd drivers?

Regards,

Brian