Topics

NPM Fetcher & License collection in Yocto Zeus


Oliver Westermann
 

Hey,

We’ve upgraded to zeus some time ago and also updated some of our recipes around node-red and it’s nodes. Before I created my own class to easily fetch several nodes, now we used devtool to create those recipes. In zeus, this happens using the npm:// fetcher.

This worked fine, but we sometimes had QA errors, hard to reproduce. Sometimes on some peoples PC, the license files were missing when do_populate_lic was running.

ERROR: node-red-contrib-socketio-1.0.7-r0 do_populate_lic: QA Issue: node-red-contrib-socketio: LIC_FILES_CHKSUM points to an invalid file: /data/workspace/yocto-build/build/cognex-myna/tmp/work/aarch64-poky-linux/node-red-contrib-socketio/1.0.7-r0/npmpkg/node_modules/socket.io/node_modules/@types/cookie/LICENSE [license-checksum]

Turned out that do_unpack would download the dependencies of the node-red-module (so in the example above, node-red-contrib-socketio depends on socket.io), but do_install would package those away/clean them up.
Since they go into the resulting package, we still have to list those licenses, so I fixed it by adding

do_install[depends] += "${PN}:do_populate_lic"

to my packages to fix the order.

However I would like to fix this upstream, but couldn't find the correct source location to add this dependency. Can sb help me figure this out as I would like to contribute 😉

Best regards, Olli


Jean-Marie Lemetayer
 

Hi Olivier,

Thanks for your implication :)

I think the modification you want to make resides in the npm class file:

But, starting with dunfell, the npm related mechanism have been reworked and also the npm class:

Is it possible to test if you have the same issue with dunfell ?

Best Regards,
Jean-Marie

On Fri, Jan 22, 2021 at 10:00 AM Oliver Westermann <oliver.westermann@...> wrote:
Hey,

We’ve upgraded to zeus some time ago and also updated some of our recipes around node-red and it’s nodes. Before I created my own class to easily fetch several nodes, now we used devtool to create those recipes. In zeus, this happens using the npm:// fetcher.

This worked fine, but we sometimes had QA errors, hard to reproduce. Sometimes on some peoples PC, the license files were missing when do_populate_lic was running.

ERROR: node-red-contrib-socketio-1.0.7-r0 do_populate_lic: QA Issue: node-red-contrib-socketio: LIC_FILES_CHKSUM points to an invalid file: /data/workspace/yocto-build/build/cognex-myna/tmp/work/aarch64-poky-linux/node-red-contrib-socketio/1.0.7-r0/npmpkg/node_modules/socket.io/node_modules/@types/cookie/LICENSE [license-checksum]

Turned out that do_unpack would download the dependencies of the node-red-module (so in the example above, node-red-contrib-socketio depends on socket.io), but do_install would package those away/clean them up.
Since they go into the resulting package, we still have to list those licenses, so I fixed it by adding

do_install[depends] += "${PN}:do_populate_lic"

to my packages to fix the order.

However I would like to fix this upstream, but couldn't find the correct source location to add this dependency. Can sb help me figure this out as I would like to contribute 😉

Best regards, Olli




Oliver Westermann
 

Hey,

 

I figured to squeeze in a test and it seems that this is solved in Dunfell, at least I can’t trigger it even when trying and the folder structure used in the work directory has changed -> No issues there I guess.

 

For Zeus: I’ve checked https://git.yoctoproject.org/cgit/cgit.cgi/poky/log/?h=zeus and it seems that there are no new commits. From https://wiki.yoctoproject.org/wiki/Releases zeus is in the Community phase.
Does this mean no new commits -> Add it to some kind of “known issues” page? Or does it make sense to provide a patch for the zeus branch?

 

Best regards, Olli

 

Von: Jean-Marie Lemetayer <jeanmarie.lemetayer@...>
Gesendet: Freitag, 22. Januar 2021 11:31
An: Westermann, Oliver <Oliver.Westermann@...>
Cc: yocto@...
Betreff: [EXTERNAL] Re: [yocto] NPM Fetcher & License collection in Yocto Zeus

 

Hi Olivier,

 

Thanks for your implication :)

 

I think the modification you want to make resides in the npm class file:

 

But, starting with dunfell, the npm related mechanism have been reworked and also the npm class:

 

Is it possible to test if you have the same issue with dunfell ?

 

Best Regards,

Jean-Marie

 

On Fri, Jan 22, 2021 at 10:00 AM Oliver Westermann <oliver.westermann@...> wrote:

Hey,

We’ve upgraded to zeus some time ago and also updated some of our recipes around node-red and it’s nodes. Before I created my own class to easily fetch several nodes, now we used devtool to create those recipes. In zeus, this happens using the npm:// fetcher.

This worked fine, but we sometimes had QA errors, hard to reproduce. Sometimes on some peoples PC, the license files were missing when do_populate_lic was running.

ERROR: node-red-contrib-socketio-1.0.7-r0 do_populate_lic: QA Issue: node-red-contrib-socketio: LIC_FILES_CHKSUM points to an invalid file: /data/workspace/yocto-build/build/cognex-myna/tmp/work/aarch64-poky-linux/node-red-contrib-socketio/1.0.7-r0/npmpkg/node_modules/
socket.io/node_modules/@types/cookie/LICENSE [license-checksum]

Turned out that do_unpack would download the dependencies of the node-red-module (so in the example above, node-red-contrib-socketio depends on
socket.io), but do_install would package those away/clean them up.
Since they go into the resulting package, we still have to list those licenses, so I fixed it by adding

do_install[depends] += "${PN}:do_populate_lic"

to my packages to fix the order.

However I would like to fix this upstream, but couldn't find the correct source location to add this dependency. Can sb help me figure this out as I would like to contribute
😉

Best regards, Olli