yocto@lists.yoctoproject.org | License reporting for golang (and rust)

Home Messages Hashtags Subgroups

Date Date 1 - 5 of 5

License reporting for golang (and rust)

Irving ST #49713 Hello, I am building a device that uses some Go (and Rust) dependencies. This is on Yocto 2.7 / Warrior. I noticed that after building an image, the generated license.manifest and package.manifest (in tmp/deploy/licenses/) does not contain any mention of go packages or rust crates. The go packages seem to generate directories in tmp/deploy/licenses/ but they do not seem to be reported in the final image. For example, an image that contains docker (from meta-virtualization layer) has build dependencies on, say, go-cli (also from meta-virtualization layer). This is specified in the docker recipe having DEPENDS on go-cli recipe. I saw tmp/deploy/licenses/go-cli directory exists with licensing information, but tmp/deploy/licenses//license.manifest does not contain any reference to go-cli. My best guess at the moment is because go packages are build dependencies (DEPENDS instead of RDEPENDS), they are not considered shipped packages by bitbake (because they are not shipped, the final docker package is) and are not listed in the manifest files. This makes licensing compliance more difficult since I still need to show the copyright notices even for permissive licenses like MIT - but I can't show it if I don't know that it has been shipped with the image (since it's not in the manifest). Is there a way to get a manifest of go packages shipped into the image? This issue seems to happen with rust crates as well, so a solution / explanation for rust would be greatly appreciated too. Sorry if this has been resolved somewhere, I'm not exactly super familiar with the golang build system or its integration with Yocto. Best regards, Irving Tjiptowarsono
More All Messages By This Member
Randy MacLeod #49739 On 2020-06-22 4:53 a.m., Irving ST wrote: Hello, I am building a device that uses some Go (and Rust) dependencies. This is on Yocto 2.7 / Warrior. I noticed that after building an image, the generated license.manifest and package.manifest (in tmp/deploy/licenses/) does not contain any mention of go packages or rust crates. The go packages seem to generate directories in tmp/deploy/licenses/ but they do not seem to be reported in the final image. For example, an image that contains docker (from meta-virtualization layer) has build dependencies on, say, go-cli (also from meta-virtualization layer). This is specified in the docker recipe having DEPENDS on go-cli recipe. I saw tmp/deploy/licenses/go-cli directory exists with licensing information, but tmp/deploy/licenses//license.manifest does not contain any reference to go-cli. My best guess at the moment is because go packages are build dependencies (DEPENDS instead of RDEPENDS), they are not considered shipped packages by bitbake (because they are not shipped, the final docker package is) and are not listed in the manifest files. This makes licensing compliance more difficult since I still need to show the copyright notices even for permissive licenses like MIT - but I can't show it if I don't know that it has been shipped with the image (since it's not in the manifest). Is there a way to get a manifest of go packages shipped into the image? This issue seems to happen with rust crates as well, so a solution / explanation for rust would be greatly appreciated too. Sorry if this has been resolved somewhere, I'm not exactly super familiar with the golang build system or its integration with Yocto. Hi Irving, I haven't heard or such a problem until now. If possible, please check if it's still happening on a newer release or the master branch. If it is still an issue, please open a bug with detailed steps to reproduce in: https://bugzilla.yoctoproject.org/ See: https://wiki.yoctoproject.org/wiki/Bug_reporting_and_Information_levels We're a little short on people to fix bugs so you could assign the defect to yourself. If you're not able to do that, we review new bugs once a week so maybe someone will decide to work on the defect. Thanks, ../Randy Best regards, Irving Tjiptowarsono -- # Randy MacLeod # Wind River Linux
More All Messages By This Member
Robert Berger #49771 Hi, My comments are in-line On 22/06/2020 11:53, Irving ST wrote: Hello, I am building a device that uses some Go (and Rust) dependencies. This is on Yocto 2.7 / Warrior. I noticed that after building an image, the generated license.manifest and package.manifest (in tmp/deploy/licenses/) does not contain any mention of go packages or rust crates. The go packages seem to generate directories in tmp/deploy/licenses/ but they do not seem to be reported in the final image. On dunfell 3.1.1 I see this: licenses/ └── github.com-influxdata-influxdb ├── generic_MIT ├── LICENSE └── recipeinfo which is wrong, since I would need to add all the licenses of all the dependencies golang pulls in as well to the recipe. It's shows only the top level license. In my license manifest I do have the influxdb: tmp/deploy/licenses/image-influxdb-from-source-container-x86-64-20200629205620/license.manifest PACKAGE NAME: github.com-influxdata-influxdb PACKAGE VERSION: 1.8.0 RECIPE NAME: github.com-influxdata-influxdb LICENSE: MIT Regards, Robert
More All Messages By This Member
Irving ST #49785 Hi, For anyone else having the same problem, here's what I eventually did for my go stuff: I created a bbclass containing a task that gets inherited by all packages, but the task only does some work when it detects that it is inherited by a golang recipe. The task basically sets up GOPATH and calls https://github.com/rancher/trash - this tool populates all the source dependencies and generate a trash.lock file in the repository. Then I used https://github.com/pivotal/LicenseFinder to generate a json report that can be postprocessed with python. To me it seems that Go used the vendor directory approach in the past, and there are multiple tools that can be used for this; and nowadays go seems to have switched to a different approach called go modules. With this multiple tools and methods of managing dependencies, I am not sure whether it is feasible to integrate this in Yocto at all - though I think I have seen mailing list posts on meta-virtualization adding vendor information to their recipes, so maybe some progress is happening there. Best regards, Irving Tjiptowarsono On Tue, Jun 30, 2020 at 7:32 AM Robert Berger@... <robert.berger.yocto.user@...> wrote: Hi, My comments are in-line On 22/06/2020 11:53, Irving ST wrote: Hello, I am building a device that uses some Go (and Rust) dependencies. This is on Yocto 2.7 / Warrior. I noticed that after building an image, the generated license.manifest and package.manifest (in tmp/deploy/licenses/) does not contain any mention of go packages or rust crates. The go packages seem to generate directories in tmp/deploy/licenses/ but they do not seem to be reported in the final image. On dunfell 3.1.1 I see this: licenses/ └── github.com-influxdata-influxdb ├── generic_MIT ├── LICENSE └── recipeinfo which is wrong, since I would need to add all the licenses of all the dependencies golang pulls in as well to the recipe. It's shows only the top level license. In my license manifest I do have the influxdb: tmp/deploy/licenses/image-influxdb-from-source-container-x86-64-20200629205620/license.manifest PACKAGE NAME: github.com-influxdata-influxdb PACKAGE VERSION: 1.8.0 RECIPE NAME: github.com-influxdata-influxdb LICENSE: MIT Regards, Robert
More All Messages By This Member
Robert Berger #49804 Hi, My comments are in-line On 30/06/2020 14:37, Irving ST wrote: Hi, For anyone else having the same problem, here's what I eventually did for my go stuff: I created a bbclass containing a task that gets inherited by all packages, but the task only does some work when it detects that it is inherited by a golang recipe. The task basically sets up GOPATH and calls https://github.com/rancher/trash - this tool populates all the source dependencies and generate a trash.lock file in the repository. Then I used https://github.com/pivotal/LicenseFinder to generate a json report that can be postprocessed with python. Is your work available somewhere in the public? I would like to give it a try with dunfell 3.1.1 and influxdb.[1] Also it would be interesting what happens if you include a prebuilt go executable[2]. [1] https://gitlab.com/meta-layers/meta-tig/-/blob/master/recipes-from-source/github.com-influxdata-influxdb/github.com-influxdata-influxdb_1.8.0.bb (from khem and slightly hacked) [2] https://gitlab.com/meta-layers/meta-tig/-/blob/master/recipes-prebuilt/influxdb/influxdb_1.8.0.bb To me it seems that Go used the vendor directory approach in the past, and there are multiple tools that can be used for this; and nowadays go seems to have switched to a different approach called go modules. It looks like master and dunfell 3.1.1 support go modules now.[3][4] [3] http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?h=dunfell&id=7892a056f7e03e9c086148f8f027bd1cebf2aa68 [4] http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?h=dunfell&id=ce277ec45f1f241b1eb17ea6999dbe5a718b898a Now we have both go dep and go modules support ;) With this multiple tools and methods of managing dependencies, I am not sure whether it is feasible to integrate this in Yocto at all - though I think I have seen mailing list posts on meta-virtualization adding vendor information to their recipes, so maybe some progress is happening there. Currently arm 32 Golang support is broken in meta-virt (docker), but I am confident it's going to be fixed soon.[5] [5] https://lists.yoctoproject.org/g/meta-virtualization/message/5488?p=,,,20,0,0,0::Created,,docker,20,2,0,75127700 Best regards, Irving Tjiptowarsono Regards, Robert
More All Messages By This Member

1 - 5 of 5

1

Previous Topic Next Topic

Home Hashtags Subgroups Terms