Re: #golang: go fetches dependencies in compile phase


Robert Berger
 

Hi,

My comments are inline.

On 12/04/2021 14:47, Juergen Landwehr wrote:
Hi Robert,
thanks for your thoughts.
I see your point and the last thing I want is "NOT reproducable builds".
But dependency management in go is not that arbitrary as it may seem.
... if everybody does what they are supposed to - which is not the case.

Dependencies and their version is stored in "go.mod". To ensure reproducable builds, hashes for each dependency and version are stored in "go.sum". Both files are in git and together with a local golang proxy, this should ensure reproducable builds, right?
This does not sound too bad. This would also mean, that if the outside repo dies you can still build and that you know what's on your proxy.

To ensure that licences are valid and did not change over time, we developed a little tool, that goes through all dependencies and creates the following output, which we insert into our recipe:
LICENSE = "Apache-2.0 & MIT & MIT & BSD-2-Clause & BSD-3-Clause & ...
Here you

1) Totally lost which License comes from which module and I hope 2 times MIT is just a typo.

Of course if you are really serious about licensing you should use a tool like fossology, upload all your sources there and inspect them.
You will be surprised.

There are a couple of tools out there which scan your sources and some which claim to do stuff with golang as well.

2) Do you check if the licenses are compatible?

MIT and Apache are compatible:

some googling:

"Both the Apache License and the MIT license are permissive, so incorporating MIT licensed code into your Apache licensed project is certainly allowed. Just be sure to attribute the original author for the parts your incorporated and include a copy of the MIT License terms, as required by the license."

Apache and BSD should also be OK:

some googling:

"In both of them you must:

Include copyright

But in Apache, unlike BSD you must:

Include License
State Changes
Include Notice
"

LIC_FILES_CHKSUM = " \
file://${S}/src/${GO_IMPORT}/vendor/github.com/coreos/go-oidc/LICENSE;md5=d2794c0df5b907fdace235a619d80314 \
file://${S}/src/${GO_IMPORT}/vendor/github.com/go-playground/locales/LICENSE;md5=3ccbda375ee345400ad1da85ba522301 \
file://${S}/src/${GO_IMPORT}/vendor/github.com/go-playground/universal-translator/LICENSE;md5=2e2b21ef8f61057977d27c727c84bef1 \
file://${S}/src/${GO_IMPORT}/vendor/github.com/godbus/dbus/v5/LICENSE;md5=09042bd5c6c96a2b9e45ddf1bc517eed \
file://${S}/src/${GO_IMPORT}/vendor/github.com/golang/gddo/LICENSE;md5=4c728948788b1a02f33ae4e906546eef \
...
This is a manual step and whenever dependencies change we have to create this list again and add it to the recipe, but it contains all the required license information and makes them visible in the recipe.
really all?

You search through every single file of a go module and it's dependencies? Or just the license text, if there is one?

It might pollute the recipe a bit, but luckily we do not have thousands of dependencies like some npm projects. So I think it is still manageable. And is it much less work, than defining a recipe for each dependency.
So we would
* guarantee reproducable builds
* use the programming language (in our case golang) to handle dependency management
* ensure that licenses are visible and correct
I mean it is not perfect and still a compromise, but it should work without breaking reproducable builds or causing license issues?
What do you think?
Regards,
J├╝rgen
PS: Thanks for the link. I was not aware of this discussion (it is quite a bit to read:))
Regards,

Robert

Join yocto@lists.yoctoproject.org to automatically receive all group messages.