Date
1 - 9 of 9
repost: how to create a SPDX "notice file" from a build?
|
Robert P. J. Day
i asked about this a couple months ago but didn't see any replies,
so i'll ask again with a little more detail. colleague wants to, from YP (actually petalinux but should be irrelevant), some sort of SPDX "notice file", along the lines of what can be generated by black duck. it doesn't need to be identical, but it would be useful to at least have a first pass that people can look at and say what they want tweaked. is there an example of how to (using the spdx.bbclass class file, i assume) do something like this? thanks. rday -- ======================================================================== Robert P. J. Day Ottawa, Ontario, CANADA http://crashcourse.ca Twitter: http://twitter.com/rpjday LinkedIn: http://ca.linkedin.com/in/rpjday ======================================================================== |
|
|
|
Robert P. J. Day
On Fri, 22 Nov 2019, Robert P. J. Day wrote:
i asked about this a couple months ago but didn't see any replies,a lilttle more progress, and then an error -- i installed fossology on my ubuntu 18.04 dev box, then added INHERIT += "spdx" to local.conf, and tried to build a simple qemuarmx86-64, and very quickly had a number of native packages fail with: //////////// start ////////// DEBUG: Executing python function do_spdx ERROR: Error executing a python function in exec_python_func() autogenerated: The stack trace of python calls that resulted in this exception/failure was: File: 'exec_python_func() autogenerated', lineno: 2, function: <module> 0001: *** 0002:do_spdx(d) 0003: File: '/home/rpjday/oe/dist/layers/poky/meta/classes/spdx.bbclass', lineno: 83, function: do_spdx 0079: foss_full_spdx = d.getVar('FOSS_FULL_SPDX') == "true" or False 0080: foss_command = "wget %s --post-file=%s %s"\ 0081: % (foss_flags, info['tar_file'], foss_server) 0082: *** 0083: foss_result = run_fossology(foss_command, foss_full_spdx) 0084: if foss_result is not None: 0085: (foss_package_info, foss_file_info, foss_license_info) = foss_result 0086: spdx_file_info = create_spdx_doc(local_file_info, foss_file_info) 0087: ## write to cache File: '/home/rpjday/oe/dist/layers/poky/meta/classes/spdx.bbclass', lineno: 230, function: run_fossology 0226: # Package info 0227: package_info = {} 0228: if full_spdx: 0229: # All mandatory, only one occurrence *** 0230: package_info['PackageCopyrightText'] = re.findall('PackageCopyrightText: (.*?</text>)', foss_output, re.S)[0] 0231: package_info['PackageLicenseDeclared'] = re.findall('PackageLicenseDeclared: (.*)', foss_output)[0] 0232: package_info['PackageLicenseConcluded'] = re.findall('PackageLicenseConcluded: (.*)', foss_output)[0] 0233: # These may be more than one 0234: package_info['PackageLicenseInfoFromFiles'] = re.findall('PackageLicenseInfoFromFiles: (.*)', foss_output) Exception: IndexError: list index out of range ERROR: list index out of range DEBUG: Python function do_spdx finished ERROR: Function failed: do_spdx /////////// end ///////// i have absolutely no idea what to think of this, and am open to suggestions. does anyone have a working scenario to simply demonstrate the usage of spdx.bbclass? rday -- ======================================================================== Robert P. J. Day Ottawa, Ontario, CANADA http://crashcourse.ca Twitter: http://twitter.com/rpjday LinkedIn: http://ca.linkedin.com/in/rpjday ======================================================================== |
|
|
|
On Fri, Nov 22, 2019 at 9:03 AM rpjday@...
<rpjday@...> wrote: it once generated SPDX manifest from patched sources for package and file level it has sadly regressed, so would need some love as you see above.
|
|
|
|
akuster808 <akuster808@...>
On 11/22/19 9:03 AM,
rpjday@... wrote:
On Fri, 22 Nov 2019, Robert P. J. Day wrote:i asked about this a couple months ago but didn't see any replies, so i'll ask again with a little more detail. colleague wants to, from YP (actually petalinux but should be irrelevant), some sort of SPDX "notice file", along the lines of what can be generated by black duck. it doesn't need to be identical, but it would be useful to at least have a first pass that people can look at and say what they want tweaked. is there an example of how to (using the spdx.bbclass class file, i assume) do something like this? thanks. Would you mind opening a Yocto defect. - armin rday |
|
|
|
Richard Purdie
On Fri, 2019-11-22 at 09:59 -0800, akuster808 wrote:
That code hasn't been touched in a while and needs some serious attention. The underlying tools and processes have changed so much it may be a case of starting again and we should perhaps consider removing that class... Cheers, Richard |
|
|
|
Robert P. J. Day
On Sat, 23 Nov 2019, Richard Purdie wrote:
On Fri, 2019-11-22 at 09:59 -0800, akuster808 wrote:yeah, that the impression i got ... is there a newer alternative toThat code hasn't been touched in a while and needs some serious that, or just the standard license info that's been generated all this time? rday -- ======================================================================== Robert P. J. Day Ottawa, Ontario, CANADA http://crashcourse.ca Twitter: http://twitter.com/rpjday LinkedIn: http://ca.linkedin.com/in/rpjday ======================================================================== |
|
|
|
Mark Hatle
On 11/23/19 6:01 AM, Richard Purdie wrote:
On Fri, 2019-11-22 at 09:59 -0800, akuster808 wrote:I think the use-cases have changed over time, even though parts and pieces areThat code hasn't been touched in a while and needs some serious still valid. There are really a few groups to consider. 1) (old case) someone is building a system and wants to construct SPDX files for the things they are building. Contacting, uploading, getting a report from fossology may still be the best way of doing this. 2) (new case) things could be shipped with prebuilt SPDX files (based on fossology run by the system, maintainer, an addon layer, OSV, etc..) In this case we would want to simply tie a recipe to an SPDX and be able to correlate them. 3) In either case, we have a list of SPDX files, but that doesn't meet Robert's question. Something needs to process these SPDX files and generate notice files and similar. To me this is an external tool, that could optionally be invoked at image creation time (or by the user directly.) Further, a 4th case.. what is the license of the components I've actually deployed. I've wanted to do this for a long time, but using the dwarf debug information you can determine what files were actually used to construct the binaries in your images. From that you can go back to the SPDX files and correlated to exactly what was deployed including file level copyright, notice, and license requirements (not just recipe) and produce an incredibly accurate report. Add to this that SPDX has the ability for custom fields that can be used to track other IP issues like patents, legal concerns, etc. And you could construct a report in a form for the legal organization of a company to review prior to product shipment. Right now, we have an old way to do 1, but it doesn't solve Robert's issue -- even if it DID work. and no way to do the rest (that I am aware of). --Mark Cheers, |
|
|
|
Robert P. J. Day
On Sat, 23 Nov 2019, Richard Purdie wrote:
On Fri, 2019-11-22 at 09:59 -0800, akuster808 wrote:as one more followup to this, first, i was asked by a colleagueThat code hasn't been touched in a while and needs some serious about the possibility of generating an "SPDX notice file," and as i don't know what the format of one of those things is, it's hard for me to know how to answer. it seems clear at this point that spdx.bbclass is not going to cut it, as richard points out. more to the point, has anyone had to do something that seems to resemble what i was asked about? i'm open to suggestions as to what *is* supported so i can try it out and hand it over and at least say, "is this close to what you're looking for?" thanks for any pointers. rday -- ======================================================================== Robert P. J. Day Ottawa, Ontario, CANADA http://crashcourse.ca Twitter: http://twitter.com/rpjday LinkedIn: http://ca.linkedin.com/in/rpjday ======================================================================== |
|
|
|
Mark Hatle
(resending, the list rejected the first reply)
On 11/24/19 4:11 AM, rpjday@... wrote: On Sat, 23 Nov 2019, Richard Purdie wrote:I know individual companies have created their own tools to do this, but sinceOn Fri, 2019-11-22 at 09:59 -0800, akuster808 wrote:as one more followup to this, first, i was asked by a colleagueThat code hasn't been touched in a while and needs some serious it's both required and could include liability (if you get it wrong), I'm not aware of any open source way of doing this. If there is one, it would likely be published as part of the SPDX project/mailing lists. --Mark thanks for any pointers. |
|
|