Urgent need for SPDX-formatted bill of materials.


Randy MacLeod
 

Saul,

Richard mentioned in today's call that there's a new US mandate around
computer security to have a software bill of materials. I mentioned that we
have a need for this as well and that it was in your queue.


To get started, all that's really needed is a list of components and their version.

Richard mentioned that this layer:
   https://github.com/doubleopen-project/meta-doubleopen/
generates reports in SPDX already.


RP's idea is to get rid of the existing manifest files and
replace it with an spdx formatted one. Start with something simple.

Joshua mentioned that he has an internal tool but he just started
talking with the developer about it. Any comments Joshua?


--
# Randy MacLeod
# Wind River Linux


Trevor Woerner
 

I'd like to recommend this be a round-table topic for next week's OE Developers Meeting?

Saul, Randy, Joshua, (and anyone else) I'll offer to moderate if you'd be interesting in joining the discussions?


Joshua Watt
 

On Tue, May 18, 2021 at 1:32 PM Trevor Woerner <twoerner@gmail.com> wrote:

I'd like to recommend this be a round-table topic for next week's OE Developers Meeting?

Saul, Randy, Joshua, (and anyone else) I'll offer to moderate if you'd be interesting in joining the discussions?
Sure



Trevor Woerner
 

Does anyone know if Saul is on this mailing list? I can't seem to find any past posts of his (which isn't definitive).


On Tue, May 18, 2021 at 2:44 PM Joshua Watt <jpewhacker@...> wrote:
On Tue, May 18, 2021 at 1:32 PM Trevor Woerner <twoerner@...> wrote:
>
> I'd like to recommend this be a round-table topic for next week's OE Developers Meeting?
>
> Saul, Randy, Joshua, (and anyone else) I'll offer to moderate if you'd be interesting in joining the discussions?

Sure

>
>


Armin Kuster
 

On 5/18/21 11:55 AM, Trevor Woerner wrote:
Does anyone know if Saul is on this mailing list? I can't seem to find
any past posts of his (which isn't definitive).
he was cc'ed in the original email

-armin

On Tue, May 18, 2021 at 2:44 PM Joshua Watt <jpewhacker@gmail.com
<mailto:jpewhacker@gmail.com>> wrote:

On Tue, May 18, 2021 at 1:32 PM Trevor Woerner <twoerner@gmail.com
<mailto:twoerner@gmail.com>> wrote:
>
> I'd like to recommend this be a round-table topic for next
week's OE Developers Meeting?
>
> Saul, Randy, Joshua, (and anyone else) I'll offer to moderate if
you'd be interesting in joining the discussions?

Sure

>
>




Armin Kuster
 

On 5/18/21 11:32 AM, Trevor Woerner wrote:
I'd like to recommend this be a round-table topic for next week's OE
Developers Meeting?
If meta-doubleopen addresses the issue for folks, what is the topic of
this Round-table?

-armin


Saul, Randy, Joshua, (and anyone else) I'll offer to moderate if you'd
be interesting in joining the discussions?



Trevor Woerner
 

On Tue, May 18, 2021 at 4:59 PM akuster808 <akuster808@...> wrote:
On 5/18/21 11:32 AM, Trevor Woerner wrote:
> I'd like to recommend this be a round-table topic for next week's OE
> Developers Meeting?
If meta-doubleopen addresses the issue for folks, what is the topic of
this Round-table?

I'm still investigating and putting together a set of ideas.

meta-doubleopen says "This meta layer is intended for use with Double Open's open source license compliance workflow". *license* workflow, we're talking about SBOMs. The fact they produce SPDX files isn't all that's required to create an SBOM. SPDX is just a file format. In fact there's nothing in that layer that says anything about SBOM. From what I can tell, all meta-doubleopen is producing is an SPDX version of the various manifest files one would find if buildhistory is enabled.

SPDX is only one of several file formats that can be used to generate an SBOM in a standard way. It could be worth a discussion to at least mention the others.

We could look at what an SBOM is, and what are the minimum required fields to produce an SBOM.

Another question for the round table: should we integrate this into oe-core, or leave it as a separate layer?

The round table is also a great way of introducing this important topic to the community at large. I bet you half the people attending the conference have never heard of an SBOM, but might be interested to know YP/OE is looking into integrating it into the build system (especially now that the US government has released an Executive Order regarding SBOMs, and that the EU is also looking into these sorts of things as well).

I'll look into inviting the DoubleOpen people to the meeting.

Joshua mentioned that the company he works for is also investigating generating SBOMs from YP/OE builds, so let's make sure everyone is working on one project, instead of scattering the community.

So there are a couple things we could talk about :-)
 
 
>
> Saul, Randy, Joshua, (and anyone else) I'll offer to moderate if you'd
> be interesting in joining the discussions?
>
>
>


Richard Purdie
 

On Tue, 2021-05-18 at 17:28 -0400, Trevor Woerner wrote:
On Tue, May 18, 2021 at 4:59 PM akuster808 <akuster808@gmail.com> wrote:
On 5/18/21 11:32 AM, Trevor Woerner wrote:
I'd like to recommend this be a round-table topic for next week's OE
Developers Meeting?
If meta-doubleopen addresses the issue for folks, what is the topic of
this Round-table?

I'm still investigating and putting together a set of ideas.
I'd like to try and put some guidance around the discussion if I may?
There are a ton of things we could do here but I think the need
is comparatively clear and pressing. Discussions are good where
the outcome is unknown and options need to be explored. I think
some of this is relatively clear for the reasons I'll mention below.

meta-doubleopen says "This meta layer is intended for use with 
Double Open's open source license compliance workflow". *license*
workflow, we're talking about SBOMs. The fact they produce SPDX 
files isn't all that's required to create an SBOM. SPDX is just
a file format. In fact there's nothing in that layer that says 
anything about SBOM. From what I can tell, all meta-doubleopen is 
producing is an SPDX version of the various manifest files one 
would find if buildhistory is enabled.

SPDX is only one of several file formats that can be used to 
generate an SBOM in a standard way. It could be worth a discussion 
to at least mention the others.
Can I suggest we adopt the position that we aim for SPDX unless someone
produces a strong argument that something else has advantages?

The reason I say this is that it is the standard most projects are
consolidating around, it shows alignment with other work at the LF
and SDPX is aiming to become an ISO standard. To do something different
would put us in a difficult position IMO. People complain that LF 
projects don't collaborate enough, here we have an opportunity I want
to make work.


We could look at what an SBOM is, and what are the minimum required 
fields to produce an SBOM.
We do need to find out what the legislation says about this so we can
meet it.

Another question for the round table: should we integrate this into 
oe-core, or leave it as a separate layer?
We need to be able to say that OE/YP generates SBOM manifests for
images out the box, preferably by default. If we don't do that, we will
lose out to projects which can claim this. I think that makes the 
decision clear.


The round table is also a great way of introducing this important topic
to the community at large. I bet you half the people attending the 
conference have never heard of an SBOM, but might be interested to 
know YP/OE is looking into integrating it into the build system 
especially now that the US government has released an Executive Order 
regarding SBOMs, and that the EU is also looking into these sorts of things as well).

I'll look into inviting the DoubleOpen people to the meeting.

Joshua mentioned that the company he works for is also investigating 
generating SBOMs from YP/OE builds, so let's make sure everyone is 
working on one project, instead of scattering the community.

So there are a couple things we could talk about :-)
I think aspects which do need discussion is how to handle:

* SPDX data at the do_package/do_packagedata level
* SPDX data at the archiver and do_populate_lic level
* whether we can replace existing image manifests
* whether we need tooling to take an SPDX image manifest and process it to 
various forms for end user/tool use (e.g. actual file output or API?).

This probably translates into some kind of plan with different phases.

Cheers,

Richard


Trevor Woerner
 

Richard, this is all awesome! Thanks for your input :-)

On Tue, May 18, 2021 at 6:03 PM Richard Purdie <richard.purdie@...> wrote:
* whether we need tooling to take an SPDX image manifest and process it to 
  various forms for end user/tool use (e.g. actual file output or API?).

Kate Stewart recently did a webinar on this topic, you can find the video and slides:  

She also talked about this at the most recent FOSDEM:

I'm thinking of inviting her to the discussion.

If you look at her slides from the webinar, around slide 27 she talks about the ecosystem of tools for working with SBOMs depending on whether you're a producer, consumer, or user of a product. Given what she says, we only have to concern ourselves with producing a proper, compliant SBOM. Other tools in the ecosystem will handle the other things.


Trevor Woerner
 

The LF also recently released this blog post, which mentions YP in a positive light wrt SBOMs:


Kate Stewart
 

"we only have to concern ourselves with producing a proper, compliant SBOM".

+1  Being able to generate the SBOM as a byproduct of the build is going to have the most trust. 
Yocto is in a unique position to do this,  and provide guidance on extending the next generation
of SPDX as well.   Richard convinced me a couple of years ago that the necessary information is present
in the debug info,  challenge is extracting it out and outputting the document.   

Possible approach 
- mark all licensing as NOASSERTION for now, and focus on the components and mapping
the relationships between them.  
- Next phase, add in the licensing information when its available as SPDX headers (ie. no scanning 
tools needed),  use declared vs detected to separate out the info at the package level on what you're
getting from sources.

The example of how it's being done in Zephyr is based on hooking into CMake see:

Kubernetes approach:

AGL might be a good testbed for this capability with Yocto, as there is a PoC starting
in the Auto-ISAC,  and they'll be looking for SBOMs, so many eyes.

In terms of validating the output format produced - 
any document created conforms to the specification.

If there are questions about the way to partition the information, etc.
Steve Winslow and myself are happy to weigh in.

HTH,
Kate





On Tue, May 18, 2021 at 5:15 PM Trevor Woerner <twoerner@...> wrote:
Richard, this is all awesome! Thanks for your input :-)

On Tue, May 18, 2021 at 6:03 PM Richard Purdie <richard.purdie@...> wrote:
* whether we need tooling to take an SPDX image manifest and process it to 
  various forms for end user/tool use (e.g. actual file output or API?).

Kate Stewart recently did a webinar on this topic, you can find the video and slides:  

She also talked about this at the most recent FOSDEM:

I'm thinking of inviting her to the discussion.

If you look at her slides from the webinar, around slide 27 she talks about the ecosystem of tools for working with SBOMs depending on whether you're a producer, consumer, or user of a product. Given what she says, we only have to concern ourselves with producing a proper, compliant SBOM. Other tools in the ecosystem will handle the other things.




Konrad Weihmann
 

When is the round table going to happen exactly? I'd be interested to join the call, as I've been working on something aim for exactly such a SBOM


Trevor Woerner
 

On Wed, May 19, 2021 at 2:29 AM Konrad Weihmann <kweihmann@...> wrote:
When is the round table going to happen exactly? I'd be interested to join the call, as I've been working on something aim for exactly such a SBOM

Next week, on Tuesday and Wednesday the Yocto Project is having a virtual Summit.

As part of the Summit, on Tuesday a 4.5 hour block has been set aside for a developers meeting/round-table on a bunch of different topics of interest to the project. The list of topics is still being worked out, once that's worked out then we'll assign time slots to the individual topics. I've added BSOM to the list of topics. The exact time will be worked out on Monday. https://www.openembedded.org/wiki/OEDVM_2021

The block of time set aside for the developer meeting is: 1530 UTC to 2000 UTC Tuesday May 25.
The details of the developers meeting (including the link to join) can be found here: https://pretalx.com/yocto-project-summit-2021/talk/BVZMYW/


Trevor Woerner
 

On Wed, May 19, 2021 at 1:27 PM Saul Wold <Saul.Wold@...> wrote:
+1 on RP's comments here, see below

On 5/18/21 3:02 PM, Richard Purdie wrote:
> On Tue, 2021-05-18 at 17:28 -0400, Trevor Woerner wrote:
>> On Tue, May 18, 2021 at 4:59 PM akuster808 <akuster808@...> wrote:
>>> On 5/18/21 11:32 AM, Trevor Woerner wrote:
>>>> I'd like to recommend this be a round-table topic for next week's OE
>>>> Developers Meeting?
>>> If meta-doubleopen addresses the issue for folks, what is the topic of
>>> this Round-table?
>>>
>>
>>
>> I'm still investigating and putting together a set of ideas.
>
> I'd like to try and put some guidance around the discussion if I may?
> There are a ton of things we could do here but I think the need
> is comparatively clear and pressing. Discussions are good where
> the outcome is unknown and options need to be explored. I think
> some of this is relatively clear for the reasons I'll mention below.
>
>> meta-doubleopen says "This meta layer is intended for use with
>> Double Open's open source license compliance workflow". *license*
>> workflow, we're talking about SBOMs. The fact they produce SPDX
>> files isn't all that's required to create an SBOM. SPDX is just
>> a file format. In fact there's nothing in that layer that says
>> anything about SBOM. From what I can tell, all meta-doubleopen is
>> producing is an SPDX version of the various manifest files one
>> would find if buildhistory is enabled.
>>
>> SPDX is only one of several file formats that can be used to
>> generate an SBOM in a standard way. It could be worth a discussion
>> to at least mention the others.
>
> Can I suggest we adopt the position that we aim for SPDX unless someone
> produces a strong argument that something else has advantages?
>
> The reason I say this is that it is the standard most projects are
> consolidating around, it shows alignment with other work at the LF
> and SDPX is aiming to become an ISO standard. To do something different
> would put us in a difficult position IMO. People complain that LF
> projects don't collaborate enough, here we have an opportunity I want
> to make work.
>
>
>> We could look at what an SBOM is, and what are the minimum required
>> fields to produce an SBOM.
>
> We do need to find out what the legislation says about this so we can
> meet it.
>
https://www.ntia.gov/sbom may be the starting point, I am just getting
around to looking at it. I don't know if there is someone in this thread
(Kate maybe?) that can interpret legislation to technical!

>> Another question for the round table: should we integrate this into
>> oe-core, or leave it as a separate layer?
>
> We need to be able to say that OE/YP generates SBOM manifests for
> images out the box, preferably by default. If we don't do that, we will
> lose out to projects which can claim this. I think that makes the
> decision clear.
>
>
>> The round table is also a great way of introducing this important topic
>> to the community at large. I bet you half the people attending the
>> conference have never heard of an SBOM, but might be interested to
>> know YP/OE is looking into integrating it into the build system
>> especially now that the US government has released an Executive Order
>> regarding SBOMs, and that the EU is also looking into these sorts of things as well).
>>
>> I'll look into inviting the DoubleOpen people to the meeting.
>>
>> Joshua mentioned that the company he works for is also investigating
>> generating SBOMs from YP/OE builds, so let's make sure everyone is
>> working on one project, instead of scattering the community.
>>
Windriver also needs to provide this data, so there are probably more
people interested for sure.

>> So there are a couple things we could talk about :-)
>
> I think aspects which do need discussion is how to handle:
>
> * SPDX data at the do_package/do_packagedata level
> * SPDX data at the archiver and do_populate_lic level
> * whether we can replace existing image manifests
> * whether we need tooling to take an SPDX image manifest and process it to
>    various forms for end user/tool use (e.g. actual file output or API?).
>
I recently submitted the image-manifest script which produces a JSON
output from the image manifest so limited to what is actually being
built. This was based on a tinfoil script from Paul Eggelton.

This currently just reads the recipe LICENSE info which is human
generated, we need to figure out how to do a better job with SPDX.

Sounds good.

Saul: will you be able to join us for the discussion next Tuesday?
The developers meeting is free to join (you don't have to be registered for the conference to attend)
 

> This probably translates into some kind of plan with different phases.
>
> Cheers,
>
> Richard
>
>
>
>
>

--
Sau!


Armin Kuster
 

On 5/18/21 3:02 PM, Richard Purdie wrote:
On Tue, 2021-05-18 at 17:28 -0400, Trevor Woerner wrote:
On Tue, May 18, 2021 at 4:59 PM akuster808 <akuster808@gmail.com> wrote:
On 5/18/21 11:32 AM, Trevor Woerner wrote:
I'd like to recommend this be a round-table topic for next week's OE
Developers Meeting?
If meta-doubleopen addresses the issue for folks, what is the topic of
this Round-table?
I'm still investigating and putting together a set of ideas.
I'd like to try and put some guidance around the discussion if I may?
There are a ton of things we could do here but I think the need
is comparatively clear and pressing. Discussions are good where
the outcome is unknown and options need to be explored. I think
some of this is relatively clear for the reasons I'll mention below.

meta-doubleopen says "This meta layer is intended for use with 
Double Open's open source license compliance workflow". *license*
workflow, we're talking about SBOMs. The fact they produce SPDX 
files isn't all that's required to create an SBOM. SPDX is just
a file format. In fact there's nothing in that layer that says 
anything about SBOM. From what I can tell, all meta-doubleopen is 
producing is an SPDX version of the various manifest files one 
would find if buildhistory is enabled.

SPDX is only one of several file formats that can be used to 
generate an SBOM in a standard way. It could be worth a discussion 
to at least mention the others.
Can I suggest we adopt the position that we aim for SPDX unless someone
produces a strong argument that something else has advantages?

The reason I say this is that it is the standard most projects are
consolidating around, it shows alignment with other work at the LF
and SDPX is aiming to become an ISO standard. To do something different
would put us in a difficult position IMO. People complain that LF 
projects don't collaborate enough, here we have an opportunity I want
to make work.


We could look at what an SBOM is, and what are the minimum required 
fields to produce an SBOM.
We do need to find out what the legislation says about this so we can
meet it.
Why? OE nor YP sell to the US Government?  This seems to be more of a
concern for commercial vendors.  They are the ones who need to come to
the table and help support these efforts. Being this is such a new
thing, things will change.  Remember, this is the US Government, just
because the President said this, does not mean each Federal agency will
adopt it for verbatim. 

Another question for the round table: should we integrate this into 
oe-core, or leave it as a separate layer?
We need to be able to say that OE/YP generates SBOM manifests for
images out the box, preferably by default. If we don't do that, we will
lose out to projects which can claim this. I think that makes the 
decision clear.


The round table is also a great way of introducing this important topic
to the community at large. I bet you half the people attending the 
conference have never heard of an SBOM, but might be interested to 
know YP/OE is looking into integrating it into the build system 
especially now that the US government has released an Executive Order 
regarding SBOMs, and that the EU is also looking into these sorts of things as well).

I'll look into inviting the DoubleOpen people to the meeting.

Joshua mentioned that the company he works for is also investigating 
generating SBOMs from YP/OE builds, so let's make sure everyone is 
working on one project, instead of scattering the community.

So there are a couple things we could talk about :-)
I think aspects which do need discussion is how to handle:

* SPDX data at the do_package/do_packagedata level
* SPDX data at the archiver and do_populate_lic level
* whether we can replace existing image manifests
* whether we need tooling to take an SPDX image manifest and process it to 
various forms for end user/tool use (e.g. actual file output or API?).
Transitioning to a more accepted format is something YP/OE should
discuss but don't do it to meet the needs of the US Government.

-armin



This probably translates into some kind of plan with different phases.

Cheers,

Richard