[Webhob] task types and outcomes

Dragomir, CalinX L calinx.l.dragomir at intel.com
Thu May 30 06:10:43 PDT 2013

Hi everyone,

I have a couple of questions regarding the tasks if you could help me understand better what's going on in a build operation.

1. From what I've read in the e-mails so far, it seems that there are three types of tasks: built, sstate and skipped. Can you tell me what each of them mean and how are they different ?
2. It also seems that there can be multiple results for each of them. Can you detail the meaning of each final state? (failed, missed, etc.) ?
3. How can I tell what type of task is the current executed task (if it is sstate or built or anything else) ?

Thank you,

-----Original Message-----
From: Barros Pena, Belen 
Sent: Thursday, May 30, 2013 1:58 PM
To: Richard Purdie; Paul Eggleton
Cc: Damian, Alexandru; Dragomir, CalinX L; Zhang, Jessica; webhob at yoctoproject.org
Subject: Re: task types and outcomes

On 29/05/2013 22:39, "Richard Purdie" <richard.purdie at linuxfoundation.org>

>On Wed, 2013-05-29 at 16:57 +0100, Paul Eggleton wrote:
>> On Tuesday 28 May 2013 17:17:35 Barros Pena, Belen wrote:
>> > Thanks, Paul. I guess we need to nail down what is that we are 
>> > trying
>> > achieve by reporting sstate tasks vs. built tasks vs. previously 
>> > built tasks. From what I can remember, the main questions users 
>> > need
>> > are:
>> > 
>> > * Why is this a built task when I was expecting it to reuse sstate
>> > * Why is this an sstate task when I was expecting it to build from
>> These are common questions yes.
>> > If those are indeed the main questions we need to answer, reporting
>> > sstate tasks and built tasks as one probably makes things easier. 
>> > The metaphor would be a single task with a certain life cycle, and 
>> > not 2 separate tasks. So, "BitBake detected changes in the inputs 
>> > for task
>>x and
>> > reran the task" would result in Web Hob reporting a single built 
>> > task
>> > an sstate missed flag. If we report this case through 2 separate 
>> > tasks we'll need to link them to each other, so that users can 
>> > reach one
>> > the other and get a full picture of what BitBake did during the build.
>> > This is also a viable option, but it will result in a much longer
>>table of
>> > tasks, with 2 entries for a certain task + recipe combination.
>> > 
>> > Which way we report tasks impacts the way we structure the Web Hob
>> > table. From this classification of tasks:
>> > 
>> > * Sstate
>> > ** Missed
>> > ** Completed
>> > ** Failed
>> > 
>> > * Built
>> > ** Previously built
>> > ** Completed
>> > ** Failed
>> > 
>> > We would move to this one:
>> > 
>> > * Completed
>> > ** Previously built
>> > ** Sstate
>> > ** Built
>> > *** Sstate missed? Y/N
>> > *** Sstae failed? Y/N
>> > 
>> > * Failed
>> > ** Built
>> I think this is a reasonable approach. Although it does slightly 
>>obscure the  details of the tasks bitbake is running it's going to be 
>>a bit easier for the  user to follow with respect to the information 
>>they're most interested in.
>Sorry about the delayed reply, as you can tell I have a bit of a 
>backlog today :(.
>I hate to complicate things however there is a touch of added 
>complexity to consider.

>Imagine we have tasks A which depends upon B which depends upon C.
>In the no sstate available case, it builds C, then B then A, easy.
>With sstate, it will try A first. If A is available, it will just 
>install A from sstate and skip B/C. If A is not available, it will try 
>B, install that and skip C. It will then build any missing pieces so it 
>might install B from sstate, then run C.

I see. So it seems we have also 'skipped' tasks.

>This means that giving the user meaningful numbers of tasks is hard for 
>This can probably be presented in the above form fine, I just want to 
>be clear about the subtleties involved. Ideally we do want to try and 
>better convey to the user what happened too in this UI. It could be 
>good if for example we could show that we didn't run C as B was from 
>sstate and hence C wasn't needed.

I agree we should show those as well. So, it looks like we have a few different kinds of tasks (previously built, skipped, sstate and built), but those can be grouped into tasks that build something (the 'built'
kind, which we could call 'executed'), and those that don't build anything (previously built, skipped and sstate, which we could call 'not executed'). 

So for the 'all tasks' table, we could have a 'task type' column classifying tasks as 'executed' and 'not executed'. Then, we could have an 'outcome' column that would classify tasks as 'succeeded', 'failed', 'previously built', 'sstate' and 'skipped'. That information is enough to give me an overview of what BitBake has done, and if a task was handled in an unexpected way (I thought it would build but it didn't, for example).

When I select a single task from that table, we could provide a 'task history' section that would expose the BitBake process for that task (in your example of sstate dependent tasks, for skipped task C the history would tell me that task C was brought in by skipped task B, which in turn was brought in by sstate task A. For sstate task A, the history would tell me that the checksum did not change, that task A depended on skipped task B, which path was searched to find the sstate object, and so on).

Would that work?

>I think the key concept we might also need to work into the UI is "dry 
>run". A mode where you run the build however rather than doing it, it 
>tells you what it would do (i.e. that it would need to run X tasks, Y 
>would be done from sstate). Combined with an indication of why 
>something didn't match, this would be extremely powerful/useful to most users.
>I have no objection to the form you present above, the key detail is 
>more the dry run concept from the user perspective and start answering 
>questions like "what would bitbake do?".

We should keep that 'dry run' mode in mind (even I can see it would be incredibly useful), but it won't happen in 1.5 :(


More information about the toaster mailing list