Date   

Yocto Project Newcomer & Unassigned Bugs - Help Needed

Stephen Jolley
 

All,

 

The triage team is starting to try and collect up and classify bugs which a newcomer to the project would be able to work on in a way which means people can find them. They're being listed on the triage page under the appropriate heading:

https://wiki.yoctoproject.org/wiki/Bug_Triage#Newcomer_Bugs  Also please review: https://www.openembedded.org/wiki/How_to_submit_a_patch_to_OpenEmbedded and how to create a bugzilla account at: https://bugzilla.yoctoproject.org/createaccount.cgi

The idea is these bugs should be straight forward for a person to help work on who doesn't have deep experience with the project.  If anyone can help, please take ownership of the bug and send patches!  If anyone needs help/advice there are people on irc who can likely do so, or some of the more experienced contributors will likely be happy to help too.

 

Also, the triage team meets weekly and does its best to handle the bugs reported into the Bugzilla. The number of people attending that meeting has fallen, as have the number of people available to help fix bugs. One of the things we hear users report is they don't know how to help. We (the triage team) are therefore going to start reporting out the currently 360 unassigned or newcomer bugs.

 

We're hoping people may be able to spare some time now and again to help out with these.  Bugs are split into two types, "true bugs" where things don't work as they should and "enhancements" which are features we'd want to add to the system.  There are also roughly four different "priority" classes right now, “3.2”, “3.3, "3.99" and "Future", the more pressing/urgent issues being in "3.2" and then “3.3”.

 

Please review this link and if a bug is something you would be able to help with either take ownership of the bug, or send me (sjolley.yp.pm@...) an e-mail with the bug number you would like and I will assign it to you (please make sure you have a Bugzilla account).  The list is at: https://wiki.yoctoproject.org/wiki/Bug_Triage_Archive#Unassigned_or_Newcomer_Bugs

 

Thanks,

 

Stephen K. Jolley

Yocto Project Program Manager

(    Cell:                (208) 244-4460

* Email:              sjolley.yp.pm@...

 


[yocto-autobuilder-helper] [PATCH] config.json: Use pregen-hostkeys on all qemu targets

Richard Purdie
 

Rather than just ppc/mips, use the pregen-hostkeys on all the qemu targets
since this is using a lot of time on the autobuilders when we don't really
need to. This should avoid some of the testing failures seen on qemuarm
recently.

Signed-off-by: Richard Purdie <richard.purdie@...>
---
config.json | 18 ++++++------------
1 file changed, 6 insertions(+), 12 deletions(-)

diff --git a/config.json b/config.json
index 4f6044e..1c52d60 100644
--- a/config.json
+++ b/config.json
@@ -71,6 +71,9 @@
"arch-qemu" : {
"BUILDINFO" : true,
"BUILDHISTORY" : true,
+ "extravars" : [
+ "IMAGE_INSTALL_append = ' ssh-pregen-hostkeys'"
+ ],
"step1" : {
"BBTARGETS" : "core-image-sato core-image-sato-sdk core-image-minimal core-image-minimal-dev core-image-sato:do_populate_sdk",
"SANITYTARGETS" : "core-image-minimal:do_testimage core-image-sato:do_testimage core-image-sato-sdk:do_testimage core-image-sato:do_testsdk"
@@ -91,6 +94,9 @@
"DISTRO" : "poky-altcfg",
"BUILDINFO" : true,
"BUILDHISTORY" : true,
+ "extravars" : [
+ "IMAGE_INSTALL_append = ' ssh-pregen-hostkeys'"
+ ],
"step1" : {
"BBTARGETS" : "core-image-full-cmdline core-image-sato core-image-sato-sdk",
"SANITYTARGETS" : "core-image-full-cmdline:do_testimage core-image-sato:do_testimage core-image-sato-sdk:do_testimage"
@@ -383,16 +389,10 @@
},
"qemumips" : {
"MACHINE" : "qemumips",
- "extravars" : [
- "IMAGE_INSTALL_append = ' ssh-pregen-hostkeys'"
- ],
"TEMPLATE" : "arch-qemu"
},
"qemumips-alt" : {
"MACHINE" : "qemumips",
- "extravars" : [
- "IMAGE_INSTALL_append = ' ssh-pregen-hostkeys'"
- ],
"TEMPLATE" : "altcfg-qemu"
},
"edgerouter" : {
@@ -409,16 +409,10 @@
},
"qemuppc" : {
"MACHINE" : "qemuppc",
- "extravars" : [
- "IMAGE_INSTALL_append = ' ssh-pregen-hostkeys'"
- ],
"TEMPLATE" : "arch-qemu"
},
"qemuppc-alt" : {
"MACHINE" : "qemuppc",
- "extravars" : [
- "IMAGE_INSTALL_append = ' ssh-pregen-hostkeys'"
- ],
"TEMPLATE" : "altcfg-qemu"
},
"qemux86" : {
--
2.30.2


Re: [meta-spdxscanner] Question about meta-spdxscanner

Marco Cavallini
 

On 13/07/21 10:57, leimaohui@... wrote:
Hi Marco

I see that meta-spdxscanner has been moved to http://git.yoctoproject.org,
and doesn't maintained on github
Yes, you can contact me or contribute to meta-spdxscanner to this ML.

I'd like to have advice from you to understand if is it possible to test it without
any external service and discover what kind of artefacts are generated into
deploy/spdx.
I have not maintained scancode-tk for long time. And recently there are somebody else asked me about scancode-tk.
Yes, I planned to make scancode-tk.bbclass work without external service. But there are always issues because environment.
The problem of scancode-tk.bbclass that scancode must work with specify a version of python(latest requires 3.6), but you know that for YP or your host, it is hard to meet the requirement.
I found the newest version of scancode-tk support running in docker. So I plan to make scancode-tk.bbclass work with scancode command by docker next.
Of course, if you have good ideas, please tell me, or contribute to meta-spdxscanner directlly.
By the way, why not try fossology? It is really good that you can browse the compliance information on fossology server after you get spdx files by bitbake of YP.
Best regards
Lei

Hi Lei,
thank you for answering.

Considering the problems I encounter with scancode-tk and that the artifacts it produces are simple text files that need further analysis, I was just deciding to migrate to Fossology with "fossology-rest".
The only drawback is having to install the server, but I don't think it will be a problem.

Thanks again, I will contact you again if you have any problems with this mode.

Best,
--
Marco Cavallini | KOAN sas
Bergamo - Italia
embedded software engineering
https://KoanSoftware.com


Re: [meta-spdxscanner] Question about meta-spdxscanner

leimaohui
 

Hi Marco

I see that meta-spdxscanner has been moved to http://git.yoctoproject.org,
and doesn't maintained on github
Yes, you can contact me or contribute to meta-spdxscanner to this ML.

I'd like to have advice from you to understand if is it possible to test it without
any external service and discover what kind of artefacts are generated into
deploy/spdx.
I have not maintained scancode-tk for long time. And recently there are somebody else asked me about scancode-tk.
Yes, I planned to make scancode-tk.bbclass work without external service. But there are always issues because environment.
The problem of scancode-tk.bbclass that scancode must work with specify a version of python(latest requires 3.6), but you know that for YP or your host, it is hard to meet the requirement.
I found the newest version of scancode-tk support running in docker. So I plan to make scancode-tk.bbclass work with scancode command by docker next.
Of course, if you have good ideas, please tell me, or contribute to meta-spdxscanner directlly.

By the way, why not try fossology? It is really good that you can browse the compliance information on fossology server after you get spdx files by bitbake of YP.

Best regards
Lei


-----Original Message-----
From: yocto@... <yocto@...> On Behalf
Of Marco Cavallini
Sent: Monday, July 12, 2021 9:09 PM
To: yocto@...
Cc: Lei, Maohui <leimaohui@...>
Subject: [yocto] [meta-spdxscanner] Question about meta-spdxscanner

Hello,
I see that meta-spdxscanner has been moved to http://git.yoctoproject.org,
and doesn't maintained on github
anymore.https://github.com/dl9pf/meta-spdxscanner/issues/21

Therefore the only way to get in contact to developers is writing to this ML

I am trying to understand how meta-spdxscanner works.
I'd like to test it without any external service (no Fossology) therefore I am
trying the INHERIT += "scancode-tk" approach.

I am testing it with a dunfell version and I noticed a lot of errors so I switched to
master and the bitbake build started but again I am facing to a problem

ERROR: bash-completion-2.10-r0 do_get_report: Could not invoke scancode
Command

I'd like to have advice from you to understand if is it possible to test it without
any external service and discover what kind of artefacts are generated into
deploy/spdx.

Thank you

--
Marco


Per recipes simple smoke tests

Samuel Dolt
 

Hi everyone,

I was asked to add some basic tests to an internal recipe in order to improve our CI system.

The purpose would be to produce an error if some check doesn't pass like
- a file is not present
- the generated file is of size 0
- the generated file is bigger than x bytes

I'm thinking to add the functionality to a class, in order to be able to use these tests in more than one recipe. Something like that: https://gist.github.com/samdolt/cf1c557f73f4f2098a19ba7ad6bc092f

Does something like that already exists? And if not, would it be useful to have this contributed upstream? Maybe as an addition to insane.bbclass?

Cheers,
Samuel


Re: #yocto Preferred development workflow #yocto

Chuck Wolber
 

We have many recipes appended to IMAGE_INSTALL in our image recipes and it works just fine.

From your description, it sounds like you could go either way. I personally prefer to keep my image recipes as clean as possible, so I push as much as I can to individual recipes. This helps with debugging later on as your project (inevitably) gets more complex.

Also... Experience has shown that the following form is a lot more readable:

IMAGE_INSTALL  = "recipe_1"
IMAGE_INSTALL += "recipe_2"
IMAGE_INSTALL += "recipe_3"
IMAGE_INSTALL += "recipe_4"
IMAGE_INSTALL += "recipe_5"

Than this:

IMAGE_INSTALL = "\
         recipe_1 \
         recipe_2 \
         recipe_3 \
         recipe_4 \
         recipe_5"

When your repository grows large, a recursive grep for recipe_2 will make a lot more sense if "IMAGE_INSTALL += "recipe_2"" is returned instead of "recipe_2 \". The latter form gives no clue as to how recipe_2 is being referenced.


..Ch:W..


On Mon, Jul 12, 2021 at 9:36 PM ivin.lim via lists.yoctoproject.org <ivin.lim=aei.com@...> wrote:
sorry about that, right now the config files are placed in the files directory of the recipe which builds the module. 

Sample snippets of the recipe concerning this config file, I removed the rest of lines.
SRC_URI = " \
file://config.json \
    "

do_install_append() {
install -m 0644 ${WORKDIR}/config.json ${D}/etc/directory/
}


If I were to have the config files placed on a separate recipe, would that mean making multiple of these recipes depending on the number of projects? 
I did thought also of adding it in the image recipe but probably this wouldn't look well if more things get added.


Thanks






--
"Perfection must be reached by degrees; she requires the slow hand of time." - Voltaire


Re: #yocto Preferred development workflow #yocto

ivin.lim@...
 

sorry about that, right now the config files are placed in the files directory of the recipe which builds the module. 

Sample snippets of the recipe concerning this config file, I removed the rest of lines.
SRC_URI = " \
file://config.json \
    "

do_install_append() {
install -m 0644 ${WORKDIR}/config.json ${D}/etc/directory/
}


If I were to have the config files placed on a separate recipe, would that mean making multiple of these recipes depending on the number of projects? 
I did thought also of adding it in the image recipe but probably this wouldn't look well if more things get added.


Thanks


Re: #yocto Preferred development workflow #yocto

Chuck Wolber
 

You left a few details out about how the configuration files are managed. Ideally you could just make bitbake recipes for the config files themselves. This way you can create an image recipe for each project and use the IMAGE_INSTALL variable to append whatever configs and modules you want on a per-project basis.

..Ch:W..


On Mon, Jul 12, 2021 at 8:40 PM ivin.lim via lists.yoctoproject.org <ivin.lim=aei.com@...> wrote:
Hi everyone,

I'm curious on a good approach when dealing with multiple projects

I've just started out  yocto and have already  created our own meta-layer which contains our modules. Now these modules would sometimes have config files and this could vary when each project that we have would use these recipes. I'm thinking of a way to control this. And I don't think having one branch per project is a good style. Right now the best way I can think of is creating seperate "project layers" (ex. meta-project1, meta-project2) and from there create those .bbappends which would point to the project-specific conf files. 

But I was wondering if there is a better way of handling it. Thanks.

Cheers.




--
"Perfection must be reached by degrees; she requires the slow hand of time." - Voltaire


#yocto Preferred development workflow #yocto

ivin.lim@...
 

Hi everyone,

I'm curious on a good approach when dealing with multiple projects

I've just started out  yocto and have already  created our own meta-layer which contains our modules. Now these modules would sometimes have config files and this could vary when each project that we have would use these recipes. I'm thinking of a way to control this. And I don't think having one branch per project is a good style. Right now the best way I can think of is creating seperate "project layers" (ex. meta-project1, meta-project2) and from there create those .bbappends which would point to the project-specific conf files. 

But I was wondering if there is a better way of handling it. Thanks.

Cheers.


Re: [layerindex-web][PATCH] models.py: extend max_length of name in YPCompatibleVersions to 100

Changqing Li
 

add Paul Eggleton

On 7/13/21 10:44 AM, Changqing Li wrote:
From: Changqing Li <changqing.li@...>

Now, YPCompatibleVersions's name is only designed for using version like
2.0 3.0, the max_length is 25. but we mostly use Codename for layer
compatibility, eg: LAYERSERIES_COMPAT_dpdk = "dunfell gatesgarth
hardknott", in this case, it's not enough to save the compatible
version. so extend it to 100.

Signed-off-by: Changqing Li <changqing.li@...>
---
 .../migrations/0045_yp_compatible_extend.py   | 24 +++++++++++++++++++
 layerindex/models.py                          |  2 +-
 2 files changed, 25 insertions(+), 1 deletion(-)
 create mode 100644 layerindex/migrations/0045_yp_compatible_extend.py

diff --git a/layerindex/migrations/0045_yp_compatible_extend.py b/layerindex/migrations/0045_yp_compatible_extend.py
new file mode 100644
index 0000000..3544b4b
--- /dev/null
+++ b/layerindex/migrations/0045_yp_compatible_extend.py
@@ -0,0 +1,24 @@
+# Generated by Django 2.2 on 2021-07-13 02:43
+
+from django.db import migrations, models
+import django.db.models.deletion
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('layerindex', '0044_extendedprovides'),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name='classicrecipe',
+            name='cover_layerbranch',
+            field=models.ForeignKey(blank=True, limit_choices_to={'branch__name': 'master'}, null=True, on_delete=django.db.models.deletion.SET_NULL, to='layerindex.LayerBranch', verbose_name='Covering layer'),
+        ),
+        migrations.AlterField(
+            model_name='ypcompatibleversion',
+            name='name',
+            field=models.CharField(help_text='Name of this Yocto Project compatible version (e.g. "2.0")', max_length=100, unique=True, verbose_name='Yocto Project Version'),
+        ),
+    ]
diff --git a/layerindex/models.py b/layerindex/models.py
index 329cc33..2317740 100644
--- a/layerindex/models.py
+++ b/layerindex/models.py
@@ -217,7 +217,7 @@ class LayerRecipeExtraURL(models.Model):
 
 
 class YPCompatibleVersion(models.Model):
-    name = models.CharField('Yocto Project Version', max_length=25, unique=True, help_text='Name of this Yocto Project compatible version (e.g. "2.0")')
+    name = models.CharField('Yocto Project Version', max_length=100, unique=True, help_text='Name of this Yocto Project compatible version (e.g. "2.0")')
     description = models.TextField(blank=True)
     image_url = models.CharField('Image URL', max_length=300, blank=True)
     link_url = models.CharField('Link URL', max_length=100, blank=True)




[layerindex-web][PATCH] models.py: extend max_length of name in YPCompatibleVersions to 100

Changqing Li
 

From: Changqing Li <changqing.li@...>

Now, YPCompatibleVersions's name is only designed for using version like
2.0 3.0, the max_length is 25. but we mostly use Codename for layer
compatibility, eg: LAYERSERIES_COMPAT_dpdk = "dunfell gatesgarth
hardknott", in this case, it's not enough to save the compatible
version. so extend it to 100.

Signed-off-by: Changqing Li <changqing.li@...>
---
.../migrations/0045_yp_compatible_extend.py | 24 +++++++++++++++++++
layerindex/models.py | 2 +-
2 files changed, 25 insertions(+), 1 deletion(-)
create mode 100644 layerindex/migrations/0045_yp_compatible_extend.py

diff --git a/layerindex/migrations/0045_yp_compatible_extend.py b/layerindex/migrations/0045_yp_compatible_extend.py
new file mode 100644
index 0000000..3544b4b
--- /dev/null
+++ b/layerindex/migrations/0045_yp_compatible_extend.py
@@ -0,0 +1,24 @@
+# Generated by Django 2.2 on 2021-07-13 02:43
+
+from django.db import migrations, models
+import django.db.models.deletion
+
+
+class Migration(migrations.Migration):
+
+ dependencies = [
+ ('layerindex', '0044_extendedprovides'),
+ ]
+
+ operations = [
+ migrations.AlterField(
+ model_name='classicrecipe',
+ name='cover_layerbranch',
+ field=models.ForeignKey(blank=True, limit_choices_to={'branch__name': 'master'}, null=True, on_delete=django.db.models.deletion.SET_NULL, to='layerindex.LayerBranch', verbose_name='Covering layer'),
+ ),
+ migrations.AlterField(
+ model_name='ypcompatibleversion',
+ name='name',
+ field=models.CharField(help_text='Name of this Yocto Project compatible version (e.g. "2.0")', max_length=100, unique=True, verbose_name='Yocto Project Version'),
+ ),
+ ]
diff --git a/layerindex/models.py b/layerindex/models.py
index 329cc33..2317740 100644
--- a/layerindex/models.py
+++ b/layerindex/models.py
@@ -217,7 +217,7 @@ class LayerRecipeExtraURL(models.Model):


class YPCompatibleVersion(models.Model):
- name = models.CharField('Yocto Project Version', max_length=25, unique=True, help_text='Name of this Yocto Project compatible version (e.g. "2.0")')
+ name = models.CharField('Yocto Project Version', max_length=100, unique=True, help_text='Name of this Yocto Project compatible version (e.g. "2.0")')
description = models.TextField(blank=True)
image_url = models.CharField('Image URL', max_length=300, blank=True)
link_url = models.CharField('Link URL', max_length=100, blank=True)
--
2.17.1


wic automount of a data partition #zeus

Gary Huband
 

I'm using Zeus.  I'm trying to create and mount a data partition using wic.  My wks file is

part u-boot --source rawcopy --sourceparams="file=u-boot.imx" --ondisk mmc --no-table --align 1
part /boot --source bootimg-partition --ondisk mmc --fstype=vfat --label bootA --active --align 4096 --size 20
part / --source rootfs --ondisk mmc --fstype=ext4 --label rootfsA --align 4096 --size 4096
part /boot --source bootimg-partition --ondisk mmc --fstype=vfat --label bootB --size 20
part / --source rootfs --ondisk mmc --fstype=ext4 --label rootfsB --align 4096 --size 4096
part /data --ondisk mmc --fstype=ext4 --fsoptions="defaults" --label data --align 4096 --size 2048
part --ondisk mmc --fstype=ext4 --label recovery --align 4096 --size 1024

This creates the data on partition 6 but does not mount create the /data directory nor mount the partition.
If I manually create /data and manually modify the /etc/fstab file everything works.

Any suggestions for fixing the wks file?

Thanks

Gary


Re: [yocto-autobuilder-helper][PATCH] config.json: track system load with PARALLEL_MAKE

Trevor Gamblin
 


On 2021-07-12 12:04 p.m., Alexander Kanavin wrote:

[Please note: This e-mail is from an EXTERNAL e-mail address]

I seem to vaguely remember that -l is not actually taking a percentage, but an absolute value that is specific to the amount of CPU cores a system has. Have you verified your assumption?
You are right that it's not actually a percentage and is instead tied to the number of cores. I'll correct the patch body to reflect this when I inevitably send a v2. With the latest testing on the AB, my suspicion that "-l 100" is too generous for most machines is being confirmed...

Alex

On Mon, 12 Jul 2021 at 14:11, Trevor Gamblin <trevor.gamblin@...> wrote:
This adds the "-l" option to PARALLEL_MAKE in config.json with an
initial testing value of 100 (100% system load). This option is supported
by both Make and Ninja. However, we also require the "--debug=j" option
to be passed to Make in order for the latter to report perceived system
load in the do_compile logs, and since this option is not supported by
Ninja, also add EXTRA_OEMAKE to the EXTRAVARS so that we can determine if
the target load percentage needs to be adjusted.

Signed-off-by: Trevor Gamblin <trevor.gamblin@...>
---
 config.json | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/config.json b/config.json
index f54081b..7fc89ea 100644
--- a/config.json
+++ b/config.json
@@ -44,7 +44,7 @@
             "PREMIRRORS = ''",
             "BB_GENERATE_MIRROR_TARBALLS = '1'",
             "BB_NUMBER_THREADS = '16'",
-            "PARALLEL_MAKE = '-j 16'",
+            "PARALLEL_MAKE = '-j 16 -l 100'",
             "XZ_MEMLIMIT = '5%'",
             "XZ_THREADS = '8'",
             "BB_TASK_NICE_LEVEL = '5'",
@@ -61,7 +61,8 @@
             "RUNQEMU_TMPFS_DIR = '/home/pokybuild/tmp'",
             "BB_HEARTBEAT_EVENT = '60'",
             "BB_LOG_HOST_STAT_ON_INTERVAL = '1'",
-            "BB_LOG_HOST_STAT_CMDS_INTERVAL = 'oe-time-dd-test.sh 100'"
+            "BB_LOG_HOST_STAT_CMDS_INTERVAL = 'oe-time-dd-test.sh 100'",
+            "EXTRA_OEMAKE = ' --debug=j'"
         ]
     },
     "templates" : {
--
2.31.1





Reminder: Yocto Project Technical Team Meeting @ Monthly from 8am on the first Tuesday (PDT)

Stephen Jolley
 

All,

 

Just a reminder we will hold the monthly Yocto Project Technical Meeting at 8am PST tomorrow. (7/13) 

 

Yocto Project Technical Team Meeting: We encourage people attending the meeting to logon and announce themselves on the Yocto Project IRC chancel during the meeting (optional):

Yocto IRC: http://webchat.freenode.net/?channels=#yocto

 

Wiki: https://www.yoctoproject.org/public-virtual-meetings/

 

When            Monthly from 8am to 9am on the first Tuesday Pacific Time

Where           Zoom Meeting: https://zoom.us/j/990892712?pwd=cHU1MjhoM2x6ck81bkcrYjRrcmJsUT09

 

We are tracking the minutes at: https://docs.google.com/document/d/1ly8nyhO14kDNnFcW2QskANXW3ZT7QwKC5wWVDg9dDH4/edit?pli=1 Please request access if you want to assist in editing them.  The world should have view access.

 

Thanks,

 

Stephen K. Jolley

Yocto Project Program Manager

(    Cell:                (208) 244-4460

* Email:              sjolley.yp.pm@...

 


Re: [yocto-autobuilder-helper][PATCH] config.json: track system load with PARALLEL_MAKE

Alexander Kanavin
 

I seem to vaguely remember that -l is not actually taking a percentage, but an absolute value that is specific to the amount of CPU cores a system has. Have you verified your assumption?

Alex


On Mon, 12 Jul 2021 at 14:11, Trevor Gamblin <trevor.gamblin@...> wrote:
This adds the "-l" option to PARALLEL_MAKE in config.json with an
initial testing value of 100 (100% system load). This option is supported
by both Make and Ninja. However, we also require the "--debug=j" option
to be passed to Make in order for the latter to report perceived system
load in the do_compile logs, and since this option is not supported by
Ninja, also add EXTRA_OEMAKE to the EXTRAVARS so that we can determine if
the target load percentage needs to be adjusted.

Signed-off-by: Trevor Gamblin <trevor.gamblin@...>
---
 config.json | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/config.json b/config.json
index f54081b..7fc89ea 100644
--- a/config.json
+++ b/config.json
@@ -44,7 +44,7 @@
             "PREMIRRORS = ''",
             "BB_GENERATE_MIRROR_TARBALLS = '1'",
             "BB_NUMBER_THREADS = '16'",
-            "PARALLEL_MAKE = '-j 16'",
+            "PARALLEL_MAKE = '-j 16 -l 100'",
             "XZ_MEMLIMIT = '5%'",
             "XZ_THREADS = '8'",
             "BB_TASK_NICE_LEVEL = '5'",
@@ -61,7 +61,8 @@
             "RUNQEMU_TMPFS_DIR = '/home/pokybuild/tmp'",
             "BB_HEARTBEAT_EVENT = '60'",
             "BB_LOG_HOST_STAT_ON_INTERVAL = '1'",
-            "BB_LOG_HOST_STAT_CMDS_INTERVAL = 'oe-time-dd-test.sh 100'"
+            "BB_LOG_HOST_STAT_CMDS_INTERVAL = 'oe-time-dd-test.sh 100'",
+            "EXTRA_OEMAKE = ' --debug=j'"
         ]
     },
     "templates" : {
--
2.31.1





Re: Yocto Autobuilder: Latency Monitor and AB-INT - Meeting notes: July 8, 2021

Trevor Gamblin
 

On 2021-07-08 9:53 a.m., Randy MacLeod wrote:

YP AB Intermittent failures meeting
===================================
July 1, 2021, 9 AM ET
https://windriver.zoom.us/j/3696693975

Attendees:  Tony, Richard, Trevor, Randy


Summary:

========



The autobuilder RCU hang is still fixed ;-),
   master branch builds greener.

ptest failures are the top problem now.



Add Michael Halstead, see questions below in section 4.



If anyone wants to help, we could use more eyes on the logs,

particularly the summary logs and understanding iostat #

when the dd test times out.





Plans for the week:



===================





  Richard: glibc upgrade, etc.



  Alex: ?



  Sakib: pub/non-release link upgrade, script clean-up.



  Trevor: make job server test. Try it on YP AB!!! What type of build?



  Tony: fix/work-around valgrind ptest bug:
     none/tests/amd64/fb_test_amd64
  Saul: nothing this week for YP.



  Randy: vacation, then email catch-up!




Meeting Notes:

==============


1, runqemu

Tony having trouble with runqemu on some Wind River machine.

Richard has a fix for a race in runqemu in master-next.

These might be related but if not Tony should debug the
issue/ collect logs.

2. job server

- Trevor has conclusive evidence that the 'make' job server is useful.
  email summary to come. Need to fix some assumptoins in code that
  parses PARALLEL_MAKE then send patch to yocto-autobuilder-helper.

- ninja will have to be done next.
I've submitted an initial patch to yocto@.... My short notes to go along with it:

MAKE JOBSERVERS, SYSTEM LOADS, AND THE YOCTO AUTOBUILDER

INTRO

- Yocto Autobuilder has been experiencing several issues with nightly builds, some intermittent
- Previous discussions looking for ways to limit CPU/RAM loading (see " [yocto] bitbake controlling memory use" on yocto@... mailing list)
- Source packages built as part of YP builds rely on various build tools, including Make, Ninja. Can we use them to limit system resource loading, and therefore reduce build failures that occur near/beyond these limits?

MAKE JOBSERVER

- When the "-j" option is provided (e.g. by PARALLEL_MAKE and/or EXTRA_OEMAKE in local.conf), this tells Make to use that many job slots, e.g. PARALLEL_MAKE = "-j 10" tells make to use at most 10 job slots during compilation
- When the "-l" or "--max-load" option is provided (also via PARALLEL_MAKE or EXTRA_OEMAKE in local.conf), this tells Make not to start *more than one job* if the system load average (as perceived by Make) exceeds that value. If the system load exceeds the average specified with the "-l" argument and there is already a Make job running, no new ones are started until either the existing jobs finish, or the system load falls below the threshold.
- No limit by default

Sources:
https://www.gnu.org/software/make/manual/make.html#Parallel
http://make.mad-scientist.net/papers/jobserver-implementation/

PROPOSED SOLUTION

- "Just do it" - put a patch in for use with the Autobuilder and see what shakes out
- add 'PARALLEL_MAKE = "-j N -l M"' to the yocto-autobuilder-helper config.json default
- Complicated slightly by Ninja not supporting the same "--debug" functionality as Make - this is what has been used to track the system load in the initial testing. To get around this, we can temporarily set EXTRA_OEMAKE. Without "--debug", the "-l" option still works, but there is no output in the do_compile logs to tell us how the actual system load compares to the percentage targeted by the "-l" option

I did not see any issues in my manual testing with PARALLEL_MAKE and EXTRA_OEMAKE set in the same manner. It appears from a recursive grep through the poky repository that PARALLEL_MAKE gets referenced in various places, but it is usually either passed directly to the build tool of choice, or the argument to the "-j" flag is explicitly stripped out for use. An example of the former is meta/classes/meson.bbclass:

...

meson_do_compile() {
    ninja -v ${PARALLEL_MAKE}
}

...

It's possible that we will run into some edge cases of course, but I haven't seen them in my manual testing. Builds are running now at https://autobuilder.yoctoproject.org/typhoon/#/builders/85/builds/1478 , so we should have some initial results soon.

- Trevor



3. AB status

 generally better but...

 ptests are having some recurring problems.
 - parted - only on arm?
 - valgrind - none/tests/amd64/fb_test_amd64
 - gdb test failing again. - Randy!

4. Richard reported
   - something really flaky going on with serial ports.
   - particularly bad on qemuppc but also x86.
   - related to Saul's QMP data dump?

5. Sakib needs to send patch to make testimage failures
   generate summary logs.

6. Richard says that we may need to redesign the data collection system
   that Sakib's AB INT tests are based on.





Still relevant parts of
Previous Meeting Notes:
=======================

1. The qemu RCU hang has been fixed to not deadlock anymore!

It still  hangs at times but this dramatically reduces the
AB failures.



4. bitbake server timeout.

   "Timeout while waiting for a reply from the bitbake server (60s)"

   Randy mentioned that the bitbake server timeouts seen in the
   Wind River build cluster have gone away after upgrading to
   a newer version of docker.

   Old: Docker Version: Docker version 18.09.4, build d14af54266
   New: Docker Version: Docker version 20.10.7, build f0df350


   Clearly the YP ABs aren't running in docker but what
   about firmware and kernel tunings.

   Michael,

   Is the BIOS/firmware kept up to date on most nodes?

   It seems that we are running stock kernels which makes sense but
   given that we don't have concerns about privacy since system access
   is controlled and the nodes are being used to test open
   source software, we might consider optimizing for performance
   rather than security.

   Alex pointed at: https://make-linux-fast-again.com/
   Which just lists a set of kernel boot options:
      noibrs noibpb nopti nospectre_v2 nospectre_v1  \
      l1tf=off nospec_store_bypass_disable no_stf_barrier \
      mds=off tsx=on tsx_async_abort=off mitigations=off

   Can we enable some or all of these on a node to see what the
   performance difference is?


5. io stalls

   Richard said that it would make sense to write an ftrace utility
   / script to monitor io latency and we could install it with sudo
   Ch^W mentioned ftrace on IRC.
   Sakib and Randy will work on that but not for a week or two.

6. Switch the pub/non-release links from full log to summary.
   The host data links on:
     https://autobuilder.yocto.io/pub/non-release/
   should include links to the summary data. I think we have room to
   include both links like this:
   0 1 2 3  (Full: 0 1 2 3 )





../Randy


[meta-spdxscanner] Question about meta-spdxscanner

Marco Cavallini
 

Hello,
I see that meta-spdxscanner has been moved to http://git.yoctoproject.org, and doesn't maintained on github anymore.https://github.com/dl9pf/meta-spdxscanner/issues/21

Therefore the only way to get in contact to developers is writing to this ML

I am trying to understand how meta-spdxscanner works.
I'd like to test it without any external service (no Fossology) therefore I am trying the INHERIT += "scancode-tk" approach.

I am testing it with a dunfell version and I noticed a lot of errors so I switched to master and the bitbake build started but again I am facing to a problem

ERROR: bash-completion-2.10-r0 do_get_report: Could not invoke scancode Command

I'd like to have advice from you to understand if is it possible to test it without any external service and discover what kind of artefacts are generated into deploy/spdx.

Thank you

--
Marco


[yocto-autobuilder-helper][PATCH] config.json: track system load with PARALLEL_MAKE

Trevor Gamblin
 

This adds the "-l" option to PARALLEL_MAKE in config.json with an
initial testing value of 100 (100% system load). This option is supported
by both Make and Ninja. However, we also require the "--debug=j" option
to be passed to Make in order for the latter to report perceived system
load in the do_compile logs, and since this option is not supported by
Ninja, also add EXTRA_OEMAKE to the EXTRAVARS so that we can determine if
the target load percentage needs to be adjusted.

Signed-off-by: Trevor Gamblin <trevor.gamblin@...>
---
config.json | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/config.json b/config.json
index f54081b..7fc89ea 100644
--- a/config.json
+++ b/config.json
@@ -44,7 +44,7 @@
"PREMIRRORS = ''",
"BB_GENERATE_MIRROR_TARBALLS = '1'",
"BB_NUMBER_THREADS = '16'",
- "PARALLEL_MAKE = '-j 16'",
+ "PARALLEL_MAKE = '-j 16 -l 100'",
"XZ_MEMLIMIT = '5%'",
"XZ_THREADS = '8'",
"BB_TASK_NICE_LEVEL = '5'",
@@ -61,7 +61,8 @@
"RUNQEMU_TMPFS_DIR = '/home/pokybuild/tmp'",
"BB_HEARTBEAT_EVENT = '60'",
"BB_LOG_HOST_STAT_ON_INTERVAL = '1'",
- "BB_LOG_HOST_STAT_CMDS_INTERVAL = 'oe-time-dd-test.sh 100'"
+ "BB_LOG_HOST_STAT_CMDS_INTERVAL = 'oe-time-dd-test.sh 100'",
+ "EXTRA_OEMAKE = ' --debug=j'"
]
},
"templates" : {
--
2.31.1


Re: [meta-security][PATCH 2/2] apparmor: use its own initscript and service files

Armin Kuster
 

merged.

thanks for the reminder.

-armin

On 7/6/21 2:03 AM, Yi Zhao wrote:

Ping ...


On 6/23/21 5:15 PM, Yi Zhao wrote:
Use initscript and service files provided by apparmor.

Signed-off-by: Yi Zhao <yi.zhao@...>
---
recipes-mac/AppArmor/apparmor_3.0.1.bb | 33 +--
...x-hardcoded-installation-directories.patch | 51 ++++
...pparmor.debian-add-missing-functions.patch | 57 ++++
recipes-mac/AppArmor/files/apparmor | 226 ---------------
recipes-mac/AppArmor/files/apparmor.rc | 98 -------
recipes-mac/AppArmor/files/apparmor.service | 22 --
recipes-mac/AppArmor/files/functions | 271 ------------------
7 files changed, 118 insertions(+), 640 deletions(-)
create mode 100644 recipes-mac/AppArmor/files/0001-Makefile-fix-hardcoded-installation-directories.patch
create mode 100644 recipes-mac/AppArmor/files/0001-rc.apparmor.debian-add-missing-functions.patch
delete mode 100644 recipes-mac/AppArmor/files/apparmor
delete mode 100644 recipes-mac/AppArmor/files/apparmor.rc
delete mode 100644 recipes-mac/AppArmor/files/apparmor.service
delete mode 100644 recipes-mac/AppArmor/files/functions

diff --git a/recipes-mac/AppArmor/apparmor_3.0.1.bb b/recipes-mac/AppArmor/apparmor_3.0.1.bb
index 6377683..ff5b39b 100644
--- a/recipes-mac/AppArmor/apparmor_3.0.1.bb
+++ b/recipes-mac/AppArmor/apparmor_3.0.1.bb
@@ -15,15 +15,13 @@ DEPENDS = "bison-native apr gettext-native coreutils-native swig-native"

SRC_URI = " \
git://gitlab.com/apparmor/apparmor.git;protocol=https;branch=apparmor-3.0 \
+ file://run-ptest \
file://disable_perl_h_check.patch \
file://crosscompile_perl_bindings.patch \
- file://apparmor.rc \
- file://functions \
- file://apparmor \
- file://apparmor.service \
file://0001-Makefile.am-suppress-perllocal.pod.patch \
- file://run-ptest \
file://0001-Revert-profiles-Update-make-check-to-select-tools-ba.patch \
+ file://0001-Makefile-fix-hardcoded-installation-directories.patch \
+ file://0001-rc.apparmor.debian-add-missing-functions.patch \
"

SRCREV = "b0f08aa9d678197b8e3477c2fbff790f50a1de5e"
@@ -79,8 +77,6 @@ do_compile () {
}

do_install () {
- install -d ${D}/${INIT_D_DIR}
- install -d ${D}/lib/apparmor
oe_runmake -C ${B}/libraries/libapparmor DESTDIR="${D}" install
oe_runmake -C ${B}/binutils DESTDIR="${D}" install
oe_runmake -C ${B}/utils DESTDIR="${D}" install
@@ -96,16 +92,16 @@ do_install () {
fi

if ${@bb.utils.contains('DISTRO_FEATURES', 'pam', 'true', 'false', d)}; then
- install -d ${D}/lib/security
oe_runmake -C ${B}/changehat/pam_apparmor DESTDIR="${D}" install
fi

- install -m 755 ${WORKDIR}/apparmor ${D}/${INIT_D_DIR}/apparmor
- install -m 755 ${WORKDIR}/functions ${D}/lib/apparmor
+ if ${@bb.utils.contains('DISTRO_FEATURES','sysvinit','true','false',d)}; then
+ install -d ${D}${sysconfdir}/init.d
+ install -m 755 ${B}/parser/rc.apparmor.debian ${D}${sysconfdir}/init.d/apparmor
+ fi

if ${@bb.utils.contains('DISTRO_FEATURES','systemd','true','false',d)}; then
- install -d ${D}${systemd_system_unitdir}
- install -m 0644 ${WORKDIR}/apparmor.service ${D}${systemd_system_unitdir}
+ oe_runmake -C ${B}/parser DESTDIR="${D}" install-systemd
fi
}

@@ -152,15 +148,6 @@ do_install_ptest_arm() {
:
}

-pkg_postinst_ontarget_${PN} () {
-if [ ! -d /etc/apparmor.d/cache ] ; then
- mkdir /etc/apparmor.d/cache
-fi
-}
-
-# We need the init script so don't rm it
-RMINITDIR_class-target_remove = " rm_sysvinit_initddir"
-
INITSCRIPT_PACKAGES = "${PN}"
INITSCRIPT_NAME = "apparmor"
INITSCRIPT_PARAMS = "start 16 2 3 4 5 . stop 35 0 1 6 ."
@@ -171,9 +158,9 @@ SYSTEMD_AUTO_ENABLE ?= "enable"

PACKAGES += "mod-${PN}"

-FILES_${PN} += "/lib/apparmor/ /lib/security/ ${sysconfdir}/apparmor ${nonarch_libdir}/${PYTHON_DIR}/site-packages"
+FILES_${PN} += "${nonarch_base_libdir}/apparmor/ ${base_libdir}/security/ ${sysconfdir}/apparmor ${nonarch_libdir}/${PYTHON_DIR}/site-packages"
FILES_mod-${PN} = "${libdir}/apache2/modules/*"
-FILES_${PN}-dbg += "/lib/security/"
+FILES_${PN}-dbg += "${base_libdir}/security/.debug"

DEPENDS_append_libc-musl = " fts "
RDEPENDS_${PN}_libc-musl += "musl-utils"
diff --git a/recipes-mac/AppArmor/files/0001-Makefile-fix-hardcoded-installation-directories.patch b/recipes-mac/AppArmor/files/0001-Makefile-fix-hardcoded-installation-directories.patch
new file mode 100644
index 0000000..f10acb1
--- /dev/null
+++ b/recipes-mac/AppArmor/files/0001-Makefile-fix-hardcoded-installation-directories.patch
@@ -0,0 +1,51 @@
+From 363114dcd72abf1c0dcd637c66037227b8be229b Mon Sep 17 00:00:00 2001
+From: Yi Zhao <yi.zhao@...>
+Date: Mon, 21 Jun 2021 14:18:30 +0800
+Subject: [PATCH 1/2] Makefile: fix hardcoded installation directories
+
+Update the installation directories to fix the do_install error for
+multilib and usrmerge.
+
+Upstream-Status: Inappropriate [configuration]
+
+Signed-off-by: Yi Zhao <yi.zhao@...>
+---
+ changehat/pam_apparmor/Makefile | 2 +-
+ parser/Makefile | 8 ++++----
+ 2 files changed, 5 insertions(+), 5 deletions(-)
+
+diff --git a/changehat/pam_apparmor/Makefile b/changehat/pam_apparmor/Makefile
+index f6ece2d1..0143ae9f 100644
+--- a/changehat/pam_apparmor/Makefile
++++ b/changehat/pam_apparmor/Makefile
+@@ -77,7 +77,7 @@ $(NAME).so: ${OBJECTS}
+
+ # need some better way of determining this
+ DESTDIR=/
+-SECDIR ?= ${DESTDIR}/lib/security
++SECDIR ?= ${DESTDIR}/${base_libdir}/security
+
+ .PHONY: install
+ install: $(NAME).so
+diff --git a/parser/Makefile b/parser/Makefile
+index 8250ac45..cf18bc11 100644
+--- a/parser/Makefile
++++ b/parser/Makefile
+@@ -23,10 +23,10 @@ COMMONDIR=../common/
+ include $(COMMONDIR)/Make.rules
+
+ DESTDIR=/
+-APPARMOR_BIN_PREFIX=${DESTDIR}/lib/apparmor
+-SBINDIR=${DESTDIR}/sbin
+-USR_SBINDIR=${DESTDIR}/usr/sbin
+-SYSTEMD_UNIT_DIR=${DESTDIR}/usr/lib/systemd/system
++APPARMOR_BIN_PREFIX=${DESTDIR}/${nonarch_base_libdir}/apparmor
++SBINDIR=${DESTDIR}/${base_sbindir}
++USR_SBINDIR=${DESTDIR}/${sbindir}
++SYSTEMD_UNIT_DIR=${DESTDIR}/${systemd_system_unitdir}
+ CONFDIR=/etc/apparmor
+ INSTALL_CONFDIR=${DESTDIR}${CONFDIR}
+ LOCALEDIR=/usr/share/locale
+--
+2.17.1
+
diff --git a/recipes-mac/AppArmor/files/0001-rc.apparmor.debian-add-missing-functions.patch b/recipes-mac/AppArmor/files/0001-rc.apparmor.debian-add-missing-functions.patch
new file mode 100644
index 0000000..53bdde8
--- /dev/null
+++ b/recipes-mac/AppArmor/files/0001-rc.apparmor.debian-add-missing-functions.patch
@@ -0,0 +1,57 @@
+From a737c95ac0f887c365fe8f16583ea95da79de1e9 Mon Sep 17 00:00:00 2001
+From: Yi Zhao <yi.zhao@...>
+Date: Mon, 21 Jun 2021 16:53:39 +0800
+Subject: [PATCH] rc.apparmor.debian: add missing functions
+
+Add missing functions:
+ aa_log_action_start
+ aa_log_action_end
+ aa_log_daemon_msg
+ aa_log_end_msg
+
+Fixes:
+$ /etc/init.d/apparmor start
+/lib/apparmor/rc.apparmor.functions: line 294: aa_log_daemon_msg: command not found
+/lib/apparmor/rc.apparmor.functions: line 214: aa_log_action_start: command not found
+
+Upstream-Status: Pending
+
+Signed-off-by: Yi Zhao <yi.zhao@...>
+---
+ parser/rc.apparmor.debian | 20 ++++++++++++++++++++
+ 1 file changed, 20 insertions(+)
+
+diff --git a/parser/rc.apparmor.debian b/parser/rc.apparmor.debian
+index 8efd4400..f35124e8 100644
+--- a/parser/rc.apparmor.debian
++++ b/parser/rc.apparmor.debian
+@@ -70,6 +70,26 @@ aa_log_skipped_msg() {
+ echo ": Skipped."
+ }
+
++aa_log_action_start()
++{
++ echo "$@"
++}
++
++aa_log_action_end()
++{
++ printf ""
++}
++
++aa_log_daemon_msg()
++{
++ echo "$@"
++}
++
++aa_log_end_msg()
++{
++ printf ""
++}
++
+ usage() {
+ echo "Usage: $0 {start|stop|restart|try-restart|reload|force-reload|status|kill}"
+ }
+--
+2.17.1
+
diff --git a/recipes-mac/AppArmor/files/apparmor b/recipes-mac/AppArmor/files/apparmor
deleted file mode 100644
index 604e48d..0000000
--- a/recipes-mac/AppArmor/files/apparmor
+++ /dev/null
@@ -1,226 +0,0 @@
-#!/bin/sh
-# ----------------------------------------------------------------------
-# Copyright (c) 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007
-# NOVELL (All rights reserved)
-# Copyright (c) 2008, 2009 Canonical, Ltd.
-#
-# This program is free software; you can redistribute it and/or
-# modify it under the terms of version 2 of the GNU General Public
-# License published by the Free Software Foundation.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program; if not, contact Novell, Inc.
-# ----------------------------------------------------------------------
-# Authors:
-# Steve Beattie <steve.beattie@...>
-# Kees Cook <kees@...>
-#
-# /etc/init.d/apparmor
-#
-### BEGIN INIT INFO
-# Provides: apparmor
-# Required-Start: $local_fs
-# Required-Stop: umountfs
-# Default-Start: S
-# Default-Stop:
-# Short-Description: AppArmor initialization
-# Description: AppArmor init script. This script loads all AppArmor profiles.
-### END INIT INFO
-
-log_daemon_msg() {
- echo $*
-}
-
-log_end_msg () {
- retval=$1
- if [ $retval -eq 0 ]; then
- echo "."
- else
- echo " failed!"
- fi
- return $retval
-}
-
-. /lib/apparmor/functions
-
-usage() {
- echo "Usage: $0 {start|stop|restart|reload|force-reload|status|recache}"
-}
-
-test -x ${PARSER} || exit 0 # by debian policy
-# LSM is built-in, so it is either there or not enabled for this boot
-test -d /sys/module/apparmor || exit 0
-
-securityfs() {
- # Need securityfs for any mode
- if [ ! -d "${AA_SFS}" ]; then
- if cut -d" " -f2,3 /proc/mounts | grep -q "^${SECURITYFS} securityfs"'$' ; then
- log_daemon_msg "AppArmor not available as kernel LSM."
- log_end_msg 1
- exit 1
- else
- log_daemon_msg "Mounting securityfs on ${SECURITYFS}"
- if ! mount -t securityfs none "${SECURITYFS}"; then
- log_end_msg 1
- exit 1
- fi
- fi
- fi
- if [ ! -w "$AA_SFS"/.load ]; then
- log_daemon_msg "Insufficient privileges to change profiles."
- log_end_msg 1
- exit 1
- fi
-}
-
-handle_system_policy_package_updates() {
- apparmor_was_updated=0
-
- if ! compare_previous_version ; then
- # On snappy flavors, if the current and previous versions are
- # different then clear the system cache. snappy will handle
- # "$PROFILES_CACHE_VAR" itself (on Touch flavors
- # compare_previous_version always returns '0' since snappy
- # isn't available).
- clear_cache_system
- apparmor_was_updated=1
- elif ! compare_and_save_debsums apparmor ; then
- # If the system policy has been updated since the last time we
- # ran, clear the cache to prevent potentially stale binary
- # cache files after an Ubuntu image based upgrade (LP:
- # #1350673). This can be removed once all system image flavors
- # move to snappy (on snappy systems compare_and_save_debsums
- # always returns '0' since /var/lib/dpkg doesn't exist).
- clear_cache
- apparmor_was_updated=1
- fi
-
- if [ -x /usr/bin/aa-clickhook ] || [ -x /usr/bin/aa-profile-hook ] ; then
- # If packages for system policy that affect click packages have
- # been updated since the last time we ran, run aa-clickhook -f
- force_clickhook=0
- force_profile_hook=0
- if ! compare_and_save_debsums apparmor-easyprof-ubuntu ; then
- force_clickhook=1
- fi
- if ! compare_and_save_debsums apparmor-easyprof-ubuntu-snappy ; then
- force_clickhook=1
- fi
- if ! compare_and_save_debsums click-apparmor ; then
- force_clickhook=1
- force_profile_hook=1
- fi
- if [ -x /usr/bin/aa-clickhook ] && ([ $force_clickhook -eq 1 ] || [ $apparmor_was_updated -eq 1 ]) ; then
- aa-clickhook -f
- fi
- if [ -x /usr/bin/aa-profile-hook ] && ([ $force_profile_hook -eq 1 ] || [ $apparmor_was_updated -eq 1 ]) ; then
- aa-profile-hook -f
- fi
- fi
-}
-
-# Allow "recache" even when running on the liveCD
-if [ "$1" = "recache" ]; then
- log_daemon_msg "Recaching AppArmor profiles"
- recache_profiles
- rc=$?
- log_end_msg "$rc"
- exit $rc
-fi
-
-# do not perform start/stop/reload actions when running from liveCD
-test -d /rofs/etc/apparmor.d && exit 0
-
-rc=255
-case "$1" in
- start)
- if test -x /sbin/systemd-detect-virt && \
- systemd-detect-virt --quiet --container && \
- ! is_container_with_internal_policy; then
- log_daemon_msg "Not starting AppArmor in container"
- log_end_msg 0
- exit 0
- fi
- log_daemon_msg "Starting AppArmor profiles"
- securityfs
- # That is only useful for click, snappy and system images,
- # i.e. not in Debian. And it reads and writes to /var, that
- # can be remote-mounted, so it would prevent us from using
- # Before=sysinit.target without possibly introducing dependency
- # loops.
- handle_system_policy_package_updates
- load_configured_profiles
- rc=$?
- log_end_msg "$rc"
- ;;
- stop)
- log_daemon_msg "Clearing AppArmor profiles cache"
- clear_cache
- rc=$?
- log_end_msg "$rc"
- cat >&2 <<EOM
-All profile caches have been cleared, but no profiles have been unloaded.
-Unloading profiles will leave already running processes permanently
-unconfined, which can lead to unexpected situations.
-
-To set a process to complain mode, use the command line tool
-'aa-complain'. To really tear down all profiles, run the init script
-with the 'teardown' option."
-EOM
- ;;
- teardown)
- if test -x /sbin/systemd-detect-virt && \
- systemd-detect-virt --quiet --container && \
- ! is_container_with_internal_policy; then
- log_daemon_msg "Not tearing down AppArmor in container"
- log_end_msg 0
- exit 0
- fi
- log_daemon_msg "Unloading AppArmor profiles"
- securityfs
- running_profile_names | while read profile; do
- if ! unload_profile "$profile" ; then
- log_end_msg 1
- exit 1
- fi
- done
- rc=0
- log_end_msg $rc
- ;;
- restart|reload|force-reload)
- if test -x /sbin/systemd-detect-virt && \
- systemd-detect-virt --quiet --container && \
- ! is_container_with_internal_policy; then
- log_daemon_msg "Not reloading AppArmor in container"
- log_end_msg 0
- exit 0
- fi
- log_daemon_msg "Reloading AppArmor profiles"
- securityfs
- clear_cache
- load_configured_profiles
- rc=$?
- unload_obsolete_profiles
-
- log_end_msg "$rc"
- ;;
- status)
- securityfs
- if [ -x /usr/sbin/aa-status ]; then
- aa-status --verbose
- else
- cat "$AA_SFS"/profiles
- fi
- rc=$?
- ;;
- *)
- usage
- rc=1
- ;;
- esac
-exit $rc
diff --git a/recipes-mac/AppArmor/files/apparmor.rc b/recipes-mac/AppArmor/files/apparmor.rc
deleted file mode 100644
index 1507d7b..0000000
--- a/recipes-mac/AppArmor/files/apparmor.rc
+++ /dev/null
@@ -1,98 +0,0 @@
-description "Pre-cache and pre-load apparmor profiles"
-author "Dimitri John Ledkov <xnox@...> and Jamie Strandboge <jamie@...>"
-
-task
-
-start on starting rc-sysinit
-
-script
- [ -d /rofs/etc/apparmor.d ] && exit 0 # do not load on liveCD
- [ -d /sys/module/apparmor ] || exit 0 # do not load without AppArmor
- [ -x /sbin/apparmor_parser ] || exit 0 # do not load without parser
-
- . /lib/apparmor/functions
-
- systemd-detect-virt --quiet --container && ! is_container_with_internal_policy && exit 0 || true
-
- # Need securityfs for any mode
- if [ ! -d /sys/kernel/security/apparmor ]; then
- if cut -d" " -f2,3 /proc/mounts | grep -q "^/sys/kernel/security securityfs"'$' ; then
- exit 0
- else
- mount -t securityfs none /sys/kernel/security || exit 0
- fi
- fi
-
- [ -w /sys/kernel/security/apparmor/.load ] || exit 0
-
- apparmor_was_updated=0
- if ! compare_previous_version ; then
- # On snappy flavors, if the current and previous versions are
- # different then clear the system cache. snappy will handle
- # "$PROFILES_CACHE_VAR" itself (on Touch flavors
- # compare_previous_version always returns '0' since snappy
- # isn't available).
- clear_cache_system
- apparmor_was_updated=1
- elif ! compare_and_save_debsums apparmor ; then
- # If the system policy has been updated since the last time we
- # ran, clear the cache to prevent potentially stale binary
- # cache files after an Ubuntu image based upgrade (LP:
- # #1350673). This can be removed once all system image flavors
- # move to snappy (on snappy systems compare_and_save_debsums
- # always returns '0' since /var/lib/dpkg doesn't exist).
- clear_cache
- apparmor_was_updated=1
- fi
-
- if [ -x /usr/bin/aa-clickhook ] || [ -x /usr/bin/aa-profile-hook ] ; then
- # If packages for system policy that affect click packages have
- # been updated since the last time we ran, run aa-clickhook -f
- force_clickhook=0
- force_profile_hook=0
- if ! compare_and_save_debsums apparmor-easyprof-ubuntu ; then
- force_clickhook=1
- fi
- if ! compare_and_save_debsums apparmor-easyprof-ubuntu-snappy ; then
- force_clickhook=1
- fi
- if ! compare_and_save_debsums click-apparmor ; then
- force_clickhook=1
- force_profile_hook=1
- fi
- if [ -x /usr/bin/aa-clickhook ] && ([ $force_clickhook -eq 1 ] || [ $apparmor_was_updated -eq 1 ]) ; then
- aa-clickhook -f
- fi
- if [ -x /usr/bin/aa-profile-hook ] && ([ $force_profile_hook -eq 1 ] || [ $apparmor_was_updated -eq 1 ]) ; then
- aa-profile-hook -f
- fi
- fi
-
- if [ "$ACTION" = "teardown" ]; then
- running_profile_names | while read profile; do
- unload_profile "$profile"
- done
- exit 0
- fi
-
- if [ "$ACTION" = "clear" ]; then
- clear_cache
- exit 0
- fi
-
- if [ "$ACTION" = "reload" ] || [ "$ACTION" = "force-reload" ]; then
- clear_cache
- load_configured_profiles
- unload_obsolete_profiles
- exit 0
- fi
-
- # Note: if apparmor-easyprof-ubuntu md5sums didn't match up above,
- # aa-clickhook will have already compiled the policy, generated the cache
- # files and loaded them into the kernel by this point, so reloading click
- # policy from cache, while fairly fast (<2 seconds for 250 profiles on
- # armhf), is redundant. Fixing this would complicate the logic quite a bit
- # and it wouldn't improve the (by far) common case (ie, when
- # 'aa-clickhook -f' is not run).
- load_configured_profiles
-end script
diff --git a/recipes-mac/AppArmor/files/apparmor.service b/recipes-mac/AppArmor/files/apparmor.service
deleted file mode 100644
index e66afe4..0000000
--- a/recipes-mac/AppArmor/files/apparmor.service
+++ /dev/null
@@ -1,22 +0,0 @@
-[Unit]
-Description=AppArmor initialization
-After=local-fs.target
-Before=sysinit.target
-AssertPathIsReadWrite=/sys/kernel/security/apparmor/.load
-ConditionSecurity=apparmor
-DefaultDependencies=no
-Documentation=man:apparmor(7)
-Documentation=http://wiki.apparmor.net/
-
-# Don't start this unit on the Ubuntu Live CD
-ConditionPathExists=!/rofs/etc/apparmor.d
-
-[Service]
-Type=oneshot
-RemainAfterExit=yes
-ExecStart=/etc/init.d/apparmor start
-ExecStop=/etc/init.d/apparmor stop
-ExecReload=/etc/init.d/apparmor reload
-
-[Install]
-WantedBy=sysinit.target
diff --git a/recipes-mac/AppArmor/files/functions b/recipes-mac/AppArmor/files/functions
deleted file mode 100644
index e9e2bbf..0000000
--- a/recipes-mac/AppArmor/files/functions
+++ /dev/null
@@ -1,271 +0,0 @@
-# /lib/apparmor/functions for Debian -*- shell-script -*-
-# ----------------------------------------------------------------------
-# Copyright (c) 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007
-# NOVELL (All rights reserved)
-# Copyright (c) 2008-2010 Canonical, Ltd.
-#
-# This program is free software; you can redistribute it and/or
-# modify it under the terms of version 2 of the GNU General Public
-# License published by the Free Software Foundation.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program; if not, contact Novell, Inc.
-# ----------------------------------------------------------------------
-# Authors:
-# Kees Cook <kees@...>
-
-PROFILES="/etc/apparmor.d"
-PROFILES_CACHE="$PROFILES/cache"
-PROFILES_VAR="/var/lib/apparmor/profiles"
-PROFILES_SNAPPY="/var/lib/snapd/apparmor/profiles"
-PROFILES_CACHE_VAR="/var/cache/apparmor"
-PARSER="/sbin/apparmor_parser"
-SECURITYFS="/sys/kernel/security"
-export AA_SFS="$SECURITYFS/apparmor"
-
-# Suppress warnings when booting in quiet mode
-quiet_arg=""
-[ "${QUIET:-no}" = yes ] && quiet_arg="-q"
-[ "${quiet:-n}" = y ] && quiet_arg="-q"
-
-foreach_configured_profile() {
- rc_all="0"
- for pdir in "$PROFILES" "$PROFILES_VAR" "$PROFILES_SNAPPY" ; do
- if [ ! -d "$pdir" ]; then
- continue
- fi
- num=`find "$pdir" -type f ! -name '*.md5sums' | wc -l`
- if [ "$num" = "0" ]; then
- continue
- fi
-
- cache_dir="$PROFILES_CACHE"
- if [ -d "$PROFILES_CACHE_VAR" ] && [ "$pdir" = "$PROFILES_VAR" ] || [ "$pdir" = "$PROFILES_SNAPPY" ]; then
- cache_dir="$PROFILES_CACHE_VAR"
- fi
- cache_args="--cache-loc=$cache_dir"
- if [ ! -d "$cache_dir" ]; then
- cache_args=
- fi
-
- # LP: #1383858 - expr tree simplification is too slow for
- # Touch policy on ARM, so disable it for now
- cache_extra_args=
- if [ -d "$PROFILES_CACHE_VAR" ] && [ "$pdir" = "$PROFILES_VAR" ] || [ "$pdir" = "$PROFILES_SNAPPY" ]; then
- cache_extra_args="-O no-expr-simplify"
- fi
-
- # If need to compile everything, then use -n1 with xargs to
- # take advantage of -P. When cache files are in use, omit -n1
- # since it is considerably faster on moderately sized profile
- # sets to give the parser all the profiles to load at once
- n1_args=
- num=`find "$cache_dir" -type f ! -name '.features' | wc -l`
- if [ "$num" = "0" ]; then
- n1_args="-n1"
- fi
-
- (ls -1 "$pdir" | egrep -v '(\.dpkg-(new|old|dist|bak)|~)$' | \
- while read profile; do
- if [ -f "$pdir"/"$profile" ]; then
- echo "$pdir"/"$profile"
- fi
- done) | \
- xargs $n1_args -d"\n" -P$(getconf _NPROCESSORS_ONLN) "$PARSER" "$@" $cache_args $cache_extra_args -- || {
- rc_all="$?"
- # FIXME: when the parser properly handles broken
- # profiles (LP: #1377338), remove this if statement.
- # For now, if the xargs returns with error, just run
- # through everything with -n1. (This could be broken
- # out and refactored, but this is temporary so make it
- # easy to understand and revert)
- if [ "$rc_all" != "0" ]; then
- (ls -1 "$pdir" | \
- egrep -v '(\.dpkg-(new|old|dist|bak)|~)$' | \
- while read profile; do
- if [ -f "$pdir"/"$profile" ]; then
- echo "$pdir"/"$profile"
- fi
- done) | \
- xargs -n1 -d"\n" -P$(getconf _NPROCESSORS_ONLN) "$PARSER" "$@" $cache_args $cache_extra_args -- || {
- rc_all="$?"
- }
- fi
- }
- done
- return $rc_all
-}
-
-load_configured_profiles() {
- clear_cache_if_outdated
- foreach_configured_profile $quiet_arg --write-cache --replace
-}
-
-load_configured_profiles_without_caching() {
- foreach_configured_profile $quiet_arg --replace
-}
-
-recache_profiles() {
- clear_cache
- foreach_configured_profile $quiet_arg --write-cache --skip-kernel-load
-}
-
-configured_profile_names() {
- foreach_configured_profile $quiet_arg -N 2>/dev/null | LC_COLLATE=C sort | grep -v '//'
-}
-
-running_profile_names() {
- # Output a sorted list of loaded profiles, skipping libvirt's
- # dynamically generated files
- cat "$AA_SFS"/profiles | sed -e "s/ (\(enforce\|complain\))$//" | egrep -v '^libvirt-[0-9a-f\-]+$' | LC_COLLATE=C sort | grep -v '//'
-}
-
-unload_profile() {
- echo -n "$1" > "$AA_SFS"/.remove
-}
-
-clear_cache() {
- clear_cache_system
- clear_cache_var
-}
-
-clear_cache_system() {
- find "$PROFILES_CACHE" -maxdepth 1 -type f -print0 | xargs -0 rm -f --
-}
-
-clear_cache_var() {
- find "$PROFILES_CACHE_VAR" -maxdepth 1 -type f -print0 | xargs -0 rm -f --
-}
-
-read_features_dir()
-{
- for f in `ls -A "$1"` ; do
- if [ -f "$1/$f" ] ; then
- read -r KF < "$1/$f" || true
- echo -n "$f {$KF } "
- elif [ -d "$1/$f" ] ; then
- echo -n "$f {"
- KF=`read_features_dir "$1/$f"` || true
- echo -n "$KF} "
- fi
- done
-}
-
-clear_cache_if_outdated() {
- if [ -r "$PROFILES_CACHE"/.features ]; then
- if [ -d "$AA_SFS"/features ]; then
- KERN_FEATURES=`read_features_dir "$AA_SFS"/features`
- else
- read -r KERN_FEATURES < "$AA_SFS"/features
- fi
- CACHE_FEATURES=`tr '\n' ' ' < "$PROFILES_CACHE"/.features`
- if [ "$KERN_FEATURES" != "$CACHE_FEATURES" ]; then
- clear_cache
- fi
- fi
-}
-
-unload_obsolete_profiles() {
- # Currently we must re-parse all the profiles to get policy names. :(
- aa_configured=$(mktemp -t aa-XXXXXX)
- configured_profile_names > "$aa_configured" || true
- aa_loaded=$(mktemp -t aa-XXXXXX)
- running_profile_names > "$aa_loaded" || true
- LC_COLLATE=C comm -2 -3 "$aa_loaded" "$aa_configured" | while read profile ; do
- unload_profile "$profile"
- done
- rm -f "$aa_configured" "$aa_loaded"
-}
-
-# If the system debsum differs from the saved debsum, the new system debsum is
-# saved and non-zero is returned. Returns 0 if the two debsums matched or if
-# the system debsum file does not exist. This can be removed when system image
-# flavors all move to snappy.
-compare_and_save_debsums() {
- pkg="$1"
-
- if [ -n $pkg ] && [ -d "$PROFILES_VAR" ]; then
- sums="/var/lib/dpkg/info/${pkg}.md5sums"
- # store saved md5sums in /var/lib/apparmor/profiles since
- # /var/cache/apparmor might be cleared by apparmor
- saved_sums="${PROFILES_VAR}/.${pkg}.md5sums"
-
- if [ -f "$sums" ] && \
- ! diff -q "$sums" "$saved_sums" 2>&1 >/dev/null ; then
- cp -f "$sums" "$saved_sums"
- return 1
- fi
- fi
-
- return 0
-}
-
-compare_previous_version() {
- installed="/usr/share/snappy/security-policy-version"
- previous="/var/lib/snappy/security-policy-version"
-
- # When just $previous doesn't exist, assume this is a new system with
- # no cache and don't do anything special.
- if [ -f "$installed" ] && [ -f "$previous" ]; then
- pv=`grep '^apparmor/' "$previous" | cut -d ' ' -f 2`
- iv=`grep '^apparmor/' "$installed" | cut -d ' ' -f 2`
- if [ -n "$iv" ] && [ -n "$pv" ] && [ "$iv" != "$pv" ]; then
- # snappy updates $previous elsewhere, so just return
- return 1
- fi
- fi
-
- return 0
-}
-
-# Checks to see if the current container is capable of having internal AppArmor
-# profiles that should be loaded. Callers of this function should have already
-# verified that they're running inside of a container environment with
-# something like `systemd-detect-virt --container`.
-#
-# The only known container environments capable of supporting internal policy
-# are LXD and LXC environment.
-#
-# Returns 0 if the container environment is capable of having its own internal
-# policy and non-zero otherwise.
-#
-# IMPORTANT: This function will return 0 in the case of a non-LXD/non-LXC
-# system container technology being nested inside of a LXD/LXC container that
-# utilized an AppArmor namespace and profile stacking. The reason 0 will be
-# returned is because .ns_stacked will be "yes" and .ns_name will still match
-# "lx[dc]-*" since the nested system container technology will not have set up
-# a new AppArmor profile namespace. This will result in the nested system
-# container's boot process to experience failed policy loads but the boot
-# process should continue without any loss of functionality. This is an
-# unsupported configuration that cannot be properly handled by this function.
-is_container_with_internal_policy() {
- local ns_stacked_path="${AA_SFS}/.ns_stacked"
- local ns_name_path="${AA_SFS}/.ns_name"
- local ns_stacked
- local ns_name
-
- if ! [ -f "$ns_stacked_path" ] || ! [ -f "$ns_name_path" ]; then
- return 1
- fi
-
- read -r ns_stacked < "$ns_stacked_path"
- if [ "$ns_stacked" != "yes" ]; then
- return 1
- fi
-
- # LXD and LXC set up AppArmor namespaces starting with "lxd-" and
- # "lxc-", respectively. Return non-zero for all other namespace
- # identifiers.
- read -r ns_name < "$ns_name_path"
- if [ "${ns_name#lxd-*}" = "$ns_name" ] && \
- [ "${ns_name#lxc-*}" = "$ns_name" ]; then
- return 1
- fi
-
- return 0
-}



[meta-security][PATCH] tpm-tools: fix build issue

Armin Kuster
 

This error occurs randomly.
/bin/bash: pod2man: command not found

[Yocto #14304]

minor space/tab cleanup

Signed-off-by: Armin Kuster <akuster808@...>
Cc: Ben <koncept1@...>
---
meta-tpm/recipes-tpm/tpm-tools/tpm-tools_1.3.9.2.bb | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/meta-tpm/recipes-tpm/tpm-tools/tpm-tools_1.3.9.2.bb b/meta-tpm/recipes-tpm/tpm-tools/tpm-tools_1.3.9.2.bb
index 8aeb8ac..9e0a686 100644
--- a/meta-tpm/recipes-tpm/tpm-tools/tpm-tools_1.3.9.2.bb
+++ b/meta-tpm/recipes-tpm/tpm-tools/tpm-tools_1.3.9.2.bb
@@ -9,16 +9,16 @@ SECTION = "tpm"
LICENSE = "CPL-1.0"
LIC_FILES_CHKSUM = "file://LICENSE;md5=059e8cd6165cb4c31e351f2b69388fd9"

-DEPENDS = "libtspi openssl"
+DEPENDS = "libtspi openssl perl"
DEPENDS_class-native = "trousers-native"

SRCREV = "bf43837575c5f7d31865562dce7778eae970052e"
SRC_URI = " \
- git://git.code.sf.net/p/trousers/tpm-tools \
- file://tpm-tools-extendpcr.patch \
- file://04-fix-FTBFS-clang.patch \
- file://openssl1.1_fix.patch \
- "
+ git://git.code.sf.net/p/trousers/tpm-tools \
+ file://tpm-tools-extendpcr.patch \
+ file://04-fix-FTBFS-clang.patch \
+ file://openssl1.1_fix.patch \
+ "

inherit autotools-brokensep gettext

--
2.25.1

3321 - 3340 of 57397