Date   

Re: Why python3 is build native in yocto ??? #python

Ross Burton
 

On Fri, 6 Nov 2020 at 08:23, <william.jacob@sama.com.sg> wrote:
I am trying to compile python package for yocto and I found python3 (python3.7) is build native then it cross compiled.I want to know why it is done like this in python.
The python3 build for target doesn't directly depend on python3-native.

Some recipes do depend on python3-native (although by definition there
is a python3 on your host) as they need to run non-standard Python
modules as part of their build. We don't want to assume that you've
installed all of these modules (as the list is ever growing) and we
don't want to assume that you've installed python3-dev on the host (as
we want to have minimal build requirements, and C modules are not
portable between Python versions).

Basically I want python3 binary cross compiled for arm architecture should be available in recipe-sysroot and recipe-sysroot-native directory . Right now it is available only in recipe-sysroot-native.
Please explain what you are actually trying to do. For 99% of python
modules, simply inheriting setuptool3 or even better pypi is
sufficient. If a package has hand-coded Python detection then it gets
a lot more complicated as Python doesn't have great support for
cross-compiling, and most people writing the build scripts don't
consider it at all.

Ross


Re: Why python3 is build native in yocto ??? #python

Josef Holzmayr
 

Howdy!

Am Fr., 6. Nov. 2020 um 09:23 Uhr schrieb <william.jacob@sama.com.sg>:

Hi,
I am trying to compile python package for yocto and I found python3 (python3.7) is build native then it cross compiled.I want to know why it is done like this in python.

Basically I want python3 binary cross compiled for arm architecture should be available in recipe-sysroot and recipe-sysroot-native directory . Right now it is available only in recipe-sysroot-native.
I'm sorry, but I don't really understand the question. You say you
want python3 crosscompiled. This will happen if something that goes
into your image depends/rdepends on it, or if you explicitly add it to
said image via IMAGE_INSTALL. Does this not work for you?

Greetz

Thanks

William Jacob


Why python3 is build native in yocto ??? #python

William Jacob
 

Hi,
I am trying to compile python package for yocto and I found  python3 (python3.7) is  build native then it cross compiled.I want to know why it is done like this in python.

Basically I want python3 binary cross compiled for arm architecture should be available in recipe-sysroot and  recipe-sysroot-native directory . Right now it is available only in recipe-sysroot-native.

 

Thanks 

William Jacob


Re: : how to fix "do not ask to insatll a package providing xxxx"?

Chen Qi
 

It's basically about some package conflict & absence.

You could provide more info such as you bb file, the layers you're using, the result of `grep packagegroup-obmc-apps-network', etc.

Regards,
Chen Qi

On 11/05/2020 09:10 PM, ouyangxuan10@... wrote:
Dear all,

I build a libarary in other project. And add a bb file in yocto to add to my yocto project, but it appear error.

who can tell me how to fix it?

thanks,
Byron



 







What part of do_package automatically scoops up source files?

Sean McKay
 

Hi all,

 

I’m hoping that someone can just answer this off the top of their head – don’t bother if you have to dig much.

We’ve got a C recipe (built with cmake, if that’s relevant) that seems to be otherwise identical to our other C recipes, but for some reason it’s not picking up the source code during the do_package step automagically like our other recipes do.

Anyone happen to know offhand where in the do_package code that gets scooped up so I can focus my attention there and on the variables used therein? (my run.do_package files for those tasks are about 3k lines of python code. And a quick glance through seemed to yield obvious answers that don’t seem to match what’s coming out in log.do_package)

 

We’re on Zeus at the moment.

 

Cheers!

-Sean McKay

 


[meta-zephyr][PATCH ] zephyr: Yocto Image Tests Fix

yock.gen.mah@...
 

From: yockgenm <yock.gen.mah@intel.com>

Fix bug on Image Test, previously the Image Tests was not working due to update on Yocto Image Test Framework.

The fix has rewritten and restructured existing Image Tests code to latest Yocto testimage class requirement to make meta-zephyr able to run Image Tests as expected.

Signed-off-by: yockgenm <yock.gen.mah@intel.com>
---
README.txt | 14 +++++++++-----
lib/oeqa/controllers/__init__.py | 3 ---
lib/oeqa/controllers/zephyrtargetcontrol.py | 24 +++++++++++++-----------
lib/oeqa/runtime/{ => cases}/zephyr.py | 15 ++++++++++++---
lib/oeqa/utils/qemuzephyrrunner.py | 12 ++++++------
5 files changed, 40 insertions(+), 28 deletions(-)
rename lib/oeqa/runtime/{ => cases}/zephyr.py (80%)

diff --git a/README.txt b/README.txt
index a02659a..6463339 100644
--- a/README.txt
+++ b/README.txt
@@ -49,24 +49,28 @@ Building and Running Zephyr Tests
Presently only toolchains for ARM, x86, IAMCU and Nios2 are supported.
(For ARM we use CortexM3 toolchain)

+To run Zephyr Test using Yocto Image Tests, ensure following in local.conf:
+
+ INHERIT += "testimage"
+
You can build and test an individual existing Zephyr test.
This is done by appending the actual test name to the "zephyr-kernel-test",
for example:

- $ MACHINE=qemu-x86 bitbake zephyr-kernel-test-test_sleep
- $ MACHINE=qemu-x86 bitbake zephyr-kernel-test-test_sleep -ctestimage
+ $ MACHINE=qemu-x86 bitbake zephyr-kernel-test-sleep
+ $ MACHINE=qemu-x86 bitbake zephyr-kernel-test-sleep -c testimage

You can also build and run all Zephyr existing tests (as listed in the file
zephyr-kernel-test.inc). For example:

$ MACHINE=qemu-x86 bitbake zephyr-kernel-test-all
- $ MACHINE=qemu-x86 bitbake zephyr-kernel-test-all -ctestimage
+ $ MACHINE=qemu-x86 bitbake zephyr-kernel-test-all -c testimage
or
$ MACHINE=qemu-cortex-m3 bitbake zephyr-kernel-test-all
- $ MACHINE=qemu-cortex-m3 bitbake zephyr-kernel-test-all -ctestimage
+ $ MACHINE=qemu-cortex-m3 bitbake zephyr-kernel-test-all -c testimage
or
$ MACHINE=qemu-nios2 bitbake zephyr-kernel-test-all
- $ MACHINE=qemu-nios2 bitbake zephyr-kernel-test-all -ctestimage
+ $ MACHINE=qemu-nios2 bitbake zephyr-kernel-test-all -c testimage


Contributing
diff --git a/lib/oeqa/controllers/__init__.py b/lib/oeqa/controllers/__init__.py
index 8eda927..e69de29 100644
--- a/lib/oeqa/controllers/__init__.py
+++ b/lib/oeqa/controllers/__init__.py
@@ -1,3 +0,0 @@
-# Enable other layers to have modules in the same named directory
-from pkgutil import extend_path
-__path__ = extend_path(__path__, __name__)
diff --git a/lib/oeqa/controllers/zephyrtargetcontrol.py b/lib/oeqa/controllers/zephyrtargetcontrol.py
index 84ba3be..8e94cb5 100644
--- a/lib/oeqa/controllers/zephyrtargetcontrol.py
+++ b/lib/oeqa/controllers/zephyrtargetcontrol.py
@@ -11,11 +11,16 @@ from oeqa.utils.qemuzephyrrunner import QemuZephyrRunner
supported_fstypes = ['elf']

class QemuTargetZephyr(OETarget):
- def __init__(self, logger, ip, server_ip, target_modules_path,
- timeout=300, user='root',
- port=None, machine='', rootfs='', kernel='', kvm=False,
- dump_dir='', dump_host_cmds='', display='', bootlog='',
- tmpdir='', dir_image='', boottime=60):
+ def __init__(self, logger, ip, server_ip,
+ machine='', rootfs='', tmpdir ='',dir_image ='',display=None,
+ kernel='',boottime=60,bootlog='',kvm=False,slirp=False,
+ dump_dir='',serial_ports=0,ovmf=None,target_modules_path='',powercontrol_cmd='',powercontrol_extra_args='',
+ serialcontrol_cmd=None,serialcontrol_extra_args='',testimage_dump_target='' ):
+
+ timeout = 300
+ user = 'root'
+ port = serial_ports
+ dump_host_cmds = testimage_dump_target

super(QemuTargetZephyr, self).__init__(logger, ip, server_ip, timeout,
user, port)
@@ -42,19 +47,16 @@ class QemuTargetZephyr(OETarget):
deploy_dir_image=dir_image, display=display,
logfile=self.qemulog, boottime=boottime,
use_kvm=kvm, dump_dir=dump_dir,
- dump_host_cmds=dump_host_cmds)
+ dump_host_cmds=dump_host_cmds,
+ logger = logger)


- def start(self, params=None, extra_bootparams=None):
+ def start(self, params=None, runqemuparams=None, extra_bootparams=None):
if self.runner.start(params, extra_bootparams=extra_bootparams):
self.ip = self.runner.ip
self.server_ip = self.runner.server_ip
else:
- self.stop()
raise RuntimeError("FAILED to start qemu - check the task log and the boot log")

- def stop(self):
- self.runner.stop()
-
def serial_readline(self):
return self.runner.serial_readline()
diff --git a/lib/oeqa/runtime/zephyr.py b/lib/oeqa/runtime/cases/zephyr.py
similarity index 80%
rename from lib/oeqa/runtime/zephyr.py
rename to lib/oeqa/runtime/cases/zephyr.py
index 96a119a..8751aba 100644
--- a/lib/oeqa/runtime/zephyr.py
+++ b/lib/oeqa/runtime/cases/zephyr.py
@@ -1,11 +1,20 @@
-import unittest
-from oeqa.oetest import oeRuntimeTest
+#
+# SPDX-License-Identifier: MIT
+#

-class ZephyrTest(oeRuntimeTest):
+from subprocess import Popen, PIPE

+from oeqa.runtime.case import OERuntimeTestCase
+from oeqa.core.decorator.oetimeout import OETimeout
+
+class ZephyrTest(OERuntimeTestCase):
+
+ @OETimeout(120)
def test_boot_zephyr(self):
+
success = False
logfile = self.target.qemurunnerlog
+
while True:
line = self.target.serial_readline().decode("utf-8")

diff --git a/lib/oeqa/utils/qemuzephyrrunner.py b/lib/oeqa/utils/qemuzephyrrunner.py
index 9d7badb..0308f1e 100644
--- a/lib/oeqa/utils/qemuzephyrrunner.py
+++ b/lib/oeqa/utils/qemuzephyrrunner.py
@@ -18,10 +18,11 @@ from oeqa.utils.qemurunner import QemuRunner

class QemuZephyrRunner(QemuRunner):

- def __init__(self, machine, rootfs, display, tmpdir, deploy_dir_image, logfile, boottime, dump_dir, dump_host_cmds, use_kvm):
+ def __init__(self, machine, rootfs, display, tmpdir, deploy_dir_image, logfile, boottime, dump_dir, dump_host_cmds, use_kvm, logger):
+
QemuRunner.__init__(self, machine, rootfs, display, tmpdir,
deploy_dir_image, logfile, boottime, None,
- None, use_kvm)
+ None, use_kvm, logger)

# Popen object for runqemu
self.socketfile = tempfile.NamedTemporaryFile()
@@ -59,7 +60,8 @@ class QemuZephyrRunner(QemuRunner):
return False
return True

- def start(self, params=None, extra_bootparams=None):
+ def start(self, params=None,runqemuparams=None, extra_bootparams=None):
+
if not os.path.exists(self.tmpdir):
bb.error("Invalid TMPDIR path %s" % self.tmpdir)
#logger.error("Invalid TMPDIR path %s" % self.tmpdir)
@@ -83,7 +85,7 @@ class QemuZephyrRunner(QemuRunner):
qemu_machine_args = '-machine lm3s6965evb'
elif 'x86' in self.machine:
qemu_binary = 'qemu-system-i386'
- qemu_machine_args = '-machine type=pc-0.14'
+ qemu_machine_args = '-machine type=pc-1.3 -no-acpi -nographic -cpu qemu32,+nx,+pae'
elif 'nios2' in self.machine:
qemu_binary = 'qemu-system-nios2'
qemu_machine_args = '-machine altera_10m50_zephyr'
@@ -155,5 +157,3 @@ class QemuZephyrRunner(QemuRunner):

self.log(line)
return line
-
-
--
2.7.4


Re: Reproducible builds and RPM packages

Anders Montonen
 



On 4 Nov 2020, at 23:23, Randy MacLeod <randy.macleod@...> wrote:

On 2020-11-03 6:16 a.m., Anders Montonen wrote:
Hi,

When going from Zeus to Dunfell, I noticed that all files on the rootfs had timestamps long in the past, which I assume is from reproducible builds now being on by default. While that is a good thing, running “rpm -V” on any installed package now reports that the mtime differs. Is this the intentional behavior?

Hi Anders,

I haven't played with that for a while but I'm pretty sure the answer is yes, it's intentional.

You can read about reproducible builds here:
   https://wiki.yoctoproject.org/wiki/Reproducible_Builds
and compare to the source if needed:
   https://git.openembedded.org/openembedded-core/tree/meta/classes/reproducible_build.bbclass?id=189630ca6cdf7ceb6cf9b8f9d86c58997f505efc&h=dunfell



Just to be clear, I’m referring to the RPM metadata. If you build a package, and then run "rpm -q --qf '[%{FILENAMES} %{FILEMTIMES}\n]' -p package” on the output, the listed timestamps will not match either SOURCE_DATE_EPOCH, or REPRODUCIBLE_TIMESTAMP_ROOTFS. When you build an image, the timestamps in the filesystem will not be consistent with the timestamps in the RPM database, leading to errors if you try to verify an installed package. I would consider this a bug or an oversight.

Regards,
Anders Montonen


Re: #yocto overriding a task in a recipe #yocto

Monsees, Steven C (US)
 

Thanks... that's what I saw this morning...

-----Original Message-----
From: yocto@lists.yoctoproject.org [mailto:yocto@lists.yoctoproject.org] On Behalf Of Khem Raj
Sent: Thursday, November 05, 2020 12:34 PM
To: Monsees, Steven C (US) <steven.monsees@baesystems.com>; yocto@lists.yoctoproject.org
Subject: Re: [yocto] #yocto overriding a task in a recipe

*** WARNING ***
EXTERNAL EMAIL -- This message originates from outside our organization.



On 11/5/20 4:13 AM, Monsees, Steven C (US) via lists.yoctoproject.org wrote:
If a recipe has a task "do_something() {...stuff...}", and for some reason
I do not want that task to execute its steps, but a different series
of steps, can I over-ride the task from a .bbapend by supplying the
same routine with different steps ("do_something() {...new stuff...}") ?
it depends, if the task is having appends etc. they will still be applied. but main task you can override.

Thanks,

Steve





Re: #yocto overriding a task in a recipe #yocto

Khem Raj
 

On 11/5/20 4:13 AM, Monsees, Steven C (US) via lists.yoctoproject.org wrote:
If a recipe has a task “do_something() {…stuff…}”, and for some reason I do not want that task to execute its steps, but a different series of steps, can I over-ride the task from a .bbapend by supplying the same routine with different steps (“do_something() {…new stuff…}”) ?
it depends, if the task is having appends etc. they will still be applied. but main task you can override.

Thanks,
Steve


# yocto zeus -building a grub image

Monsees, Steven C (US)
 

 

Can someone provide an answer on how to correct the following grub build issue ?

 

I am attempting to build a grub image under zeus, and I am seeing the error output below…

 

It appears that `./grub-core/verify.*' –do not appear to have been built under the “./grub-core” build directory for zeus.

 

Under our “rocko” based build the verify.marker/verify.mod/verify.module are all present.

 

How do I generate the referenced modules ?

 

Thanks,

Steve

 

ERROR OUTPUT SEEN:

 

Initialising tasks: 100% |###########################################################################################| Time: 0:00:04

Sstate summary: Wanted 2127 Found 2096 Missed 31 Current 0 (98% match, 0% complete)

NOTE: Executing Tasks

NOTE: Setscene tasks completed

ERROR: grub-efi-2.04-r0 do_mkimage: Execution of '/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/sbcb-default/tmp/work/corei7-64-poky-linux/grub-efi/2.04-r0/temp/run.do_mkimage.7707' failed with exit code 1:

/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/cyberobjects/core /disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/sbcb-default

/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/sbcb-default

grub-mkimage: error: cannot open `./grub-core/verify.mod': No such file or directory.

WARNING: /disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/sbcb-default/tmp/work/corei7-64-poky-linux/grub-efi/2.04-r0/temp/run.do_mkimage.7707:1 exit 1 from 'grub-mkimage -c "/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/sbcb-default/tmp/work/corei7-64-poky-linux/grub-efi/2.04-r0/image/ofp/proc4-initial.cfg" -p /ofp -k "/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/cyberobjects/core/proc4.key" -d "./grub-core" -O "x86_64-efi" -C "auto" -o "/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/sbcb-default/tmp/work/corei7-64-poky-linux/grub-efi/2.04-r0/build/proc4.efi" configfile echo efinet gcry_rsa loopback password_pbkdf2 reboot sleep tftp verify boot linux ext2 fat serial part_msdos part_gpt normal efi_gop iso9660 configfile search loadenv test chain efivar mok2verify password_pbkdf2'

 

ERROR: Logfile of failure stored in: /disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/sbcb-default/tmp/work/corei7-64-poky-linux/grub-efi/2.04-r0/temp/log.do_mkimage.7707

Log data follows:

| DEBUG: Executing shell function do_mkimage

| /disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/cyberobjects/core /disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/sbcb-default

| /disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/sbcb-default

| grub-mkimage: error: cannot open `./grub-core/verify.mod': No such file or directory.

| WARNING: /disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/sbcb-default/tmp/work/corei7-64-poky-linux/grub-efi/2.04-r0/temp/run.do_mkimage.7707:1 exit 1 from 'grub-mkimage -c "/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/sbcb-default/tmp/work/corei7-64-poky-linux/grub-efi/2.04-r0/image/ofp/proc4-initial.cfg" -p /ofp -k "/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/cyberobjects/core/proc4.key" -d "./grub-core" -O "x86_64-efi" -C "auto" -o "/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/sbcb-default/tmp/work/corei7-64-poky-linux/grub-efi/2.04-r0/build/proc4.efi" configfile echo efinet gcry_rsa loopback password_pbkdf2 reboot sleep tftp verify boot linux ext2 fat serial part_msdos part_gpt normal efi_gop iso9660 configfile search loadenv test chain efivar mok2verify password_pbkdf2'

| ERROR: Execution of '/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/sbcb-default/tmp/work/corei7-64-poky-linux/grub-efi/2.04-r0/temp/run.do_mkimage.7707' failed with exit code 1:

| /disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/cyberobjects/core /disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/sbcb-default

| /disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/sbcb-default

| grub-mkimage: error: cannot open `./grub-core/verify.mod': No such file or directory.

| WARNING: /disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/sbcb-default/tmp/work/corei7-64-poky-linux/grub-efi/2.04-r0/temp/run.do_mkimage.7707:1 exit 1 from 'grub-mkimage -c "/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/sbcb-default/tmp/work/corei7-64-poky-linux/grub-efi/2.04-r0/image/ofp/proc4-initial.cfg" -p /ofp -k "/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/cyberobjects/core/proc4.key" -d "./grub-core" -O "x86_64-efi" -C "auto" -o "/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/sbcb-default/tmp/work/corei7-64-poky-linux/grub-efi/2.04-r0/build/proc4.efi" configfile echo efinet gcry_rsa loopback password_pbkdf2 reboot sleep tftp verify boot linux ext2 fat serial part_msdos part_gpt normal efi_gop iso9660 configfile search loadenv test chain efivar mok2verify password_pbkdf2'

|

ERROR: Task (/disk0/scratch/smonsees/yocto/workspace_3/poky/meta/recipes-bsp/grub/grub-efi_2.04.bb:do_mkimage) failed with exit code '1'

NOTE: Tasks Summary: Attempted 5951 tasks of which 5421 didn't need to be rerun and 1 failed.

 

Summary: 1 task failed:

  /disk0/scratch/smonsees/yocto/workspace_3/poky/meta/recipes-bsp/grub/grub-efi_2.04.bb:do_mkimage

Summary: There was 1 WARNING message shown.

Summary: There was 1 ERROR message shown, returning a non-zero exit code.

 


Re: : how to fix "do not ask to insatll a package providing xxxx"?

Rudolf J Streif
 

Hi Byron,

It's a dependency problem. The build system cannot find a recipe to build libdebuglog1. You either need to provide that or you don't install anything that requires libdebuglog1.

:rjs

On 11/5/20 5:10 AM, ouyangxuan10@... wrote:
Dear all,

I build a libarary in other project. And add a bb file in yocto to add to my yocto project, but it appear error.

who can tell me how to fix it?

thanks,
Byron



 




-- 
Rudolf J Streif
CEO/CTO ibeeto
+1.855.442.3386 x700


#yocto overriding a task in a recipe #yocto

Monsees, Steven C (US)
 

 

If a recipe has a task “do_something() {…stuff…}”, and for some reason I do not want that task to execute its steps, but a different series of steps, can I over-ride the task from a .bbapend by supplying the same routine with different steps (“do_something() {…new stuff…}”) ?

 

Thanks,

Steve


: how to fix "do not ask to insatll a package providing xxxx"?

ouyangxuan10@163.com
 

Dear all,

I build a libarary in other project. And add a bb file in yocto to add to my yocto project, but it appear error.

who can tell me how to fix it?

thanks,
Byron



 


[meta-security][PATCH] scap-security-guide: fix build with Python 3.9

Yi Zhao
 

The getchildren and getiterator functions are deprecated in Python 3.9.
Backport 3 patches to fix the build issue.

Fixes:
File
"/build/tmp/work/cortexa8hf-neon-poky-linux-gnueabi/scap-security-guide/0.1.44+gitAUTOINC+5fdfdcb2e9-r0/git/ssg/build_stig.py",
line 41, in add_references
index = rule.getchildren().index(ref)
AttributeError: 'xml.etree.ElementTree.Element' object has no attribute 'getchildren'

Signed-off-by: Yi Zhao <yi.zhao@windriver.com>
---
...ated-instance-of-element.getchildren.patch | 43 ++++++++++++++
...-fix-deprecated-getiterator-function.patch | 58 +++++++++++++++++++
...etchildren-and-getiterator-functions.patch | 57 ++++++++++++++++++
.../scap-security-guide_git.bb | 3 +
4 files changed, 161 insertions(+)
create mode 100644 meta-security-compliance/recipes-openscap/scap-security-guide/files/0001-fix-deprecated-instance-of-element.getchildren.patch
create mode 100644 meta-security-compliance/recipes-openscap/scap-security-guide/files/0002-fix-deprecated-getiterator-function.patch
create mode 100644 meta-security-compliance/recipes-openscap/scap-security-guide/files/0003-fix-remaining-getchildren-and-getiterator-functions.patch

diff --git a/meta-security-compliance/recipes-openscap/scap-security-guide/files/0001-fix-deprecated-instance-of-element.getchildren.patch b/meta-security-compliance/recipes-openscap/scap-security-guide/files/0001-fix-deprecated-instance-of-element.getchildren.patch
new file mode 100644
index 0000000..01e3dd6
--- /dev/null
+++ b/meta-security-compliance/recipes-openscap/scap-security-guide/files/0001-fix-deprecated-instance-of-element.getchildren.patch
@@ -0,0 +1,43 @@
+From e435bf2dc59d652710104a1c59332e410b12bb64 Mon Sep 17 00:00:00 2001
+From: Vojtech Polasek <vpolasek@redhat.com>
+Date: Mon, 8 Jun 2020 12:33:48 +0200
+Subject: [PATCH] fix deprecated instance of element.getchildren
+
+Upstream-Status: Backport
+[https://github.com/ComplianceAsCode/content/commit/e435bf2dc59d652710104a1c59332e410b12bb64]
+
+Signed-off-by: Yi Zhao <yi.zhao@windriver.com>
+---
+ ssg/build_remediations.py | 2 +-
+ ssg/build_stig.py | 2 +-
+ 2 files changed, 2 insertions(+), 2 deletions(-)
+
+diff --git a/ssg/build_remediations.py b/ssg/build_remediations.py
+index fdde0f268..c18d6bd54 100644
+--- a/ssg/build_remediations.py
++++ b/ssg/build_remediations.py
+@@ -735,7 +735,7 @@ def expand_xccdf_subs(fix, remediation_type, remediation_functions):
+ # First concat output form of modified fix text (including text appended
+ # to all children of the fix)
+ modfix = [fix.text]
+- for child in fix.getchildren():
++ for child in list(fix):
+ if child is not None and child.text is not None:
+ modfix.append(child.text)
+ modfixtext = "".join(modfix)
+diff --git a/ssg/build_stig.py b/ssg/build_stig.py
+index 528285f3d..6122981fc 100644
+--- a/ssg/build_stig.py
++++ b/ssg/build_stig.py
+@@ -38,7 +38,7 @@ def add_references(reference, destination):
+ for ref in refs:
+ if (ref.get('href').startswith(stig_refs) and
+ ref.text in dictionary):
+- index = rule.getchildren().index(ref)
++ index = list(rule).index(ref)
+ new_ref = ET.Element(
+ '{%s}reference' % XCCDF11_NS, {'href': stig_ns})
+ new_ref.text = dictionary[ref.text]
+--
+2.17.1
+
diff --git a/meta-security-compliance/recipes-openscap/scap-security-guide/files/0002-fix-deprecated-getiterator-function.patch b/meta-security-compliance/recipes-openscap/scap-security-guide/files/0002-fix-deprecated-getiterator-function.patch
new file mode 100644
index 0000000..84271c4
--- /dev/null
+++ b/meta-security-compliance/recipes-openscap/scap-security-guide/files/0002-fix-deprecated-getiterator-function.patch
@@ -0,0 +1,58 @@
+From b0adc1d53780def4a95e310b6d26bb91ee97177e Mon Sep 17 00:00:00 2001
+From: Vojtech Polasek <vpolasek@redhat.com>
+Date: Mon, 8 Jun 2020 13:27:41 +0200
+Subject: [PATCH] fix deprecated getiterator function
+
+Upstream-Status: Backport
+[https://github.com/ComplianceAsCode/content/commit/b0adc1d53780def4a95e310b6d26bb91ee97177e]
+
+Signed-off-by: Yi Zhao <yi.zhao@windriver.com>
+---
+ ssg/build_cpe.py | 6 +++---
+ ssg/id_translate.py | 2 +-
+ 2 files changed, 4 insertions(+), 4 deletions(-)
+
+diff --git a/ssg/build_cpe.py b/ssg/build_cpe.py
+index 2e5d24a5d..8c046777a 100644
+--- a/ssg/build_cpe.py
++++ b/ssg/build_cpe.py
+@@ -17,7 +17,7 @@ def extract_subelement(objects, sub_elem_type):
+ """
+
+ for obj in objects:
+- for subelement in obj.getiterator():
++ for subelement in obj.iter():
+ if subelement.get(sub_elem_type):
+ sub_element = subelement.get(sub_elem_type)
+ return sub_element
+@@ -44,12 +44,12 @@ def extract_referred_nodes(tree_with_refs, tree_with_ids, attrname):
+ reflist = []
+ elementlist = []
+
+- for element in tree_with_refs.getiterator():
++ for element in tree_with_refs.iter():
+ value = element.get(attrname)
+ if value is not None:
+ reflist.append(value)
+
+- for element in tree_with_ids.getiterator():
++ for element in tree_with_ids.iter():
+ if element.get("id") in reflist:
+ elementlist.append(element)
+
+diff --git a/ssg/id_translate.py b/ssg/id_translate.py
+index 72b07be18..ba9225904 100644
+--- a/ssg/id_translate.py
++++ b/ssg/id_translate.py
+@@ -64,7 +64,7 @@ class IDTranslator(object):
+ )
+
+ def translate(self, tree, store_defname=False):
+- for element in tree.getiterator():
++ for element in tree.iter():
+ idname = element.get("id")
+ if idname:
+ # store the old name if requested (for OVAL definitions)
+--
+2.17.1
+
diff --git a/meta-security-compliance/recipes-openscap/scap-security-guide/files/0003-fix-remaining-getchildren-and-getiterator-functions.patch b/meta-security-compliance/recipes-openscap/scap-security-guide/files/0003-fix-remaining-getchildren-and-getiterator-functions.patch
new file mode 100644
index 0000000..8162292
--- /dev/null
+++ b/meta-security-compliance/recipes-openscap/scap-security-guide/files/0003-fix-remaining-getchildren-and-getiterator-functions.patch
@@ -0,0 +1,57 @@
+From a0da16c5eeb9a7414f7f2a37a6b270c8d04b2ddf Mon Sep 17 00:00:00 2001
+From: Vojtech Polasek <vpolasek@redhat.com>
+Date: Mon, 8 Jun 2020 14:01:55 +0200
+Subject: [PATCH] fix remaining getchildren and getiterator functions
+
+Upstream-Status: Backport
+[https://github.com/ComplianceAsCode/content/commit/a0da16c5eeb9a7414f7f2a37a6b270c8d04b2ddf]
+
+Signed-off-by: Yi Zhao <yi.zhao@windriver.com>
+---
+ build-scripts/sds_move_ocil_to_checks.py | 2 +-
+ build-scripts/verify_references.py | 2 +-
+ shared/transforms/pcidss/transform_benchmark_to_pcidss.py | 2 +-
+ 3 files changed, 3 insertions(+), 3 deletions(-)
+
+diff --git a/build-scripts/sds_move_ocil_to_checks.py b/build-scripts/sds_move_ocil_to_checks.py
+index 5f5139659..64dc19084 100755
+--- a/build-scripts/sds_move_ocil_to_checks.py
++++ b/build-scripts/sds_move_ocil_to_checks.py
+@@ -106,7 +106,7 @@ def move_ocil_content_from_ds_extended_component_to_ds_component(datastreamtree,
+ timestamp = extendedcomp.get('timestamp')
+
+ # Get children elements of <ds:extended-component> containing OCIL content
+- extchildren = extendedcomp.getchildren()
++ extchildren = list(extendedcomp)
+ # There should be just one OCIL subcomponent in <ds:extended-component>
+ if len(extchildren) != 1:
+ sys.stderr.write("ds:extended-component contains more than one element!"
+diff --git a/build-scripts/verify_references.py b/build-scripts/verify_references.py
+index 69b3e2d1f..95d387f46 100755
+--- a/build-scripts/verify_references.py
++++ b/build-scripts/verify_references.py
+@@ -179,7 +179,7 @@ def main():
+ check_content_refs = xccdftree.findall(".//{%s}check-content-ref"
+ % xccdf_ns)
+
+- xccdf_parent_map = dict((c, p) for p in xccdftree.getiterator() for c in p)
++ xccdf_parent_map = dict((c, p) for p in xccdftree.iter() for c in p)
+ # now we can actually do the verification work here
+ if options.rules_with_invalid_checks or options.all_checks:
+ for check_content_ref in check_content_refs:
+diff --git a/shared/transforms/pcidss/transform_benchmark_to_pcidss.py b/shared/transforms/pcidss/transform_benchmark_to_pcidss.py
+index 0ceaf727d..c94b12c45 100755
+--- a/shared/transforms/pcidss/transform_benchmark_to_pcidss.py
++++ b/shared/transforms/pcidss/transform_benchmark_to_pcidss.py
+@@ -111,7 +111,7 @@ def main():
+ benchmark.findall(".//{%s}Value" % (XCCDF_NAMESPACE)):
+ values.append(value)
+
+- parent_map = dict((c, p) for p in benchmark.getiterator() for c in p)
++ parent_map = dict((c, p) for p in benchmark.iter() for c in p)
+ for rule in \
+ benchmark.findall(".//{%s}Rule" % (XCCDF_NAMESPACE)):
+ parent_map[rule].remove(rule)
+--
+2.17.1
+
diff --git a/meta-security-compliance/recipes-openscap/scap-security-guide/scap-security-guide_git.bb b/meta-security-compliance/recipes-openscap/scap-security-guide/scap-security-guide_git.bb
index f35d769..6e7180f 100644
--- a/meta-security-compliance/recipes-openscap/scap-security-guide/scap-security-guide_git.bb
+++ b/meta-security-compliance/recipes-openscap/scap-security-guide/scap-security-guide_git.bb
@@ -4,6 +4,9 @@ SRCREV = "5fdfdcb2e95afbd86ace555beca5d20cbf1043ed"
SRC_URI = "git://github.com/akuster/scap-security-guide.git;branch=oe-0.1.44; \
file://0001-Fix-XML-parsing-of-the-remediation-functions-file.patch \
file://0002-Fixed-the-broken-fix-when-greedy-regex-ate-the-whole.patch \
+ file://0001-fix-deprecated-instance-of-element.getchildren.patch \
+ file://0002-fix-deprecated-getiterator-function.patch \
+ file://0003-fix-remaining-getchildren-and-getiterator-functions.patch \
"
PV = "0.1.44+git${SRCPV}"

--
2.17.1


Re: how to use meta-cpan layer in a reproducible way?

Khem Raj
 

On 11/4/20 6:29 AM, Robert P. J. Day wrote:
colleague wants to add some perl recipes to a zeus-based image, and
found the meta-cpan layer, which contains a number of desired recipes,
but the structure of that layer seems a bit sparse.
first, here's the entire set of branches:
$ git branch -a
* master
thud
remotes/origin/HEAD -> origin/master
remotes/origin/daisy
remotes/origin/jethro
remotes/origin/jethro-next
remotes/origin/master
remotes/origin/master-next
remotes/origin/rocko
remotes/origin/thud
$
where you can see that the branches jump from thud to master with
nothing in between, so there is no "zeus" branch to focus on. also,
there are absolutely no tags.
finally, it seems that the recipes simply upgrade versions
arbitrarily, all on the master branch, so if you grabbed the "master"
version of any recipe, there's no telling when it might change, which
makes reproducibility a bit iffy.
right, so usually it means maintainers keep master moving forward and perhaps test with latest release or master or perhaps all releases after the thud branch, you have to ask this question to the maintainers. but yes forking and then branching your development from master is perhaps fine.

what is the proper way to take advantage of a layer like this? you
can't even use PREFERRED_VERSION for a given recipe since you can
never predict when that version will suddenly disappear from the
master branch.
i imagine the best approach is to simply clone the layer, check it
out to a specific commit ID that works for all desired recipes, and
just leave it there. or just individually copy each version of each
recipe one wants into a local layer (yuck).
thoughts?
rday


Re: Reproducible builds and RPM packages

Randy MacLeod
 

On 2020-11-03 6:16 a.m., Anders Montonen wrote:
Hi,

When going from Zeus to Dunfell, I noticed that all files on the rootfs had timestamps long in the past, which I assume is from reproducible builds now being on by default. While that is a good thing, running “rpm -V” on any installed package now reports that the mtime differs. Is this the intentional behavior?

Hi Anders,

I haven't played with that for a while but I'm pretty sure the answer is yes, it's intentional.

You can read about reproducible builds here:
   https://wiki.yoctoproject.org/wiki/Reproducible_Builds
and compare to the source if needed:
   https://git.openembedded.org/openembedded-core/tree/meta/classes/reproducible_build.bbclass?id=189630ca6cdf7ceb6cf9b8f9d86c58997f505efc&h=dunfell

I don't see it in the documentation yet:
   http://docs.yoctoproject.org/search.html?q=reproducible&check_keywords=yes&area=default

but I didn't do more than skim those search results.

If you also can't find it in the docs, would you be able to make a first draft of that and contribute to the project?

If so please read and ask for guidance on: docs@...

Thanks,

../Randy


Regards,
Anders Montonen




-- 
# Randy MacLeod
# Wind River Linux


Re: Removing postinst script in bbappend

Jim Krantz
 

It's a simple operation. They check for the existence of /etc/apparmor.d/cache. If it doesn't exist, then they create it as a directory. 

I'm not sure why it can't be moved elsewhere, as we're not the maintainers. I could reach out and find out, though. 

On Wed, Nov 4, 2020 at 3:38 PM Alexander Kanavin <alex.kanavin@...> wrote:
The right way would be to fix the recipe I think? What is doing that has to happen only on target and can't be done during do_rootfs?

Alex

On Wed, 4 Nov 2020 at 21:19, Jim Krantz <krantzj84@...> wrote:
The recipe is apparmor, layer is meta-security. 

We're currently using the 2.x recipe, however the 3.x recipe appears to have the same function.


On Wed, Nov 4, 2020 at 2:32 PM Alexander Kanavin <alex.kanavin@...> wrote:
What recipe is it and what layer is it in?

Alex

On Wed 4. Nov 2020 at 19.04, Jim Krantz <krantzj84@...> wrote:
Hey folks,

We're doing builds using Yocto 3.1 and are attempting to revise a recipe from another layer we don't control. The recipe we want to make alterations to is from an open source project, so we'd prefer to not make changes directly to the recipe, rather change it using a bbappend. 

The issue we're having is that the recipe has a pkg_postinst_ontarget function defined, but we have our filesystem set to read only via 'read-only-rootfs'. The result is a build time failure:

'The following packages could not be configured offline and rootfs is read-only: ['<package>']

Looking at the Yocto code, it appears that this error is detected by checking for the presence of any files in the image rootfs/rpm-postinsts directory. If something is found and the rootfs is read only, then the error above is thrown and the build fails. 

We also looked at emit_pkgdata() and noted it was checking for the pkg_postinst/pkg_postinst_ontarget globals, so we attempted to add a task that set those to 'None', hoping it would result in the conditional checks for these resolving to 'false', however this did not seem to work either. I'm somewhat new to Yocto, so may have misunderstood how this code is applied.

We've also tried simpler solutions, like implementing a blank pkg_postinst_ontarget_{PN} and using deltask, however this also failed. 

So the question I have is: is there a way for us to remove the pkg_postinst_ontarget_{PN} from the recipe in the original layer using a bbappend file in our layer, so that the read only error is not thrown?

Thanks!




Re: Removing postinst script in bbappend

Alexander Kanavin
 

The right way would be to fix the recipe I think? What is doing that has to happen only on target and can't be done during do_rootfs?

Alex


On Wed, 4 Nov 2020 at 21:19, Jim Krantz <krantzj84@...> wrote:
The recipe is apparmor, layer is meta-security. 

We're currently using the 2.x recipe, however the 3.x recipe appears to have the same function.


On Wed, Nov 4, 2020 at 2:32 PM Alexander Kanavin <alex.kanavin@...> wrote:
What recipe is it and what layer is it in?

Alex

On Wed 4. Nov 2020 at 19.04, Jim Krantz <krantzj84@...> wrote:
Hey folks,

We're doing builds using Yocto 3.1 and are attempting to revise a recipe from another layer we don't control. The recipe we want to make alterations to is from an open source project, so we'd prefer to not make changes directly to the recipe, rather change it using a bbappend. 

The issue we're having is that the recipe has a pkg_postinst_ontarget function defined, but we have our filesystem set to read only via 'read-only-rootfs'. The result is a build time failure:

'The following packages could not be configured offline and rootfs is read-only: ['<package>']

Looking at the Yocto code, it appears that this error is detected by checking for the presence of any files in the image rootfs/rpm-postinsts directory. If something is found and the rootfs is read only, then the error above is thrown and the build fails. 

We also looked at emit_pkgdata() and noted it was checking for the pkg_postinst/pkg_postinst_ontarget globals, so we attempted to add a task that set those to 'None', hoping it would result in the conditional checks for these resolving to 'false', however this did not seem to work either. I'm somewhat new to Yocto, so may have misunderstood how this code is applied.

We've also tried simpler solutions, like implementing a blank pkg_postinst_ontarget_{PN} and using deltask, however this also failed. 

So the question I have is: is there a way for us to remove the pkg_postinst_ontarget_{PN} from the recipe in the original layer using a bbappend file in our layer, so that the read only error is not thrown?

Thanks!




Re: Removing postinst script in bbappend

Jim Krantz
 

The recipe is apparmor, layer is meta-security. 

We're currently using the 2.x recipe, however the 3.x recipe appears to have the same function.


On Wed, Nov 4, 2020 at 2:32 PM Alexander Kanavin <alex.kanavin@...> wrote:
What recipe is it and what layer is it in?

Alex

On Wed 4. Nov 2020 at 19.04, Jim Krantz <krantzj84@...> wrote:
Hey folks,

We're doing builds using Yocto 3.1 and are attempting to revise a recipe from another layer we don't control. The recipe we want to make alterations to is from an open source project, so we'd prefer to not make changes directly to the recipe, rather change it using a bbappend. 

The issue we're having is that the recipe has a pkg_postinst_ontarget function defined, but we have our filesystem set to read only via 'read-only-rootfs'. The result is a build time failure:

'The following packages could not be configured offline and rootfs is read-only: ['<package>']

Looking at the Yocto code, it appears that this error is detected by checking for the presence of any files in the image rootfs/rpm-postinsts directory. If something is found and the rootfs is read only, then the error above is thrown and the build fails. 

We also looked at emit_pkgdata() and noted it was checking for the pkg_postinst/pkg_postinst_ontarget globals, so we attempted to add a task that set those to 'None', hoping it would result in the conditional checks for these resolving to 'false', however this did not seem to work either. I'm somewhat new to Yocto, so may have misunderstood how this code is applied.

We've also tried simpler solutions, like implementing a blank pkg_postinst_ontarget_{PN} and using deltask, however this also failed. 

So the question I have is: is there a way for us to remove the pkg_postinst_ontarget_{PN} from the recipe in the original layer using a bbappend file in our layer, so that the read only error is not thrown?

Thanks!




Re: Removing postinst script in bbappend

Alexander Kanavin
 

What recipe is it and what layer is it in?

Alex

On Wed 4. Nov 2020 at 19.04, Jim Krantz <krantzj84@...> wrote:
Hey folks,

We're doing builds using Yocto 3.1 and are attempting to revise a recipe from another layer we don't control. The recipe we want to make alterations to is from an open source project, so we'd prefer to not make changes directly to the recipe, rather change it using a bbappend. 

The issue we're having is that the recipe has a pkg_postinst_ontarget function defined, but we have our filesystem set to read only via 'read-only-rootfs'. The result is a build time failure:

'The following packages could not be configured offline and rootfs is read-only: ['<package>']

Looking at the Yocto code, it appears that this error is detected by checking for the presence of any files in the image rootfs/rpm-postinsts directory. If something is found and the rootfs is read only, then the error above is thrown and the build fails. 

We also looked at emit_pkgdata() and noted it was checking for the pkg_postinst/pkg_postinst_ontarget globals, so we attempted to add a task that set those to 'None', hoping it would result in the conditional checks for these resolving to 'false', however this did not seem to work either. I'm somewhat new to Yocto, so may have misunderstood how this code is applied.

We've also tried simpler solutions, like implementing a blank pkg_postinst_ontarget_{PN} and using deltask, however this also failed. 

So the question I have is: is there a way for us to remove the pkg_postinst_ontarget_{PN} from the recipe in the original layer using a bbappend file in our layer, so that the read only error is not thrown?

Thanks!



1201 - 1220 of 52464