Date   

[meta-selinux][PATCH 05/16] libselinux-python: upgrade to 3.0 (20191204)

Yi Zhao
 

* Inherit python3native as the libselinux uses python distutils to install
selinux python bindings now.
* Add a patch to fix python modules install path for multilib.

Signed-off-by: Yi Zhao <yi.zhao@...>
---
.../selinux/libselinux-python.inc | 9 +++---
...python_2.9.bb => libselinux-python_3.0.bb} | 8 ++++--
...hon-modules-install-path-for-multili.patch | 28 +++++++++++++++++++
3 files changed, 38 insertions(+), 7 deletions(-)
rename recipes-security/selinux/{libselinux-python_2.9.bb => libselinux-python_3.0.bb} (61%)
create mode 100644 recipes-security/selinux/libselinux/0001-Makefile-fix-python-modules-install-path-for-multili.patch

diff --git a/recipes-security/selinux/libselinux-python.inc b/recipes-security/selinux/libselinux-python.inc
index 6a64473..3760fd8 100644
--- a/recipes-security/selinux/libselinux-python.inc
+++ b/recipes-security/selinux/libselinux-python.inc
@@ -7,9 +7,9 @@ LICENSE = "PD"

FILESEXTRAPATHS_prepend := "${THISDIR}/libselinux:"

-inherit python3-dir
+inherit python3native

-DEPENDS += "python3 swig-native"
+DEPENDS += "python3 swig-native libpcre libsepol"
RDEPENDS_${PN} += "libselinux python3-core python3-shell"

def get_policyconfigarch(d):
@@ -24,6 +24,7 @@ EXTRA_OEMAKE += "LDFLAGS='${LDFLAGS} -lpcre' LIBSEPOLA='${STAGING_LIBDIR}/libsep
EXTRA_OEMAKE_append_libc-musl = " FTS_LDLIBS=-lfts"

FILES_${PN} = "${libdir}/python${PYTHON_BASEVERSION}/site-packages/*"
+INSANE_SKIP_${PN} = "dev-so"

do_compile() {
oe_runmake pywrap -j1 \
@@ -34,7 +35,7 @@ do_compile() {

do_install() {
oe_runmake install-pywrap \
- PYCEXT='.so' \
+ DESTDIR=${D} \
PYLIBVER='python${PYTHON_BASEVERSION}${PYTHON_ABI}' \
- PYTHONLIBDIR='${D}${libdir}/python${PYTHON_BASEVERSION}/site-packages'
+ PYTHONLIBDIR='${libdir}/python${PYTHON_BASEVERSION}/site-packages'
}
diff --git a/recipes-security/selinux/libselinux-python_2.9.bb b/recipes-security/selinux/libselinux-python_3.0.bb
similarity index 61%
rename from recipes-security/selinux/libselinux-python_2.9.bb
rename to recipes-security/selinux/libselinux-python_3.0.bb
index 8e3aae1..e024a22 100644
--- a/recipes-security/selinux/libselinux-python_2.9.bb
+++ b/recipes-security/selinux/libselinux-python_3.0.bb
@@ -1,4 +1,4 @@
-SELINUX_RELEASE = "20190315"
+SELINUX_RELEASE = "20191204"

SRC_URI = "https://github.com/SELinuxProject/selinux/releases/download/${SELINUX_RELEASE}/libselinux-${PV}.tar.gz"

@@ -6,13 +6,15 @@ require ${BPN}.inc

LIC_FILES_CHKSUM = "file://LICENSE;md5=84b4d2c6ef954a2d4081e775a270d0d0"

-SRC_URI[md5sum] = "bb449431b6ed55a0a0496dbc366d6e31"
-SRC_URI[sha256sum] = "1bccc8873e449587d9a2b2cf253de9b89a8291b9fbc7c59393ca9e5f5f4d2693"
+SRC_URI[md5sum] = "b387a66f087b6d97713570e85ec89d89"
+SRC_URI[sha256sum] = "2ea2b30f671dae9d6b1391cbe8fb2ce5d36a3ee4fb1cd3c32f0d933c31b82433"

SRC_URI += "\
file://libselinux-drop-Wno-unused-but-set-variable.patch \
file://libselinux-make-O_CLOEXEC-optional.patch \
file://libselinux-make-SOCK_CLOEXEC-optional.patch \
file://libselinux-define-FD_CLOEXEC-as-necessary.patch \
+ file://0001-Fix-building-against-musl-and-uClibc-libc-libraries.patch \
+ file://0001-Makefile-fix-python-modules-install-path-for-multili.patch \
"
S = "${WORKDIR}/libselinux-${PV}"
diff --git a/recipes-security/selinux/libselinux/0001-Makefile-fix-python-modules-install-path-for-multili.patch b/recipes-security/selinux/libselinux/0001-Makefile-fix-python-modules-install-path-for-multili.patch
new file mode 100644
index 0000000..f0fee23
--- /dev/null
+++ b/recipes-security/selinux/libselinux/0001-Makefile-fix-python-modules-install-path-for-multili.patch
@@ -0,0 +1,28 @@
+From 930514c1b93335ccf6d70adf46ca7e3f8183603d Mon Sep 17 00:00:00 2001
+From: Yi Zhao <yi.zhao@...>
+Date: Mon, 13 Apr 2020 12:44:23 +0800
+Subject: [PATCH] Makefile: fix python modules install path for multilib
+
+Upstream-Status: Inappropriate [embedded specific]
+
+Signed-off-by: Yi Zhao <yi.zhao@...>
+---
+ src/Makefile | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/src/Makefile b/src/Makefile
+index b0ce2c8..a384a10 100644
+--- a/src/Makefile
++++ b/src/Makefile
+@@ -173,7 +173,7 @@ install: all
+ ln -sf --relative $(DESTDIR)$(SHLIBDIR)/$(LIBSO) $(DESTDIR)$(LIBDIR)/$(TARGET)
+
+ install-pywrap: pywrap
+- $(PYTHON) setup.py install --prefix=$(PREFIX) `test -n "$(DESTDIR)" && echo --root $(DESTDIR)`
++ $(PYTHON) setup.py install --prefix=$(PREFIX) --root=$(DESTDIR) --install-lib=$(PYTHONLIBDIR)
+ install -m 644 $(SWIGPYOUT) $(DESTDIR)$(PYTHONLIBDIR)/selinux/__init__.py
+ ln -sf --relative $(DESTDIR)$(PYTHONLIBDIR)/selinux/_selinux$(PYCEXT) $(DESTDIR)$(PYTHONLIBDIR)/_selinux$(PYCEXT)
+
+--
+2.7.4
+
--
2.17.1


[meta-selinux][PATCH 01/16] setools: upgrade 4.2.2 -> 4.3.0

Yi Zhao
 

Remove __pycache__ directories when do_install.

Signed-off-by: Yi Zhao <yi.zhao@...>
---
.../setools/{setools_4.2.2.bb => setools_4.3.0.bb} | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
rename recipes-security/setools/{setools_4.2.2.bb => setools_4.3.0.bb} (78%)

diff --git a/recipes-security/setools/setools_4.2.2.bb b/recipes-security/setools/setools_4.3.0.bb
similarity index 78%
rename from recipes-security/setools/setools_4.2.2.bb
rename to recipes-security/setools/setools_4.3.0.bb
index 6e5a950..ec73f7c 100644
--- a/recipes-security/setools/setools_4.2.2.bb
+++ b/recipes-security/setools/setools_4.3.0.bb
@@ -9,17 +9,17 @@ SECTION = "base"
LICENSE = "GPLv2 & LGPLv2.1"

S = "${WORKDIR}/git"
-SRC_URI = "git://github.com/SELinuxProject/${BPN}.git;branch=4.2 \
+SRC_URI = "git://github.com/SELinuxProject/${BPN}.git;branch=4.3 \
file://setools4-fixes-for-cross-compiling.patch \
"

-SRCREV = "15bffa7823b9a999f9d51533785ade18fe44df08"
+SRCREV = "a57ad3cdb669a39f785c4e85d63416a469c8d445"

LIC_FILES_CHKSUM = "file://${S}/COPYING;md5=83a5eb6974c11f30785e90d0eeccf40c \
file://${S}/COPYING.GPL;md5=b234ee4d69f5fce4486a80fdaf4a4263 \
file://${S}/COPYING.LGPL;md5=4fbd65380cdd255951079008b364516c"

-DEPENDS += "bison-native flex-native swig-native python3 python3-cython-native libsepol"
+DEPENDS += "bison-native flex-native swig-native python3 python3-cython-native libsepol libselinux"

RDEPENDS_${PN} += "python3-networkx python3-decorator python3-setuptools \
python3-logging python3-json libselinux-python"
@@ -32,4 +32,6 @@ do_install_append() {
# Need PyQt5 support, disable gui tools
rm -f ${D}${bindir}/apol
rm -rf ${D}${libdir}/${PYTHON_DIR}/site-packages/setoolsgui
+ rm -rf ${D}${libdir}/${PYTHON_DIR}/site-packages/setools/__pycache__
+ rm -rf ${D}${libdir}/${PYTHON_DIR}/site-packages/setools/*/__pycache__
}
--
2.17.1


[meta-selinux][PATCH 03/16] libsepol: upgrade to 3.0 (20191204)

Yi Zhao
 

Signed-off-by: Yi Zhao <yi.zhao@...>
---
recipes-security/selinux/libsepol_2.9.bb | 7 -------
recipes-security/selinux/libsepol_3.0.bb | 7 +++++++
2 files changed, 7 insertions(+), 7 deletions(-)
delete mode 100644 recipes-security/selinux/libsepol_2.9.bb
create mode 100644 recipes-security/selinux/libsepol_3.0.bb

diff --git a/recipes-security/selinux/libsepol_2.9.bb b/recipes-security/selinux/libsepol_2.9.bb
deleted file mode 100644
index cd55be6..0000000
--- a/recipes-security/selinux/libsepol_2.9.bb
+++ /dev/null
@@ -1,7 +0,0 @@
-require selinux_20190315.inc
-require ${BPN}.inc
-
-LIC_FILES_CHKSUM = "file://COPYING;md5=a6f89e2100d9b6cdffcea4f398e37343"
-
-SRC_URI[md5sum] = "2fdefe870a61424d8f2d5d37551c6259"
-SRC_URI[sha256sum] = "a34b12b038d121e3e459b1cbaca3c9202e983137819c16baf63658390e3f1d5d"
diff --git a/recipes-security/selinux/libsepol_3.0.bb b/recipes-security/selinux/libsepol_3.0.bb
new file mode 100644
index 0000000..6c85256
--- /dev/null
+++ b/recipes-security/selinux/libsepol_3.0.bb
@@ -0,0 +1,7 @@
+require selinux_20191204.inc
+require ${BPN}.inc
+
+LIC_FILES_CHKSUM = "file://COPYING;md5=a6f89e2100d9b6cdffcea4f398e37343"
+
+SRC_URI[md5sum] = "22ddb9994910cb9cfff5cb9663cb7ae7"
+SRC_URI[sha256sum] = "5b7ae1881909f1048b06f7a0c364c5c8a86ec12e0ec76e740fe9595a6033eb79"
--
2.17.1


[meta-selinux][PATCH 02/16] selinux: upgrade inc files to 3.0 (20191204)

Yi Zhao
 

Signed-off-by: Yi Zhao <yi.zhao@...>
---
.../selinux/{selinux_20190315.inc => selinux_20191204.inc} | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
rename recipes-security/selinux/{selinux_20190315.inc => selinux_20191204.inc} (90%)

diff --git a/recipes-security/selinux/selinux_20190315.inc b/recipes-security/selinux/selinux_20191204.inc
similarity index 90%
rename from recipes-security/selinux/selinux_20190315.inc
rename to recipes-security/selinux/selinux_20191204.inc
index e79dd54..113fc30 100644
--- a/recipes-security/selinux/selinux_20190315.inc
+++ b/recipes-security/selinux/selinux_20191204.inc
@@ -1,4 +1,4 @@
-SELINUX_RELEASE = "20190315"
+SELINUX_RELEASE = "20191204"

SRC_URI = "https://github.com/SELinuxProject/selinux/releases/download/${SELINUX_RELEASE}/${BPN}-${PV}.tar.gz"

--
2.17.1


Re: [meta-intel][PATCH] conf:machine: Use weak reference for SERIAL_CONSOLES variable

Anuj Mittal
 

On Tue, 2020-04-14 at 08:25 +0200, Marek Belisko wrote:
This add possibility to override in custom layer.

Signed-off-by: Marek Belisko <marek.belisko@...>
---
conf/machine/intel-core2-32.conf | 2 +-
conf/machine/intel-corei7-64.conf | 2 +-
conf/machine/intel-skylake-64.conf | 2 +-
3 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/conf/machine/intel-core2-32.conf b/conf/machine/intel-
core2-32.conf
index 20c9872..384ad1e 100644
--- a/conf/machine/intel-core2-32.conf
+++ b/conf/machine/intel-core2-32.conf
@@ -27,7 +27,7 @@ XSERVER ?= "${XSERVER_X86_BASE} \
"

SYSLINUX_OPTS = "serial 0 115200"
-SERIAL_CONSOLES = "115200;ttyS0 115200;ttyS1 115200;ttyPCH0"
+SERIAL_CONSOLES ?= "115200;ttyS0 115200;ttyS1 115200;ttyPCH0"
I think having just this would make this statement not do anything as
these machine files includes x86-base.inc which also sets this variable
using a ?=.

This would probably also need a ??= in x86-base.inc.

Thanks,

Anuj

APPEND += "rootwait console=ttyS0,115200 console=ttyPCH0,115200
console=tty0"

IMAGE_FSTYPES += "wic"
diff --git a/conf/machine/intel-corei7-64.conf b/conf/machine/intel-
corei7-64.conf
index 6164bf3..2009537 100644
--- a/conf/machine/intel-corei7-64.conf
+++ b/conf/machine/intel-corei7-64.conf
@@ -32,7 +32,7 @@ XSERVER ?= "${XSERVER_X86_BASE} \
"

SYSLINUX_OPTS = "serial 0 115200"
-SERIAL_CONSOLES = "115200;ttyS0 115200;ttyS1 115200;ttyS2"
+SERIAL_CONSOLES ?= "115200;ttyS0 115200;ttyS1 115200;ttyS2"
APPEND += "rootwait console=ttyS0,115200 console=tty0"

IMAGE_FSTYPES += "wic"
diff --git a/conf/machine/intel-skylake-64.conf b/conf/machine/intel-
skylake-64.conf
index e367951..503a982 100644
--- a/conf/machine/intel-skylake-64.conf
+++ b/conf/machine/intel-skylake-64.conf
@@ -25,7 +25,7 @@ XSERVER ?= "${XSERVER_X86_BASE} \
"

SYSLINUX_OPTS = "serial 0 115200"
-SERIAL_CONSOLES = "115200;ttyS0 115200;ttyS1 115200;ttyS2"
+SERIAL_CONSOLES ?= "115200;ttyS0 115200;ttyS1 115200;ttyS2"
APPEND += "rootwait console=ttyS0,115200 console=tty0"

IMAGE_FSTYPES += "wic"


Re: meta-intel: Override SERIAL_CONSOLES variable

Marek Belisko
 

On Sat, Apr 11, 2020 at 8:09 AM Anuj Mittal <anuj.mittal@...> wrote:

Could you send a patch please?
Done: https://lists.yoctoproject.org/g/yocto/message/49131

Thanks,

Anuj

-----Original Message-----
From: yocto@... <yocto@...> On Behalf Of
Leon Woestenberg
Sent: Saturday, April 11, 2020 04:48 AM
To: Marek Belisko <marek.belisko@...>
Cc: yocto <yocto@...>
Subject: Re: [yocto] meta-intel: Override SERIAL_CONSOLES variable

Hi all,

On Fri, Apr 10, 2020 at 8:56 AM Marek Belisko <marek.belisko@...> wrote:

Hi,

in meta-intel in machine configuration SERIAL_CONSOLES are defined as
: SERIAL_CONSOLES = "115200;ttyS0 115200;ttyS1 115200;ttyS2"
For a more generic solution, could this be made a weak assignment?

I am on a SMARC x86-64 module which only one serial is useful, so I would like to
downtune this variable as well.

Regards,

Leon.
BR,

marek


[meta-intel][PATCH] conf:machine: Use weak reference for SERIAL_CONSOLES variable

Marek Belisko
 

This add possibility to override in custom layer.

Signed-off-by: Marek Belisko <marek.belisko@...>
---
conf/machine/intel-core2-32.conf | 2 +-
conf/machine/intel-corei7-64.conf | 2 +-
conf/machine/intel-skylake-64.conf | 2 +-
3 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/conf/machine/intel-core2-32.conf b/conf/machine/intel-core2-32.conf
index 20c9872..384ad1e 100644
--- a/conf/machine/intel-core2-32.conf
+++ b/conf/machine/intel-core2-32.conf
@@ -27,7 +27,7 @@ XSERVER ?= "${XSERVER_X86_BASE} \
"

SYSLINUX_OPTS = "serial 0 115200"
-SERIAL_CONSOLES = "115200;ttyS0 115200;ttyS1 115200;ttyPCH0"
+SERIAL_CONSOLES ?= "115200;ttyS0 115200;ttyS1 115200;ttyPCH0"
APPEND += "rootwait console=ttyS0,115200 console=ttyPCH0,115200 console=tty0"

IMAGE_FSTYPES += "wic"
diff --git a/conf/machine/intel-corei7-64.conf b/conf/machine/intel-corei7-64.conf
index 6164bf3..2009537 100644
--- a/conf/machine/intel-corei7-64.conf
+++ b/conf/machine/intel-corei7-64.conf
@@ -32,7 +32,7 @@ XSERVER ?= "${XSERVER_X86_BASE} \
"

SYSLINUX_OPTS = "serial 0 115200"
-SERIAL_CONSOLES = "115200;ttyS0 115200;ttyS1 115200;ttyS2"
+SERIAL_CONSOLES ?= "115200;ttyS0 115200;ttyS1 115200;ttyS2"
APPEND += "rootwait console=ttyS0,115200 console=tty0"

IMAGE_FSTYPES += "wic"
diff --git a/conf/machine/intel-skylake-64.conf b/conf/machine/intel-skylake-64.conf
index e367951..503a982 100644
--- a/conf/machine/intel-skylake-64.conf
+++ b/conf/machine/intel-skylake-64.conf
@@ -25,7 +25,7 @@ XSERVER ?= "${XSERVER_X86_BASE} \
"

SYSLINUX_OPTS = "serial 0 115200"
-SERIAL_CONSOLES = "115200;ttyS0 115200;ttyS1 115200;ttyS2"
+SERIAL_CONSOLES ?= "115200;ttyS0 115200;ttyS1 115200;ttyS2"
APPEND += "rootwait console=ttyS0,115200 console=tty0"

IMAGE_FSTYPES += "wic"
--
2.7.4


Re: What is the recommended method to patch a recipe?

Philip Balister
 

On 4/13/20 10:31 AM, Nicolas Dechesne wrote:
On Mon, Apr 13, 2020 at 12:55 PM Alexander Kanavin <alex.kanavin@...>
wrote:

On Mon, 13 Apr 2020 at 12:46, Nicolas Dechesne <
nicolas.dechesne@...> wrote:

A good rule of thumb is to never modify a metadata that is not yours.
That applies to everything:
* if you need to modify a bb file from a 3rd party library, create a
bbappend file in your layer
* if you need to modify a bbappend from a 3rd party, then create another
bbappend in your layer

And by 3rd party i mean any layer/tree which is not yours (OE-core, or
any other layer your are using).
I beg to differ! Bbappends are making it difficult to reason about what
the recipe with all its appends is doing (because they destroy spatial
locality and aren't easily visible), and I would actually discourage using
them if the modification can go to the original recipe.
Ok.. you're right. Well at least my answer is not complete ;)


Is it fixing a problem? Is it adding a build option absent from the
original recipe? Then please take the trouble to submit the change to the
owners of the recipe.
This is correct. We should encourage everyone to report, and even better
propose a fix, to the relevant layer, when the 'problem' is applicable. I
didn't want to discourage that, for sure!
When developing against master, it's a no brainer. When developing against
a stable branch (like probably many users), it's not as obvious. In that
case I think trying to avoid 'touching' somebody else layer remains a good
advice, since it makes upgrades and maintenance trickier.




Similarly, if a bbappend is re-setting the component version to something
else (yes, people do that despite my best efforts to prohibit the
practice), then a much cleaner approach is to actually make a fully copy of
the recipe, or go and fix the original.
I just ran across a bbappend that change configure options, but looking
at the original recipe, there is a PACKAGECONFIG option to do the same
thing. So they could have used that mechanism instram of the bbappend.

Philip

Recipes version upgrades (or downgrade) should be done in a dedicated layer
owned by the 'developer', and in that case, I agree that bbappend should be
prohibited, and the entire recipe should be copied.


Alex



Re: Files get sporadically lost for native packages

Alexander Kanavin
 

Adding Michael (lack of confirmtation email from bugzilla).

Alex


On Mon, 13 Apr 2020 at 19:59, Konrad Weihmann <kweihmann@...> wrote:
Hi Randy,

I'm trying all day to create an account at bugzilla to file the issue,
but somehow I don't get any confirmation mail  (although I tried several
mail accounts today, and no it didn't got stuck in spam;-)) - guess
that's not how it is supposed to be, right :-)? - I don't know where to
address such a problem, so take this reply as FYI.

Best
Konrad

On 03.04.20 21:28, Randy MacLeod wrote:
> On 2020-04-02 4:44 a.m., Konrad Weihmann wrote:
>>
>> To answer your others questions... see below
>>
>> On 02.04.20 05:42, Randy MacLeod wrote:
>>> On 2020-03-28 8:26 a.m., Konrad Weihmann wrote:
>>>> Hi,
>>>>
>>>> I'm facing the following error message sporadically on all branches
>>>> I tried so far (master, zeus, warrior and thud)
>>>>
>>>> The stack trace of python calls that resulted in this
>>>> exception/failure was:
>>>> File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
>>>>       0001:
>>>>   *** 0002:extend_recipe_sysroot(d)
>>>>       0003:
>>>> File: '/build/poky/meta/classes/staging.bbclass', lineno: 551,
>>>> function: extend_recipe_sysroot
>>>>       0547:                    dest = newmanifest[l]
>>>>       0548:                    if l.endswith("/"):
>>>>       0549:                        staging_copydir(l, targetdir,
>>>> dest, seendirs)
>>>>       0550:                        continue
>>>>   *** 0551:                    staging_copyfile(l, targetdir, dest,
>>>> postinsts, seendirs)
>>>>       0552:
>>>>       0553:    bb.note("Installed into sysroot: %s" % str(msg_adding))
>>>>       0554:    bb.note("Skipping as already exists in sysroot: %s" %
>>>> str(msg_exists))
>>>>       0555:
>>>> File: '/build/poky/meta/classes/staging.bbclass', lineno: 152,
>>>> function: staging_copyfile
>>>>       0148:        os.symlink(linkto, dest)
>>>>       0149:        #bb.warn(c)
>>>>       0150:    else:
>>>>       0151:        try:
>>>>   *** 0152:            os.link(c, dest)
>>>>       0153:        except OSError as err:
>>>>       0154:            if err.errno == errno.EXDEV:
>>>>       0155:                bb.utils.copyfile(c, dest)
>>>>       0156:            else:
>>>> Exception: FileNotFoundError: [Errno 2] No such file or directory:
>>>> '/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/__init__.cpython-37.pyc'
>>>> ->
>>>> '/build/poky/build/tmp/work/qemux86_64-mine-linux/core-image-minimal-mine/1.0-r0/recipe-sysroot-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/__init__.cpython-37.pyc'
>>>>
>>>> I already had a look at the manifest
>>>>
>>>> cat manifest-x86_64-python3-msgcheck-native.populate_sysroot
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__init__.py
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/po.py
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/msgcheck.py
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/__init__.cpython-37.pyc
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/po.cpython-37.pyc
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/msgcheck.cpython-37.pyc
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/dependency_links.txt
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/requires.txt
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/top_level.txt
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/SOURCES.txt
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/PKG-INFO
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/entry_points.txt
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/bin/msgcheck
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/sysroot-providers/python3-msgcheck-native
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/sysroot-providers/
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/share/
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/bin/
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/
>>>>
>>>>
>>>> which states the file should be there, but when doing
>>>>
>>>> find
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__init__.py
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/po.cpython-37.pyc
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/msgcheck.cpython-37.pyc
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/po.py
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/msgcheck.py
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/dependency_links.txt
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/requires.txt
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/top_level.txt
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/SOURCES.txt
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/PKG-INFO
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/entry_points.txt
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/bin
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/bin/msgcheck
>>>>
>>>> /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/share
>>>>
>>>>
>>>> the file isn't there.
>>>>
>>>> This happens to random python packages compiled as native (sometimes
>>>> even for python-native itself), but (afaik) not for cross or target
>>>> packages (I'm pretty sure because of the different packaging applied).
>>>> So I digged a little into the code
>>>> classes/sstate.bbclass:sstate_install, which seems to create the
>>>> sysroot-component dir and the manifest.
>>>> There is a gap between the manifest creation (line 285) and the
>>>> hardlinking (line till 311).
>>>>
>>>> Now my question is there any place where a file potentially could
>>>> get lost? - at first glance there shouldn't be one - I have to admit
>>>> that I don't fully understand all this subprocess magic in
>>>> lib/oe/path.py:copyhardlinktree.
>>>> What I do to fix the issue is reopening the manifest and double
>>>> check for missing files and remove them from the manifest, but this
>>>> feels wrong - so any advise is welcome...
>>>>
>>>> Hope that someone more familiar with the topic could have a look.
>>>
>>> Hi Konrad,
>>>
>>> I'm not really familiar with that code but it's being run buy 1000s of
>>> builder around the world so let's try to eliminate a few possibilities.
>>>
>>> When did you start having this problem?
>> Since the start of the test distribution I'm working on. But also for
>> plain poky builds if I forcefully inject all of the python-native
>> site-packages via local.conf (DEPENDS_class-native += "..."), without
>> actually using them in the recipe scope
>>> How often do you think it's happening: 1 in 3 builds, 1 in 10?
>> See the other mail - looks like it heavily depends on the host
>>> Tell us about your machine: OS,version, disk, CPUs, ram
>> See the other mail
>>> Do you do anything special in your conf dir? Send local.conf perhaps.
>> No custom modification (just for testing the DEPENDS-injection)
>>> Do you have any local bbappends or commits on top of poky or
>>> in other layers?
>> No
>>> Have you tried to simplify the build to eliminate problems
>>> potentially caused by other layers?
>> I did - see above
>>> Are you able to reproduce the problem on more than one build machine?
>> See the other mail
>>> Are you able to reproduce the problem on a different Linux distro?
>> Not really - Debian 9 was fine all other Hosts are Ubuntu based
>>> Are there other builds or users on the machine that may be causing
>>> extra load?
>> No the hosts are just being poorly equipped - at least the ones that
>> produce this issue
>
>
> Hi Konrad,
>
> Thanks for the detailed and complete replies.
>
> I don't think I've seen this error and we do 100s of builds
> per day using local many-core systems running Ubuntu-18.04
> but with the builds in docker containers using a variety of
> OS distributions.
>
> My first *wild* guess is that the problem might go away on the Azure
> systems if you allocate more memory. That might be an easy
> test to do so that we can confirm that it happens more frequently
> when there is a memory constraint. Can you do that test?
>
> I've also BCCed someone who might know someone who
> would be interested in fixing Azure + Yocto bugs. Let's see
> if they can help. :)
>
> It would also be helpful if you created a defect in:
>
> https://bugzilla.yoctoproject.org/
>
> and hopefully add a patch in that defect including the -native recipes that
> are required to make the problem happen.
>
> Thanks,
>
> ../Randy
>
>
>>>
>>>
>>> ../Randy
>>>
>>>>
>>>> Thanks
>>>>
>>>> Konrad
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>
>>
>
>
> --
> # Randy MacLeod
> # Wind River Linux
>


Re: Files get sporadically lost for native packages

Konrad Weihmann <kweihmann@...>
 

Hi Randy,

I'm trying all day to create an account at bugzilla to file the issue, but somehow I don't get any confirmation mail (although I tried several mail accounts today, and no it didn't got stuck in spam;-)) - guess that's not how it is supposed to be, right :-)? - I don't know where to address such a problem, so take this reply as FYI.

Best
Konrad

On 03.04.20 21:28, Randy MacLeod wrote:
On 2020-04-02 4:44 a.m., Konrad Weihmann wrote:

To answer your others questions... see below

On 02.04.20 05:42, Randy MacLeod wrote:
On 2020-03-28 8:26 a.m., Konrad Weihmann wrote:
Hi,

I'm facing the following error message sporadically on all branches I tried so far (master, zeus, warrior and thud)

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
      0001:
  *** 0002:extend_recipe_sysroot(d)
      0003:
File: '/build/poky/meta/classes/staging.bbclass', lineno: 551, function: extend_recipe_sysroot
      0547:                    dest = newmanifest[l]
      0548:                    if l.endswith("/"):
      0549:                        staging_copydir(l, targetdir, dest, seendirs)
      0550:                        continue
  *** 0551:                    staging_copyfile(l, targetdir, dest, postinsts, seendirs)
      0552:
      0553:    bb.note("Installed into sysroot: %s" % str(msg_adding))
      0554:    bb.note("Skipping as already exists in sysroot: %s" % str(msg_exists))
      0555:
File: '/build/poky/meta/classes/staging.bbclass', lineno: 152, function: staging_copyfile
      0148:        os.symlink(linkto, dest)
      0149:        #bb.warn(c)
      0150:    else:
      0151:        try:
  *** 0152:            os.link(c, dest)
      0153:        except OSError as err:
      0154:            if err.errno == errno.EXDEV:
      0155:                bb.utils.copyfile(c, dest)
      0156:            else:
Exception: FileNotFoundError: [Errno 2] No such file or directory: '/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/__init__.cpython-37.pyc' -> '/build/poky/build/tmp/work/qemux86_64-mine-linux/core-image-minimal-mine/1.0-r0/recipe-sysroot-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/__init__.cpython-37.pyc'

I already had a look at the manifest

cat manifest-x86_64-python3-msgcheck-native.populate_sysroot
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__init__.py
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/po.py
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/msgcheck.py
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/__init__.cpython-37.pyc
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/po.cpython-37.pyc
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/msgcheck.cpython-37.pyc
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/dependency_links.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/requires.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/top_level.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/SOURCES.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/PKG-INFO
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/entry_points.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/bin/msgcheck
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/sysroot-providers/python3-msgcheck-native
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/sysroot-providers/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/share/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/bin/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/

which states the file should be there, but when doing

find /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__init__.py
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/po.cpython-37.pyc
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/msgcheck.cpython-37.pyc
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/po.py
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/msgcheck.py
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/dependency_links.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/requires.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/top_level.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/SOURCES.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/PKG-INFO
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/entry_points.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/bin
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/bin/msgcheck
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/share

the file isn't there.

This happens to random python packages compiled as native (sometimes even for python-native itself), but (afaik) not for cross or target packages (I'm pretty sure because of the different packaging applied).
So I digged a little into the code classes/sstate.bbclass:sstate_install, which seems to create the sysroot-component dir and the manifest.
There is a gap between the manifest creation (line 285) and the hardlinking (line till 311).

Now my question is there any place where a file potentially could get lost? - at first glance there shouldn't be one - I have to admit that I don't fully understand all this subprocess magic in lib/oe/path.py:copyhardlinktree.
What I do to fix the issue is reopening the manifest and double check for missing files and remove them from the manifest, but this
feels wrong - so any advise is welcome...

Hope that someone more familiar with the topic could have a look.
Hi Konrad,

I'm not really familiar with that code but it's being run buy 1000s of
builder around the world so let's try to eliminate a few possibilities.

When did you start having this problem?
Since the start of the test distribution I'm working on. But also for plain poky builds if I forcefully inject all of the python-native site-packages via local.conf (DEPENDS_class-native += "..."), without actually using them in the recipe scope
How often do you think it's happening: 1 in 3 builds, 1 in 10?
See the other mail - looks like it heavily depends on the host
Tell us about your machine: OS,version, disk, CPUs, ram
See the other mail
Do you do anything special in your conf dir? Send local.conf perhaps.
No custom modification (just for testing the DEPENDS-injection)
Do you have any local bbappends or commits on top of poky or
in other layers?
No
Have you tried to simplify the build to eliminate problems
potentially caused by other layers?
I did - see above
Are you able to reproduce the problem on more than one build machine?
See the other mail
Are you able to reproduce the problem on a different Linux distro?
Not really - Debian 9 was fine all other Hosts are Ubuntu based
Are there other builds or users on the machine that may be causing
extra load?
No the hosts are just being poorly equipped - at least the ones that produce this issue
Hi Konrad,
Thanks for the detailed and complete replies.
I don't think I've seen this error and we do 100s of builds
per day using local many-core systems running Ubuntu-18.04
but with the builds in docker containers using a variety of
OS distributions.
My first *wild* guess is that the problem might go away on the Azure
systems if you allocate more memory. That might be an easy
test to do so that we can confirm that it happens more frequently
when there is a memory constraint. Can you do that test?
I've also BCCed someone who might know someone who
would be interested in fixing Azure + Yocto bugs. Let's see
if they can help. :)
It would also be helpful if you created a defect in:
https://bugzilla.yoctoproject.org/
and hopefully add a patch in that defect including the -native recipes that
are required to make the problem happen.
Thanks,
../Randy



../Randy


Thanks

Konrad




--
# Randy MacLeod
# Wind River Linux


Re: Image size reduction

Randy MacLeod
 

On 2020-04-04 12:27 p.m., Ajam Ali wrote:
Hi All,

Thanks for your suggestions. I am working on your suggestions.
I will let you know if it did not work for me.

Ajam,


It would be useful if you could reply either way.


If there's still a problem then perhaps someone can help.


If you are able to resolve the issue, then a brief summary

of what you did could help the next person who has a similar

issue.


Thanks,

../Randy



Regards,
Ajam Ali



From: yocto@... <yocto@...> on behalf of Gmane Admin via lists.yoctoproject.org <gley-yocto=m.gmane-mx.org@...>
Sent: Saturday, April 4, 2020 9:36 PM
To: yocto@... <yocto@...>
Subject: Re: [yocto] Image size reduction
 
[CAUTION: This Email is from outside the Organization. Do not click links or open attachments unless you trust the sender.]

Op 29-03-2020 om 18:28 schreef Ajam Ali:
> Hi All,
>
> Actually my current image size is 3.9GB because of heavy size packages
> required by my project and without project required packages my image
> size in Yocto is 800MB.

So your project adds 3.1GB right

> I want to reduce the image size as maximum as possible.

3.1GB is huge. Compare to f.i. Libreoffice (installed 0.5GB or so).
Either your own code has a lot of fat, or you are pulling in lots of
packages you don't really need. But is there no way to advise you
without knowing more.

> Please suggest the best possible way so that I could reduce the maximum
> possible size(desirable below 1.5 GB).
>
>
> Thanks in advance,
> Ajam Ali
>
>
> Sent from Outlook Mobile <https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2Fblhgte&amp;data=02%7C01%7Cajama%40hcl.com%7Cf719cb14fe674f86f02408d7d8b236b0%7C189de737c93a4f5a8b686f4ca9941912%7C0%7C0%7C637216132261322392&amp;sdata=8in2t%2B6mXtrsRwa7GmgGQDl69uRzL7g4smj%2Ba3QG9gU%3D&amp;reserved=0>
> ::DISCLAIMER::
> ------------------------------------------------------------------------
> The contents of this e-mail and any attachment(s) are confidential and
> intended for the named recipient(s) only. E-mail transmission is not
> guaranteed to be secure or error-free as information could be
> intercepted, corrupted, lost, destroyed, arrive late or incomplete, or
> may contain viruses in transmission. The e mail and its contents (with
> or without referred errors) shall therefore not attach any liability on
> the originator or HCL or its affiliates. Views or opinions, if any,
> presented in this email are solely those of the author and may not
> necessarily reflect the views or opinions of HCL or its affiliates. Any
> form of reproduction, dissemination, copying, disclosure, modification,
> distribution and / or publication of this message without the prior
> written consent of authorized representative of HCL is strictly
> prohibited. If you have received this email in error please delete it
> and notify the sender immediately. Before opening any email and/or
> attachments, please check them for viruses and other defects.
> ------------------------------------------------------------------------
>
>
>




    


-- 
# Randy MacLeod
# Wind River Linux


M+ & H bugs with Milestone Movements WW15

Stephen Jolley
 

All,

YP M+ or high bugs which moved to a new milestone in WW15 are listed below:

Priority

Bug ID

Short Description

Changer

Owner

Was

Became

High

13604

[master-next] Distrodata.test_maintainers  fails

randy.macleod@...

kai.kang@...

3.1 M4

3.2 M1

 

13802

ltp tests fail with ssh connection lost  error intermittently

randy.macleod@...

yang.wang@...

3.1 M4

3.2 M1

Medium+

12090

bitbake resident server reconnect needed ?

richard.purdie@...

richard.purdie@...

3.99

3.2 M1

 

12367

moving or removing tmp breaks persistent bitbake server

richard.purdie@...

richard.purdie@...

3.99

3.2 M1

 

12368

persistent bitbake server does not re-parse if previous build was ctrl+C'd

richard.purdie@...

richard.purdie@...

3.99

3.2 M1

 

Thanks,

 

Stephen K. Jolley

Yocto Project Program Manager

(    Cell:                (208) 244-4460

* Email:              sjolley.yp.pm@...

 


Enhancements/Bugs closed WW15!

Stephen Jolley
 

All,

The below were the owners of enhancements or bugs closed during the last week!

Who

Count

randy.macleod@...

7

richard.purdie@...

5

timothy.t.orling@...

2

akuster@...

2

ee.peng.yeoh@...

1

changqing.li@...

1

michael@...

1

jpuhlman@...

1

david.reyna@...

1

Grand Total

21

 

Thanks,

 

Stephen K. Jolley

Yocto Project Program Manager

(    Cell:                (208) 244-4460

* Email:              sjolley.yp.pm@...

 


Current Top developers on Yocto Project 3.1

Stephen Jolley
 

All,

Below is the list as of top 43 developers as of the end of WW15 of who have open medium or higher bugs and enhancements against YP 3.1.   There are 13 possible work days left until the final release candidates for YP 3.1 needs to be released.

Who

Count

richard.purdie@...

36

liezhi.yang@...

27

david.reyna@...

21

akuster808@...

16

bluelightning@...

13

bruce.ashfield@...

11

mark.morton@...

11

kai.kang@...

10

randy.macleod@...

10

Qi.Chen@...

10

ross@...

7

timothy.t.orling@...

7

trevor.gamblin@...

6

michael@...

6

pbarker@...

5

JPEWhacker@...

5

changqing.li@...

5

jon.mason@...

5

alejandro@...

4

kexin.hao@...

4

alex.kanavin@...

4

mingli.yu@...

4

matthew.zeng@...

3

hongxu.jia@...

3

mark.hatle@...

2

kergoth@...

2

mostthingsweb@...

2

jaewon@...

2

jpuhlman@...

2

yang.wang@...

2

ycnakajsph@...

2

raj.khem@...

2

seebs@...

2

limon.anibal@...

2

rpjday@...

2

amber.n.elliot@...

1

naveen.kumar.saini@...

1

anuj.mittal@...

1

denis@...

1

ee.peng.yeoh@...

1

mshah@...

1

apoorv.sangal@...

1

kai.ruhnau@...

1

scott.branden@...

1

yi.zhao@...

1

dl9pf@...

1

jonathan.yong@...

1

Martin.Jansa@...

1

maxime.roussinbelanger@...

1

 

Thanks,

 

Stephen K. Jolley

Yocto Project Program Manager

(    Cell:                (208) 244-4460

* Email:              sjolley.yp.pm@...

 


Yocto Project Newcomer & Unassigned Bugs - Help Needed

Stephen Jolley
 

All,

 

The triage team is starting to try and collect up and classify bugs which a newcomer to the project would be able to work on in a way which means people can find them. They're being listed on the triage page under the appropriate heading:

 

https://wiki.yoctoproject.org/wiki/Bug_Triage#Newcomer_Bugs

 

The idea is these bugs should be straight forward for a person to help work on who doesn't have deep experience with the project.  If anyone can help, please take ownership of the bug and send patches!  If anyone needs help/advice there are people on irc who can likely do so, or some of the more experienced contributors will likely be happy to help too.

 

Also, the triage team meets weekly and does its best to handle the bugs reported into the Bugzilla. The number of people attending that meeting has fallen, as have the number of people available to help fix bugs. One of the things we hear users report is they don't know how to help. We (the triage team) are therefore going to start reporting out the currently 309 unassigned or newcomer bugs.

 

We're hoping people may be able to spare some time now and again to help out with these.  Bugs are split into two types, "true bugs" where things don't work as they should and "enhancements" which are features we'd want to add to the system.  There are also roughly four different "priority" classes right now, “3.1”, “3.2, "3.99" and "Future", the more pressing/urgent issues being in "3.1" and then “3.2”.

 

Please review this link and if a bug is something you would be able to help with either take ownership of the bug, or send me (sjolley.yp.pm@...) an e-mail with the bug number you would like and I will assign it to you (please make sure you have a Bugzilla account).  The list is at: https://wiki.yoctoproject.org/wiki/Bug_Triage_Archive#Unassigned_or_Newcomer_Bugs

 

Thanks,

 

Stephen K. Jolley

Yocto Project Program Manager

(    Cell:                (208) 244-4460

* Email:              sjolley.yp.pm@...

 


Re: What is the recommended method to patch a recipe?

Nicolas Dechesne
 



On Mon, Apr 13, 2020 at 12:55 PM Alexander Kanavin <alex.kanavin@...> wrote:
On Mon, 13 Apr 2020 at 12:46, Nicolas Dechesne <nicolas.dechesne@...> wrote:
A good rule of thumb is to never modify a metadata that is not yours. That applies to everything:
* if you need to modify a bb file from a 3rd party library, create a bbappend file in your layer
* if you need to modify a bbappend from a 3rd party, then create another bbappend in your layer

And by 3rd party i mean any layer/tree which is not yours (OE-core, or any other layer your are using).

I beg to differ! Bbappends are making it difficult to reason about what the recipe with all its appends is doing (because they destroy spatial locality and aren't easily visible), and I would actually discourage using them if the modification can go to the original recipe.

Ok.. you're right. Well at least my answer is not complete ;)


Is it fixing a problem? Is it adding a build option absent from the original recipe? Then please take the trouble to submit the change to the owners of the recipe.

This is correct. We should encourage everyone to report, and even better propose a fix, to the relevant layer, when the 'problem' is applicable. I didn't want to discourage that, for sure!
When developing against master, it's a no brainer. When developing against a stable branch (like probably many users), it's not as obvious. In that case I think trying to avoid 'touching' somebody else layer remains a good advice, since it makes upgrades and maintenance trickier.

 

Similarly, if a bbappend is re-setting the component version to something else (yes, people do that despite my best efforts to prohibit the practice), then a much cleaner approach is to actually make a fully copy of the recipe, or go and fix the original.

Recipes version upgrades (or downgrade) should be done in a dedicated layer owned by the 'developer', and in that case, I agree that bbappend should be prohibited, and the entire recipe should be copied.


Alex


Re: What is the recommended method to patch a recipe?

Alexander Kanavin
 

On Mon, 13 Apr 2020 at 12:46, Nicolas Dechesne <nicolas.dechesne@...> wrote:
A good rule of thumb is to never modify a metadata that is not yours. That applies to everything:
* if you need to modify a bb file from a 3rd party library, create a bbappend file in your layer
* if you need to modify a bbappend from a 3rd party, then create another bbappend in your layer

And by 3rd party i mean any layer/tree which is not yours (OE-core, or any other layer your are using).

I beg to differ! Bbappends are making it difficult to reason about what the recipe with all its appends is doing (because they destroy spatial locality and aren't easily visible), and I would actually discourage using them if the modification can go to the original recipe.

Is it fixing a problem? Is it adding a build option absent from the original recipe? Then please take the trouble to submit the change to the owners of the recipe.

Similarly, if a bbappend is re-setting the component version to something else (yes, people do that despite my best efforts to prohibit the practice), then a much cleaner approach is to actually make a fully copy of the recipe, or go and fix the original.

Alex


Re: What is the recommended method to patch a recipe?

Nicolas Dechesne
 



On Mon, Apr 13, 2020 at 10:20 AM nus1998 <nus1998@...> wrote:
Hi All,

I googled some topics on how to patch a recipe, most of them recommend to generate a corresponding <foo>.bbappend to apply the patch, I wonder if there is already a bbappend file, shall I modify the bbappend file directly, or create another layer to overwrite the bbappend file (and including the patchs in original bbappend file)?

A good rule of thumb is to never modify a metadata that is not yours. That applies to everything:
* if you need to modify a bb file from a 3rd party library, create a bbappend file in your layer
* if you need to modify a bbappend from a 3rd party, then create another bbappend in your layer

And by 3rd party i mean any layer/tree which is not yours (OE-core, or any other layer your are using).


Thanks in advance
Nus


Re: What is the recommended method to patch a recipe?

Thomas Goodwin
 

As with a lot of things, I’m sure it all depends on the scope of the change.  Personally, if I ‘own’ the original bbappend, then I consider if it’s worth establishing an override or some other similar mechanism within that bbappend, depending on what I’m developing (an image feature or something machine-specific).  If it’s a one-off for a specific build that I can lump together with other changes, then I go with a new layer specific those changes.

One thing to keep in mind is that if you create your own layer with a bbappend ,then layer priority will impact which portions of your bbappend overwrite the others and the original recipe.  You can use bitbake -e package-name to get a view of how bitbake is overlaying all changes for it.

Cheers,

Thomas

On Apr 13, 2020, 4:20 AM -0400, nus1998 <nus1998@...>, wrote:
Hi All,

I googled some topics on how to patch a recipe, most of them recommend to generate a corresponding <foo>.bbappend to apply the patch, I wonder if there is already a bbappend file, shall I modify the bbappend file directly, or create another layer to overwrite the bbappend file (and including the patchs in original bbappend file)?

Thanks in advance
Nus


What is the recommended method to patch a recipe?

nus1998 <nus1998@...>
 

Hi All,

I googled some topics on how to patch a recipe, most of them recommend to generate a corresponding <foo>.bbappend to apply the patch, I wonder if there is already a bbappend file, shall I modify the bbappend file directly, or create another layer to overwrite the bbappend file (and including the patchs in original bbappend file)?

Thanks in advance
Nus

8281 - 8300 of 57410