Date   

[dunfell][PATCH v2 1/3] bitbake: git.py: skip smudging if lfs=0 is set

Mikko Rapeli <mikko.rapeli@...>
 

From: Mauro Queirós <maurofrqueiros@...>

Git-LFS objects were being fetched even when lfs=0 was not set.
This patch disables LFS smudging when lfs=0. That way, only the LFS pointers
are downloaded during checkout.

(Bitbake rev: 646d86df7de774255246a3d7051c308e43eb257d)

Signed-off-by: Mauro Queiros <maurofrqueiros@...>
Signed-off-by: Richard Purdie <richard.purdie@...>
---
bitbake/lib/bb/fetch2/git.py | 3 +++
1 file changed, 3 insertions(+)

diff --git a/bitbake/lib/bb/fetch2/git.py b/bitbake/lib/bb/fetch2/git.py
index dcecff5d38..59afc32de1 100644
--- a/bitbake/lib/bb/fetch2/git.py
+++ b/bitbake/lib/bb/fetch2/git.py
@@ -475,6 +475,9 @@ class Git(FetchMethod):

need_lfs = ud.parm.get("lfs", "1") == "1"

+ if not need_lfs:
+ ud.basecmd = "GIT_LFS_SKIP_SMUDGE=1 " + ud.basecmd
+
source_found = False
source_error = []

--
2.20.1


Re: [OE-core] [dunfell][PATCH 1/2] bitbake: git.py: Use the correct branch to check if the repository has LFS objects.

Mikko Rapeli <mikko.rapeli@...>
 

Sorry, third patch needs to be backported from master for git lfs to work
nicely on dunfell. Will send a v2.

-Mikko


SDKMACHINE and ppc64le issue with latest head?

Andrew Geissler
 

Over in OpenBMC, we utilize a mix of x86 and ppc64le machines for our CI.

Our latest rebase of poky master (https://gerrit.openbmc-project.xyz/c/openbmc/openbmc/+/39533/) has started failing to compile on our ppc64le machines with the below error.

I can workaround it by setting "SDKMACHINE=x86_64" but that seems a bit weird on a ppc64le machine?

+ bitbake obmc-phosphor-image obmc-phosphor-debug-tarball
ERROR: OE-core's config sanity checker detected a potential misconfiguration.
    Either fix the cause of this error or at your own risk disable the checker (see sanity.conf).
    Following is the list of potential problems / advisories:

    Specified SDKMACHINE value is not valid


[dunfell][PATCH 2/2] bitbake: fetch/git: download LFS content too during do_fetch

Mikko Rapeli <mikko.rapeli@...>
 

From: Matt Hoosier <matt.hoosier@...>

Insert an explicit pass to fetch all blobs needed by Git LFS, during the
fetch() function. This avoids the default behavior of Git LFS to wait
until 'git checkout' to begin downloading the blobs pointed to by LFS rec=
ords.
Network access is not allowed at that point in the recipe's lifecycle.

[YOCTO #14191]

(Bitbake rev: 0efac66043662e7a2027192f50e92e982db2ba1c)

Signed-off-by: Matt Hoosier <matt.hoosier@...>
Signed-off-by: Richard Purdie <richard.purdie@...>
---
bitbake/lib/bb/fetch2/git.py | 44 ++++++++++++++++++++++++++++++++---
bitbake/lib/bb/tests/fetch.py | 28 +++++++++++++++-------
2 files changed, 61 insertions(+), 11 deletions(-)

diff --git a/bitbake/lib/bb/fetch2/git.py b/bitbake/lib/bb/fetch2/git.py
index fc14192e24..85c45d4dd4 100644
--- a/bitbake/lib/bb/fetch2/git.py
+++ b/bitbake/lib/bb/fetch2/git.py
@@ -378,6 +378,35 @@ class Git(FetchMethod):
if missing_rev:
raise bb.fetch2.FetchError("Unable to find revision %s e=
ven from upstream" % missing_rev)
=20
+ if self._contains_lfs(ud, d, ud.clonedir) and self._need_lfs(ud)=
:
+ # Unpack temporary working copy, use it to run 'git checkout=
' to force pre-fetching
+ # of all LFS blobs needed at the the srcrev.
+ #
+ # It would be nice to just do this inline here by running 'g=
it-lfs fetch'
+ # on the bare clonedir, but that operation requires a workin=
g copy on some
+ # releases of Git LFS.
+ tmpdir =3D tempfile.mkdtemp(dir=3Dd.getVar('DL_DIR'))
+ try:
+ # Do the checkout. This implicitly involves a Git LFS fe=
tch.
+ self.unpack(ud, tmpdir, d)
+
+ # Scoop up a copy of any stuff that Git LFS downloaded. =
Merge them into
+ # the bare clonedir.
+ #
+ # As this procedure is invoked repeatedly on incremental=
fetches as
+ # a recipe's SRCREV is bumped throughout its lifetime, t=
his will
+ # result in a gradual accumulation of LFS blobs in <ud.c=
lonedir>/lfs
+ # corresponding to all the blobs reachable from the diff=
erent revs
+ # fetched across time.
+ #
+ # Only do this if the unpack resulted in a .git/lfs dire=
ctory being
+ # created; this only happens if at least one blob needed=
to be
+ # downloaded.
+ if os.path.exists(os.path.join(tmpdir, "git", ".git", "l=
fs")):
+ runfetchcmd("tar -cf - lfs | tar -xf - -C %s" % ud.c=
lonedir, d, workdir=3D"%s/git/.git" % tmpdir)
+ finally:
+ bb.utils.remove(tmpdir, recurse=3DTrue)
+
def build_mirror_data(self, ud, d):
if ud.shallow and ud.write_shallow_tarballs:
if not os.path.exists(ud.fullshallow):
@@ -473,7 +502,7 @@ class Git(FetchMethod):
if os.path.exists(destdir):
bb.utils.prunedir(destdir)
=20
- need_lfs =3D ud.parm.get("lfs", "1") =3D=3D "1"
+ need_lfs =3D self._need_lfs(ud)
=20
source_found =3D False
source_error =3D []
@@ -559,6 +588,9 @@ class Git(FetchMethod):
raise bb.fetch2.FetchError("The command '%s' gave output wit=
h more then 1 line unexpectedly, output: '%s'" % (cmd, output))
return output.split()[0] !=3D "0"
=20
+ def _need_lfs(self, ud):
+ return ud.parm.get("lfs", "1") =3D=3D "1"
+
def _contains_lfs(self, ud, d, wd):
"""
Check if the repository has 'lfs' (large file) content
@@ -569,8 +601,14 @@ class Git(FetchMethod):
else:
branchname =3D "master"
=20
- cmd =3D "%s grep lfs origin/%s:.gitattributes | wc -l" % (
- ud.basecmd, ud.branches[ud.names[0]])
+ # The bare clonedir doesn't use the remote names; it has the bra=
nch immediately.
+ if wd =3D=3D ud.clonedir:
+ refname =3D ud.branches[ud.names[0]]
+ else:
+ refname =3D "origin/%s" % ud.branches[ud.names[0]]
+
+ cmd =3D "%s grep lfs %s:.gitattributes | wc -l" % (
+ ud.basecmd, refname)
=20
try:
output =3D runfetchcmd(cmd, d, quiet=3DTrue, workdir=3Dwd)
diff --git a/bitbake/lib/bb/tests/fetch.py b/bitbake/lib/bb/tests/fetch.p=
y
index 4702c99a7e..9453c90d2b 100644
--- a/bitbake/lib/bb/tests/fetch.py
+++ b/bitbake/lib/bb/tests/fetch.py
@@ -2046,13 +2046,14 @@ class GitLfsTest(FetcherTest):
cwd =3D self.gitdir
return bb.process.run(cmd, cwd=3Dcwd)[0]
=20
- def fetch(self, uri=3DNone):
+ def fetch(self, uri=3DNone, download=3DTrue):
uris =3D self.d.getVar('SRC_URI').split()
uri =3D uris[0]
d =3D self.d
=20
fetcher =3D bb.fetch2.Fetch(uris, d)
- fetcher.download()
+ if download:
+ fetcher.download()
ud =3D fetcher.ud[uri]
return fetcher, ud
=20
@@ -2062,16 +2063,21 @@ class GitLfsTest(FetcherTest):
uri =3D 'git://%s;protocol=3Dfile;subdir=3D${S};lfs=3D1' % self.=
srcdir
self.d.setVar('SRC_URI', uri)
=20
- fetcher, ud =3D self.fetch()
+ # Careful: suppress initial attempt at downloading until
+ # we know whether git-lfs is installed.
+ fetcher, ud =3D self.fetch(uri=3DNone, download=3DFalse)
self.assertIsNotNone(ud.method._find_git_lfs)
=20
- # If git-lfs can be found, the unpack should be successful
- ud.method._find_git_lfs =3D lambda d: True
- shutil.rmtree(self.gitdir, ignore_errors=3DTrue)
- fetcher.unpack(self.d.getVar('WORKDIR'))
+ # If git-lfs can be found, the unpack should be successful. Only
+ # attempt this with the real live copy of git-lfs installed.
+ if ud.method._find_git_lfs(self.d):
+ fetcher.download()
+ shutil.rmtree(self.gitdir, ignore_errors=3DTrue)
+ fetcher.unpack(self.d.getVar('WORKDIR'))
=20
# If git-lfs cannot be found, the unpack should throw an error
with self.assertRaises(bb.fetch2.FetchError):
+ fetcher.download()
ud.method._find_git_lfs =3D lambda d: False
shutil.rmtree(self.gitdir, ignore_errors=3DTrue)
fetcher.unpack(self.d.getVar('WORKDIR'))
@@ -2082,10 +2088,16 @@ class GitLfsTest(FetcherTest):
uri =3D 'git://%s;protocol=3Dfile;subdir=3D${S};lfs=3D0' % self.=
srcdir
self.d.setVar('SRC_URI', uri)
=20
+ # In contrast to test_lfs_enabled(), allow the implicit download
+ # done by self.fetch() to occur here. The point of this test cas=
e
+ # is to verify that the fetcher can survive even if the source
+ # repository has Git LFS usage configured.
fetcher, ud =3D self.fetch()
self.assertIsNotNone(ud.method._find_git_lfs)
=20
- # If git-lfs can be found, the unpack should be successful
+ # If git-lfs can be found, the unpack should be successful. A
+ # live copy of git-lfs is not required for this case, so
+ # unconditionally forge its presence.
ud.method._find_git_lfs =3D lambda d: True
shutil.rmtree(self.gitdir, ignore_errors=3DTrue)
fetcher.unpack(self.d.getVar('WORKDIR'))
--=20
2.20.1


[dunfell][PATCH 1/2] bitbake: git.py: Use the correct branch to check if the repository has LFS objects.

Mikko Rapeli <mikko.rapeli@...>
 

From: Mauro Queirós <maurofrqueiros@...>

Function "contains_lfs" was only looking at the master branch when searching for LFS
content. LFS may be configured in specific branches only, so we need to use the
correct branch.

(Bitbake rev: 4fa67c2af830035a1ddedc14592ee25a15ebff22)

Signed-off-by: Mauro Queiros <maurofrqueiros@...>
Signed-off-by: Richard Purdie <richard.purdie@...>
---
bitbake/lib/bb/fetch2/git.py | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/bitbake/lib/bb/fetch2/git.py b/bitbake/lib/bb/fetch2/git.py
index dcecff5d38..fc14192e24 100644
--- a/bitbake/lib/bb/fetch2/git.py
+++ b/bitbake/lib/bb/fetch2/git.py
@@ -563,8 +563,15 @@ class Git(FetchMethod):
"""
Check if the repository has 'lfs' (large file) content
"""
- cmd = "%s grep lfs HEAD:.gitattributes | wc -l" % (
- ud.basecmd)
+
+ if not ud.nobranch:
+ branchname = ud.branches[ud.names[0]]
+ else:
+ branchname = "master"
+
+ cmd = "%s grep lfs origin/%s:.gitattributes | wc -l" % (
+ ud.basecmd, ud.branches[ud.names[0]])
+
try:
output = runfetchcmd(cmd, d, quiet=True, workdir=wd)
if int(output) > 0:
--
2.20.1


Re: git lfs not working in dunfell

Richard Purdie
 

On Thu, 2021-01-28 at 16:18 +0100, Marek Belisko wrote:
Hi,

I have repo where I'm using lfs. I've added to my SRC_URI =
"git://...... ;lfs=1" and the project is fetched but the issue is that
lfs files are shown as references only (content is not fetched).
I briefly checked git fetcher in bitbake and seems it checks that repo
have lfs content but I cannot see anywhere git lfs pull to actually
pull lfs content. Am I missing something? When checked poky master it
looks like proper fetching is in place. Is there any way how to get it
to dunfell LTS release? Thanks.
There is a patch in bitbake master related to this which you may want
to test and maybe request for adding to dunfell if it helps.

https://git.openembedded.org/bitbake/commit/?id=0efac66043662e7a2027192f50e92e982db2ba1c

Cheers,

Richard


[meta-mingw] [PATCH 2/2] README: Add instructions for configuring git repo for sending patches

Richard Purdie
 

Signed-off-by: Richard Purdie <richard.purdie@...>
---
README | 6 ++++++
1 file changed, 6 insertions(+)

diff --git a/README b/README
index c17f74a..af3ace9 100644
--- a/README
+++ b/README
@@ -15,3 +15,9 @@ Layer Maintainer: Joshua Watt <jpewhacker@...>

Please send changes to the yocto mailing list with [meta-mingw] in the subject line,
cc'ing the maintainer.
+
+This can be configured within the repository with the commands:
+
+git config sendemail.to yocto@...
+git config sendemail.cc jpewhacker@...
+git config format.subjectprefix "meta-mingw] [PATCH"
--
2.27.0


[meta-mingw] [PATCH 1/2] libiconv: Update to work with autoconf 2.70

Richard Purdie
 

Signed-off-by: Richard Purdie <richard.purdie@...>
---
.../libiconv/libiconv/autoconf270.patch | 17 +++++++++++++++++
recipes-support/libiconv/libiconv_1.15.bb | 3 ++-
2 files changed, 19 insertions(+), 1 deletion(-)
create mode 100644 recipes-support/libiconv/libiconv/autoconf270.patch

diff --git a/recipes-support/libiconv/libiconv/autoconf270.patch b/recipes-support/libiconv/libiconv/autoconf270.patch
new file mode 100644
index 0000000..17e6e0b
--- /dev/null
+++ b/recipes-support/libiconv/libiconv/autoconf270.patch
@@ -0,0 +1,17 @@
+Update to add the required gettext version to work with autoconf 2.70
+
+Upstream-Status: Pending
+RP - 2021/1/28
+
+Index: libiconv-1.15/configure.ac
+===================================================================
+--- libiconv-1.15.orig/configure.ac
++++ libiconv-1.15/configure.ac
+@@ -98,6 +98,7 @@ fi
+ gl_VISIBILITY
+ AM_ICONV
+ AM_GNU_GETTEXT([external], [need-ngettext])
++AM_GNU_GETTEXT_VERSION([0.21])
+
+ dnl checks for typedefs
+
diff --git a/recipes-support/libiconv/libiconv_1.15.bb b/recipes-support/libiconv/libiconv_1.15.bb
index e3eacd9..df7f527 100644
--- a/recipes-support/libiconv/libiconv_1.15.bb
+++ b/recipes-support/libiconv/libiconv_1.15.bb
@@ -10,7 +10,8 @@ LICENSE = "LGPLv3"
LIC_FILES_CHKSUM = "file://COPYING.LIB;md5=9f604d8a4f8e74f4f5140845a21b6674 \
file://libcharset/COPYING.LIB;md5=9f604d8a4f8e74f4f5140845a21b6674"

-SRC_URI = "${GNU_MIRROR}/${BPN}/${BPN}-${PV}.tar.gz"
+SRC_URI = "${GNU_MIRROR}/${BPN}/${BPN}-${PV}.tar.gz \
+ file://autoconf270.patch"

SRC_URI[md5sum] = "ace8b5f2db42f7b3b3057585e80d9808"
SRC_URI[sha256sum] = "ccf536620a45458d26ba83887a983b96827001e92a13847b45e4925cc8913178"
--
2.27.0


[meta-gplv2] [PATCH 3/3] grep: Update to work with autoconf 2.70

Richard Purdie
 

Signed-off-by: Richard Purdie <richard.purdie@...>
---
.../grep/grep-2.5.1a/autoconf270.patch | 27 +++++++++++++++++++
recipes-extended/grep/grep_2.5.1a.bb | 1 +
2 files changed, 28 insertions(+)
create mode 100644 recipes-extended/grep/grep-2.5.1a/autoconf270.patch

diff --git a/recipes-extended/grep/grep-2.5.1a/autoconf270.patch b/recipes-extended/grep/grep-2.5.1a/autoconf270.patch
new file mode 100644
index 0000000..fe2cb0b
--- /dev/null
+++ b/recipes-extended/grep/grep-2.5.1a/autoconf270.patch
@@ -0,0 +1,27 @@
+Update to add the required gettext version to work with autoconf 2.70
+
+Also the script needs CPP so add the appropriate macro.
+
+Upstream-Status: Inappropriate [needed for old GPLv2 version only]
+RP - 2021/1/28
+
+Index: grep-2.5.1a/configure.in
+===================================================================
+--- grep-2.5.1a.orig/configure.in
++++ grep-2.5.1a/configure.in
+@@ -27,6 +27,7 @@ AC_PROG_AWK
+ AC_PROG_CC
+ AC_PROG_INSTALL
+ AC_PROG_RANLIB
++AC_PROG_CPP
+
+ dnl Checks for typedefs, structures, and compiler characteristics.
+ AC_SYS_LARGEFILE
+@@ -73,6 +74,7 @@ esac
+ dnl I18N feature
+ ALL_LINGUAS="cs de el eo es et fr gl hr id it ja ko nl no pl pt_BR ru sl sv"
+ AM_GNU_GETTEXT([external])
++AM_GNU_GETTEXT_VERSION([0.21])
+
+ dnl DOS file name convention
+ dnl sets HAVE_DOS_FILE_NAMES
diff --git a/recipes-extended/grep/grep_2.5.1a.bb b/recipes-extended/grep/grep_2.5.1a.bb
index d3b4ddc..5445c7e 100644
--- a/recipes-extended/grep/grep_2.5.1a.bb
+++ b/recipes-extended/grep/grep_2.5.1a.bb
@@ -18,6 +18,7 @@ SRC_URI = "${GNU_MIRROR}/grep/grep-${PV}.tar.bz2 \
file://grep-egrep-fgrep-Fix-LSB-NG-cases.patch \
file://search-fix-compilation-error-with-security-flags-ena.patch \
file://0001-Fix-builds-with-gettext-0.20.patch \
+ file://autoconf270.patch \
"

SRC_URI[md5sum] = "52202fe462770fa6be1bb667bd6cf30c"
--
2.27.0


[meta-gplv2] [PATCH 2/3] diffutils: Update to work with autoconf 2.70

Richard Purdie
 

Signed-off-by: Richard Purdie <richard.purdie@...>
---
.../diffutils/diffutils-2.8.1/autoconf270.patch | 17 +++++++++++++++++
recipes-extended/diffutils/diffutils_2.8.1.bb | 1 +
2 files changed, 18 insertions(+)
create mode 100644 recipes-extended/diffutils/diffutils-2.8.1/autoconf270.patch

diff --git a/recipes-extended/diffutils/diffutils-2.8.1/autoconf270.patch b/recipes-extended/diffutils/diffutils-2.8.1/autoconf270.patch
new file mode 100644
index 0000000..5651675
--- /dev/null
+++ b/recipes-extended/diffutils/diffutils-2.8.1/autoconf270.patch
@@ -0,0 +1,17 @@
+Update to add the required gettext version to work with autoconf 2.70
+
+Upstream-Status: Inappropriate [needed for obsolete GPLv2 version]
+RP - 2021/1/28
+
+Index: diffutils-2.8.1/configure.ac
+===================================================================
+--- diffutils-2.8.1.orig/configure.ac
++++ diffutils-2.8.1/configure.ac
+@@ -62,6 +62,7 @@ AC_CHECK_TYPE(ptrdiff_t, int)
+ AC_CHECK_TYPE(ssize_t, int)
+ jm_AC_TYPE_UINTMAX_T
+ AM_GNU_GETTEXT([external])
++AM_GNU_GETTEXT_VERSION([0.21])
+ XGETTEXT="AWK='$AWK' \$(SHELL) \$(top_srcdir)/exgettext $XGETTEXT"
+ AC_HEADER_DIRENT
+ AC_HEADER_STAT
diff --git a/recipes-extended/diffutils/diffutils_2.8.1.bb b/recipes-extended/diffutils/diffutils_2.8.1.bb
index 7c43c4b..88d189d 100644
--- a/recipes-extended/diffutils/diffutils_2.8.1.bb
+++ b/recipes-extended/diffutils/diffutils_2.8.1.bb
@@ -11,6 +11,7 @@ SRC_URI = "${GNU_MIRROR}/diffutils/diffutils-${PV}.tar.gz \
file://0001-Make-it-build-with-compile-time-hardening-enabled.patch \
file://0002-included-libc-use-mempcpy-instead-of.patch \
file://0003-context-fix-compilation-with-64bit-time_t-on-32bit-a.patch \
+ file://autoconf270.patch \
"

SRC_URI[md5sum] = "71f9c5ae19b60608f6c7f162da86a428"
--
2.27.0


[meta-gplv2] [PATCH 1/3] bash: Update to work with autoconf 2.70

Richard Purdie
 

Signed-off-by: Richard Purdie <richard.purdie@...>
---
.../bash/bash-3.2.57/autoconf270.patch | 17 +++++++++++++++++
recipes-extended/bash/bash_3.2.57.bb | 1 +
2 files changed, 18 insertions(+)
create mode 100644 recipes-extended/bash/bash-3.2.57/autoconf270.patch

diff --git a/recipes-extended/bash/bash-3.2.57/autoconf270.patch b/recipes-extended/bash/bash-3.2.57/autoconf270.patch
new file mode 100644
index 0000000..2249bb9
--- /dev/null
+++ b/recipes-extended/bash/bash-3.2.57/autoconf270.patch
@@ -0,0 +1,17 @@
+Update to add the required gettext version to work with autoconf 2.70
+
+Upstream-Status: Inappropriate [required for bash 3.2.57 (GPLv2) recipe only]
+RP - 2021/1/28
+
+Index: bash-3.2.57/configure.in
+===================================================================
+--- bash-3.2.57.orig/configure.in
++++ bash-3.2.57/configure.in
+@@ -625,6 +625,7 @@ AC_C_CHAR_UNSIGNED
+
+ dnl initialize GNU gettext
+ AM_GNU_GETTEXT([no-libtool], [need-ngettext], [lib/intl])
++AM_GNU_GETTEXT_VERSION([0.21])
+
+ dnl header files
+ AC_HEADER_DIRENT
diff --git a/recipes-extended/bash/bash_3.2.57.bb b/recipes-extended/bash/bash_3.2.57.bb
index 5c288b3..f4db7a5 100644
--- a/recipes-extended/bash/bash_3.2.57.bb
+++ b/recipes-extended/bash/bash_3.2.57.bb
@@ -10,6 +10,7 @@ SRC_URI = "${GNU_MIRROR}/${BPN}/${BP}.tar.gz \
file://run-ptest \
file://dont-include-target-CFLAGS-in-host-LDFLAGS.patch \
file://string-format.patch \
+ file://autoconf270.patch \
"

SRC_URI[md5sum] = "237a8767c990b43ae2c89895c2dbc062"
--
2.27.0


Re: Kernel Initramfs Problems

Martin Townsend <mtownsend1973@...>
 

On Wed, Jan 27, 2021 at 8:01 PM Martin Townsend <mtownsend1973@...> wrote:

On Wed, Jan 27, 2021 at 6:21 PM Martin Townsend <mtownsend1973@...> wrote:

Hi,

I'm trying to get an initramfs working so I can load firmware early
enough in the boot. So I've created an initramfs with the firmware
files and a script to switch to the main root. I've set the following

# Use the FIT image format
KERNEL_IMAGETYPES += "fitImage"
KERNEL_CLASSES += "kernel-fitimage"
KERNEL_IMAGETYPE = "fitImage"
KERNEL_IMAGETYPE_aarch64 = "fitImage"

# Add the initramfs for loading the firmware files
INITRAMFS_IMAGE_BUNDLE = "1"
INITRAMFS_IMAGE = "octave-cad-image-initramfs"
INITRAMFS_FSTYPES = "cpio.gz"

in a configuration file. According to the docs for INITRAMFS_IMAGE_BUNDLE:
"The unpacked initramfs image is then passed to the kernel's Makefile
using the CONFIG_INITRAMFS_SOURCE variable, allowing the initramfs
image to be built into the kernel normally. "

Perfect, this is exactly what I want. Now I'm expecting the kernel
image to contain the initramfs using CONFIG_INITRAMFS_SOURCE and not
include the initramfs in the FIT image as an image configuration that
U-Boot extracts and passes to the kernel.

But I'm not seeing this. I get two FIT images deployed, one with an
initramfs included as a configuration and a FIT image without. This
doesn't seem right as the initramfs should already be in the kernel so
including it in the FIT image means it would be there twice. Ignoring
this the FIT image without an initramfs in the filename should still
have the initramfs in the kernel via CONFIG_INITRAMFS_SOURCE but it's
not there. When booting this image I don't see the firmware loading

[ 0.629648][ T12] imx-sdma 302b0000.dma-controller: Direct
firmware load for imx/sdma/sdma-imx7d.bin failed with error -2
[ 0.638320][ T12] imx-sdma 302b0000.dma-controller: Falling back
to sysfs fallback for: imx/sdma/sdma-imx7d.bin

Now if I boot the FIT image with the initramfs in the filename I get

[ 0.713335][ T12] imx-sdma 302b0000.dma-controller: loaded firmware 4.5

early in the boot so I know the initramfs works.

So you could say use the FIT image with the initramfs in its filename
but this isn't the FIT image that gets installed by
packagegroup-core-boot and I can't find out how to use the other FIT
image. I would prefer to bundle the initramfs into the kernel and not
rely on the FIT image mechanism.

I've tried building the kernel with all the INITRAMFS_* variables
commented out and the file sizes are nearly the same, not quite but
nearly. The initramfs is around 4MiB in size due to firmware files,
busybox and the libraries that this requires to perform the
switch_root. This also suggests that CONFIG_INITRAMFS_SOURCE isn't
working.

I've built the kernel with verbose on and see
+ use_alternate_initrd=CONFIG_INITRAMFS_SOURCE=/ws/rufilla/octopus/octave-dunfell/build-release/tmp/work/octave_imx8mnevk-poky-linux/linux-imx/5.4.24+gitAUTOINC+babac008e5-r0/build/usr/octave-cad-image-initramfs-octave-imx8mnevk.cpio


and also
+ oe_runmake [snip]
CONFIG_INITRAMFS_SOURCE=/ws/rufilla/octopus/octave-dunfell/build-release/tmp/work/octave_imx8mnevk-poky-linux/linux-imx/5.4.24+gitAUTOINC+babac008e5-r0/build/usr/octave-cad-image-initramfs-octave-imx8mnevk.cpio

So it looks like this doesn't work for some reason. I am using NXP's
vendor specific kernel but I don't think they've touched the kernel's
build system. The kernel version is 5.4. The log does contain what
looks like the Makefile actually performing the steps required

GEN Makefile

scripts/kconfig/conf --syncconfig Kconfig

GEN Makefile

CALL /ws/rufilla/octopus/octave-dunfell/build-release/tmp/work-shared/octave-imx8mnevk/kernel-source/scripts/checksyscalls.sh

CALL /ws/rufilla/octopus/octave-dunfell/build-release/tmp/work-shared/octave-imx8mnevk/kernel-source/scripts/atomic/check-atomics.sh

CHK include/generated/compile.h

GEN usr/initramfs_data.cpio

AS usr/initramfs_data.o

AR usr/built-in.a

GEN .version

CHK include/generated/compile.h

LD vmlinux.o

MODPOST vmlinux.o

MODINFO modules.builtin.modinfo

LD .tmp_vmlinux1

KSYM .tmp_kallsyms1.o

LD .tmp_vmlinux2

KSYM .tmp_kallsyms2.o

LD vmlinux

SORTEX vmlinux

SYSMAP System.map

OBJCOPY arch/arm64/boot/Image

${B}/usr seems ok

drwxr-xr-x 2 martin martin 4096 Jan 27 17:19 .
drwxr-xr-x 20 martin martin 4096 Jan 27 17:19 ..
-rw-rw-r-- 1 martin martin 146 Jan 27 17:19 built-in.a
-rw-r--r-- 1 martin martin 109 Jan 27 17:19 .built-in.a.cmd
-rwxr-xr-x 1 martin martin 31320 Jan 27 17:14 gen_init_cpio
-rw-r--r-- 1 martin martin 6642 Jan 27 17:14 .gen_init_cpio.cmd
-rw-r--r-- 1 martin martin 512 Jan 27 17:14 initramfs_data.cpio
-rw-r--r-- 1 martin martin 188 Jan 27 17:14 .initramfs_data.cpio.cmd
-rw-r--r-- 1 martin martin 142 Jan 27 17:15 .initramfs_data.cpio.d
-rw-rw-r-- 1 martin martin 1504249 Jan 27 17:19 initramfs_data.cpio.gz
-rw-rw-r-- 1 martin martin 389 Jan 27 17:19 .initramfs_data.cpio.gz.cmd
-rw-rw-r-- 1 martin martin 329 Jan 27 17:19 .initramfs_data.cpio.gz.d
-rw-rw-r-- 1 martin martin 1505240 Jan 27 17:19 initramfs_data.o
-rw-r--r-- 1 martin martin 4631 Jan 27 17:19 .initramfs_data.o.cmd
-rw-r--r-- 1 martin martin 2939392 Jan 27 17:15
octave-cad-image-initramfs-octave-imx8mnevk.cpio

I'm wondering if there is maybe a race condition somewhere in the
build. I say this because after the initial build I see the following

ls -la arch/arm64/boot/
total 50684
drwxr-xr-x 3 martin martin 4096 Jan 27 18:01 .
drwxr-xr-x 9 martin martin 4096 Jan 27 18:00 ..
drwxr-xr-x 3 martin martin 4096 Jan 27 18:00 dts
-rwx------ 1 martin martin 7218640 Jan 27 18:00 fitImage
-rwx------ 1 martin martin 10239756 Jan 27 18:01
fitImage-octave-cad-image-initramfs
-rw-r--r-- 1 martin martin 15876608 Jan 27 18:00 Image
-rw-r--r-- 1 martin martin 143 Jan 27 18:01 .Image.cmd
-rw-rw-r-- 1 martin martin 18825728 Jan 27 18:01 Image.initramfs
lrwxrwxrwx 1 martin martin 16 Jan 27 18:00 vmlinux -> ../../../vmlinux


Now if I manually from do_assemble_fitimage again I get
./temp/run.do_assemble_fitimage
gzip
fit-image.its:8.26-20.19: Warning (unit_address_vs_reg):
/images/kernel@1: node has a unit name, but no reg property
fit-image.its:17.32-19.27: Warning (unit_address_vs_reg):
/images/kernel@1/hash@1: node has a unit name, but no reg property
fit-image.its:21.60-31.19: Warning (unit_address_vs_reg):
/images/fdt@freescale_octopus,octave-imx8mnevk.dtb: node has a unit
name, but no reg property
fit-image.its:28.32-30.27: Warning (unit_address_vs_reg):
/images/fdt@freescale_octopus,octave-imx8mnevk.dtb/hash@1: node has a
unit name, but no reg property
fit-image.its:36.61-45.19: Warning (unit_address_vs_reg):
/configurations/conf@freescale_octopus,octave-imx8mnevk.dtb: node has
a unit name, but no reg property
fit-image.its:42.32-44.27: Warning (unit_address_vs_reg):
/configurations/conf@freescale_octopus,octave-imx8mnevk.dtb/hash@1:
node has a unit name, but no reg property
FIT description: U-Boot fitImage for Octopus Energy Octave Linux
OS/5.4.24+gitAUTOINC+babac008e5/octave-imx8mnevk
Created: Sat May 30 07:25:46 2020
Image 0 (kernel@1)
Description: Linux kernel
Created: Sat May 30 07:25:46 2020
Type: Kernel Image
Compression: gzip compressed
Data Size: 8692100 Bytes = 8488.38 KiB = 8.29 MiB
Architecture: AArch64
OS: Linux
Load Address: 0x40480000
Entry Point: 0x40480000
Hash algo: sha256
Hash value: 3c2213b560910ff59f04307e6531ad6bd55c21e17d559c3a5d4f1927e23d4c1a
Image 1 (fdt@freescale_octopus,octave-imx8mnevk.dtb)
Description: Flattened Device Tree blob
Created: Sat May 30 07:25:46 2020
Type: Flat Device Tree
Compression: uncompressed
Data Size: 35902 Bytes = 35.06 KiB = 0.03 MiB
Architecture: AArch64
Hash algo: sha256
Hash value: 63b81215384c3b243a538f8f738e3d9ea0c8496373cf47c2a138838f88bbe90c
Default Configuration: 'conf@freescale_octopus,octave-imx8mnevk.dtb'
Configuration 0 (conf@freescale_octopus,octave-imx8mnevk.dtb)
Description: 1 Linux kernel, FDT blob
Kernel: kernel@1
FDT: fdt@freescale_octopus,octave-imx8mnevk.dtb
Hash algo: sha256
Hash value: unavailable
martin@martin-X570-AORUS-ELITE:/ws/rufilla/octopus/octave-dunfell/build-release/tmp/work/octave_imx8mnevk-poky-linux/linux-imx/5.4.24+gitAUTOINC+babac008e5-r0$
ls -la build/arch/arm64/boot/total 52160
drwxr-xr-x 3 martin martin 4096 Jan 27 18:06 .
drwxr-xr-x 9 martin martin 4096 Jan 27 18:00 ..
drwxr-xr-x 3 martin martin 4096 Jan 27 18:00 dts
-rwx------ 1 martin martin 8730032 Jan 27 18:06 fitImage
-rwx------ 1 martin martin 10239756 Jan 27 18:01
fitImage-octave-cad-image-initramfs
-rw-r--r-- 1 martin martin 15876608 Jan 27 18:00 Image
-rw-r--r-- 1 martin martin 143 Jan 27 18:01 .Image.cmd
-rw-rw-r-- 1 martin martin 18825728 Jan 27 18:01 Image.initramfs
lrwxrwxrwx 1 martin martin 16 Jan 27 18:00 vmlinux -> ../../../vmlinux

Now the FIT image looks like it contains the initramfs. I tried
booting it and the firmware files were loaded. Looking at the
original log for do_assemble_fitimage I see

FIT description: U-Boot fitImage for Octopus Energy Octave Linux
OS/5.4.24+gitAUTOINC+babac008e5/octave-imx8mnevk
Created: Sat May 30 07:25:46 2020
Image 0 (kernel@1)
Description: Linux kernel
Created: Sat May 30 07:25:46 2020
Type: Kernel Image
Compression: gzip compressed
Data Size: 7180707 Bytes = 7012.41 KiB = 6.85 MiB
Architecture: AArch64
OS: Linux
Load Address: 0x40480000
Entry Point: 0x40480000
Hash algo: sha256

Note that the kernel image size is 7180707 originally whereas the
second time I run do_assemble_fitimage it's 8692100. It looks like
linux.bin the first time doesn't include the initramfs but on the
second time it does. Note my PC is a fairly rapid AMD Ryzen 9 5950X
16-Core (32 thread) Processor.

I'll keep digging but I would be grateful for any suggestions.

Many Thanks,
Martin.
I see the problem now, here is the task order from do_compile onwards.

do_compile

do_shared_workdir
do_kernel_link_images
do_assemble_fitimage

do_compile_kernelmodules

do_strip

do_sizecheck

do_install

do_package
do_populate_sysroot

do_packagedata

do_package_write_rpm
do_package_qa

do_bundle_initramfs
use_alternate_initrd=CONFIG_INITRAMFS_SOURCE=/ws/rufilla/octopus/octave-dunfell/build-release/tmp/work/octave_imx8mnevk-poky-linux/linux-imx/5.4.24+gitAUTOINC+babac008e5-r0/build/usr/octave-cad-image-initramfs-octave-imx8mnevk.cpio
kernel_do_compile

do_assemble_fitimage_initramfs

do_deploy


the problem is that do_assemble_fitimage occurs before
do_bundle_initramfs so the linux.bin file doesn't contain the
initramfs. I tried re-ordering the tasks but due to the dependencies
between u-boot for mkimge and the initramfs recipe I keep on getting
into a circular dependency hell.

I'll try to work out how to install the FIT image with initramfs into
/boot from kernel-image any help appreciated :) .

Ideally though the kernel should produce a linux.bin on the first
compile that gets used in the FIT Image with initramfs and then on the
second compile in do_bundle_initramfs another linux.bin is created
(with initramfs compiled in) that gets used with the normal FIT image.
Then both FIT image files deployed would have an initramfs just using
the different methods.

-Martin.
I managed to get a workaround by using a do_rootfs postfunc, here it
is in case anyone else finds it useful

do_rootfs[depends] += "virtual/kernel:do_deploy"
do_rootfs[postfuncs] += "rootfs_add_kernel"
rootfs_add_kernel() {
rm ${IMAGE_ROOTFS}/boot/fitImage*
install -m 0644
${DEPLOY_DIR_IMAGE}/fitImage-${INITRAMFS_IMAGE}-${MACHINE}-${MACHINE}
${IMAGE_ROOTFS}/boot
ln -snf fitImage-${INITRAMFS_IMAGE}-${MACHINE}-${MACHINE}
${IMAGE_ROOTFS}/boot/fitImage
}


Re: Task only for target recipe

Richard Purdie
 

On Thu, 2021-01-28 at 18:53 +0100, Ayoub Zaki via
lists.yoctoproject.org wrote:
Hello I created a new task that I want to run for every recipe of my
image but only for target recipes and skip all native, sdk,...
mytask.bbclass:

addtask do_mytask after do_install

do_mytask() {
    :
}

do_mytask_class-target() {

    bbwarn "mytask"
    :
}

I added to local.conf :

INHERIT += "mytask"

I don't see it running! what is the proper way to achieve this?
How are you running it? You've said you want it to run after do_install
but not what should trigger it.

Adding "before do_build" to the addtask line for example might trigger
it (do_build is the default task).

Cheers,

Richard


Task only for target recipe

Ayoub Zaki
 

Hello I created a new task that I want to run for every recipe of my image but only for target recipes and skip all native, sdk,...

mytask.bbclass:

addtask do_mytask after do_install

do_mytask() {
    :
}

do_mytask_class-target() {

    bbwarn "mytask"
    :
}

I added to local.conf :

INHERIT += "mytask"

I don't see it running! what is the proper way to achieve this?

cheers


git lfs not working in dunfell

Marek Belisko
 

Hi,

I have repo where I'm using lfs. I've added to my SRC_URI =
"git://...... ;lfs=1" and the project is fetched but the issue is that
lfs files are shown as references only (content is not fetched).
I briefly checked git fetcher in bitbake and seems it checks that repo
have lfs content but I cannot see anywhere git lfs pull to actually
pull lfs content. Am I missing something? When checked poky master it
looks like proper fetching is in place. Is there any way how to get it
to dunfell LTS release? Thanks.

BR,

marek


[meta-selinux][PATCH] policycoreutils: Improve reproducibility

Oleksiy Obitotskyy
 

LOCALEDIR should be set to target path,
e.g. /usr/share/locale not host absolute path.
This prevent to build reproducible package.

LOCALEDIR constructed from:
$(DESTDIR)$(PREFIX)/share/locale

Change PREFIX from ${D} to ${prefix}.
DESTDIR is not set during compilation and
is set to proper value during install.

Signed-off-by: Oleksiy Obitotskyy <oobitots@...>
---
recipes-security/selinux/policycoreutils.inc | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/recipes-security/selinux/policycoreutils.inc b/recipes-security/selinux/policycoreutils.inc
index 92f7a75..43a641d 100644
--- a/recipes-security/selinux/policycoreutils.inc
+++ b/recipes-security/selinux/policycoreutils.inc
@@ -118,7 +118,7 @@ EXTRA_OEMAKE += "\
${@bb.utils.contains('PACKAGECONFIG', 'libpam', 'PAMH=y', 'PAMH=', d)} \
${@bb.utils.contains('PACKAGECONFIG', 'audit', 'AUDITH=y', 'AUDITH=', d)} \
INOTIFYH=n \
- PREFIX=${D} \
+ PREFIX=${prefix} \
SBINDIR=${base_sbindir} \
"

--
2.26.2.Cisco


Re: any interest in an official "meta-rubygems" layer?

Robert P. J. Day
 

On Wed, 27 Jan 2021, akuster wrote:

On 1/27/21 1:29 PM, Robert P. J. Day wrote:
On Wed, 27 Jan 2021, Armin Kuster wrote:


On 1/27/21 12:04 PM, Robert P. J. Day wrote:
regarding the proposed "meta-rubygems" layer --- which sounds like
it's going to take off as it can be initialized quite a lot just from
konrad's existing meta-sca layer, what are the options for "official"
YP hosting?

i notice that quite a number of layers live at git.yoctoproject.org
-- would this proposed layer even be eligible for that and, if so,
what are the benefits of living under the git.YP.org umbrella?
Are you a member of the Yocto Project?
i personally am not, so i suspect github is the option.
Regardless of its home, there seems to be interest in such a layer.
There is nothing stopping you from registering in the layer in the
layer index and see where it goes from there.

Ruby was part of meta-openembedded. Maybe OE would want to host such
a layer or include it within meta-openembedded.
interesting idea, i hadn't considered that. in the meantime, i think
we'll ponder what the structure of the new layer will look like, and
put together a first pass.

rday


Re: any interest in an official "meta-rubygems" layer?

VIVAVIS AG
 

since it appears that i will be diving head-first into messing with ruby in some
current YP builds, is there any interest in creating a meta-rubygems layer to start
collecting recipes based on what konrad weihmann has done in his meta-sca
layer here?
We would also find an official layer for gems interesting.
We are currently using the old ruby.bbclass from the meta-ruby layer
from openemebedded to build the gems.
We are also particularly interested in gems that need cross compiling, e.g. ruby-pcap.

srp


Re: [meta-zephyr] bitbake zephyr-helloworld configure failure

Tim Orling
 



On Wed, Jan 27, 2021 at 12:51 AM Peter Smith <salerio@...> wrote:
Using master branch

MACHINE=96b_nitrogen bitbake zephyr-helloworld creates a configure error due to a failure for native python to import ruamel.

I fixed this temporarily by creating a python3-ruamel-yaml_%.bbappend that includes the required BBEXTEND and adding python3-ruamel-yaml-native to zephyr-kernel-common.inc.

I don't know (not enough experience) if this is actually a problem in the meta-openembedded recipe or meta-zephyr?


BBEXTEND = “native” is a perfectly fine patch to submit to meta-python. We tend to only make those changes when needed (as in your use case) rather than universally. Please submit a patch :)

Best Regards
Peter




Re: Including binary application as part of the image output

Richard Purdie
 

On Wed, 2021-01-27 at 10:32 -0800, chuck kamas via
lists.yoctoproject.org wrote:

As part of our image we build our company's application. This
application becomes part of the image and is executed when the image boots.

My question is how to have the binary image also be available along with
the image's wic file? Is there a line I can add to a recipe to have it
copy that image file to the tmp/deploy directory or other appropriate
directory so that it is available?
You could add a "do_deploy" task that places it there. See
meta/classes/deploy.bbclass and other recipes which add such a task.

Cheers,

Richard