Date   

Re: Error building Xen device tree for qemuarm64

Bruce Ashfield
 




On Tue, Oct 26, 2021 at 9:27 AM Edward Pickup <Edward.Pickup@...> wrote:

Hi Bruce,
Sorry for the formatting issues, email client config issues

 

From: Bruce Ashfield <bruce.ashfield@...>
Sent: 26 October 2021 14:22
To: Edward Pickup <Edward.Pickup@...>
Cc: meta-virtualization@...; Bertrand Marquis <Bertrand.Marquis@...>; christopher.w.clark@...; nd <nd@...>
Subject: Re: [meta-virtualization] Error building Xen device tree for qemuarm64

 

 

 

On Tue, Oct 26, 2021 at 8:58 AM Edward Pickup <edward.pickup@...> wrote:

Hi all,

I have encountered an error while building the arm64 version of the xen 4.16 recipe in meta-virtualisation, using the master version of poky and meta-virtuaisation.
I was wondering if anyone was familiar with the issue or knew how to fix it. The error is as follows:

ERROR: xen-image-minimal-1.0-r0 do_write_xen_qemuboot_dtb: ExecutionError('/home/edwpic01/work/master-update-xen/build/metavirt-host-arm64-4.16-release.prj/tmp/work/qemuarm64-poky-linux/xen-image-minimal/1.0-r0/temp/run.do_write_xen_qemuboot_dtb.3679135', 2, None, None)
ERROR: Logfile of failure stored in: /home/edwpic01/work/master-update-xen/build/metavirt-host-arm64-4.16-release.prj/tmp/work/qemuarm64-poky-linux/xen-image-minimal/1.0-r0/temp/log.do_write_xen_qemuboot_dtb.3679135 .

I believe the relevant part of the log is as follows:

| [NOTE]: '3' lopper operation files will be processed
| [INFO]: dtb output format detected, writing /home/edwpic01/work/master-update-xen/build/metavirt-host-arm64-4.16-release.prj/tmp/work/qemuarm64-poky-linux/xen-image-minimal/1.0-r0/deploy-xen-image-minimal-image-complete/xen-image-minimal-qemuarm64-20211026121502.qemuboot.dtb
| [INFO]: writing output dtb: /home/edwpic01/work/master-update-xen/build/metavirt-host-arm64-4.16-release.prj/tmp/work/qemuarm64-poky-linux/xen-image-minimal/1.0-r0/deploy-xen-image-minimal-image-complete/xen-image-minimal-qemuarm64-20211026121502.qemuboot.dtb
| /home/edwpic01/work/master-update-xen/build/metavirt-host-arm64-4.16-release.prj/tmp/work/qemuarm64-poky-linux/xen-image-minimal/1.0-r0/xen-image-minimal-1.0/output.dts: ERROR (explicit_phandles): /apb-pclk:phandle: bad length (1) phandle property
| /home/edwpic01/work/master-update-xen/build/metavirt-host-arm64-4.16-release.prj/tmp/work/qemuarm64-poky-linux/xen-image-minimal/1.0-r0/xen-image-minimal-1.0/output.dts: ERROR (explicit_phandles): /intc@8000000:phandle: bad length (1) phandle property
| /home/edwpic01/work/master-update-xen/build/metavirt-host-arm64-4.16-release.prj/tmp/work/qemuarm64-poky-linux/xen-image-minimal/1.0-r0/xen-image-minimal-1.0/output.dts: ERROR (explicit_phandles): /pl061@9030000:phandle: bad length (1) phandle property
| ERROR: Input tree has errors, aborting (use -f to force output)
| WARNING: exit code 2 from a shell command.

The above error was the result from building on x86_64-linux, target system was aarch64-poky-linux, running on a qemuarch64 machine, itself running on x86_64 linux.
As mentioned this was done on the master version of meta-virtualisation.
The bitbake command to generate this was bitbake xen-image-mininal.
The error is also present when building the xen 4.14 and xen 4.15 arm64 versions, and is not present on the arm32 or x86 builds for xen.
If anyone knows the cause of the error, the information would be appreciated.

 

There's something wrong with the input DTS to Lopper .. luckily, I maintain Lopper, so if there's an actual bug in the processing of the input DTS .. I can fix that.

 

I can fire up a build and see if it happens here.

 

So this is just qemuarm64, with xen + virtualization in the distro features ? Any other layers ? Anything else ?

 

Bruce

Yes, this is just as you say, no other layers or anything else. Thank you for the speedy reply!


There's a really bad interaction between libfdt and python 3.10. 

I've traced the error down to libfdt not being able to read anything from the dtbs and hence the chained errors.

I'm building some older branches to confirm behaviours .. it will take me a few days to sort through this, but I don't want to mask the issue, as I need to understand exactly what is going on here.

Bruce

 

 

Edward

 


Cheers!


IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.


 

--

- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.


--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II


Re: [PATCH] xen,xen-tools: update xen to new releases

Bertrand Marquis
 

Hi Bruce,

On 26 Oct 2021, at 15:06, Bruce Ashfield <bruce.ashfield@...> wrote:



On Tue, Oct 26, 2021 at 9:32 AM Edward Pickup <edward.pickup@...> wrote:
Update the SHA in recipes for xen and xen-tools to use the latest releases;
update xen 4.14 to xen 4.14.3 release, update xen 4.15 to 4.15.1 release, and
xen 4.16 to the current master version.

Add a comment to each recipe indicating which version of Xen is used in recipe.

Do not apply patches that have been merged/solved in Xen and remove patch files
not needed anymore.

Create a test package to pack new tests introduced in 4.16.

Add a depencency to util-linux-prlimit, as the xen 4.16 x86 requires prlimit,
without this boot of dom0 fails.


Given your other email, I assume that the 4.16 bump isn't building for qemuarm64 ? Otherwise, they are all building for the supported targets ?
This is building for all qemu target (arm32, 64 and x86_64). For arm64 I checked that by commenting out the device tree modification in the class.

This is properly booting on qemu x86 and on other arm targets (using meta-arm layer for the BSPs).

I did review and compile test this so you can add my:

Reviewed-by: Bertrand Marquis <bertrand.marquis@...>

Cheers
Bertrand


Bruce


Signed-off-by: Edward Pickup <Edward.Pickup@...>
---
...ry-Fix-osdep_xenforeignmemory_map-pr.patch | 33 ------
...x86-make-hypervisor-build-with-gcc11.patch | 111 ------------------
...-around-build-issue-with-GNU-ld-2.37.patch | 36 ------
recipes-extended/xen/xen-tools.inc | 29 +++++
recipes-extended/xen/xen-tools_4.14.bb | 5 +-
recipes-extended/xen/xen-tools_4.15.bb | 5 +-
recipes-extended/xen/xen-tools_git.bb | 5 +-
recipes-extended/xen/xen_4.14.bb | 4 +-
recipes-extended/xen/xen_4.15.bb | 5 +-
recipes-extended/xen/xen_git.bb | 5 +-
10 files changed, 41 insertions(+), 197 deletions(-)
delete mode 100644 recipes-extended/xen/files/0001-libs-foreignmemory-Fix-osdep_xenforeignmemory_map-pr.patch
delete mode 100644 recipes-extended/xen/files/0001-x86-make-hypervisor-build-with-gcc11.patch
delete mode 100644 recipes-extended/xen/files/0001-x86-work-around-build-issue-with-GNU-ld-2.37.patch

diff --git a/recipes-extended/xen/files/0001-libs-foreignmemory-Fix-osdep_xenforeignmemory_map-pr.patch b/recipes-extended/xen/files/0001-libs-foreignmemory-Fix-osdep_xenforeignmemory_map-pr.patch
deleted file mode 100644
index 2e3b1a6..0000000
--- a/recipes-extended/xen/files/0001-libs-foreignmemory-Fix-osdep_xenforeignmemory_map-pr.patch
+++ /dev/null
@@ -1,33 +0,0 @@
-From 5d3e4ebb5c71477d74a0c503438545a0126d3863 Mon Sep 17 00:00:00 2001
-From: Anthony PERARD <anthony.perard@...>
-Date: Tue, 1 Jun 2021 16:41:47 +0100
-Subject: [PATCH] libs/foreignmemory: Fix osdep_xenforeignmemory_map prototype
-
-Commit cf8c4d3d13b8 made some preparation to have one day
-variable-length-array argument, but didn't declare the array in the
-function prototype the same way as in the function definition. And now
-GCC 11 complains about it.
-
-Fixes: cf8c4d3d13b8 ("tools/libs/foreignmemory: pull array length argument to map forward")
-Signed-off-by: Anthony PERARD <anthony.perard@...>
-Reviewed-by: Jan Beulich <jbeulich@...>
----
- tools/libs/foreignmemory/private.h | 2 +-
- 1 file changed, 1 insertion(+), 1 deletion(-)
-
-diff --git a/tools/libs/foreignmemory/private.h b/tools/libs/foreignmemory/private.h
-index 1ee3626dd2..5bb0cefb09 100644
---- a/tools/libs/foreignmemory/private.h
-+++ b/tools/libs/foreignmemory/private.h
-@@ -32,7 +32,7 @@ int osdep_xenforeignmemory_close(xenforeignmemory_handle *fmem);
- void *osdep_xenforeignmemory_map(xenforeignmemory_handle *fmem,
- uint32_t dom, void *addr,
- int prot, int flags, size_t num,
-- const xen_pfn_t arr[num], int err[num]);
-+ const xen_pfn_t arr[/*num*/], int err[/*num*/]);
- int osdep_xenforeignmemory_unmap(xenforeignmemory_handle *fmem,
- void *addr, size_t num);
-
---
-2.25.1
-
diff --git a/recipes-extended/xen/files/0001-x86-make-hypervisor-build-with-gcc11.patch b/recipes-extended/xen/files/0001-x86-make-hypervisor-build-with-gcc11.patch
deleted file mode 100644
index 72592c4..0000000
--- a/recipes-extended/xen/files/0001-x86-make-hypervisor-build-with-gcc11.patch
+++ /dev/null
@@ -1,111 +0,0 @@
-From 722f59d38c710a940ab05e542a83020eb5546dea Mon Sep 17 00:00:00 2001
-From: Jan Beulich <jbeulich@...>
-Date: Thu, 27 May 2021 14:40:29 +0200
-Subject: [PATCH] x86: make hypervisor build with gcc11
-MIME-Version: 1.0
-Content-Type: text/plain; charset=UTF-8
-Content-Transfer-Encoding: 8bit
-
-Gcc 11 looks to make incorrect assumptions about valid ranges that
-pointers may be used for addressing when they are derived from e.g. a
-plain constant. See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100680.
-
-Utilize RELOC_HIDE() to work around the issue, which for x86 manifests
-in at least
-- mpparse.c:efi_check_config(),
-- tboot.c:tboot_probe(),
-- tboot.c:tboot_gen_frametable_integrity(),
-- x86_emulate.c:x86_emulate() (at -O2 only).
-The last case is particularly odd not just because it only triggers at
-higher optimization levels, but also because it only affects one of at
-least three similar constructs. Various "note" diagnostics claim the
-valid index range to be [0, 2⁶³-1].
-
-Signed-off-by: Jan Beulich <jbeulich@...>
-Tested-by: Jason Andryuk <jandryuk@...>
-Acked-by: Roger Pau Monné <roger.pau@...>
----
- tools/tests/x86_emulator/x86-emulate.c | 7 +++++++
- xen/arch/x86/x86_emulate/x86_emulate.c | 2 +-
- xen/include/asm-x86/fixmap.h | 2 +-
- xen/include/xen/compiler.h | 6 ++++++
- xen/include/xen/pdx.h | 2 +-
- 5 files changed, 16 insertions(+), 3 deletions(-)
-
-diff --git a/tools/tests/x86_emulator/x86-emulate.c b/tools/tests/x86_emulator/x86-emulate.c
-index 07f892dbbb..ea286d6ad8 100644
---- a/tools/tests/x86_emulator/x86-emulate.c
-+++ b/tools/tests/x86_emulator/x86-emulate.c
-@@ -8,6 +8,13 @@
-
- #define ERR_PTR(val) NULL
-
-+/* See gcc bug 100680, but here don't bother making this version dependent. */
-+#define gcc11_wrap(x) ({ \
-+ unsigned long x_; \
-+ __asm__ ( "" : "=g" (x_) : "0" (x) ); \
-+ (typeof(x))x_; \
-+})
-+
- #define cpu_has_amd_erratum(nr) 0
- #define cpu_has_mpx false
- #define read_bndcfgu() 0
-diff --git a/xen/arch/x86/x86_emulate/x86_emulate.c b/xen/arch/x86/x86_emulate/x86_emulate.c
-index c25d88d0d8..31fdec030c 100644
---- a/xen/arch/x86/x86_emulate/x86_emulate.c
-+++ b/xen/arch/x86/x86_emulate/x86_emulate.c
-@@ -726,7 +726,7 @@ union vex {
- #define copy_VEX(ptr, vex) ({ \
- if ( !mode_64bit() ) \
- (vex).reg |= 8; \
-- (ptr)[0 - PFX_BYTES] = ext < ext_8f08 ? 0xc4 : 0x8f; \
-+ gcc11_wrap(ptr)[0 - PFX_BYTES] = ext < ext_8f08 ? 0xc4 : 0x8f; \
- (ptr)[1 - PFX_BYTES] = (vex).raw[0]; \
- (ptr)[2 - PFX_BYTES] = (vex).raw[1]; \
- container_of((ptr) + 1 - PFX_BYTES, typeof(vex), raw[0]); \
-diff --git a/xen/include/asm-x86/fixmap.h b/xen/include/asm-x86/fixmap.h
-index 0db314baeb..20746afd0a 100644
---- a/xen/include/asm-x86/fixmap.h
-+++ b/xen/include/asm-x86/fixmap.h
-@@ -78,7 +78,7 @@ extern void __set_fixmap(
-
- #define clear_fixmap(idx) __set_fixmap(idx, 0, 0)
-
--#define __fix_to_virt(x) (FIXADDR_TOP - ((x) << PAGE_SHIFT))
-+#define __fix_to_virt(x) gcc11_wrap(FIXADDR_TOP - ((x) << PAGE_SHIFT))
- #define __virt_to_fix(x) ((FIXADDR_TOP - ((x)&PAGE_MASK)) >> PAGE_SHIFT)
-
- #define fix_to_virt(x) ((void *)__fix_to_virt(x))
-diff --git a/xen/include/xen/compiler.h b/xen/include/xen/compiler.h
-index 17cf00e1ec..696c7eb89e 100644
---- a/xen/include/xen/compiler.h
-+++ b/xen/include/xen/compiler.h
-@@ -140,6 +140,12 @@
- __asm__ ("" : "=r"(__ptr) : "0"(ptr)); \
- (typeof(ptr)) (__ptr + (off)); })
-
-+#if CONFIG_GCC_VERSION >= 110000 /* See gcc bug 100680. */
-+# define gcc11_wrap(x) RELOC_HIDE(x, 0)
-+#else
-+# define gcc11_wrap(x) (x)
-+#endif
-+
- #ifdef __GCC_ASM_FLAG_OUTPUTS__
- # define ASM_FLAG_OUT(yes, no) yes
- #else
-diff --git a/xen/include/xen/pdx.h b/xen/include/xen/pdx.h
-index 770fadc06c..9fcfb0ce52 100644
---- a/xen/include/xen/pdx.h
-+++ b/xen/include/xen/pdx.h
-@@ -19,7 +19,7 @@ extern u64 pdx_region_mask(u64 base, u64 len);
- extern void set_pdx_range(unsigned long smfn, unsigned long emfn);
-
- #define page_to_pdx(pg) ((pg) - frame_table)
--#define pdx_to_page(pdx) (frame_table + (pdx))
-+#define pdx_to_page(pdx) gcc11_wrap(frame_table + (pdx))
-
- bool __mfn_valid(unsigned long mfn);
-
---
-2.25.1
-
diff --git a/recipes-extended/xen/files/0001-x86-work-around-build-issue-with-GNU-ld-2.37.patch b/recipes-extended/xen/files/0001-x86-work-around-build-issue-with-GNU-ld-2.37.patch
deleted file mode 100644
index f5108be..0000000
--- a/recipes-extended/xen/files/0001-x86-work-around-build-issue-with-GNU-ld-2.37.patch
+++ /dev/null
@@ -1,36 +0,0 @@
-From 58ad654ebce7ccb272a3f4f3482c03aaad850d31 Mon Sep 17 00:00:00 2001
-From: Jan Beulich <jbeulich@...>
-Date: Thu, 22 Jul 2021 11:20:38 +0200
-Subject: [PATCH] x86: work around build issue with GNU ld 2.37
-
-I suspect it is commit 40726f16a8d7 ("ld script expression parsing")
-which broke the hypervisor build, by no longer accepting section names
-with a dash in them inside ADDR() (and perhaps other script directives
-expecting just a section name, not an expression): .note.gnu.build-id
-is such a section.
-
-Quoting all section names passed to ADDR() via DECL_SECTION() works
-around the regression.
-
-Signed-off-by: Jan Beulich <jbeulich@...>
-Acked-by: Andrew Cooper <andrew.cooper3@...>
----
- xen/arch/x86/xen.lds.S | 2 +-
- 1 file changed, 1 insertion(+), 1 deletion(-)
-
-diff --git a/xen/arch/x86/xen.lds.S b/xen/arch/x86/xen.lds.S
-index 9c6c1c8005..955d5cf4a0 100644
---- a/xen/arch/x86/xen.lds.S
-+++ b/xen/arch/x86/xen.lds.S
-@@ -18,7 +18,7 @@ ENTRY(efi_start)
- #else /* !EFI */
-
- #define FORMAT "elf64-x86-64"
--#define DECL_SECTION(x) x : AT(ADDR(x) - __XEN_VIRT_START)
-+#define DECL_SECTION(x) x : AT(ADDR(#x) - __XEN_VIRT_START)
-
- ENTRY(start_pa)
-
---
-2.25.1
-
diff --git a/recipes-extended/xen/xen-tools.inc b/recipes-extended/xen/xen-tools.inc
index 614f557..d831941 100644
--- a/recipes-extended/xen/xen-tools.inc
+++ b/recipes-extended/xen/xen-tools.inc
@@ -102,6 +102,7 @@ RSUGGESTS:${PN}-xencov = "${PN}-xencov-split"

RDEPENDS:${PN}-xencommons = "\
bash \
+ util-linux-prlimit \
${PN}-console \
${PN}-xenstore \
virtual/xenstored \
@@ -213,6 +214,9 @@ PACKAGES = " \
${PN}-xm-examples \
${PN}-xen-access \
${PN}-xen-memshare \
+ ${PN}-test \
+ ${PN}-xen-vmtrace \
+ ${PN}-xen-mceinj \
"

PROVIDES =+ " \
@@ -684,6 +688,31 @@ FILES:${PN}-xen-memshare += "\
# memshare is only built for x86, so allow empty package for other archs
ALLOW_EMPTY:${PN}-xen-memshare = "1"

+FILES:${PN}-test += "\
+ ${libdir}/xen/bin/test-xenstore \
+ ${libdir}/xen/bin/test-resource \
+ ${libdir}/xen/bin/test-cpu-policy \
+ ${libdir}/xen/bin/test-tsx \
+ "
+
+# test-xenstore and test-resource currently only exist in 4.16
+# test-cpu-policy and test-tsx only exist in 4.16 for x86
+ALLOW_EMPTY:${PN}-test = "1"
+
+FILES:${PN}-xen-mceinj +="\
+ ${sbindir}/xen-mceinj \
+ "
+
+# xen-mceinj is only built for x86 4.16, so allow empty package
+ALLOW_EMPTY:${PN}-xen-mceinj = "1"
+
+FILES:${PN}-xen-vmtrace +="\
+ ${sbindir}/xen-vmtrace \
+ "
+
+# xen-vmtrace is only built for x86 4.16, so allow empty package
+ALLOW_EMPTY:${PN}-xen-vmtrace = "1"
+
INSANE_SKIP:${PN} = "already-stripped"

# configure init.d scripts
diff --git a/recipes-extended/xen/xen-tools_4.14.bb b/recipes-extended/xen/xen-tools_4.14.bb
index ce5dc2e..9d78e44 100644
--- a/recipes-extended/xen/xen-tools_4.14.bb
+++ b/recipes-extended/xen/xen-tools_4.14.bb
@@ -1,4 +1,5 @@
-SRCREV ?= "b6a8c4f72def4d1135ff42660a86276ce2565c8c"
+# 4.14.3 release SHA
+SRCREV ?= "9f2b6c5ec2ded4c1caf149743e862c5f15d6d083"

XEN_REL ?= "4.14"
XEN_BRANCH ?= "stable-${XEN_REL}"
@@ -8,8 +9,6 @@ SRC_URI = " \
file://0001-python-pygrub-pass-DISTUTILS-xen-4.14.patch \
file://0001-firmware-provide-a-stand-alone-set-of-headers-Xen-4.14.patch \
file://0001-tools-firmware-Build-firmware-as-ffreestanding-Xen-4.14.patch \
- file://0001-libs-foreignmemory-Fix-osdep_xenforeignmemory_map-pr.patch \
- file://0001-x86-work-around-build-issue-with-GNU-ld-2.37.patch \
"

LIC_FILES_CHKSUM ?= "file://COPYING;md5=419739e325a50f3d7b4501338e44a4e5"
diff --git a/recipes-extended/xen/xen-tools_4.15.bb b/recipes-extended/xen/xen-tools_4.15.bb
index e9c3bd6..840f162 100644
--- a/recipes-extended/xen/xen-tools_4.15.bb
+++ b/recipes-extended/xen/xen-tools_4.15.bb
@@ -1,4 +1,5 @@
-SRCREV ?= "e25aa9939ae0cd8317605be3d5c5611b76bc4ab4"
+# xen 4.15.1 release sha
+SRCREV ?= "84fa99099b920f7bcde8899e1a9b756078719d7d"

XEN_REL ?= "4.15"
XEN_BRANCH ?= "stable-${XEN_REL}"
@@ -6,8 +7,6 @@ XEN_BRANCH ?= "stable-${XEN_REL}"
SRC_URI = " \
git://xenbits.xen.org/xen.git;branch=${XEN_BRANCH} \
file://0001-python-pygrub-pass-DISTUTILS-xen-4.15.patch \
- file://0001-libs-foreignmemory-Fix-osdep_xenforeignmemory_map-pr.patch \
- file://0001-x86-work-around-build-issue-with-GNU-ld-2.37.patch \
"

LIC_FILES_CHKSUM ?= "file://COPYING;md5=419739e325a50f3d7b4501338e44a4e5"
diff --git a/recipes-extended/xen/xen-tools_git.bb b/recipes-extended/xen/xen-tools_git.bb
index 6926bd9..8ff9c4c 100644
--- a/recipes-extended/xen/xen-tools_git.bb
+++ b/recipes-extended/xen/xen-tools_git.bb
@@ -1,4 +1,5 @@
-SRCREV ?= "e680cc48b7184d3489873d6776f84ba1fc238ced"
+# master status on 2020-10-21
+SRCREV ?= "23ec1ebc8acbfd2bf06f6085a776f0db923f9fa9"

XEN_REL ?= "4.16"
XEN_BRANCH ?= "master"
@@ -6,8 +7,6 @@ XEN_BRANCH ?= "master"
SRC_URI = " \
git://xenbits.xen.org/xen.git;branch=${XEN_BRANCH} \
file://0001-python-pygrub-pass-DISTUTILS-xen-4.15.patch \
- file://0001-libs-foreignmemory-Fix-osdep_xenforeignmemory_map-pr.patch \
- file://0001-x86-work-around-build-issue-with-GNU-ld-2.37.patch \
"

LIC_FILES_CHKSUM ?= "file://COPYING;md5=419739e325a50f3d7b4501338e44a4e5"
diff --git a/recipes-extended/xen/xen_4.14.bb b/recipes-extended/xen/xen_4.14.bb
index 291ba16..0d7d20b 100644
--- a/recipes-extended/xen/xen_4.14.bb
+++ b/recipes-extended/xen/xen_4.14.bb
@@ -1,4 +1,5 @@
-SRCREV ?= "b6a8c4f72def4d1135ff42660a86276ce2565c8c"
+# 4.14.3 release SHA
+SRCREV ?= "9f2b6c5ec2ded4c1caf149743e862c5f15d6d083"

XEN_REL ?= "4.14"
XEN_BRANCH ?= "stable-${XEN_REL}"
@@ -7,7 +8,6 @@ SRC_URI = " \
git://xenbits.xen.org/xen.git;branch=${XEN_BRANCH} \
file://xen-arm64-implement-atomic-fetch-add.patch \
file://0001-menuconfig-mconf-cfg-Allow-specification-of-ncurses-location.patch \
- file://0001-x86-work-around-build-issue-with-GNU-ld-2.37.patch \
"

LIC_FILES_CHKSUM ?= "file://COPYING;md5=419739e325a50f3d7b4501338e44a4e5"
diff --git a/recipes-extended/xen/xen_4.15.bb b/recipes-extended/xen/xen_4.15.bb
index 0605441..0cfaf0c 100644
--- a/recipes-extended/xen/xen_4.15.bb
+++ b/recipes-extended/xen/xen_4.15.bb
@@ -1,4 +1,5 @@
-SRCREV ?= "e25aa9939ae0cd8317605be3d5c5611b76bc4ab4"
+# xen 4.15.1 release sha
+SRCREV ?= "84fa99099b920f7bcde8899e1a9b756078719d7d"

XEN_REL ?= "4.15"
XEN_BRANCH ?= "stable-${XEN_REL}"
@@ -7,8 +8,6 @@ SRC_URI = " \
git://xenbits.xen.org/xen.git;branch=${XEN_BRANCH} \
file://xen-arm64-implement-atomic-fetch-add.patch \
file://0001-menuconfig-mconf-cfg-Allow-specification-of-ncurses-location.patch \
- file://0001-x86-make-hypervisor-build-with-gcc11.patch \
- file://0001-x86-work-around-build-issue-with-GNU-ld-2.37.patch \
"

LIC_FILES_CHKSUM ?= "file://COPYING;md5=419739e325a50f3d7b4501338e44a4e5"
diff --git a/recipes-extended/xen/xen_git.bb b/recipes-extended/xen/xen_git.bb
index 12c9403..20227e0 100644
--- a/recipes-extended/xen/xen_git.bb
+++ b/recipes-extended/xen/xen_git.bb
@@ -1,4 +1,5 @@
-SRCREV ?= "e680cc48b7184d3489873d6776f84ba1fc238ced"
+# master status on 2020-10-21
+SRCREV ?= "23ec1ebc8acbfd2bf06f6085a776f0db923f9fa9"

XEN_REL ?= "4.16"
XEN_BRANCH ?= "master"
@@ -7,8 +8,6 @@ SRC_URI = " \
git://xenbits.xen.org/xen.git;branch=${XEN_BRANCH} \
file://xen-arm64-implement-atomic-fetch-add.patch \
file://0001-menuconfig-mconf-cfg-Allow-specification-of-ncurses-location.patch \
- file://0001-x86-make-hypervisor-build-with-gcc11.patch \
- file://0001-x86-work-around-build-issue-with-GNU-ld-2.37.patch \
"

LIC_FILES_CHKSUM ?= "file://COPYING;md5=419739e325a50f3d7b4501338e44a4e5"
--
2.25.1






--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.


Re: [PATCH] xen,xen-tools: update xen to new releases

Bruce Ashfield
 



On Tue, Oct 26, 2021 at 9:32 AM Edward Pickup <edward.pickup@...> wrote:
Update the SHA in recipes for xen and xen-tools to use the latest releases;
update xen 4.14 to xen 4.14.3 release, update xen 4.15 to 4.15.1 release, and
xen 4.16 to the current master version.

Add a comment to each recipe indicating which version of Xen is used in recipe.

Do not apply patches that have been merged/solved in Xen and remove patch files
not needed anymore.

Create a test package to pack new tests introduced in 4.16.

Add a depencency to util-linux-prlimit, as the xen 4.16 x86 requires prlimit,
without this boot of dom0 fails.


Given your other email, I assume that the 4.16 bump isn't building for qemuarm64 ? Otherwise, they are all building for the supported targets ?

Bruce
 

Signed-off-by: Edward Pickup <Edward.Pickup@...>
---
 ...ry-Fix-osdep_xenforeignmemory_map-pr.patch |  33 ------
 ...x86-make-hypervisor-build-with-gcc11.patch | 111 ------------------
 ...-around-build-issue-with-GNU-ld-2.37.patch |  36 ------
 recipes-extended/xen/xen-tools.inc            |  29 +++++
 recipes-extended/xen/xen-tools_4.14.bb        |   5 +-
 recipes-extended/xen/xen-tools_4.15.bb        |   5 +-
 recipes-extended/xen/xen-tools_git.bb         |   5 +-
 recipes-extended/xen/xen_4.14.bb              |   4 +-
 recipes-extended/xen/xen_4.15.bb              |   5 +-
 recipes-extended/xen/xen_git.bb               |   5 +-
 10 files changed, 41 insertions(+), 197 deletions(-)
 delete mode 100644 recipes-extended/xen/files/0001-libs-foreignmemory-Fix-osdep_xenforeignmemory_map-pr.patch
 delete mode 100644 recipes-extended/xen/files/0001-x86-make-hypervisor-build-with-gcc11.patch
 delete mode 100644 recipes-extended/xen/files/0001-x86-work-around-build-issue-with-GNU-ld-2.37.patch

diff --git a/recipes-extended/xen/files/0001-libs-foreignmemory-Fix-osdep_xenforeignmemory_map-pr.patch b/recipes-extended/xen/files/0001-libs-foreignmemory-Fix-osdep_xenforeignmemory_map-pr.patch
deleted file mode 100644
index 2e3b1a6..0000000
--- a/recipes-extended/xen/files/0001-libs-foreignmemory-Fix-osdep_xenforeignmemory_map-pr.patch
+++ /dev/null
@@ -1,33 +0,0 @@
-From 5d3e4ebb5c71477d74a0c503438545a0126d3863 Mon Sep 17 00:00:00 2001
-From: Anthony PERARD <anthony.perard@...>
-Date: Tue, 1 Jun 2021 16:41:47 +0100
-Subject: [PATCH] libs/foreignmemory: Fix osdep_xenforeignmemory_map prototype
-
-Commit cf8c4d3d13b8 made some preparation to have one day
-variable-length-array argument, but didn't declare the array in the
-function prototype the same way as in the function definition. And now
-GCC 11 complains about it.
-
-Fixes: cf8c4d3d13b8 ("tools/libs/foreignmemory: pull array length argument to map forward")
-Signed-off-by: Anthony PERARD <anthony.perard@...>
-Reviewed-by: Jan Beulich <jbeulich@...>
----
- tools/libs/foreignmemory/private.h | 2 +-
- 1 file changed, 1 insertion(+), 1 deletion(-)
-
-diff --git a/tools/libs/foreignmemory/private.h b/tools/libs/foreignmemory/private.h
-index 1ee3626dd2..5bb0cefb09 100644
---- a/tools/libs/foreignmemory/private.h
-+++ b/tools/libs/foreignmemory/private.h
-@@ -32,7 +32,7 @@ int osdep_xenforeignmemory_close(xenforeignmemory_handle *fmem);
- void *osdep_xenforeignmemory_map(xenforeignmemory_handle *fmem,
-                                  uint32_t dom, void *addr,
-                                  int prot, int flags, size_t num,
--                                 const xen_pfn_t arr[num], int err[num]);
-+                                 const xen_pfn_t arr[/*num*/], int err[/*num*/]);
- int osdep_xenforeignmemory_unmap(xenforeignmemory_handle *fmem,
-                                  void *addr, size_t num);
-
---
-2.25.1
-
diff --git a/recipes-extended/xen/files/0001-x86-make-hypervisor-build-with-gcc11.patch b/recipes-extended/xen/files/0001-x86-make-hypervisor-build-with-gcc11.patch
deleted file mode 100644
index 72592c4..0000000
--- a/recipes-extended/xen/files/0001-x86-make-hypervisor-build-with-gcc11.patch
+++ /dev/null
@@ -1,111 +0,0 @@
-From 722f59d38c710a940ab05e542a83020eb5546dea Mon Sep 17 00:00:00 2001
-From: Jan Beulich <jbeulich@...>
-Date: Thu, 27 May 2021 14:40:29 +0200
-Subject: [PATCH] x86: make hypervisor build with gcc11
-MIME-Version: 1.0
-Content-Type: text/plain; charset=UTF-8
-Content-Transfer-Encoding: 8bit
-
-Gcc 11 looks to make incorrect assumptions about valid ranges that
-pointers may be used for addressing when they are derived from e.g. a
-plain constant. See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100680.
-
-Utilize RELOC_HIDE() to work around the issue, which for x86 manifests
-in at least
-- mpparse.c:efi_check_config(),
-- tboot.c:tboot_probe(),
-- tboot.c:tboot_gen_frametable_integrity(),
-- x86_emulate.c:x86_emulate() (at -O2 only).
-The last case is particularly odd not just because it only triggers at
-higher optimization levels, but also because it only affects one of at
-least three similar constructs. Various "note" diagnostics claim the
-valid index range to be [0, 2⁶³-1].
-
-Signed-off-by: Jan Beulich <jbeulich@...>
-Tested-by: Jason Andryuk <jandryuk@...>
-Acked-by: Roger Pau Monné <roger.pau@...>
----
- tools/tests/x86_emulator/x86-emulate.c | 7 +++++++
- xen/arch/x86/x86_emulate/x86_emulate.c | 2 +-
- xen/include/asm-x86/fixmap.h           | 2 +-
- xen/include/xen/compiler.h             | 6 ++++++
- xen/include/xen/pdx.h                  | 2 +-
- 5 files changed, 16 insertions(+), 3 deletions(-)
-
-diff --git a/tools/tests/x86_emulator/x86-emulate.c b/tools/tests/x86_emulator/x86-emulate.c
-index 07f892dbbb..ea286d6ad8 100644
---- a/tools/tests/x86_emulator/x86-emulate.c
-+++ b/tools/tests/x86_emulator/x86-emulate.c
-@@ -8,6 +8,13 @@
-
- #define ERR_PTR(val) NULL
-
-+/* See gcc bug 100680, but here don't bother making this version dependent. */
-+#define gcc11_wrap(x) ({                  \
-+    unsigned long x_;                     \
-+    __asm__ ( "" : "=g" (x_) : "0" (x) ); \
-+    (typeof(x))x_;                        \
-+})
-+
- #define cpu_has_amd_erratum(nr) 0
- #define cpu_has_mpx false
- #define read_bndcfgu() 0
-diff --git a/xen/arch/x86/x86_emulate/x86_emulate.c b/xen/arch/x86/x86_emulate/x86_emulate.c
-index c25d88d0d8..31fdec030c 100644
---- a/xen/arch/x86/x86_emulate/x86_emulate.c
-+++ b/xen/arch/x86/x86_emulate/x86_emulate.c
-@@ -726,7 +726,7 @@ union vex {
- #define copy_VEX(ptr, vex) ({ \
-     if ( !mode_64bit() ) \
-         (vex).reg |= 8; \
--    (ptr)[0 - PFX_BYTES] = ext < ext_8f08 ? 0xc4 : 0x8f; \
-+    gcc11_wrap(ptr)[0 - PFX_BYTES] = ext < ext_8f08 ? 0xc4 : 0x8f; \
-     (ptr)[1 - PFX_BYTES] = (vex).raw[0]; \
-     (ptr)[2 - PFX_BYTES] = (vex).raw[1]; \
-     container_of((ptr) + 1 - PFX_BYTES, typeof(vex), raw[0]); \
-diff --git a/xen/include/asm-x86/fixmap.h b/xen/include/asm-x86/fixmap.h
-index 0db314baeb..20746afd0a 100644
---- a/xen/include/asm-x86/fixmap.h
-+++ b/xen/include/asm-x86/fixmap.h
-@@ -78,7 +78,7 @@ extern void __set_fixmap(
-
- #define clear_fixmap(idx) __set_fixmap(idx, 0, 0)
-
--#define __fix_to_virt(x) (FIXADDR_TOP - ((x) << PAGE_SHIFT))
-+#define __fix_to_virt(x) gcc11_wrap(FIXADDR_TOP - ((x) << PAGE_SHIFT))
- #define __virt_to_fix(x) ((FIXADDR_TOP - ((x)&PAGE_MASK)) >> PAGE_SHIFT)
-
- #define fix_to_virt(x)   ((void *)__fix_to_virt(x))
-diff --git a/xen/include/xen/compiler.h b/xen/include/xen/compiler.h
-index 17cf00e1ec..696c7eb89e 100644
---- a/xen/include/xen/compiler.h
-+++ b/xen/include/xen/compiler.h
-@@ -140,6 +140,12 @@
-     __asm__ ("" : "=r"(__ptr) : "0"(ptr));      \
-     (typeof(ptr)) (__ptr + (off)); })
-
-+#if CONFIG_GCC_VERSION >= 110000 /* See gcc bug 100680. */
-+# define gcc11_wrap(x) RELOC_HIDE(x, 0)
-+#else
-+# define gcc11_wrap(x) (x)
-+#endif
-+
- #ifdef __GCC_ASM_FLAG_OUTPUTS__
- # define ASM_FLAG_OUT(yes, no) yes
- #else
-diff --git a/xen/include/xen/pdx.h b/xen/include/xen/pdx.h
-index 770fadc06c..9fcfb0ce52 100644
---- a/xen/include/xen/pdx.h
-+++ b/xen/include/xen/pdx.h
-@@ -19,7 +19,7 @@ extern u64 pdx_region_mask(u64 base, u64 len);
- extern void set_pdx_range(unsigned long smfn, unsigned long emfn);
-
- #define page_to_pdx(pg)  ((pg) - frame_table)
--#define pdx_to_page(pdx) (frame_table + (pdx))
-+#define pdx_to_page(pdx) gcc11_wrap(frame_table + (pdx))
-
- bool __mfn_valid(unsigned long mfn);
-
---
-2.25.1
-
diff --git a/recipes-extended/xen/files/0001-x86-work-around-build-issue-with-GNU-ld-2.37.patch b/recipes-extended/xen/files/0001-x86-work-around-build-issue-with-GNU-ld-2.37.patch
deleted file mode 100644
index f5108be..0000000
--- a/recipes-extended/xen/files/0001-x86-work-around-build-issue-with-GNU-ld-2.37.patch
+++ /dev/null
@@ -1,36 +0,0 @@
-From 58ad654ebce7ccb272a3f4f3482c03aaad850d31 Mon Sep 17 00:00:00 2001
-From: Jan Beulich <jbeulich@...>
-Date: Thu, 22 Jul 2021 11:20:38 +0200
-Subject: [PATCH] x86: work around build issue with GNU ld 2.37
-
-I suspect it is commit 40726f16a8d7 ("ld script expression parsing")
-which broke the hypervisor build, by no longer accepting section names
-with a dash in them inside ADDR() (and perhaps other script directives
-expecting just a section name, not an expression): .note.gnu.build-id
-is such a section.
-
-Quoting all section names passed to ADDR() via DECL_SECTION() works
-around the regression.
-
-Signed-off-by: Jan Beulich <jbeulich@...>
-Acked-by: Andrew Cooper <andrew.cooper3@...>
----
- xen/arch/x86/xen.lds.S | 2 +-
- 1 file changed, 1 insertion(+), 1 deletion(-)
-
-diff --git a/xen/arch/x86/xen.lds.S b/xen/arch/x86/xen.lds.S
-index 9c6c1c8005..955d5cf4a0 100644
---- a/xen/arch/x86/xen.lds.S
-+++ b/xen/arch/x86/xen.lds.S
-@@ -18,7 +18,7 @@ ENTRY(efi_start)
- #else /* !EFI */
-
- #define FORMAT "elf64-x86-64"
--#define DECL_SECTION(x) x : AT(ADDR(x) - __XEN_VIRT_START)
-+#define DECL_SECTION(x) x : AT(ADDR(#x) - __XEN_VIRT_START)
-
- ENTRY(start_pa)
-
---
-2.25.1
-
diff --git a/recipes-extended/xen/xen-tools.inc b/recipes-extended/xen/xen-tools.inc
index 614f557..d831941 100644
--- a/recipes-extended/xen/xen-tools.inc
+++ b/recipes-extended/xen/xen-tools.inc
@@ -102,6 +102,7 @@ RSUGGESTS:${PN}-xencov = "${PN}-xencov-split"

 RDEPENDS:${PN}-xencommons = "\
     bash \
+    util-linux-prlimit \
     ${PN}-console \
     ${PN}-xenstore \
     virtual/xenstored \
@@ -213,6 +214,9 @@ PACKAGES = " \
     ${PN}-xm-examples \
     ${PN}-xen-access \
     ${PN}-xen-memshare \
+    ${PN}-test \
+    ${PN}-xen-vmtrace \
+    ${PN}-xen-mceinj \
     "

 PROVIDES =+ " \
@@ -684,6 +688,31 @@ FILES:${PN}-xen-memshare += "\
 # memshare is only built for x86, so allow empty package for other archs
 ALLOW_EMPTY:${PN}-xen-memshare = "1"

+FILES:${PN}-test += "\
+    ${libdir}/xen/bin/test-xenstore \
+    ${libdir}/xen/bin/test-resource \
+    ${libdir}/xen/bin/test-cpu-policy \
+    ${libdir}/xen/bin/test-tsx \
+    "
+
+# test-xenstore and test-resource currently only exist in 4.16
+# test-cpu-policy and test-tsx only exist in 4.16 for x86
+ALLOW_EMPTY:${PN}-test = "1"
+
+FILES:${PN}-xen-mceinj +="\
+    ${sbindir}/xen-mceinj \
+    "
+
+# xen-mceinj is only built for x86 4.16, so allow empty package
+ALLOW_EMPTY:${PN}-xen-mceinj = "1"
+
+FILES:${PN}-xen-vmtrace +="\
+    ${sbindir}/xen-vmtrace \
+    "
+
+# xen-vmtrace is only built for x86 4.16, so allow empty package
+ALLOW_EMPTY:${PN}-xen-vmtrace = "1"
+
 INSANE_SKIP:${PN} = "already-stripped"

 # configure init.d scripts
diff --git a/recipes-extended/xen/xen-tools_4.14.bb b/recipes-extended/xen/xen-tools_4.14.bb
index ce5dc2e..9d78e44 100644
--- a/recipes-extended/xen/xen-tools_4.14.bb
+++ b/recipes-extended/xen/xen-tools_4.14.bb
@@ -1,4 +1,5 @@
-SRCREV ?= "b6a8c4f72def4d1135ff42660a86276ce2565c8c"
+# 4.14.3 release SHA
+SRCREV ?= "9f2b6c5ec2ded4c1caf149743e862c5f15d6d083"

 XEN_REL ?= "4.14"
 XEN_BRANCH ?= "stable-${XEN_REL}"
@@ -8,8 +9,6 @@ SRC_URI = " \
     file://0001-python-pygrub-pass-DISTUTILS-xen-4.14.patch \
     file://0001-firmware-provide-a-stand-alone-set-of-headers-Xen-4.14.patch \
     file://0001-tools-firmware-Build-firmware-as-ffreestanding-Xen-4.14.patch \
-    file://0001-libs-foreignmemory-Fix-osdep_xenforeignmemory_map-pr.patch \
-    file://0001-x86-work-around-build-issue-with-GNU-ld-2.37.patch \
     "

 LIC_FILES_CHKSUM ?= "file://COPYING;md5=419739e325a50f3d7b4501338e44a4e5"
diff --git a/recipes-extended/xen/xen-tools_4.15.bb b/recipes-extended/xen/xen-tools_4.15.bb
index e9c3bd6..840f162 100644
--- a/recipes-extended/xen/xen-tools_4.15.bb
+++ b/recipes-extended/xen/xen-tools_4.15.bb
@@ -1,4 +1,5 @@
-SRCREV ?= "e25aa9939ae0cd8317605be3d5c5611b76bc4ab4"
+# xen 4.15.1 release sha
+SRCREV ?= "84fa99099b920f7bcde8899e1a9b756078719d7d"

 XEN_REL ?= "4.15"
 XEN_BRANCH ?= "stable-${XEN_REL}"
@@ -6,8 +7,6 @@ XEN_BRANCH ?= "stable-${XEN_REL}"
 SRC_URI = " \
     git://xenbits.xen.org/xen.git;branch=${XEN_BRANCH} \
     file://0001-python-pygrub-pass-DISTUTILS-xen-4.15.patch \
-    file://0001-libs-foreignmemory-Fix-osdep_xenforeignmemory_map-pr.patch \
-    file://0001-x86-work-around-build-issue-with-GNU-ld-2.37.patch \
     "

 LIC_FILES_CHKSUM ?= "file://COPYING;md5=419739e325a50f3d7b4501338e44a4e5"
diff --git a/recipes-extended/xen/xen-tools_git.bb b/recipes-extended/xen/xen-tools_git.bb
index 6926bd9..8ff9c4c 100644
--- a/recipes-extended/xen/xen-tools_git.bb
+++ b/recipes-extended/xen/xen-tools_git.bb
@@ -1,4 +1,5 @@
-SRCREV ?= "e680cc48b7184d3489873d6776f84ba1fc238ced"
+# master status on 2020-10-21
+SRCREV ?= "23ec1ebc8acbfd2bf06f6085a776f0db923f9fa9"

 XEN_REL ?= "4.16"
 XEN_BRANCH ?= "master"
@@ -6,8 +7,6 @@ XEN_BRANCH ?= "master"
 SRC_URI = " \
     git://xenbits.xen.org/xen.git;branch=${XEN_BRANCH} \
     file://0001-python-pygrub-pass-DISTUTILS-xen-4.15.patch \
-    file://0001-libs-foreignmemory-Fix-osdep_xenforeignmemory_map-pr.patch \
-    file://0001-x86-work-around-build-issue-with-GNU-ld-2.37.patch \
     "

 LIC_FILES_CHKSUM ?= "file://COPYING;md5=419739e325a50f3d7b4501338e44a4e5"
diff --git a/recipes-extended/xen/xen_4.14.bb b/recipes-extended/xen/xen_4.14.bb
index 291ba16..0d7d20b 100644
--- a/recipes-extended/xen/xen_4.14.bb
+++ b/recipes-extended/xen/xen_4.14.bb
@@ -1,4 +1,5 @@
-SRCREV ?= "b6a8c4f72def4d1135ff42660a86276ce2565c8c"
+# 4.14.3 release SHA
+SRCREV ?= "9f2b6c5ec2ded4c1caf149743e862c5f15d6d083"

 XEN_REL ?= "4.14"
 XEN_BRANCH ?= "stable-${XEN_REL}"
@@ -7,7 +8,6 @@ SRC_URI = " \
     git://xenbits.xen.org/xen.git;branch=${XEN_BRANCH} \
     file://xen-arm64-implement-atomic-fetch-add.patch \
     file://0001-menuconfig-mconf-cfg-Allow-specification-of-ncurses-location.patch \
-    file://0001-x86-work-around-build-issue-with-GNU-ld-2.37.patch \
     "

 LIC_FILES_CHKSUM ?= "file://COPYING;md5=419739e325a50f3d7b4501338e44a4e5"
diff --git a/recipes-extended/xen/xen_4.15.bb b/recipes-extended/xen/xen_4.15.bb
index 0605441..0cfaf0c 100644
--- a/recipes-extended/xen/xen_4.15.bb
+++ b/recipes-extended/xen/xen_4.15.bb
@@ -1,4 +1,5 @@
-SRCREV ?= "e25aa9939ae0cd8317605be3d5c5611b76bc4ab4"
+# xen 4.15.1 release sha
+SRCREV ?= "84fa99099b920f7bcde8899e1a9b756078719d7d"

 XEN_REL ?= "4.15"
 XEN_BRANCH ?= "stable-${XEN_REL}"
@@ -7,8 +8,6 @@ SRC_URI = " \
     git://xenbits.xen.org/xen.git;branch=${XEN_BRANCH} \
     file://xen-arm64-implement-atomic-fetch-add.patch \
     file://0001-menuconfig-mconf-cfg-Allow-specification-of-ncurses-location.patch \
-    file://0001-x86-make-hypervisor-build-with-gcc11.patch \
-    file://0001-x86-work-around-build-issue-with-GNU-ld-2.37.patch \
     "

 LIC_FILES_CHKSUM ?= "file://COPYING;md5=419739e325a50f3d7b4501338e44a4e5"
diff --git a/recipes-extended/xen/xen_git.bb b/recipes-extended/xen/xen_git.bb
index 12c9403..20227e0 100644
--- a/recipes-extended/xen/xen_git.bb
+++ b/recipes-extended/xen/xen_git.bb
@@ -1,4 +1,5 @@
-SRCREV ?= "e680cc48b7184d3489873d6776f84ba1fc238ced"
+# master status on 2020-10-21
+SRCREV ?= "23ec1ebc8acbfd2bf06f6085a776f0db923f9fa9"

 XEN_REL ?= "4.16"
 XEN_BRANCH ?= "master"
@@ -7,8 +8,6 @@ SRC_URI = " \
     git://xenbits.xen.org/xen.git;branch=${XEN_BRANCH} \
     file://xen-arm64-implement-atomic-fetch-add.patch \
     file://0001-menuconfig-mconf-cfg-Allow-specification-of-ncurses-location.patch \
-    file://0001-x86-make-hypervisor-build-with-gcc11.patch \
-    file://0001-x86-work-around-build-issue-with-GNU-ld-2.37.patch \
     "

 LIC_FILES_CHKSUM ?= "file://COPYING;md5=419739e325a50f3d7b4501338e44a4e5"
--
2.25.1






--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II


[PATCH] xen,xen-tools: update xen to new releases

Edward Pickup
 

Update the SHA in recipes for xen and xen-tools to use the latest releases;
update xen 4.14 to xen 4.14.3 release, update xen 4.15 to 4.15.1 release, and
xen 4.16 to the current master version.

Add a comment to each recipe indicating which version of Xen is used in recipe.

Do not apply patches that have been merged/solved in Xen and remove patch files
not needed anymore.

Create a test package to pack new tests introduced in 4.16.

Add a depencency to util-linux-prlimit, as the xen 4.16 x86 requires prlimit,
without this boot of dom0 fails.

Signed-off-by: Edward Pickup <Edward.Pickup@...>
---
...ry-Fix-osdep_xenforeignmemory_map-pr.patch | 33 ------
...x86-make-hypervisor-build-with-gcc11.patch | 111 ------------------
...-around-build-issue-with-GNU-ld-2.37.patch | 36 ------
recipes-extended/xen/xen-tools.inc | 29 +++++
recipes-extended/xen/xen-tools_4.14.bb | 5 +-
recipes-extended/xen/xen-tools_4.15.bb | 5 +-
recipes-extended/xen/xen-tools_git.bb | 5 +-
recipes-extended/xen/xen_4.14.bb | 4 +-
recipes-extended/xen/xen_4.15.bb | 5 +-
recipes-extended/xen/xen_git.bb | 5 +-
10 files changed, 41 insertions(+), 197 deletions(-)
delete mode 100644 recipes-extended/xen/files/0001-libs-foreignmemory-Fix-osdep_xenforeignmemory_map-pr.patch
delete mode 100644 recipes-extended/xen/files/0001-x86-make-hypervisor-build-with-gcc11.patch
delete mode 100644 recipes-extended/xen/files/0001-x86-work-around-build-issue-with-GNU-ld-2.37.patch

diff --git a/recipes-extended/xen/files/0001-libs-foreignmemory-Fix-osdep_xenforeignmemory_map-pr.patch b/recipes-extended/xen/files/0001-libs-foreignmemory-Fix-osdep_xenforeignmemory_map-pr.patch
deleted file mode 100644
index 2e3b1a6..0000000
--- a/recipes-extended/xen/files/0001-libs-foreignmemory-Fix-osdep_xenforeignmemory_map-pr.patch
+++ /dev/null
@@ -1,33 +0,0 @@
-From 5d3e4ebb5c71477d74a0c503438545a0126d3863 Mon Sep 17 00:00:00 2001
-From: Anthony PERARD <anthony.perard@...>
-Date: Tue, 1 Jun 2021 16:41:47 +0100
-Subject: [PATCH] libs/foreignmemory: Fix osdep_xenforeignmemory_map prototype
-
-Commit cf8c4d3d13b8 made some preparation to have one day
-variable-length-array argument, but didn't declare the array in the
-function prototype the same way as in the function definition. And now
-GCC 11 complains about it.
-
-Fixes: cf8c4d3d13b8 ("tools/libs/foreignmemory: pull array length argument to map forward")
-Signed-off-by: Anthony PERARD <anthony.perard@...>
-Reviewed-by: Jan Beulich <jbeulich@...>
----
- tools/libs/foreignmemory/private.h | 2 +-
- 1 file changed, 1 insertion(+), 1 deletion(-)
-
-diff --git a/tools/libs/foreignmemory/private.h b/tools/libs/foreignmemory/private.h
-index 1ee3626dd2..5bb0cefb09 100644
---- a/tools/libs/foreignmemory/private.h
-+++ b/tools/libs/foreignmemory/private.h
-@@ -32,7 +32,7 @@ int osdep_xenforeignmemory_close(xenforeignmemory_handle *fmem);
- void *osdep_xenforeignmemory_map(xenforeignmemory_handle *fmem,
- uint32_t dom, void *addr,
- int prot, int flags, size_t num,
-- const xen_pfn_t arr[num], int err[num]);
-+ const xen_pfn_t arr[/*num*/], int err[/*num*/]);
- int osdep_xenforeignmemory_unmap(xenforeignmemory_handle *fmem,
- void *addr, size_t num);
-
---
-2.25.1
-
diff --git a/recipes-extended/xen/files/0001-x86-make-hypervisor-build-with-gcc11.patch b/recipes-extended/xen/files/0001-x86-make-hypervisor-build-with-gcc11.patch
deleted file mode 100644
index 72592c4..0000000
--- a/recipes-extended/xen/files/0001-x86-make-hypervisor-build-with-gcc11.patch
+++ /dev/null
@@ -1,111 +0,0 @@
-From 722f59d38c710a940ab05e542a83020eb5546dea Mon Sep 17 00:00:00 2001
-From: Jan Beulich <jbeulich@...>
-Date: Thu, 27 May 2021 14:40:29 +0200
-Subject: [PATCH] x86: make hypervisor build with gcc11
-MIME-Version: 1.0
-Content-Type: text/plain; charset=UTF-8
-Content-Transfer-Encoding: 8bit
-
-Gcc 11 looks to make incorrect assumptions about valid ranges that
-pointers may be used for addressing when they are derived from e.g. a
-plain constant. See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100680.
-
-Utilize RELOC_HIDE() to work around the issue, which for x86 manifests
-in at least
-- mpparse.c:efi_check_config(),
-- tboot.c:tboot_probe(),
-- tboot.c:tboot_gen_frametable_integrity(),
-- x86_emulate.c:x86_emulate() (at -O2 only).
-The last case is particularly odd not just because it only triggers at
-higher optimization levels, but also because it only affects one of at
-least three similar constructs. Various "note" diagnostics claim the
-valid index range to be [0, 2⁶³-1].
-
-Signed-off-by: Jan Beulich <jbeulich@...>
-Tested-by: Jason Andryuk <jandryuk@...>
-Acked-by: Roger Pau Monné <roger.pau@...>
----
- tools/tests/x86_emulator/x86-emulate.c | 7 +++++++
- xen/arch/x86/x86_emulate/x86_emulate.c | 2 +-
- xen/include/asm-x86/fixmap.h | 2 +-
- xen/include/xen/compiler.h | 6 ++++++
- xen/include/xen/pdx.h | 2 +-
- 5 files changed, 16 insertions(+), 3 deletions(-)
-
-diff --git a/tools/tests/x86_emulator/x86-emulate.c b/tools/tests/x86_emulator/x86-emulate.c
-index 07f892dbbb..ea286d6ad8 100644
---- a/tools/tests/x86_emulator/x86-emulate.c
-+++ b/tools/tests/x86_emulator/x86-emulate.c
-@@ -8,6 +8,13 @@
-
- #define ERR_PTR(val) NULL
-
-+/* See gcc bug 100680, but here don't bother making this version dependent. */
-+#define gcc11_wrap(x) ({ \
-+ unsigned long x_; \
-+ __asm__ ( "" : "=g" (x_) : "0" (x) ); \
-+ (typeof(x))x_; \
-+})
-+
- #define cpu_has_amd_erratum(nr) 0
- #define cpu_has_mpx false
- #define read_bndcfgu() 0
-diff --git a/xen/arch/x86/x86_emulate/x86_emulate.c b/xen/arch/x86/x86_emulate/x86_emulate.c
-index c25d88d0d8..31fdec030c 100644
---- a/xen/arch/x86/x86_emulate/x86_emulate.c
-+++ b/xen/arch/x86/x86_emulate/x86_emulate.c
-@@ -726,7 +726,7 @@ union vex {
- #define copy_VEX(ptr, vex) ({ \
- if ( !mode_64bit() ) \
- (vex).reg |= 8; \
-- (ptr)[0 - PFX_BYTES] = ext < ext_8f08 ? 0xc4 : 0x8f; \
-+ gcc11_wrap(ptr)[0 - PFX_BYTES] = ext < ext_8f08 ? 0xc4 : 0x8f; \
- (ptr)[1 - PFX_BYTES] = (vex).raw[0]; \
- (ptr)[2 - PFX_BYTES] = (vex).raw[1]; \
- container_of((ptr) + 1 - PFX_BYTES, typeof(vex), raw[0]); \
-diff --git a/xen/include/asm-x86/fixmap.h b/xen/include/asm-x86/fixmap.h
-index 0db314baeb..20746afd0a 100644
---- a/xen/include/asm-x86/fixmap.h
-+++ b/xen/include/asm-x86/fixmap.h
-@@ -78,7 +78,7 @@ extern void __set_fixmap(
-
- #define clear_fixmap(idx) __set_fixmap(idx, 0, 0)
-
--#define __fix_to_virt(x) (FIXADDR_TOP - ((x) << PAGE_SHIFT))
-+#define __fix_to_virt(x) gcc11_wrap(FIXADDR_TOP - ((x) << PAGE_SHIFT))
- #define __virt_to_fix(x) ((FIXADDR_TOP - ((x)&PAGE_MASK)) >> PAGE_SHIFT)
-
- #define fix_to_virt(x) ((void *)__fix_to_virt(x))
-diff --git a/xen/include/xen/compiler.h b/xen/include/xen/compiler.h
-index 17cf00e1ec..696c7eb89e 100644
---- a/xen/include/xen/compiler.h
-+++ b/xen/include/xen/compiler.h
-@@ -140,6 +140,12 @@
- __asm__ ("" : "=r"(__ptr) : "0"(ptr)); \
- (typeof(ptr)) (__ptr + (off)); })
-
-+#if CONFIG_GCC_VERSION >= 110000 /* See gcc bug 100680. */
-+# define gcc11_wrap(x) RELOC_HIDE(x, 0)
-+#else
-+# define gcc11_wrap(x) (x)
-+#endif
-+
- #ifdef __GCC_ASM_FLAG_OUTPUTS__
- # define ASM_FLAG_OUT(yes, no) yes
- #else
-diff --git a/xen/include/xen/pdx.h b/xen/include/xen/pdx.h
-index 770fadc06c..9fcfb0ce52 100644
---- a/xen/include/xen/pdx.h
-+++ b/xen/include/xen/pdx.h
-@@ -19,7 +19,7 @@ extern u64 pdx_region_mask(u64 base, u64 len);
- extern void set_pdx_range(unsigned long smfn, unsigned long emfn);
-
- #define page_to_pdx(pg) ((pg) - frame_table)
--#define pdx_to_page(pdx) (frame_table + (pdx))
-+#define pdx_to_page(pdx) gcc11_wrap(frame_table + (pdx))
-
- bool __mfn_valid(unsigned long mfn);
-
---
-2.25.1
-
diff --git a/recipes-extended/xen/files/0001-x86-work-around-build-issue-with-GNU-ld-2.37.patch b/recipes-extended/xen/files/0001-x86-work-around-build-issue-with-GNU-ld-2.37.patch
deleted file mode 100644
index f5108be..0000000
--- a/recipes-extended/xen/files/0001-x86-work-around-build-issue-with-GNU-ld-2.37.patch
+++ /dev/null
@@ -1,36 +0,0 @@
-From 58ad654ebce7ccb272a3f4f3482c03aaad850d31 Mon Sep 17 00:00:00 2001
-From: Jan Beulich <jbeulich@...>
-Date: Thu, 22 Jul 2021 11:20:38 +0200
-Subject: [PATCH] x86: work around build issue with GNU ld 2.37
-
-I suspect it is commit 40726f16a8d7 ("ld script expression parsing")
-which broke the hypervisor build, by no longer accepting section names
-with a dash in them inside ADDR() (and perhaps other script directives
-expecting just a section name, not an expression): .note.gnu.build-id
-is such a section.
-
-Quoting all section names passed to ADDR() via DECL_SECTION() works
-around the regression.
-
-Signed-off-by: Jan Beulich <jbeulich@...>
-Acked-by: Andrew Cooper <andrew.cooper3@...>
----
- xen/arch/x86/xen.lds.S | 2 +-
- 1 file changed, 1 insertion(+), 1 deletion(-)
-
-diff --git a/xen/arch/x86/xen.lds.S b/xen/arch/x86/xen.lds.S
-index 9c6c1c8005..955d5cf4a0 100644
---- a/xen/arch/x86/xen.lds.S
-+++ b/xen/arch/x86/xen.lds.S
-@@ -18,7 +18,7 @@ ENTRY(efi_start)
- #else /* !EFI */
-
- #define FORMAT "elf64-x86-64"
--#define DECL_SECTION(x) x : AT(ADDR(x) - __XEN_VIRT_START)
-+#define DECL_SECTION(x) x : AT(ADDR(#x) - __XEN_VIRT_START)
-
- ENTRY(start_pa)
-
---
-2.25.1
-
diff --git a/recipes-extended/xen/xen-tools.inc b/recipes-extended/xen/xen-tools.inc
index 614f557..d831941 100644
--- a/recipes-extended/xen/xen-tools.inc
+++ b/recipes-extended/xen/xen-tools.inc
@@ -102,6 +102,7 @@ RSUGGESTS:${PN}-xencov = "${PN}-xencov-split"

RDEPENDS:${PN}-xencommons = "\
bash \
+ util-linux-prlimit \
${PN}-console \
${PN}-xenstore \
virtual/xenstored \
@@ -213,6 +214,9 @@ PACKAGES = " \
${PN}-xm-examples \
${PN}-xen-access \
${PN}-xen-memshare \
+ ${PN}-test \
+ ${PN}-xen-vmtrace \
+ ${PN}-xen-mceinj \
"

PROVIDES =+ " \
@@ -684,6 +688,31 @@ FILES:${PN}-xen-memshare += "\
# memshare is only built for x86, so allow empty package for other archs
ALLOW_EMPTY:${PN}-xen-memshare = "1"

+FILES:${PN}-test += "\
+ ${libdir}/xen/bin/test-xenstore \
+ ${libdir}/xen/bin/test-resource \
+ ${libdir}/xen/bin/test-cpu-policy \
+ ${libdir}/xen/bin/test-tsx \
+ "
+
+# test-xenstore and test-resource currently only exist in 4.16
+# test-cpu-policy and test-tsx only exist in 4.16 for x86
+ALLOW_EMPTY:${PN}-test = "1"
+
+FILES:${PN}-xen-mceinj +="\
+ ${sbindir}/xen-mceinj \
+ "
+
+# xen-mceinj is only built for x86 4.16, so allow empty package
+ALLOW_EMPTY:${PN}-xen-mceinj = "1"
+
+FILES:${PN}-xen-vmtrace +="\
+ ${sbindir}/xen-vmtrace \
+ "
+
+# xen-vmtrace is only built for x86 4.16, so allow empty package
+ALLOW_EMPTY:${PN}-xen-vmtrace = "1"
+
INSANE_SKIP:${PN} = "already-stripped"

# configure init.d scripts
diff --git a/recipes-extended/xen/xen-tools_4.14.bb b/recipes-extended/xen/xen-tools_4.14.bb
index ce5dc2e..9d78e44 100644
--- a/recipes-extended/xen/xen-tools_4.14.bb
+++ b/recipes-extended/xen/xen-tools_4.14.bb
@@ -1,4 +1,5 @@
-SRCREV ?= "b6a8c4f72def4d1135ff42660a86276ce2565c8c"
+# 4.14.3 release SHA
+SRCREV ?= "9f2b6c5ec2ded4c1caf149743e862c5f15d6d083"

XEN_REL ?= "4.14"
XEN_BRANCH ?= "stable-${XEN_REL}"
@@ -8,8 +9,6 @@ SRC_URI = " \
file://0001-python-pygrub-pass-DISTUTILS-xen-4.14.patch \
file://0001-firmware-provide-a-stand-alone-set-of-headers-Xen-4.14.patch \
file://0001-tools-firmware-Build-firmware-as-ffreestanding-Xen-4.14.patch \
- file://0001-libs-foreignmemory-Fix-osdep_xenforeignmemory_map-pr.patch \
- file://0001-x86-work-around-build-issue-with-GNU-ld-2.37.patch \
"

LIC_FILES_CHKSUM ?= "file://COPYING;md5=419739e325a50f3d7b4501338e44a4e5"
diff --git a/recipes-extended/xen/xen-tools_4.15.bb b/recipes-extended/xen/xen-tools_4.15.bb
index e9c3bd6..840f162 100644
--- a/recipes-extended/xen/xen-tools_4.15.bb
+++ b/recipes-extended/xen/xen-tools_4.15.bb
@@ -1,4 +1,5 @@
-SRCREV ?= "e25aa9939ae0cd8317605be3d5c5611b76bc4ab4"
+# xen 4.15.1 release sha
+SRCREV ?= "84fa99099b920f7bcde8899e1a9b756078719d7d"

XEN_REL ?= "4.15"
XEN_BRANCH ?= "stable-${XEN_REL}"
@@ -6,8 +7,6 @@ XEN_BRANCH ?= "stable-${XEN_REL}"
SRC_URI = " \
git://xenbits.xen.org/xen.git;branch=${XEN_BRANCH} \
file://0001-python-pygrub-pass-DISTUTILS-xen-4.15.patch \
- file://0001-libs-foreignmemory-Fix-osdep_xenforeignmemory_map-pr.patch \
- file://0001-x86-work-around-build-issue-with-GNU-ld-2.37.patch \
"

LIC_FILES_CHKSUM ?= "file://COPYING;md5=419739e325a50f3d7b4501338e44a4e5"
diff --git a/recipes-extended/xen/xen-tools_git.bb b/recipes-extended/xen/xen-tools_git.bb
index 6926bd9..8ff9c4c 100644
--- a/recipes-extended/xen/xen-tools_git.bb
+++ b/recipes-extended/xen/xen-tools_git.bb
@@ -1,4 +1,5 @@
-SRCREV ?= "e680cc48b7184d3489873d6776f84ba1fc238ced"
+# master status on 2020-10-21
+SRCREV ?= "23ec1ebc8acbfd2bf06f6085a776f0db923f9fa9"

XEN_REL ?= "4.16"
XEN_BRANCH ?= "master"
@@ -6,8 +7,6 @@ XEN_BRANCH ?= "master"
SRC_URI = " \
git://xenbits.xen.org/xen.git;branch=${XEN_BRANCH} \
file://0001-python-pygrub-pass-DISTUTILS-xen-4.15.patch \
- file://0001-libs-foreignmemory-Fix-osdep_xenforeignmemory_map-pr.patch \
- file://0001-x86-work-around-build-issue-with-GNU-ld-2.37.patch \
"

LIC_FILES_CHKSUM ?= "file://COPYING;md5=419739e325a50f3d7b4501338e44a4e5"
diff --git a/recipes-extended/xen/xen_4.14.bb b/recipes-extended/xen/xen_4.14.bb
index 291ba16..0d7d20b 100644
--- a/recipes-extended/xen/xen_4.14.bb
+++ b/recipes-extended/xen/xen_4.14.bb
@@ -1,4 +1,5 @@
-SRCREV ?= "b6a8c4f72def4d1135ff42660a86276ce2565c8c"
+# 4.14.3 release SHA
+SRCREV ?= "9f2b6c5ec2ded4c1caf149743e862c5f15d6d083"

XEN_REL ?= "4.14"
XEN_BRANCH ?= "stable-${XEN_REL}"
@@ -7,7 +8,6 @@ SRC_URI = " \
git://xenbits.xen.org/xen.git;branch=${XEN_BRANCH} \
file://xen-arm64-implement-atomic-fetch-add.patch \
file://0001-menuconfig-mconf-cfg-Allow-specification-of-ncurses-location.patch \
- file://0001-x86-work-around-build-issue-with-GNU-ld-2.37.patch \
"

LIC_FILES_CHKSUM ?= "file://COPYING;md5=419739e325a50f3d7b4501338e44a4e5"
diff --git a/recipes-extended/xen/xen_4.15.bb b/recipes-extended/xen/xen_4.15.bb
index 0605441..0cfaf0c 100644
--- a/recipes-extended/xen/xen_4.15.bb
+++ b/recipes-extended/xen/xen_4.15.bb
@@ -1,4 +1,5 @@
-SRCREV ?= "e25aa9939ae0cd8317605be3d5c5611b76bc4ab4"
+# xen 4.15.1 release sha
+SRCREV ?= "84fa99099b920f7bcde8899e1a9b756078719d7d"

XEN_REL ?= "4.15"
XEN_BRANCH ?= "stable-${XEN_REL}"
@@ -7,8 +8,6 @@ SRC_URI = " \
git://xenbits.xen.org/xen.git;branch=${XEN_BRANCH} \
file://xen-arm64-implement-atomic-fetch-add.patch \
file://0001-menuconfig-mconf-cfg-Allow-specification-of-ncurses-location.patch \
- file://0001-x86-make-hypervisor-build-with-gcc11.patch \
- file://0001-x86-work-around-build-issue-with-GNU-ld-2.37.patch \
"

LIC_FILES_CHKSUM ?= "file://COPYING;md5=419739e325a50f3d7b4501338e44a4e5"
diff --git a/recipes-extended/xen/xen_git.bb b/recipes-extended/xen/xen_git.bb
index 12c9403..20227e0 100644
--- a/recipes-extended/xen/xen_git.bb
+++ b/recipes-extended/xen/xen_git.bb
@@ -1,4 +1,5 @@
-SRCREV ?= "e680cc48b7184d3489873d6776f84ba1fc238ced"
+# master status on 2020-10-21
+SRCREV ?= "23ec1ebc8acbfd2bf06f6085a776f0db923f9fa9"

XEN_REL ?= "4.16"
XEN_BRANCH ?= "master"
@@ -7,8 +8,6 @@ SRC_URI = " \
git://xenbits.xen.org/xen.git;branch=${XEN_BRANCH} \
file://xen-arm64-implement-atomic-fetch-add.patch \
file://0001-menuconfig-mconf-cfg-Allow-specification-of-ncurses-location.patch \
- file://0001-x86-make-hypervisor-build-with-gcc11.patch \
- file://0001-x86-work-around-build-issue-with-GNU-ld-2.37.patch \
"

LIC_FILES_CHKSUM ?= "file://COPYING;md5=419739e325a50f3d7b4501338e44a4e5"
--
2.25.1


Re: Error building Xen device tree for qemuarm64

Edward Pickup
 

Hi Bruce,
Sorry for the formatting issues, email client config issues

 

From: Bruce Ashfield <bruce.ashfield@...>
Sent: 26 October 2021 14:22
To: Edward Pickup <Edward.Pickup@...>
Cc: meta-virtualization@...; Bertrand Marquis <Bertrand.Marquis@...>; christopher.w.clark@...; nd <nd@...>
Subject: Re: [meta-virtualization] Error building Xen device tree for qemuarm64

 

 

 

On Tue, Oct 26, 2021 at 8:58 AM Edward Pickup <edward.pickup@...> wrote:

Hi all,

I have encountered an error while building the arm64 version of the xen 4.16 recipe in meta-virtualisation, using the master version of poky and meta-virtuaisation.
I was wondering if anyone was familiar with the issue or knew how to fix it. The error is as follows:

ERROR: xen-image-minimal-1.0-r0 do_write_xen_qemuboot_dtb: ExecutionError('/home/edwpic01/work/master-update-xen/build/metavirt-host-arm64-4.16-release.prj/tmp/work/qemuarm64-poky-linux/xen-image-minimal/1.0-r0/temp/run.do_write_xen_qemuboot_dtb.3679135', 2, None, None)
ERROR: Logfile of failure stored in: /home/edwpic01/work/master-update-xen/build/metavirt-host-arm64-4.16-release.prj/tmp/work/qemuarm64-poky-linux/xen-image-minimal/1.0-r0/temp/log.do_write_xen_qemuboot_dtb.3679135 .

I believe the relevant part of the log is as follows:

| [NOTE]: '3' lopper operation files will be processed
| [INFO]: dtb output format detected, writing /home/edwpic01/work/master-update-xen/build/metavirt-host-arm64-4.16-release.prj/tmp/work/qemuarm64-poky-linux/xen-image-minimal/1.0-r0/deploy-xen-image-minimal-image-complete/xen-image-minimal-qemuarm64-20211026121502.qemuboot.dtb
| [INFO]: writing output dtb: /home/edwpic01/work/master-update-xen/build/metavirt-host-arm64-4.16-release.prj/tmp/work/qemuarm64-poky-linux/xen-image-minimal/1.0-r0/deploy-xen-image-minimal-image-complete/xen-image-minimal-qemuarm64-20211026121502.qemuboot.dtb
| /home/edwpic01/work/master-update-xen/build/metavirt-host-arm64-4.16-release.prj/tmp/work/qemuarm64-poky-linux/xen-image-minimal/1.0-r0/xen-image-minimal-1.0/output.dts: ERROR (explicit_phandles): /apb-pclk:phandle: bad length (1) phandle property
| /home/edwpic01/work/master-update-xen/build/metavirt-host-arm64-4.16-release.prj/tmp/work/qemuarm64-poky-linux/xen-image-minimal/1.0-r0/xen-image-minimal-1.0/output.dts: ERROR (explicit_phandles): /intc@8000000:phandle: bad length (1) phandle property
| /home/edwpic01/work/master-update-xen/build/metavirt-host-arm64-4.16-release.prj/tmp/work/qemuarm64-poky-linux/xen-image-minimal/1.0-r0/xen-image-minimal-1.0/output.dts: ERROR (explicit_phandles): /pl061@9030000:phandle: bad length (1) phandle property
| ERROR: Input tree has errors, aborting (use -f to force output)
| WARNING: exit code 2 from a shell command.

The above error was the result from building on x86_64-linux, target system was aarch64-poky-linux, running on a qemuarch64 machine, itself running on x86_64 linux.
As mentioned this was done on the master version of meta-virtualisation.
The bitbake command to generate this was bitbake xen-image-mininal.
The error is also present when building the xen 4.14 and xen 4.15 arm64 versions, and is not present on the arm32 or x86 builds for xen.
If anyone knows the cause of the error, the information would be appreciated.

 

There's something wrong with the input DTS to Lopper .. luckily, I maintain Lopper, so if there's an actual bug in the processing of the input DTS .. I can fix that.

 

I can fire up a build and see if it happens here.

 

So this is just qemuarm64, with xen + virtualization in the distro features ? Any other layers ? Anything else ?

 

Bruce

Yes, this is just as you say, no other layers or anything else. Thank you for the speedy reply!

 

Edward

 


Cheers!


IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.


 

--

- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.


Re: Error building Xen device tree for qemuarm64

Bruce Ashfield
 



On Tue, Oct 26, 2021 at 8:58 AM Edward Pickup <edward.pickup@...> wrote:
Hi all,

I have encountered an error while building the arm64 version of the xen 4.16 recipe in meta-virtualisation, using the master version of poky and meta-virtuaisation.
I was wondering if anyone was familiar with the issue or knew how to fix it. The error is as follows:

ERROR: xen-image-minimal-1.0-r0 do_write_xen_qemuboot_dtb: ExecutionError('/home/edwpic01/work/master-update-xen/build/metavirt-host-arm64-4.16-release.prj/tmp/work/qemuarm64-poky-linux/xen-image-minimal/1.0-r0/temp/run.do_write_xen_qemuboot_dtb.3679135', 2, None, None)
ERROR: Logfile of failure stored in: /home/edwpic01/work/master-update-xen/build/metavirt-host-arm64-4.16-release.prj/tmp/work/qemuarm64-poky-linux/xen-image-minimal/1.0-r0/temp/log.do_write_xen_qemuboot_dtb.3679135 .

I believe the relevant part of the log is as follows:

| [NOTE]: '3' lopper operation files will be processed
| [INFO]: dtb output format detected, writing /home/edwpic01/work/master-update-xen/build/metavirt-host-arm64-4.16-release.prj/tmp/work/qemuarm64-poky-linux/xen-image-minimal/1.0-r0/deploy-xen-image-minimal-image-complete/xen-image-minimal-qemuarm64-20211026121502.qemuboot.dtb
| [INFO]: writing output dtb: /home/edwpic01/work/master-update-xen/build/metavirt-host-arm64-4.16-release.prj/tmp/work/qemuarm64-poky-linux/xen-image-minimal/1.0-r0/deploy-xen-image-minimal-image-complete/xen-image-minimal-qemuarm64-20211026121502.qemuboot.dtb
| /home/edwpic01/work/master-update-xen/build/metavirt-host-arm64-4.16-release.prj/tmp/work/qemuarm64-poky-linux/xen-image-minimal/1.0-r0/xen-image-minimal-1.0/output.dts: ERROR (explicit_phandles): /apb-pclk:phandle: bad length (1) phandle property
| /home/edwpic01/work/master-update-xen/build/metavirt-host-arm64-4.16-release.prj/tmp/work/qemuarm64-poky-linux/xen-image-minimal/1.0-r0/xen-image-minimal-1.0/output.dts: ERROR (explicit_phandles): /intc@8000000:phandle: bad length (1) phandle property
| /home/edwpic01/work/master-update-xen/build/metavirt-host-arm64-4.16-release.prj/tmp/work/qemuarm64-poky-linux/xen-image-minimal/1.0-r0/xen-image-minimal-1.0/output.dts: ERROR (explicit_phandles): /pl061@9030000:phandle: bad length (1) phandle property
| ERROR: Input tree has errors, aborting (use -f to force output)
| WARNING: exit code 2 from a shell command.

The above error was the result from building on x86_64-linux, target system was aarch64-poky-linux, running on a qemuarch64 machine, itself running on x86_64 linux.
As mentioned this was done on the master version of meta-virtualisation.
The bitbake command to generate this was bitbake xen-image-mininal.
The error is also present when building the xen 4.14 and xen 4.15 arm64 versions, and is not present on the arm32 or x86 builds for xen.
If anyone knows the cause of the error, the information would be appreciated.

There's something wrong with the input DTS to Lopper .. luckily, I maintain Lopper, so if there's an actual bug in the processing of the input DTS .. I can fix that.

I can fire up a build and see if it happens here.

So this is just qemuarm64, with xen + virtualization in the distro features ? Any other layers ? Anything else ?

Bruce

 

Cheers!


IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.





--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II


Error building Xen device tree for qemuarm64

Edward Pickup
 

Hi all,

I have encountered an error while building the arm64 version of the xen 4.16 recipe in meta-virtualisation, using the master version of poky and meta-virtuaisation.
I was wondering if anyone was familiar with the issue or knew how to fix it. The error is as follows:

ERROR: xen-image-minimal-1.0-r0 do_write_xen_qemuboot_dtb: ExecutionError('/home/edwpic01/work/master-update-xen/build/metavirt-host-arm64-4.16-release.prj/tmp/work/qemuarm64-poky-linux/xen-image-minimal/1.0-r0/temp/run.do_write_xen_qemuboot_dtb.3679135', 2, None, None)
ERROR: Logfile of failure stored in: /home/edwpic01/work/master-update-xen/build/metavirt-host-arm64-4.16-release.prj/tmp/work/qemuarm64-poky-linux/xen-image-minimal/1.0-r0/temp/log.do_write_xen_qemuboot_dtb.3679135 .

I believe the relevant part of the log is as follows:

| [NOTE]: '3' lopper operation files will be processed
| [INFO]: dtb output format detected, writing /home/edwpic01/work/master-update-xen/build/metavirt-host-arm64-4.16-release.prj/tmp/work/qemuarm64-poky-linux/xen-image-minimal/1.0-r0/deploy-xen-image-minimal-image-complete/xen-image-minimal-qemuarm64-20211026121502.qemuboot.dtb
| [INFO]: writing output dtb: /home/edwpic01/work/master-update-xen/build/metavirt-host-arm64-4.16-release.prj/tmp/work/qemuarm64-poky-linux/xen-image-minimal/1.0-r0/deploy-xen-image-minimal-image-complete/xen-image-minimal-qemuarm64-20211026121502.qemuboot.dtb
| /home/edwpic01/work/master-update-xen/build/metavirt-host-arm64-4.16-release.prj/tmp/work/qemuarm64-poky-linux/xen-image-minimal/1.0-r0/xen-image-minimal-1.0/output.dts: ERROR (explicit_phandles): /apb-pclk:phandle: bad length (1) phandle property
| /home/edwpic01/work/master-update-xen/build/metavirt-host-arm64-4.16-release.prj/tmp/work/qemuarm64-poky-linux/xen-image-minimal/1.0-r0/xen-image-minimal-1.0/output.dts: ERROR (explicit_phandles): /intc@8000000:phandle: bad length (1) phandle property
| /home/edwpic01/work/master-update-xen/build/metavirt-host-arm64-4.16-release.prj/tmp/work/qemuarm64-poky-linux/xen-image-minimal/1.0-r0/xen-image-minimal-1.0/output.dts: ERROR (explicit_phandles): /pl061@9030000:phandle: bad length (1) phandle property
| ERROR: Input tree has errors, aborting (use -f to force output)
| WARNING: exit code 2 from a shell command.

The above error was the result from building on x86_64-linux, target system was aarch64-poky-linux, running on a qemuarch64 machine, itself running on x86_64 linux.
As mentioned this was done on the master version of meta-virtualisation.
The bitbake command to generate this was bitbake xen-image-mininal.
The error is also present when building the xen 4.14 and xen 4.15 arm64 versions, and is not present on the arm32 or x86 builds for xen.
If anyone knows the cause of the error, the information would be appreciated.

Cheers!


IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.


Re: Problems Building Docker with dunfell

Bruce Ashfield
 



On Mon, Oct 25, 2021 at 8:26 PM Jay Foster <jay.foster@...> wrote:
Sorry for the noise.  I think I found my silly mistake.  I checked out
the dunfell branch instead of the yocto-3.1.11 tag, which results in a
newer, incompatible, version of go.

The tip of dunfell should be fine, it doesn't take those sorts of upgrades (incompatible go updates).

I fired off a clean build from the latest dunfell, and didn't have any issues.
 
Bruce


Jay

On 10/25/2021 3:38 PM, Jay Foster wrote:
> I am trying to build docker (virtual/docker) from dunfell
> meta/virtualization layer with go from the dunfell
> meta/recipes-devtools/go and cannot get it to build.  docker, runc,
> and containerd (the go based programs) each fail with an error like:
>
> DEBUG: Executing shell function do_compile
> NOTE: make -j 6 GODEBUG=1 binaries
> 🇩 bin/ctr
> go: cannot find main module, but found vendor.conf in
> /home2/jay/work/sc-docker/sc20_linux/poky/build/tmp-glibc/work/armv7ahf-neon-oe-linux-gnueabi/containerd-opencontainers/v1.2.14+gitAUTOINC+3b3e9d5f62-r0/git/src/import
>     to create a module there, run:
>     go mod init
> Makefile:173: recipe for target 'bin/ctr' failed
> make: *** [bin/ctr] Error 1
> ERROR: oe_runmake failed
> WARNING:
> /home2/jay/work/sc-docker/sc20_linux/poky/build/tmp-glibc/work/armv7ahf-neon-oe-linux-gnueabi/containerd-opencontainers/v1.2.14+gitAUTOINC+3b3e9d5f62-r0/temp/run.do_compile.23657:1
> exit 1 from 'exit 1'
> ERROR: Function failed: do_compile (log file is located at
> /home2/jay/work/sc-docker/sc20_linux/poky/build/tmp-glibc/work/armv7ahf-neon-oe-linux-gnueabi/containerd-opencontainers/v1.2.14+gitAUTOINC+3b3e9d5f62-r0/temp/log.do_compile.23657)
>
>
> DEBUG: Executing shell function do_compile
> NOTE: make -j 6 BUILDTAGS=${PACKAGECONFIG_CONFARGS}
> GO=arm-oe-linux-gnueabi-go static
> CGO_ENABLED=1 arm-oe-linux-gnueabi-go build  -tags " netgo osusergo
> static_build" -installsuffix netgo -ldflags "-w -extldflags -static -X
> main.gitCommit="425e105d5a03fabd737a126ad93d62a9eeede87f-dirty" -X
> main.version=1.0.0-rc8 " -o runc .
> go: cannot find main module, but found vendor.conf in
> /home2/jay/work/sc-docker/sc20_linux/poky/build/tmp-glibc/work/armv7ahf-neon-oe-linux-gnueabi/runc-docker/1.0.0-rc8+gitAUTOINC+425e105d5a-r0/git/src/import
>     to create a module there, run:
>     go mod init
> Makefile:43: recipe for target 'static' failed
> make: *** [static] Error 1
> ERROR: oe_runmake failed
> WARNING:
> /home2/jay/work/sc-docker/sc20_linux/poky/build/tmp-glibc/work/armv7ahf-neon-oe-linux-gnueabi/runc-docker/1.0.0-rc8+gitAUTOINC+425e105d5a-r0/temp/run.do_compile.23438:1
> exit 1 from 'exit 1'
> ERROR: Function failed: do_compile (log file is located at
> /home2/jay/work/sc-docker/sc20_linux/poky/build/tmp-glibc/work/armv7ahf-neon-oe-linux-gnueabi/runc-docker/1.0.0-rc8+gitAUTOINC+425e105d5a-r0/temp/log.do_compile.23438)
>
> Am I missing something obvious or is meta/virtualization (dunfell) out
> of sync with meta (dunfell)?
>
> Jay
>



--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II


[m-c-s][hardknott][PATCH] python3-posix-ipc: Add LDSHARED to recipe

Trevor Gamblin
 

Doing "import posix_ipc" on hardknott builds gives the following error:

ImportError: /usr/lib/python3.9/site-packages/posix_ipc.cpython-39-x86_64-linux-gnu.so: undefined symbol: shm_unlink

Set LDSHARED for the recipe with "-lrt" included to fix this.

Signed-off-by: Trevor Gamblin <trevor.gamblin@...>
---
.../recipes-devtools/python/python3-posix-ipc_1.0.4.bb | 2 ++
1 file changed, 2 insertions(+)

diff --git a/meta-openstack/recipes-devtools/python/python3-posix-ipc_1.0.4.bb b/meta-openstack/recipes-devtools/python/python3-posix-ipc_1.0.4.bb
index 2e8dca06..cded499d 100644
--- a/meta-openstack/recipes-devtools/python/python3-posix-ipc_1.0.4.bb
+++ b/meta-openstack/recipes-devtools/python/python3-posix-ipc_1.0.4.bb
@@ -20,6 +20,8 @@ DEPENDS += " \
${PYTHON_PN}-pip \
"

+LDSHARED="${CCLD} -shared -lrt"
+
# RDEPENDS_default:
RDEPENDS_${PN} += " \
"
--
2.33.0


[PATCH] ceph: 15.2.12 -> 15.2.15

kai
 

From: Kai Kang <kai.kang@...>

Update ceph to latest 15.x version:

* remove 0001-add-missing-include-for-atomic-bool.patch which has been
fixed in upstream
* update patch to support python 3.10
* add gawk to RDEPENDS that script 'rgw-gap-list-comparator' requires it

Signed-off-by: Kai Kang <kai.kang@...>
---
...-add-missing-include-for-atomic-bool.patch | 25 -------------------
...01-cmake-add-support-for-python3.10.patch} | 12 ++++++---
.../ceph/{ceph_15.2.12.bb => ceph_15.2.15.bb} | 9 +++----
3 files changed, 11 insertions(+), 35 deletions(-)
delete mode 100644 recipes-extended/ceph/ceph/0001-add-missing-include-for-atomic-bool.patch
rename recipes-extended/ceph/ceph/{0001-cmake-add-support-for-python3.9.patch => 0001-cmake-add-support-for-python3.10.patch} (76%)
rename recipes-extended/ceph/{ceph_15.2.12.bb => ceph_15.2.15.bb} (89%)

diff --git a/recipes-extended/ceph/ceph/0001-add-missing-include-for-atomic-bool.patch b/recipes-extended/ceph/ceph/0001-add-missing-include-for-atomic-bool.patch
deleted file mode 100644
index ffcbc583..00000000
--- a/recipes-extended/ceph/ceph/0001-add-missing-include-for-atomic-bool.patch
+++ /dev/null
@@ -1,25 +0,0 @@
-From 502be73be3c3645094525aa552f5e53d8782a004 Mon Sep 17 00:00:00 2001
-From: Sakib Sajal <sakib.sajal@...>
-Date: Fri, 28 Aug 2020 19:36:19 -0400
-Subject: [PATCH] add missing include for atomic<bool>
-
-Signed-off-by: Sakib Sajal <sakib.sajal@...>
----
- src/tools/rbd/action/Bench.cc | 1 +
- 1 file changed, 1 insertion(+)
-
-diff --git a/src/tools/rbd/action/Bench.cc b/src/tools/rbd/action/Bench.cc
-index aa6edbc1..90c551c1 100644
---- a/src/tools/rbd/action/Bench.cc
-+++ b/src/tools/rbd/action/Bench.cc
-@@ -9,6 +9,7 @@
- #include "common/ceph_mutex.h"
- #include "include/types.h"
- #include "global/signal_handler.h"
-+#include <atomic>
- #include <iostream>
- #include <boost/accumulators/accumulators.hpp>
- #include <boost/accumulators/statistics/stats.hpp>
---
-2.27.0
-
diff --git a/recipes-extended/ceph/ceph/0001-cmake-add-support-for-python3.9.patch b/recipes-extended/ceph/ceph/0001-cmake-add-support-for-python3.10.patch
similarity index 76%
rename from recipes-extended/ceph/ceph/0001-cmake-add-support-for-python3.9.patch
rename to recipes-extended/ceph/ceph/0001-cmake-add-support-for-python3.10.patch
index d9116c4f..94c4b8a6 100644
--- a/recipes-extended/ceph/ceph/0001-cmake-add-support-for-python3.9.patch
+++ b/recipes-extended/ceph/ceph/0001-cmake-add-support-for-python3.10.patch
@@ -1,13 +1,17 @@
From 1060f2e4362ebd6db23870d442dcd158d219ee92 Mon Sep 17 00:00:00 2001
From: Yanfei Xu <yanfei.xu@...>
Date: Tue, 10 Nov 2020 17:17:30 +0800
-Subject: [PATCH] cmake: add support for python3.9
+Subject: [PATCH] cmake: add support for python 3.9 and 3.10

add support for python3.9.

-Upstream-Status: Pending
-
Signed-off-by: Yanfei Xu <yanfei.xu@...>
+
+Add support for python 3.10.
+
+Upstream-Status: Submitted [https://github.com/ceph/ceph/pull/43630]
+
+Signed-off-by: Kai Kang <kai.kang@...>
---
cmake/modules/FindPython/Support.cmake | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
@@ -21,7 +25,7 @@ index 6584699b79..c05bbe3306 100644
endif()
if (_${_PYTHON_PREFIX}_REQUIRED_VERSION_MAJOR EQUAL 3)
- set(_${_PYTHON_PREFIX}_VERSIONS 3.8 3.7 3.6 3.5 3.4 3.3 3.2 3.1 3.0)
-+ set(_${_PYTHON_PREFIX}_VERSIONS 3.9 3.8 3.7 3.6 3.5 3.4 3.3 3.2 3.1 3.0)
++ set(_${_PYTHON_PREFIX}_VERSIONS 3.10 3.9 3.8 3.7 3.6 3.5 3.4 3.3 3.2 3.1 3.0)
elseif (_${_PYTHON_PREFIX}_REQUIRED_VERSION_MAJOR EQUAL 2)
set(_${_PYTHON_PREFIX}_VERSIONS 2.7 2.6 2.5 2.4 2.3 2.2 2.1 2.0)
else()
diff --git a/recipes-extended/ceph/ceph_15.2.12.bb b/recipes-extended/ceph/ceph_15.2.15.bb
similarity index 89%
rename from recipes-extended/ceph/ceph_15.2.12.bb
rename to recipes-extended/ceph/ceph_15.2.15.bb
index 693b5259..0d9bae02 100644
--- a/recipes-extended/ceph/ceph_15.2.12.bb
+++ b/recipes-extended/ceph/ceph_15.2.15.bb
@@ -12,15 +12,11 @@ SRC_URI = "http://download.ceph.com/tarballs/ceph-${PV}.tar.gz \
file://0001-ceph-fix-build-errors-for-cross-compile.patch \
file://0001-fix-host-library-paths-were-used.patch \
file://ceph.conf \
- file://0001-add-missing-include-for-atomic-bool.patch \
- file://0001-cmake-add-support-for-python3.9.patch \
+ file://0001-cmake-add-support-for-python3.10.patch \
file://0001-SnappyCompressor.h-fix-snappy-compiler-error.patch \
"

-SRC_URI[sha1sum] = "9c76b298e1e659b9c5501003ac77fb28aab7823d"
-SRC_URI[sha256sum] = "8710f6700de83765138df8eb4fc7a5e7c66104dd838033e0827a1850f250ac85"
-SRC_URI[sha384sum] = "2a2f177e5b68f6f5d13b253a8d4daba9e956b4a79a40a4775aee288b60c612fe13b417d6baf77944c32b1181b2136f68"
-SRC_URI[sha512sum] = "08266a2e9a7ca5a37e03f340873571fd012c630dd898a87241cd97c29760b2bf41fe64c8883e7b08e0f83c26245072cd26e69ecb1db02378d1ec38335ba1cd0f"
+SRC_URI[sha256sum] = "5dccdaff2ebe18d435b32bfc06f8b5f474bf6ac0432a6a07d144b7c56700d0bf"

DEPENDS = "boost bzip2 curl expat gperf-native \
keyutils libaio libibverbs lz4 \
@@ -133,6 +129,7 @@ RDEPENDS:${PN} += "\
python3-modules \
python3-prettytable \
${PN}-python \
+ gawk \
"
COMPATIBLE_HOST = "(x86_64).*"
PACKAGES += " \
--
2.17.1


Re: [PATCH 1/2] k3s: Update k3s service configuration

Kamil Dziezyk
 

Hello Bruce,

Can you backport these two k3s patches to honister branch?

Thanks,

Kamil


Re: [PATCH 1/2] k3s: Update k3s service configuration

Bruce Ashfield
 

Thanks for the tweaks, they look good to me.

I have some other pending k3s changes I'm working on, but there was no
reason to hold these up, so I've pulled them into master.

Bruce

On Tue, Oct 19, 2021 at 11:13 AM Kamil Dziezyk <kamil.dziezyk@...> wrote:

Update k3s.service with the latest changes from install.sh script.
Add k3s-killall.sh script to stop all of the K3s containers and reset
the containerd state.

The killall script cleans up containers, K3s directories, and networking
components while also removing the iptables chain with all the associated
rules. The cluster data will not be deleted.

Signed-off-by: Kamil Dziezyk <kamil.dziezyk@...>
Change-Id: If1794367cabfc18fc8e3ecaf26badd4d0bc25114
---
recipes-containers/k3s/k3s/k3s-killall.sh | 82 +++++++++++++++++++++++
recipes-containers/k3s/k3s/k3s.service | 12 +++-
recipes-containers/k3s/k3s_git.bb | 2 +
3 files changed, 95 insertions(+), 1 deletion(-)
create mode 100644 recipes-containers/k3s/k3s/k3s-killall.sh

diff --git a/recipes-containers/k3s/k3s/k3s-killall.sh b/recipes-containers/k3s/k3s/k3s-killall.sh
new file mode 100644
index 0000000..9e72615
--- /dev/null
+++ b/recipes-containers/k3s/k3s/k3s-killall.sh
@@ -0,0 +1,82 @@
+#!/bin/sh
+
+# Based on: k3s-killall.sh installed when running Rancher Lab's K3S install.sh
+# In open-source project: https://github.com/k3s-io/k3s
+#
+# Original file: Copyright (c) 2021 Rancher Labs and Contributors.
+# Modifications: Copyright (c) 2021 Arm Limited and Contributors. All rights reserved.
+#
+# Modifications:
+# - Change systemd service directory location
+# - Fix PID parsing to run on core image
+# - Remove service stopping code (as this is intended to run as part of service
+# stop)
+# - Changes to resolve warnings from the ShellCheck static analysis tool
+#
+# SPDX-License-Identifier: Apache License 2.0
+
+[ "$(id -u)" -eq 0 ] || exec sudo "$0" "$@"
+
+for bin in /var/lib/rancher/k3s/data/**/bin/; do
+ [ -d "$bin" ] && export PATH=$PATH:$bin:$bin/aux
+done
+
+set -x
+
+pschildren() {
+ ps -e -o ppid= -o pid= | sed -e 's/^\s*//g; s/\s\s*/\t/g;' | grep -w "^$1" | cut -f2
+}
+
+pstree() {
+ for pid in "$@"; do
+ echo "$pid"
+ for child in $(pschildren "$pid"); do
+ pstree "$child"
+ done
+ done
+}
+
+killtree() {
+ while read -r pid; do
+ if [ -n "${pid}" ]; then
+ kill -9 "${pid}" 2>/dev/null
+ fi
+ done <<EOF
+$({ set +x; } 2>/dev/null; pstree "$@"; set -x;)
+EOF
+}
+
+getshims() {
+ ps -e -o pid= -o args= | sed -e 's/^ *//; s/\s\s*/\t/;' | grep -w '[^/]*/bin/containerd-shim' | cut -f1
+}
+
+killtree "$({ set +x; } 2>/dev/null; getshims; set -x)"
+
+# shellcheck disable=SC2016
+do_unmount_and_remove() {
+ set +x
+ while read -r _ path _; do
+ case "$path" in $1*) echo "$path" ;; esac
+ done < /proc/self/mounts | sort -r | xargs -r -t -n 1 sh -c 'umount "$0" && rm -rf "$0"'
+ set -x
+}
+
+do_unmount_and_remove '/run/k3s'
+do_unmount_and_remove '/var/lib/rancher/k3s'
+do_unmount_and_remove '/var/lib/kubelet/pods'
+do_unmount_and_remove '/var/lib/kubelet/plugins'
+do_unmount_and_remove '/run/netns/cni-'
+
+# Remove CNI namespaces
+ip netns show 2>/dev/null | grep cni- | xargs -r -t -n 1 ip netns delete
+
+# Delete network interface(s) that match 'master cni0'
+ip link show 2>/dev/null | grep 'master cni0' | while read -r _ iface _; do
+ iface=${iface%%@*}
+ [ -z "$iface" ] || ip link delete "$iface"
+done
+ip link delete cni0
+ip link delete flannel.1
+ip link delete flannel-v6.1
+rm -rf /var/lib/cni/
+iptables-save | grep -v KUBE- | grep -v CNI- | iptables-restore
diff --git a/recipes-containers/k3s/k3s/k3s.service b/recipes-containers/k3s/k3s/k3s.service
index 34c7a80..33d3ee7 100644
--- a/recipes-containers/k3s/k3s/k3s.service
+++ b/recipes-containers/k3s/k3s/k3s.service
@@ -4,12 +4,17 @@ Description=Lightweight Kubernetes
Documentation=https://k3s.io
Requires=containerd.service
After=containerd.service
+After=network-online.target
+Wants=network-online.target

[Install]
WantedBy=multi-user.target

[Service]
Type=notify
+EnvironmentFile=-/etc/default/%N
+EnvironmentFile=-/etc/sysconfig/%N
+EnvironmentFile=-/etc/systemd/system/k3s.service.env
KillMode=process
Delegate=yes
# Having non-zero Limit*s causes performance problems due to accounting overhead
@@ -21,7 +26,12 @@ TasksMax=infinity
TimeoutStartSec=0
Restart=always
RestartSec=5s
+ExecStartPre=/bin/sh -xc '! systemctl is-enabled --quiet nm-cloud-setup.service'
ExecStartPre=-/sbin/modprobe br_netfilter
ExecStartPre=-/sbin/modprobe overlay
ExecStart=/usr/local/bin/k3s server
-
+# Avoid any delay due to this service when the system is rebooting or shutting
+# down by using the k3s-killall.sh script to kill all of the running k3s
+# services and containers
+ExecStopPost=/bin/sh -c "if systemctl is-system-running | grep -i \
+ 'stopping'; then /usr/local/bin/k3s-killall.sh; fi"
diff --git a/recipes-containers/k3s/k3s_git.bb b/recipes-containers/k3s/k3s_git.bb
index 5971dde..0300fc3 100644
--- a/recipes-containers/k3s/k3s_git.bb
+++ b/recipes-containers/k3s/k3s_git.bb
@@ -11,6 +11,7 @@ SRC_URI = "git://github.com/rancher/k3s.git;branch=release-1.22;name=k3s \
file://k3s-clean \
file://cni-containerd-net.conf \
file://0001-Finding-host-local-in-usr-libexec.patch;patchdir=src/import \
+ file://k3s-killall.sh \
"
SRC_URI[k3s.md5sum] = "363d3a08dc0b72ba6e6577964f6e94a5"
SRCREV_k3s = "737f722315b9832e9180fa232253d28ae1f2272f"
@@ -60,6 +61,7 @@ do_install() {
# ln -sr "${D}/${BIN_PREFIX}/bin/k3s" "${D}${BIN_PREFIX}/bin/ctr"
ln -sr "${D}/${BIN_PREFIX}/bin/k3s" "${D}${BIN_PREFIX}/bin/kubectl"
install -m 755 "${WORKDIR}/k3s-clean" "${D}${BIN_PREFIX}/bin"
+ install -m 755 "${WORKDIR}/k3s-killall.sh" "${D}${BIN_PREFIX}/bin"

if ${@bb.utils.contains('DISTRO_FEATURES','systemd','true','false',d)}; then
install -D -m 0644 "${WORKDIR}/k3s.service" "${D}${systemd_system_unitdir}/k3s.service"
--
2.17.1



--
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II


Re: [PATCH] rename git branch to "main"

Bruce Ashfield
 

Thanks for the patch!

i have the same changes locally, as I ran into this today. I've
queued it for master and the supported release branches. I expect to
push it shortly, once I've sorted out another set of changes.

Bruce

On Thu, Oct 21, 2021 at 6:08 PM Nicolas Adell <nicolas.nadell@...> wrote:

From 6193c0b2415073d2bba106f013ac1ddd3c1ecf4c Mon Sep 17 00:00:00 2001
From: Nicolas Adell <nicolas.nadell@...>
Date: Thu, 21 Oct 2021 23:49:47 +0200
Subject: [PATCH] umoci: rename git branch to "main"

The upstream project switched from master to main, so we align
to avoid fetch errors (as the default of master no longer works).

Signed-off-by: Nicolas Adell <nicolas.nadell@...>
---
recipes-containers/umoci/umoci_git.bb | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/recipes-containers/umoci/umoci_git.bb
b/recipes-containers/umoci/umoci_git.bb
index 69325d1..3c8b2fb 100644
--- a/recipes-containers/umoci/umoci_git.bb
+++ b/recipes-containers/umoci/umoci_git.bb
@@ -7,7 +7,7 @@ RDEPENDS:${PN} = "skopeo"
RDEPENDS:${PN}:class-native = ""

SRCREV_umoci = "758044fc26ad65eb900143e90d1e22c2d6e4484d"
-SRC_URI = "git://github.com/opencontainers/umoci.git;branch=master;name=umoci;destsuffix=github.com/opencontainers/umoci
\
+SRC_URI = "git://github.com/opencontainers/umoci.git;branch=main;name=umoci;destsuffix=github.com/opencontainers/umoci
\
"

PV = "v0.4.7-dev+git${SRCPV}"
--
2.32.0
--
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II


[PATCH] rename git branch to "main"

nicolas.nadell@...
 

From 6193c0b2415073d2bba106f013ac1ddd3c1ecf4c Mon Sep 17 00:00:00 2001
From: Nicolas Adell <nicolas.nadell@...>
Date: Thu, 21 Oct 2021 23:49:47 +0200
Subject: [PATCH] umoci: rename git branch to "main"

The upstream project switched from master to main, so we rename
to avoid fetch errors (as the default of master no longer works).

Signed-off-by: Nicolas Adell <nicolas.nadell@...>
---
recipes-containers/umoci/umoci_git.bb | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/recipes-containers/umoci/umoci_git.bb
b/recipes-containers/umoci/umoci_git.bb
index 69325d1..3c8b2fb 100644
--- a/recipes-containers/umoci/umoci_git.bb
+++ b/recipes-containers/umoci/umoci_git.bb
@@ -7,7 +7,7 @@ RDEPENDS:${PN} = "skopeo"
RDEPENDS:${PN}:class-native = ""

SRCREV_umoci = "758044fc26ad65eb900143e90d1e22c2d6e4484d"
-SRC_URI = "git://github.com/opencontainers/umoci.git;branch=master;name=umoci;destsuffix=github.com/opencontainers/umoci
\
+SRC_URI = "git://github.com/opencontainers/umoci.git;branch=main;name=umoci;destsuffix=github.com/opencontainers/umoci
\
"

PV = "v0.4.7-dev+git${SRCPV}"
--
2.32.0


[PATCH 2/2] k3s: Add xt-statistic kernel module to rrecommends

Kamil Dziezyk
 

Include the xt-statistic kernel module required by K3S for iptables
configuration.

Signed-off-by: Kamil Dziezyk <kamil.dziezyk@...>
Change-Id: I7592261c65c7c0831ce553ee907fba9e3e458b6f
---
recipes-containers/k3s/k3s_git.bb | 1 +
1 file changed, 1 insertion(+)

diff --git a/recipes-containers/k3s/k3s_git.bb b/recipes-containers/k3s/k3s_git.bb
index 0300fc3..e341ad7 100644
--- a/recipes-containers/k3s/k3s_git.bb
+++ b/recipes-containers/k3s/k3s_git.bb
@@ -95,6 +95,7 @@ RRECOMMENDS:${PN} = "\
kernel-module-xt-connmark \
kernel-module-vxlan \
kernel-module-xt-masquerade \
+ kernel-module-xt-statistic \
"

RCONFLICTS:${PN} = "kubectl"
--
2.17.1


[PATCH 1/2] k3s: Update k3s service configuration

Kamil Dziezyk
 

Update k3s.service with the latest changes from install.sh script.
Add k3s-killall.sh script to stop all of the K3s containers and reset
the containerd state.

The killall script cleans up containers, K3s directories, and networking
components while also removing the iptables chain with all the associated
rules. The cluster data will not be deleted.

Signed-off-by: Kamil Dziezyk <kamil.dziezyk@...>
Change-Id: If1794367cabfc18fc8e3ecaf26badd4d0bc25114
---
recipes-containers/k3s/k3s/k3s-killall.sh | 82 +++++++++++++++++++++++
recipes-containers/k3s/k3s/k3s.service | 12 +++-
recipes-containers/k3s/k3s_git.bb | 2 +
3 files changed, 95 insertions(+), 1 deletion(-)
create mode 100644 recipes-containers/k3s/k3s/k3s-killall.sh

diff --git a/recipes-containers/k3s/k3s/k3s-killall.sh b/recipes-containers/k3s/k3s/k3s-killall.sh
new file mode 100644
index 0000000..9e72615
--- /dev/null
+++ b/recipes-containers/k3s/k3s/k3s-killall.sh
@@ -0,0 +1,82 @@
+#!/bin/sh
+
+# Based on: k3s-killall.sh installed when running Rancher Lab's K3S install.sh
+# In open-source project: https://github.com/k3s-io/k3s
+#
+# Original file: Copyright (c) 2021 Rancher Labs and Contributors.
+# Modifications: Copyright (c) 2021 Arm Limited and Contributors. All rights reserved.
+#
+# Modifications:
+# - Change systemd service directory location
+# - Fix PID parsing to run on core image
+# - Remove service stopping code (as this is intended to run as part of service
+# stop)
+# - Changes to resolve warnings from the ShellCheck static analysis tool
+#
+# SPDX-License-Identifier: Apache License 2.0
+
+[ "$(id -u)" -eq 0 ] || exec sudo "$0" "$@"
+
+for bin in /var/lib/rancher/k3s/data/**/bin/; do
+ [ -d "$bin" ] && export PATH=$PATH:$bin:$bin/aux
+done
+
+set -x
+
+pschildren() {
+ ps -e -o ppid= -o pid= | sed -e 's/^\s*//g; s/\s\s*/\t/g;' | grep -w "^$1" | cut -f2
+}
+
+pstree() {
+ for pid in "$@"; do
+ echo "$pid"
+ for child in $(pschildren "$pid"); do
+ pstree "$child"
+ done
+ done
+}
+
+killtree() {
+ while read -r pid; do
+ if [ -n "${pid}" ]; then
+ kill -9 "${pid}" 2>/dev/null
+ fi
+ done <<EOF
+$({ set +x; } 2>/dev/null; pstree "$@"; set -x;)
+EOF
+}
+
+getshims() {
+ ps -e -o pid= -o args= | sed -e 's/^ *//; s/\s\s*/\t/;' | grep -w '[^/]*/bin/containerd-shim' | cut -f1
+}
+
+killtree "$({ set +x; } 2>/dev/null; getshims; set -x)"
+
+# shellcheck disable=SC2016
+do_unmount_and_remove() {
+ set +x
+ while read -r _ path _; do
+ case "$path" in $1*) echo "$path" ;; esac
+ done < /proc/self/mounts | sort -r | xargs -r -t -n 1 sh -c 'umount "$0" && rm -rf "$0"'
+ set -x
+}
+
+do_unmount_and_remove '/run/k3s'
+do_unmount_and_remove '/var/lib/rancher/k3s'
+do_unmount_and_remove '/var/lib/kubelet/pods'
+do_unmount_and_remove '/var/lib/kubelet/plugins'
+do_unmount_and_remove '/run/netns/cni-'
+
+# Remove CNI namespaces
+ip netns show 2>/dev/null | grep cni- | xargs -r -t -n 1 ip netns delete
+
+# Delete network interface(s) that match 'master cni0'
+ip link show 2>/dev/null | grep 'master cni0' | while read -r _ iface _; do
+ iface=${iface%%@*}
+ [ -z "$iface" ] || ip link delete "$iface"
+done
+ip link delete cni0
+ip link delete flannel.1
+ip link delete flannel-v6.1
+rm -rf /var/lib/cni/
+iptables-save | grep -v KUBE- | grep -v CNI- | iptables-restore
diff --git a/recipes-containers/k3s/k3s/k3s.service b/recipes-containers/k3s/k3s/k3s.service
index 34c7a80..33d3ee7 100644
--- a/recipes-containers/k3s/k3s/k3s.service
+++ b/recipes-containers/k3s/k3s/k3s.service
@@ -4,12 +4,17 @@ Description=Lightweight Kubernetes
Documentation=https://k3s.io
Requires=containerd.service
After=containerd.service
+After=network-online.target
+Wants=network-online.target

[Install]
WantedBy=multi-user.target

[Service]
Type=notify
+EnvironmentFile=-/etc/default/%N
+EnvironmentFile=-/etc/sysconfig/%N
+EnvironmentFile=-/etc/systemd/system/k3s.service.env
KillMode=process
Delegate=yes
# Having non-zero Limit*s causes performance problems due to accounting overhead
@@ -21,7 +26,12 @@ TasksMax=infinity
TimeoutStartSec=0
Restart=always
RestartSec=5s
+ExecStartPre=/bin/sh -xc '! systemctl is-enabled --quiet nm-cloud-setup.service'
ExecStartPre=-/sbin/modprobe br_netfilter
ExecStartPre=-/sbin/modprobe overlay
ExecStart=/usr/local/bin/k3s server
-
+# Avoid any delay due to this service when the system is rebooting or shutting
+# down by using the k3s-killall.sh script to kill all of the running k3s
+# services and containers
+ExecStopPost=/bin/sh -c "if systemctl is-system-running | grep -i \
+ 'stopping'; then /usr/local/bin/k3s-killall.sh; fi"
diff --git a/recipes-containers/k3s/k3s_git.bb b/recipes-containers/k3s/k3s_git.bb
index 5971dde..0300fc3 100644
--- a/recipes-containers/k3s/k3s_git.bb
+++ b/recipes-containers/k3s/k3s_git.bb
@@ -11,6 +11,7 @@ SRC_URI = "git://github.com/rancher/k3s.git;branch=release-1.22;name=k3s \
file://k3s-clean \
file://cni-containerd-net.conf \
file://0001-Finding-host-local-in-usr-libexec.patch;patchdir=src/import \
+ file://k3s-killall.sh \
"
SRC_URI[k3s.md5sum] = "363d3a08dc0b72ba6e6577964f6e94a5"
SRCREV_k3s = "737f722315b9832e9180fa232253d28ae1f2272f"
@@ -60,6 +61,7 @@ do_install() {
# ln -sr "${D}/${BIN_PREFIX}/bin/k3s" "${D}${BIN_PREFIX}/bin/ctr"
ln -sr "${D}/${BIN_PREFIX}/bin/k3s" "${D}${BIN_PREFIX}/bin/kubectl"
install -m 755 "${WORKDIR}/k3s-clean" "${D}${BIN_PREFIX}/bin"
+ install -m 755 "${WORKDIR}/k3s-killall.sh" "${D}${BIN_PREFIX}/bin"

if ${@bb.utils.contains('DISTRO_FEATURES','systemd','true','false',d)}; then
install -D -m 0644 "${WORKDIR}/k3s.service" "${D}${systemd_system_unitdir}/k3s.service"
--
2.17.1


Re: Issues with Setscene tasks

Bruce Ashfield
 

On Tue, Oct 19, 2021 at 6:51 AM Diego Sueiro <Diego.Sueiro@...> wrote:

Hi Bruce,

This is the current fix that I have for docker-moby and cni recipes and I think it is aligned with the patch that you linked since you are also setting the SRCREV_FORMAT:
Indeed.

I'll take care of the changes, since I can run tests as I go.

What branches are you using ? I can backport to wherever it makes sense.

Bruce

```
diff --git a/recipes-containers/docker/docker-moby_git.bb b/recipes-containers/docker/docker-moby_git.bb
index defb321..f2ccfcd 100644
--- a/recipes-containers/docker/docker-moby_git.bb
+++ b/recipes-containers/docker/docker-moby_git.bb
@@ -47,6 +47,8 @@ SRC_URI = "\
file://0001-dynbinary-use-go-cross-compiler.patch \
"

+SRCRV_FORMAT = "moby"
+
require docker.inc

# Apache-2.0 for docker
diff --git a/recipes-networking/cni/cni_git.bb b/recipes-networking/cni/cni_git.bb
index cb8d6ff..385d170 100644
--- a/recipes-networking/cni/cni_git.bb
+++ b/recipes-networking/cni/cni_git.bb
@@ -24,6 +24,8 @@ LIC_FILES_CHKSUM = "file://src/import/LICENSE;md5=fa818a259cbed7ce8bc2a22d35a464

GO_IMPORT = "import"

+SRCREV_FORMAT = "cni"
+
PV = "v0.8.0+git${SRCREV_cni}"

inherit go
```

________________________________
From: Bruce Ashfield <bruce.ashfield@...>
Sent: 18 October 2021 20:44
To: Diego Sueiro <Diego.Sueiro@...>
Cc: meta-virtualization@... <meta-virtualization@...>; nd <nd@...>
Subject: Re: [meta-virtualization] Issues with Setscene tasks

On Mon, Oct 18, 2021 at 3:30 PM Diego Sueiro <diego.sueiro@...> wrote:

Hi,



Recently I realized that we are having issues with setscene tasks (for docker-moby and cni recipes) where the packages were rebuild even already have their tasks in the sstate mirror and bitbake-diffsigs not showing any difference.
This is the sample of the output that I was getting:
```

WARNING: Logfile for failed setscene task is /<snip>/build/tmp/work/neoversen1-poky-linux/cni/v0.8.0+gitb5ab16f010e822936eb974690ecec38ba69afc01-r0/temp/log.do_package_qa_setscene.3335

WARNING: Setscene task (/<snip>/layers/meta-virtualization/recipes-networking/cni/cni_git.bb:do_package_qa_setscene) failed with exit code '1' - real task will be run instead

WARNING: Logfile for failed setscene task is /<snip>/build/tmp/work/neoversen1-poky-linux/cni/v0.8.0+gitb5ab16f010e822936eb974690ecec38ba69afc01-0/temp/log.do_package_write_rpm_setscene.3336

WARNING: Setscene task (/<snip>/layers/meta-virtualization/recipes-networking/cni/cni_git.bb:do_package_write_rpm_setscene) failed with exit code '1' - real task will be run instead

WARNING: Logfile for failed setscene task is /<snip>/build/tmp/work/neoversen1-poky-linux/cni/v0.8.0+gitb5ab16f010e822936eb974690ecec38ba69afc01-r0/temp/log.do_populate_lic_setscene.3337

WARNING: Setscene task (/<snip>/layers/meta-virtualization/recipes-networking/cni/cni_git.bb:do_populate_lic_setscene) failed with exit code '1' - real task will be run instead

WARNING: Logfile for failed setscene task is /<snip>/build/tmp/work/neoversen1-poky-linux/cni/v0.8.0+gitb5ab16f010e822936eb974690ecec38ba69afc01-r0/temp/log.do_populate_sysroot_setscene.3338

WARNING: Setscene task (/<snip>/layers/meta-virtualization/recipes-networking/cni/cni_git.bb:do_populate_sysroot_setscene) failed with exit code '1' - real task will be run instead

WARNING: Logfile for failed setscene task is /<snip>/build/tmp/work/neoversen1-poky-linux/cni/v0.8.0+gitb5ab16f010e822936eb974690ecec38ba69afc01-r0/temp/log.do_package_qa_setscene.3339

WARNING: Setscene task (/<snip>/layers/meta-virtualization/recipes-networking/cni/cni_git.bb:do_packagedata_setscene) failed with exit code '1' - real task will be run instead

WARNING: Logfile for failed setscene task is /<snip>/build/tmp/work/neoversen1-poky-linux/cni/v0.8.0+gitb5ab16f010e822936eb974690ecec38ba69afc01-r0/temp/log.do_package_setscene.3380

WARNING: Setscene task (/<snip>/layers/meta-virtualization/recipes-networking/cni/cni_git.bb:do_package_setscene) failed with exit code '1' - real task will be run instead

WARNING: Logfile for failed setscene task is /<snip>/build/tmp/work/neoversen1-poky-linux/cni/v0.8.0+gitb5ab16f010e822936eb974690ecec38ba69afc01-r0/temp/log.do_deploy_source_date_epoch_setscene.3383

WARNING: Setscene task (/<snip>/layers/meta-virtualization/recipes-networking/cni/cni_git.bb:do_deploy_source_date_epoch_setscene) failed with exit code '1' - real task will be run instead

```



After some debugging I found that the “failed with exit code 1” comes from this line http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/classes/sstate.bbclass#n764:
```

localdata.setVar('SRCPV', d.getVar('SRCPV'))
```


The root cause is because the cni and docker-moby recipes are using named SRC_URI (a.k.a http://...;name=foo), setting the PV with it and were not setting the SRCREV_FORMAT which consequentially not set the SRCPV accordingly.



Setting the SRCREV_FORMAT for cni and docker-moby recipes fixed the issue, but I also suspect that the following recipes are potentially having the same issue:

```
$ grep -r PV | grep SRCREV_

recipes-core/runx/runx_git.bb:PV = "v1.0-git${SRCREV_runx}"

recipes-core/runv/runv_git.bb:PV = "0.4.0+git${SRCREV_runv}"

recipes-containers/docker/docker-ce_git.bb:PV = "${DOCKER_VERSION}+git${SRCREV_docker}"

recipes-containers/docker/docker-moby_git.bb:PV = "${DOCKER_VERSION}+git${SRCREV_moby}"

recipes-containers/cri-o/cri-o_git.bb:PV = "1.22.0+git${SRCREV_cri-o}"

recipes-containers/kubernetes/kubernetes_git.bb:PV = "v1.22.2+git${SRCREV_kubernetes}"

recipes-containers/cri-tools/cri-tools_git.bb:PV = "1.21.0+git${SRCREV_cri-tools}"

recipes-containers/crun/crun_git.bb:PV = "0.18+git${SRCREV_crun}"

recipes-networking/cni/cni_git.bb:PV = "v0.8.0+git${SRCREV_cni}"

recipes-devtools/yq/yq_git.bb:PV = "1.13.1+git${SRCREV_yq}"

recipes-devtools/go/go-build_git.bb:PV = "0.1-git${SRCREV_runx}"

```



I can send a patch to fix cni and docker-moby but unfortunately I don’t have resources to test the other mentioned recipes.
I've actually been going through and fixing these as they pop up, and
just had to do it for kubernetes:

https://git.yoctoproject.org/cgit/cgit.cgi/meta-virtualization/commit/?h=master-next&id=45e5f2e821784f0b55a41ee5f08d7acb98f8f288

If you were doing anything other than what you see that I did in that
recipe, let me know. Otherwise, I'll make a similar change globally to
the layer.

Bruce



--
Diego Sueiro
Staff Software Developer – Automotive and Industrial Solutions (CE-OSS)






--
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II


--
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II


Re: Issues with Setscene tasks

Diego Sueiro
 

Hi Bruce,

This is the current fix that I have for docker-moby and cni recipes and I think it is aligned with the patch that you linked since you are also setting the SRCREV_FORMAT:
```
diff --git a/recipes-containers/docker/docker-moby_git.bb b/recipes-containers/docker/docker-moby_git.bb
index defb321..f2ccfcd 100644
--- a/recipes-containers/docker/docker-moby_git.bb
+++ b/recipes-containers/docker/docker-moby_git.bb
@@ -47,6 +47,8 @@ SRC_URI = "\
         file://0001-dynbinary-use-go-cross-compiler.patch \
        "
 
+SRCRV_FORMAT = "moby"
+
 require docker.inc
 
 # Apache-2.0 for docker
diff --git a/recipes-networking/cni/cni_git.bb b/recipes-networking/cni/cni_git.bb
index cb8d6ff..385d170 100644
--- a/recipes-networking/cni/cni_git.bb
+++ b/recipes-networking/cni/cni_git.bb
@@ -24,6 +24,8 @@ LIC_FILES_CHKSUM = "file://src/import/LICENSE;md5=fa818a259cbed7ce8bc2a22d35a464
 
 GO_IMPORT = "import"
 
+SRCREV_FORMAT = "cni"
+
 PV = "v0.8.0+git${SRCREV_cni}"
 
 inherit go
```


From: Bruce Ashfield <bruce.ashfield@...>
Sent: 18 October 2021 20:44
To: Diego Sueiro <Diego.Sueiro@...>
Cc: meta-virtualization@... <meta-virtualization@...>; nd <nd@...>
Subject: Re: [meta-virtualization] Issues with Setscene tasks
 
On Mon, Oct 18, 2021 at 3:30 PM Diego Sueiro <diego.sueiro@...> wrote:
>
> Hi,
>
>
>
> Recently I realized that we are having issues with setscene tasks (for docker-moby and cni recipes) where the packages were rebuild even already have their tasks in the sstate mirror and bitbake-diffsigs not showing any difference.
> This is the sample of the output that I was getting:
> ```
>
> WARNING: Logfile for failed setscene task is /<snip>/build/tmp/work/neoversen1-poky-linux/cni/v0.8.0+gitb5ab16f010e822936eb974690ecec38ba69afc01-r0/temp/log.do_package_qa_setscene.3335
>
> WARNING: Setscene task (/<snip>/layers/meta-virtualization/recipes-networking/cni/cni_git.bb:do_package_qa_setscene) failed with exit code '1' - real task will be run instead
>
> WARNING: Logfile for failed setscene task is /<snip>/build/tmp/work/neoversen1-poky-linux/cni/v0.8.0+gitb5ab16f010e822936eb974690ecec38ba69afc01-0/temp/log.do_package_write_rpm_setscene.3336
>
> WARNING: Setscene task (/<snip>/layers/meta-virtualization/recipes-networking/cni/cni_git.bb:do_package_write_rpm_setscene) failed with exit code '1' - real task will be run instead
>
> WARNING: Logfile for failed setscene task is /<snip>/build/tmp/work/neoversen1-poky-linux/cni/v0.8.0+gitb5ab16f010e822936eb974690ecec38ba69afc01-r0/temp/log.do_populate_lic_setscene.3337
>
> WARNING: Setscene task (/<snip>/layers/meta-virtualization/recipes-networking/cni/cni_git.bb:do_populate_lic_setscene) failed with exit code '1' - real task will be run instead
>
> WARNING: Logfile for failed setscene task is /<snip>/build/tmp/work/neoversen1-poky-linux/cni/v0.8.0+gitb5ab16f010e822936eb974690ecec38ba69afc01-r0/temp/log.do_populate_sysroot_setscene.3338
>
> WARNING: Setscene task (/<snip>/layers/meta-virtualization/recipes-networking/cni/cni_git.bb:do_populate_sysroot_setscene) failed with exit code '1' - real task will be run instead
>
> WARNING: Logfile for failed setscene task is /<snip>/build/tmp/work/neoversen1-poky-linux/cni/v0.8.0+gitb5ab16f010e822936eb974690ecec38ba69afc01-r0/temp/log.do_package_qa_setscene.3339
>
> WARNING: Setscene task (/<snip>/layers/meta-virtualization/recipes-networking/cni/cni_git.bb:do_packagedata_setscene) failed with exit code '1' - real task will be run instead
>
> WARNING: Logfile for failed setscene task is /<snip>/build/tmp/work/neoversen1-poky-linux/cni/v0.8.0+gitb5ab16f010e822936eb974690ecec38ba69afc01-r0/temp/log.do_package_setscene.3380
>
> WARNING: Setscene task (/<snip>/layers/meta-virtualization/recipes-networking/cni/cni_git.bb:do_package_setscene) failed with exit code '1' - real task will be run instead
>
> WARNING: Logfile for failed setscene task is /<snip>/build/tmp/work/neoversen1-poky-linux/cni/v0.8.0+gitb5ab16f010e822936eb974690ecec38ba69afc01-r0/temp/log.do_deploy_source_date_epoch_setscene.3383
>
> WARNING: Setscene task (/<snip>/layers/meta-virtualization/recipes-networking/cni/cni_git.bb:do_deploy_source_date_epoch_setscene) failed with exit code '1' - real task will be run instead
>
> ```
>
>
>
> After some debugging I found that the “failed with exit code 1” comes from this line http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/classes/sstate.bbclass#n764:
> ```
>
> localdata.setVar('SRCPV', d.getVar('SRCPV'))
> ```
>
>
> The root cause is because the cni and docker-moby recipes are using named SRC_URI (a.k.a http://...;name=foo), setting the PV with it and were not setting the SRCREV_FORMAT which consequentially not set the SRCPV accordingly.
>
>
>
> Setting the SRCREV_FORMAT for cni and docker-moby recipes fixed the issue, but I also suspect that the following recipes are potentially having the same issue:
>
> ```
> $ grep -r PV | grep SRCREV_
>
> recipes-core/runx/runx_git.bb:PV = "v1.0-git${SRCREV_runx}"
>
> recipes-core/runv/runv_git.bb:PV = "0.4.0+git${SRCREV_runv}"
>
> recipes-containers/docker/docker-ce_git.bb:PV = "${DOCKER_VERSION}+git${SRCREV_docker}"
>
> recipes-containers/docker/docker-moby_git.bb:PV = "${DOCKER_VERSION}+git${SRCREV_moby}"
>
> recipes-containers/cri-o/cri-o_git.bb:PV = "1.22.0+git${SRCREV_cri-o}"
>
> recipes-containers/kubernetes/kubernetes_git.bb:PV = "v1.22.2+git${SRCREV_kubernetes}"
>
> recipes-containers/cri-tools/cri-tools_git.bb:PV = "1.21.0+git${SRCREV_cri-tools}"
>
> recipes-containers/crun/crun_git.bb:PV = "0.18+git${SRCREV_crun}"
>
> recipes-networking/cni/cni_git.bb:PV = "v0.8.0+git${SRCREV_cni}"
>
> recipes-devtools/yq/yq_git.bb:PV = "1.13.1+git${SRCREV_yq}"
>
> recipes-devtools/go/go-build_git.bb:PV = "0.1-git${SRCREV_runx}"
>
> ```
>
>
>
> I can send a patch to fix cni and docker-moby but unfortunately I don’t have resources to test the other mentioned recipes.
>

I've actually been going through and fixing these as they pop up, and
just had to do it for kubernetes:

https://git.yoctoproject.org/cgit/cgit.cgi/meta-virtualization/commit/?h=master-next&id=45e5f2e821784f0b55a41ee5f08d7acb98f8f288

If you were doing anything other than what you see that I did in that
recipe, let me know. Otherwise, I'll make a similar change globally to
the layer.

Bruce

>
>
> --
> Diego Sueiro
> Staff Software Developer – Automotive and Industrial Solutions (CE-OSS)
>
>
>
>
>
>


--
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II


Re: [PATCH v2] ceph: inherit pkgconfig.bbclass

kai
 

On 10/19/21 11:04 AM, Bruce Ashfield wrote:
On Mon, Oct 18, 2021 at 10:53 PM kai <kai.kang@...> wrote:
From: Kai Kang <kai.kang@...>

According to oe-core commit

8e26252b45 layer.conf: Extend recipes not to install without explict dependencies

pkgconfig-native is not deployed in sysroot by default any more. Inherit
pkgconfig.bbclass to make it available for ceph.

Signed-off-by: Kai Kang <kai.kang@...>
---
v2:
* correct referenced commit
v2 is staged on master-next.

If this is needed anywhere else than master (i.e. honister), let me
know ... It wouldn't break anything on honister, but I'm trying to
keep the changes there to a minimum.
It is only required by master that the oe-core commit only exists in oe-core master and next release(kirkstone) branches:

$ git br -r --contains 8e26252b45

  origin/HEAD -> origin/master
  origin/kirkstone-next
  origin/master
  origin/master-next

Regards,
Kai


Cheers,

Bruce

recipes-extended/ceph/ceph_15.2.12.bb | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/recipes-extended/ceph/ceph_15.2.12.bb b/recipes-extended/ceph/ceph_15.2.12.bb
index 6636d7a..693b525 100644
--- a/recipes-extended/ceph/ceph_15.2.12.bb
+++ b/recipes-extended/ceph/ceph_15.2.12.bb
@@ -4,7 +4,7 @@ LIC_FILES_CHKSUM = "file://COPYING-LGPL2.1;md5=fbc093901857fcd118f065f900982c24
file://COPYING-GPL2;md5=b234ee4d69f5fce4486a80fdaf4a4263 \
file://COPYING;md5=4eb012c221c5fd4b760029a2981a6754 \
"
-inherit cmake python3native python3-dir systemd
+inherit cmake pkgconfig python3native python3-dir systemd
# Disable python pybind support for ceph temporary, when corss compiling pybind,
# pybind mix cmake and python setup environment, would case a lot of errors.

--
2.17.1


--
Kai Kang
Wind River Linux


Re: [PATCH v2] ceph: inherit pkgconfig.bbclass

Bruce Ashfield
 

On Mon, Oct 18, 2021 at 10:53 PM kai <kai.kang@...> wrote:

From: Kai Kang <kai.kang@...>

According to oe-core commit

8e26252b45 layer.conf: Extend recipes not to install without explict dependencies

pkgconfig-native is not deployed in sysroot by default any more. Inherit
pkgconfig.bbclass to make it available for ceph.

Signed-off-by: Kai Kang <kai.kang@...>
---
v2:
* correct referenced commit
v2 is staged on master-next.

If this is needed anywhere else than master (i.e. honister), let me
know ... It wouldn't break anything on honister, but I'm trying to
keep the changes there to a minimum.

Cheers,

Bruce


recipes-extended/ceph/ceph_15.2.12.bb | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/recipes-extended/ceph/ceph_15.2.12.bb b/recipes-extended/ceph/ceph_15.2.12.bb
index 6636d7a..693b525 100644
--- a/recipes-extended/ceph/ceph_15.2.12.bb
+++ b/recipes-extended/ceph/ceph_15.2.12.bb
@@ -4,7 +4,7 @@ LIC_FILES_CHKSUM = "file://COPYING-LGPL2.1;md5=fbc093901857fcd118f065f900982c24
file://COPYING-GPL2;md5=b234ee4d69f5fce4486a80fdaf4a4263 \
file://COPYING;md5=4eb012c221c5fd4b760029a2981a6754 \
"
-inherit cmake python3native python3-dir systemd
+inherit cmake pkgconfig python3native python3-dir systemd
# Disable python pybind support for ceph temporary, when corss compiling pybind,
# pybind mix cmake and python setup environment, would case a lot of errors.

--
2.17.1




--
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

541 - 560 of 7394