Date   

Re: [ptest-runner 5/5] main: Do not return number of failed tests when calling ptest-runner

Tero Kinnunen
 

I would favor to keep non-zero exit value if any tests failed. We have relied on exit value since ptest started to support it, others may be doing that as well. This would be backward incompatible change, making tests silently fail. It is somewhat clumsy to try to parse failed tests from output, so non-zero rc was very welcome change to me.

Looking at other test runners, it is very common to return non-zero if any tests failed. (Some examples: pytest, python unittest, googletest, shunit2, robot framework.) See pytest exit codes for example: https://docs.pytest.org/en/latest/reference/exit-codes.html

Indeed the failed count is not a good exit value though because of rollover, should be fixed. Could consider using own rc for "any tests failed" differentiating from other errors.

    - Tero


[PATCH yocto-autobuilder-helper] auh-config: enable systemd only for glibc

Alexander Kanavin
 

It is not guaranteed to build or work otherwise, and causes
update failures in unrelated recipes when building for musl for instance.

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
---
scripts/auh-config/local.conf.append | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/scripts/auh-config/local.conf.append b/scripts/auh-config/local.conf.append
index b18590f..417d228 100644
--- a/scripts/auh-config/local.conf.append
+++ b/scripts/auh-config/local.conf.append
@@ -1,4 +1,5 @@

INHERIT += "buildhistory"
LICENSE_FLAGS_WHITELIST = "commercial"
-DISTRO_FEATURES_append = ' systemd pam'
+DISTRO_FEATURES_append_libc-glibc = ' systemd'
+DISTRO_FEATURES_append = ' pam'
--
2.31.1


Re: [PATCH yocto-autobuilder-helper] Enable canary builds for meta-aws[aws-iot-device-sdk-cpp-v2] on x86-64 and arm64.

Richard Purdie
 

On Thu, 2021-07-22 at 17:29 -0400, elberger via lists.yoctoproject.org wrote:
From: Richard Elberger <elberger@amazon.com>

This patch provides the buildbot system to make the aws-iot-device-sdk-cpp-v2.
- build on poky
- meta-openembedded and meta-aws are additional repo dependencies
- build for the two dominant architectures: x86-64 and arm64.

Signed-off-by: Richard Elberger <elberger@amazon.com>
---
 config.json | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/config.json b/config.json
index 3709b40..f04fac8 100644
--- a/config.json
+++ b/config.json
@@ -350,6 +350,23 @@
                 "BBTARGETS": "agl-image-core-autobuilder"
             }
         },
+ "meta-aws" : {
+ "NEEDREPOS" : ["poky", "meta-openembedded", "meta-aws"],
+ "ADDLAYER" : [
+ "${BUILDDIR}/../meta-openembedded/meta-oe",
+ "${BUILDDIR}/../meta-openembedded/meta-networking",
+ "${BUILDDIR}/../meta-openembedded/meta-python",
+ "${BUILDDIR}/../meta-aws"
The ordering isn't correct here since they have to be added in dependency 
order and for some reason this was different to the patch we discussed 
previously. I've just gone ahead and fixed it.

Cheers,

Richard


Re: [poky] [PATCH] local.conf.sample: disable prelink

Robert Berger
 

On 22/07/2021 22:05, Alexander Kanavin wrote:
PIE is nowadays more or less the only available option and is expected for improved security; Yocto does not even test non-PIE builds or provide an off the shelf way to turn it off.
I am worried about those libraries, which are non-PIE libraries by default. My theory is, that they are non-PIE since prelink is able to operate on them. So prelink can "at least" be used a PIE detector.

They are:

lib/libdl-2.33.so is prelinked
lib/ld-2.33.so is prelinked
lib/libpthread-2.33.so is prelinked
lib/libc-2.33.so is prelinked

Is there are rational explanation why they are not compiled in PIE mode and/or if they are compiled in PIE mode how cross-prelink can operate on them? If cross-prelink can operate on them why not on the others?

I also have to note that prelink does show higher RAM consumption in your tests as well (MemFree column). On the constrained systems which would benefit most from improved prelink timings that might be a bigger loss than not prelinking.
I guess we agree that MemFree shows free physical memory (user and kernel space).

My experiments show, that non-PIE and no prelink leaves the biggest amount of free physical memory.

They also show that non-PIE and prelink leave the smallest amount of free physical memory ;)

The difference is significant
prelinked-no-pie/no-prelink-no-pie: 4552 (kB)

If we leave things are they are:
prelinked-no-pie/prelinked-with-pie: 3972 (kB)

If we disable prelink (as you suggest - and I tend to agree since it does not make sense as it is right now)
prelinked-no-pie/no-prelink-with-pie: 4120 (kB)

...

but

if you look at the next line MemAvailable kB things looks a bit differently.

My interpretation of MemAvailable is, that it is an estimate of virtual memory available after reclaimable parts of memory (caches, buffer, slab,...) have been reclaimed without getting swap involved.

I see this:

MemAvailable kB

prelinked-with-pie 939412
no-prelink-with-pie 939696
prelinked-no-pie 940344
no-prelink-no-pie 941216

Which means, that our current default setting is the worst possible solution ;)

no-prelink-no-pie would (theoretically) be the best.

I will try to update my second article and try to explain a bit more my interpretation of the results and maybe also try to see what bootchart says to all this.

Don't get me wrong. I am neither pro nor con prelink. I just would like to understand what it does, if it does something ;)

I spent quite some time on this - also discussing with most of you offline.
If you ask me, we should use your patch, since people didn't even notice that prelink can not prelink on PIE binaries for a couple of years.

So there does not seem to be much demand for it ;)

We can keep a "placebo" in for the homeopaths who think they use prelink in their images since PIE was enabled ;)

But yes, there is a timing benefit visible in the tests: 0.01s vs 0.1s.
Also less CPU usage can be seen. I hope I'll find time to run some test with bootchart. Maybe then we can also see boot time, memory, CPU,...

Regards,

Robert

Alex
On Mon, 19 Jul 2021 at 22:58, Robert Berger@yocto.user <robert.berger.yocto.user@gmail.com <mailto:robert.berger.yocto.user@gmail.com>> wrote:
Hi Alex, RP, Mark,
I did some research on the subject in order to try to figure out
what is
going on.
1) I come to a similar conclusion with what found, but tried to look a
bit deeper for the reason.
1.1) The reason that cross-prelink is not prelinking is, that for a
quite some time by default everything is built with PIE mode by default
and cross-prelink does not seem to be able to work on exe/libs compiled
with PIE mode. So seeing the same behavior with and without prelinking
is what I would expect as long as everything is compiled with PIE mode.
A more detailed analysis of my tests can be found on my not yet
officially published site:
https://rlbl.me/prelink-1 <https://rlbl.me/prelink-1>
https://rlbl.me/prelink-2 <https://rlbl.me/prelink-2>
Alex:
Can you please rebuild your test images without PIE mode and re-run the
tests?
Then we should have the 4 test cases:
prelinked-with-pie
no-prelink-with-pie
prelink-no-pie
no-prelink-no-pie
I guess then we can discuss what are the next steps.
In my opinion the current default settings, which compile close to
everything in PIE mode, but invoke also cross-prelink do not make much
sense.
The question is: "Do we want to drop cross-prelink, or do we want to
drag it along and come up more fine-grained configuration options?"
We could e.g. exclude certain files from pre-linking.
IMHO cross-prelink still works, but not on exe/libs which were compiled
in PIE mode.
Regards,
Robert


Re: #yocto #yocto

Yocto
 

maybe clear your build tmp cache sstate and rebuild


On Friday 23 July 2021 13:26:08 PM (+07:00), sateesh m wrote:

Hi Guys,

              I am facing below error while compiling consul package. can anybody know this issue please suggest me. 
ERROR: consul-git-r0 do_compile: Execution of '/home/sources/fu540-build/tmp-glibc/work/riscv64-oe-linux/consul/git-r0/temp/run.do_compile.26832' failed with exit code 2:
internal/unsafeheader
unicode/utf16
container/list
internal/cpu
crypto/internal/subtle
crypto/subtle
vendor/golang.org/x/crypto/cryptobyte/asn1
internal/nettrace
vendor/golang.org/x/crypto/internal/subtle
internal/race
unicode/utf8
runtime/internal/atomic
sync/atomic
internal/testlog
runtime/internal/sys
runtime/internal/math
internal/bytealg
encoding
unicode
math/bits
math
runtime
internal/reflectlite
sync
errors
internal/singleflight
sort
math/rand
strconv
internal/oserror
io
vendor/golang.org/x/net/dns/dnsmessage
container/heap
crypto/rc4
reflect
strings
bytes
syscall
hash
crypto
crypto/internal/randutil
crypto/hmac
hash/crc32
vendor/golang.org/x/crypto/hkdf
bufio
vendor/golang.org/x/text/transform
path
text/tabwriter
regexp/syntax
encoding/base32
html
hash/crc64
hash/fnv
internal/fmtsort
hash/adler32
encoding/binary
regexp
time
internal/syscall/unix
internal/syscall/execenv
encoding/base64
crypto/cipher
crypto/sha512
crypto/ed25519/internal/edwards25519
crypto/md5
crypto/aes
crypto/des
context
crypto/sha1
internal/poll
crypto/sha256
encoding/pem
vendor/golang.org/x/crypto/chacha20
vendor/golang.org/x/crypto/poly1305
vendor/golang.org/x/crypto/chacha20poly1305
os
fmt
path/filepath
net
io/ioutil
encoding/json
encoding/hex
vendor/golang.org/x/net/http2/hpack
mime
mime/quotedprintable
vendor/golang.org/x/crypto/curve25519
os/signal
log
compress/flate
vendor/golang.org/x/text/unicode/norm
vendor/golang.org/x/text/unicode/bidi
compress/gzip
runtime/debug
net/http/internal
math/big
net/textproto
net/url
go/token
compress/lzw
encoding/gob
os/exec
github.com/hashicorp/consul/vendor/golang.org/x/net/internal/socket
vendor/golang.org/x/text/secure/bidirule
vendor/golang.org/x/net/idna
github.com/hashicorp/consul/vendor/golang.org/x/sys/unix
flag
runtime/trace
testing
os/user
encoding/asn1
crypto/elliptic
text/template/parse
crypto/rand
vendor/golang.org/x/crypto/cryptobyte
crypto/ed25519
crypto/rsa
crypto/dsa
crypto/x509/pkix
mime/multipart
vendor/golang.org/x/net/http/httpproxy
vendor/golang.org/x/net/http/httpguts
internal/lazyregexp
go/scanner
crypto/ecdsa
text/template
compress/zlib
crypto/x509
go/ast
archive/tar
github.com/hashicorp/consul/vendor/github.com/boltdb/bolt
html/template
github.com/hashicorp/consul/vendor/github.com/shirou/gopsutil/host
go/parser
go/doc
crypto/tls
net/mail
log/syslog
 
encoding/xml
# github.com/hashicorp/consul/vendor/golang.org/x/net/internal/socket
src/github.com/hashicorp/consul/vendor/golang.org/x/net/internal/socket/cmsghdr.go:9:10: undefined: cmsghdr
src/github.com/hashicorp/consul/vendor/golang.org/x/net/internal/socket/cmsghdr.go:10:10: undefined: cmsghdr
src/github.com/hashicorp/consul/vendor/golang.org/x/net/internal/socket/cmsghdr.go:11:10: undefined: cmsghdr
src/github.com/hashicorp/consul/vendor/golang.org/x/net/internal/socket/mmsghdr_unix.go:11:17: undefined: mmsghdr
encoding/csv
net/http/httptrace
internal/profile
runtime/pprof
net/http
# github.com/hashicorp/consul/vendor/github.com/shirou/gopsutil/host
src/github.com/hashicorp/consul/vendor/github.com/shirou/gopsutil/host/host_linux.go:208:22: undefined: sizeOfUtmp
src/github.com/hashicorp/consul/vendor/github.com/shirou/gopsutil/host/host_linux.go:213:14: undefined: sizeOfUtmp
src/github.com/hashicorp/consul/vendor/github.com/shirou/gopsutil/host/host_linux.go:215:9: undefined: utmp
# github.com/hashicorp/consul/vendor/github.com/boltdb/bolt
src/github.com/hashicorp/consul/vendor/github.com/boltdb/bolt/bolt_unix.go:62:15: undefined: maxMapSize
src/github.com/hashicorp/consul/vendor/github.com/boltdb/bolt/bucket.go:135:15: undefined: brokenUnaligned
src/github.com/hashicorp/consul/vendor/github.com/boltdb/bolt/db.go:101:13: undefined: maxMapSize
src/github.com/hashicorp/consul/vendor/github.com/boltdb/bolt/db.go:317:12: undefined: maxMapSize
src/github.com/hashicorp/consul/vendor/github.com/boltdb/bolt/db.go:335:10: undefined: maxMapSize
src/github.com/hashicorp/consul/vendor/github.com/boltdb/bolt/db.go:336:8: undefined: maxMapSize
src/github.com/hashicorp/consul/vendor/github.com/boltdb/bolt/freelist.go:169:19: undefined: maxAllocSize
src/github.com/hashicorp/consul/vendor/github.com/boltdb/bolt/freelist.go:176:14: undefined: maxAllocSize
src/github.com/hashicorp/consul/vendor/github.com/boltdb/bolt/freelist.go:204:17: undefined: maxAllocSize
src/github.com/hashicorp/consul/vendor/github.com/boltdb/bolt/freelist.go:207:7: undefined: maxAllocSize
src/github.com/hashicorp/consul/vendor/github.com/boltdb/bolt/freelist.go:207:7: too many errors
expvar
net/http/httptest
net/rpc
net/http/httputil
net/http/cookiejar
# github.com/hashicorp/consul/vendor/golang.org/x/sys/unix
src/github.com/hashicorp/consul/vendor/golang.org/x/sys/unix/syscall_linux_gc.go:10:6: missing function body
src/github.com/hashicorp/consul/vendor/golang.org/x/sys/unix/syscall_linux_gc.go:14:6: missing function body
src/github.com/hashicorp/consul/vendor/golang.org/x/sys/unix/syscall_unix_gc.go:12:6: missing function body
src/github.com/hashicorp/consul/vendor/golang.org/x/sys/unix/syscall_unix_gc.go:13:6: missing function body
src/github.com/hashicorp/consul/vendor/golang.org/x/sys/unix/syscall_unix_gc.go:14:6: missing function body
src/github.com/hashicorp/consul/vendor/golang.org/x/sys/unix/syscall_unix_gc.go:15:6: missing function body
net/http/pprof
WARNING: exit code 2 from a shell command.
 
ERROR: Logfile of failure stored in: /home/sources/fu540-build/tmp-glibc/work/riscv64-oe-linux/consul/git-r0/temp/log.do_compile.26832
 
--
Regards,
Sateesh

-- 
Sent with Vivaldi Mail. Download Vivaldi for free at vivaldi.com


#yocto #yocto

sateesh m
 

Hi Guys,

              I am facing below error while compiling consul package. can anybody know this issue please suggest me. 
ERROR: consul-git-r0 do_compile: Execution of '/home/sources/fu540-build/tmp-glibc/work/riscv64-oe-linux/consul/git-r0/temp/run.do_compile.26832' failed with exit code 2:
internal/unsafeheader
unicode/utf16
container/list
internal/cpu
crypto/internal/subtle
crypto/subtle
vendor/golang.org/x/crypto/cryptobyte/asn1
internal/nettrace
vendor/golang.org/x/crypto/internal/subtle
internal/race
unicode/utf8
runtime/internal/atomic
sync/atomic
internal/testlog
runtime/internal/sys
runtime/internal/math
internal/bytealg
encoding
unicode
math/bits
math
runtime
internal/reflectlite
sync
errors
internal/singleflight
sort
math/rand
strconv
internal/oserror
io
vendor/golang.org/x/net/dns/dnsmessage
container/heap
crypto/rc4
reflect
strings
bytes
syscall
hash
crypto
crypto/internal/randutil
crypto/hmac
hash/crc32
vendor/golang.org/x/crypto/hkdf
bufio
vendor/golang.org/x/text/transform
path
text/tabwriter
regexp/syntax
encoding/base32
html
hash/crc64
hash/fnv
internal/fmtsort
hash/adler32
encoding/binary
regexp
time
internal/syscall/unix
internal/syscall/execenv
encoding/base64
crypto/cipher
crypto/sha512
crypto/ed25519/internal/edwards25519
crypto/md5
crypto/aes
crypto/des
context
crypto/sha1
internal/poll
crypto/sha256
encoding/pem
vendor/golang.org/x/crypto/chacha20
vendor/golang.org/x/crypto/poly1305
vendor/golang.org/x/crypto/chacha20poly1305
os
fmt
path/filepath
net
io/ioutil
encoding/json
encoding/hex
vendor/golang.org/x/net/http2/hpack
mime
mime/quotedprintable
vendor/golang.org/x/crypto/curve25519
os/signal
log
compress/flate
vendor/golang.org/x/text/unicode/norm
vendor/golang.org/x/text/unicode/bidi
compress/gzip
runtime/debug
net/http/internal
math/big
net/textproto
net/url
go/token
compress/lzw
encoding/gob
os/exec
github.com/hashicorp/consul/vendor/golang.org/x/net/internal/socket
vendor/golang.org/x/text/secure/bidirule
vendor/golang.org/x/net/idna
github.com/hashicorp/consul/vendor/golang.org/x/sys/unix
flag
runtime/trace
testing
os/user
encoding/asn1
crypto/elliptic
text/template/parse
crypto/rand
vendor/golang.org/x/crypto/cryptobyte
crypto/ed25519
crypto/rsa
crypto/dsa
crypto/x509/pkix
mime/multipart
vendor/golang.org/x/net/http/httpproxy
vendor/golang.org/x/net/http/httpguts
internal/lazyregexp
go/scanner
crypto/ecdsa
text/template
compress/zlib
crypto/x509
go/ast
archive/tar
github.com/hashicorp/consul/vendor/github.com/boltdb/bolt
html/template
github.com/hashicorp/consul/vendor/github.com/shirou/gopsutil/host
go/parser
go/doc
crypto/tls
net/mail
log/syslog
 
encoding/xml
# github.com/hashicorp/consul/vendor/golang.org/x/net/internal/socket
src/github.com/hashicorp/consul/vendor/golang.org/x/net/internal/socket/cmsghdr.go:9:10: undefined: cmsghdr
src/github.com/hashicorp/consul/vendor/golang.org/x/net/internal/socket/cmsghdr.go:10:10: undefined: cmsghdr
src/github.com/hashicorp/consul/vendor/golang.org/x/net/internal/socket/cmsghdr.go:11:10: undefined: cmsghdr
src/github.com/hashicorp/consul/vendor/golang.org/x/net/internal/socket/mmsghdr_unix.go:11:17: undefined: mmsghdr
encoding/csv
net/http/httptrace
internal/profile
runtime/pprof
net/http
# github.com/hashicorp/consul/vendor/github.com/shirou/gopsutil/host
src/github.com/hashicorp/consul/vendor/github.com/shirou/gopsutil/host/host_linux.go:208:22: undefined: sizeOfUtmp
src/github.com/hashicorp/consul/vendor/github.com/shirou/gopsutil/host/host_linux.go:213:14: undefined: sizeOfUtmp
src/github.com/hashicorp/consul/vendor/github.com/shirou/gopsutil/host/host_linux.go:215:9: undefined: utmp
# github.com/hashicorp/consul/vendor/github.com/boltdb/bolt
src/github.com/hashicorp/consul/vendor/github.com/boltdb/bolt/bolt_unix.go:62:15: undefined: maxMapSize
src/github.com/hashicorp/consul/vendor/github.com/boltdb/bolt/bucket.go:135:15: undefined: brokenUnaligned
src/github.com/hashicorp/consul/vendor/github.com/boltdb/bolt/db.go:101:13: undefined: maxMapSize
src/github.com/hashicorp/consul/vendor/github.com/boltdb/bolt/db.go:317:12: undefined: maxMapSize
src/github.com/hashicorp/consul/vendor/github.com/boltdb/bolt/db.go:335:10: undefined: maxMapSize
src/github.com/hashicorp/consul/vendor/github.com/boltdb/bolt/db.go:336:8: undefined: maxMapSize
src/github.com/hashicorp/consul/vendor/github.com/boltdb/bolt/freelist.go:169:19: undefined: maxAllocSize
src/github.com/hashicorp/consul/vendor/github.com/boltdb/bolt/freelist.go:176:14: undefined: maxAllocSize
src/github.com/hashicorp/consul/vendor/github.com/boltdb/bolt/freelist.go:204:17: undefined: maxAllocSize
src/github.com/hashicorp/consul/vendor/github.com/boltdb/bolt/freelist.go:207:7: undefined: maxAllocSize
src/github.com/hashicorp/consul/vendor/github.com/boltdb/bolt/freelist.go:207:7: too many errors
expvar
net/http/httptest
net/rpc
net/http/httputil
net/http/cookiejar
# github.com/hashicorp/consul/vendor/golang.org/x/sys/unix
src/github.com/hashicorp/consul/vendor/golang.org/x/sys/unix/syscall_linux_gc.go:10:6: missing function body
src/github.com/hashicorp/consul/vendor/golang.org/x/sys/unix/syscall_linux_gc.go:14:6: missing function body
src/github.com/hashicorp/consul/vendor/golang.org/x/sys/unix/syscall_unix_gc.go:12:6: missing function body
src/github.com/hashicorp/consul/vendor/golang.org/x/sys/unix/syscall_unix_gc.go:13:6: missing function body
src/github.com/hashicorp/consul/vendor/golang.org/x/sys/unix/syscall_unix_gc.go:14:6: missing function body
src/github.com/hashicorp/consul/vendor/golang.org/x/sys/unix/syscall_unix_gc.go:15:6: missing function body
net/http/pprof
WARNING: exit code 2 from a shell command.
 
ERROR: Logfile of failure stored in: /home/sources/fu540-build/tmp-glibc/work/riscv64-oe-linux/consul/git-r0/temp/log.do_compile.26832
 
--
Regards,
Sateesh


Re: consul

Yocto
 

SRC_URI = "git://${GO_IMPORT}.git;branch=main \


On Friday 23 July 2021 12:37:47 PM (+07:00), sateesh m wrote:

Hi Guys,

                I am trying to integrate consul package to my image.facing issue while fetching the sources.

WARNING: consul-git-r0 do_fetch: Failed to fetch URL git://github.com/hashicorp/consul.git, attempting MIRRORS if available
ERROR: consul-git-r0 do_fetch: Fetcher failure: Unable to find revision 944cc71026c007e7de9467ec3f38f0ad14464fcc in branch master even from upstream
ERROR: consul-git-r0 do_fetch: Fetcher failure for URL: 'git://github.com/hashicorp/consul.git'. Unable to fetch URL from any source.

Can anybody know please  tell  which revision ID i need to used to fix this issue. or any patch available suggest me. 

Thanking you in advance. 
--
Regards,
Sateesh

-- 
Sent with Vivaldi Mail. Download Vivaldi for free at vivaldi.com


Re: consul

Yocto
 

add branch=main to the giturl github chaged a huge swatch from master to main


On Friday 23 July 2021 12:37:47 PM (+07:00), sateesh m wrote:

Hi Guys,

                I am trying to integrate consul package to my image.facing issue while fetching the sources.

WARNING: consul-git-r0 do_fetch: Failed to fetch URL git://github.com/hashicorp/consul.git, attempting MIRRORS if available
ERROR: consul-git-r0 do_fetch: Fetcher failure: Unable to find revision 944cc71026c007e7de9467ec3f38f0ad14464fcc in branch master even from upstream
ERROR: consul-git-r0 do_fetch: Fetcher failure for URL: 'git://github.com/hashicorp/consul.git'. Unable to fetch URL from any source.

Can anybody know please  tell  which revision ID i need to used to fix this issue. or any patch available suggest me. 

Thanking you in advance. 
--
Regards,
Sateesh

-- 
Sent with Vivaldi Mail. Download Vivaldi for free at vivaldi.com


consul

sateesh m
 

Hi Guys,

                I am trying to integrate consul package to my image.facing issue while fetching the sources.

WARNING: consul-git-r0 do_fetch: Failed to fetch URL git://github.com/hashicorp/consul.git, attempting MIRRORS if available
ERROR: consul-git-r0 do_fetch: Fetcher failure: Unable to find revision 944cc71026c007e7de9467ec3f38f0ad14464fcc in branch master even from upstream
ERROR: consul-git-r0 do_fetch: Fetcher failure for URL: 'git://github.com/hashicorp/consul.git'. Unable to fetch URL from any source.

Can anybody know please  tell  which revision ID i need to used to fix this issue. or any patch available suggest me. 

Thanking you in advance. 
--
Regards,
Sateesh


[PATCH yocto-autobuilder-helper] Enable canary builds for meta-aws[aws-iot-device-sdk-cpp-v2] on x86-64 and arm64.

elberger@...
 

From: Richard Elberger <elberger@amazon.com>

This patch provides the buildbot system to make the aws-iot-device-sdk-cpp-v2.
- build on poky
- meta-openembedded and meta-aws are additional repo dependencies
- build for the two dominant architectures: x86-64 and arm64.

Signed-off-by: Richard Elberger <elberger@amazon.com>
---
config.json | 17 +++++++++++++++++
1 file changed, 17 insertions(+)

diff --git a/config.json b/config.json
index 3709b40..f04fac8 100644
--- a/config.json
+++ b/config.json
@@ -350,6 +350,23 @@
"BBTARGETS": "agl-image-core-autobuilder"
}
},
+ "meta-aws" : {
+ "NEEDREPOS" : ["poky", "meta-openembedded", "meta-aws"],
+ "ADDLAYER" : [
+ "${BUILDDIR}/../meta-openembedded/meta-oe",
+ "${BUILDDIR}/../meta-openembedded/meta-networking",
+ "${BUILDDIR}/../meta-openembedded/meta-python",
+ "${BUILDDIR}/../meta-aws"
+ ],
+ "step1" : {
+ "MACHINE": "qemux86-64",
+ "BBTARGETS": "aws-iot-device-sdk-cpp-v2"
+ },
+ "step2" : {
+ "MACHINE": "qemuarm64",
+ "BBTARGETS": "aws-iot-device-sdk-cpp-v2"
+ }
+ },
"bringup" : {
"BUILDINFO" : true,
"SSTATEDIR" : ["SSTATE_DIR ?= '${HELPERBUILDDIR}/sstate'"],
--
2.17.1


Re: [PATCH yocto-autobuilder-helper] Enable canary builds for meta-aws[aws-iot-device-sdk-cpp-v2] on x86-64 and arm64.

elberger@...
 

Ok let me give it a shot again - I'll get there :)

On 7/22/21 4:51 PM, Richard Purdie wrote:
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.



On Thu, 2021-07-22 at 16:05 -0400, elberger via lists.yoctoproject.org wrote:
Hello,

What can I do to move this patch along - or did I miss anything?
Apologies for my ignorance, it's my first time submitting a patch to YP.
I can't see the patch in the mailing list archives:

https://lists.yoctoproject.org/g/yocto/topics?

only your reply:

https://lists.yoctoproject.org/g/yocto/topic/patch/84387478?

so I don't think the original patch made it to the list? Since I never
saw it there, I couldn't apply it!

Cheers,

Richard





-- 
Richard Elberger
Principal Technologist
AWS IoT

+1 646.927.6897
+1 203.942.8039 – mobile
elberger@... – email


Re: [PATCH yocto-autobuilder-helper] Enable canary builds for meta-aws[aws-iot-device-sdk-cpp-v2] on x86-64 and arm64.

Richard Purdie
 

On Thu, 2021-07-22 at 16:05 -0400, elberger via lists.yoctoproject.org wrote:
Hello,

What can I do to move this patch along - or did I miss anything? 
Apologies for my ignorance, it's my first time submitting a patch to YP.
I can't see the patch in the mailing list archives:

https://lists.yoctoproject.org/g/yocto/topics?

only your reply:

https://lists.yoctoproject.org/g/yocto/topic/patch/84387478?

so I don't think the original patch made it to the list? Since I never
saw it there, I couldn't apply it!

Cheers,

Richard


Re: [PATCH yocto-autobuilder-helper] Enable canary builds for meta-aws[aws-iot-device-sdk-cpp-v2] on x86-64 and arm64.

elberger@...
 

Hello,

What can I do to move this patch along - or did I miss anything?  Apologies for my ignorance, it's my first time submitting a patch to YP.

Richard Elberger

On 7/20/21 7:08 AM, rpcme.github@gmail.com wrote:
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.



From: Richard Elberger <elberger@amazon.com>

This patch provides the buildbot system to make the aws-iot-device-sdk-cpp-v2.
- build on poky
- meta-openembedded and meta-aws are additional repo dependencies
- build for the two dominant architectures: x86-64 and arm64.

Signed-off-by: Richard Elberger <elberger@amazon.com>
---
config.json | 17 +++++++++++++++++
1 file changed, 17 insertions(+)

diff --git a/config.json b/config.json
index 3709b40..f04fac8 100644
--- a/config.json
+++ b/config.json
@@ -350,6 +350,23 @@
"BBTARGETS": "agl-image-core-autobuilder"
}
},
+ "meta-aws" : {
+ "NEEDREPOS" : ["poky", "meta-openembedded", "meta-aws"],
+ "ADDLAYER" : [
+ "${BUILDDIR}/../meta-openembedded/meta-oe",
+ "${BUILDDIR}/../meta-openembedded/meta-networking",
+ "${BUILDDIR}/../meta-openembedded/meta-python",
+ "${BUILDDIR}/../meta-aws"
+ ],
+ "step1" : {
+ "MACHINE": "qemux86-64",
+ "BBTARGETS": "aws-iot-device-sdk-cpp-v2"
+ },
+ "step2" : {
+ "MACHINE": "qemuarm64",
+ "BBTARGETS": "aws-iot-device-sdk-cpp-v2"
+ }
+ },
"bringup" : {
"BUILDINFO" : true,
"SSTATEDIR" : ["SSTATE_DIR ?= '${HELPERBUILDDIR}/sstate'"],
--
2.17.1
--
Richard Elberger
Principal Technologist
AWS IoT

+1 646.927.6897
+1 203.942.8039 – mobile
elberger@amazon.com – email


Re: [poky] [PATCH] local.conf.sample: disable prelink

Alexander Kanavin
 

PIE is nowadays more or less the only available option and is expected for improved security; Yocto does not even test non-PIE builds or provide an off the shelf way to turn it off.

I also have to note that prelink does show higher RAM consumption in your tests as well (MemFree column). On the constrained systems which would benefit most from improved prelink timings that might be a bigger loss than not prelinking.

But yes, there is a timing benefit visible in the tests: 0.01s vs 0.1s.

Alex


On Mon, 19 Jul 2021 at 22:58, Robert Berger@... <robert.berger.yocto.user@...> wrote:
Hi Alex, RP, Mark,

I did some research on the subject in order to try to figure out what is
going on.

1) I come to a similar conclusion with what found, but tried to look a
bit deeper for the reason.

1.1) The reason that cross-prelink is not prelinking is, that for a
quite some time by default everything is built with PIE mode by default
and cross-prelink does not seem to be able to work on exe/libs compiled
with PIE mode. So seeing the same behavior with and without prelinking
is what I would expect as long as everything is compiled with PIE mode.

A more detailed analysis of my tests can be found on my not yet
officially published site:

https://rlbl.me/prelink-1

https://rlbl.me/prelink-2

Alex:

Can you please rebuild your test images without PIE mode and re-run the
tests?

Then we should have the 4 test cases:

prelinked-with-pie
no-prelink-with-pie
prelink-no-pie
no-prelink-no-pie

I guess then we can discuss what are the next steps.

In my opinion the current default settings, which compile close to
everything in PIE mode, but invoke also cross-prelink do not make much
sense.

The question is: "Do we want to drop cross-prelink, or do we want to
drag it along and come up more fine-grained configuration options?"

We could e.g. exclude certain files from pre-linking.

IMHO cross-prelink still works, but not on exe/libs which were compiled
in PIE mode.

Regards,

Robert


[yocto-autobuilder-helper][hardknott][PATCH] config.json: track system load with PARALLEL_MAKE

Trevor Gamblin
 

Backport for hardknott since we have been running for ~1 week with this
patch in master and the same load limiting may help reduce build
failures for hardknott itself as well as when its builds are running
alongside other branches.

This adds the "-l" option to PARALLEL_MAKE in config.json with an
initial testing value of 50 (the previous iteration of this patch both
had it set to 100 and mistakenly stated that this was a percentage,
rather than an absolute representation of system load ala top/htop).
This option is supported by both Make and Ninja. However, we also
require the "--debug=j" option to be passed to Make in order for the
latter to report perceived system load in the do_compile logs, (which we
need in order to track test values), and since this option is not
supported by Ninja, also add EXTRA_OEMAKE to the EXTRAVARS so that we
can determine if the target load percentage needs to be adjusted.

Signed-off-by: Trevor Gamblin <trevor.gamblin@windriver.com>
---
config.json | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/config.json b/config.json
index 110b0d9..5689cb9 100644
--- a/config.json
+++ b/config.json
@@ -43,7 +43,7 @@
"PREMIRRORS = ''",
"BB_GENERATE_MIRROR_TARBALLS = '1'",
"BB_NUMBER_THREADS = '16'",
- "PARALLEL_MAKE = '-j 16'",
+ "PARALLEL_MAKE = '-j 16 -l 50'",
"XZ_MEMLIMIT = '5%'",
"XZ_THREADS = '8'",
"BB_TASK_NICE_LEVEL = '5'",
@@ -57,7 +57,8 @@
"SDK_INCLUDE_TOOLCHAIN = '1'",
"BB_DISKMON_DIRS = 'STOPTASKS,${TMPDIR},1G,100K STOPTASKS,${DL_DIR},1G STOPTASKS,${SSTATE_DIR},1G STOPTASKS,/tmp,100M,100K ABORT,${TMPDIR},100M,1K ABORT,${DL_DIR},100M ABORT,${SSTATE_DIR},100M ABORT,/tmp,10M,1K'",
"BB_HASHSERVE = 'typhoon.yocto.io:8686'",
- "RUNQEMU_TMPFS_DIR = '/home/pokybuild/tmp'"
+ "RUNQEMU_TMPFS_DIR = '/home/pokybuild/tmp'",
+ "EXTRA_OEMAKE = ' --debug=j'"
]
},
"templates" : {
--
2.31.1


[yocto-autobuilder-helper][gatesgarth][PATCH] config.json: track system load with PARALLEL_MAKE

Trevor Gamblin
 

Backport for gatesgarth since we have been running for ~1 week with this
patch in master and the same load limiting may help reduce build
failures for gatesgarth itself as well as when its builds are running
alongside other branches.

This adds the "-l" option to PARALLEL_MAKE in config.json with an
initial testing value of 50 (the previous iteration of this patch both
had it set to 100 and mistakenly stated that this was a percentage,
rather than an absolute representation of system load ala top/htop).
This option is supported by both Make and Ninja. However, we also
require the "--debug=j" option to be passed to Make in order for the
latter to report perceived system load in the do_compile logs, (which we
need in order to track test values), and since this option is not
supported by Ninja, also add EXTRA_OEMAKE to the EXTRAVARS so that we
can determine if the target load percentage needs to be adjusted.

Signed-off-by: Trevor Gamblin <trevor.gamblin@windriver.com>
---
config.json | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/config.json b/config.json
index 5dda653..8a4343a 100644
--- a/config.json
+++ b/config.json
@@ -43,7 +43,7 @@
"PREMIRRORS = ''",
"BB_GENERATE_MIRROR_TARBALLS = '1'",
"BB_NUMBER_THREADS = '16'",
- "PARALLEL_MAKE = '-j 16'",
+ "PARALLEL_MAKE = '-j 16 -l 50'",
"XZ_MEMLIMIT = '5%'",
"XZ_THREADS = '8'",
"BB_TASK_NICE_LEVEL = '5'",
@@ -56,7 +56,8 @@
"SDK_EXT_TYPE = 'minimal'",
"SDK_INCLUDE_TOOLCHAIN = '1'",
"BB_DISKMON_DIRS = 'STOPTASKS,${TMPDIR},1G,100K STOPTASKS,${DL_DIR},1G STOPTASKS,${SSTATE_DIR},1G STOPTASKS,/tmp,100M,100K ABORT,${TMPDIR},100M,1K ABORT,${DL_DIR},100M ABORT,${SSTATE_DIR},100M ABORT,/tmp,10M,1K'",
- "BB_HASHSERVE = 'typhoon.yocto.io:8686'"
+ "BB_HASHSERVE = 'typhoon.yocto.io:8686'",
+ "EXTRA_OEMAKE = ' --debug=j'"
]
},
"templates" : {
--
2.31.1


[yocto-autobuilder-helper][dunfell][PATCH] config.json: track system load with PARALLEL_MAKE

Trevor Gamblin
 

Backport for dunfell since we have been running for ~1 week with this
patch in master and the same load limiting may help reduce build
failures for dunfell itself as well as when its builds are running
alongside other branches.

This adds the "-l" option to PARALLEL_MAKE in config.json with an
initial testing value of 50 (the previous iteration of this patch both
had it set to 100 and mistakenly stated that this was a percentage,
rather than an absolute representation of system load ala top/htop).
This option is supported by both Make and Ninja. However, we also
require the "--debug=j" option to be passed to Make in order for the
latter to report perceived system load in the do_compile logs, (which we
need in order to track test values), and since this option is not
supported by Ninja, also add EXTRA_OEMAKE to the EXTRAVARS so that we
can determine if the target load percentage needs to be adjusted.

Signed-off-by: Trevor Gamblin <trevor.gamblin@windriver.com>
---
config.json | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/config.json b/config.json
index 3cff363..8323864 100644
--- a/config.json
+++ b/config.json
@@ -43,7 +43,7 @@
"PREMIRRORS = ''",
"BB_GENERATE_MIRROR_TARBALLS = '1'",
"BB_NUMBER_THREADS = '16'",
- "PARALLEL_MAKE = '-j 16'",
+ "PARALLEL_MAKE = '-j 16 -l 50'",
"XZ_MEMLIMIT = '5%'",
"XZ_THREADS = '8'",
"BB_TASK_NICE_LEVEL = '5'",
@@ -56,7 +56,8 @@
"SDK_EXT_TYPE = 'minimal'",
"SDK_INCLUDE_TOOLCHAIN = '1'",
"BB_DISKMON_DIRS = 'STOPTASKS,${TMPDIR},1G,100K STOPTASKS,${DL_DIR},1G STOPTASKS,${SSTATE_DIR},1G STOPTASKS,/tmp,100M,100K ABORT,${TMPDIR},100M,1K ABORT,${DL_DIR},100M ABORT,${SSTATE_DIR},100M ABORT,/tmp,10M,1K'",
- "BB_HASHSERVE = 'typhoon.yocto.io:8686'"
+ "BB_HASHSERVE = 'typhoon.yocto.io:8686'",
+ "EXTRA_OEMAKE = ' --debug=j'"
]
},
"templates" : {
--
2.31.1


[meta-spdxscanner][PATCH] MAINTAINERS & README.md: Updated the email address where send patches

Marco Cavallini
 

From: Marco Cavallini <m.cavallini@koansoftware.com>

Was yocto@yoctoproject.org and now is yocto@lists.yoctoproject.org

Signed-off-by: Marco Cavallini <koansoftware@gmail.com>
---
MAINTAINERS | 2 +-
README.md | 4 ++++
2 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/MAINTAINERS b/MAINTAINERS
index 23761bf..ec34c24 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1,7 +1,7 @@
This file contains a list of maintainers for the meta-spdxscanner layer.

Please submit any patches against meta-spdxscanner to the Yocto Project
-mailing list (yocto@yoctoproject.org) with'[meta-spdxscanner]' in the
+mailing list (yocto@lists.yoctoproject.org) with'[meta-spdxscanner]' in the
subject.

You may also contact the maintainers directly.
diff --git a/README.md b/README.md
index eb0c389..62b59f5 100644
--- a/README.md
+++ b/README.md
@@ -69,3 +69,7 @@ Note
- scancode-tk has to install on host development under python2. So, please make sure that python2 exists on your build server.
- If you want to use scancode.bbclass, There is no need to install anything on your host.

+---
+
+Please submit any patches against meta-spdxscanner to the Yocto Project mailing list (yocto@lists.yoctoproject.org)
+with'[meta-spdxscanner]' in the subject.
\ No newline at end of file
--
2.17.1


[meta-spdxscanner][PATCH] fossology-rest.bbclass: Moved check if spdx already exists out of the task

Marco Cavallini
 

From: Marco Cavallini <m.cavallini@koansoftware.com>

With the previous code was impossible to run the do_get_report
task whenever a correspondent .spdx file was already present
in tmp/deploy/spdx.
Now the check if spdx already exists is at a higher level
therefore is possible to run : bitbake -c get_report

Signed-off-by: Marco Cavallini <koansoftware@gmail.com>
---
classes/fossology-rest.bbclass | 18 +++++++++---------
1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/classes/fossology-rest.bbclass b/classes/fossology-rest.bbclass
index 3694440..e55e859 100644
--- a/classes/fossology-rest.bbclass
+++ b/classes/fossology-rest.bbclass
@@ -84,6 +84,15 @@ python () {
create_manifest(info,sstatefile)
return

+ # if spdx already exists
+ if os.path.exists(info['outfile']):
+ bb.note(info['pn'] + "spdx file has been exist, do nothing")
+ return
+ if os.path.exists( sstatefile ):
+ bb.note(info['pn'] + "spdx file has been exist, do nothing")
+ create_manifest(info,sstatefile)
+ return
+
d.appendVarFlag('do_spdx', 'depends', ' %s:do_get_report' % pn)
bb.build.addtask('do_get_report', 'do_configure', 'do_patch' , d)
bb.build.addtask('do_spdx', 'do_configure', 'do_get_report', d)
@@ -157,15 +166,6 @@ python do_get_report () {
info['outfile'] = os.path.join(manifest_dir, info['pn'] + "-" + info['pv'] + ".spdx" )
sstatefile = os.path.join(spdx_outdir, info['pn'] + "-" + info['pv'] + ".spdx" )

- # if spdx has been exist
- if os.path.exists(info['outfile']):
- bb.note(info['pn'] + "spdx file has been exist, do nothing")
- return
- if os.path.exists( sstatefile ):
- bb.note(info['pn'] + "spdx file has been exist, do nothing")
- create_manifest(info,sstatefile)
- return
-
spdx_get_src(d)

bb.note('SPDX: Archiving the patched source...')
--
2.17.1


Re: [ptest-runner 0/5] ptest: Various memory fixes and enhancements

?ukasz Majewski
 

Hi Randy,

On 2021-07-21 5:46 a.m., ?ukasz Majewski wrote:
This patch series provides some memory management fixes and
corrected return code handling for ptest-runner2.

Adrian Freihofer (4):
git: Extend the gitignore
mem: Fix memleak for ptest_opts
mem: Simplify memory management
mem: Refactor ptest_list cleanup

Lukasz Majewski (1):
main: Do not return number of failed tests when calling
ptest-runner

.gitignore | 3 +++
main.c | 33 ++++++++++++++++++++++++++++-----
ptest_list.c | 13 ++++---------
ptest_list.h | 8 +-------
tests/ptest_list.c | 13 +++++++------
tests/utils.c | 22 +++++++++++-----------
utils.c | 11 ++++-------
7 files changed, 58 insertions(+), 45 deletions(-)
create mode 100644 .gitignore




Looks good to me.

Did you happen to check the compile with clang as well?
If not, I may do that eventually.
No, I've just used gcc.


Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de

1241 - 1260 of 55412