#yocto #kernel BB_GENERATE_MIRROR_TARBALLS #yocto #kernel


Monsees, Steven C (US)
 

When I generate MIRROR TARBALLS for my build it does not appear to generate all the required tarballs…

 

I tested by tested by rebuilding with “BB_NO_NETWORK = 1”, and received the following build Error (see below).

 

Is there something else I need to define to get all the required tarballs in build ?

 

Thanks,

Steve

 

Initialising tasks: 100% |##########################################################| Time: 0:00:03

Sstate summary: Wanted 1412 Found 0 Missed 1412 Current 0 (0% match, 0% complete)

NOTE: Executing Tasks

NOTE: Setscene tasks completed

ERROR: ncurses-native-6.1+20190803-r0 do_fetch: Network access disabled through BB_NO_NETWORK (or set indirectly due to use of BB_FETCH_PREMIRRORONLY) but access requested with command LANG=C git -c core.fsyncobjectfiles=0 clone --bare --mirror https://salsa.debian.org/debian/ncurses.git /disk0/scratch/smonsees/yocto/workspace_3/downloads/aarch64/git2/salsa.debian.org.debian.ncurses.git --progress (for url git://salsa.debian.org/debian/ncurses.git;protocol=https)

ERROR: Logfile of failure stored in: /disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/aiox-default/tmp/work/x86_64-linux/ncurses-native/6.1+20190803-r0/temp/log.do_fetch.16824

ERROR: Task (virtual:native:/disk0/scratch/smonsees/yocto/workspace_3/poky/meta/recipes-core/ncurses/ncurses_6.1+20190803.bb:do_fetch) failed with exit code '1'

NOTE: Tasks Summary: Attempted 99 tasks of which 0 didn't need to be rerun and 1 failed.

 

Summary: 1 task failed:

  virtual:native:/disk0/scratch/smonsees/yocto/workspace_3/poky/meta/recipes-core/ncurses/ncurses_6.1+20190803.bb:do_fetch

 

ncurses_6.1+20190803.bb:

 

require ncurses.inc

 

SRC_URI += "file://0001-tic-hang.patch \

            file://0002-configure-reproducible.patch \

            file://config.cache \

            file://0001-ncurses-selective-backport-of-20191012-patch.patch \

"

# commit id corresponds to the revision in package version

SRCREV = "3c9b2677c96c645496997321bf2fe465a5e7e21f"

S = "${WORKDIR}/git"

EXTRA_OECONF += "--with-abi-version=5 --cache-file=${B}/config.cache"

UPSTREAM_CHECK_GITTAGREGEX = "(?P<pver>\d+(\.\d+)+(\+\d+)*)"

 

CVE_VERSION = "6.1.${@d.getVar("PV").split('+')[1]}"

 

/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/aiox-default/tmp/work/x86_64-linux/ncurses-native/6.1+20190803-r0/temp/log.do_fetch.16824:

 

DEBUG: Executing python function extend_recipe_sysroot

NOTE: Direct dependencies are []

NOTE: Installed into sysroot: []

NOTE: Skipping as already exists in sysroot: []

DEBUG: Python function extend_recipe_sysroot finished

DEBUG: Executing python function do_fetch

DEBUG: Executing python function base_do_fetch

DEBUG: Trying PREMIRRORS

DEBUG: For url ['git', 'salsa.debian.org', '/debian/ncurses.git', '', '', OrderedDict([('protocol', 'https')])] comparing ['git', '.*', '/.*', '', '', OrderedDict()] to ['file', '', '//ede/tms/yocto/zeus/downloads/aarch64', '', '', OrderedDict()]

DEBUG: For url git://salsa.debian.org/debian/ncurses.git;protocol=https returning file:///ede/tms/yocto/zeus/downloads/aarch64/git2_salsa.debian.org.debian.ncurses.git.tar.gz

DEBUG: For url ['file', '', '/ede/tms/yocto/zeus/downloads/aarch64/git2_salsa.debian.org.debian.ncurses.git.tar.gz', '', '', OrderedDict()] comparing ['ftp', '.*', '/.*', '', '', OrderedDict()] to ['file', '', '//ede/tms/yocto/zeus/downloads/aarch64', '', '', OrderedDict()]

DEBUG: For url ['file', '', '/ede/tms/yocto/zeus/downloads/aarch64/git2_salsa.debian.org.debian.ncurses.git.tar.gz', '', '', OrderedDict()] comparing ['http', '.*', '/.*', '', '', OrderedDict()] to ['file', '', '//ede/tms/yocto/zeus/downloads/aarch64', '', '', OrderedDict()]

DEBUG: For url ['file', '', '/ede/tms/yocto/zeus/downloads/aarch64/git2_salsa.debian.org.debian.ncurses.git.tar.gz', '', '', OrderedDict()] comparing ['https', '.*', '/.*', '', '', OrderedDict()] to ['file', '', '//ede/tms/yocto/zeus/downloads/aarch64', '', '', OrderedDict()]

DEBUG: For url ['git', 'salsa.debian.org', '/debian/ncurses.git', '', '', OrderedDict([('protocol', 'https')])] comparing ['ftp', '.*', '/.*', '', '', OrderedDict()] to ['file', '', '//ede/tms/yocto/zeus/downloads/aarch64', '', '', OrderedDict()]

DEBUG: For url ['git', 'salsa.debian.org', '/debian/ncurses.git', '', '', OrderedDict([('protocol', 'https')])] comparing ['http', '.*', '/.*', '', '', OrderedDict()] to ['file', '', '//ede/tms/yocto/zeus/downloads/aarch64', '', '', OrderedDict()]

DEBUG: For url ['git', 'salsa.debian.org', '/debian/ncurses.git', '', '', OrderedDict([('protocol', 'https')])] comparing ['https', '.*', '/.*', '', '', OrderedDict()] to ['file', '', '//ede/tms/yocto/zeus/downloads/aarch64', '', '', OrderedDict()]

DEBUG: Mirror fetch failure for url file:///ede/tms/yocto/zeus/downloads/aarch64/git2_salsa.debian.org.debian.ncurses.git.tar.gz (original url: git://salsa.debian.org/debian/ncurses.git;protocol=https)

DEBUG: Fetcher failure: Unable to find file file:///ede/tms/yocto/zeus/downloads/aarch64/git2_salsa.debian.org.debian.ncurses.git.tar.gz anywhere. The paths that were searched were:

    /disk0/scratch/smonsees/yocto/workspace_3/poky/meta/recipes-core/ncurses/ncurses-6.1+20190803/limws

    /disk0/scratch/smonsees/yocto/workspace_3/poky/meta/recipes-core/ncurses/ncurses/limws

    /disk0/scratch/smonsees/yocto/workspace_3/poky/meta/recipes-core/ncurses/files/limws

    /disk0/scratch/smonsees/yocto/workspace_3/poky/meta/recipes-core/ncurses/ncurses-6.1+20190803/

    /disk0/scratch/smonsees/yocto/workspace_3/poky/meta/recipes-core/ncurses/ncurses/

    /disk0/scratch/smonsees/yocto/workspace_3/poky/meta/recipes-core/ncurses/files/

    /disk0/scratch/smonsees/yocto/workspace_3/poky/meta/recipes-core/ncurses/ncurses-6.1+20190803/x86-64

    /disk0/scratch/smonsees/yocto/workspace_3/poky/meta/recipes-core/ncurses/ncurses/x86-64

    /disk0/scratch/smonsees/yocto/workspace_3/poky/meta/recipes-core/ncurses/files/x86-64

    /disk0/scratch/smonsees/yocto/workspace_3/poky/meta/recipes-core/ncurses/ncurses-6.1+20190803/

    /disk0/scratch/smonsees/yocto/workspace_3/poky/meta/recipes-core/ncurses/ncurses/

    /disk0/scratch/smonsees/yocto/workspace_3/poky/meta/recipes-core/ncurses/files/

    /disk0/scratch/smonsees/yocto/workspace_3/downloads/aarch64

DEBUG: Trying Upstream

ERROR: Network access disabled through BB_NO_NETWORK (or set indirectly due to use of BB_FETCH_PREMIRRORONLY) but access requested with command LANG=C git -c core.fsyncobjectfiles=0 clone --bare --mirror https://salsa.debian.org/debian/ncurses.git /disk0/scratch/smonsees/yocto/workspace_3/downloads/aarch64/git2/salsa.debian.org.debian.ncurses.git --progress (for url git://salsa.debian.org/debian/ncurses.git;protocol=https)

DEBUG: Python function base_do_fetch finished

 

 


Richard Purdie
 

On Tue, 2021-01-19 at 13:42 +0000, Monsees, Steven C (US) via
lists.yoctoproject.org wrote:
When I generate MIRROR TARBALLS for my build it does not appear to
generate all the required tarballs…
 
I tested by tested by rebuilding with “BB_NO_NETWORK = 1”, and
received the following build Error (see below).
 
Is there something else I need to define to get all the required
tarballs in build ?
Did you set:

BB_GENERATE_MIRROR_TARBALLS = "1"

If not you need to set that. Also, you may need to refetch things
initially to ensure all the right things are generated with that set.

Cheers,

Richard


Monsees, Steven C (US)
 

That was set : export BB_GENERATE_MIRROR_TARBALLS=1

-----Original Message-----
From: yocto@... <yocto@...> On Behalf Of Richard Purdie
Sent: Tuesday, January 19, 2021 9:02 AM
To: Monsees, Steven C (US) <steven.monsees@...>; yocto@...
Subject: Re: [yocto] #yocto #kernel BB_GENERATE_MIRROR_TARBALLS

*** WARNING ***
EXTERNAL EMAIL -- This message originates from outside our organization.


On Tue, 2021-01-19 at 13:42 +0000, Monsees, Steven C (US) via lists.yoctoproject.org wrote:
When I generate MIRROR TARBALLS for my build it does not appear to
generate all the required tarballs…
 
I tested by tested by rebuilding with “BB_NO_NETWORK = 1”, and
received the following build Error (see below).
 
Is there something else I need to define to get all the required
tarballs in build ?
Did you set:

BB_GENERATE_MIRROR_TARBALLS = "1"

If not you need to set that. Also, you may need to refetch things initially to ensure all the right things are generated with that set.

Cheers,

Richard


Richard Purdie
 

On Tue, 2021-01-19 at 14:08 +0000, Monsees, Steven C (US) wrote:
That was set : export BB_GENERATE_MIRROR_TARBALLS=1
Just to be clear, where did you set that? In a .conf file? If so, you
don't need the export bit. If it was in a shell it probably wouldn't
work.

Also, you didn't answer my other question, did you refetch everything
with that set?

Cheers,

Richard


Monsees, Steven C (US)
 

From shell... I just checked the mega-manual, will rerun as described...

-----Original Message-----
From: Richard Purdie <richard.purdie@...>
Sent: Tuesday, January 19, 2021 9:13 AM
To: Monsees, Steven C (US) <steven.monsees@...>; yocto@...
Subject: Re: [yocto] #yocto #kernel BB_GENERATE_MIRROR_TARBALLS

*** WARNING ***
EXTERNAL EMAIL -- This message originates from outside our organization.


On Tue, 2021-01-19 at 14:08 +0000, Monsees, Steven C (US) wrote:
That was set : export BB_GENERATE_MIRROR_TARBALLS=1
Just to be clear, where did you set that? In a .conf file? If so, you don't need the export bit. If it was in a shell it probably wouldn't work.

Also, you didn't answer my other question, did you refetch everything with that set?

Cheers,

Richard


Monsees, Steven C (US)
 

All but the following downloaded except these which failed (timing out):

https://sourceware.org/elfutils/ftp/0.177/elfutils-0.177.tar.bz2
https://download.qemu.org/qemu-4.1.0.tar.xz
http://libarchive.org/downloads/libarchive-3.4.0.tar.gz

fyi... I am using zeus 3.0.4.

Steve

-----Original Message-----
From: yocto@... <yocto@...> On Behalf Of Monsees, Steven C (US) via lists.yoctoproject.org
Sent: Tuesday, January 19, 2021 9:25 AM
To: Richard Purdie <richard.purdie@...>; yocto@...
Subject: Re: [yocto] #yocto #kernel BB_GENERATE_MIRROR_TARBALLS

*** WARNING ***
EXTERNAL EMAIL -- This message originates from outside our organization.



From shell... I just checked the mega-manual, will rerun as described...

-----Original Message-----
From: Richard Purdie <richard.purdie@...>
Sent: Tuesday, January 19, 2021 9:13 AM
To: Monsees, Steven C (US) <steven.monsees@...>; yocto@...
Subject: Re: [yocto] #yocto #kernel BB_GENERATE_MIRROR_TARBALLS

*** WARNING ***
EXTERNAL EMAIL -- This message originates from outside our organization.


On Tue, 2021-01-19 at 14:08 +0000, Monsees, Steven C (US) wrote:
That was set : export BB_GENERATE_MIRROR_TARBALLS=1
Just to be clear, where did you set that? In a .conf file? If so, you don't need the export bit. If it was in a shell it probably wouldn't work.

Also, you didn't answer my other question, did you refetch everything with that set?

Cheers,

Richard


Monsees, Steven C (US)
 

This is the types of error I am seeing:


WARNING: qemu-native-4.1.0-r0 do_fetch: Failed to fetch URL https://download.qemu.org/qemu-4.1.0.tar.xz, attempting MIRRORS if available
WARNING: libarchive-native-3.4.0-r0 do_fetch: Failed to fetch URL http://libarchive.org/downloads/libarchive-3.4.0.tar.gz, attempting MIRRORS if available
ERROR: libarchive-native-3.4.0-r0 do_fetch: Fetcher failure: Fetch command export PSEUDO_DISABLED=1; unset _PYTHON_SYSCONFIGDATA_NAME; export DBUS_SESSION_BUS_ADDRESS="unix:abstract=/tmp/dbus-WGeN1VUN5f,guid=5f3398a95f5d5aa6a98efe74600732fe"; export SSH_AGENT_PID="3519"; export SSH_AUTH_SOCK="/run/user/11588/keyring/ssh"; export PATH="/disk0/scratch/smonsees/yocto/workspace_3/poky/scripts/native-intercept:/disk0/scratch/smonsees/yocto/workspace_3/poky/scripts:/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/aiox-default/tmp/work/x86_64-linux/libarchive-native/3.4.0-r0/recipe-sysroot-native/usr/bin/x86_64-linux:/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/aiox-default/tmp/work/x86_64-linux/libarchive-native/3.4.0-r0/recipe-sysroot-native/usr/bin:/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/aiox-default/tmp/work/x86_64-linux/libarchive-native/3.4.0-r0/recipe-sysroot-native/usr/sbin:/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/aiox-default/tmp/work/x86_64-linux/libarchive-native/3.4.0-r0/recipe-sysroot-native/usr/bin:/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/aiox-default/tmp/work/x86_64-linux/libarchive-native/3.4.0-r0/recipe-sysroot-native/sbin:/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/aiox-default/tmp/work/x86_64-linux/libarchive-native/3.4.0-r0/recipe-sysroot-native/bin:/disk0/scratch/smonsees/yocto/workspace_3/poky/bitbake/bin:/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/aiox-default/tmp/hosttools"; export HOME="/ede/smonsees"; /usr/bin/env wget -t 2 -T 30 --passive-ftp --no-check-certificate -P /disk0/scratch/smonsees/yocto/workspace_3/downloads/aarch64 'http://libarchive.org/downloads/libarchive-3.4.0.tar.gz' --progress=dot -v failed with exit code 3, output:
--2021-01-19 20:29:29-- http://libarchive.org/downloads/libarchive-3.4.0.tar.gz
Resolving libarchive.org (libarchive.org)... 99.84.191.71, 99.84.191.4, 99.84.191.92, ...
Connecting to libarchive.org (libarchive.org)|99.84.191.71|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 6908093 (6.6M) [application/octet-stream]
Saving to: ‘/disk0/scratch/smonsees/yocto/workspace_3/downloads/aarch64/libarchive-3.4.0.tar.gz’

0K ........ 0% 290 =31s

2021-01-19 20:30:10 (290 B/s) - Connection closed at byte 8875. Retrying.

--2021-01-19 20:30:11-- (try: 2) http://libarchive.org/downloads/libarchive-3.4.0.tar.gz
Connecting to libarchive.org (libarchive.org)|99.84.191.71|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 6908093 (6.6M) [application/octet-stream]
Saving to: ‘/disk0/scratch/smonsees/yocto/workspace_3/downloads/aarch64/libarchive-3.4.0.tar.gz’

0K 0% 0.00 =0.002s


Cannot write to ‘/disk0/scratch/smonsees/yocto/workspace_3/downloads/aarch64/libarchive-3.4.0.tar.gz’ (Success).

ERROR: libarchive-native-3.4.0-r0 do_fetch: Fetcher failure for URL: 'http://libarchive.org/downloads/libarchive-3.4.0.tar.gz'. Unable to fetch URL from any source.
ERROR: Logfile of failure stored in: /disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/aiox-default/tmp/work/x86_64-linux/libarchive-native/3.4.0-r0/temp/log.do_fetch.11418
ERROR: Task (virtual:native:/disk0/scratch/smonsees/yocto/workspace_3/poky/meta/recipes-extended/libarchive/libarchive_3.4.0.bb:do_fetch) failed with exit code '1'
ERROR: qemu-native-4.1.0-r0 do_fetch: Fetcher failure: Fetch command export PSEUDO_DISABLED=1; unset _PYTHON_SYSCONFIGDATA_NAME; export DBUS_SESSION_BUS_ADDRESS="unix:abstract=/tmp/dbus-WGeN1VUN5f,guid=5f3398a95f5d5aa6a98efe74600732fe"; export SSH_AGENT_PID="3519"; export SSH_AUTH_SOCK="/run/user/11588/keyring/ssh"; export PATH="/disk0/scratch/smonsees/yocto/workspace_3/poky/scripts/native-intercept:/disk0/scratch/smonsees/yocto/workspace_3/poky/scripts:/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/aiox-default/tmp/work/x86_64-linux/qemu-native/4.1.0-r0/recipe-sysroot-native/usr/bin/x86_64-linux:/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/aiox-default/tmp/work/x86_64-linux/qemu-native/4.1.0-r0/recipe-sysroot-native/usr/bin:/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/aiox-default/tmp/work/x86_64-linux/qemu-native/4.1.0-r0/recipe-sysroot-native/usr/sbin:/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/aiox-default/tmp/work/x86_64-linux/qemu-native/4.1.0-r0/recipe-sysroot-native/usr/bin:/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/aiox-default/tmp/work/x86_64-linux/qemu-native/4.1.0-r0/recipe-sysroot-native/sbin:/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/aiox-default/tmp/work/x86_64-linux/qemu-native/4.1.0-r0/recipe-sysroot-native/bin:/disk0/scratch/smonsees/yocto/workspace_3/poky/bitbake/bin:/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/aiox-default/tmp/hosttools"; export HOME="/ede/smonsees"; /usr/bin/env wget -t 2 -T 30 --passive-ftp --no-check-certificate -P /disk0/scratch/smonsees/yocto/workspace_3/downloads/aarch64 'https://download.qemu.org/qemu-4.1.0.tar.xz' --progress=dot -v failed with exit code 3, output:
--2021-01-19 20:28:05-- https://download.qemu.org/qemu-4.1.0.tar.xz
Resolving download.qemu.org (download.qemu.org)... 172.99.69.163
Connecting to download.qemu.org (download.qemu.org)|172.99.69.163|:443... connected.
WARNING: cannot verify download.qemu.org's certificate, issued by ‘/C=US/ST=VA/L=Reston/O=BAE Systems, Inc./OU=Cybersecurity Engineering/CN=BAEPROXY’:
Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 200 OK
Length: 54001708 (52M) [application/x-xz]
Saving to: ‘/disk0/scratch/smonsees/yocto/workspace_3/downloads/aarch64/qemu-4.1.0.tar.xz’

0K ........ 0% 188 =47s

2021-01-19 20:28:58 (188 B/s) - Connection closed at byte 8828. Retrying.

--2021-01-19 20:28:59-- (try: 2) https://download.qemu.org/qemu-4.1.0.tar.xz
Connecting to download.qemu.org (download.qemu.org)|172.99.69.163|:443... connected.
WARNING: cannot verify download.qemu.org's certificate, issued by ‘/C=US/ST=VA/L=Reston/O=BAE Systems, Inc./OU=Cybersecurity Engineering/CN=BAEPROXY’:
Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 200 OK
Length: 54001708 (52M) [application/x-xz]
Saving to: ‘/disk0/scratch/smonsees/yocto/workspace_3/downloads/aarch64/qemu-4.1.0.tar.xz’

0K 0% 0.00 =0s


Cannot write to ‘/disk0/scratch/smonsees/yocto/workspace_3/downloads/aarch64/qemu-4.1.0.tar.xz’ (Success).

ERROR: qemu-native-4.1.0-r0 do_fetch: Fetcher failure for URL: 'https://download.qemu.org/qemu-4.1.0.tar.xz'. Unable to fetch URL from any source.
ERROR: Logfile of failure stored in: /disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/aiox-default/tmp/work/x86_64-linux/qemu-native/4.1.0-r0/temp/log.do_fetch.11272
ERROR: Task (/disk0/scratch/smonsees/yocto/workspace_3/poky/meta/recipes-devtools/qemu/qemu-native_4.1.0.bb:do_fetch) failed with exit code '1'

-----Original Message-----
From: Monsees, Steven C (US) <steven.monsees@...>
Sent: Tuesday, January 19, 2021 3:22 PM
To: Monsees, Steven C (US) <steven.monsees@...>; Richard Purdie <richard.purdie@...>; yocto@...
Subject: RE: [yocto] #yocto #kernel BB_GENERATE_MIRROR_TARBALLS


All but the following downloaded except these which failed (timing out):

https://sourceware.org/elfutils/ftp/0.177/elfutils-0.177.tar.bz2
https://download.qemu.org/qemu-4.1.0.tar.xz
http://libarchive.org/downloads/libarchive-3.4.0.tar.gz

fyi... I am using zeus 3.0.4.

Steve

-----Original Message-----
From: yocto@... <yocto@...> On Behalf Of Monsees, Steven C (US) via lists.yoctoproject.org
Sent: Tuesday, January 19, 2021 9:25 AM
To: Richard Purdie <richard.purdie@...>; yocto@...
Subject: Re: [yocto] #yocto #kernel BB_GENERATE_MIRROR_TARBALLS

*** WARNING ***
EXTERNAL EMAIL -- This message originates from outside our organization.



From shell... I just checked the mega-manual, will rerun as described...

-----Original Message-----
From: Richard Purdie <richard.purdie@...>
Sent: Tuesday, January 19, 2021 9:13 AM
To: Monsees, Steven C (US) <steven.monsees@...>; yocto@...
Subject: Re: [yocto] #yocto #kernel BB_GENERATE_MIRROR_TARBALLS

*** WARNING ***
EXTERNAL EMAIL -- This message originates from outside our organization.


On Tue, 2021-01-19 at 14:08 +0000, Monsees, Steven C (US) wrote:
That was set : export BB_GENERATE_MIRROR_TARBALLS=1
Just to be clear, where did you set that? In a .conf file? If so, you don't need the export bit. If it was in a shell it probably wouldn't work.

Also, you didn't answer my other question, did you refetch everything with that set?

Cheers,

Richard


Richard Purdie
 

On Tue, 2021-01-19 at 20:22 +0000, Monsees, Steven C (US) wrote:
All but the following downloaded except these which failed (timing out):

 https://sourceware.org/elfutils/ftp/0.177/elfutils-0.177.tar.bz2
 https://download.qemu.org/qemu-4.1.0.tar.xz
 http://libarchive.org/downloads/libarchive-3.4.0.tar.gz

fyi... I am using zeus 3.0.4.
I did just check and these do seem to download ok here:

$ wget https://download.qemu.org/qemu-4.1.0.tar.xz
--2021-01-19 21:09:28-- https://download.qemu.org/qemu-4.1.0.tar.xz
Resolving download.qemu.org (download.qemu.org)... 172.99.69.163
Connecting to download.qemu.org (download.qemu.org)|172.99.69.163|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 54001708 (52M) [application/x-xz]
Saving to: ‘qemu-4.1.0.tar.xz’

qemu-4.1.0.tar.xz 100%[=======================================================================>] 51.50M 8.77MB/s in 6.3s

2021-01-19 21:09:35 (8.21 MB/s) - ‘qemu-4.1.0.tar.xz’ saved [54001708/54001708]

$ wget https://sourceware.org/elfutils/ftp/0.177/elfutils-0.177.tar.bz2
--2021-01-19 21:09:38-- https://sourceware.org/elfutils/ftp/0.177/elfutils-0.177.tar.bz2
Resolving sourceware.org (sourceware.org)... 2620:52:3:1:0:246e:9693:128c, 8.43.85.97
Connecting to sourceware.org (sourceware.org)|2620:52:3:1:0:246e:9693:128c|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 8852413 (8.4M) [application/x-bzip2]
Saving to: ‘elfutils-0.177.tar.bz2’

elfutils-0.177.tar.bz2 100%[=======================================================================>] 8.44M 5.87MB/s in 1.4s

2021-01-19 21:09:40 (5.87 MB/s) - ‘elfutils-0.177.tar.bz2’ saved [8852413/8852413]

$ wget http://libarchive.org/downloads/libarchive-3.4.0.tar.gz
--2021-01-19 21:09:46-- http://libarchive.org/downloads/libarchive-3.4.0.tar.gz
Resolving libarchive.org (libarchive.org)... 2600:9000:21c7:e600:16:e6b0:f440:93a1, 2600:9000:21c7:9000:16:e6b0:f440:93a1, 2600:9000:21c7:2e00:16:e6b0:f440:93a1, ...
Connecting to libarchive.org (libarchive.org)|2600:9000:21c7:e600:16:e6b0:f440:93a1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 6908093 (6.6M) [application/octet-stream]
Saving to: ‘libarchive-3.4.0.tar.gz’

libarchive-3.4.0.tar.gz 100%[=======================================================================>] 6.59M 8.25MB/s in 0.8s

2021-01-19 21:09:47 (8.25 MB/s) - ‘libarchive-3.4.0.tar.gz’ saved [6908093/6908093]

$ sha256sum libarchive-3.4.0.tar.gz elfutils-0.177.tar.bz2 qemu-4.1.0.tar.xz
8643d50ed40c759f5412a3af4e353cffbce4fdf3b5cf321cb72cacf06b2d825e libarchive-3.4.0.tar.gz
fa489deccbcae7d8c920f60d85906124c1989c591196d90e0fd668e3dc05042e elfutils-0.177.tar.bz2
656e60218689bdeec69903087fd7582d5d3e72238d02f4481d8dc6d79fd909c6 qemu-4.1.0.tar.xz

Cheers,

Richard


Konrad Weihmann <kweihmann@...>
 

Just a wild guess, but could your corporate proxy, which is kind of visible in the provided logs, intervene with this - to me it seems to be some MITM box.
Do you have any chance to try it without that proxy involved?

On 19.01.21 21:42, Monsees, Steven C (US) via lists.yoctoproject.org wrote:
This is the types of error I am seeing:
WARNING: qemu-native-4.1.0-r0 do_fetch: Failed to fetch URL https://download.qemu.org/qemu-4.1.0.tar.xz, attempting MIRRORS if available
WARNING: libarchive-native-3.4.0-r0 do_fetch: Failed to fetch URL http://libarchive.org/downloads/libarchive-3.4.0.tar.gz, attempting MIRRORS if available
ERROR: libarchive-native-3.4.0-r0 do_fetch: Fetcher failure: Fetch command export PSEUDO_DISABLED=1; unset _PYTHON_SYSCONFIGDATA_NAME; export DBUS_SESSION_BUS_ADDRESS="unix:abstract=/tmp/dbus-WGeN1VUN5f,guid=5f3398a95f5d5aa6a98efe74600732fe"; export SSH_AGENT_PID="3519"; export SSH_AUTH_SOCK="/run/user/11588/keyring/ssh"; export PATH="/disk0/scratch/smonsees/yocto/workspace_3/poky/scripts/native-intercept:/disk0/scratch/smonsees/yocto/workspace_3/poky/scripts:/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/aiox-default/tmp/work/x86_64-linux/libarchive-native/3.4.0-r0/recipe-sysroot-native/usr/bin/x86_64-linux:/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/aiox-default/tmp/work/x86_64-linux/libarchive-native/3.4.0-r0/recipe-sysroot-native/usr/bin:/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/aiox-default/tmp/work/x86_64-linux/libarchive-native/3.4.0-r0/recipe-sysroot-native/usr/sbin:/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/aiox-default/tmp/work/x86_64-linux/libarchive-native/3.4.0-r0/recipe-sysroot-native/usr/bin:/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/aiox-default/tmp/work/x86_64-linux/libarchive-native/3.4.0-r0/recipe-sysroot-native/sbin:/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/aiox-default/tmp/work/x86_64-linux/libarchive-native/3.4.0-r0/recipe-sysroot-native/bin:/disk0/scratch/smonsees/yocto/workspace_3/poky/bitbake/bin:/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/aiox-default/tmp/hosttools"; export HOME="/ede/smonsees"; /usr/bin/env wget -t 2 -T 30 --passive-ftp --no-check-certificate -P /disk0/scratch/smonsees/yocto/workspace_3/downloads/aarch64 'http://libarchive.org/downloads/libarchive-3.4.0.tar.gz' --progress=dot -v failed with exit code 3, output:
--2021-01-19 20:29:29-- http://libarchive.org/downloads/libarchive-3.4.0.tar.gz
Resolving libarchive.org (libarchive.org)... 99.84.191.71, 99.84.191.4, 99.84.191.92, ...
Connecting to libarchive.org (libarchive.org)|99.84.191.71|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 6908093 (6.6M) [application/octet-stream]
Saving to: ‘/disk0/scratch/smonsees/yocto/workspace_3/downloads/aarch64/libarchive-3.4.0.tar.gz’
0K ........ 0% 290 =31s
2021-01-19 20:30:10 (290 B/s) - Connection closed at byte 8875. Retrying.
--2021-01-19 20:30:11-- (try: 2) http://libarchive.org/downloads/libarchive-3.4.0.tar.gz
Connecting to libarchive.org (libarchive.org)|99.84.191.71|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 6908093 (6.6M) [application/octet-stream]
Saving to: ‘/disk0/scratch/smonsees/yocto/workspace_3/downloads/aarch64/libarchive-3.4.0.tar.gz’
0K 0% 0.00 =0.002s
Cannot write to ‘/disk0/scratch/smonsees/yocto/workspace_3/downloads/aarch64/libarchive-3.4.0.tar.gz’ (Success).
ERROR: libarchive-native-3.4.0-r0 do_fetch: Fetcher failure for URL: 'http://libarchive.org/downloads/libarchive-3.4.0.tar.gz'. Unable to fetch URL from any source.
ERROR: Logfile of failure stored in: /disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/aiox-default/tmp/work/x86_64-linux/libarchive-native/3.4.0-r0/temp/log.do_fetch.11418
ERROR: Task (virtual:native:/disk0/scratch/smonsees/yocto/workspace_3/poky/meta/recipes-extended/libarchive/libarchive_3.4.0.bb:do_fetch) failed with exit code '1'
ERROR: qemu-native-4.1.0-r0 do_fetch: Fetcher failure: Fetch command export PSEUDO_DISABLED=1; unset _PYTHON_SYSCONFIGDATA_NAME; export DBUS_SESSION_BUS_ADDRESS="unix:abstract=/tmp/dbus-WGeN1VUN5f,guid=5f3398a95f5d5aa6a98efe74600732fe"; export SSH_AGENT_PID="3519"; export SSH_AUTH_SOCK="/run/user/11588/keyring/ssh"; export PATH="/disk0/scratch/smonsees/yocto/workspace_3/poky/scripts/native-intercept:/disk0/scratch/smonsees/yocto/workspace_3/poky/scripts:/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/aiox-default/tmp/work/x86_64-linux/qemu-native/4.1.0-r0/recipe-sysroot-native/usr/bin/x86_64-linux:/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/aiox-default/tmp/work/x86_64-linux/qemu-native/4.1.0-r0/recipe-sysroot-native/usr/bin:/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/aiox-default/tmp/work/x86_64-linux/qemu-native/4.1.0-r0/recipe-sysroot-native/usr/sbin:/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/aiox-default/tmp/work/x86_64-linux/qemu-native/4.1.0-r0/recipe-sysroot-native/usr/bin:/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/aiox-default/tmp/work/x86_64-linux/qemu-native/4.1.0-r0/recipe-sysroot-native/sbin:/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/aiox-default/tmp/work/x86_64-linux/qemu-native/4.1.0-r0/recipe-sysroot-native/bin:/disk0/scratch/smonsees/yocto/workspace_3/poky/bitbake/bin:/disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/aiox-default/tmp/hosttools"; export HOME="/ede/smonsees"; /usr/bin/env wget -t 2 -T 30 --passive-ftp --no-check-certificate -P /disk0/scratch/smonsees/yocto/workspace_3/downloads/aarch64 'https://download.qemu.org/qemu-4.1.0.tar.xz' --progress=dot -v failed with exit code 3, output:
--2021-01-19 20:28:05-- https://download.qemu.org/qemu-4.1.0.tar.xz
Resolving download.qemu.org (download.qemu.org)... 172.99.69.163
Connecting to download.qemu.org (download.qemu.org)|172.99.69.163|:443... connected.
WARNING: cannot verify download.qemu.org's certificate, issued by ‘/C=US/ST=VA/L=Reston/O=BAE Systems, Inc./OU=Cybersecurity Engineering/CN=BAEPROXY’:
Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 200 OK
Length: 54001708 (52M) [application/x-xz]
Saving to: ‘/disk0/scratch/smonsees/yocto/workspace_3/downloads/aarch64/qemu-4.1.0.tar.xz’
0K ........ 0% 188 =47s
2021-01-19 20:28:58 (188 B/s) - Connection closed at byte 8828. Retrying.
--2021-01-19 20:28:59-- (try: 2) https://download.qemu.org/qemu-4.1.0.tar.xz
Connecting to download.qemu.org (download.qemu.org)|172.99.69.163|:443... connected.
WARNING: cannot verify download.qemu.org's certificate, issued by ‘/C=US/ST=VA/L=Reston/O=BAE Systems, Inc./OU=Cybersecurity Engineering/CN=BAEPROXY’:
Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 200 OK
Length: 54001708 (52M) [application/x-xz]
Saving to: ‘/disk0/scratch/smonsees/yocto/workspace_3/downloads/aarch64/qemu-4.1.0.tar.xz’
0K 0% 0.00 =0s
Cannot write to ‘/disk0/scratch/smonsees/yocto/workspace_3/downloads/aarch64/qemu-4.1.0.tar.xz’ (Success).
ERROR: qemu-native-4.1.0-r0 do_fetch: Fetcher failure for URL: 'https://download.qemu.org/qemu-4.1.0.tar.xz'. Unable to fetch URL from any source.
ERROR: Logfile of failure stored in: /disk0/scratch/smonsees/yocto/workspace_3/meta-bae/meta-limws/builds/aiox-default/tmp/work/x86_64-linux/qemu-native/4.1.0-r0/temp/log.do_fetch.11272
ERROR: Task (/disk0/scratch/smonsees/yocto/workspace_3/poky/meta/recipes-devtools/qemu/qemu-native_4.1.0.bb:do_fetch) failed with exit code '1'
-----Original Message-----
From: Monsees, Steven C (US) <steven.monsees@...>
Sent: Tuesday, January 19, 2021 3:22 PM
To: Monsees, Steven C (US) <steven.monsees@...>; Richard Purdie <richard.purdie@...>; yocto@...
Subject: RE: [yocto] #yocto #kernel BB_GENERATE_MIRROR_TARBALLS
All but the following downloaded except these which failed (timing out):
https://sourceware.org/elfutils/ftp/0.177/elfutils-0.177.tar.bz2
https://download.qemu.org/qemu-4.1.0.tar.xz
http://libarchive.org/downloads/libarchive-3.4.0.tar.gz
fyi... I am using zeus 3.0.4.
Steve
-----Original Message-----
From: yocto@... <yocto@...> On Behalf Of Monsees, Steven C (US) via lists.yoctoproject.org
Sent: Tuesday, January 19, 2021 9:25 AM
To: Richard Purdie <richard.purdie@...>; yocto@...
Subject: Re: [yocto] #yocto #kernel BB_GENERATE_MIRROR_TARBALLS
*** WARNING ***
EXTERNAL EMAIL -- This message originates from outside our organization.
From shell... I just checked the mega-manual, will rerun as described...
-----Original Message-----
From: Richard Purdie <richard.purdie@...>
Sent: Tuesday, January 19, 2021 9:13 AM
To: Monsees, Steven C (US) <steven.monsees@...>; yocto@...
Subject: Re: [yocto] #yocto #kernel BB_GENERATE_MIRROR_TARBALLS
*** WARNING ***
EXTERNAL EMAIL -- This message originates from outside our organization.
On Tue, 2021-01-19 at 14:08 +0000, Monsees, Steven C (US) wrote:
That was set : export BB_GENERATE_MIRROR_TARBALLS=1
Just to be clear, where did you set that? In a .conf file? If so, you don't need the export bit. If it was in a shell it probably wouldn't work.
Also, you didn't answer my other question, did you refetch everything with that set?
Cheers,
Richard