yocto@lists.yoctoproject.org | Yocto [thud], [zeus] do_fetch and do_unpack failures with offline/online svn build!

Georgi Georgiev

#48225

Hello Community,
This is the third time I am asking for support on this issue. This time I decided to use the web form.
In our project we have a requirement to be able to build the project offline. E.g. on the field without any network connection. When we are connected with the recipe mentioned below we don't have issues:

SUMMARY = "PACMan - Parameter And Configuration MANager"

LICENSE = "CLOSED"

inherit systemd useradd

REQUIRED_DISTRO_FEATURES = "systemd"

# SVN revision
PV = "121026"
# Name of SVN project
PACMAN_PROJ_NAME="SOCPACManEnvEngKeys"

SRC_URI = "svn://cocosubversion/svn/Embedded/Valve\&Actuator/DVPII/trunk/${PACMAN_PROJ_NAME};module=trunk;protocol=http;externals=allowed;rev=${PV}"
SRC_URI += "file://ww-authpacman.service"
SRC_URI += "file://ww-pacman.service"

S = "${WORKDIR}/trunk/"

# ${PN}-sys: content related to system, which goes to base rootfs (only .service file and symlinks)
# ${PN}: real content which may go to separate partition
PACKAGES =+ " ${PN}-sys"
.........
When disconnect the network, erase sstate-cache, cache and tmp I see log file attached (log.do_fetch.32757) and the following output:

ERROR: ww-pacman-121026-r0 do_fetch: Fetcher failure: Fetch command export PSEUDO_DISABLED=1; unset _PYTHON_SYSCONFIGDATA_NAME; export DBUS_SESSION_BUS_ADDRESS="unix:path=/run/user/1000/bus"; export SSH_AGENT_PID="11412"; export SSH_AUTH_SOCK="/run/user/1000/keyring/ssh"; export PATH="/home/w23698/projects/proj_dvp2/build_dvp2/tmp/sysroots-uninative/x86_64-linux/usr/bin:/home/w23698/projects/proj_dvp2/sources/poky/scripts:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/usr/bin/arm-poky-linux-gnueabi:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot/usr/bin/crossscripts:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/usr/sbin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/usr/bin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/sbin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/bin:/home/w23698/projects/proj_dvp2/sources/poky/bitbake/bin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/hosttools"; export HOME="/home/w23698"; /usr/bin/env svn --non-interactive --trust-server-cert update --no-auth-cache -r 121026 failed with exit code 1, output:
Updating '.':

svn: E170013: Unable to connect to a repository at URL 'http://cocosubversion/svn/Embedded/Valve&Actuator/DVPII/trunk/SOCPACManEnvEngKeys/trunk'
svn: E670003: Temporary failure in name resolution

ERROR: ww-pacman-121026-r0 do_fetch: Fetcher failure for URL: 'svn://cocosubversion/svn/Embedded/Valve\&Actuator/DVPII/trunk/SOCPACManEnvEngKeys;module=trunk;protocol=http;externals=allowed;rev=121026'. Unable to fetch URL from any source.
ERROR: Logfile of failure stored in: /home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/temp/log.do_fetch.32757
ERROR: Task (/home/w23698/projects/proj_dvp2/build_dvp2/../sources/meta-ww-dvp2/recipes-ww/ww-pacman/ww-pacman.bb:do_fetch) failed with exit code '1'

When remove the '\' character in SRC_URI, e.g. to become:

SRC_URI = "svn://cocosubversion/svn/Embedded/Valve&Actuator/DVPII/trunk/${PACMAN_PROJ_NAME};module=trunk;protocol=http;externals=allowed;rev=${PV}"

In connected and not connected to network do_fetch() passes successfully but I see one and same error (log.do_unpack.25226) output:

ERROR: ww-pacman-121026-r0 do_unpack: Unpack failure for URL: 'svn://cocosubversion/svn/Embedded/Valve&Actuator/DVPII/trunk/SOCPACManEnvEngKeys;module=trunk;protocol=http;externals=allowed;rev=121026'. Unpack command PATH="/home/w23698/projects/proj_dvp2/build_dvp2/tmp/sysroots-uninative/x86_64-linux/usr/bin:/home/w23698/projects/proj_dvp2/sources/poky/scripts:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/usr/bin/arm-poky-linux-gnueabi:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot/usr/bin/crossscripts:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/usr/sbin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/usr/bin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/sbin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/bin:/home/w23698/projects/proj_dvp2/sources/poky/bitbake/bin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/hosttools" tar xz --no-same-owner -f /home/w23698/projects/proj_dvp2/build_dvp2/downloads/trunk_cocosubversion_.svn.Embedded.Valve&Actuator.DVPII.trunk.SOCPACManEnvEngKeys_121026_.tar.gz failed with return value 127
ERROR: Logfile of failure stored in: /home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/temp/log.do_unpack.25226
ERROR: Task (/home/w23698/projects/proj_dvp2/build_dvp2/../sources/meta-ww-dvp2/recipes-ww/ww-pacman/ww-pacman.bb:do_unpack) failed with exit code '1'

I don't it this matters but, the build machine is baremetal Ubuntu 18.04.3 LTS. In all cases the packed tar.gz remain in downloads directory with one name!

Cordially,
Georgi

log.do_fetch.32757

log.do_unpack.25226

Mikko Rapeli

#48226

Hi,

I can't comment on svn fetcher but I use subversion via http
fetcher. This way the local download cache can have the exact same file
and builds can continue off line. One major gotcha is that all files
uploaded to svn must have unique and versioned file names. Updating the
same file to different content will result in using the old cache file version.

Maybe this helps,

-Mikko

Khem Raj

#48229

On 1/31/20 4:02 AM, georgi.georgiev via Lists.Yoctoproject.Org wrote:
Hello Community,
This is the third time I am asking for support on this issue. This time I decided to use the web form.
In our project we have a requirement to be able to build the project offline. E.g. on the field without any network connection. When we are connected with the recipe mentioned below we don't have issues:

svn: E170013: Unable to connect to a repository at URL 'http://cocosubversion/svn/Embedded/Valve&Actuator/DVPII/trunk/SOCPACManEnvEngKeys/trunk'
svn: E670003: Temporary failure in name resolution

can you try checking out the repo out side of fetcher and see if the machine can fetch the url you are using in SRC_URI ?

*SUMMARY = "PACMan - Parameter And Configuration MANager"*
*LICENSE = "CLOSED"*
*inherit systemd useradd*
*REQUIRED_DISTRO_FEATURES = "systemd"*
*# SVN revision*
*PV = "121026"*
*# Name of SVN project*
*PACMAN_PROJ_NAME="SOCPACManEnvEngKeys"*
*SRC_URI = "svn://cocosubversion/svn/Embedded/Valve\&Actuator/DVPII/trunk/${PACMAN_PROJ_NAME};module=trunk;protocol=http;externals=allowed;rev=${PV}"*
*SRC_URI += "file://ww-authpacman.service"*
*SRC_URI += "file://ww-pacman.service"*
*S = "${WORKDIR}/trunk/"*
*# ${PN}-sys: content related to system, which goes to base rootfs (only .service file and symlinks)*
*# ${PN}:      real content which may go to separate partition*
*PACKAGES =+ " ${PN}-sys"*
.........
When disconnect the network, erase sstate-cache, cache and tmp I see log file attached (log.do_fetch.32757) and the following output:
*ERROR: ww-pacman-121026-r0 do_fetch: Fetcher failure: Fetch command export PSEUDO_DISABLED=1; unset _PYTHON_SYSCONFIGDATA_NAME; export DBUS_SESSION_BUS_ADDRESS="unix:path=/run/user/1000/bus"; export SSH_AGENT_PID="11412"; export SSH_AUTH_SOCK="/run/user/1000/keyring/ssh"; export PATH="/home/w23698/projects/proj_dvp2/build_dvp2/tmp/sysroots-uninative/x86_64-linux/usr/bin:/home/w23698/projects/proj_dvp2/sources/poky/scripts:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/usr/bin/arm-poky-linux-gnueabi:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot/usr/bin/crossscripts:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/usr/sbin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/usr/bin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/sbin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/bin:/home/w23698/projects/proj_dvp2/sources/poky/bitbake/bin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/hosttools"; export HOME="/home/w23698"; /usr/bin/env svn --non-interactive --trust-server-cert update --no-auth-cache -r 121026 failed with exit code 1, output:*
*Updating '.':*
*svn: E170013: Unable to connect to a repository at URL 'http://cocosubversion/svn/Embedded/Valve&Actuator/DVPII/trunk/SOCPACManEnvEngKeys/trunk'*
*svn: E670003: Temporary failure in name resolution*
*ERROR: ww-pacman-121026-r0 do_fetch: Fetcher failure for URL: 'svn://cocosubversion/svn/Embedded/Valve\&Actuator/DVPII/trunk/SOCPACManEnvEngKeys;module=trunk;protocol=http;externals=allowed;rev=121026'. Unable to fetch URL from any source.*
*ERROR: Logfile of failure stored in: /home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/temp/log.do_fetch.32757*
*ERROR: Task (/home/w23698/projects/proj_dvp2/build_dvp2/../sources/meta-ww-dvp2/recipes-ww/ww-pacman/ww-pacman.bb:do_fetch) failed with exit code '1'*
When remove the '\' character in SRC_URI, e.g. to become:
*SRC_URI = "svn://cocosubversion/svn/Embedded/Valve&Actuator/DVPII/trunk/${PACMAN_PROJ_NAME};module=trunk;protocol=http;externals=allowed;rev=${PV}"*
In connected and not connected to network do_fetch() passes successfully but I see one and same error (log.do_unpack.25226) output:
*ERROR: ww-pacman-121026-r0 do_unpack: Unpack failure for URL: 'svn://cocosubversion/svn/Embedded/Valve&Actuator/DVPII/trunk/SOCPACManEnvEngKeys;module=trunk;protocol=http;externals=allowed;rev=121026'. Unpack command PATH="/home/w23698/projects/proj_dvp2/build_dvp2/tmp/sysroots-uninative/x86_64-linux/usr/bin:/home/w23698/projects/proj_dvp2/sources/poky/scripts:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/usr/bin/arm-poky-linux-gnueabi:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot/usr/bin/crossscripts:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/usr/sbin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/usr/bin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/sbin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/bin:/home/w23698/projects/proj_dvp2/sources/poky/bitbake/bin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/hosttools" tar xz --no-same-owner -f /home/w23698/projects/proj_dvp2/build_dvp2/downloads/trunk_cocosubversion_.svn.Embedded.Valve&Actuator.DVPII.trunk.SOCPACManEnvEngKeys_121026_.tar.gz failed with return value 127*
*ERROR: Logfile of failure stored in: /home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/temp/log.do_unpack.25226*
*ERROR: Task (/home/w23698/projects/proj_dvp2/build_dvp2/../sources/meta-ww-dvp2/recipes-ww/ww-pacman/ww-pacman.bb:do_unpack) failed with exit code '1'
*
I don't it this matters but, the build machine is baremetal Ubuntu 18.04.3 LTS. In all cases the packed tar.gz remain in downloads directory with one name!
Cordially,
Georgi

Georgi Georgiev

#48282

Sorry Khem,
With esc character '\' before & it can't take the full path. So briefly:

Yocto build with char '\' before & in SRC_URI:
	online - OK
	offline - Error - svn: E170013: Unable to connect to a repository...
Yocto build without '\' in SRC_URI:
	online and offline same error - do_unpack: Unpack failure for URL - "package".tar.gz is present in DL_DIR

Thanks

Show quoted text

-----Original Message-----
From: yocto@lists.yoctoproject.org [mailto:yocto@lists.yoctoproject.org] On Behalf Of Georgi Georgiev via Lists.Yoctoproject.Org
Sent: Wednesday, February 05, 2020 5:26 PM
To: Khem Raj <raj.khem@gmail.com>; yocto@lists.yoctoproject.org
Cc: yocto@lists.yoctoproject.org
Subject: Re: [EXTERNAL] Re: [yocto] Yocto [thud], [zeus] do_fetch and do_unpack failures with offline/online svn build! #yocto #python

Hi Khem,

Yes, no issues with that. When I am connected to network it fetches the code with and without escape character before & in the path.
svn co " https://urldefense.proofpoint.com/v2/url?u=http-3A__cocosubversion_svn_Embedded_Valve-26Actuator_DVPII_trunk_SOCPACManEnvEngKeys&d=DwIGaQ&c=y6L7g950KfMp92YmLM0QlMdXcRn6b-Cq4AApnSJOenE&r=kHtJrQGzfH1ZmfsNkJpYuH-jtNpv8yMDkqAmsRP99mc&m=vXia-xlcxuLr2fSmBogZpMIaOntXLKWv3mkxcRgtUc4&s=UYH9lueDIeMGZndMxgbZgb76A5EfqN-MR58DBv1iriI&e= " works fine and fetches the package. 

Georgi

-----Original Message-----
From: Khem Raj [mailto:raj.khem@gmail.com]
Sent: Friday, January 31, 2020 8:33 PM
To: Georgi Georgiev <Georgi.Georgiev@woodward.com>; yocto@lists.yoctoproject.org
Subject: [EXTERNAL] Re: [yocto] Yocto [thud], [zeus] do_fetch and do_unpack failures with offline/online svn build! #yocto #python

On 1/31/20 4:02 AM, georgi.georgiev via Lists.Yoctoproject.Org wrote:
Hello Community,
This is the third time I am asking for support on this issue. This 
time I decided to use the web form.
In our project we have a requirement to be able to build the project 
offline. E.g. on the field without any network connection. When we are 
connected with the recipe mentioned below we don't have issues:

svn: E170013: Unable to connect to a repository at URL 'https://urldefense.proofpoint.com/v2/url?u=http-3A__cocosubversion_svn_Embedded_Valve-26Actuator_DVPII_trunk_SOCPACManEnvEngKeys_trunk&d=DwIGaQ&c=y6L7g950KfMp92YmLM0QlMdXcRn6b-Cq4AApnSJOenE&r=kHtJrQGzfH1ZmfsNkJpYuH-jtNpv8yMDkqAmsRP99mc&m=ocx6BoVSrDQGWUvUUVvgKtuJbT7eH7jFSjCy1Ys73Vw&s=OTunnmsi_tCUt3JytmU2Hs7i7Xnqhl8-2CMsqmzIl90&e= '
svn: E670003: Temporary failure in name resolution

can you try checking out the repo out side of fetcher and see if the machine can fetch the url you are using in SRC_URI ?

*SUMMARY = "PACMan - Parameter And Configuration MANager"* *LICENSE =
"CLOSED"* *inherit systemd useradd* *REQUIRED_DISTRO_FEATURES =
"systemd"* *# SVN revision* *PV = "121026"* *# Name of SVN project*
*PACMAN_PROJ_NAME="SOCPACManEnvEngKeys"*
*SRC_URI =
"svn://cocosubversion/svn/Embedded/Valve%5C&Actuator/DVPII/trunk/$%7BP
ACMAN_PROJ_NAME%7D;module=trunk;protocol=http;externals=allowed;rev=$%
7BPV%7D%22* *SRC_URI += "file://ww-authpacman.service%22* *SRC_URI +=
"file://ww-pacman.service%22* *S = "${WORKDIR}/trunk/"* *# ${PN}-sys: 
content related to system, which goes to base rootfs (only .service 
file and symlinks)* *# ${PN}:      real content which may go to 
separate partition* *PACKAGES =+ " ${PN}-sys"* .........
When disconnect the network, erase sstate-cache, cache and tmp I see 
log file attached (log.do_fetch.32757) and the following output:

*ERROR: ww-pacman-121026-r0 do_fetch: Fetcher failure: Fetch command 
export PSEUDO_DISABLED=1; unset _PYTHON_SYSCONFIGDATA_NAME; export 
DBUS_SESSION_BUS_ADDRESS="unix:path=/run/user/1000/bus"; export 
SSH_AGENT_PID="11412"; export 
SSH_AUTH_SOCK="/run/user/1000/keyring/ssh"; export 
PATH="/home/w23698/projects/proj_dvp2/build_dvp2/tmp/sysroots-uninativ
e/x86_64-linux/usr/bin:/home/w23698/projects/proj_dvp2/sources/poky/sc
ripts:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-n
eon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/usr/b
in/arm-poky-linux-gnueabi:/home/w23698/projects/proj_dvp2/build_dvp2/t
mp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-
sysroot/usr/bin/crossscripts:/home/w23698/projects/proj_dvp2/build_dvp
2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/reci
pe-sysroot-native/usr/sbin:/home/w23698/projects/proj_dvp2/build_dvp2/
tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe
-sysroot-native/usr/bin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp
/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sy
sroot-native/sbin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/
cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-
native/bin:/home/w23698/projects/proj_dvp2/sources/poky/bitbake/bin:/h
ome/w23698/projects/proj_dvp2/build_dvp2/tmp/hosttools";
export HOME="/home/w23698"; /usr/bin/env svn --non-interactive 
--trust-server-cert update --no-auth-cache -r 121026 failed with exit 
code 1, output:* *Updating '.':*
*svn: E170013: Unable to connect to a repository at URL 
'https://urldefense.proofpoint.com/v2/url?u=http-3A__cocosubversion_sv
n_Embedded_Valve-26Actuator_DVPII_trunk_SOCPACManEnvEngKeys_trunk-27-2
A&d=DwIGaQ&c=y6L7g950KfMp92YmLM0QlMdXcRn6b-Cq4AApnSJOenE&r=kHtJrQGzfH1
ZmfsNkJpYuH-jtNpv8yMDkqAmsRP99mc&m=ocx6BoVSrDQGWUvUUVvgKtuJbT7eH7jFSjC
y1Ys73Vw&s=Ffv1JU1QYBh4g49fmoLnsDSFgMMBc_5MbOpy59QUS18&e=
*svn: E670003: Temporary failure in name resolution*
*ERROR: ww-pacman-121026-r0 do_fetch: Fetcher failure for URL: 
'svn://cocosubversion/svn/Embedded/Valve%5C&Actuator/DVPII/trunk/SOCPACManEnvEngKeys;module=trunk;protocol=http;externals=allowed;rev=121026'. 
Unable to fetch URL from any source.*
*ERROR: Logfile of failure stored in: 
/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-po
ky-linux-gnueabi/ww-pacman/121026-r0/temp/log.do_fetch.32757*
*ERROR: Task
(/home/w23698/projects/proj_dvp2/build_dvp2/../sources/meta-ww-dvp2/re
cipes-ww/ww-pacman/ww-pacman.bb:do_fetch)
failed with exit code '1'*

When remove the '\' character in SRC_URI, e.g. to become:

*SRC_URI =
"svn://cocosubversion/svn/Embedded/Valve&Actuator/DVPII/trunk/$%7BPACM
AN_PROJ_NAME%7D;module=trunk;protocol=http;externals=allowed;rev=$%7BP
V%7D%22*

In connected and not connected to network do_fetch() passes 
successfully but I see one and same error (log.do_unpack.25226) output:

*ERROR: ww-pacman-121026-r0 do_unpack: Unpack failure for URL: 
'svn://cocosubversion/svn/Embedded/Valve&Actuator/DVPII/trunk/SOCPACManEnvEngKeys;module=trunk;protocol=http;externals=allowed;rev=121026'. 
Unpack command
PATH="/home/w23698/projects/proj_dvp2/build_dvp2/tmp/sysroots-uninative/x86_64-linux/usr/bin:/home/w23698/projects/proj_dvp2/sources/poky/scripts:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/usr/bin/arm-poky-linux-gnueabi:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot/usr/bin/crossscripts:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/usr/sbin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/usr/bin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/sbin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/bin:/home/w23698/projects/proj_dvp2/sources/poky/bitbake/bin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/hosttools" 
tar xz --no-same-owner -f
/home/w23698/projects/proj_dvp2/build_dvp2/downloads/trunk_cocosubvers
ion_.svn.Embedded.Valve&Actuator.DVPII.trunk.SOCPACManEnvEngKeys_12102
6_.tar.gz
failed with return value 127*
*ERROR: Logfile of failure stored in: 
/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-po
ky-linux-gnueabi/ww-pacman/121026-r0/temp/log.do_unpack.25226*
*ERROR: Task
(/home/w23698/projects/proj_dvp2/build_dvp2/../sources/meta-ww-dvp2/re
cipes-ww/ww-pacman/ww-pacman.bb:do_unpack)
failed with exit code '1'
*
I don't it this matters but, the build machine is baremetal Ubuntu
18.04.3 LTS. In all cases the packed tar.gz remain in downloads 
directory with one name!

Cordially,
Georgi

***
The information in this email is confidential and intended solely for the individual or entity to whom it is addressed.  If you have received this email in error please notify the sender by return e-mail, delete this email, and refrain from any disclosure or action based on the information.
***

***
The information in this email is confidential and intended solely for the individual or entity to whom it is addressed.  If you have received this email in error please notify the sender by return e-mail, delete this email, and refrain from any disclosure or action based on the information.
***

Georgi Georgiev

#48451

Another serious issue with svn fetcher, always occurring when doing incremental builds. I am putting it here because I think both issues are related. Please suggest what to do. The error we get is shown below. This time it happens always. The only workaround is to clean and build. As soon as we make modification of the recipe (PV change only) it fails:

ERROR: ww-pacman-121204-r0 do_fetch: Error executing a python function in exec_python_func() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:base_do_fetch(d)
     0003:
File: '/data/home/w23698/projects/proj/build/../sources/poky/meta/classes/base.bbclass', lineno: 163, function: base_do_fetch
     0159:        return
     0160:
     0161:    try:
     0162:        fetcher = bb.fetch2.Fetch(src_uri, d)
 *** 0163:        fetcher.download()
     0164:    except bb.fetch2.BBFetchException as e:
     0165:        bb.fatal(str(e))
     0166:}
     0167:
File: '/data/home/w23698/projects/proj/sources/poky/bitbake/lib/bb/fetch2/__init__.py', lineno: 1678, function: download
     1674:                    try:
     1675:                        if not trusted_network(self.d, ud.url):
     1676:                            raise UntrustedUrl(ud.url)
     1677:                        logger.debug(1, "Trying Upstream")
 *** 1678:                        m.download(ud, self.d)
     1679:                        if hasattr(m, "build_mirror_data"):
     1680:                            m.build_mirror_data(ud, self.d)
     1681:                        localpath = ud.localpath
     1682:                        # early checksum verify, so that if checksum mismatched,
File: '/data/home/w23698/projects/proj/sources/poky/bitbake/lib/bb/fetch2/svn.py', lineno: 150, function: download
     0146:            if not ("externals" in ud.parm and ud.parm["externals"] == "nowarn"):
     0147:                # Warn the user if this had externals (won't catch them all)
     0148:                output = runfetchcmd("svn propget svn:externals || true", d, workdir=ud.moddir)
     0149:                if output:
 *** 0150:                    if "--ignore-externals" in svnfetchcmd.split():
     0151:                        bb.warn("%s contains svn:externals." % ud.url)
     0152:                        bb.warn("These should be added to the recipe SRC_URI as necessary.")
     0153:                        bb.warn("svn fetch has ignored externals:\n%s" % output)
     0154:                        bb.warn("To disable this warning add ';externals=nowarn' to the url.")
Exception: UnboundLocalError: local variable 'svnfetchcmd' referenced before assignment

ERROR: Logfile of failure stored in: /data/home/w23698/projects/proj/build/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121204-r0/temp/log.do_fetch.18853

Show quoted text

-----Original Message-----
From: Georgi Georgiev 
Sent: Wednesday, February 05, 2020 6:57 PM
To: Georgi Georgiev <Georgi.Georgiev@woodward.com>; Khem Raj <raj.khem@gmail.com>; yocto@lists.yoctoproject.org
Subject: Re: [yocto] Yocto [thud], [zeus] do_fetch and do_unpack failures with offline/online svn build! #yocto #python

Sorry Khem,
With esc character '\' before & it can't take the full path. So briefly:

Yocto build with char '\' before & in SRC_URI:
	online - OK
	offline - Error - svn: E170013: Unable to connect to a repository...
Yocto build without '\' in SRC_URI:
	online and offline same error - do_unpack: Unpack failure for URL - "package".tar.gz is present in DL_DIR

Thanks

-----Original Message-----
From: yocto@lists.yoctoproject.org [mailto:yocto@lists.yoctoproject.org] On Behalf Of Georgi Georgiev via Lists.Yoctoproject.Org
Sent: Wednesday, February 05, 2020 5:26 PM
To: Khem Raj <raj.khem@gmail.com>; yocto@lists.yoctoproject.org
Cc: yocto@lists.yoctoproject.org
Subject: Re: [EXTERNAL] Re: [yocto] Yocto [thud], [zeus] do_fetch and do_unpack failures with offline/online svn build! #yocto #python

Hi Khem,

Yes, no issues with that. When I am connected to network it fetches the code with and without escape character before & in the path.
svn co " https://urldefense.proofpoint.com/v2/url?u=http-3A__cocosubversion_svn_Embedded_Valve-26Actuator_DVPII_trunk_SOCPACManEnvEngKeys&d=DwIGaQ&c=y6L7g950KfMp92YmLM0QlMdXcRn6b-Cq4AApnSJOenE&r=kHtJrQGzfH1ZmfsNkJpYuH-jtNpv8yMDkqAmsRP99mc&m=vXia-xlcxuLr2fSmBogZpMIaOntXLKWv3mkxcRgtUc4&s=UYH9lueDIeMGZndMxgbZgb76A5EfqN-MR58DBv1iriI&e= " works fine and fetches the package. 

Georgi

-----Original Message-----
From: Khem Raj [mailto:raj.khem@gmail.com]
Sent: Friday, January 31, 2020 8:33 PM
To: Georgi Georgiev <Georgi.Georgiev@woodward.com>; yocto@lists.yoctoproject.org
Subject: [EXTERNAL] Re: [yocto] Yocto [thud], [zeus] do_fetch and do_unpack failures with offline/online svn build! #yocto #python

On 1/31/20 4:02 AM, georgi.georgiev via Lists.Yoctoproject.Org wrote:
Hello Community,
This is the third time I am asking for support on this issue. This 
time I decided to use the web form.
In our project we have a requirement to be able to build the project 
offline. E.g. on the field without any network connection. When we are 
connected with the recipe mentioned below we don't have issues:

svn: E170013: Unable to connect to a repository at URL 'https://urldefense.proofpoint.com/v2/url?u=http-3A__cocosubversion_svn_Embedded_Valve-26Actuator_DVPII_trunk_SOCPACManEnvEngKeys_trunk&d=DwIGaQ&c=y6L7g950KfMp92YmLM0QlMdXcRn6b-Cq4AApnSJOenE&r=kHtJrQGzfH1ZmfsNkJpYuH-jtNpv8yMDkqAmsRP99mc&m=ocx6BoVSrDQGWUvUUVvgKtuJbT7eH7jFSjCy1Ys73Vw&s=OTunnmsi_tCUt3JytmU2Hs7i7Xnqhl8-2CMsqmzIl90&e= '
svn: E670003: Temporary failure in name resolution

can you try checking out the repo out side of fetcher and see if the machine can fetch the url you are using in SRC_URI ?

*SUMMARY = "PACMan - Parameter And Configuration MANager"* *LICENSE =
"CLOSED"* *inherit systemd useradd* *REQUIRED_DISTRO_FEATURES =
"systemd"* *# SVN revision* *PV = "121026"* *# Name of SVN project*
*PACMAN_PROJ_NAME="SOCPACManEnvEngKeys"*
*SRC_URI =
"svn://cocosubversion/svn/Embedded/Valve%5C&Actuator/DVPII/trunk/$%7BP
ACMAN_PROJ_NAME%7D;module=trunk;protocol=http;externals=allowed;rev=$%
7BPV%7D%22* *SRC_URI += "file://ww-authpacman.service%22* *SRC_URI +=
"file://ww-pacman.service%22* *S = "${WORKDIR}/trunk/"* *# ${PN}-sys: 
content related to system, which goes to base rootfs (only .service 
file and symlinks)* *# ${PN}:      real content which may go to 
separate partition* *PACKAGES =+ " ${PN}-sys"* .........
When disconnect the network, erase sstate-cache, cache and tmp I see 
log file attached (log.do_fetch.32757) and the following output:

*ERROR: ww-pacman-121026-r0 do_fetch: Fetcher failure: Fetch command 
export PSEUDO_DISABLED=1; unset _PYTHON_SYSCONFIGDATA_NAME; export 
DBUS_SESSION_BUS_ADDRESS="unix:path=/run/user/1000/bus"; export 
SSH_AGENT_PID="11412"; export 
SSH_AUTH_SOCK="/run/user/1000/keyring/ssh"; export 
PATH="/home/w23698/projects/proj_dvp2/build_dvp2/tmp/sysroots-uninativ
e/x86_64-linux/usr/bin:/home/w23698/projects/proj_dvp2/sources/poky/sc
ripts:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-n
eon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/usr/b
in/arm-poky-linux-gnueabi:/home/w23698/projects/proj_dvp2/build_dvp2/t
mp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-
sysroot/usr/bin/crossscripts:/home/w23698/projects/proj_dvp2/build_dvp
2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/reci
pe-sysroot-native/usr/sbin:/home/w23698/projects/proj_dvp2/build_dvp2/
tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe
-sysroot-native/usr/bin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp
/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sy
sroot-native/sbin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/
cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-
native/bin:/home/w23698/projects/proj_dvp2/sources/poky/bitbake/bin:/h
ome/w23698/projects/proj_dvp2/build_dvp2/tmp/hosttools";
export HOME="/home/w23698"; /usr/bin/env svn --non-interactive 
--trust-server-cert update --no-auth-cache -r 121026 failed with exit 
code 1, output:* *Updating '.':*
*svn: E170013: Unable to connect to a repository at URL 
'https://urldefense.proofpoint.com/v2/url?u=http-3A__cocosubversion_sv
n_Embedded_Valve-26Actuator_DVPII_trunk_SOCPACManEnvEngKeys_trunk-27-2
A&d=DwIGaQ&c=y6L7g950KfMp92YmLM0QlMdXcRn6b-Cq4AApnSJOenE&r=kHtJrQGzfH1
ZmfsNkJpYuH-jtNpv8yMDkqAmsRP99mc&m=ocx6BoVSrDQGWUvUUVvgKtuJbT7eH7jFSjC
y1Ys73Vw&s=Ffv1JU1QYBh4g49fmoLnsDSFgMMBc_5MbOpy59QUS18&e=
*svn: E670003: Temporary failure in name resolution*
*ERROR: ww-pacman-121026-r0 do_fetch: Fetcher failure for URL: 
'svn://cocosubversion/svn/Embedded/Valve%5C&Actuator/DVPII/trunk/SOCPACManEnvEngKeys;module=trunk;protocol=http;externals=allowed;rev=121026'. 
Unable to fetch URL from any source.*
*ERROR: Logfile of failure stored in: 
/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-po
ky-linux-gnueabi/ww-pacman/121026-r0/temp/log.do_fetch.32757*
*ERROR: Task
(/home/w23698/projects/proj_dvp2/build_dvp2/../sources/meta-ww-dvp2/re
cipes-ww/ww-pacman/ww-pacman.bb:do_fetch)
failed with exit code '1'*

When remove the '\' character in SRC_URI, e.g. to become:

*SRC_URI =
"svn://cocosubversion/svn/Embedded/Valve&Actuator/DVPII/trunk/$%7BPACM
AN_PROJ_NAME%7D;module=trunk;protocol=http;externals=allowed;rev=$%7BP
V%7D%22*

In connected and not connected to network do_fetch() passes 
successfully but I see one and same error (log.do_unpack.25226) output:

*ERROR: ww-pacman-121026-r0 do_unpack: Unpack failure for URL: 
'svn://cocosubversion/svn/Embedded/Valve&Actuator/DVPII/trunk/SOCPACManEnvEngKeys;module=trunk;protocol=http;externals=allowed;rev=121026'. 
Unpack command
PATH="/home/w23698/projects/proj_dvp2/build_dvp2/tmp/sysroots-uninative/x86_64-linux/usr/bin:/home/w23698/projects/proj_dvp2/sources/poky/scripts:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/usr/bin/arm-poky-linux-gnueabi:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot/usr/bin/crossscripts:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/usr/sbin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/usr/bin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/sbin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/bin:/home/w23698/projects/proj_dvp2/sources/poky/bitbake/bin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/hosttools" 
tar xz --no-same-owner -f
/home/w23698/projects/proj_dvp2/build_dvp2/downloads/trunk_cocosubvers
ion_.svn.Embedded.Valve&Actuator.DVPII.trunk.SOCPACManEnvEngKeys_12102
6_.tar.gz
failed with return value 127*
*ERROR: Logfile of failure stored in: 
/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-po
ky-linux-gnueabi/ww-pacman/121026-r0/temp/log.do_unpack.25226*
*ERROR: Task
(/home/w23698/projects/proj_dvp2/build_dvp2/../sources/meta-ww-dvp2/re
cipes-ww/ww-pacman/ww-pacman.bb:do_unpack)
failed with exit code '1'
*
I don't it this matters but, the build machine is baremetal Ubuntu
18.04.3 LTS. In all cases the packed tar.gz remain in downloads 
directory with one name!

Cordially,
Georgi

***
The information in this email is confidential and intended solely for the individual or entity to whom it is addressed.  If you have received this email in error please notify the sender by return e-mail, delete this email, and refrain from any disclosure or action based on the information.
***

***
The information in this email is confidential and intended solely for the individual or entity to whom it is addressed.  If you have received this email in error please notify the sender by return e-mail, delete this email, and refrain from any disclosure or action based on the information.
***

Mikko Rapeli

#48452

On Mon, Feb 17, 2020 at 12:56:13PM +0000, Georgi Georgiev via Lists.Yoctoproject.Org wrote:
Another serious issue with svn fetcher, always occurring when doing incremental builds. I am putting it here because I think both issues are related. Please suggest what to do. The error we get is shown below. This time it happens always. The only workaround is to clean and build. As soon as we make modification of the recipe (PV change only) it fails:

ERROR: ww-pacman-121204-r0 do_fetch: Error executing a python function in exec_python_func() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:base_do_fetch(d)
     0003:
File: '/data/home/w23698/projects/proj/build/../sources/poky/meta/classes/base.bbclass', lineno: 163, function: base_do_fetch
     0159:        return
     0160:
     0161:    try:
     0162:        fetcher = bb.fetch2.Fetch(src_uri, d)
 *** 0163:        fetcher.download()
     0164:    except bb.fetch2.BBFetchException as e:
     0165:        bb.fatal(str(e))
     0166:}
     0167:
File: '/data/home/w23698/projects/proj/sources/poky/bitbake/lib/bb/fetch2/__init__.py', lineno: 1678, function: download
     1674:                    try:
     1675:                        if not trusted_network(self.d, ud.url):
     1676:                            raise UntrustedUrl(ud.url)
     1677:                        logger.debug(1, "Trying Upstream")
 *** 1678:                        m.download(ud, self.d)
     1679:                        if hasattr(m, "build_mirror_data"):
     1680:                            m.build_mirror_data(ud, self.d)
     1681:                        localpath = ud.localpath
     1682:                        # early checksum verify, so that if checksum mismatched,
File: '/data/home/w23698/projects/proj/sources/poky/bitbake/lib/bb/fetch2/svn.py', lineno: 150, function: download
     0146:            if not ("externals" in ud.parm and ud.parm["externals"] == "nowarn"):
     0147:                # Warn the user if this had externals (won't catch them all)
     0148:                output = runfetchcmd("svn propget svn:externals || true", d, workdir=ud.moddir)
     0149:                if output:
 *** 0150:                    if "--ignore-externals" in svnfetchcmd.split():
     0151:                        bb.warn("%s contains svn:externals." % ud.url)
     0152:                        bb.warn("These should be added to the recipe SRC_URI as necessary.")
     0153:                        bb.warn("svn fetch has ignored externals:\n%s" % output)
     0154:                        bb.warn("To disable this warning add ';externals=nowarn' to the url.")
Exception: UnboundLocalError: local variable 'svnfetchcmd' referenced before assignment

ERROR: Logfile of failure stored in: /data/home/w23698/projects/proj/build/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121204-r0/temp/log.do_fetch.18853Hmm. Rings a bell. I run builds with a local non-upstreamed patch:

Author:     Mikko Rapeli <mikko.rapeli@bmw.de>
AuthorDate: Fri Sep 6 14:15:20 2019 +0200
Commit:     Mikko Rapeli <mikko.rapeli@bmw.de>
CommitDate: Fri Sep 6 14:25:15 2019 +0200

    svn fetcher: allow "svn propget svn:externals" to fail

    Not all servers and repositories have this property set
    which results in failures like this when actual svn checkout
    command succeeded:

    svn: warning: W200017: Property 'svn:externals' not found on ''
    svn: E200000: A problem occurred; see other errors for details

Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de>

--- a/bitbake/lib/bb/fetch2/svn.py
+++ b/bitbake/lib/bb/fetch2/svn.py
@@ -145,7 +145,7 @@ class Svn(FetchMethod):
 
                 if not ("externals" in ud.parm and ud.parm["externals"] == "nowarn"):
                     # Warn the user if this had externals (won't catch them all)
-                    output = runfetchcmd("svn propget svn:externals", d, workdir=ud.moddir)
+                    output = runfetchcmd("svn propget svn:externals || true", d, workdir=ud.moddir)
                     if output:
                         if "--ignore-externals" in svnfetchcmd.split():
                             bb.warn("%s contains svn:externals." % ud.url)

I think this would help.

Cheers,

-Mikko


-----Original Message-----
From: Georgi Georgiev 
Sent: Wednesday, February 05, 2020 6:57 PM
To: Georgi Georgiev <Georgi.Georgiev@woodward.com>; Khem Raj <raj.khem@gmail.com>; yocto@lists.yoctoproject.org
Subject: Re: [yocto] Yocto [thud], [zeus] do_fetch and do_unpack failures with offline/online svn build! #yocto #python

Sorry Khem,
With esc character '\' before & it can't take the full path. So briefly:

Yocto build with char '\' before & in SRC_URI:
	online - OK
	offline - Error - svn: E170013: Unable to connect to a repository...
Yocto build without '\' in SRC_URI:
	online and offline same error - do_unpack: Unpack failure for URL - "package".tar.gz is present in DL_DIR

Thanks



-----Original Message-----
From: yocto@lists.yoctoproject.org [mailto:yocto@lists.yoctoproject.org] On Behalf Of Georgi Georgiev via Lists.Yoctoproject.Org
Sent: Wednesday, February 05, 2020 5:26 PM
To: Khem Raj <raj.khem@gmail.com>; yocto@lists.yoctoproject.org
Cc: yocto@lists.yoctoproject.org
Subject: Re: [EXTERNAL] Re: [yocto] Yocto [thud], [zeus] do_fetch and do_unpack failures with offline/online svn build! #yocto #python

Hi Khem,

Yes, no issues with that. When I am connected to network it fetches the code with and without escape character before & in the path.
svn co " https://urldefense.proofpoint.com/v2/url?u=http-3A__cocosubversion_svn_Embedded_Valve-26Actuator_DVPII_trunk_SOCPACManEnvEngKeys&d=DwIGaQ&c=y6L7g950KfMp92YmLM0QlMdXcRn6b-Cq4AApnSJOenE&r=kHtJrQGzfH1ZmfsNkJpYuH-jtNpv8yMDkqAmsRP99mc&m=vXia-xlcxuLr2fSmBogZpMIaOntXLKWv3mkxcRgtUc4&s=UYH9lueDIeMGZndMxgbZgb76A5EfqN-MR58DBv1iriI&e= " works fine and fetches the package. 

Georgi

-----Original Message-----
From: Khem Raj [mailto:raj.khem@gmail.com]
Sent: Friday, January 31, 2020 8:33 PM
To: Georgi Georgiev <Georgi.Georgiev@woodward.com>; yocto@lists.yoctoproject.org
Subject: [EXTERNAL] Re: [yocto] Yocto [thud], [zeus] do_fetch and do_unpack failures with offline/online svn build! #yocto #python

On 1/31/20 4:02 AM, georgi.georgiev via Lists.Yoctoproject.Org wrote:
Hello Community,
This is the third time I am asking for support on this issue. This 
time I decided to use the web form.
In our project we have a requirement to be able to build the project 
offline. E.g. on the field without any network connection. When we are 
connected with the recipe mentioned below we don't have issues:

svn: E170013: Unable to connect to a repository at URL 'https://urldefense.proofpoint.com/v2/url?u=http-3A__cocosubversion_svn_Embedded_Valve-26Actuator_DVPII_trunk_SOCPACManEnvEngKeys_trunk&d=DwIGaQ&c=y6L7g950KfMp92YmLM0QlMdXcRn6b-Cq4AApnSJOenE&r=kHtJrQGzfH1ZmfsNkJpYuH-jtNpv8yMDkqAmsRP99mc&m=ocx6BoVSrDQGWUvUUVvgKtuJbT7eH7jFSjCy1Ys73Vw&s=OTunnmsi_tCUt3JytmU2Hs7i7Xnqhl8-2CMsqmzIl90&e= '
svn: E670003: Temporary failure in name resolution

can you try checking out the repo out side of fetcher and see if the machine can fetch the url you are using in SRC_URI ?

*SUMMARY = "PACMan - Parameter And Configuration MANager"* *LICENSE =
"CLOSED"* *inherit systemd useradd* *REQUIRED_DISTRO_FEATURES =
"systemd"* *# SVN revision* *PV = "121026"* *# Name of SVN project*
*PACMAN_PROJ_NAME="SOCPACManEnvEngKeys"*
*SRC_URI =
"svn://cocosubversion/svn/Embedded/Valve%5C&Actuator/DVPII/trunk/$%7BP
ACMAN_PROJ_NAME%7D;module=trunk;protocol=http;externals=allowed;rev=$%
7BPV%7D%22* *SRC_URI += "file://ww-authpacman.service%22* *SRC_URI +=
"file://ww-pacman.service%22* *S = "${WORKDIR}/trunk/"* *# ${PN}-sys: 
content related to system, which goes to base rootfs (only .service 
file and symlinks)* *# ${PN}:      real content which may go to 
separate partition* *PACKAGES =+ " ${PN}-sys"* .........
When disconnect the network, erase sstate-cache, cache and tmp I see 
log file attached (log.do_fetch.32757) and the following output:

*ERROR: ww-pacman-121026-r0 do_fetch: Fetcher failure: Fetch command 
export PSEUDO_DISABLED=1; unset _PYTHON_SYSCONFIGDATA_NAME; export 
DBUS_SESSION_BUS_ADDRESS="unix:path=/run/user/1000/bus"; export 
SSH_AGENT_PID="11412"; export 
SSH_AUTH_SOCK="/run/user/1000/keyring/ssh"; export 
PATH="/home/w23698/projects/proj_dvp2/build_dvp2/tmp/sysroots-uninativ
e/x86_64-linux/usr/bin:/home/w23698/projects/proj_dvp2/sources/poky/sc
ripts:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-n
eon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/usr/b
in/arm-poky-linux-gnueabi:/home/w23698/projects/proj_dvp2/build_dvp2/t
mp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-
sysroot/usr/bin/crossscripts:/home/w23698/projects/proj_dvp2/build_dvp
2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/reci
pe-sysroot-native/usr/sbin:/home/w23698/projects/proj_dvp2/build_dvp2/
tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe
-sysroot-native/usr/bin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp
/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sy
sroot-native/sbin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/
cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-
native/bin:/home/w23698/projects/proj_dvp2/sources/poky/bitbake/bin:/h
ome/w23698/projects/proj_dvp2/build_dvp2/tmp/hosttools";
export HOME="/home/w23698"; /usr/bin/env svn --non-interactive 
--trust-server-cert update --no-auth-cache -r 121026 failed with exit 
code 1, output:* *Updating '.':*
*svn: E170013: Unable to connect to a repository at URL 
'https://urldefense.proofpoint.com/v2/url?u=http-3A__cocosubversion_sv
n_Embedded_Valve-26Actuator_DVPII_trunk_SOCPACManEnvEngKeys_trunk-27-2
A&d=DwIGaQ&c=y6L7g950KfMp92YmLM0QlMdXcRn6b-Cq4AApnSJOenE&r=kHtJrQGzfH1
ZmfsNkJpYuH-jtNpv8yMDkqAmsRP99mc&m=ocx6BoVSrDQGWUvUUVvgKtuJbT7eH7jFSjC
y1Ys73Vw&s=Ffv1JU1QYBh4g49fmoLnsDSFgMMBc_5MbOpy59QUS18&e=
*svn: E670003: Temporary failure in name resolution*
*ERROR: ww-pacman-121026-r0 do_fetch: Fetcher failure for URL: 
'svn://cocosubversion/svn/Embedded/Valve%5C&Actuator/DVPII/trunk/SOCPACManEnvEngKeys;module=trunk;protocol=http;externals=allowed;rev=121026'. 
Unable to fetch URL from any source.*
*ERROR: Logfile of failure stored in: 
/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-po
ky-linux-gnueabi/ww-pacman/121026-r0/temp/log.do_fetch.32757*
*ERROR: Task
(/home/w23698/projects/proj_dvp2/build_dvp2/../sources/meta-ww-dvp2/re
cipes-ww/ww-pacman/ww-pacman.bb:do_fetch)
failed with exit code '1'*

When remove the '\' character in SRC_URI, e.g. to become:

*SRC_URI =
"svn://cocosubversion/svn/Embedded/Valve&Actuator/DVPII/trunk/$%7BPACM
AN_PROJ_NAME%7D;module=trunk;protocol=http;externals=allowed;rev=$%7BP
V%7D%22*

In connected and not connected to network do_fetch() passes 
successfully but I see one and same error (log.do_unpack.25226) output:

*ERROR: ww-pacman-121026-r0 do_unpack: Unpack failure for URL: 
'svn://cocosubversion/svn/Embedded/Valve&Actuator/DVPII/trunk/SOCPACManEnvEngKeys;module=trunk;protocol=http;externals=allowed;rev=121026'. 
Unpack command
PATH="/home/w23698/projects/proj_dvp2/build_dvp2/tmp/sysroots-uninative/x86_64-linux/usr/bin:/home/w23698/projects/proj_dvp2/sources/poky/scripts:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/usr/bin/arm-poky-linux-gnueabi:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot/usr/bin/crossscripts:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/usr/sbin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/usr/bin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/sbin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/bin:/home/w23698/projects/proj_dvp2/sources/poky/bitbake/bin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/hosttools" 
tar xz --no-same-owner -f
/home/w23698/projects/proj_dvp2/build_dvp2/downloads/trunk_cocosubvers
ion_.svn.Embedded.Valve&Actuator.DVPII.trunk.SOCPACManEnvEngKeys_12102
6_.tar.gz
failed with return value 127*
*ERROR: Logfile of failure stored in: 
/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-po
ky-linux-gnueabi/ww-pacman/121026-r0/temp/log.do_unpack.25226*
*ERROR: Task
(/home/w23698/projects/proj_dvp2/build_dvp2/../sources/meta-ww-dvp2/re
cipes-ww/ww-pacman/ww-pacman.bb:do_unpack)
failed with exit code '1'
*
I don't it this matters but, the build machine is baremetal Ubuntu
18.04.3 LTS. In all cases the packed tar.gz remain in downloads 
directory with one name!

Cordially,
Georgi




***
The information in this email is confidential and intended solely for the individual or entity to whom it is addressed.  If you have received this email in error please notify the sender by return e-mail, delete this email, and refrain from any disclosure or action based on the information.
***

***
The information in this email is confidential and intended solely for the individual or entity to whom it is addressed.  If you have received this email in error please notify the sender by return e-mail, delete this email, and refrain from any disclosure or action based on the information.
***

Georgi Georgiev

#48454

Hi Mikko,
Your patch is upstreamed and...It is actually released. If you see the error output you will see your codeline.  I decided to try with your old version but the result is the same. Can you say something for the previous issue I have about offline build.
It seem svn is not very well supported by yocto.

Cordially
Georgi

Show quoted text

-----Original Message-----
From: yocto@lists.yoctoproject.org [mailto:yocto@lists.yoctoproject.org] On Behalf Of Mikko Rapeli
Sent: Monday, February 17, 2020 3:04 PM
To: Georgi Georgiev <Georgi.Georgiev@woodward.com>
Cc: raj.khem@gmail.com; yocto@lists.yoctoproject.org
Subject: [EXTERNAL] Re: [yocto] Yocto [thud], [zeus] do_fetch and do_unpack failures with offline/online svn build! #yocto #python

On Mon, Feb 17, 2020 at 12:56:13PM +0000, Georgi Georgiev via Lists.Yoctoproject.Org wrote:
Another serious issue with svn fetcher, always occurring when doing incremental builds. I am putting it here because I think both issues are related. Please suggest what to do. The error we get is shown below. This time it happens always. The only workaround is to clean and build. As soon as we make modification of the recipe (PV change only) it fails:

ERROR: ww-pacman-121204-r0 do_fetch: Error executing a python function in exec_python_func() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:base_do_fetch(d)
     0003:
File: '/data/home/w23698/projects/proj/build/../sources/poky/meta/classes/base.bbclass', lineno: 163, function: base_do_fetch
     0159:        return
     0160:
     0161:    try:
     0162:        fetcher = bb.fetch2.Fetch(src_uri, d)
 *** 0163:        fetcher.download()
     0164:    except bb.fetch2.BBFetchException as e:
     0165:        bb.fatal(str(e))
     0166:}
     0167:
File: '/data/home/w23698/projects/proj/sources/poky/bitbake/lib/bb/fetch2/__init__.py', lineno: 1678, function: download
     1674:                    try:
     1675:                        if not trusted_network(self.d, ud.url):
     1676:                            raise UntrustedUrl(ud.url)
     1677:                        logger.debug(1, "Trying Upstream")
 *** 1678:                        m.download(ud, self.d)
     1679:                        if hasattr(m, "build_mirror_data"):
     1680:                            m.build_mirror_data(ud, self.d)
     1681:                        localpath = ud.localpath
     1682:                        # early checksum verify, so that if checksum mismatched,
File: '/data/home/w23698/projects/proj/sources/poky/bitbake/lib/bb/fetch2/svn.py', lineno: 150, function: download
     0146:            if not ("externals" in ud.parm and ud.parm["externals"] == "nowarn"):
     0147:                # Warn the user if this had externals (won't catch them all)
     0148:                output = runfetchcmd("svn propget svn:externals || true", d, workdir=ud.moddir)
     0149:                if output:
 *** 0150:                    if "--ignore-externals" in svnfetchcmd.split():
     0151:                        bb.warn("%s contains svn:externals." % ud.url)
     0152:                        bb.warn("These should be added to the recipe SRC_URI as necessary.")
     0153:                        bb.warn("svn fetch has ignored externals:\n%s" % output)
     0154:                        bb.warn("To disable this warning add ';externals=nowarn' to the url.")
Exception: UnboundLocalError: local variable 'svnfetchcmd' referenced 
before assignment

ERROR: Logfile of failure stored in: 
/data/home/w23698/projects/proj/build/tmp/work/cortexa9hf-neon-poky-li
nux-gnueabi/ww-pacman/121204-r0/temp/log.do_fetch.18853Hmm. Rings a bell. I run builds with a local non-upstreamed patch:

Author:     Mikko Rapeli <mikko.rapeli@bmw.de>
AuthorDate: Fri Sep 6 14:15:20 2019 +0200
Commit:     Mikko Rapeli <mikko.rapeli@bmw.de>
CommitDate: Fri Sep 6 14:25:15 2019 +0200

    svn fetcher: allow "svn propget svn:externals" to fail
    
    Not all servers and repositories have this property set
    which results in failures like this when actual svn checkout
    command succeeded:
    
    svn: warning: W200017: Property 'svn:externals' not found on ''
    svn: E200000: A problem occurred; see other errors for details
    
Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de>

--- a/bitbake/lib/bb/fetch2/svn.py
+++ b/bitbake/lib/bb/fetch2/svn.py
@@ -145,7 +145,7 @@ class Svn(FetchMethod):
 
                 if not ("externals" in ud.parm and ud.parm["externals"] == "nowarn"):
                     # Warn the user if this had externals (won't catch them all)
-                    output = runfetchcmd("svn propget svn:externals", d, workdir=ud.moddir)
+                    output = runfetchcmd("svn propget svn:externals || 
+ true", d, workdir=ud.moddir)
                     if output:
                         if "--ignore-externals" in svnfetchcmd.split():
                             bb.warn("%s contains svn:externals." % ud.url)

I think this would help.

Cheers,

-Mikko


-----Original Message-----
From: Georgi Georgiev
Sent: Wednesday, February 05, 2020 6:57 PM
To: Georgi Georgiev <Georgi.Georgiev@woodward.com>; Khem Raj 
<raj.khem@gmail.com>; yocto@lists.yoctoproject.org
Subject: Re: [yocto] Yocto [thud], [zeus] do_fetch and do_unpack 
failures with offline/online svn build! #yocto #python

Sorry Khem,
With esc character '\' before & it can't take the full path. So briefly:

Yocto build with char '\' before & in SRC_URI:
	online - OK
	offline - Error - svn: E170013: Unable to connect to a repository...
Yocto build without '\' in SRC_URI:
	online and offline same error - do_unpack: Unpack failure for URL - 
"package".tar.gz is present in DL_DIR

Thanks



-----Original Message-----
From: yocto@lists.yoctoproject.org 
[mailto:yocto@lists.yoctoproject.org] On Behalf Of Georgi Georgiev via 
Lists.Yoctoproject.Org
Sent: Wednesday, February 05, 2020 5:26 PM
To: Khem Raj <raj.khem@gmail.com>; yocto@lists.yoctoproject.org
Cc: yocto@lists.yoctoproject.org
Subject: Re: [EXTERNAL] Re: [yocto] Yocto [thud], [zeus] do_fetch and 
do_unpack failures with offline/online svn build! #yocto #python

Hi Khem,

Yes, no issues with that. When I am connected to network it fetches the code with and without escape character before & in the path.
svn co " https://urldefense.proofpoint.com/v2/url?u=http-3A__cocosubversion_svn_Embedded_Valve-26Actuator_DVPII_trunk_SOCPACManEnvEngKeys&d=DwIGaQ&c=y6L7g950KfMp92YmLM0QlMdXcRn6b-Cq4AApnSJOenE&r=kHtJrQGzfH1ZmfsNkJpYuH-jtNpv8yMDkqAmsRP99mc&m=vXia-xlcxuLr2fSmBogZpMIaOntXLKWv3mkxcRgtUc4&s=UYH9lueDIeMGZndMxgbZgb76A5EfqN-MR58DBv1iriI&e= " works fine and fetches the package. 

Georgi

-----Original Message-----
From: Khem Raj [mailto:raj.khem@gmail.com]
Sent: Friday, January 31, 2020 8:33 PM
To: Georgi Georgiev <Georgi.Georgiev@woodward.com>; 
yocto@lists.yoctoproject.org
Subject: [EXTERNAL] Re: [yocto] Yocto [thud], [zeus] do_fetch and 
do_unpack failures with offline/online svn build! #yocto #python

On 1/31/20 4:02 AM, georgi.georgiev via Lists.Yoctoproject.Org wrote:
Hello Community,
This is the third time I am asking for support on this issue. This 
time I decided to use the web form.
In our project we have a requirement to be able to build the project 
offline. E.g. on the field without any network connection. When we 
are connected with the recipe mentioned below we don't have issues:

svn: E170013: Unable to connect to a repository at URL 'https://urldefense.proofpoint.com/v2/url?u=http-3A__cocosubversion_svn_Embedded_Valve-26Actuator_DVPII_trunk_SOCPACManEnvEngKeys_trunk&d=DwIGaQ&c=y6L7g950KfMp92YmLM0QlMdXcRn6b-Cq4AApnSJOenE&r=kHtJrQGzfH1ZmfsNkJpYuH-jtNpv8yMDkqAmsRP99mc&m=ocx6BoVSrDQGWUvUUVvgKtuJbT7eH7jFSjCy1Ys73Vw&s=OTunnmsi_tCUt3JytmU2Hs7i7Xnqhl8-2CMsqmzIl90&e= '
svn: E670003: Temporary failure in name resolution

can you try checking out the repo out side of fetcher and see if the machine can fetch the url you are using in SRC_URI ?

*SUMMARY = "PACMan - Parameter And Configuration MANager"* *LICENSE 
=
"CLOSED"* *inherit systemd useradd* *REQUIRED_DISTRO_FEATURES =
"systemd"* *# SVN revision* *PV = "121026"* *# Name of SVN project*
*PACMAN_PROJ_NAME="SOCPACManEnvEngKeys"*
*SRC_URI =
"svn://cocosubversion/svn/Embedded/Valve%5C&Actuator/DVPII/trunk/$%7
BP 
ACMAN_PROJ_NAME%7D;module=trunk;protocol=http;externals=allowed;rev=
$%
7BPV%7D%22* *SRC_URI += "file://ww-authpacman.service%22* *SRC_URI 
+=
"file://ww-pacman.service%22* *S = "${WORKDIR}/trunk/"* *# ${PN}-sys: 
content related to system, which goes to base rootfs (only .service 
file and symlinks)* *# ${PN}:      real content which may go to 
separate partition* *PACKAGES =+ " ${PN}-sys"* .........
When disconnect the network, erase sstate-cache, cache and tmp I see 
log file attached (log.do_fetch.32757) and the following output:

*ERROR: ww-pacman-121026-r0 do_fetch: Fetcher failure: Fetch command 
export PSEUDO_DISABLED=1; unset _PYTHON_SYSCONFIGDATA_NAME; export 
DBUS_SESSION_BUS_ADDRESS="unix:path=/run/user/1000/bus"; export 
SSH_AGENT_PID="11412"; export 
SSH_AUTH_SOCK="/run/user/1000/keyring/ssh"; export 
PATH="/home/w23698/projects/proj_dvp2/build_dvp2/tmp/sysroots-uninat
iv 
e/x86_64-linux/usr/bin:/home/w23698/projects/proj_dvp2/sources/poky/
sc 
ripts:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf
-n 
eon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/usr
/b 
in/arm-poky-linux-gnueabi:/home/w23698/projects/proj_dvp2/build_dvp2
/t
mp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recip
e- 
sysroot/usr/bin/crossscripts:/home/w23698/projects/proj_dvp2/build_d
vp 
2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/re
ci 
pe-sysroot-native/usr/sbin:/home/w23698/projects/proj_dvp2/build_dvp
2/ 
tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/reci
pe 
-sysroot-native/usr/bin:/home/w23698/projects/proj_dvp2/build_dvp2/t
mp 
/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-
sy 
sroot-native/sbin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/wor
k/
cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroo
t- 
native/bin:/home/w23698/projects/proj_dvp2/sources/poky/bitbake/bin:
/h ome/w23698/projects/proj_dvp2/build_dvp2/tmp/hosttools";
export HOME="/home/w23698"; /usr/bin/env svn --non-interactive 
--trust-server-cert update --no-auth-cache -r 121026 failed with 
exit code 1, output:* *Updating '.':*
*svn: E170013: Unable to connect to a repository at URL 
'https://urldefense.proofpoint.com/v2/url?u=http-3A__cocosubversion_
sv
n_Embedded_Valve-26Actuator_DVPII_trunk_SOCPACManEnvEngKeys_trunk-27
-2
A&d=DwIGaQ&c=y6L7g950KfMp92YmLM0QlMdXcRn6b-Cq4AApnSJOenE&r=kHtJrQGzf
H1 
ZmfsNkJpYuH-jtNpv8yMDkqAmsRP99mc&m=ocx6BoVSrDQGWUvUUVvgKtuJbT7eH7jFS
jC y1Ys73Vw&s=Ffv1JU1QYBh4g49fmoLnsDSFgMMBc_5MbOpy59QUS18&e=
*svn: E670003: Temporary failure in name resolution*
*ERROR: ww-pacman-121026-r0 do_fetch: Fetcher failure for URL: 
'svn://cocosubversion/svn/Embedded/Valve%5C&Actuator/DVPII/trunk/SOCPACManEnvEngKeys;module=trunk;protocol=http;externals=allowed;rev=121026'. 
Unable to fetch URL from any source.*
*ERROR: Logfile of failure stored in: 
/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-
po
ky-linux-gnueabi/ww-pacman/121026-r0/temp/log.do_fetch.32757*
*ERROR: Task
(/home/w23698/projects/proj_dvp2/build_dvp2/../sources/meta-ww-dvp2/
re
cipes-ww/ww-pacman/ww-pacman.bb:do_fetch)
failed with exit code '1'*

When remove the '\' character in SRC_URI, e.g. to become:

*SRC_URI =
"svn://cocosubversion/svn/Embedded/Valve&Actuator/DVPII/trunk/$%7BPA
CM 
AN_PROJ_NAME%7D;module=trunk;protocol=http;externals=allowed;rev=$%7
BP
V%7D%22*

In connected and not connected to network do_fetch() passes 
successfully but I see one and same error (log.do_unpack.25226) output:

*ERROR: ww-pacman-121026-r0 do_unpack: Unpack failure for URL: 
'svn://cocosubversion/svn/Embedded/Valve&Actuator/DVPII/trunk/SOCPACManEnvEngKeys;module=trunk;protocol=http;externals=allowed;rev=121026'. 
Unpack command
PATH="/home/w23698/projects/proj_dvp2/build_dvp2/tmp/sysroots-uninative/x86_64-linux/usr/bin:/home/w23698/projects/proj_dvp2/sources/poky/scripts:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/usr/bin/arm-poky-linux-gnueabi:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot/usr/bin/crossscripts:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/usr/sbin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/usr/bin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/sbin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/ww-pacman/121026-r0/recipe-sysroot-native/bin:/home/w23698/projects/proj_dvp2/sources/poky/bitbake/bin:/home/w23698/projects/proj_dvp2/build_dvp2/tmp/hosttools" 
tar xz --no-same-owner -f
/home/w23698/projects/proj_dvp2/build_dvp2/downloads/trunk_cocosubve
rs
ion_.svn.Embedded.Valve&Actuator.DVPII.trunk.SOCPACManEnvEngKeys_121
02
6_.tar.gz
failed with return value 127*
*ERROR: Logfile of failure stored in: 
/home/w23698/projects/proj_dvp2/build_dvp2/tmp/work/cortexa9hf-neon-
po
ky-linux-gnueabi/ww-pacman/121026-r0/temp/log.do_unpack.25226*
*ERROR: Task
(/home/w23698/projects/proj_dvp2/build_dvp2/../sources/meta-ww-dvp2/
re
cipes-ww/ww-pacman/ww-pacman.bb:do_unpack)
failed with exit code '1'
*
I don't it this matters but, the build machine is baremetal Ubuntu
18.04.3 LTS. In all cases the packed tar.gz remain in downloads 
directory with one name!

Cordially,
Georgi




***
The information in this email is confidential and intended solely for the individual or entity to whom it is addressed.  If you have received this email in error please notify the sender by return e-mail, delete this email, and refrain from any disclosure or action based on the information.
***

***
The information in this email is confidential and intended solely for the individual or entity to whom it is addressed.  If you have received this email in error please notify the sender by return e-mail, delete this email, and refrain from any disclosure or action based on the information.
******
The information in this email is confidential and intended solely for the individual or entity to whom it is addressed.  If you have received this email in error please notify the sender by return e-mail, delete this email, and refrain from any disclosure or action based on the information.
***

Mikko Rapeli

#48455

On Mon, Feb 17, 2020 at 01:35:02PM +0000, Georgi Georgiev via Lists.Yoctoproject.Org wrote:
Hi Mikko,
Your patch is upstreamed and...It is actually released. If you see the error output you will see your codeline.  I decided to try with your old version but the result is the same. Can you say something for the previous issue I have about offline build.
It seem svn is not very well supported by yocto.Ah, indeed.

svn is supported and for me it's working quite well. It has some issues
like download cache locking when multiple recipes include the same URLs.

Where possible, I've switched to using http fetcher with tar balls and large
binary blobs in svn, but there one needs to remember to include version
details into file names...

-Mikko

Georgi Georgiev

#48465

OK,
I read some code and I added the next line:

--- a/bitbake/lib/bb/fetch2/svn.py
+++ b/bitbake/lib/bb/fetch2/svn.py
@@ -145,6 +145,7 @@ class Svn(FetchMethod):

             if not ("externals" in ud.parm and ud.parm["externals"] == "nowarn"):
                 # Warn the user if this had externals (won't catch them all)
+                svnfetchcmd = self._buildsvncommand(ud, d, "fetch")
                 output = runfetchcmd("svn propget svn:externals || true", d, workdir=ud.moddir)
                 if output:
                     if "--ignore-externals" in svnfetchcmd.split():

This works for me perfectly. I may look little redundant but...
Any comments? 
@Mikko, 
We have only one svn repo in the whole project :-)

Cordially,
Georgi

Show quoted text

-----Original Message-----
From: Mikko.Rapeli@bmw.de [mailto:Mikko.Rapeli@bmw.de] 
Sent: Monday, February 17, 2020 3:47 PM
To: Georgi Georgiev <Georgi.Georgiev@woodward.com>
Cc: yocto@lists.yoctoproject.org
Subject: [EXTERNAL] Re: [yocto] Yocto [thud], [zeus] do_fetch and do_unpack failures with offline/online svn build! #yocto #python

On Mon, Feb 17, 2020 at 01:35:02PM +0000, Georgi Georgiev via Lists.Yoctoproject.Org wrote:
Hi Mikko,
Your patch is upstreamed and...It is actually released. If you see the error output you will see your codeline.  I decided to try with your old version but the result is the same. Can you say something for the previous issue I have about offline build.
It seem svn is not very well supported by yocto.Ah, indeed.

svn is supported and for me it's working quite well. It has some issues like download cache locking when multiple recipes include the same URLs.

Where possible, I've switched to using http fetcher with tar balls and large binary blobs in svn, but there one needs to remember to include version details into file names...

-Mikko

***
The information in this email is confidential and intended solely for the individual or entity to whom it is addressed.  If you have received this email in error please notify the sender by return e-mail, delete this email, and refrain from any disclosure or action based on the information.
***

Mikko Rapeli

#48512

On Tue, Feb 18, 2020 at 07:25:15AM +0000, Georgi Georgiev via Lists.Yoctoproject.Org wrote:
OK,
I read some code and I added the next line:

--- a/bitbake/lib/bb/fetch2/svn.py
+++ b/bitbake/lib/bb/fetch2/svn.py
@@ -145,6 +145,7 @@ class Svn(FetchMethod):

             if not ("externals" in ud.parm and ud.parm["externals"] == "nowarn"):
                 # Warn the user if this had externals (won't catch them all)
+                svnfetchcmd = self._buildsvncommand(ud, d, "fetch")
                 output = runfetchcmd("svn propget svn:externals || true", d, workdir=ud.moddir)
                 if output:
                     if "--ignore-externals" in svnfetchcmd.split():

This works for me perfectly. I may look little redundant but...
Any comments? 
@Mikko, Looks good to me.

We have only one svn repo in the whole project :-)Lucky you :)

-Mikko