Date   

apt-get destroying itself when trying to install package #apt #yocto

stefan.wenninger@...
 

Hi,
We are trying to install packages on our arm64 architecture directly from debian.org.
Our sources.list contains "deb [ arch=arm64 ] http://ftp.de.debian.org/debian buster main".

1. When executing apt-get update we get the following output:
root@imx8mq-var-dart:~# apt-get update
Get:1 http://ftp.de.debian.org/debian buster InRelease [122 kB]
Ign:1 http://ftp.de.debian.org/debian buster InRelease
Get:2 http://ftp.de.debian.org/debian buster/main arm64 Packages [7737 kB]
Fetched 7858 kB in 5s (1424 kB/s)
Reading package lists... Done
W: GPG error: http://ftp.de.debian.org/debian buster InRelease: Unknown error executing apt-key
W: The repository 'http://ftp.de.debian.org/debian buster InRelease' is not signed.
N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: Failed to fetch /
E: Some index files failed to download. They have been ignored, or old ones used instead.
We did not worry about the missing gpg authentification since we were able to apt-cache search this source successfully.

2. When we tried to apt-get install python3-psutil we were promted with a large list of mainly perl related packages that were about to be removed:
12 upgraded, 66 newly installed, 531 to remove and 212 not upgraded
[complete output in attachments.]
Allowing these changes to be made led to dpkg trying to overwrite files that are also in other packages. This caused the command to fail.

3. We then tried to pass the "--force-overwrite" option to dpkg with the command:
sudo apt-get -o Dpkg::Options::="--force-overwrite" install python3-psutil
This command told us there were unmet dependencies:
The following packages have unmet dependencies:
 libc6-dev : Depends: libc6 (= 2.28-10) but 2.27-r0 is to be installed
             Depends: libc-dev-bin (= 2.28-10) but it is not going to be installed
             Depends: linux-libc-dev but it is not going to be installed
 python3-psutil : Depends: python3 (< 3.8)
                  Depends: python3 (>= 3.7~)
                  Depends: python3:any
E: Unmet dependencies. Try 'apt-get -f install' with no packages (or specify a solution).
[complete output in attachments]

4. We followed the suggestion and ran apt-get -f install.
[complete output in attachments]
That led to dpkg trying to unpack "libc6" but failing because /sbin/ldconfig is not present.
We confirmed that /sbin/ldconfig is in place and an executable on a fresh image.


We have tried to install different packages and to use different orders of commands, but ultimately we always end up with ldconfig being deleted and dpkg failing because of that.
Our best guess it that the way apt-get is set up in our image is faulty.

Relevant Yocto info:
Version: Yocto sumo 2.5
Image: fsl-image-qt5
IMAGE_INSTALL += " apt "
We have PACKAGE_CLASSES = "package_deb" and PACKAGE_FEED_URIS="http://<host_ip>:5678" in our local.conf. However we deleted the sources.list entries created by this.
Is this a known issue? Are we missing an important configuration for apt within Yocto?
Is there another way to install and setup apt-get in our image that does not include Yocto?

Thanks,
Stefan


[meta-spdxscanner][PATCH V2] Remove redundant code.

Li, Xiaoming
 

FOLDER_ID has already been assigned a defalut value "1", so there is no
need add 'or "1"' here.

Signed-off-by: Li Xiaoming <lixm.fnst@...>
---
classes/fossology-rest.bbclass | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/classes/fossology-rest.bbclass b/classes/fossology-rest.bbclass
index 5c5ef70..69f4998 100644
--- a/classes/fossology-rest.bbclass
+++ b/classes/fossology-rest.bbclass
@@ -230,7 +230,7 @@ def get_folder_id(d):
folder_name = d.getVar('FOLDER_NAME')
folder_id = create_folder(d, folder_name)
else:
- folder_id = (d.getVar('FOLDER_ID', True) or "1")
+ folder_id = d.getVar('FOLDER_ID', False)

bb.note("Folder Id = " + str(folder_id))
return str(folder_id)
--
2.17.1


Re: Files get sporadically lost for native packages

Konrad Weihmann <kweihmann@...>
 

To answer your others questions... see below

On 02.04.20 05:42, Randy MacLeod wrote:
On 2020-03-28 8:26 a.m., Konrad Weihmann wrote:
Hi,

I'm facing the following error message sporadically on all branches I tried so far (master, zeus, warrior and thud)

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
      0001:
  *** 0002:extend_recipe_sysroot(d)
      0003:
File: '/build/poky/meta/classes/staging.bbclass', lineno: 551, function: extend_recipe_sysroot
      0547:                    dest = newmanifest[l]
      0548:                    if l.endswith("/"):
      0549:                        staging_copydir(l, targetdir, dest, seendirs)
      0550:                        continue
  *** 0551:                    staging_copyfile(l, targetdir, dest, postinsts, seendirs)
      0552:
      0553:    bb.note("Installed into sysroot: %s" % str(msg_adding))
      0554:    bb.note("Skipping as already exists in sysroot: %s" % str(msg_exists))
      0555:
File: '/build/poky/meta/classes/staging.bbclass', lineno: 152, function: staging_copyfile
      0148:        os.symlink(linkto, dest)
      0149:        #bb.warn(c)
      0150:    else:
      0151:        try:
  *** 0152:            os.link(c, dest)
      0153:        except OSError as err:
      0154:            if err.errno == errno.EXDEV:
      0155:                bb.utils.copyfile(c, dest)
      0156:            else:
Exception: FileNotFoundError: [Errno 2] No such file or directory: '/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/__init__.cpython-37.pyc' -> '/build/poky/build/tmp/work/qemux86_64-mine-linux/core-image-minimal-mine/1.0-r0/recipe-sysroot-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/__init__.cpython-37.pyc'

I already had a look at the manifest

cat manifest-x86_64-python3-msgcheck-native.populate_sysroot
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__init__.py
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/po.py
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/msgcheck.py
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/__init__.cpython-37.pyc
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/po.cpython-37.pyc
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/msgcheck.cpython-37.pyc
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/dependency_links.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/requires.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/top_level.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/SOURCES.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/PKG-INFO
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/entry_points.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/bin/msgcheck
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/sysroot-providers/python3-msgcheck-native
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/sysroot-providers/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/share/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/bin/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/

which states the file should be there, but when doing

find /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__init__.py
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/po.cpython-37.pyc
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/msgcheck.cpython-37.pyc
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/po.py
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/msgcheck.py
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/dependency_links.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/requires.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/top_level.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/SOURCES.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/PKG-INFO
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/entry_points.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/bin
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/bin/msgcheck
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/share

the file isn't there.

This happens to random python packages compiled as native (sometimes even for python-native itself), but (afaik) not for cross or target packages (I'm pretty sure because of the different packaging applied).
So I digged a little into the code classes/sstate.bbclass:sstate_install, which seems to create the sysroot-component dir and the manifest.
There is a gap between the manifest creation (line 285) and the hardlinking (line till 311).

Now my question is there any place where a file potentially could get lost? - at first glance there shouldn't be one - I have to admit that I don't fully understand all this subprocess magic in lib/oe/path.py:copyhardlinktree.
What I do to fix the issue is reopening the manifest and double check for missing files and remove them from the manifest, but this
feels wrong - so any advise is welcome...

Hope that someone more familiar with the topic could have a look.

Hi Konrad,

I'm not really familiar with that code but it's being run buy 1000s of
builder around the world so let's try to eliminate a few possibilities.

When did you start having this problem?
Since the start of the test distribution I'm working on. But also for plain poky builds if I forcefully inject all of the python-native site-packages via local.conf (DEPENDS_class-native += "..."), without actually using them in the recipe scope
How often do you think it's happening: 1 in 3 builds, 1 in 10?
See the other mail - looks like it heavily depends on the host
Tell us about your machine: OS,version, disk, CPUs, ram
See the other mail
Do you do anything special in your conf dir? Send local.conf perhaps.
No custom modification (just for testing the DEPENDS-injection)
Do you have any local bbappends or commits on top of poky or
in other layers?
No
Have you tried to simplify the build to eliminate problems
potentially caused by other layers?
I did - see above
Are you able to reproduce the problem on more than one build machine?
See the other mail
Are you able to reproduce the problem on a different Linux distro?
Not really - Debian 9 was fine all other Hosts are Ubuntu based
Are there other builds or users on the machine that may be causing
extra load?
No the hosts are just being poorly equipped - at least the ones that produce this issue


../Randy


Thanks

Konrad








    


Re: Files get sporadically lost for native packages

Konrad Weihmann <kweihmann@...>
 

HI,

sorry I didn't specify it further.
To me it seems to be a corner case scenario.

This happens when doing a from scratch build with a lot on native python libraries (as it is setup in a test distribution I'm working on).
Roughly it pulls in ~100 native python site packages.

The setup where I almost every time can reproduce the issue

- Ubuntu 16.04 or 18.04 container
- 2 VCPUs (can't say what it actually is)
- 14GB of hard disk (I guess it's some virtual storage)
- 12GB of RAM

-> as offered by GitHub's Actions Azure interface

The setup where it occasionally happens

- Ubuntu 19.04/19.10
- 8 cores i7-6th gen
- 16GB Ram
- some Samsung-SSD with more than enough space

The setup where it never happened so far

- Ubuntu 18.04
- 48 core threadripper
- lots of RAM and SSD

and

- Debian 9
- 24 core XEN guest
- >80GB of RAM
- HDD doesn't matter as it was a ramdisk only build

This doesn't happen (afaik) on differential builds when the initial one is good (same for sstate cache backed builds).

I just wanted to know if somebody ever encountered the same issue.
As the nature of this issue is hard to catch, I'm open to ideas...
What currently works for me is to remove all python cache files before they are packaged - at least none of my recent build failed so far.

Any ideas how to tackle this problem, any suggestions?

Best
Konrad

On 02.04.20 05:42, Randy MacLeod wrote:
On 2020-03-28 8:26 a.m., Konrad Weihmann wrote:
Hi,

I'm facing the following error message sporadically on all branches I tried so far (master, zeus, warrior and thud)

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
      0001:
  *** 0002:extend_recipe_sysroot(d)
      0003:
File: '/build/poky/meta/classes/staging.bbclass', lineno: 551, function: extend_recipe_sysroot
      0547:                    dest = newmanifest[l]
      0548:                    if l.endswith("/"):
      0549:                        staging_copydir(l, targetdir, dest, seendirs)
      0550:                        continue
  *** 0551:                    staging_copyfile(l, targetdir, dest, postinsts, seendirs)
      0552:
      0553:    bb.note("Installed into sysroot: %s" % str(msg_adding))
      0554:    bb.note("Skipping as already exists in sysroot: %s" % str(msg_exists))
      0555:
File: '/build/poky/meta/classes/staging.bbclass', lineno: 152, function: staging_copyfile
      0148:        os.symlink(linkto, dest)
      0149:        #bb.warn(c)
      0150:    else:
      0151:        try:
  *** 0152:            os.link(c, dest)
      0153:        except OSError as err:
      0154:            if err.errno == errno.EXDEV:
      0155:                bb.utils.copyfile(c, dest)
      0156:            else:
Exception: FileNotFoundError: [Errno 2] No such file or directory: '/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/__init__.cpython-37.pyc' -> '/build/poky/build/tmp/work/qemux86_64-mine-linux/core-image-minimal-mine/1.0-r0/recipe-sysroot-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/__init__.cpython-37.pyc'

I already had a look at the manifest

cat manifest-x86_64-python3-msgcheck-native.populate_sysroot
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__init__.py
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/po.py
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/msgcheck.py
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/__init__.cpython-37.pyc
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/po.cpython-37.pyc
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/msgcheck.cpython-37.pyc
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/dependency_links.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/requires.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/top_level.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/SOURCES.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/PKG-INFO
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/entry_points.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/bin/msgcheck
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/sysroot-providers/python3-msgcheck-native
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/sysroot-providers/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/share/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/bin/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/

which states the file should be there, but when doing

find /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__init__.py
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/po.cpython-37.pyc
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/msgcheck.cpython-37.pyc
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/po.py
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/msgcheck.py
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/dependency_links.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/requires.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/top_level.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/SOURCES.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/PKG-INFO
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/entry_points.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/bin
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/bin/msgcheck
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/share

the file isn't there.

This happens to random python packages compiled as native (sometimes even for python-native itself), but (afaik) not for cross or target packages (I'm pretty sure because of the different packaging applied).
So I digged a little into the code classes/sstate.bbclass:sstate_install, which seems to create the sysroot-component dir and the manifest.
There is a gap between the manifest creation (line 285) and the hardlinking (line till 311).

Now my question is there any place where a file potentially could get lost? - at first glance there shouldn't be one - I have to admit that I don't fully understand all this subprocess magic in lib/oe/path.py:copyhardlinktree.
What I do to fix the issue is reopening the manifest and double check for missing files and remove them from the manifest, but this
feels wrong - so any advise is welcome...

Hope that someone more familiar with the topic could have a look.

Hi Konrad,

I'm not really familiar with that code but it's being run buy 1000s of
builder around the world so let's try to eliminate a few possibilities.

When did you start having this problem?
How often do you think it's happening: 1 in 3 builds, 1 in 10?
Tell us about your machine: OS,version, disk, CPUs, ram
Do you do anything special in your conf dir? Send local.conf perhaps.
Do you have any local bbappends or commits on top of poky or
in other layers?
Have you tried to simplify the build to eliminate problems
potentially caused by other layers?
Are you able to reproduce the problem on more than one build machine?
Are you able to reproduce the problem on a different Linux distro?
Are there other builds or users on the machine that may be causing
extra load?


../Randy


Thanks

Konrad








    


[meta-spdxscanner][PATCH] Remove redundant code.

Li, Xiaoming
 

As folder_id can not be specified be in conf file, we do not need
getVar to read it.

Signed-off-by: Li Xiaoming <lixm.fnst@...>
---
classes/fossology-rest.bbclass | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/classes/fossology-rest.bbclass b/classes/fossology-rest.bbclass
index 5c5ef70..c8d29ff 100644
--- a/classes/fossology-rest.bbclass
+++ b/classes/fossology-rest.bbclass
@@ -230,7 +230,7 @@ def get_folder_id(d):
folder_name = d.getVar('FOLDER_NAME')
folder_id = create_folder(d, folder_name)
else:
- folder_id = (d.getVar('FOLDER_ID', True) or "1")
+ folder_id = FOLDER_ID

bb.note("Folder Id = " + str(folder_id))
return str(folder_id)
--
2.17.1


post-install - how to "reset" an image

rob.haycock@...
 

Hi,

 

Newbie here.

 

I’ve been handed a PCB with a Yocto image running on it. I’m reading up on Yocto to get a better understanding but in the meantime I would appreciate help with a, hopefully, quick question.

 

I know the image has post-install actions (that talk to other peripherals) and I’m assuming there is flag somewhere that I can reset to make the post-install actions run again?

 

My understanding is, the long way round would be to get a build host and build the image again. I’m currently waiting for a build host to be made available so I can try this out. I’m hoping for a quick resolution in the meantime.

 

Thanks,

Rob

 


Using CROPS inside Kubernetes

Assaf Katz
 

Hi, Currently we have VMs that are created on demand and are used for build of versions of our distribution by Yocto. We want to use docker images (based on CROPS) inside Kubernetes. I found out that that buildbot supports Kubernetes (http://docs.buildbot.net/current/relnotes/1.x.html?highlight=kubernetes#features). But it isn't clear how to use it for building distribution by Yocoto. Is anyone try to do such configuration? We want to use OpenShift (Kubernetes), with full build system will be configured inside so any information will be useful :-) Thanks, Assaf Katz
 


[meta-selinux][PATCH] layer.conf: update LAYERSERIES_COMPAT for dunfell

Yi Zhao
 

Signed-off-by: Yi Zhao <yi.zhao@...>
---
conf/layer.conf | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/conf/layer.conf b/conf/layer.conf
index cdee27a..da24359 100644
--- a/conf/layer.conf
+++ b/conf/layer.conf
@@ -23,7 +23,7 @@ BBFILE_PRIORITY_selinux = "5"
# cause compatibility issues with other layers
LAYERVERSION_selinux = "1"

-LAYERSERIES_COMPAT_selinux = "zeus"
+LAYERSERIES_COMPAT_selinux = "dunfell"

LAYERDEPENDS_selinux = " \
core \
--
2.17.1


Re: Files get sporadically lost for native packages

Randy MacLeod
 

On 2020-03-28 8:26 a.m., Konrad Weihmann wrote:
Hi,
I'm facing the following error message sporadically on all branches I tried so far (master, zeus, warrior and thud)
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:extend_recipe_sysroot(d)
     0003:
File: '/build/poky/meta/classes/staging.bbclass', lineno: 551, function: extend_recipe_sysroot
     0547:                    dest = newmanifest[l]
     0548:                    if l.endswith("/"):
     0549:                        staging_copydir(l, targetdir, dest, seendirs)
     0550:                        continue
 *** 0551:                    staging_copyfile(l, targetdir, dest, postinsts, seendirs)
     0552:
     0553:    bb.note("Installed into sysroot: %s" % str(msg_adding))
     0554:    bb.note("Skipping as already exists in sysroot: %s" % str(msg_exists))
     0555:
File: '/build/poky/meta/classes/staging.bbclass', lineno: 152, function: staging_copyfile
     0148:        os.symlink(linkto, dest)
     0149:        #bb.warn(c)
     0150:    else:
     0151:        try:
 *** 0152:            os.link(c, dest)
     0153:        except OSError as err:
     0154:            if err.errno == errno.EXDEV:
     0155:                bb.utils.copyfile(c, dest)
     0156:            else:
Exception: FileNotFoundError: [Errno 2] No such file or directory: '/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/__init__.cpython-37.pyc' -> '/build/poky/build/tmp/work/qemux86_64-mine-linux/core-image-minimal-mine/1.0-r0/recipe-sysroot-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/__init__.cpython-37.pyc'
I already had a look at the manifest
cat manifest-x86_64-python3-msgcheck-native.populate_sysroot
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__init__.py
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/po.py
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/msgcheck.py
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/__init__.cpython-37.pyc
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/po.cpython-37.pyc
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/msgcheck.cpython-37.pyc
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/dependency_links.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/requires.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/top_level.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/SOURCES.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/PKG-INFO
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/entry_points.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/bin/msgcheck
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/sysroot-providers/python3-msgcheck-native
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/sysroot-providers/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/share/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/bin/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/
which states the file should be there, but when doing
find /build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__init__.py
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/po.cpython-37.pyc
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/msgcheck.cpython-37.pyc
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/po.py
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/msgcheck.py
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/dependency_links.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/requires.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/top_level.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/SOURCES.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/PKG-INFO
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck-2.8-py3.7.egg-info/entry_points.txt
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/bin
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/bin/msgcheck
/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/share
the file isn't there.
This happens to random python packages compiled as native (sometimes even for python-native itself), but (afaik) not for cross or target packages (I'm pretty sure because of the different packaging applied).
So I digged a little into the code classes/sstate.bbclass:sstate_install, which seems to create the sysroot-component dir and the manifest.
There is a gap between the manifest creation (line 285) and the hardlinking (line till 311).
Now my question is there any place where a file potentially could get lost? - at first glance there shouldn't be one - I have to admit that I don't fully understand all this subprocess magic in lib/oe/path.py:copyhardlinktree.
What I do to fix the issue is reopening the manifest and double check for missing files and remove them from the manifest, but this
feels wrong - so any advise is welcome...
Hope that someone more familiar with the topic could have a look.
Hi Konrad,

I'm not really familiar with that code but it's being run buy 1000s of
builder around the world so let's try to eliminate a few possibilities.

When did you start having this problem?
How often do you think it's happening: 1 in 3 builds, 1 in 10?
Tell us about your machine: OS,version, disk, CPUs, ram
Do you do anything special in your conf dir? Send local.conf perhaps.
Do you have any local bbappends or commits on top of poky or
in other layers?
Have you tried to simplify the build to eliminate problems
potentially caused by other layers?
Are you able to reproduce the problem on more than one build machine?
Are you able to reproduce the problem on a different Linux distro?
Are there other builds or users on the machine that may be causing
extra load?


../Randy

Thanks
Konrad

--
# Randy MacLeod
# Wind River Linux


Re: [yocto-autobuilder-helper][PATCH 1/2] utils: Add getconfiglistfilter() to return config based on regexpr

Aaron Chan
 

Could someone please review my patches ? Thanks.


Re: [yocto-autobuilder2][PATCH] config: Fix giturl for meta-virtualization Layer

Aaron Chan
 

Could someone please fix this simple piece?


Re: [PATCH] Add localhost to noproxy list.

leimaohui
 

Merged. Thank you.

-----Original Message-----
From: Li Xiaoming <lixm.fnst@...>
Sent: Monday, March 30, 2020 9:37 PM
To: yocto@...
Cc: Lei, Maohui <leimaohui@...>
Subject: [PATCH] Add localhost to noproxy list.

Signed-off-by: Li Xiaoming <lixm.fnst@...>
---
classes/fossology-rest.bbclass | 16 +++++++++-------
1 file changed, 9 insertions(+), 7 deletions(-)

diff --git a/classes/fossology-rest.bbclass b/classes/fossology-rest.bbclass index
48e16f3..000eefe 100644
--- a/classes/fossology-rest.bbclass
+++ b/classes/fossology-rest.bbclass
@@ -25,6 +25,8 @@ HOSTTOOLS_NONFATAL += "curl"

CREATOR_TOOL = "fossology-rest.bbclass in meta-spdxscanner"

+NO_PROXY = "localhost, 127.0.0.1"
+
# If ${S} isn't actually the top-level source directory, set SPDX_S to point at # the
real top-level directory.
SPDX_S ?= "${S}"
@@ -159,7 +161,7 @@ def get_folder_id_by_name(d, folder_name):

rest_api_cmd = "curl -k -s -S -X GET " + server_url + "/api/v1/folders" \
+ " -H \"Authorization: Bearer " + token + "\"" \
- + " --noproxy 127.0.0.1"
+ + " --noproxy " + NO_PROXY
bb.note("Invoke rest_api_cmd = " + rest_api_cmd )
try:
all_folder = subprocess.check_output(rest_api_cmd,
stderr=subprocess.STDOUT, shell=True) @@ -200,7 +202,7 @@ def
create_folder(d, folder_name):
+ " -H \'parentFolder: 1\'" \
+ " -H \'folderName: " + folder_name + "\'" \
+ " -H \"Authorization: Bearer " + token + "\"" \
- + " --noproxy 127.0.0.1"
+ + " --noproxy " + NO_PROXY
bb.note("Invoke rest_api_cmd = " + rest_api_cmd)
try:
add_folder = subprocess.check_output(rest_api_cmd,
stderr=subprocess.STDOUT, shell=True) @@ -251,7 +253,7 @@ def
has_upload(d, tar_file, folder_id):

rest_api_cmd = "curl -k -s -S -X GET " + server_url + "/api/v1/uploads" \
+ " -H \"Authorization: Bearer " + token + "\"" \
- + " --noproxy 127.0.0.1"
+ + " --noproxy " + NO_PROXY
bb.note("Invoke rest_api_cmd = " + rest_api_cmd )

try:
@@ -302,7 +304,7 @@ def upload(d, tar_file, folder):
+ " -H \'public: public\'" \
+ " -H \'Content-Type: multipart/form-data\'" \
+ " -F \'fileInput=@\"" + tar_file + "\";type=application/octet-
stream\'" \
- + " --noproxy 127.0.0.1"
+ + " --noproxy " + NO_PROXY
bb.note("Upload : Invoke rest_api_cmd = " + rest_api_cmd )
while i < 10:
time.sleep(delaytime)
@@ -343,7 +345,7 @@ def analysis(d, folder_id, upload_id):
+ " -H \"Authorization: Bearer " + token + "\"" \
+ " -H \'Content-Type: application/json\'" \
+ " --data \'{\"analysis\": {\"bucket\":
true,\"copyright_email_author\": true,\"ecc\": true, \"keyword\": true,\"mime\":
true,\"monk\": true,\"nomos\": true,\"package\": true},\"decider\":
{\"nomos_monk\": true,\"bulk_reused\": true,\"new_scanner\": true}}\'" \
- + " --noproxy 127.0.0.1"
+ + " --noproxy " + NO_PROXY
bb.note("Analysis : Invoke rest_api_cmd = " + rest_api_cmd )
while i < 10:
try:
@@ -389,7 +391,7 @@ def trigger(d, folder_id, upload_id):
+ " -H \"Authorization: Bearer " + token + "\"" \
+ " -H \"uploadId: " + str(upload_id) + "\"" \
+ " -H \'reportFormat: spdx2tv\'" \
- + " --noproxy 127.0.0.1"
+ + " --noproxy " + NO_PROXY
bb.note("trigger : Invoke rest_api_cmd = " + rest_api_cmd )
while i < 10:
time.sleep(delaytime)
@@ -431,7 +433,7 @@ def get_spdx(d, report_id, spdx_file):
rest_api_cmd = "curl -k -s -S -X GET " + server_url + "/api/v1/report/" +
report_id \
+ " -H \'accept: text/plain\'" \
+ " -H \"Authorization: Bearer " + token + "\"" \
- + " --noproxy 127.0.0.1"
+ + " --noproxy " + NO_PROXY
bb.note("get_spdx : Invoke rest_api_cmd = " + rest_api_cmd )
while i < 10:
time.sleep(delaytime)
--
2.17.1


Re: python argparse or alternate method

Khem Raj
 

On 4/1/20 2:20 PM, Joel Winarske wrote:
Hi Konrad,
Thanks, that helps a lot.  It works!  I used straight split() which prevented the first value from being null.
Yes, I agree less pythonic :)  The only recipe I've seen building gn based projects is OSSystem's meta-browser/chromium.  I followed it's pattern, as the Flutter Engine is a derivative of Chrome.
The issue with these large gn based projects is that they're self contained builds.  A build system within a build system.  I must admit though the gclient DEPS download scheme is efficient.  I initially looked at parsing out the DEPs file into a large SRC_URI list, but quickly hit Yocto serializing them.  Perhaps OE could benefit from tighter integration with gn.
perhaps a generic gn class might be helpful, there have been attemps to add depot_tools/glient to fetchers in past but there is no consistent apis there, it kept on changing in myriad different ways. gn is more or less same story, its too tightly integrated into the software its building, we should abstract it if we can.

I'm open to any other alternative approaches.
Cheers,
Joel
On Wed, Apr 1, 2020 at 1:09 PM Konrad Weihmann <kweihmann@... <mailto:kweihmann@...>> wrote:
Hi Joel,
okay, I see - although I have to admit the argparse idea looks very
weird, the error is that according to
https://docs.python.org/3/library/argparse.html#argparse.ArgumentParser.parse_args
the pkgconfig_args variable should be a list not a single string.
so doing pkgconfig_args = d.getVar("PACKAGECONFIG_CONFARGS").split("
") should fix the issue.
But from a bitbake perspective you should really think about
choosing a less pythonic version
Best
Konrad
On 01.04.20 22:00, Joel Winarske wrote:
Hi Konrad,

I left out the details of the path encoding :)  Effectively I was
attempting to duplicate get_out_dir() as found here:
https://github.com/flutter/engine/blob/master/tools/gn

My recipe is here:
https://github.com/jwinarske/meta-flutter/blob/zeus/recipes-graphics/flutter-engine/flutter-engine_git.bb

Cheers,
Joel

On Wed, Apr 1, 2020 at 10:46 AM Konrad Weihmann
<kweihmann@... <mailto:kweihmann@...>> wrote:

There is a single tick missing, but I'm sure you get the point

On 01.04.20 19:37, Konrad Weihmann wrote:

What about

do_configure() {

   cd ${@bb.utils.contains('PACKAGECONFIG_CONFARGS',
'--verbose', out/verbose', 'out', d)}

}

On 01.04.20 18:55, Joel Winarske wrote:
Hello,

I'm trying the below in a recipe.  Bitbake gives me: ERROR:
Unable to parse ...: Exited with "2"

inherit python3native

require utils.inc

do_configure() {

    cd ${@get_out_dir(d)}
}

(in utils.inc)
def get_out_dir(d):
    import os
    import argparse
    pkgconfig_args = d.getVar("PACKAGECONFIG_CONFARGS")

    parser = argparse.ArgumentParser()
    parser.add_argument("--verbose", help="increase output
verbosity",
                        action="store_true")
    args = parser.parse_args(pkgconfig_args)
    if args.verbose:
        return os.path.join('out', 'verbose')

    return 'out'



What am I doing wrong, or is there a better way?  Seems like
it doesn't like "args = parser.parse_args(pkgconfig_args)"

The output directory is a combination of the arguments.


Thanks!
Joel


Re: python argparse or alternate method

Joel Winarske
 

Hi Konrad,

Thanks, that helps a lot.  It works!  I used straight split() which prevented the first value from being null.

Yes, I agree less pythonic :)  The only recipe I've seen building gn based projects is OSSystem's meta-browser/chromium.  I followed it's pattern, as the Flutter Engine is a derivative of Chrome.

The issue with these large gn based projects is that they're self contained builds.  A build system within a build system.  I must admit though the gclient DEPS download scheme is efficient.  I initially looked at parsing out the DEPs file into a large SRC_URI list, but quickly hit Yocto serializing them.  Perhaps OE could benefit from tighter integration with gn.

I'm open to any other alternative approaches.


Cheers,
Joel


On Wed, Apr 1, 2020 at 1:09 PM Konrad Weihmann <kweihmann@...> wrote:

Hi Joel,

okay, I see - although I have to admit the argparse idea looks very weird, the error is that according to
https://docs.python.org/3/library/argparse.html#argparse.ArgumentParser.parse_args
the pkgconfig_args variable should be a list not a single string.

so doing pkgconfig_args = d.getVar("PACKAGECONFIG_CONFARGS").split(" ") should fix the issue.
But from a bitbake perspective you should really think about choosing a less pythonic version

Best

Konrad

On 01.04.20 22:00, Joel Winarske wrote:
Hi Konrad,

I left out the details of the path encoding :)  Effectively I was attempting to duplicate get_out_dir() as found here:


Cheers,
Joel

On Wed, Apr 1, 2020 at 10:46 AM Konrad Weihmann <kweihmann@...> wrote:

There is a single tick missing, but I'm sure you get the point

On 01.04.20 19:37, Konrad Weihmann wrote:

What about

do_configure() {

   cd ${@bb.utils.contains('PACKAGECONFIG_CONFARGS', '--verbose', out/verbose', 'out', d)}

}

On 01.04.20 18:55, Joel Winarske wrote:
Hello,

I'm trying the below in a recipe.  Bitbake gives me: ERROR: Unable to parse ...: Exited with "2"

inherit python3native

require utils.inc

do_configure() {

    cd ${@get_out_dir(d)}
}

(in utils.inc)
def get_out_dir(d):
    import os
    import argparse
    pkgconfig_args = d.getVar("PACKAGECONFIG_CONFARGS")

    parser = argparse.ArgumentParser()
    parser.add_argument("--verbose", help="increase output verbosity",
                        action="store_true")
    args = parser.parse_args(pkgconfig_args)
    if args.verbose:
        return os.path.join('out', 'verbose')

    return 'out'



What am I doing wrong, or is there a better way?  Seems like it doesn't like "args = parser.parse_args(pkgconfig_args)"

The output directory is a combination of the arguments.


Thanks!
Joel




Re: python argparse or alternate method

Tim Orling
 



On Wed, Apr 1, 2020 at 1:09 PM Konrad Weihmann <kweihmann@...> wrote:

Hi Joel,

okay, I see - although I have to admit the argparse idea looks very weird, the error is that according to
https://docs.python.org/3/library/argparse.html#argparse.ArgumentParser.parse_args
the pkgconfig_args variable should be a list not a single string.

so doing pkgconfig_args = d.getVar("PACKAGECONFIG_CONFARGS").split(" ") should fix the issue.
But from a bitbake perspective you should really think about choosing a less pythonic version


You might also want to review the documentation on how PACKAGECONFIG is intended to be used:


In your recipe, each is a single command line flag, without the remaining fields. Looks odd from a Yocto Project perspective.

Best

Konrad

On 01.04.20 22:00, Joel Winarske wrote:
Hi Konrad,

I left out the details of the path encoding :)  Effectively I was attempting to duplicate get_out_dir() as found here:


Cheers,
Joel

On Wed, Apr 1, 2020 at 10:46 AM Konrad Weihmann <kweihmann@...> wrote:

There is a single tick missing, but I'm sure you get the point

On 01.04.20 19:37, Konrad Weihmann wrote:

What about

do_configure() {

   cd ${@bb.utils.contains('PACKAGECONFIG_CONFARGS', '--verbose', out/verbose', 'out', d)}

}

On 01.04.20 18:55, Joel Winarske wrote:
Hello,

I'm trying the below in a recipe.  Bitbake gives me: ERROR: Unable to parse ...: Exited with "2"

inherit python3native

require utils.inc

do_configure() {

    cd ${@get_out_dir(d)}
}

(in utils.inc)
def get_out_dir(d):
    import os
    import argparse
    pkgconfig_args = d.getVar("PACKAGECONFIG_CONFARGS")

    parser = argparse.ArgumentParser()
    parser.add_argument("--verbose", help="increase output verbosity",
                        action="store_true")
    args = parser.parse_args(pkgconfig_args)
    if args.verbose:
        return os.path.join('out', 'verbose')

    return 'out'



What am I doing wrong, or is there a better way?  Seems like it doesn't like "args = parser.parse_args(pkgconfig_args)"

The output directory is a combination of the arguments.


Thanks!
Joel





Re: python argparse or alternate method

Konrad Weihmann <kweihmann@...>
 

Hi Joel,

okay, I see - although I have to admit the argparse idea looks very weird, the error is that according to
https://docs.python.org/3/library/argparse.html#argparse.ArgumentParser.parse_args
the pkgconfig_args variable should be a list not a single string.

so doing pkgconfig_args = d.getVar("PACKAGECONFIG_CONFARGS").split(" ") should fix the issue.
But from a bitbake perspective you should really think about choosing a less pythonic version

Best

Konrad

On 01.04.20 22:00, Joel Winarske wrote:
Hi Konrad,

I left out the details of the path encoding :)  Effectively I was attempting to duplicate get_out_dir() as found here:


Cheers,
Joel

On Wed, Apr 1, 2020 at 10:46 AM Konrad Weihmann <kweihmann@...> wrote:

There is a single tick missing, but I'm sure you get the point

On 01.04.20 19:37, Konrad Weihmann wrote:

What about

do_configure() {

   cd ${@bb.utils.contains('PACKAGECONFIG_CONFARGS', '--verbose', out/verbose', 'out', d)}

}

On 01.04.20 18:55, Joel Winarske wrote:
Hello,

I'm trying the below in a recipe.  Bitbake gives me: ERROR: Unable to parse ...: Exited with "2"

inherit python3native

require utils.inc

do_configure() {

    cd ${@get_out_dir(d)}
}

(in utils.inc)
def get_out_dir(d):
    import os
    import argparse
    pkgconfig_args = d.getVar("PACKAGECONFIG_CONFARGS")

    parser = argparse.ArgumentParser()
    parser.add_argument("--verbose", help="increase output verbosity",
                        action="store_true")
    args = parser.parse_args(pkgconfig_args)
    if args.verbose:
        return os.path.join('out', 'verbose')

    return 'out'



What am I doing wrong, or is there a better way?  Seems like it doesn't like "args = parser.parse_args(pkgconfig_args)"

The output directory is a combination of the arguments.


Thanks!
Joel




Re: python argparse or alternate method

Joel Winarske
 

Hi Konrad,

I left out the details of the path encoding :)  Effectively I was attempting to duplicate get_out_dir() as found here:


Cheers,
Joel

On Wed, Apr 1, 2020 at 10:46 AM Konrad Weihmann <kweihmann@...> wrote:

There is a single tick missing, but I'm sure you get the point

On 01.04.20 19:37, Konrad Weihmann wrote:

What about

do_configure() {

   cd ${@bb.utils.contains('PACKAGECONFIG_CONFARGS', '--verbose', out/verbose', 'out', d)}

}

On 01.04.20 18:55, Joel Winarske wrote:
Hello,

I'm trying the below in a recipe.  Bitbake gives me: ERROR: Unable to parse ...: Exited with "2"

inherit python3native

require utils.inc

do_configure() {

    cd ${@get_out_dir(d)}
}

(in utils.inc)
def get_out_dir(d):
    import os
    import argparse
    pkgconfig_args = d.getVar("PACKAGECONFIG_CONFARGS")

    parser = argparse.ArgumentParser()
    parser.add_argument("--verbose", help="increase output verbosity",
                        action="store_true")
    args = parser.parse_args(pkgconfig_args)
    if args.verbose:
        return os.path.join('out', 'verbose')

    return 'out'



What am I doing wrong, or is there a better way?  Seems like it doesn't like "args = parser.parse_args(pkgconfig_args)"

The output directory is a combination of the arguments.


Thanks!
Joel



    


Re: python argparse or alternate method

Konrad Weihmann <kweihmann@...>
 

There is a single tick missing, but I'm sure you get the point

On 01.04.20 19:37, Konrad Weihmann wrote:

What about

do_configure() {

   cd ${@bb.utils.contains('PACKAGECONFIG_CONFARGS', '--verbose', out/verbose', 'out', d)}

}

On 01.04.20 18:55, Joel Winarske wrote:
Hello,

I'm trying the below in a recipe.  Bitbake gives me: ERROR: Unable to parse ...: Exited with "2"

inherit python3native

require utils.inc

do_configure() {

    cd ${@get_out_dir(d)}
}

(in utils.inc)
def get_out_dir(d):
    import os
    import argparse
    pkgconfig_args = d.getVar("PACKAGECONFIG_CONFARGS")

    parser = argparse.ArgumentParser()
    parser.add_argument("--verbose", help="increase output verbosity",
                        action="store_true")
    args = parser.parse_args(pkgconfig_args)
    if args.verbose:
        return os.path.join('out', 'verbose')

    return 'out'



What am I doing wrong, or is there a better way?  Seems like it doesn't like "args = parser.parse_args(pkgconfig_args)"

The output directory is a combination of the arguments.


Thanks!
Joel



    


Re: python argparse or alternate method

Konrad Weihmann <kweihmann@...>
 

What about

do_configure() {

   cd ${@bb.utils.contains('PACKAGECONFIG_CONFARGS', '--verbose', out/verbose', 'out', d)}

}

On 01.04.20 18:55, Joel Winarske wrote:
Hello,

I'm trying the below in a recipe.  Bitbake gives me: ERROR: Unable to parse ...: Exited with "2"

inherit python3native

require utils.inc

do_configure() {

    cd ${@get_out_dir(d)}
}

(in utils.inc)
def get_out_dir(d):
    import os
    import argparse
    pkgconfig_args = d.getVar("PACKAGECONFIG_CONFARGS")

    parser = argparse.ArgumentParser()
    parser.add_argument("--verbose", help="increase output verbosity",
                        action="store_true")
    args = parser.parse_args(pkgconfig_args)
    if args.verbose:
        return os.path.join('out', 'verbose')

    return 'out'



What am I doing wrong, or is there a better way?  Seems like it doesn't like "args = parser.parse_args(pkgconfig_args)"

The output directory is a combination of the arguments.


Thanks!
Joel


    


python argparse or alternate method

Joel Winarske
 

Hello,

I'm trying the below in a recipe.  Bitbake gives me: ERROR: Unable to parse ...: Exited with "2"

inherit python3native

require utils.inc

do_configure() {

    cd ${@get_out_dir(d)}
}

(in utils.inc)
def get_out_dir(d):
    import os
    import argparse
    pkgconfig_args = d.getVar("PACKAGECONFIG_CONFARGS")

    parser = argparse.ArgumentParser()
    parser.add_argument("--verbose", help="increase output verbosity",
                        action="store_true")
    args = parser.parse_args(pkgconfig_args)
    if args.verbose:
        return os.path.join('out', 'verbose')

    return 'out'



What am I doing wrong, or is there a better way?  Seems like it doesn't like "args = parser.parse_args(pkgconfig_args)"

The output directory is a combination of the arguments.


Thanks!
Joel

8361 - 8380 of 57384