Date
1 - 9 of 9
devtool upgrade for go recipes with git "broken"
|
Thomas Roos
Hi,
Amazon Web Services EMEA SARL 38 avenue John F. Kennedy, L-1855 Luxembourg Sitz der Gesellschaft: L-1855 Luxemburg eingetragen im Luxemburgischen Handelsregister unter R.C.S. B186284 Amazon Web Services EMEA SARL, Niederlassung Deutschland Marcel-Breuer-Str. 12, D-80807 Muenchen Sitz der Zweigniederlassung: Muenchen eingetragen im Handelsregister des Amtsgerichts Muenchen unter HRB 242240, USt-ID DE317013094 |
|
|
|
Alexander Kanavin
It helps if you show the exact error, and a way to reproduce this
using a minimal setup (ideally only poky). Alex On Thu, 20 Oct 2022 at 11:18, Thomas Roos via lists.yoctoproject.org <throos=amazon.de@...> wrote:
|
|
|
|
Thomas Roos
yes, easy "devtool upgrade go-helloworld"
INFO: Extracting upgraded version source...
Traceback (most recent call last): File "/home/ubuntu/yocto/poky/scripts/lib/devtool/upgrade.py", line 567, in upgrade rev2, md5, sha256, srcbranch, srcsubdir2 = _extract_new_source(args.version, srctree, args.no_patch, File "/home/ubuntu/yocto/poky/scripts/lib/devtool/upgrade.py", line 183, in _extract_new_source __run('git checkout %s' % rev) File "/home/ubuntu/yocto/poky/scripts/lib/devtool/upgrade.py", line 170, in __run return _run(cmd, srctree) File "/home/ubuntu/yocto/poky/scripts/lib/devtool/upgrade.py", line 31, in _run return bb.process.run('%s' % cmd, cwd=cwd) File "/home/ubuntu/yocto/poky/bitbake/lib/bb/process.py", line 189, in run raise ExecutionError(cmd, pipe.returncode, stdout, stderr) bb.process.ExecutionError: Execution of 'git checkout 2e68773dfca072cb81f219fc3b97ad34fe9d9f94' failed with exit code 128: fatal: reference is not a tree: 2e68773dfca072cb81f219fc3b97ad34fe9d9f94 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/ubuntu/yocto/poky/scripts/devtool", line 338, in <module> ret = main() File "/home/ubuntu/yocto/poky/scripts/devtool", line 325, in main ret = args.func(args, config, basepath, workspace) File "/home/ubuntu/yocto/poky/scripts/lib/devtool/upgrade.py", line 575, in upgrade _upgrade_error(e, recipedir, srctree, args.keep_failure) File "/home/ubuntu/yocto/poky/scripts/lib/devtool/upgrade.py", line 134, in _upgrade_error _cleanup_on_error(rd, srctree) File "/home/ubuntu/yocto/poky/scripts/lib/devtool/upgrade.py", line 126, in _cleanup_on_error if not len(os.listdir(rdp)): FileNotFoundError: [Errno 2] No such file or directory: '/home/ubuntu/yocto/build/workspace/recipes' Von: Alexander Kanavin <alex.kanavin@...>
Gesendet: Donnerstag, 20. Oktober 2022 11:32:19 An: Roos, Thomas Cc: yocto@... Betreff: RE: [EXTERNAL][yocto] devtool upgrade for go recipes with git "broken" CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
It helps if you show the exact error, and a way to reproduce this using a minimal setup (ideally only poky). Alex On Thu, 20 Oct 2022 at 11:18, Thomas Roos via lists.yoctoproject.org <throos=amazon.de@...> wrote: > > Hi, > when doing a "devtool upgrade" for a recipe which inherits "go" and using a SRC_URI to a git repo this will fail. Complaining that the git revision is "out of tree" - I found that the additional "src" folder inside the checkout out sources is causing this problem and I wondering if this is necessary. I'm not a go programmer, but when changing this the recipe seems to produce the same result. > I did change our go recipe to don't use this additional src folder by overwriting the go_do_upack and go_do_configure method. Here: https://github.com/aws4embeddedlinux/meta-aws/blob/942651026c7a91b9703609d7658bd1649350acaf/recipes-support/amazon-ssm-agent/amazon-ssm-agent_3.1.1856.0.bb > Is this a bug or do anybody know why go use that additional src folder? > > > Cheers, > Thomas > > > > > Amazon Web Services EMEA SARL > 38 avenue John F. Kennedy, L-1855 Luxembourg > Sitz der Gesellschaft: L-1855 Luxemburg > eingetragen im Luxemburgischen Handelsregister unter R.C.S. B186284 > > Amazon Web Services EMEA SARL, Niederlassung Deutschland > Marcel-Breuer-Str. 12, D-80807 Muenchen > Sitz der Zweigniederlassung: Muenchen > eingetragen im Handelsregister des Amtsgerichts Muenchen unter HRB 242240, USt-ID DE317013094 > > > > > Amazon Web Services EMEA SARL 38 avenue John F. Kennedy, L-1855 Luxembourg Sitz der Gesellschaft: L-1855 Luxemburg eingetragen im Luxemburgischen Handelsregister unter R.C.S. B186284 Amazon Web Services EMEA SARL, Niederlassung Deutschland Marcel-Breuer-Str. 12, D-80807 Muenchen Sitz der Zweigniederlassung: Muenchen eingetragen im Handelsregister des Amtsgerichts Muenchen unter HRB 242240, USt-ID DE317013094 |
|
|
|
Alexander Kanavin
Thanks, I do not have a quick answer to this, but would appreciate if
toggle quoted message
Show quoted text
you can dig deeper into the issue. Alex On Thu, 20 Oct 2022 at 11:50, Roos, Thomas <throos@...> wrote:
|
|
|
|
Thomas Roos
yes, I already did this - it's because that go.bbclass is spangly using a subdir called "src" and that will break the devtool check for upgrades mechanism because it's not looking into a src subdir. So git will fail checkout anything in the parent dir "reference
is not a tree"
# src folder will break devtool upgrade
python go_do_unpack() { src_uri = (d.getVar('SRC_URI') or "").split() if len(src_uri) == 0: return fetcher = bb.fetch2.Fetch(src_uri, d) for url in fetcher.urls: if fetcher.ud[url].type == 'git': if fetcher.ud[url].parm.get('destsuffix') is None: s_dirname = os.path.basename(d.getVar('S')) #cause problems fetcher.ud[url].parm['destsuffix'] = os.path.join(s_dirname, 'src', d.getVar('GO_IMPORT')) + '/' #FIX
fetcher.ud[url].parm['destsuffix'] = os.path.join(s_dirname, '', d.getVar('GO_IMPORT')) + '/' #FIX
fetcher.unpack(d.getVar('WORKDIR')) } # src folder will break devtool upgrade go_do_configure() { #cause problems ln -snf ${S}/src ${B}/ #FIX
ln -snf ${S} ${B}/ #FIX
} Von: Alexander Kanavin <alex.kanavin@...>
Gesendet: Donnerstag, 20. Oktober 2022 11:56:48 An: Roos, Thomas Cc: yocto@... Betreff: RE: [EXTERNAL][yocto] devtool upgrade for go recipes with git "broken" CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
Thanks, I do not have a quick answer to this, but would appreciate if you can dig deeper into the issue. Alex On Thu, 20 Oct 2022 at 11:50, Roos, Thomas <throos@...> wrote: > > yes, easy > - set the go-helloworld bb srcrev to an older rev - e.g: > SRCREV = "ad95e7f791d8e1e7e7ec7f7cedfc2f6b29f6c9e2" > > "devtool upgrade go-helloworld" > > INFO: Extracting upgraded version source... > Traceback (most recent call last): > File "/home/ubuntu/yocto/poky/scripts/lib/devtool/upgrade.py", line 567, in upgrade > rev2, md5, sha256, srcbranch, srcsubdir2 = _extract_new_source(args.version, srctree, args.no_patch, > File "/home/ubuntu/yocto/poky/scripts/lib/devtool/upgrade.py", line 183, in _extract_new_source > __run('git checkout %s' % rev) > File "/home/ubuntu/yocto/poky/scripts/lib/devtool/upgrade.py", line 170, in __run > return _run(cmd, srctree) > File "/home/ubuntu/yocto/poky/scripts/lib/devtool/upgrade.py", line 31, in _run > return bb.process.run('%s' % cmd, cwd=cwd) > File "/home/ubuntu/yocto/poky/bitbake/lib/bb/process.py", line 189, in run > raise ExecutionError(cmd, pipe.returncode, stdout, stderr) > bb.process.ExecutionError: Execution of 'git checkout 2e68773dfca072cb81f219fc3b97ad34fe9d9f94' failed with exit code 128: > fatal: reference is not a tree: 2e68773dfca072cb81f219fc3b97ad34fe9d9f94 > > > During handling of the above exception, another exception occurred: > > Traceback (most recent call last): > File "/home/ubuntu/yocto/poky/scripts/devtool", line 338, in <module> > ret = main() > File "/home/ubuntu/yocto/poky/scripts/devtool", line 325, in main > ret = args.func(args, config, basepath, workspace) > File "/home/ubuntu/yocto/poky/scripts/lib/devtool/upgrade.py", line 575, in upgrade > _upgrade_error(e, recipedir, srctree, args.keep_failure) > File "/home/ubuntu/yocto/poky/scripts/lib/devtool/upgrade.py", line 134, in _upgrade_error > _cleanup_on_error(rd, srctree) > File "/home/ubuntu/yocto/poky/scripts/lib/devtool/upgrade.py", line 126, in _cleanup_on_error > if not len(os.listdir(rdp)): > FileNotFoundError: [Errno 2] No such file or directory: '/home/ubuntu/yocto/build/workspace/recipes' > > > > > > ________________________________ > Von: Alexander Kanavin <alex.kanavin@...> > Gesendet: Donnerstag, 20. Oktober 2022 11:32:19 > An: Roos, Thomas > Cc: yocto@... > Betreff: RE: [EXTERNAL][yocto] devtool upgrade for go recipes with git "broken" > > CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. > > > > It helps if you show the exact error, and a way to reproduce this > using a minimal setup (ideally only poky). > > Alex > > On Thu, 20 Oct 2022 at 11:18, Thomas Roos via lists.yoctoproject.org > <throos=amazon.de@...> wrote: > > > > Hi, > > when doing a "devtool upgrade" for a recipe which inherits "go" and using a SRC_URI to a git repo this will fail. Complaining that the git revision is "out of tree" - I found that the additional "src" folder inside the checkout out sources is causing this problem and I wondering if this is necessary. I'm not a go programmer, but when changing this the recipe seems to produce the same result. > > I did change our go recipe to don't use this additional src folder by overwriting the go_do_upack and go_do_configure method. Here: https://github.com/aws4embeddedlinux/meta-aws/blob/942651026c7a91b9703609d7658bd1649350acaf/recipes-support/amazon-ssm-agent/amazon-ssm-agent_3.1.1856.0.bb > > Is this a bug or do anybody know why go use that additional src folder? > > > > > > Cheers, > > Thomas > > > > > > > > > > Amazon Web Services EMEA SARL > > 38 avenue John F. Kennedy, L-1855 Luxembourg > > Sitz der Gesellschaft: L-1855 Luxemburg > > eingetragen im Luxemburgischen Handelsregister unter R.C.S. B186284 > > > > Amazon Web Services EMEA SARL, Niederlassung Deutschland > > Marcel-Breuer-Str. 12, D-80807 Muenchen > > Sitz der Zweigniederlassung: Muenchen > > eingetragen im Handelsregister des Amtsgerichts Muenchen unter HRB 242240, USt-ID DE317013094 > > > > > > > > > > > > > > Amazon Web Services EMEA SARL > 38 avenue John F. Kennedy, L-1855 Luxembourg > Sitz der Gesellschaft: L-1855 Luxemburg > eingetragen im Luxemburgischen Handelsregister unter R.C.S. B186284 > > Amazon Web Services EMEA SARL, Niederlassung Deutschland > Marcel-Breuer-Str. 12, D-80807 Muenchen > Sitz der Zweigniederlassung: Muenchen > eingetragen im Handelsregister des Amtsgerichts Muenchen unter HRB 242240, USt-ID DE317013094 > > Amazon Web Services EMEA SARL 38 avenue John F. Kennedy, L-1855 Luxembourg Sitz der Gesellschaft: L-1855 Luxemburg eingetragen im Luxemburgischen Handelsregister unter R.C.S. B186284 Amazon Web Services EMEA SARL, Niederlassung Deutschland Marcel-Breuer-Str. 12, D-80807 Muenchen Sitz der Zweigniederlassung: Muenchen eingetragen im Handelsregister des Amtsgerichts Muenchen unter HRB 242240, USt-ID DE317013094 |
|
|
|
Alexander Kanavin
Can you propose a patch to core?
toggle quoted message
Show quoted text
Alex On Thu, 20 Oct 2022 at 12:05, Roos, Thomas <throos@...> wrote:
|
|
|
|
Thomas Roos
sure, but may this src folder is necessary for something? Some go convention I don't know about... Von: Alexander Kanavin <alex.kanavin@...>
Gesendet: Donnerstag, 20. Oktober 2022 12:09:47 An: Roos, Thomas Cc: yocto@... Betreff: RE: [EXTERNAL][yocto] devtool upgrade for go recipes with git "broken" CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
Can you propose a patch to core? Alex On Thu, 20 Oct 2022 at 12:05, Roos, Thomas <throos@...> wrote: > > yes, I already did this - it's because that go.bbclass is spangly using a subdir called "src" and that will break the devtool check for upgrades mechanism because it's not looking into a src subdir. So git will fail checkout anything in the parent dir "reference is not a tree" > > This simple change in the go.bbclass fix it - but I don't know if that additional src dir is somehow necessary: > > > # src folder will break devtool upgrade > python go_do_unpack() { > src_uri = (d.getVar('SRC_URI') or "").split() > if len(src_uri) == 0: > return > > fetcher = bb.fetch2.Fetch(src_uri, d) > for url in fetcher.urls: > if fetcher.ud[url].type == 'git': > if fetcher.ud[url].parm.get('destsuffix') is None: > s_dirname = os.path.basename(d.getVar('S')) > #cause problems fetcher.ud[url].parm['destsuffix'] = os.path.join(s_dirname, 'src', d.getVar('GO_IMPORT')) + '/' > #FIX > fetcher.ud[url].parm['destsuffix'] = os.path.join(s_dirname, '', d.getVar('GO_IMPORT')) + '/' > #FIX > fetcher.unpack(d.getVar('WORKDIR')) > } > > # src folder will break devtool upgrade > go_do_configure() { > #cause problems ln -snf ${S}/src ${B}/ > #FIX > ln -snf ${S} ${B}/ > #FIX > } > > > ________________________________ > Von: Alexander Kanavin <alex.kanavin@...> > Gesendet: Donnerstag, 20. Oktober 2022 11:56:48 > An: Roos, Thomas > Cc: yocto@... > Betreff: RE: [EXTERNAL][yocto] devtool upgrade for go recipes with git "broken" > > CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. > > > > Thanks, I do not have a quick answer to this, but would appreciate if > you can dig deeper into the issue. > > Alex > > On Thu, 20 Oct 2022 at 11:50, Roos, Thomas <throos@...> wrote: > > > > yes, easy > > - set the go-helloworld bb srcrev to an older rev - e.g: > > SRCREV = "ad95e7f791d8e1e7e7ec7f7cedfc2f6b29f6c9e2" > > > > "devtool upgrade go-helloworld" > > > > INFO: Extracting upgraded version source... > > Traceback (most recent call last): > > File "/home/ubuntu/yocto/poky/scripts/lib/devtool/upgrade.py", line 567, in upgrade > > rev2, md5, sha256, srcbranch, srcsubdir2 = _extract_new_source(args.version, srctree, args.no_patch, > > File "/home/ubuntu/yocto/poky/scripts/lib/devtool/upgrade.py", line 183, in _extract_new_source > > __run('git checkout %s' % rev) > > File "/home/ubuntu/yocto/poky/scripts/lib/devtool/upgrade.py", line 170, in __run > > return _run(cmd, srctree) > > File "/home/ubuntu/yocto/poky/scripts/lib/devtool/upgrade.py", line 31, in _run > > return bb.process.run('%s' % cmd, cwd=cwd) > > File "/home/ubuntu/yocto/poky/bitbake/lib/bb/process.py", line 189, in run > > raise ExecutionError(cmd, pipe.returncode, stdout, stderr) > > bb.process.ExecutionError: Execution of 'git checkout 2e68773dfca072cb81f219fc3b97ad34fe9d9f94' failed with exit code 128: > > fatal: reference is not a tree: 2e68773dfca072cb81f219fc3b97ad34fe9d9f94 > > > > > > During handling of the above exception, another exception occurred: > > > > Traceback (most recent call last): > > File "/home/ubuntu/yocto/poky/scripts/devtool", line 338, in <module> > > ret = main() > > File "/home/ubuntu/yocto/poky/scripts/devtool", line 325, in main > > ret = args.func(args, config, basepath, workspace) > > File "/home/ubuntu/yocto/poky/scripts/lib/devtool/upgrade.py", line 575, in upgrade > > _upgrade_error(e, recipedir, srctree, args.keep_failure) > > File "/home/ubuntu/yocto/poky/scripts/lib/devtool/upgrade.py", line 134, in _upgrade_error > > _cleanup_on_error(rd, srctree) > > File "/home/ubuntu/yocto/poky/scripts/lib/devtool/upgrade.py", line 126, in _cleanup_on_error > > if not len(os.listdir(rdp)): > > FileNotFoundError: [Errno 2] No such file or directory: '/home/ubuntu/yocto/build/workspace/recipes' > > > > > > > > > > > > ________________________________ > > Von: Alexander Kanavin <alex.kanavin@...> > > Gesendet: Donnerstag, 20. Oktober 2022 11:32:19 > > An: Roos, Thomas > > Cc: yocto@... > > Betreff: RE: [EXTERNAL][yocto] devtool upgrade for go recipes with git "broken" > > > > CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. > > > > > > > > It helps if you show the exact error, and a way to reproduce this > > using a minimal setup (ideally only poky). > > > > Alex > > > > On Thu, 20 Oct 2022 at 11:18, Thomas Roos via lists.yoctoproject.org > > <throos=amazon.de@...> wrote: > > > > > > Hi, > > > when doing a "devtool upgrade" for a recipe which inherits "go" and using a SRC_URI to a git repo this will fail. Complaining that the git revision is "out of tree" - I found that the additional "src" folder inside the checkout out sources is causing this problem and I wondering if this is necessary. I'm not a go programmer, but when changing this the recipe seems to produce the same result. > > > I did change our go recipe to don't use this additional src folder by overwriting the go_do_upack and go_do_configure method. Here: https://github.com/aws4embeddedlinux/meta-aws/blob/942651026c7a91b9703609d7658bd1649350acaf/recipes-support/amazon-ssm-agent/amazon-ssm-agent_3.1.1856.0.bb > > > Is this a bug or do anybody know why go use that additional src folder? > > > > > > > > > Cheers, > > > Thomas > > > > > > > > > > > > > > > Amazon Web Services EMEA SARL > > > 38 avenue John F. Kennedy, L-1855 Luxembourg > > > Sitz der Gesellschaft: L-1855 Luxemburg > > > eingetragen im Luxemburgischen Handelsregister unter R.C.S. B186284 > > > > > > Amazon Web Services EMEA SARL, Niederlassung Deutschland > > > Marcel-Breuer-Str. 12, D-80807 Muenchen > > > Sitz der Zweigniederlassung: Muenchen > > > eingetragen im Handelsregister des Amtsgerichts Muenchen unter HRB 242240, USt-ID DE317013094 > > > > > > > > > > > > > > > > > > > > > > > Amazon Web Services EMEA SARL > > 38 avenue John F. Kennedy, L-1855 Luxembourg > > Sitz der Gesellschaft: L-1855 Luxemburg > > eingetragen im Luxemburgischen Handelsregister unter R.C.S. B186284 > > > > Amazon Web Services EMEA SARL, Niederlassung Deutschland > > Marcel-Breuer-Str. 12, D-80807 Muenchen > > Sitz der Zweigniederlassung: Muenchen > > eingetragen im Handelsregister des Amtsgerichts Muenchen unter HRB 242240, USt-ID DE317013094 > > > > > > > > Amazon Web Services EMEA SARL > 38 avenue John F. Kennedy, L-1855 Luxembourg > Sitz der Gesellschaft: L-1855 Luxemburg > eingetragen im Luxemburgischen Handelsregister unter R.C.S. B186284 > > Amazon Web Services EMEA SARL, Niederlassung Deutschland > Marcel-Breuer-Str. 12, D-80807 Muenchen > Sitz der Zweigniederlassung: Muenchen > eingetragen im Handelsregister des Amtsgerichts Muenchen unter HRB 242240, USt-ID DE317013094 > > Amazon Web Services EMEA SARL 38 avenue John F. Kennedy, L-1855 Luxembourg Sitz der Gesellschaft: L-1855 Luxemburg eingetragen im Luxemburgischen Handelsregister unter R.C.S. B186284 Amazon Web Services EMEA SARL, Niederlassung Deutschland Marcel-Breuer-Str. 12, D-80807 Muenchen Sitz der Zweigniederlassung: Muenchen eingetragen im Handelsregister des Amtsgerichts Muenchen unter HRB 242240, USt-ID DE317013094 |
|
|
|
Alexander Kanavin
It may well be that no one remembers anymore why it's there, and in
toggle quoted message
Show quoted text
this case you can just drop it. Or you can dig into git history with 'git blame' and similar techniques. Alex On Thu, 20 Oct 2022 at 12:23, Roos, Thomas <throos@...> wrote:
|
|
|
|
Monsees, Steven C (US)
Thanks...
toggle quoted message
Show quoted text
-----Original Message-----
From: yocto@... <yocto@...> On Behalf Of Alexander Kanavin Sent: Thursday, October 20, 2022 6:40 AM To: Roos, Thomas <throos@...> Cc: yocto@... Subject: Re: [yocto] devtool upgrade for go recipes with git "broken" External Email Alert This email has been sent from an account outside of the BAE Systems network. Please treat the email with caution, especially if you are requested to click on a link, decrypt/open an attachment, or enable macros. For further information on how to spot phishing, access “Cybersecurity OneSpace Page” and report phishing by clicking the button “Report Phishing” on the Outlook toolbar. It may well be that no one remembers anymore why it's there, and in this case you can just drop it. Or you can dig into git history with 'git blame' and similar techniques. Alex On Thu, 20 Oct 2022 at 12:23, Roos, Thomas <throos@...> wrote:
|
|
|