Issue with accessing network from a recipe


Gärding Antti
 

Hi,

 

I noticed that the another question was probably due to my own mistake: Accessing network from do_fetch seems to work, but I hadn’t noticed that I had left commands requiring network into do_compile even though I had moved those of them that were in do_configure to do_fetch.

 

 

Best regards,

Antti Gärding

 

From: Gärding Antti <Antti.Garding@...>
Sent: Tuesday, December 20, 2022 9:42
To: Martin Jansa <martin.jansa@...>
Cc: yocto@...
Subject: Re: [yocto] Issue with accessing network from a recipe

 

Hi,

 

Thanks! That helped my team find out a solution or at least a work-around: setting do_configure[network] and do_compile[network] to "1" in local.conf.

 

I would have another related questions, though: Should the commands requiring network access work if they are invoked from do_fetch? I tried that and it didn't work.

 

 

Best regards,

Antti Gärding

 


Kohteesta: Martin Jansa <martin.jansa@...>
Lähetetty: maanantaina 19. joulukuuta 2022 klo 16.16
Vastaanottaja: Gärding Antti <Antti.Garding@...>
Kopio: yocto@... <yocto@...>
Aihe: Re: [yocto] Issue with accessing network from a recipe

 

Hi,

 

that is intentional, recipes should fetch sources with bitbake fetcher (which respects MIRROR/PREMIRROR/DL_DIR etc) in do_fetch and network shouldn't be used in other tasks (unless explicitly enabled in well explained exceptions).

 

bitbake can now restrict network access in various tasks based on "network" flag as implemented in:
  https://git.openembedded.org/bitbake/commit/?id=0746b6a2a32fec4c18bf1a52b1454ca4c04bf543
  https://git.openembedded.org/bitbake/commit/?id=9d6341df611a1725090444f6f8eb0244aed08213
and oe now enables network mostly only in do_fetch and few special tasks as implemented in:
  https://git.openembedded.org/openembedded-core/commit/?id=7ce1e88a3ad85bbb925bb9f7167dc0a5fd1c27f4

 

Regards,

 

On Mon, Dec 19, 2022 at 2:02 PM Gärding Antti via lists.yoctoproject.org <Antti.Garding=etteplan.com@...> wrote:

Hi,

 

I am having an issue with a recipe whose do_configure requires network connections and I wonder if this issue could be related to Yocto so that this list would be the correct place to ask for ideas.

 

The problem appears when trying to build dotnet-hello-world which comes with meta-dotnet (https://github.com/jeremy-prater/meta-dotnet). I am not the only one having this issue, but many people, including the layer's author, also report having no problems. Googling gives a lot of suggestions, but as this is related to .NET, all of them are related to Windows / Visual Studio environment and I don't know how to apply them using Yocto.

 

In short, do_configure for dotnet-hello-world invokes the .NET SDK's tool dotnet as a host tool to fetch what is needed to build and run the program. As a part of that, it uses NuGet, the package manager for .NET, which says "error NU1301: Unable to load the service index for source https://api.nuget.org/v3/index.json".

 

If I modify the recipe so that it tries to curl that file, I get "Could not resolve host: api.nuget.org". If I do the same but use numeric address, I get "Couldn't connect to server".

 

If I run the commands in the recipe manually in the same environment I use for Yocto builds, using .NET SDK I have installed manually, they work.

 

My build environment is Ubuntu 20.04 run inside a Docker container.

 

 

Best regards,

Antti Gärding


 


Gärding Antti
 

Hi,

Thanks! That helped my team find out a solution or at least a work-around: setting do_configure[network] and do_compile[network] to "1" in local.conf.

I would have another related questions, though: Should the commands requiring network access work if they are invoked from do_fetch? I tried that and it didn't work.


Best regards,
Antti Gärding


Kohteesta: Martin Jansa <martin.jansa@...>
Lähetetty: maanantaina 19. joulukuuta 2022 klo 16.16
Vastaanottaja: Gärding Antti <Antti.Garding@...>
Kopio: yocto@... <yocto@...>
Aihe: Re: [yocto] Issue with accessing network from a recipe

Hi,

that is intentional, recipes should fetch sources with bitbake fetcher (which respects MIRROR/PREMIRROR/DL_DIR etc) in do_fetch and network shouldn't be used in other tasks (unless explicitly enabled in well explained exceptions).

bitbake can now restrict network access in various tasks based on "network" flag as implemented in:
  https://git.openembedded.org/bitbake/commit/?id=0746b6a2a32fec4c18bf1a52b1454ca4c04bf543
  https://git.openembedded.org/bitbake/commit/?id=9d6341df611a1725090444f6f8eb0244aed08213
and oe now enables network mostly only in do_fetch and few special tasks as implemented in:
  https://git.openembedded.org/openembedded-core/commit/?id=7ce1e88a3ad85bbb925bb9f7167dc0a5fd1c27f4

Regards,

On Mon, Dec 19, 2022 at 2:02 PM Gärding Antti via lists.yoctoproject.org <Antti.Garding=etteplan.com@...> wrote:

Hi,

 

I am having an issue with a recipe whose do_configure requires network connections and I wonder if this issue could be related to Yocto so that this list would be the correct place to ask for ideas.

 

The problem appears when trying to build dotnet-hello-world which comes with meta-dotnet (https://github.com/jeremy-prater/meta-dotnet). I am not the only one having this issue, but many people, including the layer's author, also report having no problems. Googling gives a lot of suggestions, but as this is related to .NET, all of them are related to Windows / Visual Studio environment and I don't know how to apply them using Yocto.

 

In short, do_configure for dotnet-hello-world invokes the .NET SDK's tool dotnet as a host tool to fetch what is needed to build and run the program. As a part of that, it uses NuGet, the package manager for .NET, which says "error NU1301: Unable to load the service index for source https://api.nuget.org/v3/index.json".

 

If I modify the recipe so that it tries to curl that file, I get "Could not resolve host: api.nuget.org". If I do the same but use numeric address, I get "Couldn't connect to server".

 

If I run the commands in the recipe manually in the same environment I use for Yocto builds, using .NET SDK I have installed manually, they work.

 

My build environment is Ubuntu 20.04 run inside a Docker container.

 

 

Best regards,

Antti Gärding






Martin Jansa
 

Hi,

that is intentional, recipes should fetch sources with bitbake fetcher (which respects MIRROR/PREMIRROR/DL_DIR etc) in do_fetch and network shouldn't be used in other tasks (unless explicitly enabled in well explained exceptions).

bitbake can now restrict network access in various tasks based on "network" flag as implemented in:
  https://git.openembedded.org/bitbake/commit/?id=0746b6a2a32fec4c18bf1a52b1454ca4c04bf543
  https://git.openembedded.org/bitbake/commit/?id=9d6341df611a1725090444f6f8eb0244aed08213
and oe now enables network mostly only in do_fetch and few special tasks as implemented in:
  https://git.openembedded.org/openembedded-core/commit/?id=7ce1e88a3ad85bbb925bb9f7167dc0a5fd1c27f4

Regards,

On Mon, Dec 19, 2022 at 2:02 PM Gärding Antti via lists.yoctoproject.org <Antti.Garding=etteplan.com@...> wrote:

Hi,

 

I am having an issue with a recipe whose do_configure requires network connections and I wonder if this issue could be related to Yocto so that this list would be the correct place to ask for ideas.

 

The problem appears when trying to build dotnet-hello-world which comes with meta-dotnet (https://github.com/jeremy-prater/meta-dotnet). I am not the only one having this issue, but many people, including the layer's author, also report having no problems. Googling gives a lot of suggestions, but as this is related to .NET, all of them are related to Windows / Visual Studio environment and I don't know how to apply them using Yocto.

 

In short, do_configure for dotnet-hello-world invokes the .NET SDK's tool dotnet as a host tool to fetch what is needed to build and run the program. As a part of that, it uses NuGet, the package manager for .NET, which says "error NU1301: Unable to load the service index for source https://api.nuget.org/v3/index.json".

 

If I modify the recipe so that it tries to curl that file, I get "Could not resolve host: api.nuget.org". If I do the same but use numeric address, I get "Couldn't connect to server".

 

If I run the commands in the recipe manually in the same environment I use for Yocto builds, using .NET SDK I have installed manually, they work.

 

My build environment is Ubuntu 20.04 run inside a Docker container.

 

 

Best regards,

Antti Gärding





Gärding Antti
 

Hi,

 

I am having an issue with a recipe whose do_configure requires network connections and I wonder if this issue could be related to Yocto so that this list would be the correct place to ask for ideas.

 

The problem appears when trying to build dotnet-hello-world which comes with meta-dotnet (https://github.com/jeremy-prater/meta-dotnet). I am not the only one having this issue, but many people, including the layer's author, also report having no problems. Googling gives a lot of suggestions, but as this is related to .NET, all of them are related to Windows / Visual Studio environment and I don't know how to apply them using Yocto.

 

In short, do_configure for dotnet-hello-world invokes the .NET SDK's tool dotnet as a host tool to fetch what is needed to build and run the program. As a part of that, it uses NuGet, the package manager for .NET, which says "error NU1301: Unable to load the service index for source https://api.nuget.org/v3/index.json".

 

If I modify the recipe so that it tries to curl that file, I get "Could not resolve host: api.nuget.org". If I do the same but use numeric address, I get "Couldn't connect to server".

 

If I run the commands in the recipe manually in the same environment I use for Yocto builds, using .NET SDK I have installed manually, they work.

 

My build environment is Ubuntu 20.04 run inside a Docker container.

 

 

Best regards,

Antti Gärding