<div dir="ltr"><div>Thanks for replying!</div><div><br></div>That is effectively what we have to do with the VM clone of the build machine. We spin up the VM, do the build, see what is new in the downloads directory, do our due diligence, and push the new stuff into the local mirror behind the firewall. The primary build server must remain "sterile", so everything has to be "hoisted" over the fence manually.<div><br></div><div>[Side Note: We also maintain a local clone of the openembedded and poky repositories. Our build pins to a specific hash, which gets advanced after we do our due diligence. This is when we discover that certain packages need to be added to our ${LOCAL_MIRROR}. My desire is to automate as much as possible, while exposing as little as possible.]<br><br>Whenever the nightly build fails due to a missing package, and I see a URL with a (for example) .tar.gz in the build error log, I know my job is a lot easier. I can simply wget from any Internet connected host, instead of spinning up the VM clone, waiting for a build to complete, and then figuring out what is new in the downloads directory.<div><br></div><div>I was hoping that there was an equally simple way of grabbing git repository packages from an Internet connected host without having a bitbake build cache them for me.</div><div><br></div><div>I understand that these packages are effectively a bare clone, but there seems to be some additional non-obvious (to me at least) "magic" that makes it more complicated than doing a git repo bare clone and wrapping it up into a .tar.gz. </div><div><br></div><div>I am happy to start trawling through source code and figure it out on my own, but I figured this had to be common knowledge to someone on this list.</div><div><br></div><div>..Ch:W..</div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Sep 26, 2019 at 8:50 PM Rudolf J Streif <<a href="mailto:rudolf.streif@ibeeto.com">rudolf.streif@ibeeto.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<p>Chuck,</p>
<p>Unless I am missing something (which I cannot entirely rule out),
the default PREMIRRORS should actually do what you want. For the
poky distro configuration has this:</p>
<p>PREMIRRORS ??= "\<br>
bzr://.*/.*Â Â <a class="gmail-m_-2831024795406922359moz-txt-link-freetext" href="http://downloads.yoctoproject.org/mirror/sources/" target="_blank">http://downloads.yoctoproject.org/mirror/sources/</a> \n
\<br>
cvs://.*/.*Â Â <a class="gmail-m_-2831024795406922359moz-txt-link-freetext" href="http://downloads.yoctoproject.org/mirror/sources/" target="_blank">http://downloads.yoctoproject.org/mirror/sources/</a> \n
\<br>
git://.*/.*Â Â <a class="gmail-m_-2831024795406922359moz-txt-link-freetext" href="http://downloads.yoctoproject.org/mirror/sources/" target="_blank">http://downloads.yoctoproject.org/mirror/sources/</a> \n
\<br>
gitsm://.*/.* <a class="gmail-m_-2831024795406922359moz-txt-link-freetext" href="http://downloads.yoctoproject.org/mirror/sources/" target="_blank">http://downloads.yoctoproject.org/mirror/sources/</a> \n
\<br>
hg://.*/.*Â Â Â <a class="gmail-m_-2831024795406922359moz-txt-link-freetext" href="http://downloads.yoctoproject.org/mirror/sources/" target="_blank">http://downloads.yoctoproject.org/mirror/sources/</a> \n
\<br>
osc://.*/.*Â Â <a class="gmail-m_-2831024795406922359moz-txt-link-freetext" href="http://downloads.yoctoproject.org/mirror/sources/" target="_blank">http://downloads.yoctoproject.org/mirror/sources/</a> \n
\<br>
p4://.*/.*Â Â Â <a class="gmail-m_-2831024795406922359moz-txt-link-freetext" href="http://downloads.yoctoproject.org/mirror/sources/" target="_blank">http://downloads.yoctoproject.org/mirror/sources/</a> \n
\<br>
svn://.*/.*Â Â <a class="gmail-m_-2831024795406922359moz-txt-link-freetext" href="http://downloads.yoctoproject.org/mirror/sources/" target="_blank">http://downloads.yoctoproject.org/mirror/sources/</a>
\n"<br>
</p>
<p>This causes the fetcher to intercept and translate all the SCM
protocols into http downloads from the YP mirror. Since you are
saying wget works for you this should actually automatically work
for you. If you build system can at least download using http then</p>
<p>bitbake <target> --runall=fetch</p>
<p>should download all the sources for you. Then you can disconnect
the box from the Internet and start the build offline.</p>
<p>:rjs<br>
</p>
<div class="gmail-m_-2831024795406922359moz-cite-prefix">On 9/26/19 8:06 PM, Chuck Wolber wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>Hi All,</div>
<div><br>
</div>
<div>Our build environment is stuck behind a firewall with no
access to the Internet. When a Yocto build requires a new
package, we normally extract the URL from the failed build log
and "wget" the package from the upstream source, do our due
diligence (checksums, etc), and then add it to the server
pointed to by ${LOCAL_MIRROR}.</div>
<div><br>
</div>
<div>Obviously that procedure does not work so well when a git
repository needs to be fetched. We have tried various methods,
but nothing seems to work reliably. Our current method is an
annoying hack involving a virtual machine clone of the build
server.</div>
<div><br>
</div>
<div>I have searched far and wide, and cannot find any sort of
procedure. Can someone point me at a procedure for translating
something like git://<a href="http://git.yoctoproject.org/pseudo" target="_blank">git.yoctoproject.org/pseudo</a> into
a file we can store in our ${LOCAL_MIRROR}?</div>
<div><br>
</div>
<div>Thank you,</div>
<div><br>
</div>
<div>..Ch:W..</div>
</div>
<br>
<fieldset class="gmail-m_-2831024795406922359mimeAttachmentHeader"></fieldset>
</blockquote>
<pre class="gmail-m_-2831024795406922359moz-signature" cols="72">--
-----
Rudolf J Streif
CEO/CTO ibeeto
+1.855.442.3386 x700</pre>
</div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr">“I would challenge anyone here to think of a question upon which we once had a scientific answer, however inadequate, but for which now the best answer is a religious one." -Sam Harris</div></div>