How to use mirror to download linux-yocto?


Bo YU <tsu.yubo@...>
 

Hi,

I hope the mail list is the right place to ask the simple question about yocto.

I am using meta-riscv[0] to build riscv32 qemu.
And after `$(MACHINE=qemuriscv32 bitbake core-image-full-cmdline)`
Everything is downloaded well but linux-yocto:
```
0: linux-yocto-6.1.14+gitAUTOINC+e8d08fc4c0_b05ca3429c-r0 do_fetch -
12h26m11s (pid 4110422) 28% |####### | 2.00 KiB/s
openembedded-core/meta/recipes-kernel/linux/linux-yocto_6.1.bb
```
I tried many time but nothing change.

There is one mirror[1] I can use under my network condition.
So I want to know what is right way to alter URI in
openembedded-core/meta/recipes-kernel/linux/linux-yocto_6.1.bb?

I googled one article to suggest use Insteadof in .gitconfig[2], is it ok?

Many thanks for your help.

BR,
Bo

[0]: https://github.com/riscv/meta-riscv
[1]: https://isrc.iscas.ac.cn/gitlab/mirrors/git.yoctoproject.org/linux-yocto
[3]: https://www.wpgdadatong.com/cn/blog/detail/44614


Alexander Kanavin
 

When the slow download is happening, can you please check (e.g. from a process listing), which server specifically is being accessed and with which command? Just to be sure where the problem is.

Alex

On Fri 17. Mar 2023 at 15.08, Bo YU <tsu.yubo@...> wrote:
Hi,

I hope the mail list is the right place to ask the simple question about yocto.

I am using meta-riscv[0] to build riscv32 qemu.
And after `$(MACHINE=qemuriscv32 bitbake core-image-full-cmdline)`
Everything is downloaded well but linux-yocto:
```
0: linux-yocto-6.1.14+gitAUTOINC+e8d08fc4c0_b05ca3429c-r0 do_fetch -
12h26m11s (pid 4110422)  28% |#######                    | 2.00 KiB/s
openembedded-core/meta/recipes-kernel/linux/linux-yocto_6.1.bb
```
I tried many time but nothing change.

There is one mirror[1] I can use under my network condition.
So I want to know what is right way to alter URI in
openembedded-core/meta/recipes-kernel/linux/linux-yocto_6.1.bb?

I googled one article to suggest use Insteadof in .gitconfig[2], is it ok?

Many thanks for your help.

BR,
Bo

[0]: https://github.com/riscv/meta-riscv
[1]: https://isrc.iscas.ac.cn/gitlab/mirrors/git.yoctoproject.org/linux-yocto
[3]: https://www.wpgdadatong.com/cn/blog/detail/44614




Bo YU <tsu.yubo@...>
 

Hi,
On Sat, Mar 18, 2023 at 09:29:34AM +0000, Alexander Kanavin wrote:
When the slow download is happening, can you please check (e.g. from a
process listing), which server specifically is being accessed and with
which command? Just to be sure where the problem is.
Thanks for tips. I downloaded linux-yocto about 3 days but failed and
one night to download it after just sent the mail.

I suspected the accessing to git://git.yoctoproject.org/linux-yocto.git is very slow under my network. So I think there is one way to replace

```
SRC_URI =
"git://git.yoctoproject.org/linux-yocto.git;name=machine;branch=${KBRANCH};
\
git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=yocto-6.1;destsuffix=${KMETA}"
```

Or use another bb file to download linux-yocto kernel with mirror I
mentioned. But I am fail to try this.

Anway, Thank you very much!

--
Regards,
--
Bo YU


Martin Jansa
 

On Sun, Mar 19, 2023 at 3:50 PM Bo YU <tsu.yubo@...> wrote:
Hi,
On Sat, Mar 18, 2023 at 09:29:34AM +0000, Alexander Kanavin wrote:
>   When the slow download is happening, can you please check (e.g. from a
>   process listing), which server specifically is being accessed and with
>   which command? Just to be sure where the problem is.

Thanks for tips. I downloaded linux-yocto about 3 days but failed and
one night to download it after just sent the mail.

I suspected the accessing to git://git.yoctoproject.org/linux-yocto.git
is very slow under my network. So I think there is one way to replace

```
SRC_URI =
"git://git.yoctoproject.org/linux-yocto.git;name=machine;branch=${KBRANCH};
\
git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=yocto-6.1;destsuffix=${KMETA}"
```

Or use another bb file to download linux-yocto kernel with mirror I
mentioned. But I am fail to try this.

You can and should use PREMIRROR like with any other source fetched by bitbake fetcher.

Use BB_GENERATE_MIRROR_TARBALLS, then rsync your downloads directory (exclude .done, bad-checksum files and git2, svn, ... dirs) to some shared location on your network and you have a good starting PREMIRROR to use in all the builds, so you don't ever need to re-download everything again. Update your PREMIRROR with rsync after succesful build (every time or from time to time).

You can also share whole DL_DIR directly, but that has small risk, that if fetcher fails to update e.g. linux-yocto due to temporary network glitch, it will remove whole git clone from the DL_DIR and start again, while with PREMIRROR it will start from the tarball stored there. So it's useful to combine both (PREMIRROR updated after successful builds and shared DL_DIR if you have many builders with fast access to the same NFS/CIFS share).

Cheers,