Building Yocto on M1 Mac


Abhijeet Tripathi
 

Hello yocto-devs,

I'm new to the yocto project and trying to set up a build environment on my M1 Mac which is arm64 architecture. 

I'm following the steps as mentioned in below link:

But it is always pulling up containers based on amd64 architecture. Are there any updated steps/link to set up a build environment specific to M1 Mac(arm64)?

Thanks,
Abhijeet
 


Anders Montonen
 

Hi,

On 13 Feb 2022, at 14:24, Abhijeet Tripathi <abhijeettripathi3003@...> wrote:

Hello yocto-devs,

I'm new to the yocto project and trying to set up a build environment on my M1 Mac which is arm64 architecture. 

I'm following the steps as mentioned in below link:

But it is always pulling up containers based on amd64 architecture. Are there any updated steps/link to set up a build environment specific to M1 Mac(arm64)?

Looking at Docker Hub, none of the published Crops containers are built for ARM64. You can try building the containers yourself, making sure to pick base distros that are available for ARM64.

Note that there are plenty of reports of Docker performing poorly on Macs, probably because of the file system implementation. You may be better off using a full VM via e.g. UTM.

-a


Abhijeet Tripathi
 


On Sun, Feb 13, 2022 at 7:09 PM Anders Montonen <Anders.Montonen@...> wrote:
Hi,

On 13 Feb 2022, at 14:24, Abhijeet Tripathi <abhijeettripathi3003@...> wrote:

Hello yocto-devs,

I'm new to the yocto project and trying to set up a build environment on my M1 Mac which is arm64 architecture. 

I'm following the steps as mentioned in below link:

But it is always pulling up containers based on amd64 architecture. Are there any updated steps/link to set up a build environment specific to M1 Mac(arm64)?

Looking at Docker Hub, none of the published Crops containers are built for ARM64. You can try building the containers yourself, making sure to pick base distros that are available for ARM64.

Note that there are plenty of reports of Docker performing poorly on Macs, probably because of the file system implementation. You may be better off using a full VM via e.g. UTM.
 
I initially started with UTM but the problem with virtual machines is that we can only use half the resources of the machine.
This makes the builds slower, so I was looking at the container way.

Can you point me to any documentation which I can use to build customized containers for yocto builds?

Thanks,
Abhijeet
 

-a




Alexander Kanavin
 

Even if you do manage to start a build, performance won't be great.
Yocto, by design, builds the entire stack from source, so a lot of CPU
cores is essential. Don't spend your time waiting, get an
appropriately specced Linux workstation, or an account on a Linux
server that your organization might have for the purpose.

Alex

On Sun, 13 Feb 2022 at 15:56, Abhijeet Tripathi
<abhijeettripathi3003@...> wrote:


On Sun, Feb 13, 2022 at 7:09 PM Anders Montonen <Anders.Montonen@...> wrote:

Hi,

On 13 Feb 2022, at 14:24, Abhijeet Tripathi <abhijeettripathi3003@...> wrote:

Hello yocto-devs,

I'm new to the yocto project and trying to set up a build environment on my M1 Mac which is arm64 architecture.

I'm following the steps as mentioned in below link:
https://github.com/crops/docker-win-mac-docs/wiki/Mac-Instructions

But it is always pulling up containers based on amd64 architecture. Are there any updated steps/link to set up a build environment specific to M1 Mac(arm64)?


Looking at Docker Hub, none of the published Crops containers are built for ARM64. You can try building the containers yourself, making sure to pick base distros that are available for ARM64.

Note that there are plenty of reports of Docker performing poorly on Macs, probably because of the file system implementation. You may be better off using a full VM via e.g. UTM.

I initially started with UTM but the problem with virtual machines is that we can only use half the resources of the machine.
This makes the builds slower, so I was looking at the container way.

Can you point me to any documentation which I can use to build customized containers for yocto builds?

Thanks,
Abhijeet



-a




Josef Holzmayr
 

Howdy!

On 13. Feb 2022, at 19:04, Alexander Kanavin <alex.kanavin@...> wrote:

Even if you do manage to start a build, performance won't be great.
Yocto, by design, builds the entire stack from source, so a lot of CPU
cores is essential. Don't spend your time waiting, get an
appropriately specced Linux workstation, or an account on a Linux
server that your organization might have for the purpose.

While that is perfectly true from the pure technical facts, I have to add that building in Parallels on a M1 is ok-ish. You have to crank up the VM way beyond the defaults, to something like 6-8core, 16+X GB RAM, but then it’s bearable for the occasional user. When you can’t get your hands on a proper environment, its certainly an option.

Greetz


Alex

On Sun, 13 Feb 2022 at 15:56, Abhijeet Tripathi
<abhijeettripathi3003@...> wrote:


On Sun, Feb 13, 2022 at 7:09 PM Anders Montonen <Anders.Montonen@...> wrote:

Hi,

On 13 Feb 2022, at 14:24, Abhijeet Tripathi <abhijeettripathi3003@...> wrote:

Hello yocto-devs,

I'm new to the yocto project and trying to set up a build environment on my M1 Mac which is arm64 architecture.

I'm following the steps as mentioned in below link:
https://github.com/crops/docker-win-mac-docs/wiki/Mac-Instructions

But it is always pulling up containers based on amd64 architecture. Are there any updated steps/link to set up a build environment specific to M1 Mac(arm64)?


Looking at Docker Hub, none of the published Crops containers are built for ARM64. You can try building the containers yourself, making sure to pick base distros that are available for ARM64.

Note that there are plenty of reports of Docker performing poorly on Macs, probably because of the file system implementation. You may be better off using a full VM via e.g. UTM.


I initially started with UTM but the problem with virtual machines is that we can only use half the resources of the machine.
This makes the builds slower, so I was looking at the container way.

Can you point me to any documentation which I can use to build customized containers for yocto builds?

Thanks,
Abhijeet



-a









Anders Montonen
 


On 13 Feb 2022, at 21:59, Josef Holzmayr <jester@...> wrote:

Howdy!

On 13. Feb 2022, at 19:04, Alexander Kanavin <alex.kanavin@...> wrote:

Even if you do manage to start a build, performance won't be great.
Yocto, by design, builds the entire stack from source, so a lot of CPU
cores is essential. Don't spend your time waiting, get an
appropriately specced Linux workstation, or an account on a Linux
server that your organization might have for the purpose.

While that is perfectly true from the pure technical facts, I have to add that building in Parallels on a M1 is ok-ish. You have to crank up the VM way beyond the defaults, to something like 6-8core, 16+X GB RAM, but then it’s bearable for the occasional user. When you can’t get your hands on a proper environment, its certainly an option.

Just to give some numbers, building a Dunfell core-image-minimal for the Beaglebone, with a pre-populated DL_DIR took 36 minutes in a Linux VM on a M1 Max Macbook Pro (8 cores, 16GB RAM allocated), and 21 minutes on my 5900X desktop (12/24 cores/threads, 64GB RAM).

-a


Robert Joslyn
 

On Feb 13, 2022, at 6:56 AM, Abhijeet Tripathi <abhijeettripathi3003@...> wrote:


On Sun, Feb 13, 2022 at 7:09 PM Anders Montonen <Anders.Montonen@...> wrote:
Hi,

On 13 Feb 2022, at 14:24, Abhijeet Tripathi <abhijeettripathi3003@...> wrote:

Hello yocto-devs,

I'm new to the yocto project and trying to set up a build environment on my M1 Mac which is arm64 architecture.

I'm following the steps as mentioned in below link:
https://github.com/crops/docker-win-mac-docs/wiki/Mac-Instructions

But it is always pulling up containers based on amd64 architecture. Are there any updated steps/link to set up a build environment specific to M1 Mac(arm64)?
Looking at Docker Hub, none of the published Crops containers are built for ARM64. You can try building the containers yourself, making sure to pick base distros that are available for ARM64.

Note that there are plenty of reports of Docker performing poorly on Macs, probably because of the file system implementation. You may be better off using a full VM via e.g. UTM.

I initially started with UTM but the problem with virtual machines is that we can only use half the resources of the machine.
This makes the builds slower, so I was looking at the container way.

Can you point me to any documentation which I can use to build customized containers for yocto builds?

Thanks,
Abhijeet
A Dockerfile like this works for me:

FROM ubuntu:20.04

ARG DEBIAN_FRONTEND=noninteractive
ENV LANG=en_US.UTF-8

RUN apt-get update \
&& apt-get -y install \
locales \
sudo \
vim-tiny \
&& sed -i '/en_US.UTF-8/s/^# //g' /etc/locale.gen \
&& locale-gen \
&& apt-get -y install \
binutils \
build-essential \
chrpath \
cpio \
diffstat \
gawk \
git \
lz4 \
python3 \
python3-distutils \
wget \
zstd \
&& rm -rf /var/lib/apt/lists/*

RUN useradd -m -G sudo --uid=1000 -s /bin/bash yocto
RUN install -d -o yocto -g yocto /yocto
USER yocto

Save that to a file called “Dockerfile”. You can build a container called “yocto” with:
docker build -t yocto .

You can run this with something like:
docker run —rm —mount type=volume,src=yocto,dst=/yocto -it yocto

Once in the container, go to /yocto to do work within a Docker volume. I have an M1 MacBook Air and can use this container to run builds. As others have said, it’s not fast, but it does work if it’s what you have. Normally I don’t do builds on the MacBook, I usually ssh into my Linux desktop and do my work there.

You got me curious though, so I did a quick comparison of building on my MacBook Air and my desktop. Using the container built with that Dockerfile, I ran this sequence (the download is separate to avoid download time, which is highly variable):

git clone https://git.yoctoproject.org/poky.git -b honister
cd poky
. oe-init-build-env
bitbake core-image-minimal —runonly=fetch
rm -rf tmp/ sstate-cache/
time bitbake core-image-minimal

On my 2020 M1 MacBook Air (8 cores, 16 GB RAM, docker using 8 cores and 8 GB), the build took 84 minutes. My desktop with an AMD Ryzen 9 3950X (16 cores, 64 GB RAM) it takes 21 minutes.

Docker performance has always been bad for me on MacOS, so it wouldn’t surprise me if it’s faster to use a normal VM than docker (which uses a VM internally too).

Robert


Ross Burton <ross@...>
 

On Mon, 14 Feb 2022 at 00:17, Robert Joslyn
<robert.joslyn@...> wrote:
On my 2020 M1 MacBook Air (8 cores, 16 GB RAM, docker using 8 cores and 8 GB), the build took 84 minutes. My desktop with an AMD Ryzen 9 3950X (16 cores, 64 GB RAM) it takes 21 minutes.
Try doing the MBA build with the laptop suspended in the air on a
couple of pencils. As the MBA has no fan you might find that
increasing airflow around the laptop makes a big difference. And
giving the container more RAM/cores will help, obviously, there's no
reason to limit it to half the power.

Ross


Abhijeet Tripathi
 



On Mon, Feb 14, 2022 at 5:32 PM Ross Burton <ross@...> wrote:
On Mon, 14 Feb 2022 at 00:17, Robert Joslyn
<robert.joslyn@...> wrote:
> On my 2020 M1 MacBook Air (8 cores, 16 GB RAM, docker using 8 cores and 8 GB), the build took 84 minutes. My desktop with an AMD Ryzen 9 3950X (16 cores, 64 GB RAM) it takes 21 minutes.

Try doing the MBA build with the laptop suspended in the air on a
couple of pencils.  As the MBA has no fan you might find that
increasing airflow around the laptop makes a big difference.  And
giving the container more RAM/cores will help, obviously, there's no
reason to limit it to half the power.

Ross

Thanks for the pointers, let me try it out on my Mac.
Will try to use max cpu cores and RAM to boost the performance a bit.

Regards,
Abhijeet


Abhijeet Tripathi
 



On Mon, Feb 14, 2022 at 1:29 AM Josef Holzmayr <jester@...> wrote:
Howdy!

On 13. Feb 2022, at 19:04, Alexander Kanavin <alex.kanavin@...> wrote:

Even if you do manage to start a build, performance won't be great.
Yocto, by design, builds the entire stack from source, so a lot of CPU
cores is essential. Don't spend your time waiting, get an
appropriately specced Linux workstation, or an account on a Linux
server that your organization might have for the purpose.

While that is perfectly true from the pure technical facts, I have to add that building in Parallels on a M1 is ok-ish. You have to crank up the VM way beyond the defaults, to something like 6-8core, 16+X GB RAM, but then it’s bearable for the occasional user. When you can’t get your hands on a proper environment, its certainly an option.

Greetz
I agree that a powerful Linux workstation will really help but right nowI do not have access to any of it.
I Will try to check if I can get one, but in the meantime I would like to start on my Mac so that I can continue my learning.

Thanks,
Abhijeet


Alex

On Sun, 13 Feb 2022 at 15:56, Abhijeet Tripathi
<abhijeettripathi3003@...> wrote:


On Sun, Feb 13, 2022 at 7:09 PM Anders Montonen <Anders.Montonen@...> wrote:

Hi,

On 13 Feb 2022, at 14:24, Abhijeet Tripathi <abhijeettripathi3003@...> wrote:

Hello yocto-devs,

I'm new to the yocto project and trying to set up a build environment on my M1 Mac which is arm64 architecture.

I'm following the steps as mentioned in below link:
https://github.com/crops/docker-win-mac-docs/wiki/Mac-Instructions

But it is always pulling up containers based on amd64 architecture. Are there any updated steps/link to set up a build environment specific to M1 Mac(arm64)?


Looking at Docker Hub, none of the published Crops containers are built for ARM64. You can try building the containers yourself, making sure to pick base distros that are available for ARM64.

Note that there are plenty of reports of Docker performing poorly on Macs, probably because of the file system implementation. You may be better off using a full VM via e.g. UTM.


I initially started with UTM but the problem with virtual machines is that we can only use half the resources of the machine.
This makes the builds slower, so I was looking at the container way.

Can you point me to any documentation which I can use to build customized containers for yocto builds?

Thanks,
Abhijeet



-a









Robert Joslyn
 

On Feb 14, 2022, at 4:02 AM, Ross Burton <ross@...> wrote:

On Mon, 14 Feb 2022 at 00:17, Robert Joslyn
<robert.joslyn@...> wrote:
On my 2020 M1 MacBook Air (8 cores, 16 GB RAM, docker using 8 cores and 8 GB), the build took 84 minutes. My desktop with an AMD Ryzen 9 3950X (16 cores, 64 GB RAM) it takes 21 minutes.
Try doing the MBA build with the laptop suspended in the air on a
couple of pencils. As the MBA has no fan you might find that
increasing airflow around the laptop makes a big difference. And
giving the container more RAM/cores will help, obviously, there's no
reason to limit it to half the power.

Ross
Giving docker all 16 GB of memory made the build take 124 minutes. I assume this is due to the VM docker uses fighting for memory with the build and the host, slowing things down. I’m sure someone could find an optimal amount of memory to give the docker VM for best times.

I did these builds with the MBA on its side, which should be a little better than it just being flat on a desk (or sitting on pencils). A fan would also help, but lack of a fan is one of my favorite features! This was also done at room temperature, 68 F/20 C, I’m sure it’d be a bit faster if I went outside where it’s below freezing, but I’ll let someone else run that test :-D

The MBA is obviously not intended for this kind of workload, but it does at least work.

Robert