Errors building with Windows Subsystem for Linux (aka Bash on Ubuntu on Windows)


Bryan Evenson
 

All,

Due to what our IT department can support, I am issued a Windows laptop for development. In the past I have used VMWare to make a Linux virtual machine for my Yocto Project based image builds and application development. We are starting to get Windows 10 laptops so I am evaluating doing my builds using the Windows Subsystem for Linux (WSL) by building a poky/morty image. Overall it seems to be working. I've had an issue that I've worked through and other issues that I'm not quite sure what is going on.

The first major issue was that PATH includes some Windows directory paths so that you can execute certain Windows applications, but that meant PATH included filepaths like ".../Program Files (x86) ..." and python didn't like the spaces and parentheses in PATH. The suggested fix for this issue is to set PATH in your ~/.bashrc file to either exclude the Windows directories or to escape the spaces and parentheses. This worked for me.

The second issue I'm seeing is I get the following warning:

WARNING: The Linux kernel on your build host was not configured to provide process I/O statistics. (CONFIG_TASK_IO_ACCOUNTING is not set)

I searched the Yocto Project documentation and I couldn't figure out what specifically these I/O statistics are used for. What affect does this have on my build? Does anybody know if there is a way to enable I/O statistics for the WSL?

I'm also seeing a very slow refresh of the build status on the command line. When multiple tasks are running, I can see the lines clear away from the bottom of the screen to the top and then fill in back down. The resulting flash from the refresh makes it nearly impossible to read what some of the running tasks are. What is the actual mechanism for updating the task status on the command line? I'm wondering if it's counting on some feature that Microsoft hasn't fully supported yet.

I am also getting a build error with glibc-locale. In the do_package stage I'm getting a failure in the function sstate_task_postfunc. The error is on the following command:

cp -afl --preserve=xattr <base_path>/poky/poky-build/tmp/work/armv5e-poky-linux-gnueabi/glibc-locale/2.24-r0/packages-split/* <base_path>/poky/poky-build/tmp/work/armv5e-poky-linux-gnueabi/glibc-locale/2.24-r0/sstate-build-package//packages-split/*

I am seeing a lot of errors from this command that look like the following:

cp: cannot create hard link: '<base_path>/poky/poky-build/tmp/work/armv5e-poky-linux-gnueabi/glibc-locale/2.24-r0/sstate-build-package//packages-split/glibc-binary-localedata-ks-in/usr/lib/locale/ks_IN/LC_MEASUREMENT' to '<base_path>/poky/poky-build/tmp/work/armv5e-poky-linux-gnueabi/glibc-locale/2.24-r0/packages-split/glibc-binary-localedata-ks-in/usr/lib/locale/ks_IN/LC_MEASUREMENT': Invalid argument

I looked at my working directory, and I see that in the source files are there. If I run the cp command from the command line as shown above, I get the same errors. I'm trying to figure out if this is an issue with hard links with WSL or if this is general build issue. Any suggestions on what else to try?

Thanks,
Bryan


Burton, Ross <ross.burton@...>
 

Hi,

On 26 September 2017 at 18:16, Bryan Evenson <bevenson@...> wrote:
  WARNING: The Linux kernel on your build host was not configured to provide process I/O statistics. (CONFIG_TASK_IO_ACCOUNTING is not set)

I searched the Yocto Project documentation and I couldn't figure out what specifically these I/O statistics are used for.  What affect does this have on my build?  Does anybody know if there is a way to enable I/O statistics for the WSL?

That's the buildstats class that is enabled in your local.conf.  I don't expect WSL will ever fake a Linux-compatible /proc directory structure, so you can just disable this.
 
I'm also seeing a very slow refresh of the build status on the command line.  When multiple tasks are running, I can see the lines clear away from the bottom of the screen to the top and then fill in back down.  The resulting flash from the refresh makes it nearly impossible to read what some of the running tasks are.  What is the actual mechanism for updating the task status on the command line?  I'm wondering if it's counting on some feature that Microsoft hasn't fully supported yet.

This is just standard curses, so you'll have to wait for Microsoft to make it faster.

If you do "bitbake <your targets> | cat" then bitbake will fall back to a UI that isn't interactive and will be a lot faster.
 
I am also getting a build error with glibc-locale.  In the do_package stage I'm getting a failure in the function sstate_task_postfunc.  The error is on the following command:

  cp -afl --preserve=xattr <base_path>/poky/poky-build/tmp/work/armv5e-poky-linux-gnueabi/glibc-locale/2.24-r0/packages-split/* <base_path>/poky/poky-build/tmp/work/armv5e-poky-linux-gnueabi/glibc-locale/2.24-r0/sstate-build-package//packages-split/*

I am seeing a lot of errors from this command that look like the following:

 cp: cannot create hard link: '<base_path>/poky/poky-build/tmp/work/armv5e-poky-linux-gnueabi/glibc-locale/2.24-r0/sstate-build-package//packages-split/glibc-binary-localedata-ks-in/usr/lib/locale/ks_IN/LC_MEASUREMENT' to '<base_path>/poky/poky-build/tmp/work/armv5e-poky-linux-gnueabi/glibc-locale/2.24-r0/packages-split/glibc-binary-localedata-ks-in/usr/lib/locale/ks_IN/LC_MEASUREMENT': Invalid argument

I looked at my working directory, and I see that in the source files are there.  If I run the cp command from the command line as shown above, I get the same errors.  I'm trying to figure out if this is an issue with hard links with WSL or if this is general build issue.  Any suggestions on what else to try?

Now this *is* interesting.  Try removing the repeated slashes just in case the WSL ln is being incredibly pedantic (ie sstate-build-package//packages-split), but I don't seriously expect that to be the problem.  Running stat on the source and verifying the destination doesn't exist would be helpful.  Can you tell if that is the first ln that it is trying to do, or do many work and that one fails?  Does WSL have a working strace or similar to identify which exact syscall is failing?

In all honestly I'm surprised you got the build to go as far as you have under WSL, that's impressive.  Have you been able to compare performance against a full VM on the same hardware?

Ross


Josef Holzmayr <holzmayr@...>
 

Hi Bryan,

On 26.09.2017 19:16, Bryan Evenson wrote:
Due to what our IT department can support, I am issued a Windows laptop for development. In the past I have used VMWare to make a Linux virtual machine for my Yocto Project based image builds and application development. We are starting to get Windows 10 laptops so I am evaluating doing my builds using the Windows Subsystem for Linux (WSL) by building a poky/morty image. Overall it seems to be working. I've had an issue that I've worked through and other issues that I'm not quite sure what is going on.
There's obviously a lot of stuff that the WSL does not support (yet) which is often filesystems or API limitations. I ran into a list of things that they officially marked as known good/bad, but I can't find it right now. If it shows up again, I'll forward it. For the time being, I think you might be better off looking at CROPS [1] which is the "offical" recommendation to buld on Windows/OSX, as far as I can see.

Greetz

[1] https://wiki.yoctoproject.org/wiki/TipsAndTricks/CropsCLIContainers

--
Josef Holzmayr
Software Developer Embedded Systems

Tel: +49 8444 9204-48
Fax: +49 8444 9204-50

R-S-I Elektrotechnik GmbH & Co. KG
Woelkestrasse 11
D-85301 Schweitenkirchen
www.rsi-elektrotechnik.de
———————————————
Amtsgericht Ingolstadt – GmbH: HRB 191328 – KG: HRA 170393
Geschäftsführer: Dr.-Ing. Michael Sorg, Dipl.-Ing. Franz Sorg
Ust-IdNr: DE 128592548

_____________________________________________________________
Amtsgericht Ingolstadt - GmbH: HRB 191328 - KG: HRA 170363
Geschäftsführer: Dr.-Ing. Michael Sorg, Dipl.-Ing. Franz Sorg
USt-IdNr.: DE 128592548


Bryan Evenson
 

Ross,

 

 

From: Burton, Ross [mailto:ross.burton@...]
Sent: Tuesday, September 26, 2017 3:13 PM
To: Bryan Evenson <bevenson@...>
Cc: yocto@...
Subject: Re: [yocto] Errors building with Windows Subsystem for Linux (aka Bash on Ubuntu on Windows)

 

Hi,

 

On 26 September 2017 at 18:16, Bryan Evenson <bevenson@...> wrote:

  WARNING: The Linux kernel on your build host was not configured to provide process I/O statistics. (CONFIG_TASK_IO_ACCOUNTING is not set)

I searched the Yocto Project documentation and I couldn't figure out what specifically these I/O statistics are used for.  What affect does this have on my build?  Does anybody know if there is a way to enable I/O statistics for the WSL?

 

That's the buildstats class that is enabled in your local.conf.  I don't expect WSL will ever fake a Linux-compatible /proc directory structure, so you can just disable this.

 

OK.  I removed that from my local.conf and the warning went away.

 

 

I'm also seeing a very slow refresh of the build status on the command line.  When multiple tasks are running, I can see the lines clear away from the bottom of the screen to the top and then fill in back down.  The resulting flash from the refresh makes it nearly impossible to read what some of the running tasks are.  What is the actual mechanism for updating the task status on the command line?  I'm wondering if it's counting on some feature that Microsoft hasn't fully supported yet.

 

This is just standard curses, so you'll have to wait for Microsoft to make it faster.

 

If you do "bitbake <your targets> | cat" then bitbake will fall back to a UI that isn't interactive and will be a lot faster.

 

After I disabled buildstats, the refresh noticeably improved.  From looking at buildstats.bbclass, I see a number of ‘stat’ calls.  I’m guessing the stat call has some efficiency improvements left to go on WSL.

 

I am also getting a build error with glibc-locale.  In the do_package stage I'm getting a failure in the function sstate_task_postfunc.  The error is on the following command:

  cp -afl --preserve=xattr <base_path>/poky/poky-build/tmp/work/armv5e-poky-linux-gnueabi/glibc-locale/2.24-r0/packages-split/* <base_path>/poky/poky-build/tmp/work/armv5e-poky-linux-gnueabi/glibc-locale/2.24-r0/sstate-build-package//packages-split/*

I am seeing a lot of errors from this command that look like the following:

 cp: cannot create hard link: '<base_path>/poky/poky-build/tmp/work/armv5e-poky-linux-gnueabi/glibc-locale/2.24-r0/sstate-build-package//packages-split/glibc-binary-localedata-ks-in/usr/lib/locale/ks_IN/LC_MEASUREMENT' to '<base_path>/poky/poky-build/tmp/work/armv5e-poky-linux-gnueabi/glibc-locale/2.24-r0/packages-split/glibc-binary-localedata-ks-in/usr/lib/locale/ks_IN/LC_MEASUREMENT': Invalid argument

I looked at my working directory, and I see that in the source files are there.  If I run the cp command from the command line as shown above, I get the same errors.  I'm trying to figure out if this is an issue with hard links with WSL or if this is general build issue.  Any suggestions on what else to try?

 

Now this *is* interesting.  Try removing the repeated slashes just in case the WSL ln is being incredibly pedantic (ie sstate-build-package//packages-split), but I don't seriously expect that to be the problem.  Running stat on the source and verifying the destination doesn't exist would be helpful.  Can you tell if that is the first ln that it is trying to do, or do many work and that one fails?  Does WSL have a working strace or similar to identify which exact syscall is failing?

 

I am about 60% through the full image build when it gets to glibc-locale with about half of the packages for the image fully complete.  I did a stat on one of the source files and verified it did exist, and it had 0644 for access rights and is owned by me.  I also verified that the destination file doesn’t exist.  WSL does have a working strace.  I ran strace on the failed cp command shown above and I now have a 56MB strace output file.  What should I be looking for in this file?

 

In all honestly I'm surprised you got the build to go as far as you have under WSL, that's impressive.  Have you been able to compare performance against a full VM on the same hardware?

 

I haven’t got far enough to do a good performance comparison.  Overall performance seems similar, maybe a little slower than against a full VM.

 

Ross


Burton, Ross <ross.burton@...>
 

On 26 September 2017 at 22:01, Bryan Evenson <bevenson@...> wrote:

Now this *is* interesting.  Try removing the repeated slashes just in case the WSL ln is being incredibly pedantic (ie sstate-build-package//packages-split), but I don't seriously expect that to be the problem.  Running stat on the source and verifying the destination doesn't exist would be helpful.  Can you tell if that is the first ln that it is trying to do, or do many work and that one fails?  Does WSL have a working strace or similar to identify which exact syscall is failing?

 

I am about 60% through the full image build when it gets to glibc-locale with about half of the packages for the image fully complete.  I did a stat on one of the source files and verified it did exist, and it had 0644 for access rights and is owned by me.  I also verified that the destination file doesn’t exist.  WSL does have a working strace.  I ran strace on the failed cp command shown above and I now have a 56MB strace output file.  What should I be looking for in this file?


Feel free to compress and email it directly to me, but simply grepping for EINVAL (invalid argument error) might be enough.

Ross


Bryan Evenson
 

Ross,

 

From: Burton, Ross [mailto:ross.burton@...]
Sent: Tuesday, September 26, 2017 6:15 PM
To: Bryan Evenson <bevenson@...>
Cc: yocto@...
Subject: Re: [yocto] Errors building with Windows Subsystem for Linux (aka Bash on Ubuntu on Windows)

 

On 26 September 2017 at 22:01, Bryan Evenson <bevenson@...> wrote:

Now this *is* interesting.  Try removing the repeated slashes just in case the WSL ln is being incredibly pedantic (ie sstate-build-package//packages-split), but I don't seriously expect that to be the problem.  Running stat on the source and verifying the destination doesn't exist would be helpful.  Can you tell if that is the first ln that it is trying to do, or do many work and that one fails?  Does WSL have a working strace or similar to identify which exact syscall is failing?

 

I am about 60% through the full image build when it gets to glibc-locale with about half of the packages for the image fully complete.  I did a stat on one of the source files and verified it did exist, and it had 0644 for access rights and is owned by me.  I also verified that the destination file doesn’t exist.  WSL does have a working strace.  I ran strace on the failed cp command shown above and I now have a 56MB strace output file.  What should I be looking for in this file?

 

Feel free to compress and email it directly to me, but simply grepping for EINVAL (invalid argument error) might be enough.

 

I did a search on EINVAL and I got a bunch of entries of that look like this:

 

  linkat(AT_FDCWD, “<source_full_path>”, AT_FDCWD, “<dest_full_path>”, 0) = -1 EINVAL (Invalid argument)

 

I checked a few entries by doing a stat on the source and destination.  In each case, the source file exists at the specified path, was owned by the user account with which I’m doing the build, and had access rights of 0644.  Also in each case, the destination file did not exist but the destination directory did exist, the destination directory was owned by the user account with which I’m doing the build, and the destination directory had access rights of 0755.  From reading the man page for linkat, linkat should only return EINVAL if the flags argument is invalid, and 0 should be a valid value for flags.

 

I think there may be something missing from WSL’s implementation of linkat.  I am doing OPKG packages, so I’m guessing there is something different about the resulting function calls in the libc-package class as opposed to the package_ipk class.  I think I may enable the other package types to see if it is as successfully with RPMs and DEB packages.  I’m also going to see if there is an easier way to reproduce the problem to report the issue to Microsoft.

 

Bryan


Burton, Ross <ross.burton@...>
 

On 27 September 2017 at 13:57, Bryan Evenson <bevenson@...> wrote:


Feel free to compress and email it directly to me, but simply grepping for EINVAL (invalid argument error) might be enough.

 

I did a search on EINVAL and I got a bunch of entries of that look like this:

 

  linkat(AT_FDCWD, “<source_full_path>”, AT_FDCWD, “<dest_full_path>”, 0) = -1 EINVAL (Invalid argument)

 

I checked a few entries by doing a stat on the source and destination.  In each case, the source file exists at the specified path, was owned by the user account with which I’m doing the build, and had access rights of 0644.  Also in each case, the destination file did not exist but the destination directory did exist, the destination directory was owned by the user account with which I’m doing the build, and the destination directory had access rights of 0755.  From reading the man page for linkat, linkat should only return EINVAL if the flags argument is invalid, and 0 should be a valid value for flags.

 

I think there may be something missing from WSL’s implementation of linkat.  I am doing OPKG packages, so I’m guessing there is something different about the resulting function calls in the libc-package class as opposed to the package_ipk class.  I think I may enable the other package types to see if it is as successfully with RPMs and DEB packages.  I’m also going to see if there is an easier way to reproduce the problem to report the issue to Microsoft.


Good digging.  At least you can replicate it on demand, and have a strace showing it.  This bit of packaging happens before the rpm/opkg/deb specific code, so changing the packaging format won't help.

Ross


Bryan Evenson
 

Ross,

 

 

From: Burton, Ross [mailto:ross.burton@...]
Sent: Wednesday, September 27, 2017 9:06 AM
To: Bryan Evenson <bevenson@...>
Cc: yocto@...
Subject: Re: [yocto] Errors building with Windows Subsystem for Linux (aka Bash on Ubuntu on Windows)

 

On 27 September 2017 at 13:57, Bryan Evenson <bevenson@...> wrote:


Feel free to compress and email it directly to me, but simply grepping for EINVAL (invalid argument error) might be enough.

 

I did a search on EINVAL and I got a bunch of entries of that look like this:

 

  linkat(AT_FDCWD, “<source_full_path>”, AT_FDCWD, “<dest_full_path>”, 0) = -1 EINVAL (Invalid argument)

 

I checked a few entries by doing a stat on the source and destination.  In each case, the source file exists at the specified path, was owned by the user account with which I’m doing the build, and had access rights of 0644.  Also in each case, the destination file did not exist but the destination directory did exist, the destination directory was owned by the user account with which I’m doing the build, and the destination directory had access rights of 0755.  From reading the man page for linkat, linkat should only return EINVAL if the flags argument is invalid, and 0 should be a valid value for flags.

 

I think there may be something missing from WSL’s implementation of linkat.  I am doing OPKG packages, so I’m guessing there is something different about the resulting function calls in the libc-package class as opposed to the package_ipk class.  I think I may enable the other package types to see if it is as successfully with RPMs and DEB packages.  I’m also going to see if there is an easier way to reproduce the problem to report the issue to Microsoft.

 

Good digging.  At least you can replicate it on demand, and have a strace showing it.  This bit of packaging happens before the rpm/opkg/deb specific code, so changing the packaging format won't help.

 

I think I found the problem.  I started looking at more file properties for the files that worked and the ones that didn’t, and I noticed that all the ones that failed show a link count of 1024.  The Windows filesystem has a link limit of 1023 links per file (at least as reported here: https://msdn.microsoft.com/en-us/library/windows/desktop/aa363860(v=vs.85).aspx), so I think the hard link is failing due to the Windows link limit.  If that is the case, then I don’t think it’ll be a quick fix to get a working solution under WSL.

 

Bryan


Burton, Ross <ross.burton@...>
 

On 27 September 2017 at 21:59, Bryan Evenson <bevenson@...> wrote:


I think I found the problem.  I started looking at more file properties for the files that worked and the ones that didn’t, and I noticed that all the ones that failed show a link count of 1024.  The Windows filesystem has a link limit of 1023 links per file (at least as reported here: https://msdn.microsoft.com/en-us/library/windows/desktop/aa363860(v=vs.85).aspx), so I think the hard link is failing due to the Windows link limit.  If that is the case, then I don’t think it’ll be a quick fix to get a working solution under WSL.


That link count doesn't seem feasible though...  we hardlink frequently during a recipe build, but I'd expect to see 10 links, not over a thousand.  You've definitely found the problem, just need to figure out what is causing such excessive linking,

Ross


Bryan Evenson
 

Ross,

 

From: Burton, Ross [mailto:ross.burton@...]
Sent: Wednesday, September 27, 2017 6:43 PM
To: Bryan Evenson <bevenson@...>
Cc: yocto@...
Subject: Re: [yocto] Errors building with Windows Subsystem for Linux (aka Bash on Ubuntu on Windows)

 

On 27 September 2017 at 21:59, Bryan Evenson <bevenson@...> wrote:

 

I think I found the problem.  I started looking at more file properties for the files that worked and the ones that didn’t, and I noticed that all the ones that failed show a link count of 1024.  The Windows filesystem has a link limit of 1023 links per file (at least as reported here: https://msdn.microsoft.com/en-us/library/windows/desktop/aa363860(v=vs.85).aspx), so I think the hard link is failing due to the Windows link limit.  If that is the case, then I don’t think it’ll be a quick fix to get a working solution under WSL.

 

That link count doesn't seem feasible though...  we hardlink frequently during a recipe build, but I'd expect to see 10 links, not over a thousand.  You've definitely found the problem, just need to figure out what is causing such excessive linking,

 

Two files, LC_MEASUREMENT and LC_PAPER, seem to be identical through most the locales.  I’m not sure which are copies and which are hard links, but I did a md5sum comparison and found over 1200 identical LC_MEASUREMENT files in the glibc-locale working directory.  I don’t need all the locales, so I set GLIBC_GENERATE_LOCALES = "en_GB.UTF-8 en_US.UTF-8" in my local.conf and now glibc-locale completes building.

 

I’ll report back if the rest of the build completes.

 

Bryan

 


Josef Holzmayr <holzmayr@...>
 

I don't have the slightest clue why this message was delayed so much.

As it might sound now like parodizing the ongoing efforts, please ignore/disregard.

Greetz
--
Josef Holzmayr
Software Developer Embedded Systems

Tel: +49 8444 9204-48
Fax: +49 8444 9204-50

R-S-I Elektrotechnik GmbH & Co. KG
Woelkestrasse 11
D-85301 Schweitenkirchen
www.rsi-elektrotechnik.de
———————————————
Amtsgericht Ingolstadt – GmbH: HRB 191328 – KG: HRA 170393
Geschäftsführer: Dr.-Ing. Michael Sorg, Dipl.-Ing. Franz Sorg
Ust-IdNr: DE 128592548

_____________________________________________________________
Amtsgericht Ingolstadt - GmbH: HRB 191328 - KG: HRA 170363
Geschäftsführer: Dr.-Ing. Michael Sorg, Dipl.-Ing. Franz Sorg
USt-IdNr.: DE 128592548


Dominig ar Foll (Intel Open Source) <dominig.arfoll@...>
 

Le 26/09/2017 à 19:16, Bryan Evenson a écrit :
All,

Due to what our IT department can support, I am issued a Windows laptop for development. In the past I have used VMWare to make a Linux virtual machine for my Yocto Project based image builds and application development. We are starting to get Windows 10 laptops so I am evaluating doing my builds using the Windows Subsystem for Linux (WSL) by building a poky/morty image. Overall it seems to be working. I've had an issue that I've worked through and other issues that I'm not quite sure what is going on.
Hi Brian,

I have been trying the same thing attempting to build Automotive Grade
Linux from Linux for Windows subsystem.
We have many adopter of AGL who also receive Windows PC from their IT
department. While VM work, they impose a serious limitation on memory
and CPU usage.
We all know that Linux For Windows a very green SW but, I did not
expected to have so much pain to upgrade to the current Windows10 build
(version 1703). Deactivating the Antivirus with the help of IT, was
painful and slow.

Now that I run 1703, it kind of give hope that it may work, but it does not.
I see random errors on variable expansion with an error (disk I/O
error). Looks like if the file system was not very happy.

My IT PC does not let me run Windows pre build, so I do not know if
progress is coming soon or not.

Likely will do a test when the next build of Win10 is coming to me.
In the mean time, I will stick to my Linux Machine which works fine.

But it would be nice to have it working for many corporation.

--
Dominig ar Foll
Senior Software Architect
Intel Open Source Technology Centre


Mark Hatle <mark.hatle@...>
 

On 9/28/17 8:28 AM, Bryan Evenson wrote:
Ross,

 

*From:*Burton, Ross [mailto:ross.burton@...]
*Sent:* Wednesday, September 27, 2017 6:43 PM
*To:* Bryan Evenson <bevenson@...>
*Cc:* yocto@...
*Subject:* Re: [yocto] Errors building with Windows Subsystem for Linux (aka
Bash on Ubuntu on Windows)

 

On 27 September 2017 at 21:59, Bryan Evenson <bevenson@...
<mailto:bevenson@...>> wrote:

 

I think I found the problem.  I started looking at more file properties for
the files that worked and the ones that didn’t, and I noticed that all the
ones that failed show a link count of 1024.  The Windows filesystem has a
link limit of 1023 links per file (at least as reported here:
https://msdn.microsoft.com/en-us/library/windows/desktop/aa363860(v=vs.85).aspx),
so I think the hard link is failing due to the Windows link limit.  If that
is the case, then I don’t think it’ll be a quick fix to get a working
solution under WSL.

 

That link count doesn't seem feasible though...  we hardlink frequently during a
recipe build, but I'd expect to see 10 links, not over a thousand.  You've
definitely found the problem, just need to figure out what is causing such
excessive linking,

 

Two files, LC_MEASUREMENT and LC_PAPER, seem to be identical through most the
locales.  I’m not sure which are copies and which are hard links, but I did a
md5sum comparison and found over 1200 identical LC_MEASUREMENT files in the
glibc-locale working directory.  I don’t need all the locales, so I set
GLIBC_GENERATE_LOCALES = "en_GB.UTF-8 en_US.UTF-8" in my local.conf and now
glibc-locale completes building.
FYI, this is expected. The locales have numerous copies of the same files for
the same locales. So to save space there is a step that runs and identifies the
duplicates and hard links them. (This helps a lot for the sstate-cache for
instance).

It likely wouldn't be difficult to determine the link count 'issue' and simply
link to a different file for subsequent links... but I'm no longer sure where
the linking occurs that does all of this.

--Mark


I’ll report back if the rest of the build completes.

 

Bryan

 



Alexander Kanavin <alexander.kanavin@...>
 

On 09/29/2017 05:01 PM, Dominig ar Foll (Intel Open Source) wrote:
Due to what our IT department can support, I am issued a Windows laptop for development. In the past I have used VMWare to make a Linux virtual machine for my Yocto Project based image builds and application development. We are starting to get Windows 10 laptops so I am evaluating doing my builds using the Windows Subsystem for Linux (WSL) by building a poky/morty image. Overall it seems to be working. I've had an issue that I've worked through and other issues that I'm not quite sure what is going on.
Hi Brian,
I have been trying the same thing attempting to build Automotive Grade
Linux from Linux for Windows subsystem.
We have many adopter of AGL who also receive Windows PC from their IT
department. While VM work, they impose a serious limitation on memory
and CPU usage.
Does anyone else think doing embedded linux development on windows machines is a ridiculous situation, and needs to be discussed with companies management? IT departments should either support product R&D work (because at the end of the day that is where the company makes money), or stay out of the way and let engineers self-manage their computers, including installing weird operating systems.


Alex


Alex Lennon <ajlennon@...>
 

On 01/10/2017 09:20, Alexander Kanavin wrote:
On 09/29/2017 05:01 PM, Dominig ar Foll (Intel Open Source) wrote:
Due to what our IT department can support, I am issued a Windows laptop for development.  In the past I have used VMWare to make a Linux virtual machine for my Yocto Project based image builds and application development. We are starting to get Windows 10 laptops so I am evaluating doing my builds using the Windows Subsystem for Linux (WSL) by building a poky/morty image.  Overall it seems to be working. I've had an issue that I've worked through and other issues that I'm not quite sure what is going on.
Hi Brian,

I have been trying the same thing attempting to build Automotive Grade
Linux from Linux for Windows subsystem.
We have many adopter of AGL who also receive Windows PC from their IT
department. While VM work, they impose a serious limitation on memory
and CPU usage.
Does anyone else think doing embedded linux development on windows machines is a ridiculous situation, and needs to be discussed with companies management? IT departments should either support product R&D work (because at the end of the day that is where the company makes money), or stay out of the way and let engineers self-manage their computers, including installing weird operating systems.
+1

I suspect that technical people saying technical things have very little impact on upstream bean-counters.

If one was to, say, quantify the amount of time and therefore money the entire team are putting into wrangling the Windows machines into doing a half baked job of building Linux versus the capital cost and (presumably negligible) cost of IT support, with a commentary on limiting any security impact, I suspect some spanking new Linux boxes would arrive quite quickly

Cheers,

Alex


Alex Lennon <ajlennon@...>
 

On 01/10/2017 09:30, Alex Lennon wrote:


On 01/10/2017 09:20, Alexander Kanavin wrote:
On 09/29/2017 05:01 PM, Dominig ar Foll (Intel Open Source) wrote:
Due to what our IT department can support, I am issued a Windows laptop for development.  In the past I have used VMWare to make a Linux virtual machine for my Yocto Project based image builds and application development. We are starting to get Windows 10 laptops so I am evaluating doing my builds using the Windows Subsystem for Linux (WSL) by building a poky/morty image.  Overall it seems to be working. I've had an issue that I've worked through and other issues that I'm not quite sure what is going on.
Hi Brian,

I have been trying the same thing attempting to build Automotive Grade
Linux from Linux for Windows subsystem.
We have many adopter of AGL who also receive Windows PC from their IT
department. While VM work, they impose a serious limitation on memory
and CPU usage.
Does anyone else think doing embedded linux development on windows machines is a ridiculous situation, and needs to be discussed with companies management? IT departments should either support product R&D work (because at the end of the day that is where the company makes money), or stay out of the way and let engineers self-manage their computers, including installing weird operating systems.
+1

I suspect that technical people saying technical things have very little impact on upstream bean-counters.

If one was to, say, quantify the amount of time and therefore money the entire team are putting into wrangling the Windows machines into doing a half baked job of building Linux versus the capital cost and (presumably negligible) cost of IT support, with a commentary on limiting any security impact, I suspect some spanking new Linux boxes would arrive quite quickly
That said I don't want to sound negative about the effort.

I've been following bits of the thread and technically what the guys are doing here is pretty cool stuff.

I do feel that ultimately we're still using virtualisation though, whether type I or type II, whether Docker with VirtualBox hidden away or with the Win10 subystem.

My feeling is the I/O throughput, and possibly the process forking issue, is going to ruin performance but that doesn't stop me wanting to give it a go :)

In fact for a long time I've wanted a scalable cloud server I can bring up, pull a Yocto docker image into, and do a build. Not cost effective at the minute but perhaps in the future. And I think there are great benefits to having that kind of build control management system in place.

Cheers,

Alex


Bryan Evenson
 

Alex,

-----Original Message-----
From: yocto-bounces@... [mailto:yocto-bounces@...] On Behalf Of Alexander Kanavin
Sent: Sunday, October 01, 2017 4:20 AM
To: Dominig ar Foll (Intel Open Source) <dominig.arfoll@...>; yocto@...
Subject: Re: [yocto] Errors building with Windows Subsystem for Linux (aka Bash on Ubuntu on Windows)

On 09/29/2017 05:01 PM, Dominig ar Foll (Intel Open Source) wrote:
Due to what our IT department can support, I am issued a Windows laptop for development. In the past I have used VMWare to make a Linux virtual machine for my Yocto Project based image builds and application development. We are starting to get Windows 10 laptops so I am evaluating doing my builds using the Windows Subsystem for Linux (WSL) by building a poky/morty image. Overall it seems to be working. I've had an issue that I've worked through and other issues that I'm not quite sure what is going on.
Hi Brian,

I have been trying the same thing attempting to build Automotive Grade
Linux from Linux for Windows subsystem.
We have many adopter of AGL who also receive Windows PC from their IT
department. While VM work, they impose a serious limitation on memory
and CPU usage.
Does anyone else think doing embedded linux development on windows machines is a ridiculous situation, and needs to be discussed with companies management? IT departments should either support product R&D work (because at the end of the day that is where the company makes money), or stay out of the way and let engineers self-manage their computers, including installing weird operating systems.
Yes, ideally I could just get a Linux laptop and do that for development. Unfortunately, that's not an option for me. Honestly, if WSL works well I wouldn't mind it as a working solution. I started trying this more out of curiosity if it would do anything. I was quite surprised how much did work with very little modification.

With the few changes I have already mentioned, I was able to build my complete image. I then started working on building my development image to create and SDK that has gdb and tools for the Eclipse plugin. That built all the way to rootfs, then failed with a blank error message. I then tried with my VM, and the development image failed in the same manner. I recently updated to a more recent Yocto branch, so I think I broke something with my development image build (unrelated to WSL) that I need to go back and fix.

The biggest issue I had with build times was with antivirus tools. I hadn't setup the Windows antivirus to ignore the WSL working directory, so it was scanning every new file created and modified. Once I told it to ignore the WSL working directory, the build times were comparable to under VMWare.

I don't know how far I'll take this, but I thought others would be interested in the results.

Bryan