Linux-on-Windows Tips

Different Linux “environments” on Windows

Simple Bash

As part of the GIT install, I have a Bash shell. This was my first usage of Linux on windows, using install to launch Jupyter, for example.

From this Bash shell, it gives the illusion to be a linux distribution:

aleja@CC-Labs-2 MINGW64 ~
$ ls /
bin/  dev/  git-bash.exe*  LICENSE.txt  proc/              tmp/          unins000.exe*  usr/
cmd/  etc/  git-cmd.exe*   mingw64/     ReleaseNotes.html  unins000.dat  unins000.msg
(base)

However, it’s just an illusion and many Linux commands are missing. For instance, more is not available:

aleja@CC-Labs-2 MINGW64 ~
$ which more
which: no more in (/c/Users/aleja/Documents/CodeImages/Technos/Anaconda3:/c/Users/aleja/Documents/CodeImages/Technos/
Anaconda3/Library/mingw-w64/bin:/c/Users/aleja/Documents/CodeImages/Technos/Anaconda3/Library/usr/bin:/c/Users/aleja/
Documents/CodeImages/Technos/Anaconda3/Library/bin:/c/Users/aleja/Documents/CodeImages/Technos/Anaconda3/Scripts:/c/U
sers/aleja/Documents/CodeImages/Technos/Anaconda3/bin:/c/Users/aleja/Documents/CodeImages/Technos/Anaconda3/Scripts/c
ondabin:/c/Users/aleja/bin:/mingw64/bin:/usr/local/bin:/usr/bin:/usr/bin:/mingw64/bin:/usr/bin:/c/Users/aleja/bin:/c/
Program Files (x86)/Common Files/Oracle/Java/javapath:/c/Progra~2/Common Files/Oracle/Java/javapath:/c/WINDOWS/system
32:/c/WINDOWS:/c/WINDOWS/System32/Wbem:/c/WINDOWS/System32/WindowsPowerShell/v1.0:/c/WINDOWS/System32/OpenSSH:/cmd:/u
sr/bin:/c/Users/aleja/Documents/CodeImages/Technos/graphviz-2.38/release/bin:/c/Progra~1/dotnet:/c/Progra~1/Microsoft
SQL Server/130/Tools/Binn:/c/Progra~1/Microsoft SQL Server/Client SDK/ODBC/170/Tools/Binn:/c/Progra~2/sbt/bin:/c/User
s/aleja/AppData/Local/Microsoft/WindowsApps:/c/Users/aleja/AppData/Local/Programs/Microsoft VS Code/bin:/c/Users/alej
a/Documents/CodeImages/Technos/Anaconda3:/c/Users/aleja/Documents/CodeImages/Technos/Anaconda3/Scripts:/c/Users/aleja
/Documents/CodeImages/Technos/Anaconda3/Library/bin:/c/Users/aleja/Documents/CodeImages/Technos/Anaconda3/Library/min
gw-w64/bin:/c/Users/aleja/AppData/Roaming/Python/Python37/Scripts:/c/Users/aleja/Documents/CodeImages/Technos/spark/s
park-2.4.4-bin-hadoop2.7/bin:/c/Progra~2/sbt/bin:/c/Progra~2/scala/bin:/c/Program Files/Multipass/bin:/usr/bin/vendor_
perl:/usr/bin/core_perl)

Actually, this “linux file system” is really a folder in windows, as in this Powershell snippet, showing that the “linux filesystem” of Bash is really just the Windows folder C:\Users\aleja\Documents\CodeImages\Technos\Git>:

(base) PS C:\Users\aleja\Documents\CodeImages\Technos\Git> ls


    Directory: C:\Users\aleja\Documents\CodeImages\Technos\Git


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d-----        4/15/2019     18:36                bin
d-----        4/15/2019     18:37                cmd
d-----        4/15/2019     18:37                dev
d-----        4/15/2019     18:37                etc
d-----        4/15/2019     18:36                mingw64
d-----        4/15/2019     18:37                tmp
d-----        4/15/2019     18:37                usr
-a----        2/26/2019     18:48         149784 git-bash.exe
-a----        2/26/2019     18:48         149272 git-cmd.exe
-a----        3/12/2018     17:58          18765 LICENSE.txt
-a----        2/26/2019     19:10         144911 ReleaseNotes.html
-a----        4/15/2019     18:37        1502888 unins000.dat
-a----        4/15/2019     18:34        1297048 unins000.exe
-a----        4/15/2019     18:37          22795 unins000.msg

Bottom line: this is not “real Linux”. I didn’t try, but don’t think that one could install “real software” like Web servers and such.

Windows Subsystem for Linux (WSL) v2

Then since 2019, Microsoft gradually released a new version of WSL which has a full Linux kernel. (though apparently the full WSL 2 won’t be publicly available until late May 2020).

The Ubuntu application from the Microsoft store give you a native Linux shell. In theory one can install any Linux app on it.

I have yet to try, but think that (for example) I should be able to install Nginx on it.

For now one notes that looking at the root folder it is clear that this is not the same filesystem of the “simple Bash” described above that comes with Git - and in fact the Unix user is not the same as the Windows user, unlike in the “simple Bash” case

alex@CC-Labs-2:~$ pwd
/home/alex
alex@CC-Labs-2:~$ ls /
bin  boot  dev  etc  home  init  lib  lib32  lib64  libx32  media  mnt  opt  proc  root  run  sbin  snap  srv  sys  tmp  usr  var

From the news articles this is accomplished with a light-weight virtualization, separate from the Hyper-v approach. In fact, if one runs Hyper-v and looks at the VMs that exist in Hyper-v, none of them correspond to this WSL v2 process.

Another nice thing is that VSCode comes with a Remote-WSL extension that in theory lets you see nany folder in WSL. So that would let you edit, say, use VS Code on Windows to edit configuration files for a web server running in WSL.

This will soon be available even in Windows Explorer: Explorer-Linux integration

For the history of how this gets better as Microsoft keeps investing in WSL, a good article to see is WSL in Wikepedia

To launch, in the Ubuntu app (the shell), just do:

code .

Issues in WSL with systemd, ufw, etc.

It seems some Linux functionality does not work on WSL. In particular:

  • Apparently ufw doesn’t really work, because it is a front end to iptables and WSL uses the Windows Firewall instead of iptables to set up rules.

  • And systemd, while installed as part of WSL, is not used to boot WSL. Normally, Linux uses systemd as its initialization program - it becomes the process with PID=1 and all other processes are children or descendents.

See for example how Linux boots

Thus, if we use “normal Linux” (such as a Hyper-v VM), we see that the first process is indeed systemd, even if it is called via the /sbin/init script that points to systemd:

ubuntu@ccl-vm-1:~$ ps -aux | head -n 2
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.2  0.9  77760  8844 ?        Ss   15:02   0:02 /sbin/init

ubuntu@ccl-vm-1:~$ stat /sbin/init
File: /sbin/init -> /lib/systemd/systemd
Size: 20         Blocks: 0          IO Block: 4096   symbolic link
Device: 801h/2049d Inode: 3854        Links: 1
Access: (0777/lrwxrwxrwx)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2020-05-14 17:07:03.838918369 -0700
Modify: 2020-02-06 07:00:49.000000000 -0800
Change: 2020-05-06 09:03:28.912801034 -0700
Birth: -

However, in the case of WSL, it does not use systemd to boot Linux (see here).

Indeed, if we repeat the above commands on a WSL shell, we get diferent results: WSL uses a different program /init that has nothing to do with systemd

alex@CC-Labs-2:/etc/nginx$ ps -aux | head -n 2
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0   8892   312 ?        Ssl  07:45   0:00 /init
alex@CC-Labs-2:/etc/nginx$
alex@CC-Labs-2:/etc/nginx$ stat /init
File: /init
Size: 591344          Blocks: 1160       IO Block: 4096   regular file
Device: 2h/2d   Inode: 3659174697312806  Links: 1
Access: (0755/-rwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 1969-12-31 16:00:00.000000000 -0800
Modify: 1969-12-31 16:00:00.000000000 -0800
Change: 1969-12-31 16:00:00.000000000 -0800
Birth: -

The impact of this is that commands like systemctl can’t be used to manage services, since systemctl depends on systemd.

For example, we can’t start nginx with commands like sudo systemctl start nginx. We would get an error:

alex@CC-Labs-2:/etc/nginx$ sudo systemctl start nginx
System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to connect to bus: Host is down

Instead, we should use commands like sudo service nginx start. The SystemV service command runs a script in /etc/init.d, and in my example nginx has been installed so it has such a script:

alex@CC-Labs-2:/etc/nginx$ ls /etc/init.d
apparmor          cryptdisks        iscsid             multipath-tools  plymouth-log    ssh                  x11-common
apport            cryptdisks-early  keyboard-setup.sh  nginx            procps          udev
atd               dbus              kmod               open-iscsi       rsync           ufw
console-setup.sh  hwclock.sh        lvm2               open-vm-tools    rsyslog         unattended-upgrades
cron              irqbalance        lvm2-lvmpolld      plymouth         screen-cleanup  uuidd

VMs with Hyper-v

The last way of having Linux on windows is old-fashioned virtualization with Hyper-v.

More cumbersome and resource-intensive, but one advantage it has is that one can create a cluster of VMs, whereas with WSL it appears you only have “1 unique node”, so to speak.

I found that to have a cluster it is convenient to use multipass from Canonical. It relies on Hyper-v so the VMs it creates are visible in Hyper-v, but you can manage their lifecycle more easily from shells outside Hyper-v and get more accurate information.

For example, whereas Hyper-v’s UI does not always show you the IP addresses (their UI rendering seem to come and go), with multipass you get it from a CLI:

(base) PS C:\Users\aleja> multipass list
Name                    State             IPv4             Image
primary                 Running           192.168.23.238   Ubuntu 18.04 LTS
ccl-vm-1                Running           192.168.23.231   Ubuntu 18.04 LTS

See my Ubuntu Tips for how I configured SSH on a multipass VM, and my Nginx Tips for how I then installed a Web server on it.