Networking Case Study: CCL

Real and Virtualized Networks in a Laptop

The following diagram shows the networking for laptop CC-Labs-2 when running Hyper-v with two VMs.

Key observations:

  • CC-Labs-2 creates an internal virtualized network

  • VMs on that virtual network can access other physical computers on the home network

  • However, the reverse is not true: those VMs are not visible.

This happens because we used Hyper-v with the Default Virtual Switch, which is an internal switch (see private vs internal vs public switches in Hyper-v).

Being an internal switch (as opposed to private) means that the CC-Labs-2’s operating system can see the virtual network, so can route from the virtual network to the home network.

But since the virtual swith is not public, there is no ingress from other machines in the home network to CC-Lab-2’s virtual network.

../_images/networks_in_a_laptop.svg

Where did MAC addresses come from?

MAC addresses are allocated to manufacturers by the IEEE, and you can see which manufacturer gets what MAC address at this finder.

In theory, manufacturers burn a hard-coded MAC address into each NIC and device, but as this article, explains, in real life some deviance can happen:

  • A manufacturer may “cheat” and use the same MAC address on two devices, which means they now can’t communicate.

  • The operating system can be “spoofed” into thinking the MAC address is not what it really is.

The Hyper-v Virtual Network

The examples in this section refer to the scenario and diagram from lab note Real and Virtualized Networks in a Laptop.

Hyper-v Powershell commdands

A lot of insight can be derived by inspecting the Hyper-v network with Powershell commands

Virtual Network adapters

In our scenario, we can inspect the Hyper-v network’s virtual network adapters thus:

(base) PS C:\WINDOWS\system32> Get-VMNetworkAdapter -All

Name                   IsManagementOs VMName             SwitchName     MacAddress   Status IPAddresses
----                   -------------- ------             ----------     ----------   ------ -----------
Container NIC 2cf575f1 True                              Default Switch 00155D35AF70 {Ok}
Network Adapter        False          ccl-vm-1           Default Switch 00155D00E501 {Ok}   {}
Network Adapter        False          primary            Default Switch 00155D00E502 {Ok}   {}

Notice that the first row corresponds to the virtual switch with which the hosting laptop’s O/S participates as a node in the virtual network. That participation is possible since we are using the Default Switch, which is internal as opposed to private (see private vs internal vs public switches in Hyper-v).

Virtual switch

The Hyper-v’s network switch information can be thus obtained:

(base) PS C:\WINDOWS\system32> Get-VMSwitch

Name           SwitchType NetAdapterInterfaceDescription
----           ---------- ------------------------------
Default Switch Internal

Access Control Lists & Security

Restrictions can be placed on what kind of traffic can flow in or out of VMs.

In our scenario, by default no restritions are placed:

(base) PS C:\WINDOWS\system32> Get-VMNetworkAdapterAcl -VMName ccl-vm-1
(base) PS C:\WINDOWS\system32>

More generally, we can see what security settings exist for a VM:

(base) PS C:\WINDOWS\system32> Get-VMSecurity -VMName ccl-vm-1


TpmEnabled                        : False
KsdEnabled                        : False
Shielded                          : False
EncryptStateAndVmMigrationTraffic : False
VirtualizationBasedSecurityOptOut : False
BindToHostTpm                     : False
CimSession                        : CimSession: .
ComputerName                      : CC-LABS-2
IsDeleted                         : False

Virtual Switch Extensions

The behavior of Hyper-v’s virtual switch can be enriched or modified by means of switch extensions.

Use cases can include telemetry, traffic control, and security.

In our scenario, by default we get a couple of drivers that developers can build extensions against:

  • Network Default Interface Specification, or NDIS

  • Windows Filtering Platform, or WFP

  • An Azure-related driver

(base) PS C:\WINDOWS\system32> Get-VMSwitch "Default Switch" | Get-VMSwitchExtension


Id                  : E7C3B2F0-F3C5-48DF-AF2B-10FED6D72E7A
Name                : Microsoft Windows Filtering Platform
Vendor              : Microsoft
Version             : 10.0.18362.207
ExtensionType       : Filter
ParentExtensionId   :
ParentExtensionName :
SwitchId            : c08cb7b8-9b3c-408e-8e30-5e16a3aeb444
SwitchName          : Default Switch
Enabled             : False
Running             : True
CimSession          : CimSession: .
ComputerName        : CC-LABS-2
IsDeleted           : False

Id                  : E9B59CFA-2BE1-4B21-828F-B6FBDBDDC017
Name                : Microsoft Azure VFP Switch Extension
Vendor              : Microsoft
Version             : 10.0.10011.16384
ExtensionType       : Forwarding
ParentExtensionId   :
ParentExtensionName :
SwitchId            : c08cb7b8-9b3c-408e-8e30-5e16a3aeb444
SwitchName          : Default Switch
Enabled             : False
Running             : True
CimSession          : CimSession: .
ComputerName        : CC-LABS-2
IsDeleted           : False

Id                  : EA24CD6C-D17A-4348-9190-09F0D5BE83DD
Name                : Microsoft NDIS Capture
Vendor              : Microsoft
Version             : 10.0.18362.1
ExtensionType       : Monitoring
ParentExtensionId   :
ParentExtensionName :
SwitchId            : c08cb7b8-9b3c-408e-8e30-5e16a3aeb444
SwitchName          : Default Switch
Enabled             : False
Running             : True
CimSession          : CimSession: .
ComputerName        : CC-LABS-2
IsDeleted           : False

Reserved IP addresses

The various ip addresses in the lab note Real and Virtualized Networks in a Laptop were assigned by different services: some were assigned by the router, some by Hyper-v, and some by the operating system of the laptop in question.

In doing so, these services honored IANA’s policy on reserved IP addresses which sets guidelines for such assignment.

The special 10.x, 172.14.x, 127.x addresses

In particular:

  • The home network uses 10.0.0.0/24 (256 addresses) leveraging the industry’s allotment of 10.0.0.0/8 (16,777,216 addresses) for private network use. The router configuration made this choice. It could have chosen a bigger range or could also have chosen other reserved private ip prefixes, like 192.0.0.0/24 or 172.16.0.0/12.

  • They Hyper-v network uses 172.14.214/28 (16 addresses) leveraging the industry’s allotment of 172.16.0.0/12 (1,048,576 addresses of type 172.[1-15].any.any) for a private newtork.

  • Loopback uses 127.0.0.1/8, leveraging the industry’s allotment of 16,777,216 ip addresses for localhost use. People sometimes complain about the “waste” of so many ip addresses where apparently a single one (127.0.0.1) would suffice. I found two responses on why:

    • Some say it is just a clueless bureacratic decision from IANA, to no good end, like the one on 224.0.0.0/4 and 240.0.0.0/4 (268 million addresses each).

    • Others say it can be used in testing a distributed system on a single machine. For instance, an application with multiple webservers / microservices all listening on port 80. You give each a different 127.x.y.z ip address and you can then simulate the entire distributed system on your single machine (if it has the required horsepower).

We see the occurrence of these reserved ip addresses in the result from a netstat -r command to get the routing table:

alex@CC-Labs-2:~$ netstat -r
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
255.255.255.255 0.0.0.0         255.255.255.255 U         0 0          0 eth1
224.0.0.0       0.0.0.0         240.0.0.0       U         0 0          0 eth1
169.254.255.255 0.0.0.0         255.255.255.255 U         0 0          0 eth1
link-local      0.0.0.0         255.255.0.0     U         0 0          0 eth1
169.254.147.180 0.0.0.0         255.255.255.255 U         0 0          0 eth1
224.0.0.0       0.0.0.0         240.0.0.0       U         0 0          0 eth2
255.255.255.255 0.0.0.0         255.255.255.255 U         0 0          0 eth2
172.17.214.175  0.0.0.0         255.255.255.255 U         0 0          0 eth2
172.17.214.160  0.0.0.0         255.255.255.240 U         0 0          0 eth2
172.17.214.161  0.0.0.0         255.255.255.255 U         0 0          0 eth2
127.0.0.0       0.0.0.0         255.0.0.0       U         0 0          0 lo
127.0.0.1       0.0.0.0         255.255.255.255 U         0 0          0 lo
127.255.255.255 0.0.0.0         255.255.255.255 U         0 0          0 lo
224.0.0.0       0.0.0.0         240.0.0.0       U         0 0          0 lo
255.255.255.255 0.0.0.0         255.255.255.255 U         0 0          0 lo
0.0.0.0         10.0.0.1        255.255.255.255 U         0 0          0 wifi0
10.0.0.0        0.0.0.0         255.255.255.0   U         0 0          0 wifi0
10.0.0.229      0.0.0.0         255.255.255.255 U         0 0          0 wifi0
10.0.0.255      0.0.0.0         255.255.255.255 U         0 0          0 wifi0
224.0.0.0       0.0.0.0         240.0.0.0       U         0 0          0 wifi0
255.255.255.255 0.0.0.0         255.255.255.255 U         0 0          0 wifi0

The “unassigned” 169.254.x.y ip addresses

In Linux, doing an ip address reveals several ip addresses in 169.254.0.0\16. These are reserved for link-local addresses, which means that they are not assigned at the network level (either manually or via DCHP) but at the link level.

Basically, when a device connects to a link, it uses the Address Resolution Protocol (ARP) to discover what other devices are on the same link, and in particular which 169.254.x.y addresses are already taken by “unassigned” devices.

It then sets its own 169.254.x.y ip address to its network cards, so as to be different from those of other devices on the link.

Presumably after that link connection is established, services like DCHP will assign a real network ip address whose validity goes beyond that of the local link.

Thus, normally after a device has been running more than a few seconds, say, any NIC that is supposed to be on the network should not have ip addresses of 169.254.x.y.

If any NIC has such an ip address, that may indicate a possible error in DHCP or manual IP assignment, or perhaps the media is not connected, or some other special purpose (such as the NPCAP lookup adapter, which can sniff local traffic and can be chosen in tools like Wireshark).

As an example, here are extracts from ip address for two different computers on the same Wifi link. Notice that the NICs that have a 169.254.x.y ip are marked as not running (except for one), as expected, and are a disjoint set of 169.254.x.y set across both computers, thanks to ARP. The only one running is the NPCAP lookup adapter, as shown in the subsequent Powershell ipconfig extract.

For contrast we included in the extract the network Wifi entry, which has a real ip address for a home network 10.0.0.0/24.

For computer CC-Labs-2:

alex@CC-Labs-2:~$ ip address
    # Extract of actual output
8: eth0: <> mtu 1500 group default qlen 1
    link/ether f0:6e:0b:d2:aa:e9
    inet 169.254.18.111/16 brd 169.254.255.255 scope global dynamic
    valid_lft forever preferred_lft forever
    inet6 fe80::45fe:53cd:8c92:126f/64 scope link dynamic
    valid_lft forever preferred_lft forever
15: eth1: <BROADCAST,MULTICAST,UP> mtu 1500 group default qlen 1
    link/ether 02:00:4c:4f:4f:50
    inet 169.254.147.180/16 brd 169.254.255.255 scope global dynamic
    valid_lft forever preferred_lft forever
    inet6 fe80::c16a:f58d:a7:93b4/64 scope link dynamic
    valid_lft forever preferred_lft forever
22: wifi0: <BROADCAST,MULTICAST,UP> mtu 1500 group default qlen 1
    link/ieee802.11 f0:6e:0b:d2:aa:e8
    inet 10.0.0.229/24 brd 10.0.0.255 scope global dynamic
    valid_lft 454341sec preferred_lft 454341sec
    inet6 2601:647:4c80:4000:d537:8eb2:807f:ba3a/64 scope global dynamic
    valid_lft 197772sec preferred_lft 197772sec
    inet6 2601:647:4c80:4000::4346/128 scope global dynamic
    valid_lft 250001sec preferred_lft 250001sec
    inet6 2601:647:4c80:4000:a81e:a0b8:9196:768/128 scope global dynamic
    valid_lft 197772sec preferred_lft 197772sec
    inet6 fe80::d537:8eb2:807f:ba3a/64 scope link dynamic
    valid_lft forever preferred_lft forever
2: wifi1: <> mtu 1500 group default qlen 1
    link/ieee802.11 f2:6e:0b:d2:ab:e9
    inet 169.254.249.99/16 brd 169.254.255.255 scope global dynamic
    valid_lft forever preferred_lft forever
    inet6 fe80::cd6c:dd90:bd7e:f963/64 scope link dynamic
    valid_lft forever preferred_lft forever
7: wifi2: <> mtu 1500 group default qlen 1
    link/ieee802.11 f2:6e:0b:d2:ae:e9
    inet 169.254.239.154/16 brd 169.254.255.255 scope global dynamic
    valid_lft forever preferred_lft forever
    inet6 fe80::2074:c93:f4d7:ef9a/64 scope link dynamic
    valid_lft forever preferred_lft forever

To see that the running 169.254.147.180 is actually the PCAP loopback adapter, here is an extract from Powershell (Linux did not seem to have this information, hence Powershell):

(base) PS C:\Users\aleja> ipconfig
    # Extract of actual output
Ethernet adapter Npcap Loopback Adapter:

Connection-specific DNS Suffix  . :
Link-local IPv6 Address . . . . . : fe80::c16a:f58d:a7:93b4%15
Autoconfiguration IPv4 Address. . : 169.254.147.180
Subnet Mask . . . . . . . . . . . : 255.255.0.0
Default Gateway . . . . . . . . . :

Conract the above withe the ip addresses for computer NYCLNY6X2VBH2, which are disjoint from those of CC-Labs-2:

alex@NYCLNY6X2VBH2:~$ ip address
    # Extract of actual output
11: eth0: <> mtu 1500 group default qlen 1
    link/ether 88:b1:11:7c:44:d9
    inet 169.254.248.241/16 brd 169.254.255.255 scope global dynamic
    valid_lft forever preferred_lft forever
    inet6 fe80::69a6:fe:a60c:f8f1/64 scope link dynamic
    valid_lft forever preferred_lft forever
4: eth2: <BROADCAST,MULTICAST,UP> mtu 1500 group default qlen 1
    link/ether 88:b1:11:7c:44:d5
    inet 10.0.0.63/24 brd 10.0.0.255 scope global dynamic
    valid_lft 602736sec preferred_lft 602736sec
    inet6 2601:647:4c80:4000::37f2/128 scope global dynamic
    valid_lft 408015sec preferred_lft 408015sec
    inet6 fe80::3d65:a8ec:73ce:a8a1/64 scope link dynamic
    valid_lft forever preferred_lft forever
9: wifi0: <> mtu 1500 group default qlen 1
    link/ieee802.11 8a:b1:11:7c:44:d5
    inet 169.254.132.196/16 brd 169.254.255.255 scope global dynamic
    valid_lft forever preferred_lft forever
    inet6 fe80::b0e6:4326:797c:84c4/64 scope link dynamic
    valid_lft forever preferred_lft forever

The 255 broadcast addresses

In every network, a connected device responds to two ip addresses:

  • The specific internet address (inet) that uniquely identifies the device in the network. Usually assigned by services like DHCP

  • A shared broadcast ip address that all devices use to receive messages intended to be read by every device in the network. It is usually the “highest ip number” of the valid ip addresses for the network.

For example, in the routing table below we see that each NIC has at least one such broadcast destination, usually ending in x.255 since that is the highest ip address, though not always. For the sub network 192.168.41.96/28, which only allows 16 addresses, that broadcast address would be 192.168.41.111, as shown in the routing table.

(Note: this routing table was asked after a laptop reboot, which is why the Hyper-v ips don’t match to those in the diagram in Real and Virtualized Networks in a Laptop: the diagram uses 172.254.214.0/28, whereas the table below used was taken when Hyper-v re-booted and re-assigned ips to be 192.168.41.96/28).

alex@CC-Labs-2:~$ netstat -r
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
link-local      0.0.0.0         255.255.0.0     U         0 0          0 eth1
169.254.147.180 0.0.0.0         255.255.255.255 U         0 0          0 eth1
169.254.255.255 0.0.0.0         255.255.255.255 U         0 0          0 eth1
224.0.0.0       0.0.0.0         240.0.0.0       U         0 0          0 eth1
255.255.255.255 0.0.0.0         255.255.255.255 U         0 0          0 eth1
192.168.41.96   0.0.0.0         255.255.255.240 U         0 0          0 eth2
192.168.41.97   0.0.0.0         255.255.255.255 U         0 0          0 eth2
192.168.41.111  0.0.0.0         255.255.255.255 U         0 0          0 eth2
224.0.0.0       0.0.0.0         240.0.0.0       U         0 0          0 eth2
255.255.255.255 0.0.0.0         255.255.255.255 U         0 0          0 eth2
127.0.0.0       0.0.0.0         255.0.0.0       U         0 0          0 lo
127.0.0.1       0.0.0.0         255.255.255.255 U         0 0          0 lo
127.255.255.255 0.0.0.0         255.255.255.255 U         0 0          0 lo
224.0.0.0       0.0.0.0         240.0.0.0       U         0 0          0 lo
255.255.255.255 0.0.0.0         255.255.255.255 U         0 0          0 lo
0.0.0.0         10.0.0.1        255.255.255.255 U         0 0          0 wifi0
10.0.0.0        0.0.0.0         255.255.255.0   U         0 0          0 wifi0
10.0.0.229      0.0.0.0         255.255.255.255 U         0 0          0 wifi0
10.0.0.255      0.0.0.0         255.255.255.255 U         0 0          0 wifi0
224.0.0.0       0.0.0.0         240.0.0.0       U         0 0          0 wifi0
255.255.255.255 0.0.0.0         255.255.255.255 U         0 0          0 wifi0

The multicasting 224.0.0.0/4 ip addresses

The routing table above also shows that each NIC can participate in the special network 224.0.0.0/4.

IANA reserved 224.0.0.0/4 (268 million addresses) for “multicast”.

Unlike the broadcast case earlier (that involved only 1 ip address per sub network), with multicast one has to determine the group of devices interested in receiving the information. As explaned in How IP Multicasting Works, 28 bits are used for a multicast group, which means there can be 268 million of them across a given network.

In out local network, arp indicates that there are 3 separate devices with multicast ip addresses, which are part of industry-reserved multicast addresses, though it isn’t clear how widely used they are, though some (like LLMNR) are said to be used by Windows to find what other computers are on the same link:

  • 224.0.0.22 on MAC 01-00-5e-00-00-16.
    • This is used by the Internet Group Managment Protocol (IGMP)

    • It is said to be relevant for streaming, as in games and video.

    • It is a protocol used in IPTV (IP-based television), which apparently is growing in e.g. Asia.

  • 224.0.0.251 on MAC 01-00-5e-00-00-fb
    • This is used for multicast DNS.

    • It is used for “zero configuration” discovery, in small networks without a DNS server. For example, by Apple’s Bonjour software.

  • 224.0.0.252 on MAC 01-00-5e-00-00-fc
(base) PS C:\WINDOWS\system32> arp -a

Interface: 169.254.147.180 --- 0xe
Internet Address      Physical Address      Type
169.254.255.255       ff-ff-ff-ff-ff-ff     static
224.0.0.22            01-00-5e-00-00-16     static
224.0.0.251           01-00-5e-00-00-fb     static
224.0.0.252           01-00-5e-00-00-fc     static
239.255.255.250       01-00-5e-7f-ff-fa     static
239.255.255.253       01-00-5e-7f-ff-fd     static
255.255.255.255       ff-ff-ff-ff-ff-ff     static

Interface: 10.0.0.229 --- 0x16
Internet Address      Physical Address      Type
10.0.0.1              48-f7-c0-ac-ee-46     dynamic
10.0.0.63             88-b1-11-7c-44-d5     dynamic
10.0.0.81             02-0f-b5-b7-fe-be     dynamic
10.0.0.128            3c-37-86-22-f5-fc     dynamic
10.0.0.143            30-96-fb-a6-7d-8d     dynamic
10.0.0.178            02-0f-b5-59-c9-40     dynamic
10.0.0.254            00-05-04-03-02-01     dynamic
10.0.0.255            ff-ff-ff-ff-ff-ff     static
224.0.0.22            01-00-5e-00-00-16     static
224.0.0.251           01-00-5e-00-00-fb     static
224.0.0.252           01-00-5e-00-00-fc     static
239.255.255.250       01-00-5e-7f-ff-fa     static
239.255.255.253       01-00-5e-7f-ff-fd     static
255.255.255.255       ff-ff-ff-ff-ff-ff     static

Interface: 192.168.41.97 --- 0x18
Internet Address      Physical Address      Type
192.168.41.102        00-15-5d-00-e5-02     static
192.168.41.106        00-15-5d-00-e5-06     static
192.168.41.109        00-15-5d-00-e5-01     static
192.168.41.111        ff-ff-ff-ff-ff-ff     static
224.0.0.22            01-00-5e-00-00-16     static
224.0.0.251           01-00-5e-00-00-fb     static
239.255.255.250       01-00-5e-7f-ff-fa     static
239.255.255.253       01-00-5e-7f-ff-fd     static
255.255.255.255       ff-ff-ff-ff-ff-ff     static

Multiple Virtual Networks

Hyper-v supports network virtualization. which allows having multiple virtual networks.

Useful Neworking Commands

ip command (Linux)

The ip command gives a wealth of information, and in many ways it is richer than the older ifconfig.

Example 1: See all the network interface cards (NICs) on a laptop

alex@CC-Labs-2:/etc/nginx$ ip link
8: eth0: <> mtu 1500 group default qlen 1
    link/ether f0:6e:0b:d2:aa:e9
15: eth1: <BROADCAST,MULTICAST,UP> mtu 1500 group default qlen 1
    link/ether 02:00:4c:4f:4f:50
24: eth2: <BROADCAST,MULTICAST,UP> mtu 1500 group default qlen 1
    link/ether 00:15:5d:35:af:70
1: lo: <LOOPBACK,UP> mtu 1500 group default qlen 1
    link/loopback 00:00:00:00:00:00
22: wifi0: <BROADCAST,MULTICAST,UP> mtu 1500 group default qlen 1
    link/ieee802.11 f0:6e:0b:d2:aa:e8
2: wifi1: <> mtu 1500 group default qlen 1
    link/ieee802.11 f2:6e:0b:d2:ab:e9
7: wifi2: <> mtu 1500 group default qlen 1
    link/ieee802.11 f2:6e:0b:d2:ae:e9

Example 2: See what other devices a VM has been talking to. In this case, it has been talking to the virtual switch provided in Hyper-v:

ubuntu@ccl-vm-1:~$ ip neigh
172.17.214.161 dev eth0 lladdr 00:15:5d:35:af:70 DELAY
Example 3: See detailed information about all NICs on a laptop. There are many of them because of multiple

types of link (Wifi, Bluetooth, loopback, …) and also because there are virtual NICs in Hyper-v

alex@CC-Labs-2:/etc/nginx$ ip address
8: eth0: <> mtu 1500 group default qlen 1
    link/ether f0:6e:0b:d2:aa:e9
    inet 169.254.18.111/16 brd 169.254.255.255 scope global dynamic
    valid_lft forever preferred_lft forever
    inet6 fe80::45fe:53cd:8c92:126f/64 scope link dynamic
    valid_lft forever preferred_lft forever
15: eth1: <BROADCAST,MULTICAST,UP> mtu 1500 group default qlen 1
    link/ether 02:00:4c:4f:4f:50
    inet 169.254.147.180/16 brd 169.254.255.255 scope global dynamic
    valid_lft forever preferred_lft forever
    inet6 fe80::c16a:f58d:a7:93b4/64 scope link dynamic
    valid_lft forever preferred_lft forever
24: eth2: <BROADCAST,MULTICAST,UP> mtu 1500 group default qlen 1
    link/ether 00:15:5d:35:af:70
    inet 172.17.214.161/28 brd 172.17.214.175 scope global dynamic
    valid_lft forever preferred_lft forever
    inet6 fe80::8167:6b45:b2b0:daa9/64 scope link dynamic
    valid_lft forever preferred_lft forever
1: lo: <LOOPBACK,UP> mtu 1500 group default qlen 1
    link/loopback 00:00:00:00:00:00
    inet 127.0.0.1/8 brd 127.255.255.255 scope global dynamic
    valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host dynamic
    valid_lft forever preferred_lft forever
22: wifi0: <BROADCAST,MULTICAST,UP> mtu 1500 group default qlen 1
    link/ieee802.11 f0:6e:0b:d2:aa:e8
    inet 10.0.0.229/24 brd 10.0.0.255 scope global dynamic
    valid_lft 495667sec preferred_lft 495667sec
    inet6 2601:647:4c80:4000:d537:8eb2:807f:ba3a/64 scope global dynamic
    valid_lft 197772sec preferred_lft 197772sec
    inet6 2601:647:4c80:4000::4346/128 scope global dynamic
    valid_lft 291327sec preferred_lft 291327sec
    inet6 2601:647:4c80:4000:a81e:a0b8:9196:768/128 scope global dynamic
    valid_lft 197772sec preferred_lft 197772sec
    inet6 fe80::d537:8eb2:807f:ba3a/64 scope link dynamic
    valid_lft forever preferred_lft forever
2: wifi1: <> mtu 1500 group default qlen 1
    link/ieee802.11 f2:6e:0b:d2:ab:e9
    inet 169.254.249.99/16 brd 169.254.255.255 scope global dynamic
    valid_lft forever preferred_lft forever
    inet6 fe80::cd6c:dd90:bd7e:f963/64 scope link dynamic
    valid_lft forever preferred_lft forever
7: wifi2: <> mtu 1500 group default qlen 1
    link/ieee802.11 f2:6e:0b:d2:ae:e9
    inet 169.254.239.154/16 brd 169.254.255.255 scope global dynamic
    valid_lft forever preferred_lft forever
    inet6 fe80::2074:c93:f4d7:ef9a/64 scope link dynamic
    valid_lft forever preferred_lft forever

netstat and arp (Linux and Windows)

This command works in both Linux and in Window’s powershell.

Example 1: See the routing table for a laptop. There are the rules that the O/S uses to determine the gateway to which to send a particular message, based on the destination ip. A bitwise and is done between the mask and the desired destination, and this result is used to make a lookup in the destination column of the routing table. The row that matches then determines the ip of the gateway to which to route the result.

Example 1: For a Hyper-v VM, the mask in the 2nd row of the routing table (255.255.255.240) is a set of 32 bits where all bits are 1 except the the last 4: 1111 1111 1111 1111 1111 1111 1111 0000.

ubuntu@ccl-vm-1:~$ netstat -r
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
default         CC-Labs-2.mshom 0.0.0.0         UG        0 0          0 eth0
192.168.41.96   0.0.0.0         255.255.255.240 U         0 0          0 eth0
CC-Labs-2.mshom 0.0.0.0         255.255.255.255 UH        0 0          0 eth0

For an ip to match the destination of 192.168.41.96 when the mask is applied to the ip, that means that the ip must be of the form 192.168.41.x` where ``BITWISE_AND(x, 1111 000) = 96 (decimal) = 0110 0000 (bitwise).

That means that x must be in the range from 0110 000 to 0110 1111, i.e., the ip ranges must be from 192.168.41.96 to 192.168.41.111.

In other words, that is the range of ips allowed in the virtual network to which virtual host ccl-vm-1 belongs. We could write it as 192.168.41.96/28, since only the last 4 bits out of 32 (since 4=32-28) can change for devices connected on this network.

If there is a match, then the message is routed to 0.0.0.0, which is a way of saying that there is no further hop to make since we already are at the device that should receive the message, i.e., end of the trajectory.

Other entries in the table seem to be “special”. The first row has variable CC-Labs-2.mshom in lieu of an ip address, which appears to be how Hyper-v refers to the “gateway device” to connect to the home network. We say “gateway” because of the “G” flag “UG” for row 1.

The routing table maps this “gateway” in row 3, to the local host 0.0.0.0, which is marked of type “host” because of the “H” flag in “UH”.

This treatment of CC-Labs-2.mshom as if it where an ip address seems to be corroborated by doing an arp to see the mappings between ip and physical addresses:

ubuntu@ccl-vm-1:~$ arp
Address                  HWtype  HWaddress           Flags Mask            Iface
CC-Labs-2.mshome.net     ether   00:15:5d:68:b3:e6   C                     eth0

Interestingly, the MAC address for this CC-Labs-2.mshom gateway (00:15:5d:68:b3:e6) is not that of the NIC for the VM we are in (ccl-vm-1), but for the NIC of the host (CC-Labs-2).

This can be corroborated by doing an ip address in each of the two machines:

For ccl-vm-1:

ubuntu@ccl-vm-1:~$ ip address
    # Extract of output
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:15:5d:00:e5:01 brd ff:ff:ff:ff:ff:ff
    inet 192.168.41.109/28 brd 192.168.41.111 scope global dynamic eth0
    valid_lft 86143sec preferred_lft 86143sec
    inet6 fe80::215:5dff:fe00:e501/64 scope link
    valid_lft forever preferred_lft forever

By contrast, for CC-Labs-2 we see the same 00:15:5d:68:b3:e6 we had in the arp output:

alex@CC-Labs-2:~$ ip address
    # Extract of output
24: eth2: <BROADCAST,MULTICAST,UP> mtu 1500 group default qlen 1
    link/ether 00:15:5d:68:b3:e6
    inet 192.168.41.97/28 brd 192.168.41.111 scope global dynamic
22: wifi0: <BROADCAST,MULTICAST,UP> mtu 1500 group default qlen 1
    link/ieee802.11 f0:6e:0b:d2:aa:e8
    inet 10.0.0.229/24 brd 10.0.0.255 scope global dynamic

So what the routing table is saying, then, is that any request for an ip not in 192.168.41.96/28 (row 2) should by default (row 1 in routing table) be sent to NIC used by the containing host to participate in the Hyper-v network, as seen in the first row when doing Get-VMNetworkAdapter -All.

(base) PS C:\WINDOWS\system32> Get-VMNetworkAdapter -All

Name                   IsManagementOs VMName             SwitchName     MacAddress   Status IPAddresses
----                   -------------- ------             ----------     ----------   ------ -----------
Container NIC dee36933 True                              Default Switch 00155D68B3E6 {Ok}
Network Adapter        False          ccl-vm-1           Default Switch 00155D00E501 {Ok}   {}
Network Adapter        False          primary            Default Switch 00155D00E502 {Ok}   {}
Network Adapter        False          Ubuntu 18.04.3 LTS Default Switch 00155D00E506 {Ok}   {}

I believe this is how the operating system enables the Ubuntu VM to be able to reach out to the internet:

  • A process running in ccl-vm-1 tries to access the internet.

  • O/S in ccl-vm-1 checks the routing table and sees message should be routed to the device called Container NIC dee36933 (MAC 00:15:5D:68:B3:E6) in the host.

  • The NIC for ccl-vm-1 sends message to NIC 00:15:5D:68:B3:E6

  • Presumably, Hyper-v then routes whatever arrives at the host’s Hyper-v NIC (00:15:5D:68:B3:E6) to its external-facing NIC (f0:6e:0b:d2:aa:e8 which is ip 10.0.0.229 on the home network).

  • Within the home network, message then goes from 10.0.0.229 to the router 10.0.0.1, and from there to the internet.