32 bit network prefix via DHCP

[Richard_Velox]

As a followup to 32 bit network prefix via DHCP? it seems that there are two networking stacks running on Meraki routers; One that can correctly use the information provided from DHCP and communicate with hosts on the Internet and one that checks for connectivity back to Meraki Cloud that sends ARP requests out the WAN interface for the host it's trying to communicate with.

This is unusual as devices normally exhibit one of these behaviors and are classified respectively as works with 32-bit prefixes and doesn't work. Which leads me to believe that Meraki routers use at least two networking stacks.

Network traffic looks normal for long enough to think the device might even be working:

Spoiler

15:37:30.816279 2c:3f:0b:xx:xx:xx > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 346: (tos 0x0, ttl 250, id 39387, offset 0, flags [none], proto UDP (17), length 332)
0.0.0.0.68 > 255.255.255.255.67: [udp sum ok] BOOTP/DHCP, Request from 2c:3f:0b:xx:xx:xx, length 304, xid 0x13423f53, Flags [none] (0x0000)
Client-Ethernet-Address 2c:3f:0b:xx:xx:xx
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message Option 53, length 1: Discover
Client-ID Option 61, length 7: ether 2c:3f:0b:xx:xx:xx
Hostname Option 12, length 25: "ft-gw-aylmer-2c3f0bxxxxxx"
Vendor-Class Option 60, length 6: "MERAKI"
Vendor-Option Option 43, length 3: 1.0.255
Parameter-Request Option 55, length 7:
Subnet-Mask, Domain-Name, Default-Gateway, BR
Hostname, Domain-Name-Server, MTU
15:37:30.837378 44:d9:e7:xx:xx:xx > 2c:3f:0b:xx:xx:xx, ethertype IPv4 (0x0800), length 342: (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328)
100.64.18.254.67 > 192.0.2.44.68: [udp sum ok] BOOTP/DHCP, Reply, length 300, xid 0x13423f53, Flags [none] (0x0000)
Your-IP 192.0.2.44
Client-Ethernet-Address 2c:3f:0b:xx:xx:xx
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message Option 53, length 1: Offer
Server-ID Option 54, length 4: 100.64.18.254
Lease-Time Option 51, length 4: 86400
Subnet-Mask Option 1, length 4: 255.255.255.255
Default-Gateway Option 3, length 4: 100.64.18.254
Domain-Name-Server Option 6, length 8: 100.127.255.249,100.127.255.248
15:37:30.837771 2c:3f:0b:xx:xx:xx > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 356: (tos 0x0, ttl 250, id 48399, offset 0, flags [none], proto UDP (17), length 342)
0.0.0.0.68 > 255.255.255.255.67: [udp sum ok] BOOTP/DHCP, Request from 2c:3f:0b:xx:xx:xx, length 314, xid 0x13423f53, Flags [none] (0x0000)
Client-Ethernet-Address 2c:3f:0b:xx:xx:xx
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message Option 53, length 1: Request
Server-ID Option 54, length 4: 100.64.18.254
Requested-IP Option 50, length 4: 192.0.2.44
Client-ID Option 61, length 7: ether 2c:3f:0b:xx:xx:xx
Hostname Option 12, length 25: "ft-gw-aylmer-2c3f0bxxxxxx"
Vendor-Class Option 60, length 6: "MERAKI"
Vendor-Option Option 43, length 3: 1.0.255
Parameter-Request Option 55, length 7:
Subnet-Mask, Domain-Name, Default-Gateway, BR
Hostname, Domain-Name-Server, MTU
15:37:30.892254 44:d9:e7:xx:xx:xx > 2c:3f:0b:xx:xx:xx, ethertype IPv4 (0x0800), length 342: (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328)
100.64.18.254.67 > 192.0.2.44.68: [udp sum ok] BOOTP/DHCP, Reply, length 300, xid 0x13423f53, Flags [none] (0x0000)
Your-IP 192.0.2.44
Client-Ethernet-Address 2c:3f:0b:xx:xx:xx
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message Option 53, length 1: ACK
Server-ID Option 54, length 4: 100.64.18.254
Lease-Time Option 51, length 4: 86400
Subnet-Mask Option 1, length 4: 255.255.255.255
Default-Gateway Option 3, length 4: 100.64.18.254
Domain-Name-Server Option 6, length 8: 100.127.255.249,100.127.255.248
15:37:31.207067 2c:3f:0b:xx:xx:xx > 44:d9:e7:xx:xx:xx, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 200, id 812, offset 0, flags [none], proto ICMP (1), length 84)
192.0.2.44 > 8.8.8.8: ICMP echo request, id 42246, seq 812, length 64
15:37:31.225343 2c:3f:0b:xx:xx:xx > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has 192.0.2.44 (ff:ff:ff:ff:ff:ff) tell 0.0.0.0, length 46
15:37:31.232138 44:d9:e7:xx:xx:xx > 2c:3f:0b:xx:xx:xx, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 117, id 0, offset 0, flags [none], proto ICMP (1), length 84)
8.8.8.8 > 192.0.2.44: ICMP echo reply, id 42246, seq 812, length 64

Until this ARP request you might think the router was working normally.

Spoiler

15:37:32.269932 2c:3f:0b:xx:xx:xx > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has 143.204.0.73 tell 192.0.2.44, length 46

And communication through the other stack keeps on working.

pastebin for more tcpdump output 

At any rate it's enough for the router to mark the connection as inactive.