Date: Thu, 20 Nov 2025 23:38:08 +0000 From: bugzilla-noreply@freebsd.org To: net@FreeBSD.org Subject: [Bug 291100] Reduce ARP cache time from 20 minutes to 60 seconds Message-ID: <bug-291100-7501-jgzed7F68B@https.bugs.freebsd.org/bugzilla/> In-Reply-To: <bug-291100-7501@https.bugs.freebsd.org/bugzilla/> References: <bug-291100-7501@https.bugs.freebsd.org/bugzilla/>
next in thread | previous in thread | raw e-mail | index | archive | help
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=291100 --- Comment #4 from Achilles Gaikwad <achillesgaikwad@gmail.com> --- (In reply to Zhenlei Huang from comment #3) >When the IP address is assigned to new a VM's interface, the VM will broadcast ARP request to detect duplicated IP, >and the neighbors can update their ARP cache entry to reflect the new IP-to-MAC mapping, >So it does not matter how long the ARP entry lifetime is. Yes, if an IP address that is re-assigned from VM1 to VM2, a gratuitous ARP will be sent by VM2. The problem I am trying to solve is not that of IP reassignment from one interface to another interface, but that when a VM2 boots and picks an IP that was previously used by VM1 - VM2 does not send a GARP. If I create several other VMs they do not send a GARP on their boot either. >From my testing, when a VM boots, regardless of if it is using an IP address used by a different VM in past, it does not send out a GARP packet. >Do you have troubles that the ARP cache entries are stale ? Yes, ARP entries being stale are a problem here. Example, In an AWS cloud environment - I created a NFS server on my FreeBSD system, and then created several NFS clients to exhaust the IPs in my subnet. 1. Create an NFS server on FreeBSD, and start a packet capture. 2. Create NFS clients and mount the NFS share on them, my subnet had ~26 free IPs so I created ~26 NFS clients. 3. Delete all the NFS clients after verifying that they've successfully mounted the nfs share 4. Create 26 clients again, and attempt NFS mount on them ## Packet capture analysis shows that 1. A GARP request is not sent by a VM if hypervisor if an IP address is _moved_ from one VM to another VM. 2. My VM `172.31.150.4` boots. 3. There is no ARP request sent by the VM during its boot. VM doesn't know if that IP was used by someone else. 4. My NFS server `12:49:76:66:25:e9` asks for the mac address of my NFS client `172.31.150.4` and gets a reply in frame `128` 5. Frames `129`, `130` show a successful handshake while talking to `12:e3:82:23:c1:7b` my NFS client. ``` $ tshark -ntad -r ~/Downloads/freebsd_arp.pcap -Y "(ip.addr == 172.31.150.4 and tcp.port == 111) or (arp.src.proto_ipv4 == 172.31.150.4)" | head 126 2025-11-20 16:15:20.451216 172.31.150.4 → 172.31.150.24 TCP 12:e3:82:23:c1:7b 12:49:76:66:25:e9 38872 → 111 [SYN] Seq=0 Win=62727 Len=0 MSS=8961 SACK_PERM TSval=638822548 TSecr=0 WS=128 128 2025-11-20 16:15:20.451254 0.000038 12:e3:82:23:c1:7b → 12:49:76:66:25:e9 ARP 12:e3:82:23:c1:7b 12:49:76:66:25:e9 172.31.150.4 is at 12:e3:82:23:c1:7b 129 2025-11-20 16:15:20.451262 0.000008 172.31.150.24 → 172.31.150.4 TCP 12:49:76:66:25:e9 12:e3:82:23:c1:7b 111 → 38872 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=8961 WS=64 SACK_PERM TSval=847652929 TSecr=638822548 130 2025-11-20 16:15:20.451457 0.000195 172.31.150.4 → 172.31.150.24 TCP 12:e3:82:23:c1:7b 12:49:76:66:25:e9 38872 → 111 [ACK] Seq=1 Ack=1 Win=62848 Len=0 TSval=638822549 TSecr=847652929 131 2025-11-20 16:15:20.451577 0.000120 172.31.150.4 → 172.31.150.24 Portmap 12:e3:82:23:c1:7b 12:49:76:66:25:e9 V2 GETPORT Call NFS(100003) V:3 TCP ``` 6. I terminate `172.31.150.4`. The IP address is free'd up. 7. I spin up a new VM, and free IP `172.31.150.4` is assigned to my new VM, lets call this VM2. 8. There are no gARP packets between frame `169` and `1661` as the IP was not moved, it was just reused from the pool of available IPs. ``` $ tshark -ntad -r ~/Downloads/freebsd_arp.pcap -Y "(ip.addr == 172.31.150.4 and tcp.port == 111) or (arp.src.proto_ipv4 == 172.31.150.4)" | head ... 169 2025-11-20 16:15:20.590482 0.000224 172.31.150.4 → 172.31.150.24 TCP 12:e3:82:23:c1:7b 12:49:76:66:25:e9 38874 → 111 [ACK] Seq=110 Ack=34 Win=62848 Len=0 TSval=638822688 TSecr=3403520806 <VM2 boots here and reuses the IP address from the free pool> 1661 2025-11-20 16:18:48.198831 207.608349 172.31.150.4 → 172.31.150.24 TCP 12:fd:55:e8:b6:a9 12:49:76:66:25:e9 36578 → 111 [SYN] Seq=0 Win=62727 Len=0 MSS=8961 SACK_PERM TSval=512935728 TSecr=0 WS=128 1662 2025-11-20 16:18:48.198842 0.000011 172.31.150.24 → 172.31.150.4 TCP 12:49:76:66:25:e9 12:e3:82:23:c1:7b 111 → 36578 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=8961 WS=64 SACK_PERM TSval=4022057017 TSecr=512935728 1704 2025-11-20 16:18:49.221276 1.022434 172.31.150.24 → 172.31.150.4 TCP 12:49:76:66:25:e9 12:e3:82:23:c1:7b [TCP Retransmission] 111 → 36578 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=8961 WS=64 SACK_PERM TSval=4022058040 TSecr=512935728 1706 2025-11-20 16:18:49.266915 0.045639 172.31.150.4 → 172.31.150.24 TCP 12:fd:55:e8:b6:a9 12:49:76:66:25:e9 [TCP Retransmission] 36578 → 111 [SYN] Seq=0 Win=62727 Len=0 MSS=8961 SACK_PERM TSval=512936797 TSecr=0 WS=128 1707 2025-11-20 16:18:49.266927 0.000012 172.31.150.24 → 172.31.150.4 TCP 12:49:76:66:25:e9 12:e3:82:23:c1:7b [TCP Retransmission] 111 → 36578 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=8961 WS=64 SACK_PERM TSval=4022058086 TSecr=512936797 1746 2025-11-20 16:18:50.278090 1.011163 172.31.150.24 → 172.31.150.4 TCP 12:49:76:66:25:e9 12:e3:82:23:c1:7b [TCP Retransmission] 111 → 36578 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=8961 WS=64 SACK_PERM TSval=4022059097 TSecr=512936797 1777 2025-11-20 16:18:51.346919 1.068829 172.31.150.4 → 172.31.150.24 TCP 12:fd:55:e8:b6:a9 12:49:76:66:25:e9 [TCP Retransmission] 36578 → 111 [SYN] Seq=0 Win=62727 Len=0 MSS=8961 SACK_PERM TSval=512938877 TSecr=0 WS=128 ``` 9. VM2 tries to mount the NFS share and start a tcp communication in `1661`. The packet comes in from MAC address `12:fd:55:e8:b6:a9` who is the new owner of this IP. Our previous owner was `12:e3:82:23:c1:7b` 10. FreeBSD has a stale entry for upto 20 minutes that is still pointing towards `12:e3:82:23:c1:7b`, so in frame `1662` we see the SYN+ACK is actually going to previous owner of the IP `12:e3:82:23:c1:7b` and not `12:fd:55:e8:b6:a9` 11. We observe that client never gets the SYN,ACK so it keeps retransmitting the SYN pkt. Sever keeps sending the SYN,ACK to the wrong MAC address. Therefore, what was historically OK to have an ARP cache time of 20 minutes, is not suitable for dynamic cloud environments where systems can be short lived. I therefore propose to set the ARP cache timeout from 1200s to 60s similar to what GNU/Linux does. I hope this helps. -- You are receiving this mail because: You are the assignee for the bug.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bug-291100-7501-jgzed7F68B>