Date: Thu, 14 Jul 2022 20:05:17 +0000 From: John <jwd@freebsd.org> To: FreeBSD-Hackers <freebsd-hackers@freebsd.org> Subject: Dynamic VF mac address manipulation / Bhyve Message-ID: <YtB2/UbVGa28aFCq@FreeBSD.org>
next in thread | raw e-mail | index | archive | help
Hi -
I am slowly converting a Linux based qemu/kvm solution to FreeBSD
Bhyve where VMs have mac addresses assigned to them.
Linux allows individual VF entries per $iface to be manipulated:
ip link set ${iface} vf $vface mac $vfmac
The VF is then passed into qemu/kvm and the VM sees the mac address
as assigned above ($vfmac):
-device vfio-pci,host=81:02.1,bus=pci.1,id=vfnet0
iovctl allows for mac address control of all VFs under an interface
at one time - and requires the destruction and re-creation of all VF
entries in order to modify a single entry - which breaks existing
running VMs.
Is there a way I have missed to modify the mac address of a single
VF which is presented to a VM?
I have tried the following steps.
# iovctl -C -f /root/iov-cc0.conf # Create, for instance, 4 VFs agains cc0.
# pciconf -l | grep vf
t6vf0@pci0:65:0:8: class=0x020000 rev=0x00 hdr=0x00 vendor=0x1425 device=0x6808 subvendor=0x1425 subdevice=0x0000
t6vf1@pci0:65:0:12: class=0x020000 rev=0x00 hdr=0x00 vendor=0x1425 device=0x6808 subvendor=0x1425 subdevice=0x0000
t6vf2@pci0:65:0:16: class=0x020000 rev=0x00 hdr=0x00 vendor=0x1425 device=0x6808 subvendor=0x1425 subdevice=0x0000
t6vf3@pci0:65:0:20: class=0x020000 rev=0x00 hdr=0x00 vendor=0x1425 device=0x6808 subvendor=0x1425 subdevice=0x0000
ccv0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=66ec07bb<RXCSUM,TXCSUM,VLAN_MTU,....
ether 02:00:00:00:00:00
media: Ethernet 100GBase-SR4 <full-duplex> (100GBase-SR4 <full-duplex,rxpause,txpause>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
ccv1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=66ec07bb<RXCSUM,TXCSUM,VLAN_MTU,....
ether 02:00:00:00:00:00
media: Ethernet 100GBase-SR4 <full-duplex> (100GBase-SR4 <full-duplex,rxpause,txpause>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
ccv2: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=66ec07bb<RXCSUM,TXCSUM,VLAN_MTU,....
ether 02:00:00:00:00:00
media: Ethernet 100GBase-SR4 <full-duplex> (100GBase-SR4 <full-duplex,rxpause,txpause>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
ccv3: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=66ec07bb<RXCSUM,TXCSUM,VLAN_MTU,....
ether 02:00:00:00:00:00
media: Ethernet 100GBase-SR4 <full-duplex> (100GBase-SR4 <full-duplex,rxpause,txpause>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
Note: initial mac 02:00:00:00:00:00 indicating not-in-use and/or available.
# ifconfig ccv3 ether 02:22:33:44:55:66
# ifconfig ccv3
ccv3: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=66ec07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6,HWRXTSTMP,NOMAP,VXLAN_HWCSUM,VXLAN_HWTSO>
ether 02:22:33:44:55:66
hwaddr 02:00:00:00:00:00
media: Ethernet 100GBase-SR4 <full-duplex> (100GBase-SR4 <full-duplex,rxpause,txpause>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
Ether now shows with new mac - hwaddr the original as assigned by iovctl.
Now re-assign the VF device to the pass-thru driver:
devctl set driver -f pci0:65:0:20 ppt
The VF device can now be passed into bhyve (65/0/20) but the new
value 02:22:33:44:55:66 is lost and the prior value 02:00:00:00:00:00
is seen by the vm - but we want 02:22:33:44:55:66. So the new mac is
a property of the driver, not the VF.
At this point I'm wondering if adding a -U (update) option to iovctl
or a new standalone program to target an individual iface/vf pair
for update might be the answer.
Comments welcome.
Thanks,
John
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?YtB2/UbVGa28aFCq>