Date: Fri, 26 Oct 2018 15:34:29 +1100 From: Peter Jeremy <peter@rulingia.com> To: freebsd-arm@freebsd.org Subject: Re: Poor virtio performance on Scaleway ARM systems Message-ID: <20181026043429.GA64669@server.rulingia.com> In-Reply-To: <1540309565.1771.1@smtp.migadu.com> References: <20180916184657.GB24416@server.rulingia.com> <1540309565.1771.1@smtp.migadu.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--BOKacYhQ+x31HxR3 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2018-Oct-23 18:46:05 +0300, Greg V <greg@unrelenting.technology> wrote: >echo 'hw.pci.honor_msi_blacklist=3D"0"' >> /boot/loader.conf > >With the fix, we're getting ~1000 interrupts on 'its0,4: virtio_pci1'=20 >(instead of ~30000 on gic), latency is no longer awful, and bandwidth=20 >is about 220 MB/s. YAY!! Unfortunately, this isn't working for me on 13-CURRENT. I am still seeing ~4.5MBps disk I/O with ~30,000 interrupts/sec on gic0,s5. I don't even see an its0,4. =20 Looking at a verbose boot, the relevant errors are: its0: <ARM GIC Interrupt Translation Service> mem 0x8080000-0x809ffff on gi= c0 its0: GITS_BASER0: unable to be updated: a907000040a40607 !=3D a907000040ee= 0607 device_attach: its0 attach returned 6 =2E.. virtio_pci0: <VirtIO PCI Network adapter> port 0xf040-0xf05f mem 0x3ee09000= -0x3ee09fff,0x3ee00000-0x3ee03fff at device 1.0 on pci0 vtnet0: <VirtIO Networking Adapter> on virtio_pci0 virtio_pci0: host features: 0x79bfffe7 <EventIdx,RingIndirect,0x8000000,Not= ifyOnEmpty,SetMacAddress,GuestAnnounce,RxModeExtra,VLanFilter,RxMode,Contro= lVq,Status,MrgRxBuf,TxUFO,TxTSOECN,TxTSOv6,TxTSOv4,RxUFO,RxECN,RxTSOv6,RxTS= Ov4,TxAllGSO,MacAddress,0x4,RxChecksum,TxChecksum> virtio_pci0: negotiated features: 0x308fbbe3 <EventIdx,RingIndirect,SetMacA= ddress,VLanFilter,RxMode,ControlVq,Status,MrgRxBuf,TxTSOECN,TxTSOv6,TxTSOv4= ,RxECN,RxTSOv6,RxTSOv4,TxAllGSO,MacAddress,RxChecksum,TxChecksum> vtnet0: bpf attached vtnet0: Ethernet address: de:2b:88:06:a0:4c virtio_pci0: attempting to allocate 3 MSI-X vectors (3 supported) virtio_pci0: attempting to allocate 2 MSI-X vectors (3 supported) virtio_pci0: using legacy interrupt virtio_pci1: <VirtIO PCI Block adapter> port 0xf000-0xf03f mem 0x3ee08000-0= x3ee08fff,0x3ee04000-0x3ee07fff at device 2.0 on pci0 vtblk0: <VirtIO Block Adapter> on virtio_pci1 virtio_pci1: host features: 0x79000e54 <EventIdx,RingIndirect,0x8000000,Not= ifyOnEmpty,ConfigWCE,Topology,WriteCache,BlockSize,DiskGeometry,MaxNumSegs> virtio_pci1: negotiated features: 0x10000e54 <RingIndirect,ConfigWCE,Topolo= gy,WriteCache,BlockSize,DiskGeometry,MaxNumSegs> virtio_pci1: attempting to allocate 2 MSI-X vectors (2 supported) virtio_pci1: attempting to allocate 2 MSI-X vectors (2 supported) virtio_pci1: using legacy interrupt vtblk0: 47683MB (97656250 512 byte sectors) I haven't had a chance to dig into either of these any further but hope to = do so over the weekend. -- Peter Jeremy --BOKacYhQ+x31HxR3 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEE7rKYbDBnHnTmXCJ+FqWXoOSiCzQFAlvSmVRfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEVF QjI5ODZDMzA2NzFFNzRFNjVDMjI3RTE2QTU5N0EwRTRBMjBCMzQACgkQFqWXoOSi CzQ4Yg/+Nap/BL1gukJpj5bO5V7EBC0aijNpgq72zLXqFzmtjvfUjIoAZmdIqtq4 F/U1hRz1oMA932x83fvMlChO45bjBq/BymZG508cGuL6NvlSuy4lFlQxBzR1XFsC sW/AXYBCdavZFdN6qMY3WGKOrfvqzvqCVYGAkcVwmvukBUbvoDy5amy3TXcRPKyL Tr/Bp4woIIZ7LZfbpDdur3u0PAVAfuCNYMQ8esUWGkxGZ43VqNbnmrWGQWPHqCcY 5Ozr8Y53KwdQqsruBAmCWKhcsvBg7YYAkTEfnzt9a8FTw/cc+i5NPIeS0nvaMroT NAJKoh+kt4d0xCHk2MkqIWMnk634XFczzDaOoNU94FbgezAByaou7u7padkFht+B 4tRSsf3KQtJFL6DMvNnxFcfoYdBZ00WGWBgUrFI+CtEyfsxZt990m6GySNsLJwRm h21UnS1Ro6Kne2rrvBeFnCap14qvtePdQlvOMyipcgXX8q5pLxFQ+HKSx7KsJ29v 7q/F0z8NqB0ziAO2w2axPahj9vh7nuER+9LxYq3j2sF5QY9vP6Uj9dRg8EiIEq0Q /kYwCy1I4fS+5YFyvrL1vATP/3E1RuUXn5DcdpiN2g9xhP1RaGTT089rLkyE+jr/ vdzFNKLTDLEUFPIGvc2gyT43qpi3Em/hx11Y6ul6pp37JBrcEBM= =2p1g -----END PGP SIGNATURE----- --BOKacYhQ+x31HxR3--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20181026043429.GA64669>