Date: Sat, 20 Aug 2016 15:32:01 GMT From: vincenzo@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r308091 - soc2016/vincenzo Message-ID: <201608201532.u7KFW1LL079268@socsvn.freebsd.org>
next in thread | raw e-mail | index | archive | help
Author: vincenzo Date: Sat Aug 20 15:32:00 2016 New Revision: 308091 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=308091 Log: update README with report Modified: soc2016/vincenzo/README Modified: soc2016/vincenzo/README ============================================================================== --- soc2016/vincenzo/README Sat Aug 20 14:04:51 2016 (r308090) +++ soc2016/vincenzo/README Sat Aug 20 15:32:00 2016 (r308091) @@ -4,7 +4,7 @@ Mentor: Luigi Rizzo (luigi AT freebsd DOT org) -=========================== Project proposal ========================== +======================== Original project proposal ======================= Netmap passhthrough (ptnetmap) has been recently introduced on Linux/FreeBSD platforms, where QEMU-KVM/bhyve hypervisors allow VMs to exchange over 20 Mpps @@ -96,7 +96,7 @@ interrupt-based operation under lower loads. -===================== The ptnet paravirtualized device ================ +================== The ptnet paravirtualized device =================== The original ptnetmap implementation required ptnetmap-enabled virtio-net/e1000 drivers. Only the notification functionalities of those devices were reused, @@ -163,15 +163,15 @@ Timetable: - * Week 1-2: Write ptnet FreeBSD driver supporting netmap native applications [D1] --> COMPLETED - * Week 3: Tests, bug-fixing, and performance evaluation [D1] --> COMPLETED - * Week 4-5: Write FreeBSD network interface support for the ptnet driver. [D2] --> COMPLETED - * Week 6: Tests, bug-fixing [D2]. Prepare documents for mid-term evaluation. --> COMPLETED - * Week 7-8: Add virtio-net header support to the ptnet driver [D3]. --> COMPLETED - * Week 9: Test, bug-fixes and performance evaluation [D3]. --> COMPLETED - * Week 10: Write ptnet device model emulation for bhyve [D4] --> COMPLETED - * Week 11: Test and performance evaluation over bhyve [D4]. - * Week 12: Clean code and prepare documentation for final evaluation. + * Week 1-2: Write ptnet FreeBSD driver supporting netmap native applications [D1] + * Week 3: Tests, bug-fixing, and performance evaluation [D1] + * Week 4-5: Write FreeBSD network interface support for the ptnet driver [D2] + * Week 6: Tests, bug-fixing [D2]. Prepare documents for mid-term evaluation + * Week 7-8: Add virtio-net header support to the ptnet driver [D3] + * Week 9: Test, bug-fixes and performance evaluation [D3] + * Week 10: Write ptnet device model emulation for bhyve [D4] + * Week 11: Test and performance evaluation over bhyve [D4] + * Week 12: Clean code and prepare documentation for final evaluation ========================== Final submission ================================= @@ -183,7 +183,7 @@ The complete list of my SVN commits can be obtained with the following command on the SVN repository $ svn log -r 302612:HEAD -Instructions to extract patches of the subsystem I modified are reported +Instructions to extract patches of the subsystems I modified are reported below. Moreover, all the modifications I did to netmap (see below) have also been @@ -224,9 +224,10 @@ I implemented the routines required to attach/detach the ptnet PCI device (ptnet_probe, ptnet_attach, ptnet_detach, ...). - I developed the routines necessary to open and use the ptnet NIC in + I developed the routines necessary to open and use a ptnet NIC in netmap mode (ptnet_nm_register, ptnet_nm_txsync, ptnet_nm_rxsync, - ptnet_nm_config, ...). I managed to share some routines + ptnet_nm_config, ...), and to react to MSI-X interrupts. + I managed to share some routines (ptnet_nm_krings_create, ptnet_nm_krings_delete, ptnet_nm_dtor) between the Linux and FreeBSD drivers, to improve code reuse. @@ -236,12 +237,39 @@ I also implemented a ptnet_poll() routine to support FreeBSD polling (DEVICE_POLLING). + Using as a reference the FreeBSD virtio-net driver (if_vtnet.c), I added + support for hw offloadings (TSO and TCP/UDP checksum offloading), by + means of the virtio-net header, which is prepended to each Ethernet + frame. The virtio-net header can be enabled or disabled by configuring + a register of the ptnet device. A boolean sysctl belonging to the + netmap module (named ptnet_vnet_hdr) can be tuned for this purpose. + * As a second step, I extended bhyve to emulate the ptnet device, using as a reference the emulation that I had already developed for QEMU: (https://github.com/vmaffione/qemu/blob/ptnet/hw/net/ptnetmap-netif.c). - I reused previous work from by myself, my mentor Luigi Rizzo - and Stefano Garzarella, that we did in the past to add netmap and - ptnetmap support for bhyve, and to extend libvmm and the vmm module to + + As a preliminary step, I adapted previous work from by myself, my mentor + Luigi Rizzo and Stefano Garzarella, that we did in the past to (i) add + netmap and ptnetmap support for bhyve, and (ii) extend libvmm and the vmm + module to allow netmap (kernel-space code) to inject interrupts in the + guest and intercept guest I/O register accesses. + + The emulation for ptnet is available here + https://svnweb.freebsd.org/socsvn/soc2016/vincenzo/head/usr.sbin/bhyve/pci_ptnetmap_netif.c?view=markup + which implements (i) the callbacks to manage accesses to the register of + the ptnet device (PTCTL, CSBBAH, CSBBAL, VNET_HDR_LEN, ...); (ii) the + setup necessary to let netmap inject MSI-X interrupts and intercept + writes to the ptnet kick registers. + + Finally, I reorganized and fixed the userspace netmap backend for + bhyve, completing the work we did in the past. This new backend is + intended to replace the old one that is currently in FreeBSD head. + The code in pci_virtio_net.c has been reorganized to support different + backends (TAP, netmap, ptnetmap), while previously it was only able + to support the TAP backend. The backends are now implemented in + a separate file: + https://svnweb.freebsd.org/socsvn/soc2016/vincenzo/head/usr.sbin/bhyve/net_backends.c?view=markup + Example on how to use various NICs (including ptnet) with bhyve VMs: @@ -256,6 +284,7 @@ vm1 where the bhyve VM named "vm1" has three virtual NICs: + * A virtio-net NIC which uses a traditional TAP device as a backend. The TAP backend is implemented in userspace, as a part of bhyve. * A virtio-net NIC which uses an host netmap VALE port as backend. The @@ -265,13 +294,12 @@ implemented as part of the netmap subsystem. - - ======================= Useful links ============================== - * [0] http://info.iet.unipi.it/~luigi/netmap/ - * [1] https://wiki.freebsd.org/SummerOfCode2016/PtnetDriverAndDeviceModel#preview - * [2] https://svnweb.freebsd.org/socsvn/soc2016/vincenzo/ - * [3] https://github.com/luigirizzo/netmap - * [4] https://github.com/vmaffione/qemu/tree/ptnet + * [0] http://info.iet.unipi.it/~luigi/papers/20160613-ptnet.pdf + * [1] http://info.iet.unipi.it/~luigi/netmap/ + * [2] https://wiki.freebsd.org/SummerOfCode2016/PtnetDriverAndDeviceModel#preview + * [3] https://svnweb.freebsd.org/socsvn/soc2016/vincenzo/ + * [4] https://github.com/luigirizzo/netmap + * [5] https://github.com/vmaffione/qemu/tree/ptnet
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201608201532.u7KFW1LL079268>