Date: Sun, 16 Aug 2015 08:59:11 GMT From: stefano@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r289801 - soc2015/stefano Message-ID: <201508160859.t7G8xBKg039254@socsvn.freebsd.org>
next in thread | raw e-mail | index | archive | help
Author: stefano Date: Sun Aug 16 08:59:10 2015 New Revision: 289801 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=289801 Log: update README ================================ ==================================== Modified: soc2015/stefano/README Modified: soc2015/stefano/README ============================================================================== --- soc2015/stefano/README Sun Aug 16 01:08:59 2015 (r289800) +++ soc2015/stefano/README Sun Aug 16 08:59:10 2015 (r289801) @@ -1,4 +1,4 @@ -A FreeBSD/bhyve version of the netmap virtual passthrough (ptnetmap) for VMs. +AFreeBSD/bhyve version of the netmap virtual passthrough (ptnetmap) for VMs. Student: Stefano Garzarella Mentor: Luigi Rizzo @@ -78,9 +78,130 @@ implement kernel thread in netmap host adapter [D3]. Week 9: --- DONE --- test and bugfix [D3]. -Week 10: --- STARTED --- +Week 10: --- DONE --- implement notification mechanism in bhyve/freebsd [D4]. -Week 11: +Week 11: --- DONE --- test and performance evaluation [D4]. -Week 12: +Week 12: --- DONE --- clean code and prepare documents to final evaluation. + + +=========================== Final Documentation =============================== +My code is available here: +https://svnweb.freebsd.org/socsvn/soc2015/stefano/ptnetmap/stable/10/ +I used the 10-STABLE branch for my work. +To get the patches of modified modules you can follow the following steps: + - bhyve + svn diff -r 287649 usr.sbin/bhyve + - virtio-net + svn diff -r 287649 sys/dev/virtio/network + - vmm.ko + svn diff -r 287649 lib/libvmmapi sys/modules/vmm sys/amd64 + - netmap + the changes will be shortly committed in HEAD and R10 + svn diff -r 287649 sys/conf/files sys/modules/netmap sys/dev/cxgbe \ + sys/dev/netmap sys/net + +The ptnetmap support for linux-KVM and QEMU is available here: +https://github.com/stefano-garzarella/ptnetmap + +I implemented ptnetmap on bhyve working on the following steps: + +- bhyve network backends + I reused the code developed by Luigi Rizzo (my mentor) and Vincenzo + Maffione to support multiple backend in bhyve and to interface them + with a fronteds. + The backends availabale are: + - tap + - netmap (netmap, vale) + - ptnetmap (ptnetmap, ptvale) + +- ptnetmap support on virtio-net device for FreeBSD + I modified the virtio-net guest device driver and the virtio-net fronted + of thh host hypervisor (bhyve) to support ptnetmap. + + guest: + - new PTNETMAP config flag in virtio-net device driver to check + if ptnetmap is supported. + - ptnetmap device-specific code (netmap - if_vtnet_netmap.h) + + host (hypervisor): + - ptnetmap support on virtio-net fronted (bhyve - pci_virtio_ptnetmap.h) + - ptnetmap backend [name: ptnetmap, ptvale] (bhyve - net_backends.c) + +- map netmap host memory into the guest + I added a new IOCTL to vmm.ko to map an userspace guest buffer in the + guest VM. Then I implemented a new PCI device (ptnetmap-memdev) to map + the netmap host memory in the guest through PCI MEMORY-BAR. + + kernel host (vmm.ko): + - new VM_MAP_USER_BUF ioctl to map buffer in the guest + + userspace host (bhyve): + - vm_map_user_buf() in libvmmapi + - ptnetmap‐memdev device emulation (bhyve - pci_ptnetmap_memdev.c) + + kernel guest (netmap): + - device driver for ptnetmap-memdev inluded in netmap module + +- ptnetmap support for FreeBSD host: + I implemented kernel thread in netmap module to support ptnetmap on + FreeBSD host. + + kernel host (netmap): + - nm_os_kthread_*() functions to handle netmap kthreads. + (netmap - netmap_freebsd.c ptnetmap.c) + +- netmap guest/host notification mechanisms. + I needed two mechanisms: + 1) notification from ptnetmap kthread to guest VM (interrupt/irq) + vmm.ko already has IOCTL to send interrupt to the guest and I used it + in the ptnetmap kernel threads. + 2) notification from guest VM to ptnetmap kthread (write into the + specific register) + I added new IOCTL on vmm.ko (VM_IO_REG_HANDLER) to catch write/read + on specific I/O address and send notification. + For now I've implemented only one type of handler (VM_IO_REGH_KWEVENTS) + to use these events in the kernel through wakeup() and tsleep()/msleep(), + but I wrote the code to be easily extended to support other type of + handler (cond_signal, write/ioctl on fd, etc). + + kernel host (vmm.ko): + - new VM_IO_REG_HANDLER ioctl to catch write/read on specific + I/O address and to choose an handler. (eg. wakeup(event_id) + - vm_io_reg_handler() in libvmmapi + + kernel host (netmap) + - msleep() on event_id. + + userspace host (bhyve) + - vm_io_reg_handler(VTCFG_R_QNOTIFY, event_id) + send guest notifications to ptnetmap kthreads when the guest + writes on this virtio register (bhyve - pci_virtio_ptnetmap.h) + - tell to netmap kthreads the event_id where they can wait for + events + + +Example: + +Run a 2GB single‐CPU virtual machine with three network ports which use +netmap and ptnetmap backends: + +bhyve ‐s 0,hostbridge ‐s 1,lpc \ + ‐s 2:1,virtio‐net,vale0:1 \ /* normal vale backend */ + ‐s 2:2,ptnetmap‐memdev \ /* ptnetmap-memdev is needed for each + * ptnetmap port + * (If two or more ptnetmap ports + * share the same netmap memory allocator, + * only one ptnetmap‐memdev is required) + */ + ‐s 2:3,virtio‐net,ptvale1:1 \ /* vale port in ptnetmap mode + * If "pt" prefix is used, the port + * is opened in passthrough mode (ptnetmap) + */ + ‐s 2:2,ptnetmap‐memdev \ + ‐s 2:3,virtio‐net,ptvale2{1 \ /* netmap-pipe in ptnetmap mode */ + ‐s 3,ptnetmap‐memdev \ + ‐s 4,virtio‐net,ptnetmap:ix0 \ /* NIC in ptnetmap mode */ + ‐l com1,stdio ‐A ‐H ‐P ‐m 2G netmapvm +
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201508160859.t7G8xBKg039254>