From owner-svn-soc-all@freebsd.org Fri Aug 5 13:53:36 2016 Return-Path: Delivered-To: svn-soc-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C4973BAFDE1 for ; Fri, 5 Aug 2016 13:53:36 +0000 (UTC) (envelope-from vincenzo@FreeBSD.org) Received: from socsvn.freebsd.org (socsvn.freebsd.org [IPv6:2001:1900:2254:206a::50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A45671CE3 for ; Fri, 5 Aug 2016 13:53:36 +0000 (UTC) (envelope-from vincenzo@FreeBSD.org) Received: from socsvn.freebsd.org ([127.0.1.124]) by socsvn.freebsd.org (8.15.2/8.15.2) with ESMTP id u75Drat3036187 for ; Fri, 5 Aug 2016 13:53:36 GMT (envelope-from vincenzo@FreeBSD.org) Received: (from www@localhost) by socsvn.freebsd.org (8.15.2/8.15.2/Submit) id u75DrZQF036169 for svn-soc-all@FreeBSD.org; Fri, 5 Aug 2016 13:53:35 GMT (envelope-from vincenzo@FreeBSD.org) Date: Fri, 5 Aug 2016 13:53:35 GMT Message-Id: <201608051353.u75DrZQF036169@socsvn.freebsd.org> X-Authentication-Warning: socsvn.freebsd.org: www set sender to vincenzo@FreeBSD.org using -f From: vincenzo@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r307218 - soc2016/vincenzo/head/usr.sbin/bhyve MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Aug 2016 13:53:36 -0000 Author: vincenzo Date: Fri Aug 5 13:53:35 2016 New Revision: 307218 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=307218 Log: add support for ptnetmap PCI device Added: soc2016/vincenzo/head/usr.sbin/bhyve/pci_ptnetmap_memdev.c Modified: soc2016/vincenzo/head/usr.sbin/bhyve/Makefile Modified: soc2016/vincenzo/head/usr.sbin/bhyve/Makefile ============================================================================== --- soc2016/vincenzo/head/usr.sbin/bhyve/Makefile Fri Aug 5 13:51:56 2016 (r307217) +++ soc2016/vincenzo/head/usr.sbin/bhyve/Makefile Fri Aug 5 13:53:35 2016 (r307218) @@ -32,6 +32,7 @@ pci_irq.c \ pci_lpc.c \ pci_passthru.c \ + pci_ptnetmap_memdev.c \ pci_virtio_block.c \ pci_virtio_net.c \ pci_virtio_rnd.c \ Added: soc2016/vincenzo/head/usr.sbin/bhyve/pci_ptnetmap_memdev.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ soc2016/vincenzo/head/usr.sbin/bhyve/pci_ptnetmap_memdev.c Fri Aug 5 13:53:35 2016 (r307218) @@ -0,0 +1,319 @@ +/* + * Copyright (C) 2015 Stefano Garzarella (stefano.garzarella@gmail.com) + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include +__FBSDID("$FreeBSD$"); + +#include +#include +#include +#include + +#include /* IFNAMSIZ */ +#include +#include + +#include +#include + +#include "bhyverun.h" +#include "pci_emul.h" + +/* + * ptnetmap memdev PCI device + * + * This device is used to map a netmap memory allocator on the guest VM + * through PCI_BAR. The same allocator can be shared between multiple ptnetmap + * ports in the guest. + * + * Each netmap allocator has a unique ID assigned by the netmap host module. + * + * The implementation here is based on the QEMU/KVM one. + */ +struct ptn_memdev_softc { + struct pci_devinst *pi; /* PCI device instance */ + + void *mem_ptr; /* netmap shared memory */ + uint64_t mem_size; /* netmap shared memory size */ + uint16_t mem_id; /* netmap memory allocator ID */ + + TAILQ_ENTRY(ptn_memdev_softc) next; +}; +static TAILQ_HEAD(, ptn_memdev_softc) ptn_memdevs = TAILQ_HEAD_INITIALIZER(ptn_memdevs); + +/* + * ptn_memdev_softc can be created by pe_init or ptnetmap backend, + * this depends on the order of initialization. + */ +static struct ptn_memdev_softc * +ptn_memdev_create() +{ + struct ptn_memdev_softc *sc; + + sc = calloc(1, sizeof(struct ptn_memdev_softc)); + if (sc != NULL) { + TAILQ_INSERT_TAIL(&ptn_memdevs, sc, next); + } + + return sc; +} + +static void +ptn_memdev_delete(struct ptn_memdev_softc *sc) +{ + TAILQ_REMOVE(&ptn_memdevs, sc, next); + + free(sc); +} + +/* + * Find ptn_memdev through mem_id (netmap memory allocator ID) + */ +static struct ptn_memdev_softc * +ptn_memdev_find_memid(uint16_t mem_id) +{ + struct ptn_memdev_softc *sc; + + TAILQ_FOREACH(sc, &ptn_memdevs, next) { + if (sc->mem_ptr != NULL && mem_id == sc->mem_id) { + return sc; + } + } + + return NULL; +} + +/* + * Find ptn_memdev that has not netmap memory (attached by ptnetmap backend) + */ +static struct ptn_memdev_softc * +ptn_memdev_find_empty_mem() +{ + struct ptn_memdev_softc *sc; + + TAILQ_FOREACH(sc, &ptn_memdevs, next) { + if (sc->mem_ptr == NULL) { + return sc; + } + } + + return NULL; +} + +/* + * Find ptn_memdev that has not PCI device istance (created by pe_init) + */ +static struct ptn_memdev_softc * +ptn_memdev_find_empty_pi() +{ + struct ptn_memdev_softc *sc; + + TAILQ_FOREACH(sc, &ptn_memdevs, next) { + if (sc->pi == NULL) { + return sc; + } + } + + return NULL; +} + +/* + * Handle read on ptnetmap-memdev register + */ +static uint64_t +ptn_pci_read(struct vmctx *ctx, int vcpu, struct pci_devinst *pi, + int baridx, uint64_t offset, int size) +{ + struct ptn_memdev_softc *sc = pi->pi_arg; + + if (sc == NULL) + return 0; + + if (baridx == PTNETMAP_IO_PCI_BAR) { + switch (offset) { + case PTNETMAP_IO_PCI_MEMSIZE: + return sc->mem_size; + case PTNETMAP_IO_PCI_HOSTID: + return sc->mem_id; + } + } + + printf("%s: Unexpected register read [bar %u, offset %lx size %d]\n", + __func__, baridx, offset, size); + + return 0; +} + +/* + * Handle write on ptnetmap-memdev register (unused for now) + */ +static void +ptn_pci_write(struct vmctx *ctx, int vcpu, struct pci_devinst *pi, + int baridx, uint64_t offset, int size, uint64_t value) +{ + struct ptn_memdev_softc *sc = pi->pi_arg; + + if (sc == NULL) + return; + + printf("%s: Unexpected register write [bar %u, offset %lx size %d " + "value %lx]\n", __func__, baridx, offset, size, value); +} + +/* + * Configure the ptnetmap-memdev PCI BARs. PCI BARs can only be created + * when the PCI device is created and the netmap memory is attached. + */ +static int +ptn_memdev_configure_bars(struct ptn_memdev_softc *sc) +{ + int ret; + + if (sc->pi == NULL || sc->mem_ptr == NULL) + return 0; + + /* Allocate a BAR for an I/O region. */ + ret = pci_emul_alloc_bar(sc->pi, PTNETMAP_IO_PCI_BAR, PCIBAR_IO, + PTNETMAP_IO_SIZE); + if (ret) { + printf("ptnetmap_memdev: iobar allocation error %d\n", ret); + return ret; + } + + /* Allocate a BAR for a memory region. */ + ret = pci_emul_alloc_bar(sc->pi, PTNETMAP_MEM_PCI_BAR, PCIBAR_MEM32, + sc->mem_size); + if (ret) { + printf("ptnetmap_memdev: membar allocation error %d\n", ret); + return ret; + } + + /* Map netmap memory on the memory BAR. */ + ret = vm_map_user_buf(sc->pi->pi_vmctx, + sc->pi->pi_bar[PTNETMAP_MEM_PCI_BAR].addr, + sc->mem_size, sc->mem_ptr); + if (ret) { + printf("ptnetmap_memdev: membar map error %d\n", ret); + return ret; + } + + return 0; +} + +/* + * PCI device initialization + */ +static int +ptn_memdev_init(struct vmctx *ctx, struct pci_devinst *pi, char *opts) +{ + struct ptn_memdev_softc *sc; + int ret; + + sc = ptn_memdev_find_empty_pi(); + if (sc == NULL) { + sc = ptn_memdev_create(); + if (sc == NULL) { + printf("ptnetmap_memdev: calloc error\n"); + return (ENOMEM); + } + } + + /* Link our softc in the pci_devinst. */ + pi->pi_arg = sc; + sc->pi = pi; + + /* Initialize PCI configuration space. */ + pci_set_cfgdata16(pi, PCIR_VENDOR, PTNETMAP_PCI_VENDOR_ID); + pci_set_cfgdata16(pi, PCIR_DEVICE, PTNETMAP_PCI_DEVICE_ID); + pci_set_cfgdata8(pi, PCIR_CLASS, PCIC_NETWORK); + pci_set_cfgdata16(pi, PCIR_SUBDEV_0, 1); + pci_set_cfgdata16(pi, PCIR_SUBVEND_0, PTNETMAP_PCI_VENDOR_ID); + + /* Configure PCI-BARs. */ + ret = ptn_memdev_configure_bars(sc); + if (ret) { + printf("ptnetmap_memdev: configure error\n"); + goto err; + } + + return 0; +err: + ptn_memdev_delete(sc); + pi->pi_arg = NULL; + return ret; +} + +/* + * used by ptnetmap backend to attach the netmap memory allocator to the + * ptnetmap-memdev. (shared with the guest VM through PCI-BAR) + */ +int +ptn_memdev_attach(void *mem_ptr, uint32_t mem_size, uint16_t mem_id) +{ + struct ptn_memdev_softc *sc; + int ret; + + /* if a device with the same mem_id is already attached, we are done */ + if (ptn_memdev_find_memid(mem_id)) { + printf("ptnetmap_memdev: already attched\n"); + return 0; + } + + sc = ptn_memdev_find_empty_mem(); + if (sc == NULL) { + sc = ptn_memdev_create(); + if (sc == NULL) { + printf("ptnetmap_memdev: calloc error\n"); + return (ENOMEM); + } + } + + sc->mem_ptr = mem_ptr; + sc->mem_size = mem_size; + sc->mem_id = mem_id; + + /* configure device PCI-BARs */ + ret = ptn_memdev_configure_bars(sc); + if (ret) { + printf("ptnetmap_memdev: configure error\n"); + goto err; + } + + + return 0; +err: + ptn_memdev_delete(sc); + sc->pi->pi_arg = NULL; + return ret; +} + +struct pci_devemu pci_de_ptnetmap = { + .pe_emu = PTN_MEMDEV_NAME, + .pe_init = ptn_memdev_init, + .pe_barwrite = ptn_pci_write, + .pe_barread = ptn_pci_read +}; +PCI_EMUL_SET(pci_de_ptnetmap);