From owner-svn-soc-all@freebsd.org Thu Aug 2 10:04:05 2018 Return-Path: Delivered-To: svn-soc-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4ED521061AEA for ; Thu, 2 Aug 2018 10:04:05 +0000 (UTC) (envelope-from sduo@FreeBSD.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B05298140E for ; Thu, 2 Aug 2018 10:04:04 +0000 (UTC) (envelope-from sduo@FreeBSD.org) Received: from socsvn.freebsd.org (socsvn.freebsd.org [IPv6:2001:1900:2254:206a::50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 1A89D26A3D for ; Thu, 2 Aug 2018 10:04:04 +0000 (UTC) (envelope-from sduo@FreeBSD.org) Received: from socsvn.freebsd.org ([127.0.1.124]) by socsvn.freebsd.org (8.15.2/8.15.2) with ESMTP id w72A44E4022859 for ; Thu, 2 Aug 2018 10:04:04 GMT (envelope-from sduo@FreeBSD.org) Received: (from www@localhost) by socsvn.freebsd.org (8.15.2/8.15.2/Submit) id w72A3rci022646 for svn-soc-all@FreeBSD.org; Thu, 2 Aug 2018 10:03:53 GMT (envelope-from sduo@FreeBSD.org) Date: Thu, 2 Aug 2018 10:03:53 GMT Message-Id: <201808021003.w72A3rci022646@socsvn.freebsd.org> X-Authentication-Warning: socsvn.freebsd.org: www set sender to sduo@FreeBSD.org using -f From: sduo@FreeBSD.org To: svn-soc-all@FreeBSD.org Subject: socsvn commit: r337276 - in soc2018/sduo/head/sys: dev/netmap net MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-soc-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: SVN commit messages for the entire Summer of Code repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 10:04:05 -0000 Author: sduo Date: Thu Aug 2 10:03:51 2018 New Revision: 337276 URL: http://svnweb.FreeBSD.org/socsvn/?view=rev&rev=337276 Log: Updated netmap files. Modified: soc2018/sduo/head/sys/dev/netmap/if_em_netmap.h soc2018/sduo/head/sys/dev/netmap/if_igb_netmap.h soc2018/sduo/head/sys/dev/netmap/if_ixl_netmap.h soc2018/sduo/head/sys/dev/netmap/if_lem_netmap.h soc2018/sduo/head/sys/dev/netmap/if_ptnet.c soc2018/sduo/head/sys/dev/netmap/if_re_netmap.h soc2018/sduo/head/sys/dev/netmap/if_vtnet_netmap.h soc2018/sduo/head/sys/dev/netmap/ixgbe_netmap.h soc2018/sduo/head/sys/dev/netmap/netmap.c soc2018/sduo/head/sys/dev/netmap/netmap_bdg.c soc2018/sduo/head/sys/dev/netmap/netmap_generic.c soc2018/sduo/head/sys/dev/netmap/netmap_kern.h soc2018/sduo/head/sys/dev/netmap/netmap_mem2.c soc2018/sduo/head/sys/dev/netmap/netmap_mem2.h soc2018/sduo/head/sys/dev/netmap/netmap_monitor.c soc2018/sduo/head/sys/dev/netmap/netmap_offloadings.c soc2018/sduo/head/sys/dev/netmap/netmap_pipe.c soc2018/sduo/head/sys/dev/netmap/netmap_vale.c soc2018/sduo/head/sys/net/netmap.h soc2018/sduo/head/sys/net/netmap_user.h Modified: soc2018/sduo/head/sys/dev/netmap/if_em_netmap.h ============================================================================== --- soc2018/sduo/head/sys/dev/netmap/if_em_netmap.h Thu Aug 2 00:07:30 2018 (r337275) +++ soc2018/sduo/head/sys/dev/netmap/if_em_netmap.h Thu Aug 2 10:03:51 2018 (r337276) @@ -142,9 +142,10 @@ nic_i = netmap_idx_k2n(kring, nm_i); for (n = 0; nm_i != head; n++) { struct netmap_slot *slot = &ring->slot[nm_i]; + uint64_t offset = kring->fixed_buf_offset + slot->offset; u_int len = slot->len; uint64_t paddr; - void *addr = PNMB(na, slot, &paddr); + void *addr = PNMB(kring, slot, &paddr); /* device-specific */ struct e1000_tx_desc *curr = &txr->tx_base[nic_i]; @@ -158,7 +159,8 @@ if (slot->flags & NS_BUF_CHANGED) { curr->buffer_addr = htole64(paddr); /* buffer has changed, reload map */ - netmap_reload_map(na, txr->txtag, txbuf->map, addr); + netmap_reload_map(na, txr->txtag, txbuf->map, + addr, NETMAP_BUF_SIZE(na, offset)); } slot->flags &= ~(NS_REPORT | NS_BUF_CHANGED); @@ -266,8 +268,9 @@ nic_i = netmap_idx_k2n(kring, nm_i); for (n = 0; nm_i != head; n++) { struct netmap_slot *slot = &ring->slot[nm_i]; + uint64_t offset = kring->fixed_buf_offset + slot->offset; uint64_t paddr; - void *addr = PNMB(na, slot, &paddr); + void *addr = PNMB(kring, slot, &paddr); union e1000_rx_desc_extended *curr = &rxr->rx_base[nic_i]; struct em_rxbuffer *rxbuf = &rxr->rx_buffers[nic_i]; @@ -278,7 +281,8 @@ curr->read.buffer_addr = htole64(paddr); if (slot->flags & NS_BUF_CHANGED) { /* buffer has changed, reload map */ - netmap_reload_map(na, rxr->rxtag, rxbuf->map, addr); + netmap_reload_map(na, rxr->rxtag, rxbuf->map, + addr, NETMAP_BUF_SIZE(na, offset)); slot->flags &= ~NS_BUF_CHANGED; } curr->wb.upper.status_error = 0; Modified: soc2018/sduo/head/sys/dev/netmap/if_igb_netmap.h ============================================================================== --- soc2018/sduo/head/sys/dev/netmap/if_igb_netmap.h Thu Aug 2 00:07:30 2018 (r337275) +++ soc2018/sduo/head/sys/dev/netmap/if_igb_netmap.h Thu Aug 2 10:03:51 2018 (r337276) @@ -113,9 +113,10 @@ nic_i = netmap_idx_k2n(kring, nm_i); for (n = 0; nm_i != head; n++) { struct netmap_slot *slot = &ring->slot[nm_i]; + uint64_t offset = kring->fixed_buf_offset + slot->offset; u_int len = slot->len; uint64_t paddr; - void *addr = PNMB(na, slot, &paddr); + void *addr = PNMB(kring, slot, &paddr); /* device-specific */ union e1000_adv_tx_desc *curr = @@ -129,7 +130,8 @@ if (slot->flags & NS_BUF_CHANGED) { /* buffer has changed, reload map */ - netmap_reload_map(na, txr->txtag, txbuf->map, addr); + netmap_reload_map(na, txr->txtag, txbuf->map, + addr, NETMAP_BUF_SIZE(na, offset)); } slot->flags &= ~(NS_REPORT | NS_BUF_CHANGED); @@ -246,8 +248,9 @@ nic_i = netmap_idx_k2n(kring, nm_i); for (n = 0; nm_i != head; n++) { struct netmap_slot *slot = &ring->slot[nm_i]; + uint64_t offset = kring->fixed_buf_offset + slot->offset; uint64_t paddr; - void *addr = PNMB(na, slot, &paddr); + void *addr = PNMB(kring, slot, &paddr); union e1000_adv_rx_desc *curr = &rxr->rx_base[nic_i]; struct igb_rx_buf *rxbuf = &rxr->rx_buffers[nic_i]; @@ -257,7 +260,8 @@ if (slot->flags & NS_BUF_CHANGED) { /* buffer has changed, reload map */ - netmap_reload_map(na, rxr->ptag, rxbuf->pmap, addr); + netmap_reload_map(na, rxr->ptag, rxbuf->pmap, + addr, NETMAP_BUF_SIZE(na, offset)); slot->flags &= ~NS_BUF_CHANGED; } curr->wb.upper.status_error = 0; Modified: soc2018/sduo/head/sys/dev/netmap/if_ixl_netmap.h ============================================================================== --- soc2018/sduo/head/sys/dev/netmap/if_ixl_netmap.h Thu Aug 2 00:07:30 2018 (r337275) +++ soc2018/sduo/head/sys/dev/netmap/if_ixl_netmap.h Thu Aug 2 10:03:51 2018 (r337276) @@ -214,9 +214,10 @@ for (n = 0; nm_i != head; n++) { struct netmap_slot *slot = &ring->slot[nm_i]; + uint64_t offset = kring->fixed_buf_offset + slot->offset; u_int len = slot->len; uint64_t paddr; - void *addr = PNMB(na, slot, &paddr); + void *addr = PNMB(kring, slot, &paddr); /* device-specific */ struct i40e_tx_desc *curr = &txr->base[nic_i]; @@ -233,7 +234,8 @@ if (slot->flags & NS_BUF_CHANGED) { /* buffer has changed, reload map */ - netmap_reload_map(na, txr->dma.tag, txbuf->map, addr); + netmap_reload_map(na, txr->dma.tag, txbuf->map, + addr, NETMAP_BUF_SIZE(na, offset)); } slot->flags &= ~(NS_REPORT | NS_BUF_CHANGED); @@ -376,8 +378,9 @@ nic_i = netmap_idx_k2n(kring, nm_i); for (n = 0; nm_i != head; n++) { struct netmap_slot *slot = &ring->slot[nm_i]; + uint64_t offset = kring->fixed_buf_offset + slot->offset; uint64_t paddr; - void *addr = PNMB(na, slot, &paddr); + void *addr = PNMB(kring, slot, &paddr); union i40e_32byte_rx_desc *curr = &rxr->base[nic_i]; struct ixl_rx_buf *rxbuf = &rxr->buffers[nic_i]; @@ -387,7 +390,8 @@ if (slot->flags & NS_BUF_CHANGED) { /* buffer has changed, reload map */ - netmap_reload_map(na, rxr->ptag, rxbuf->pmap, addr); + netmap_reload_map(na, rxr->ptag, rxbuf->pmap, + addr, NETMAP_BUF_SIZE(na, offset)); slot->flags &= ~NS_BUF_CHANGED; } curr->read.pkt_addr = htole64(paddr); Modified: soc2018/sduo/head/sys/dev/netmap/if_lem_netmap.h ============================================================================== --- soc2018/sduo/head/sys/dev/netmap/if_lem_netmap.h Thu Aug 2 00:07:30 2018 (r337275) +++ soc2018/sduo/head/sys/dev/netmap/if_lem_netmap.h Thu Aug 2 10:03:51 2018 (r337276) @@ -124,9 +124,10 @@ nic_i = netmap_idx_k2n(kring, nm_i); while (nm_i != head) { struct netmap_slot *slot = &ring->slot[nm_i]; + uint64_t offset = kring->fixed_buf_offset + slot->offset; u_int len = slot->len; uint64_t paddr; - void *addr = PNMB(na, slot, &paddr); + void *addr = PNMB(kring, slot, &paddr); /* device-specific */ struct e1000_tx_desc *curr = &adapter->tx_desc_base[nic_i]; @@ -140,7 +141,8 @@ if (slot->flags & NS_BUF_CHANGED) { /* buffer has changed, reload map */ curr->buffer_addr = htole64(paddr); - netmap_reload_map(na, adapter->txtag, txbuf->map, addr); + netmap_reload_map(na, adapter->txtag, txbuf->map, + addr, NETMAP_BUF_SIZE(na, offset)); } slot->flags &= ~(NS_REPORT | NS_BUF_CHANGED); @@ -258,8 +260,9 @@ nic_i = netmap_idx_k2n(kring, nm_i); for (n = 0; nm_i != head; n++) { struct netmap_slot *slot = &ring->slot[nm_i]; + uint64_t offset = kring->fixed_buf_offset + slot->offset; uint64_t paddr; - void *addr = PNMB(na, slot, &paddr); + void *addr = PNMB(kring, slot, &paddr); struct e1000_rx_desc *curr = &adapter->rx_desc_base[nic_i]; struct em_buffer *rxbuf = &adapter->rx_buffer_area[nic_i]; @@ -270,7 +273,8 @@ if (slot->flags & NS_BUF_CHANGED) { /* buffer has changed, reload map */ curr->buffer_addr = htole64(paddr); - netmap_reload_map(na, adapter->rxtag, rxbuf->map, addr); + netmap_reload_map(na, adapter->rxtag, rxbuf->map, + addr, NETMAP_BUF_SIZE(na, offset)); slot->flags &= ~NS_BUF_CHANGED; } curr->status = 0; Modified: soc2018/sduo/head/sys/dev/netmap/if_ptnet.c ============================================================================== --- soc2018/sduo/head/sys/dev/netmap/if_ptnet.c Thu Aug 2 00:07:30 2018 (r337275) +++ soc2018/sduo/head/sys/dev/netmap/if_ptnet.c Thu Aug 2 10:03:51 2018 (r337276) @@ -909,7 +909,7 @@ goto err_register; } - nm_buf_size = NETMAP_BUF_SIZE(na_dr); + nm_buf_size = NETMAP_BUF_SIZE(na_dr, 0); KASSERT(nm_buf_size > 0, ("Invalid netmap buffer size")); sc->min_tx_space = PTNET_MAX_PKT_SIZE / nm_buf_size + 2; @@ -1107,12 +1107,13 @@ ptnet_nm_config(struct netmap_adapter *na, struct nm_config_info *info) { struct ptnet_softc *sc = if_getsoftc(na->ifp); + uint64_t offset = 0; info->num_tx_rings = bus_read_4(sc->iomem, PTNET_IO_NUM_TX_RINGS); info->num_rx_rings = bus_read_4(sc->iomem, PTNET_IO_NUM_RX_RINGS); info->num_tx_descs = bus_read_4(sc->iomem, PTNET_IO_NUM_TX_SLOTS); info->num_rx_descs = bus_read_4(sc->iomem, PTNET_IO_NUM_RX_SLOTS); - info->rx_buf_maxsize = NETMAP_BUF_SIZE(na); + info->rx_buf_maxsize = NETMAP_BUF_SIZE(na, offset); device_printf(sc->dev, "txr %u, rxr %u, txd %u, rxd %u, rxbufsz %u\n", info->num_tx_rings, info->num_rx_rings, @@ -1801,7 +1802,7 @@ /* Initialize transmission state variables. */ slot = ring->slot + head; - nmbuf = NMB(na, slot); + nmbuf = NMB(kring, slot); nmbuf_bytes = 0; /* If needed, prepare the virtio-net header at the beginning @@ -1842,7 +1843,8 @@ int mlen = mf->m_len; for (;;) { - int copy = NETMAP_BUF_SIZE(na) - nmbuf_bytes; + uint64_t offset = kring->fixed_buf_offset + slot->offset; + int copy = NETMAP_BUF_SIZE(na, offset) - nmbuf_bytes; if (mlen < copy) { copy = mlen; @@ -1865,7 +1867,7 @@ KASSERT(head != ring->tail, ("Unexpectedly run out of TX space")); slot = ring->slot + head; - nmbuf = NMB(na, slot); + nmbuf = NMB(kring, slot); nmbuf_bytes = 0; } } @@ -2074,7 +2076,7 @@ /* Initialize ring state variables, possibly grabbing the * virtio-net header. */ slot = ring->slot + head; - nmbuf = NMB(na, slot); + nmbuf = NMB(kring, slot); nmbuf_len = slot->len; vh = (struct virtio_net_hdr *)nmbuf; @@ -2160,7 +2162,7 @@ } slot = ring->slot + head; - nmbuf = NMB(na, slot); + nmbuf = NMB(kring, slot); nmbuf_len = slot->len; mhead->m_pkthdr.len += nmbuf_len; } Modified: soc2018/sduo/head/sys/dev/netmap/if_re_netmap.h ============================================================================== --- soc2018/sduo/head/sys/dev/netmap/if_re_netmap.h Thu Aug 2 00:07:30 2018 (r337275) +++ soc2018/sduo/head/sys/dev/netmap/if_re_netmap.h Thu Aug 2 10:03:51 2018 (r337276) @@ -94,9 +94,10 @@ for (n = 0; nm_i != head; n++) { struct netmap_slot *slot = &ring->slot[nm_i]; + uint64_t offset = kring->fixed_buf_offset + slot->offset; u_int len = slot->len; uint64_t paddr; - void *addr = PNMB(na, slot, &paddr); + void *addr = PNMB(kring, slot, &paddr); /* device-specific */ struct rl_desc *desc = &sc->rl_ldata.rl_tx_list[nic_i]; @@ -113,7 +114,8 @@ desc->rl_bufaddr_lo = htole32(RL_ADDR_LO(paddr)); desc->rl_bufaddr_hi = htole32(RL_ADDR_HI(paddr)); netmap_reload_map(na, sc->rl_ldata.rl_tx_mtag, - txd[nic_i].tx_dmamap, addr); + txd[nic_i].tx_dmamap, addr, + NETMAP_BUF_SIZE(na, offset)); } slot->flags &= ~(NS_REPORT | NS_BUF_CHANGED); @@ -236,11 +238,12 @@ nic_i = netmap_idx_k2n(kring, nm_i); for (n = 0; nm_i != head; n++) { struct netmap_slot *slot = &ring->slot[nm_i]; + uint64_t offset = kring->fixed_buf_offset + slot->offset; uint64_t paddr; - void *addr = PNMB(na, slot, &paddr); + void *addr = PNMB(kring, slot, &paddr); struct rl_desc *desc = &sc->rl_ldata.rl_rx_list[nic_i]; - int cmd = NETMAP_BUF_SIZE(na) | RL_RDESC_CMD_OWN; + int cmd = NETMAP_BUF_SIZE(na, offset) | RL_RDESC_CMD_OWN; if (addr == NETMAP_BUF_BASE(na)) /* bad buf */ goto ring_reset; @@ -253,7 +256,8 @@ desc->rl_bufaddr_lo = htole32(RL_ADDR_LO(paddr)); desc->rl_bufaddr_hi = htole32(RL_ADDR_HI(paddr)); netmap_reload_map(na, sc->rl_ldata.rl_rx_mtag, - rxd[nic_i].rx_dmamap, addr); + rxd[nic_i].rx_dmamap, addr, + NETMAP_BUF_SIZE(na, offset)); slot->flags &= ~NS_BUF_CHANGED; } desc->rl_cmdstat = htole32(cmd); @@ -301,14 +305,17 @@ /* l points in the netmap ring, i points in the NIC ring */ for (i = 0; i < n; i++) { + struct netmap_kring *kring = na->tx_rings[0]; + uint64_t offset = kring->fixed_buf_offset; uint64_t paddr; - int l = netmap_idx_n2k(na->tx_rings[0], i); - void *addr = PNMB(na, slot + l, &paddr); + int l = netmap_idx_n2k(kring, i); + void *addr = PNMB(kring, slot + l, &paddr); desc[i].rl_bufaddr_lo = htole32(RL_ADDR_LO(paddr)); desc[i].rl_bufaddr_hi = htole32(RL_ADDR_HI(paddr)); netmap_load_map(na, sc->rl_ldata.rl_tx_mtag, - txd[i].tx_dmamap, addr); + txd[i].tx_dmamap, addr, + NETMAP_BUF_SIZE(na, offset)); } } @@ -318,6 +325,7 @@ struct netmap_adapter *na = NA(sc->rl_ifp); struct netmap_slot *slot = netmap_reset(na, NR_RX, 0, 0); struct rl_desc *desc = sc->rl_ldata.rl_rx_list; + struct netmap_kring *kring = na->rx_rings[0]; uint32_t cmdstat; uint32_t nic_i, max_avail; uint32_t const n = sc->rl_ldata.rl_rx_desc_cnt; @@ -328,21 +336,23 @@ * Do not release the slots owned by userspace, * and also keep one empty. */ - max_avail = n - 1 - nm_kr_rxspace(na->rx_rings[0]); + max_avail = n - 1 - nm_kr_rxspace(kring); for (nic_i = 0; nic_i < n; nic_i++) { + uint32_t nm_i = netmap_idx_n2k(kring, nic_i); + uint64_t offset = kring->fixed_buf_offset + (slot + nm_i)->offset; void *addr; uint64_t paddr; - uint32_t nm_i = netmap_idx_n2k(na->rx_rings[0], nic_i); - addr = PNMB(na, slot + nm_i, &paddr); + addr = PNMB(kring, slot + nm_i, &paddr); netmap_reload_map(na, sc->rl_ldata.rl_rx_mtag, - sc->rl_ldata.rl_rx_desc[nic_i].rx_dmamap, addr); + sc->rl_ldata.rl_rx_desc[nic_i].rx_dmamap, + addr, NETMAP_BUF_SIZE(na, offset)); bus_dmamap_sync(sc->rl_ldata.rl_rx_mtag, sc->rl_ldata.rl_rx_desc[nic_i].rx_dmamap, BUS_DMASYNC_PREREAD); desc[nic_i].rl_bufaddr_lo = htole32(RL_ADDR_LO(paddr)); desc[nic_i].rl_bufaddr_hi = htole32(RL_ADDR_HI(paddr)); - cmdstat = NETMAP_BUF_SIZE(na); + cmdstat = NETMAP_BUF_SIZE(na, offset); if (nic_i == n - 1) /* mark the end of ring */ cmdstat |= RL_RDESC_CMD_EOR; if (nic_i < max_avail) Modified: soc2018/sduo/head/sys/dev/netmap/if_vtnet_netmap.h ============================================================================== --- soc2018/sduo/head/sys/dev/netmap/if_vtnet_netmap.h Thu Aug 2 00:07:30 2018 (r337275) +++ soc2018/sduo/head/sys/dev/netmap/if_vtnet_netmap.h Thu Aug 2 10:03:51 2018 (r337276) @@ -140,7 +140,7 @@ struct netmap_slot *slot = &ring->slot[nm_i]; u_int len = slot->len; uint64_t paddr; - void *addr = PNMB(na, slot, &paddr); + void *addr = PNMB(kring, slot, &paddr); int err; NM_CHECK_ADDR_LEN(na, addr, len); @@ -242,8 +242,9 @@ for (n = 0; nm_i != head; n++) { static struct virtio_net_hdr_mrg_rxbuf hdr; struct netmap_slot *slot = &ring->slot[nm_i]; + uint64_t offset = kring->fixed_buf_offset + slot->offset; uint64_t paddr; - void *addr = PNMB(na, slot, &paddr); + void *addr = PNMB(kring, slot, &paddr); int err = 0; if (addr == NETMAP_BUF_BASE(na)) { /* bad buf */ @@ -254,7 +255,7 @@ slot->flags &= ~NS_BUF_CHANGED; sglist_reset(&sg); // cheap err = sglist_append(&sg, &hdr, sc->vtnet_hdr_size); - err = sglist_append_phys(&sg, paddr, NETMAP_BUF_SIZE(na)); + err = sglist_append_phys(&sg, paddr, NETMAP_BUF_SIZE(na, offset)); /* writable for the host */ err = virtqueue_enqueue(vq, rxq, &sg, 0, sg.sg_nseg); if (err < 0) { Modified: soc2018/sduo/head/sys/dev/netmap/ixgbe_netmap.h ============================================================================== --- soc2018/sduo/head/sys/dev/netmap/ixgbe_netmap.h Thu Aug 2 00:07:30 2018 (r337275) +++ soc2018/sduo/head/sys/dev/netmap/ixgbe_netmap.h Thu Aug 2 10:03:51 2018 (r337276) @@ -236,9 +236,10 @@ for (n = 0; nm_i != head; n++) { struct netmap_slot *slot = &ring->slot[nm_i]; + uint64_t offset = kring->fixed_buf_offset; u_int len = slot->len; uint64_t paddr; - void *addr = PNMB(na, slot, &paddr); + void *addr = PNMB(kring, slot, &paddr); /* device-specific */ union ixgbe_adv_tx_desc *curr = &txr->tx_base[nic_i]; @@ -255,7 +256,8 @@ if (slot->flags & NS_BUF_CHANGED) { /* buffer has changed, reload map */ - netmap_reload_map(na, txr->txtag, txbuf->map, addr); + netmap_reload_map(na, txr->txtag, txbuf->map, + addr, NETMAP_BUF_SIZE(na, offset)); } slot->flags &= ~(NS_REPORT | NS_BUF_CHANGED); @@ -437,8 +439,9 @@ nic_i = netmap_idx_k2n(kring, nm_i); for (n = 0; nm_i != head; n++) { struct netmap_slot *slot = &ring->slot[nm_i]; + uint64_t offset = kring->fixed_buf_offset + slot->offset; uint64_t paddr; - void *addr = PNMB(na, slot, &paddr); + void *addr = PNMB(kring, slot, &paddr); union ixgbe_adv_rx_desc *curr = &rxr->rx_base[nic_i]; struct ixgbe_rx_buf *rxbuf = &rxr->rx_buffers[nic_i]; @@ -448,7 +451,8 @@ if (slot->flags & NS_BUF_CHANGED) { /* buffer has changed, reload map */ - netmap_reload_map(na, rxr->ptag, rxbuf->pmap, addr); + netmap_reload_map(na, rxr->ptag, rxbuf->pmap, + addr, NETMAP_BUF_SIZE(na, offset)); slot->flags &= ~NS_BUF_CHANGED; } curr->wb.upper.status_error = 0; Modified: soc2018/sduo/head/sys/dev/netmap/netmap.c ============================================================================== --- soc2018/sduo/head/sys/dev/netmap/netmap.c Thu Aug 2 00:07:30 2018 (r337275) +++ soc2018/sduo/head/sys/dev/netmap/netmap.c Thu Aug 2 10:03:51 2018 (r337276) @@ -1163,16 +1163,17 @@ for (n = kring->nr_hwcur; n != head; n = nm_next(n, lim)) { struct mbuf *m; struct netmap_slot *slot = &kring->ring->slot[n]; + uint64_t offset = kring->fixed_buf_offset + slot->offset; if ((slot->flags & NS_FORWARD) == 0 && !force) continue; - if (slot->len < 14 || slot->len > NETMAP_BUF_SIZE(na)) { + if (slot->len < 14 || slot->len > NETMAP_BUF_SIZE(na, offset)) { RD(5, "bad pkt at %d len %d", n, slot->len); continue; } slot->flags &= ~NS_FORWARD; // XXX needed ? /* XXX TODO: adapt to the case of a multisegment packet */ - m = m_devget(NMB(na, slot), slot->len, 0, na->ifp, NULL); + m = m_devget(NMB(kring, slot), slot->len, 0, na->ifp, NULL); if (m == NULL) break; @@ -1328,10 +1329,10 @@ int len = MBUF_LEN(m); struct netmap_slot *slot = &ring->slot[nm_i]; - m_copydata(m, 0, len, NMB(na, slot)); + m_copydata(m, 0, len, NMB(kring, slot)); ND("nm %d len %d", nm_i, len); if (netmap_verbose) - D("%s", nm_dump_buf(NMB(na, slot),len, 128, NULL)); + D("%s", nm_dump_buf(NMB(kring, slot),len, 128, NULL)); slot->len = len; slot->flags = 0; @@ -1770,11 +1771,13 @@ for (i = 0; i <= lim; i++) { u_int idx = ring->slot[i].buf_idx; u_int len = ring->slot[i].len; + uint64_t offset = kring->fixed_buf_offset + ring->slot[i].offset; + if (idx < 2 || idx >= kring->na->na_lut.objtotal) { RD(5, "bad index at slot %d idx %d len %d ", i, idx, len); ring->slot[i].buf_idx = 0; ring->slot[i].len = 0; - } else if (len > NETMAP_BUF_SIZE(kring->na)) { + } else if (len > NETMAP_BUF_SIZE(kring->na, offset)) { ring->slot[i].len = 0; RD(5, "bad len at slot %d idx %d len %d", i, idx, len); } @@ -2091,7 +2094,8 @@ */ int netmap_do_regif(struct netmap_priv_d *priv, struct netmap_adapter *na, - uint32_t nr_mode, uint16_t nr_ringid, uint64_t nr_flags) + uint32_t nr_mode, uint16_t nr_ringid, uint64_t nr_flags, + uint64_t fixed_buf_offset) { struct netmap_if *nifp = NULL; int error; @@ -2128,7 +2132,7 @@ */ if (na->ifp && nm_priv_rx_enabled(priv)) { /* This netmap adapter is attached to an ifnet. */ - unsigned nbs = NETMAP_BUF_SIZE(na); + unsigned nbs = NETMAP_BUF_SIZE(na, fixed_buf_offset); unsigned mtu = nm_os_ifnet_mtu(na->ifp); ND("%s: mtu %d rx_buf_maxsize %d netmap_buf_size %d", @@ -2250,6 +2254,89 @@ return error; } +/* Checks that every every buffer, belonging to [first_ring, last_ring[ + * (tx/rx dependings on t) of the specified na, is bigger than the + * specified offset. + */ +int +check_fixed_offset(struct netmap_adapter *na, enum txrx t, u_int first_ring, + u_int last_ring, uint64_t offset) +{ + u_int i; + + for (i = first_ring; i < last_ring; ++i) { + struct netmap_kring *kring = NMR(na, t)[i]; + struct netmap_ring *ring = kring->ring; + + if (ring->nr_buf_size <= offset) { + return EINVAL; + } + } + + return 0; +} + +/* Sets the fields needed to use the specified offset for every + * buffer belonging to [first_ring, last_ring[ (tx/rx dependings on t) + * of the specified na. + */ +void +set_fixed_offset(struct netmap_adapter *na, enum txrx t, u_int first_ring, + u_int last_ring, uint64_t offset) +{ + u_int i; + + for (i = first_ring; i < last_ring; ++i) { + struct netmap_kring *kring = NMR(na, t)[i]; + struct netmap_ring *ring = kring->ring; + uint32_t buf_size; + int j; + + kring->fixed_buf_offset = offset; + ring->fixed_buf_offset = offset; + buf_size = ring->nr_buf_size - offset; + *(uint32_t *)(uintptr_t)&ring->nr_buf_size = buf_size; + for (j = 0; j < ring->num_slots; ++j) { + struct netmap_slot *slot = &ring->slot[j]; + + slot->len -= offset; + } + } +} + +/* Default callback to set fixed per-ring offsets. Applies the offset + * to every ring the request is trying to open. + */ +int +netmap_set_fixed_offset(struct netmap_adapter *na, struct netmap_priv_d *priv, + struct nmreq_opt_ring_offset *opt) +{ + uint64_t fixed_buf_offset; + enum txrx t; + int ret; + + if (opt == NULL) { + return 0; + } + + fixed_buf_offset = opt->nro_fixed_buf_offset; + for_rx_tx(t) { + ret = check_fixed_offset(na, t, priv->np_qfirst[t], + priv->np_qlast[t], fixed_buf_offset); + if (ret != 0) { + opt->nro_opt.nro_status = ret; + return ret; + } + } + + for_rx_tx(t) { + set_fixed_offset(na, t, priv->np_qfirst[t], priv->np_qlast[t], + fixed_buf_offset); + } + + opt->nro_opt.nro_status = 0; + return 0; +} /* * update kring and ring at the end of rxsync/txsync. @@ -2282,6 +2369,40 @@ static int nmreq_copyout(struct nmreq_header *, int); static int nmreq_checkoptions(struct nmreq_header *); +static int +netmap_handle_slot_offset(struct nmreq_header *hdr, struct netmap_adapter *na) +{ + struct nmreq_opt_slot_offset *off_opt; + struct nmreq_option *opt; + uint32_t mem_flag; + int error; + + if (hdr->nr_flags & NETMAP_REQ_LEGACY) { + /* Legacy requests do not support per-slot variable offsets */ + mem_flag = NETMAP_MEM_LEGACY; + opt = NULL; + } else { + opt = nmreq_findoption((struct nmreq_option *)(uintptr_t)hdr->nr_options, + NETMAP_REQ_OPT_SLOT_OFFSET); + if (opt == NULL) { + /* Default behaviour: per-slot variable offsets disabled */ + mem_flag = NETMAP_MEM_LEGACY; + } else { + /* If there's an option embedded in the request, + * do as the user says */ + off_opt = (struct nmreq_opt_slot_offset *)opt; + mem_flag = (off_opt->nro_enable == 0) ? + NETMAP_MEM_LEGACY : NETMAP_MEM_OFFSET; + } + } + + error = netmap_mem_flag_offset(na->nm_mem, mem_flag); + if (opt != NULL) { + opt->nro_status = error; + } + return error; +} + /* * ioctl(2) support for the "netmap" device. * @@ -2346,10 +2467,10 @@ /* Protect access to priv from concurrent requests. */ NMG_LOCK(); do { - u_int memflags; -#ifdef WITH_EXTMEM + struct nmreq_opt_ring_offset *off_opt = NULL; + uint64_t fixed_buf_offset; struct nmreq_option *opt; -#endif /* WITH_EXTMEM */ + u_int memflags; if (priv->np_nifp != NULL) { /* thread already registered */ error = EBUSY; @@ -2375,7 +2496,6 @@ break; } #endif /* WITH_EXTMEM */ - if (nmd == NULL && req->nr_mem_id) { /* find the allocator and get a reference */ nmd = netmap_mem_find(req->nr_mem_id); @@ -2399,8 +2519,23 @@ break; } + opt = nmreq_findoption( + (struct nmreq_option *)(uintptr_t)hdr->nr_options, + NETMAP_REQ_OPT_RING_OFFSET); + if (opt == NULL) { + fixed_buf_offset = 0; + } else if (nmreq_checkduplicate(opt) != 0) { + opt->nro_status = EINVAL; + error = opt->nro_status; + break; + } else { + off_opt = (struct nmreq_opt_ring_offset *)opt; + fixed_buf_offset = off_opt->nro_fixed_buf_offset; + } + error = netmap_do_regif(priv, na, req->nr_mode, - req->nr_ringid, req->nr_flags); + req->nr_ringid, req->nr_flags, + fixed_buf_offset); if (error) { /* reg. failed, release priv and ref */ break; } @@ -2412,6 +2547,7 @@ req->nr_tx_rings = na->num_tx_rings; req->nr_rx_slots = na->num_rx_desc; req->nr_tx_slots = na->num_tx_desc; + error = netmap_mem_get_info(na->nm_mem, &req->nr_memsize, &memflags, &req->nr_mem_id); if (error) { @@ -2437,8 +2573,29 @@ } req->nr_offset = netmap_mem_if_offset(na->nm_mem, nifp); + /* Fixed per-ring offset handling */ + if (na->nm_set_fixed_offset != NULL) { + nm_prinf("netmap: setting fixed offset for %s\n", + na->name); + error = na->nm_set_fixed_offset(na, priv, off_opt); + if (error != 0) { + nm_prinf("netmap: error while setting " + "fixed offset\n"); + netmap_do_unregif(priv); + break; + } + } + + error = netmap_handle_slot_offset(hdr, na); + if (error != 0) { + nm_prinf("netmap: failed to set slot offsets\n"); + netmap_do_unregif(priv); + break; + } + error = nmreq_checkoptions(hdr); if (error) { + nm_prinf("netmap: an option failed\n"); netmap_do_unregif(priv); break; } @@ -2558,7 +2715,8 @@ (struct netmap_vp_adapter *)na; na->virt_hdr_len = req->nr_hdr_len; if (na->virt_hdr_len) { - vpna->mfs = NETMAP_BUF_SIZE(na); + /* TODO: Retrieve biggest offset from krings? */ + vpna->mfs = NETMAP_BUF_SIZE(na, 0); } D("Using vnet_hdr_len %d for %p", na->virt_hdr_len, na); netmap_adapter_put(na); @@ -2761,6 +2919,12 @@ rv = sizeof(struct nmreq_opt_extmem); break; #endif /* WITH_EXTMEM */ + case NETMAP_REQ_OPT_RING_OFFSET: + rv = sizeof(struct nmreq_opt_ring_offset); + break; + case NETMAP_REQ_OPT_SLOT_OFFSET: + rv = sizeof(struct nmreq_opt_slot_offset); + break; } /* subtract the common header */ return rv - sizeof(struct nmreq_option); @@ -3602,6 +3766,7 @@ u_int len = MBUF_LEN(m); u_int error = ENOBUFS; unsigned int txr; + uint64_t offset; struct mbq *q; int busy; u_int i; @@ -3611,6 +3776,7 @@ i = i % na->num_host_rx_rings; } kring = NMR(na, NR_RX)[nma_get_nrings(na, NR_RX) + i]; + offset = kring->fixed_buf_offset; // XXX [Linux] we do not need this lock // if we follow the down/configure/up protocol -gl @@ -3635,9 +3801,9 @@ q = &kring->rx_queue; // XXX reconsider long packets if we handle fragments - if (len > NETMAP_BUF_SIZE(na)) { /* too long for us */ + if (len > NETMAP_BUF_SIZE(na, offset)) { /* too long for us */ D("%s from_host, drop packet size %d > %d", na->name, - len, NETMAP_BUF_SIZE(na)); + len, NETMAP_BUF_SIZE(na, offset)); goto done; } Modified: soc2018/sduo/head/sys/dev/netmap/netmap_bdg.c ============================================================================== --- soc2018/sduo/head/sys/dev/netmap/netmap_bdg.c Thu Aug 2 00:07:30 2018 (r337275) +++ soc2018/sduo/head/sys/dev/netmap/netmap_bdg.c Thu Aug 2 10:03:51 2018 (r337276) @@ -1130,7 +1130,6 @@ static int netmap_vp_rxsync_locked(struct netmap_kring *kring, int flags) { - struct netmap_adapter *na = kring->na; struct netmap_ring *ring = kring->ring; u_int nm_i, lim = kring->nkr_num_slots - 1; u_int head = kring->rhead; @@ -1151,7 +1150,7 @@ /* consistency check, but nothing really important here */ for (n = 0; likely(nm_i != head); n++) { struct netmap_slot *slot = &ring->slot[nm_i]; - void *addr = NMB(na, slot); + void *addr = NMB(kring, slot); if (addr == NETMAP_BUF_BASE(kring->na)) { /* bad buf */ D("bad buffer index %d, ignore ?", @@ -1436,10 +1435,6 @@ NMR(na, t)[i]->ring = NULL; } } - /* reset the number of host rings to default */ - for_rx_tx(t) { - nma_set_host_nrings(hwna, t, 1); - } } @@ -1658,7 +1653,7 @@ return ENOMEM; npriv->np_ifp = na->ifp; /* let the priv destructor release the ref */ error = netmap_do_regif(npriv, na, req->reg.nr_mode, - req->reg.nr_ringid, req->reg.nr_flags); + req->reg.nr_ringid, req->reg.nr_flags, 0); if (error) { netmap_priv_delete(npriv); return error; @@ -1726,24 +1721,12 @@ na->na_flags |= NAF_SW_ONLY; na->na_flags |= NAF_HOST_RINGS; hostna = &bna->host.up; - - /* limit the number of host rings to that of hw */ - nm_bound_var(&hostna->num_tx_rings, 1, 1, - nma_get_nrings(hwna, NR_TX), NULL); - nm_bound_var(&hostna->num_rx_rings, 1, 1, - nma_get_nrings(hwna, NR_RX), NULL); - snprintf(hostna->name, sizeof(hostna->name), "%s^", na->name); hostna->ifp = hwna->ifp; for_rx_tx(t) { enum txrx r = nm_txrx_swap(t); - u_int nr = nma_get_nrings(hostna, t); - - nma_set_nrings(hostna, t, nr); - nma_set_host_nrings(na, t, nr); - if (nma_get_host_nrings(hwna, t) < nr) { - nma_set_host_nrings(hwna, t, nr); - } + nma_set_nrings(hostna, t, 1); + nma_set_host_nrings(na, t, 1); nma_set_ndesc(hostna, t, nma_get_ndesc(hwna, r)); } // hostna->nm_txsync = netmap_bwrap_host_txsync; Modified: soc2018/sduo/head/sys/dev/netmap/netmap_generic.c ============================================================================== --- soc2018/sduo/head/sys/dev/netmap/netmap_generic.c Thu Aug 2 00:07:30 2018 (r337275) +++ soc2018/sduo/head/sys/dev/netmap/netmap_generic.c Thu Aug 2 10:03:51 2018 (r337276) @@ -708,8 +708,9 @@ while (nm_i != head) { struct netmap_slot *slot = &ring->slot[nm_i]; + uint64_t offset = kring->fixed_buf_offset + slot->offset; u_int len = slot->len; - void *addr = NMB(na, slot); + void *addr = NMB(kring, slot); /* device-specific */ struct mbuf *m; int tx_ret; @@ -721,7 +722,7 @@ m = kring->tx_pool[nm_i]; if (unlikely(m == NULL)) { kring->tx_pool[nm_i] = m = - nm_os_get_mbuf(ifp, NETMAP_BUF_SIZE(na)); + nm_os_get_mbuf(ifp, NETMAP_BUF_SIZE(na, offset)); if (m == NULL) { RD(2, "Failed to replenish mbuf"); /* Here we could schedule a timer which @@ -835,12 +836,14 @@ struct netmap_kring *kring; u_int work_done; u_int r = MBUF_RXQ(m); /* receive ring number */ + uint64_t offset; if (r >= na->num_rx_rings) { r = r % na->num_rx_rings; } kring = na->rx_rings[r]; + offset = kring->fixed_buf_offset; if (kring->nr_mode == NKR_NETMAP_OFF) { /* We must not intercept this mbuf. */ @@ -848,7 +851,7 @@ } /* limit the size of the queue */ - if (unlikely(!gna->rxsg && MBUF_LEN(m) > NETMAP_BUF_SIZE(na))) { + if (unlikely(!gna->rxsg && MBUF_LEN(m) > NETMAP_BUF_SIZE(na, offset))) { /* This may happen when GRO/LRO features are enabled for * the NIC driver when the generic adapter does not * support RX scatter-gather. */ @@ -892,6 +895,7 @@ { struct netmap_ring *ring = kring->ring; struct netmap_adapter *na = kring->na; + uint64_t offset = kring->fixed_buf_offset; u_int nm_i; /* index into the netmap ring */ //j, u_int n; u_int const lim = kring->nkr_num_slots - 1; @@ -899,7 +903,7 @@ int force_update = (flags & NAF_FORCE_READ) || kring->nr_kflags & NKR_PENDINTR; /* Adapter-specific variables. */ - u_int nm_buf_len = NETMAP_BUF_SIZE(na); + u_int nm_buf_len = NETMAP_BUF_SIZE(na, offset); struct mbq tmpq; struct mbuf *m; int avail; /* in bytes */ @@ -1000,7 +1004,7 @@ } do { - nmaddr = NMB(na, &ring->slot[nm_i]); + nmaddr = NMB(kring, &ring->slot[nm_i]); /* We only check the address here on generic rx rings. */ if (nmaddr == NETMAP_BUF_BASE(na)) { /* Bad buffer */ m_freem(m); @@ -1117,11 +1121,11 @@ na->ifp = ifp; na->num_tx_desc = num_tx_desc; na->num_rx_desc = num_rx_desc; - na->rx_buf_maxsize = 32768; na->nm_register = &generic_netmap_register; na->nm_txsync = &generic_netmap_txsync; na->nm_rxsync = &generic_netmap_rxsync; na->nm_dtor = &generic_netmap_dtor; + na->nm_set_fixed_offset = &netmap_set_fixed_offset; /* when using generic, NAF_NETMAP_ON is set so we force * NAF_SKIP_INTR to use the regular interrupt handler */ Modified: soc2018/sduo/head/sys/dev/netmap/netmap_kern.h ============================================================================== --- soc2018/sduo/head/sys/dev/netmap/netmap_kern.h Thu Aug 2 00:07:30 2018 (r337275) +++ soc2018/sduo/head/sys/dev/netmap/netmap_kern.h Thu Aug 2 10:03:51 2018 (r337276) @@ -538,6 +538,8 @@ int (*mon_notify)(struct netmap_kring *kring, int flags); #endif + /* Fixed per-ring offset, set during the first regif. */ + uint64_t fixed_buf_offset; } #ifdef _WIN32 __declspec(align(64)); @@ -676,30 +678,32 @@ */ NM_OS_MAGIC magic; uint32_t na_flags; /* enabled, and other flags */ -#define NAF_SKIP_INTR 1 /* use the regular interrupt handler. +#define NAF_SKIP_INTR 1 /* use the regular interrupt handler. * useful during initialization */ -#define NAF_SW_ONLY 2 /* forward packets only to sw adapter */ -#define NAF_BDG_MAYSLEEP 4 /* the bridge is allowed to sleep when +#define NAF_SW_ONLY 2 /* forward packets only to sw adapter */ +#define NAF_BDG_MAYSLEEP 4 /* the bridge is allowed to sleep when * forwarding packets coming from this * interface */ -#define NAF_MEM_OWNER 8 /* the adapter uses its own memory area +#define NAF_MEM_OWNER 8 /* the adapter uses its own memory area * that cannot be changed */ -#define NAF_NATIVE 16 /* the adapter is native. +#define NAF_NATIVE 16 /* the adapter is native. * Virtual ports (non persistent vale ports, * pipes, monitors...) should never use * this flag. */ -#define NAF_NETMAP_ON 32 /* netmap is active (either native or +#define NAF_NETMAP_ON 32 /* netmap is active (either native or * emulated). Where possible (e.g. FreeBSD) * IFCAP_NETMAP also mirrors this flag. */ -#define NAF_HOST_RINGS 64 /* the adapter supports the host rings */ -#define NAF_FORCE_NATIVE 128 /* the adapter is always NATIVE */ +#define NAF_HOST_RINGS 64 /* the adapter supports the host rings */ +#define NAF_FORCE_NATIVE 128 /* the adapter is always NATIVE */ #define NAF_PTNETMAP_HOST 256 /* the adapter supports ptnetmap in the host */ -#define NAF_MOREFRAG 512 /* the adapter supports NS_MOREFRAG */ +#define NAF_MOREFRAG 512 /* the adapter supports NS_MOREFRAG */ +#define NAF_LEGACY_OPEN (1U<<10) +#define NAF_OFFSET_OPEN (1U<<11) #define NAF_ZOMBIE (1U<<30) /* the nic driver has been unloaded */ #define NAF_BUSY (1U<<31) /* the adapter is used internally and * cannot be registered from userspace @@ -862,6 +866,10 @@ #ifdef WITH_MONITOR unsigned long monitor_id; /* debugging */ #endif + + /* Called during a NETMAP_REQ_REGISTER to handle fixed offset */ + int (*nm_set_fixed_offset)(struct netmap_adapter *, + struct netmap_priv_d *, struct nmreq_opt_ring_offset *); }; static __inline u_int @@ -1364,16 +1372,19 @@ /* check/fix address and len in tx rings */ #if 1 /* debug version */ #define NM_CHECK_ADDR_LEN(_na, _a, _l) do { \ - if (_a == NETMAP_BUF_BASE(_na) || _l > NETMAP_BUF_SIZE(_na)) { \ + uint64_t offset = kring->fixed_buf_offset; \ + if (_a == NETMAP_BUF_BASE(_na) || \ + _l > NETMAP_BUF_SIZE(_na, offset)) { \ RD(5, "bad addr/len ring %d slot %d idx %d len %d", \ kring->ring_id, nm_i, slot->buf_idx, len); \ - if (_l > NETMAP_BUF_SIZE(_na)) \ - _l = NETMAP_BUF_SIZE(_na); \ + if (_l > NETMAP_BUF_SIZE(_na, offset)) \ + _l = NETMAP_BUF_SIZE(_na, offset); \ *** DIFF OUTPUT TRUNCATED AT 1000 LINES ***