From owner-freebsd-net@freebsd.org Wed Nov 23 08:33:12 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BD3EBC516E0 for ; Wed, 23 Nov 2016 08:33:12 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: from mail-oi0-x231.google.com (mail-oi0-x231.google.com [IPv6:2607:f8b0:4003:c06::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 790439A for ; Wed, 23 Nov 2016 08:33:12 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: by mail-oi0-x231.google.com with SMTP id v84so6748785oie.3 for ; Wed, 23 Nov 2016 00:33:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=uN9vJDaYnyDUFuqKCtkA6JxgK/Ve10yi5qlvdzVIP7U=; b=0OKZIGKIZkNgT2Z6+H5W9xIVIbX4lW8B5qdh+yt8qDj0fJuAuGcQBZk4hIpuhBM6qH v3I53J7/bWkATxUOWW+XDAnRfGTO7Ru6rmX7miwu6t498Fszc1xsAjFhRN3ihGRjegwx sTQLTfgOq3vll8+wp/aHpj27P7ExXlQqw1cC34sqEP3RqYzLeaD+5BV2hA5c9hL3MfyE eZSIWVrt9FDcX3FomLhb/8tZYikolcFjEAO9vLekJ1On6aHTeeRVYF9McL7rtkiPdC9R YR9kZuBSb7fsWLB+Z2Du/liqK9LaX5G1gBJSx0Wdfp82G3seNMyQNsoqgkg2Y1krQU0q 2/Vg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=uN9vJDaYnyDUFuqKCtkA6JxgK/Ve10yi5qlvdzVIP7U=; b=SX9ZQEUh53a5xu2WUTvptRAN5jmYhWeDebMTGc8et/KhXnoSmJcmQKlMaavcBJT0AE UrH8JYpCwx8rYbZdZlik4iTfsFJPPbDn4tyueWuOuQeoAMSVMW6INbRs0GLD7hk94dFI 8H7iiM4vu2zFv1ii9p8UgflYlTIKEWoIu9dbMu9HUE+VCIgcBhqSqOnGji6HTdXtRn01 1t/gmIWQ+uy8MHaVtYIC7JohPqaFUVmHiomQdxjchoF/66M5oC4ghETqnBZHuc8RBiNN OlRzyKWqUC3Glcw/Ze12pV41J+3mTK9So6kG6aX31cYWaWIY860I8HtIH7gqyGY5G7n0 xucw== X-Gm-Message-State: AKaTC01rkiNex/GEfegg2jSHyCum6hsMHbuN4y/aN+StkpDe39hHoO4/EzcAEpQmOwqN0mVZKRpIb7NVKpYjgA== X-Received: by 10.157.20.235 with SMTP id r40mr1031224otr.143.1479889991563; Wed, 23 Nov 2016 00:33:11 -0800 (PST) MIME-Version: 1.0 Received: by 10.157.45.134 with HTTP; Wed, 23 Nov 2016 00:33:11 -0800 (PST) In-Reply-To: References: From: Vincenzo Maffione Date: Wed, 23 Nov 2016 09:33:11 +0100 Message-ID: Subject: Re: Looking for some help with netmap/bhyve To: Steven Crangle Cc: FreeBSD Net Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Nov 2016 08:33:12 -0000 2016-11-22 11:31 GMT+01:00 Steven Crangle : > > Hi, > > > I've recently been trying to boot up several bhyves so that I can test > netmap communication between instances. The problem is, no matter what > configuration I try, the guest vm running in bhyve completely hangs and > becomes unusable as soon as a packet hits the netmap interface. When > testing with pkt-gen, the TX side successfully starts sending packets, but > the RX side will reliable freeze with the only option being killing the > bhyve process. > > > The bhyve command used for the above test was: > > > bhyve -c 1 -s 0,hostbridge -s 1,lpc -s 2,virtio-blk,/dev/zvol/zroot/viper1vol > -s 3,virtio-net,tap0,mac=00:01:23:45:67:83 -s 4,virtio-net,tap4 -l > com1,/dev/nmdm0A -A -H -P -m 6g viper1 & > bhyve -c 1 -s 0,hostbridge -s 1,lpc -s 2,virtio-blk,/dev/zvol/zroot/viper2vol > -s 3,virtio-net,tap1,mac=00:01:23:45:67:84 -s 4,virtio-net,tap5 -l > com1,/dev/nmdm1A -A -H -P -m 6g viper2 > > For this test the host OS was FreeBSD-11.0-p3 and the guest OS was > FreeBSD-11.0-p3. > > After failing to get this solution working, I pulled down the source from > the following url and installed it on the host box: > > https://svnweb.freebsd.org/socsvn/soc2016/vincenzo/head/ > > I then ran the following commands to try and bring up the machines using > the ptnetmap interface (the guest still running 11.0p3) : > > bhyve -c 1 -s 0,hostbridge -s 1,lpc -s 1:1,virtio-blk,/dev/zvol/zroot/viper1vol > -s 2:0,virtio-net,tap0,mac=00:01:23:45:67:83 -s 2:1,ptnetmap-memdev -s > 2:2,ptnet,vale0:0 -l com1,/dev/nmdm0A -A -H -P -m 6g viper1 & > bhyve -c 1 -s 0,hostbridge -s 1,lpc -s 1:1,virtio-blk,/dev/zvol/zroot/viper2vol > -s 2:0,virtio-net,tap1,mac=00:01:23:45:67:84 -s 2:1,ptnetmap-memdev -s > 2:2,ptnet,vale0:1 -l com1,/dev/nmdm1A -A -H -P -m 6g viper2 > > With the above commands the vm's fail to boot with the following message: > > ptnet_init: failed to get ptnetmap > > Output in /var/log/messages seems to just show the ptnetmap driver > allocating one RX/TX ring for each vm, while bringing the device up, the > device then goes down and the above error is seen in the console. > > Is there something I'm doing wrong with regards to running netmap or > ptnetmap within a bhyve? Any pointers in the right direction will be much > appreciated! > > Kind Regards > > Steven > > Hi Steven, The code you are looking at is the final code released by my gsoc 2016 project at the end of August 2016. However, I've been working on that for a while after the gsoc, so that code is not updated anymore. My modification basically involves two subsystems: netmap and bhyve. The updates to netmap are already available in HEAD. The updates to bhyve are not yet upstream, as we are in the process to review that with the bhyve maintainers. Anyway, you need two sources to get the latest code: 1) https://github.com/luigirizzo/netmap for the latest netmap code, that you could also compile as an external kernel module. 2) https://github.com/vmaffione/freebsd to get the updates to bhyve (and vmm.ko). You can use this as the host system. There are two branches here: ptnet-10.3, and ptnet-head. My original code was developed under FreeBSD 10.3, so a first possibility is to try this as the host system (using the ptnet-10.3 branch). The other branch (ptnet-head) contains a porting of the work to a recent version of HEAD. I would be very glad if you could test the code also on FreeBSD 11. We support two combinations of bhyve networking with netmap: (A) virtio-net + netmap: that is something like "-s 2:0,virtio-net,vale0:0" in the bhyve command line. (B) ptnet + ptnetmap: that is something like "-s 2:1,ptnetmap-memdev -s 2:2,ptnet,vale0:0" in the bhyve command line. so you may want to try A first (netmap backend in user-space, slower) and then B (netmap backend in kernel-space, faster). Sorry about the confusion on the code repositories, I'll try also to update the wiki page (and/or the gsoc svn repo) to reflect these updates. It's perfectly ok for me to discuss these issues here in the ML, however for more detailed/low-level discussion and support about problems you are running into, feel free to open Github issues here https://github.com/ luigirizzo/netmap. Cheers, Vincenzo > > > > > > > > ________________________________ > > > > [Stream Logo] > > > > > > > > > Steven Crangle > > > Systems Developer | Stream Technologies | Glasgow, UK > > > [http://www.stream-technologies.com/img/phone.png] > > > +44 (0)844 800 8520 > > > | > > > [http://www.stream-technologies.com/img/mouse.png] > > > www.stream-technologies.com > > > > > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > -- Vincenzo Maffione