From owner-freebsd-net@freebsd.org Sat Oct 15 07:32:05 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D43B1C125F2; Sat, 15 Oct 2016 07:32:05 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: from mail-oi0-x22b.google.com (mail-oi0-x22b.google.com [IPv6:2607:f8b0:4003:c06::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 986F8928; Sat, 15 Oct 2016 07:32:05 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: by mail-oi0-x22b.google.com with SMTP id y2so161956221oie.0; Sat, 15 Oct 2016 00:32:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=75ToIbowHPkVtbCDF6Fe5mSLlbDBvtVurs9POYokY34=; b=yI257TKxuK8hKhI71LR3jZc5MzOITSkrKOWV43GVJinsrLdRkzNGwvmNuQhm+GxyTw a3qtMtCdA4eIzmTxfcQqTcp0JiufmRCI1S7o24HwlfzVGVOu0l+JbPc3F9/nub0JJEE+ AmpIjqrN0dR/wvkZ1wXGOG1EhyMVy1bp4nkoWonsdQkx/2anK6ARUzWk07EFo/MB9qdc AzfNK9RFr0VIC1g3XH5vy6xjULe7NYfOjW5gzNlRX5RE+ZEg+6eW46qvWf1EfsBTkJw/ 4nem0QOrmPgSnXXUJCAZ0bQ5ilRXssQ8zCeXiZMharu1s/2qSbI3MPYZ3x76RMQ4wEPD hmQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=75ToIbowHPkVtbCDF6Fe5mSLlbDBvtVurs9POYokY34=; b=MarY9jWA6eC02WUDZNssrQzKfxenUTD8UeTtbG/Q76uaPdZypX4eudXXjqQlLT1Z+d wwxmxK72TzcajDizhGtaC6FJ+jnSv+S13KCpx2GVxTW4Paun/MF8Vl5IYT8b72RNmWSK IPDTco8tvrJws+KyuKn+Bsuduc7C1UizSeDci6+X8cMh7y4gp9kJPjX6lOK9AleK4ZeA jrWGDQWjHI5t+NWD+fjJvjINW+zXX5RiLr2H++Hfx9dstZZFF1HYaTxTCsZGmhDGhgON hqu33l+dUfMWhiHXSht3cWWlxuPXjkX0d88p8o3APro0M+UClq2OydgLdajIG8MypJKd vhCA== X-Gm-Message-State: AA6/9RkEo8zjtff9osfDR88Z3tWashqJSdxjDFs919f8opQ0AjvjM6I5Cg8aU1LcNZo2Tdry+NEMoqQMv5zJfg== X-Received: by 10.157.47.232 with SMTP id b37mr8503831otd.67.1476516724794; Sat, 15 Oct 2016 00:32:04 -0700 (PDT) MIME-Version: 1.0 Received: by 10.157.61.52 with HTTP; Sat, 15 Oct 2016 00:32:04 -0700 (PDT) In-Reply-To: <5800DFB9.5030102@omnilan.de> References: <58009EB4.30708@omnilan.de> <5800DFB9.5030102@omnilan.de> From: Vincenzo Maffione Date: Sat, 15 Oct 2016 09:32:04 +0200 Message-ID: Subject: Re: vale-ctl(-8), ifconfig(8), SIOCAIFADDR: Invalid argument [utilizing netmap(4) providing virtual switches+interfaces to BHyVe] To: Harry Schmalzbauer Cc: FreeBSD Net , FreeBSD Stable , Luigi Rizzo Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Oct 2016 07:32:05 -0000 2016-10-14 15:38 GMT+02:00 Harry Schmalzbauer : > Bez=C3=BCglich Vincenzo Maffione's Nachricht vom 14.10.2016 15:08 (localt= ime): > > Hi, > > > > Thanks for your feedback. > > > =E2=80=A6 > >> Accidentally I found out that 'vale-ctl -n testif0' creates a artifici= al > >> interface, which is reported by ifconfig(8): > >> testif0: flags=3D8801 metric 0 mtu 1500 > >> options=3D80000 > >> ether 00:be:eb:8d:f8:00 > >> nd6 options=3D21 > >> > >> But I can't assign a IP address: 'ifconfig testif0 203.0.113.1/24' > >> ifconfig: ioctl (SIOCAIFADDR): Invalid argument > >> > >> I guess couldn't geti the picture of the netmap(4) world yet. > >> Probably, testif0 is available only in netmap(4) world, not in "host > >> world". > >> I'm assuming, because I found vale-ctl(-8)s "-h" switch. > >> > > > > Yes, those are the "persistent" VALE ports. They are a recent feature, > and > > probably you don't need to use them if you are going to play with Virtu= al > > Machines and jails (see below). > > Hello Vincenzo, > > thank you very much for your help!!! > > > =E2=80=A6 > >> Now my question: > >> > >> How can I plug a jail's or vmm's artificial interface to a VALE virtua= l > >> switch, bridging frames to real-world via physical interfaces? > >> (the latter part should work with vale-ctl -h vale0:em1, but what > >> interface to use for jail(8) vnet.interface and how to create/attach?) > >> > > > > If you use bhyve/vmm, you can attach the VM TAP interface to the VALE > > switch, as you would do for "em1". Regarding jails, I don't know exactl= y > > how networking works there, but I guess epair(4) interface (or similar) > are > > used. If this is the case, then you would have one end of the epair onl= y > > visible in the jail, and the other end only visible in the "host"; then > you > > I'm familar with epair(4), but not with tap(4). > I don't understand the man page for tap, perhaps I should read pty(4)=E2= =80=A6 > But I guess I don't have to know the details of tap(4), since you > confirmed that it can be connected to VALE. > It's not necessary to understand the details. However, a TAP device is conceptually similar to the two ends of an epair, with the difference that in the TAP a network interface (e.g. tap0) is conecptually "connected" back-to-back to a file descriptor. The file descriptor is written/read by the hypervisor (to inject/intercept packets to/from the network stack), while the tap0 interface can be attached to if_bridge. > > So one could summarize: > VALE (as part of netmap(4)) can act as a if_bridge(4) replacement in > FreeBSD-10/11, keeping everything else involved untouched. > Please correct me if I'm wrong. > For simple cases yes. if_bridge may have features that are not supported by netmap (i.e. configure ports as VLAN access ports). Moreover, if_bridge has a interface (br0), whereas VALE bridges doesn't. > > > > could attach the host end to a VALE switch again with "vale-ctl -a". > > Unfortunately, the performance you would get in any case is not great, > > because TAP and epair interface do not have netmap "native support". > > Moreover, when using bhyve, you have to pay the cost of the emulation o= f > > the vtnet device, since each packet passes through this device (other > than > > passing across netmap). > > I understand, thanks. > In fact, I expected that at first hand, but have had some oddities with > if_bridge(4) some years ago, so I thought I'd better try something new ;-= ) > Can I expect any resource savings over if_bridge(4)? I guess if so, the > ammount isn't relevant considering the whole bhyve scenarium. > I don't think you can expect resource savings in terms of memory, since netmap is trading off memory for performance, using pre-allocation rather than dynamic allocation (of each packet). I don't know what happens in terms of CPU saving, but using TAPs, epair and vtnet makes the system as a whole not really efficient (as if_bridge), so you may end up with nothing. > > > > However, consider the following: a consistent netmap update is going to > > happen in FreeBSD-CURRENT, in short. This is going to align the netmap > code > > which is now in FreeBSD to the code on the official github repository ( > > https://github.com/luigirizzo/netmap). Among the new features, there is > a > > new solution for bhyve networking, which will let you attach your bhyve > VMs > > directly to a VALE switch, without paying additional overheads related = to > > TAPs, epairs, and vtnet emulation. You can find additional information, > > code and performance numbers here: > > https://wiki.freebsd.org/SummerOfCode2016/PtnetDriverAndDeviceModel. > > Thanks for that hint! > I guess it's about ptnetmap(4)? I read papers but haven't considered it > could be production-ready for FreeBSD in the near future. > It's extremely interesting and I'd love to be eraly adopter, but my > (ESXi) setups are currently doing well and I don't have spare time or > any business project to try out=E2=80=A6 :-( > Yes, it's ptnetmap. However, bhyve is going to have support for VALE ports anyway (even without ptnetmap), as QEMU already does, so at least you will be able to replace TAPs with VALE ports (while still using vtnet devices for the VM). > > Is it likely that there will a MFC happen? Or will it be a exclusive > 12.0 feature? If ptnetmap will be MFCd I'll definitely give it a try > next summer and stay with 11.0 for my replacement machines for now. > Otherwise I'm unsure=E2=80=A6 > No idea, I guess this is up to the committers. First step, however, is to merge in CURRENT :) Cheers, Vincenzo > > best, > > -Harry > --=20 Vincenzo Maffione