From owner-freebsd-net@freebsd.org Thu Sep 6 08:15:21 2018 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BE8EBFF32CB for ; Thu, 6 Sep 2018 08:15:21 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 537B58C503 for ; Thu, 6 Sep 2018 08:15:21 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 15AAEFF32CA; Thu, 6 Sep 2018 08:15:21 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E7EF4FF32C9 for ; Thu, 6 Sep 2018 08:15:20 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: from mail-qt0-x229.google.com (mail-qt0-x229.google.com [IPv6:2607:f8b0:400d:c0d::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 82DA48C501; Thu, 6 Sep 2018 08:15:20 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: by mail-qt0-x229.google.com with SMTP id t39-v6so11261588qtc.8; Thu, 06 Sep 2018 01:15:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=soaEHkxXEanM9kpsmJ3aqlurS02W2x9vo/z/5KOEL50=; b=hjUtuB5NHAMCxWqsZVvtpETFwAeTdI7vJ0uUR+r6Lhrw1ZTq+Qge8hzixiv7gH6C9a M6E6UAYjCDGD6p0xXxIZpfI/syX7N9RlvmnKaTwkZlYxQVyiV48DzymlKqI0kXGZPMht 0eBxLWb9pZGZwiPgSS57gUtWKUogSg+D3biRaE5WYmfx9nR6IHznrWog6TBxnZax9brN vHE2asHGfKyQMeUs/ZUUpDV4T/ICjbeibPR284xTCYR1sda3iWl20GNh+/xHPRGjuPbg tYd5iNuBjaZtuysXWMy1a4r0GKyqtTUETXT8AeKTTr9g+/IPrlY8eEK+HxFCge/8WV8k zEUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=soaEHkxXEanM9kpsmJ3aqlurS02W2x9vo/z/5KOEL50=; b=ofQ/tfth9LGIZj4rQmtGGOcDMXFbW8vNKoFylTArWfvH5tq/dXhBoeCErEZFLozFGX HqfzirryRSk5s19jJx5SG5GJKMltmiT5E9H8M7/TUOZpJZzfG/vSig3a8pfrW6BZS50d RwBzmaJgBiTgJkodxt3Pk9Vm6O2MzdRXmcXreivjfymgFus/T1S0gOxvl2+naIMZjngx 5DQ6aL4uys+dSgLHSHmGwPZlfhaXG/N2lPXqrDazzIsaONWRCZIq/SIkG1EEhZ62Yof0 6OhO23wJzS6LBhcvXpwcS0rMzpZaAMIHIzScb2GSWCIu0p0I51dTFR3B/Kx+TAyUL8D1 paSQ== X-Gm-Message-State: APzg51CVkFKiOJieqSOzUp3OmedShSZysUHwwZa0Frgi7BqaM5UdeyXU MGI5SMYggXojMc5QnDGcs34Bj03XW1AXbJMW9Ffs/Q== X-Google-Smtp-Source: ANB0VdZuSVfHAKIsbKRI+piRRPnri1EM9OttnaO47mfNtx++4Er5ITYZC7BHNiiS5JnqSaXPuG9G+o3MWhLhnfz5kAc= X-Received: by 2002:ac8:2b01:: with SMTP id 1-v6mr1119891qtu.119.1536221719943; Thu, 06 Sep 2018 01:15:19 -0700 (PDT) MIME-Version: 1.0 References: <20180901014953.GV45503@funkthat.com> <20180901211123.GZ45503@funkthat.com> <20180904164303.68c43b6e@x23> <20180904162815.GA75530@funkthat.com> <20180905104433.08ad13b8@x23> <20180905124730.0f266c37@x23> <20180906004206.GB75530@funkthat.com> <20180906083535.36ac39ce@x23> In-Reply-To: <20180906083535.36ac39ce@x23> From: Vincenzo Maffione Date: Thu, 6 Sep 2018 10:12:57 +0200 Message-ID: Subject: Re: vale and netmap module questions To: Marko Zec Cc: jmg@funkthat.com, bz@freebsd.org, Luigi Rizzo , "freebsd-net@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Sep 2018 08:15:22 -0000 Il giorno gio 6 set 2018 alle ore 08:35 Marko Zec ha scritto: > On Wed, 5 Sep 2018 17:42:06 -0700 > John-Mark Gurney wrote: > > > Marko Zec wrote this message on Wed, Sep 05, 2018 at 12:47 +0200: > > > On Wed, 5 Sep 2018 12:36:38 +0200 > > > Vincenzo Maffione wrote: > > > > > > > Hi Marko, > > > > Thanks a lot for identifying the problem. > > > > If I understand correctly, simply adding -D VIMAGE here > > > > > https://github.com/luigirizzo/netmap/blob/master/sys/modules/netmap/Makefile#L11 > > > > would at least mitigate the issue. > > > > If you think I'm right I'll just add it. > > > > > > Right, go for it... > > > > Why not just hook up netmap to the build? > > I have no idea why it is on by default only on amd64... Perhaps due to > the lack of adopters / testers on other platforms? Perhaps the code has > some assumptions re. memory coherence model, or unaligned accesses, > which hold the water on amd64 but not necessarily elsewhere? > We used netmap on aarch64 (on Linux) with no problems. There can certainly be bugs on untested platforms, but there are no assumptions about the cache coherence model. And we do not use unaligned access, of course, which would slow down the whole thing. > > > Because if you add -DVIMAGE to the Makefile, you'll now break people > > who have kernels w/o VIMAGE.. > > No, and this is certifiable. At least in this particular case, we have > only two macro instances which set / restore curvnet prior to calling > into the network stack. If the network stack is of the non-VNET kind, > and thus doesn't care about curvnet, that won't do any harm. > Yes indeed. I just tried on FreeBSD 10.3 (with VIMAGE off) and netmap works when compiled with -DVIMAGE. > > > And the only reason to build netmap module manually is because it's not > hooked up. > > There's still a reason to retain the possibility to build netmap as > module, as this speeds up the development cycles for netmap-internal > hackers / experimenters. > > But beyond that, running netmap without native support in device > drivers is pretty much pointless. > Not really. Netmap exports software-only ports, like pipes, monitor, VALE ports, etc, that you can use with profit even without native support. The real problem, if I recall correctly, is that FreeBSD modules introduce a significant overhead every time you need to invoke a module function from the kernel core image. I guess it is a matter of reference counters and locks. So running netmap as a module leads to significantly lower performance. But it is ok for regression testing and experimentation. -- Vincenzo