From owner-freebsd-net@freebsd.org Mon Jun 6 22:47:53 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9A096B6D2C9 for ; Mon, 6 Jun 2016 22:47:53 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mail-it0-x229.google.com (mail-it0-x229.google.com [IPv6:2607:f8b0:4001:c0b::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 639C71938 for ; Mon, 6 Jun 2016 22:47:53 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: by mail-it0-x229.google.com with SMTP id i65so49269817ith.1 for ; Mon, 06 Jun 2016 15:47:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=8kDc/UaJ3ilZT/DiQMajeAEPImvAdldP2WIoruwAYnc=; b=G7G72ZDk8QULUPS54cQGCi4CfH9g0WxhM5hR6SglHaqdZZrqicVosIO9Fao3b3bHtk zfO3wdIb7vQ1/iWHoFxhzxCGIVsVMyQQKaEM2fa0ObrJzs88klQBY40BfDpgXiv4J9ry hDlLKYEsX4ZbrYMdWPtc9MtSUPukX6eGMxg1LK9LZwUAzkBZyx1BMWbw9wA7idfQ7pyR wtUQGSrIVH8qIB88rpa8zGIwcWIkDtAMQ3aH6u8cPZMALFXWl1jopaGU5ttVRKYksEMc 41Il+Pyn/cPcfp4Iff9BcqR4Hykd2AXk0dJrY3ydIvMjdZ4VSNexwj4vVlXcvzgIY6rl tOJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=8kDc/UaJ3ilZT/DiQMajeAEPImvAdldP2WIoruwAYnc=; b=DowoYMWCKJdeYla8+vEdaCLsXo8HHIGEguTXaU5T+eyQmzZPZD2LYxdAYNn51FUsQE Qq+ENlgCUR3VkMCSfzpheWi6qwmXDK90eD0lWLTJCypVd3oY0SrE2ok3Af/FGMNvCPNM CyjaFRjqslEI02a/xj0rxGw920DPUhzlz/s1XAWnvqrH9VKTZf4r+JCdiruNJvbqAZ6I oAL/txj8cDWxnakEALTioTkSyzl3bnP/wNUFpgVO3221XX/IHkt8vvJUGsEnvTwLG6e2 +z75TRwYTjq7N8SkLc6WxPeLsf71a/R9RAnexJTOFJTFWTvwVNmtkQ3vmmPDTm0gt3a1 OYfw== X-Gm-Message-State: ALyK8tK87h8ydtyXdC2af0G4apmovB2NahAG0MGuwdItSbhz8HUsrRdIHdCHUPfsc4Mc7w7OwJ1pGK0QyYagfw== X-Received: by 10.107.159.84 with SMTP id i81mr24045305ioe.29.1465253272756; Mon, 06 Jun 2016 15:47:52 -0700 (PDT) MIME-Version: 1.0 Received: by 10.107.57.9 with HTTP; Mon, 6 Jun 2016 15:47:52 -0700 (PDT) In-Reply-To: References: From: Ryan Stone Date: Mon, 6 Jun 2016 18:47:52 -0400 Message-ID: Subject: Re: Is netmap jumbo frames broken in STABLE? To: Luigi Rizzo Cc: Andrew Vylegzhanin , "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Jun 2016 22:47:53 -0000 The use of mbuf clusters larger than a single page really doesn't work. The problem is that over time physical memory becomes fragmented and eventually 9K of contiguous memory can't be allocated anymore. This is why many drivers now limit themselves to page-sized clusters. On Mon, Jun 6, 2016 at 10:03 AM, Luigi Rizzo wrote: > On Mon, Jun 6, 2016 at 3:22 PM, Andrew Vylegzhanin > wrote: > > > Hello all, > > > > > > I have an application that uses netmap for capture jumbo frames. The > frames > > are fixed size and have fixed rate (for example size 5166, rate 50000 > pps). > > The frames are pure Ethernet, without IP header. > > > > > > Everything works fine in 10.0-RELEASE, 10.1-RELEASE. > > > > > > Starting from 10.3 and actual 10-STABLE I've got wrong data from netmap > > ring. It's looks like packet data broke and packet split on two parts > 4092 > > and 1070 bytes, where original size was 5166. > > > > A code ring precessing is based on macros from netmap_user.h : > > > > > > n =3D nm_ring_space(ring); > > > > for (rx =3D 0; rx < limit; rx++) { > > > > struct netmap_slot *slot =3D &ring->slot[cur]; > > > > char *p =3D NETMAP_BUF(ring, slot->buf_idx); > > > > process_payload(p, slot->len, datapx); > > > > cur =3D nm_ring_next(ring, cur); > > > > } > > > > ring->head =3D ring->cur =3D cur; > > > > > > Here is netmap sysctl's: > > > > dev.netmap.buf_num=3D81920 > > > > dev.netmap.ring_size=3D73728 > > > > dev.netmap.buf_size=3D5248 > > > > > > Hardware is Dell R720 (2x E5-2643 v2) with four Intel Ethernet 10G 2P > X520 > > Adapter. I use only one hardware queue per interface. > > > > > > BTW, may be a new version of Intel ixgbe driver (3.1.13-k) is a reason? > > > > > =E2=80=8BHi, > yes I suspect the problem may be in the new ixgbe driver, > probably it programs the hardware to limit buffer sizes to 4k > even when large MTUs are in use, > so the receiver will split > the incoming frame in two buffers, while netmap is expecting > only one. > I suggest to have a look at the ioctl handler (in the driver) > that handles the MTU setting and compare with the code > in the previous driver. > > cheers > luigi > > > > Does it make sense to try with 11-CURRENT? > > > > > > Thank you in advance. > > > > > > -- > > > > Andrew > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > > > > -- > -----------------------------------------+------------------------------- > Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione > http://www.iet.unipi.it/~luigi/ . Universita` di Pisa > TEL +39-050-2217533 . via Diotisalvi 2 > Mobile +39-338-6809875 . 56122 PISA (Italy) > -----------------------------------------+------------------------------- > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >