From owner-freebsd-net@FreeBSD.ORG Fri Mar 21 15:01:09 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8570275C for ; Fri, 21 Mar 2014 15:01:09 +0000 (UTC) Received: from mail-qg0-x22a.google.com (mail-qg0-x22a.google.com [IPv6:2607:f8b0:400d:c04::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4106A6D2 for ; Fri, 21 Mar 2014 15:01:09 +0000 (UTC) Received: by mail-qg0-f42.google.com with SMTP id q107so7386975qgd.1 for ; Fri, 21 Mar 2014 08:01:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=pDdGRsGOdTRlmNfUPp4mQDxh8LGQyj9jVSYzW4p3VrA=; b=nRnmVeVNdVc4ZolccF18hzo9N1FvY9KkpzVUi89fYSULL+VRDIrUxBjIEvd0cHHUfZ a4kgiLDLxB+LxoZ4cziD1WkaW1h8Aaw806kDpugpuZr0yUc6hyHZQJ7PGBHYcj/ztSJo 9Cj0vC2Tf1+o5ECB4M1KTbpf76VwLe38etTbs1KoJlcS221PZc9LK6S46t2+g3mHIpvl 0znF612l1EalVRcBwsrRDjdUO1yCWc4JFJyc26lnN94n5bT9Kb+hBEZhAoUkVlk/m0Ey gq14FaiGyvA9Ls/9T8gIys3h/gqBN0y8Hc2MuihfKsaOakCQRI/BTH0Ne3+LEg/9N0Uo Wp2g== MIME-Version: 1.0 X-Received: by 10.224.163.12 with SMTP id y12mr8648917qax.25.1395414068545; Fri, 21 Mar 2014 08:01:08 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Fri, 21 Mar 2014 08:01:08 -0700 (PDT) In-Reply-To: References: <477992488.642193.1395368751685.JavaMail.root@uoguelph.ca> Date: Fri, 21 Mar 2014 12:01:08 -0300 Message-ID: Subject: Re: 9.2 ixgbe tx queue hang From: Christopher Forgeron To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , Jack Vogel , Markus Gebert X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 15:01:09 -0000 (Pardon me, for some reason my gmail is sending on my cut-n-pastes if I cr down too fast) First set of logs: Mar 21 11:07:00 SAN0 kernel: before pklen=65542 actl=65542 csum=4116 Mar 21 11:07:00 SAN0 kernel: after mbcnt=33 pklen=65542 actl=65542 Mar 21 11:07:00 SAN0 kernel: before pklen=65542 actl=65542 csum=4116 Mar 21 11:07:00 SAN0 kernel: after mbcnt=33 pklen=65542 actl=65542 Mar 21 11:07:00 SAN0 kernel: before pklen=65542 actl=65542 csum=4116 Mar 21 11:07:00 SAN0 kernel: after mbcnt=33 pklen=65542 actl=65542 Mar 21 11:07:00 SAN0 kernel: before pklen=65542 actl=65542 csum=4116 Mar 21 11:07:00 SAN0 kernel: after mbcnt=33 pklen=65542 actl=65542 Mar 21 11:07:00 SAN0 kernel: before pklen=65542 actl=65542 csum=4116 Here's a few later on. Mar 21 11:10:09 SAN0 kernel: before pklen=65538 actl=65538 csum=4116 Mar 21 11:10:09 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 Mar 21 11:10:09 SAN0 kernel: before pklen=65538 actl=65538 csum=4116 Mar 21 11:10:09 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 Mar 21 11:10:09 SAN0 kernel: before pklen=65538 actl=65538 csum=4116 Mar 21 11:10:09 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 Mar 21 11:10:09 SAN0 kernel: before pklen=65538 actl=65538 csum=4116 Mar 21 11:10:09 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 Mar 21 11:23:00 SAN0 kernel: after mbcnt=33 pklen=65546 actl=65546 Mar 21 11:23:01 SAN0 kernel: before pklen=65546 actl=65546 csum=4116 Mar 21 11:23:01 SAN0 kernel: after mbcnt=33 pklen=65546 actl=65546 Mar 21 11:23:03 SAN0 kernel: before pklen=65546 actl=65546 csum=4116 Mar 21 11:23:03 SAN0 kernel: after mbcnt=33 pklen=65546 actl=65546 Mar 21 11:23:04 SAN0 kernel: before pklen=65546 actl=65546 csum=4116 Mar 21 11:23:04 SAN0 kernel: after mbcnt=33 pklen=65546 actl=65546 Mar 21 11:41:25 SAN0 kernel: before pklen=65538 actl=65538 csum=4116 Mar 21 11:41:25 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 Mar 21 11:41:25 SAN0 kernel: before pklen=65538 actl=65538 csum=4116 Mar 21 11:41:25 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 Mar 21 11:41:25 SAN0 kernel: before pklen=65538 actl=65538 csum=4116 Mar 21 11:41:25 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 Mar 21 11:41:25 SAN0 kernel: before pklen=65538 actl=65538 csum=4116 Mar 21 11:41:25 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 Mar 21 11:41:26 SAN0 kernel: before pklen=65538 actl=65538 csum=4116 Mar 21 11:41:26 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 Mar 21 11:41:26 SAN0 kernel: before pklen=65538 actl=65538 csum=4116 Mar 21 11:41:26 SAN0 kernel: after mbcnt=33 pklen=65538 actl=65538 To be clear, I changed tp->t_tsomax to IP_MAXPACKET at ~ 777 in sys/netinet/tcp_output.c like so: if (len > IP_MAXPACKET - hdrlen) { len = IP_MAXPACKET - hdrlen; sendalot = 1; } I notice there is more that is different between 9.1 and 10 for this file: http://fxr.watson.org/fxr/diff/netinet/tcp_output.c?v=FREEBSD10;diffval=FREEBSD91;diffvar=v I'm going to attempt inserting a 9.1 tcp_output.c and see if that makes any difference. Otherwise, I wait further ideas from the list. Thanks.