From owner-freebsd-net@FreeBSD.ORG Wed May 22 20:52:28 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 1B6D413B2 for ; Wed, 22 May 2013 20:52:28 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-ob0-x22d.google.com (mail-ob0-x22d.google.com [IPv6:2607:f8b0:4003:c01::22d]) by mx1.freebsd.org (Postfix) with ESMTP id DD58E122 for ; Wed, 22 May 2013 20:52:27 +0000 (UTC) Received: by mail-ob0-f173.google.com with SMTP id eh20so3042646obb.32 for ; Wed, 22 May 2013 13:52:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=HM9+QYnUoOWsni3R5m+p0Lw8xyuVRcTdFbTHWfjKknQ=; b=X79H4GqCsSP9OkrC+4q+FsTCQWBUljZiQIgIlf6cgVfQUb16vOwkGg5uggXbJekqgE 4K6v9ygv7h0gU2Zsrfv3VjwtsVhj6YpOKP38eW3NrwBkMKlLHtDjdR+XSOnfb4fxZ8OW USfRWxq8YFSgnTCIDstfLbBWAZKYF46+bxoe7MBR3FN6SKsSEj1K5jgN+vPbE+kykSiB mVcIvB1xBdVP8UH5yQpTcEihbPHDLs2uU/XHWxMbpaZKc+WRoc9ndUsG2Rpw7yH1sho4 cr5Bk4iDtgf1jFoc9y38QmDrv3JM0epiPuHXSmR7doWMf4ZN1e1BKkfk51nr8oKH6/0d oRLg== X-Received: by 10.60.79.231 with SMTP id m7mr6437384oex.105.1369255947528; Wed, 22 May 2013 13:52:27 -0700 (PDT) Received: from gloom.sandvine.com ([64.7.137.182]) by mx.google.com with ESMTPSA id jt1sm9426426oeb.5.2013.05.22.13.52.25 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 22 May 2013 13:52:26 -0700 (PDT) Sender: Mark Johnston Date: Wed, 22 May 2013 16:52:14 -0400 From: Mark Johnston To: Giorgio Emanuel Subject: Re: em2: watchdog timeout - resetting Message-ID: <20130522205214.GE4748@gloom.sandvine.com> References: MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 May 2013 20:52:28 -0000 On Wed, May 22, 2013 at 12:05:31AM -0300, Giorgio Emanuel wrote: > Hello, I have a server pfsense in bridge mode to function as transparent > FW, the problem is that once I connect the pfsense between my router core > and my core switch catalyst a few seconds begin to appear several messages > like these: > > em2: watchdog timeout - resetting > em2: watchdog timeout - resetting > em2: watchdog timeout - resetting > em2: watchdog timeout - resetting > > And the connection falls back repeatedly and indefinitely. > Searching on google did some tweaks: > > hw.em.rxd = "4096" > hw.em.txd = "4096" > hw.em.tx_int_delay = "250" > hw.em.rx_int_delay = "250" > hw.em.tx_abs_int_delay = "250" > hw.em.rx_abs_int_delay = "250" > hw.em.enable_msix = "0" > hw.em.msix_queues = "2" > hw.em.rx_process_limit = "-1" > hw.em.fc_setting = "0" > hw.em.num_queues = 1 > > I tried several variations, but also without success. > > My infrastructure: > > (7301 cisco router) ------ em1 (pfsense bridge0) em2 ------ sw core cisco > catalyst > > I tried also a fresh installation of pfsense 2.0.3 without success. > My network card is an Intel ® PRO/1000 MT Dual Port Server Adapter PCI-X on > a 32-bit PCI slot, operating bridge in 1 <-> em2. > Motherboard ASUS P8H61-V and CPU corei5, I tried disabling all onboard > devices, also without success. > I have 4 nics identical, tested one per one all without success also. > I am using the latest bios from ASUS site, I tried various combinations of > bios, also without success. > I tried to activate the device polling unsuccessfully to solve the problem. > I tried disable acpi on boot menu but the ser without acpi wont boot. > I tried disable TOE, FLOW CONTROL, no sucess. > > This is a link to a troughput 60 Mbits with multiple VLANs (more than 200) > Wisp provider. > I tested the same server same hardware but using linux (debian 6) did the > bridges, and everything worked properly!, so it is clear that this is a > problem software (in intel em driver?) > Tested on another server (P5VD2MX + Core2Duo) but with the same NIC, and > the problem occurs in the same way. > I realized that it generates a huge number of interruptions em2 over 200 > thousand. Are you using TSO on the interface? Does the problem go away if you disable it? (ifconfig em2 -tso) I ask because I had to backport the patch below to FreeBSD 8.2 at work; I was seeing tx watchdog timeouts when TSO and hardware VLAN tagging were combined in em(4). It looks like pfSense 2.0.3 is based on FreeBSD 8.1, so unless they've already backported this fix (which is a subset of r228387), you may be running into the same issue. -Mark Index: sys/dev/e1000/if_em.c =================================================================== --- sys/dev/e1000/if_em.c (revision 250913) +++ sys/dev/e1000/if_em.c (working copy) @@ -1963,6 +1963,14 @@ em_transmit_checksum_setup(txr, m_head, ip_off, ip, &txd_upper, &txd_lower); + if (m_head->m_flags & M_VLANTAG) { + /* Set the vlan id. */ + txd_upper |= + (htole16(m_head->m_pkthdr.ether_vtag) << 16); + /* Tell hardware to add tag */ + txd_lower |= htole32(E1000_TXD_CMD_VLE); + } + i = txr->next_avail_desc; /* Set up our transmit descriptors */ @@ -2020,14 +2028,6 @@ if (tso_desc) /* TSO used an extra for sentinel */ txr->tx_avail -= txd_used; - if (m_head->m_flags & M_VLANTAG) { - /* Set the vlan id. */ - ctxd->upper.fields.special = - htole16(m_head->m_pkthdr.ether_vtag); - /* Tell hardware to add tag */ - ctxd->lower.data |= htole32(E1000_TXD_CMD_VLE); - } - tx_buffer->m_head = m_head; tx_buffer_mapped->map = tx_buffer->map; tx_buffer->map = map;