From owner-svn-src-all@FreeBSD.ORG Fri Jun 11 19:17:36 2010 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CF0DD1065672; Fri, 11 Jun 2010 19:17:36 +0000 (UTC) (envelope-from ken@FreeBSD.org) Received: from svn.freebsd.org (unknown [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id BB40C8FC1C; Fri, 11 Jun 2010 19:17:36 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id o5BJHawN065640; Fri, 11 Jun 2010 19:17:36 GMT (envelope-from ken@svn.freebsd.org) Received: (from ken@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id o5BJHa93065633; Fri, 11 Jun 2010 19:17:36 GMT (envelope-from ken@svn.freebsd.org) Message-Id: <201006111917.o5BJHa93065633@svn.freebsd.org> From: "Kenneth D. Merry" Date: Fri, 11 Jun 2010 19:17:36 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-8@freebsd.org X-SVN-Group: stable-8 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r209061 - in stable/8/sys: amd64/include/xen dev/cxgb/sys dev/xen/netfront i386/include/xen kern sys X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jun 2010 19:17:36 -0000 Author: ken Date: Fri Jun 11 19:17:36 2010 New Revision: 209061 URL: http://svn.freebsd.org/changeset/base/209061 Log: MFC 199549, 199997, 204158, 207673, and 208901. Bring in a number of netfront changes: r199549 | jhb Remove commented out reference to if_watchdog and an assignment of zero to if_timer. Reviewed by: scottl r199997 | gibbs Add media ioctl support and link notifications so that devd will attempt to run dhclient on a netfront (xn) device that is setup for DHCP in /etc/rc.conf. PR: kern/136251 (fixed differently than the submitted patch) r204158 | kmacy - make printf conditional - fix witness warnings by making configuration lock a mutex r207673 | joel Switch to our preferred 2-clause BSD license. Approved by: kmacy r208901 | ken A number of netfront fixes and stability improvements: - Re-enable TSO. This was broken previously due to CSUM_TSO clearing the CSUM_TCP flag, so our checksum flags were incorrectly set going to the netback driver. That was fixed in r206844 in tcp_output.c, so we can turn TSO back on here. - Fix the way transmit slots are calculated, so that we can't overfill the ring. - Avoid sending packets with more fragments/segments than netback can handle. The Linux netback code can only handle packets of MAX_SKB_FRAGS, which turns out to be 18 on machines with 4K pages. We can easily generate packets with 32 or so fragments with TSO turned on. Right now the solution is just to drop the packets (since netback doesn't seem to handle it gracefully), but we should come up with a way to allow a driver to tell the TCP stack the maximum number of fragments it can handle in a single packet. - Fix the way the consumer is tracked in the receive path. It could get out of sync fairly easily. - Use standard Xen ring macros to make it clearer how netfront is using the rings. - Get rid of Linux-ish negative errno return values. - Added more documentation to the driver. - Refactored code to make it easier to read. - Some other minor fixes. Reviewed by: gibbs Sponsored by: Spectra Logic Approved by: re (bz) Modified: stable/8/sys/amd64/include/xen/xenfunc.h stable/8/sys/amd64/include/xen/xenvar.h stable/8/sys/dev/cxgb/sys/mvec.h stable/8/sys/dev/cxgb/sys/uipc_mvec.c stable/8/sys/dev/xen/netfront/netfront.c stable/8/sys/i386/include/xen/xenfunc.h stable/8/sys/i386/include/xen/xenvar.h stable/8/sys/kern/subr_bufring.c stable/8/sys/sys/buf_ring.h Directory Properties: stable/8/sys/ (props changed) stable/8/sys/amd64/include/xen/ (props changed) stable/8/sys/cddl/contrib/opensolaris/ (props changed) stable/8/sys/contrib/dev/acpica/ (props changed) stable/8/sys/contrib/pf/ (props changed) stable/8/sys/dev/xen/xenpci/ (props changed) stable/8/sys/geom/sched/ (props changed) Modified: stable/8/sys/amd64/include/xen/xenfunc.h ============================================================================== --- stable/8/sys/amd64/include/xen/xenfunc.h Fri Jun 11 19:03:59 2010 (r209060) +++ stable/8/sys/amd64/include/xen/xenfunc.h Fri Jun 11 19:17:36 2010 (r209061) @@ -1,6 +1,5 @@ -/* - * - * Copyright (c) 2004,2005 Kip Macy +/*- + * Copyright (c) 2004, 2005 Kip Macy * All rights reserved. * * Redistribution and use in source and binary forms, with or without @@ -11,22 +10,22 @@ * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. - * 4. The name of the author may not be used to endorse or promote products - * derived from this software without specific prior written permission. * - * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR - * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES - * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. - * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, - * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT - * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, - * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF - * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD$ */ - #ifndef _XEN_XENFUNC_H_ #define _XEN_XENFUNC_H_ Modified: stable/8/sys/amd64/include/xen/xenvar.h ============================================================================== --- stable/8/sys/amd64/include/xen/xenvar.h Fri Jun 11 19:03:59 2010 (r209060) +++ stable/8/sys/amd64/include/xen/xenvar.h Fri Jun 11 19:17:36 2010 (r209061) @@ -1,29 +1,27 @@ -/* +/*- * Copyright (c) 2008 Kip Macy * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: - * * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. - * * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * - * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR - * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES - * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. - * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, - * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT - * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, - * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF - * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. * * $FreeBSD$ */ Modified: stable/8/sys/dev/cxgb/sys/mvec.h ============================================================================== --- stable/8/sys/dev/cxgb/sys/mvec.h Fri Jun 11 19:03:59 2010 (r209060) +++ stable/8/sys/dev/cxgb/sys/mvec.h Fri Jun 11 19:17:36 2010 (r209061) @@ -1,33 +1,31 @@ -/************************************************************************** - * - * Copyright (c) 2007,2009 Kip Macy kmacy@freebsd.org +/*- + * Copyright (c) 2007, 2009 Kip Macy * All rights reserved. * * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * 1. Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * - * 2. The name of Kip Macy nor the names of other - * contributors may be used to endorse or promote products derived from - * this software without specific prior written permission. + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" - * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE - * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE - * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR - * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF - * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS - * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN - * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) - * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE - * POSSIBILITY OF SUCH DAMAGE. + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. * * $FreeBSD$ * - ***************************************************************************/ + */ #ifndef _MVEC_H_ #define _MVEC_H_ Modified: stable/8/sys/dev/cxgb/sys/uipc_mvec.c ============================================================================== --- stable/8/sys/dev/cxgb/sys/uipc_mvec.c Fri Jun 11 19:03:59 2010 (r209060) +++ stable/8/sys/dev/cxgb/sys/uipc_mvec.c Fri Jun 11 19:17:36 2010 (r209061) @@ -1,32 +1,28 @@ -/************************************************************************** - * - * Copyright (c) 2007-2008, Kip Macy kmacy@freebsd.org +/*- + * Copyright (c) 2007-2008 Kip Macy * All rights reserved. * * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * 1. Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * - * 2. The name of Kip Macy nor the names of other - * contributors may be used to endorse or promote products derived from - * this software without specific prior written permission. + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" - * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE - * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE - * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR - * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF - * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS - * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN - * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) - * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE - * POSSIBILITY OF SUCH DAMAGE. - * - * - ***************************************************************************/ + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ #include __FBSDID("$FreeBSD$"); Modified: stable/8/sys/dev/xen/netfront/netfront.c ============================================================================== --- stable/8/sys/dev/xen/netfront/netfront.c Fri Jun 11 19:03:59 2010 (r209060) +++ stable/8/sys/dev/xen/netfront/netfront.c Fri Jun 11 19:17:36 2010 (r209061) @@ -1,19 +1,27 @@ -/* - * +/*- * Copyright (c) 2004-2006 Kip Macy * All rights reserved. * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. * - * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR - * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES - * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. - * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, - * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT - * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, - * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF - * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. */ @@ -116,7 +124,16 @@ static const int MODPARM_rx_copy = 1; static const int MODPARM_rx_flip = 0; #endif -#define MAX_SKB_FRAGS (65536/PAGE_SIZE + 2) +/** + * \brief The maximum allowed data fragments in a single transmit + * request. + * + * This limit is imposed by the backend driver. We assume here that + * we are dealing with a Linux driver domain and have set our limit + * to mirror the Linux MAX_SKB_FRAGS constant. + */ +#define MAX_TX_REQ_FRAGS (65536 / PAGE_SIZE + 2) + #define RX_COPY_THRESHOLD 256 #define net_ratelimit() 0 @@ -132,6 +149,9 @@ static void xn_tick_locked(struct netfro static void xn_tick(void *); static void xn_intr(void *); +static inline int xn_count_frags(struct mbuf *m); +static int xn_assemble_tx_request(struct netfront_info *sc, + struct mbuf *m_head); static void xn_start_locked(struct ifnet *); static void xn_start(struct ifnet *); static int xn_ioctl(struct ifnet *, u_long, caddr_t); @@ -155,6 +175,9 @@ static void netif_disconnect_backend(str static int setup_device(device_t dev, struct netfront_info *info); static void end_access(int ref, void *page); +static int xn_ifmedia_upd(struct ifnet *ifp); +static void xn_ifmedia_sts(struct ifnet *ifp, struct ifmediareq *ifmr); + /* Xenolinux helper functions */ int network_connect(struct netfront_info *); @@ -163,8 +186,8 @@ static void xn_free_rx_ring(struct netfr static void xn_free_tx_ring(struct netfront_info *); static int xennet_get_responses(struct netfront_info *np, - struct netfront_rx_info *rinfo, RING_IDX rp, struct mbuf **list, - int *pages_flipped_p); + struct netfront_rx_info *rinfo, RING_IDX rp, RING_IDX *cons, + struct mbuf **list, int *pages_flipped_p); #define virt_to_mfn(x) (vtomach(x) >> PAGE_SHIFT) @@ -176,11 +199,12 @@ static int xennet_get_responses(struct n * not the other way around. The size must track the free index arrays. */ struct xn_chain_data { - struct mbuf *xn_tx_chain[NET_TX_RING_SIZE+1]; - int xn_tx_chain_cnt; - struct mbuf *xn_rx_chain[NET_RX_RING_SIZE+1]; + struct mbuf *xn_tx_chain[NET_TX_RING_SIZE+1]; + int xn_tx_chain_cnt; + struct mbuf *xn_rx_chain[NET_RX_RING_SIZE+1]; }; +#define NUM_ELEMENTS(x) (sizeof(x)/sizeof(*x)) struct net_device_stats { @@ -230,7 +254,7 @@ struct netfront_info { struct mtx tx_lock; struct mtx rx_lock; - struct sx sc_lock; + struct mtx sc_lock; u_int handle; u_int irq; @@ -240,32 +264,29 @@ struct netfront_info { /* Receive-ring batched refills. */ #define RX_MIN_TARGET 32 #define RX_MAX_TARGET NET_RX_RING_SIZE - int rx_min_target, rx_max_target, rx_target; - - /* - * {tx,rx}_skbs store outstanding skbuffs. The first entry in each - * array is an index into a chain of free entries. - */ + int rx_min_target; + int rx_max_target; + int rx_target; grant_ref_t gref_tx_head; grant_ref_t grant_tx_ref[NET_TX_RING_SIZE + 1]; grant_ref_t gref_rx_head; grant_ref_t grant_rx_ref[NET_TX_RING_SIZE + 1]; -#define TX_MAX_TARGET min(NET_RX_RING_SIZE, 256) - device_t xbdev; - int tx_ring_ref; - int rx_ring_ref; - uint8_t mac[ETHER_ADDR_LEN]; + device_t xbdev; + int tx_ring_ref; + int rx_ring_ref; + uint8_t mac[ETHER_ADDR_LEN]; struct xn_chain_data xn_cdata; /* mbufs */ - struct mbuf_head xn_rx_batch; /* head of the batch queue */ + struct mbuf_head xn_rx_batch; /* head of the batch queue */ int xn_if_flags; struct callout xn_stat_ch; - u_long rx_pfn_array[NET_RX_RING_SIZE]; - multicall_entry_t rx_mcl[NET_RX_RING_SIZE+1]; - mmu_update_t rx_mmu[NET_RX_RING_SIZE]; + u_long rx_pfn_array[NET_RX_RING_SIZE]; + multicall_entry_t rx_mcl[NET_RX_RING_SIZE+1]; + mmu_update_t rx_mmu[NET_RX_RING_SIZE]; + struct ifmedia sc_media; }; #define rx_mbufs xn_cdata.xn_rx_chain @@ -274,7 +295,7 @@ struct netfront_info { #define XN_LOCK_INIT(_sc, _name) \ mtx_init(&(_sc)->tx_lock, #_name"_tx", "network transmit lock", MTX_DEF); \ mtx_init(&(_sc)->rx_lock, #_name"_rx", "network receive lock", MTX_DEF); \ - sx_init(&(_sc)->sc_lock, #_name"_rx") + mtx_init(&(_sc)->sc_lock, #_name"_sc", "netfront softc lock", MTX_DEF) #define XN_RX_LOCK(_sc) mtx_lock(&(_sc)->rx_lock) #define XN_RX_UNLOCK(_sc) mtx_unlock(&(_sc)->rx_lock) @@ -282,15 +303,15 @@ struct netfront_info { #define XN_TX_LOCK(_sc) mtx_lock(&(_sc)->tx_lock) #define XN_TX_UNLOCK(_sc) mtx_unlock(&(_sc)->tx_lock) -#define XN_LOCK(_sc) sx_xlock(&(_sc)->sc_lock); -#define XN_UNLOCK(_sc) sx_xunlock(&(_sc)->sc_lock); +#define XN_LOCK(_sc) mtx_lock(&(_sc)->sc_lock); +#define XN_UNLOCK(_sc) mtx_unlock(&(_sc)->sc_lock); -#define XN_LOCK_ASSERT(_sc) sx_assert(&(_sc)->sc_lock, SX_LOCKED); +#define XN_LOCK_ASSERT(_sc) mtx_assert(&(_sc)->sc_lock, MA_OWNED); #define XN_RX_LOCK_ASSERT(_sc) mtx_assert(&(_sc)->rx_lock, MA_OWNED); #define XN_TX_LOCK_ASSERT(_sc) mtx_assert(&(_sc)->tx_lock, MA_OWNED); #define XN_LOCK_DESTROY(_sc) mtx_destroy(&(_sc)->rx_lock); \ mtx_destroy(&(_sc)->tx_lock); \ - sx_destroy(&(_sc)->sc_lock); + mtx_destroy(&(_sc)->sc_lock); struct netfront_rx_info { struct netif_rx_response rx; @@ -310,18 +331,22 @@ struct netfront_rx_info { */ static inline void -add_id_to_freelist(struct mbuf **list, unsigned short id) +add_id_to_freelist(struct mbuf **list, uintptr_t id) { - KASSERT(id != 0, ("add_id_to_freelist: the head item (0) must always be free.")); + KASSERT(id != 0, + ("%s: the head item (0) must always be free.", __func__)); list[id] = list[0]; - list[0] = (void *)(u_long)id; + list[0] = (struct mbuf *)id; } static inline unsigned short get_id_from_freelist(struct mbuf **list) { - u_int id = (u_int)(u_long)list[0]; - KASSERT(id != 0, ("get_id_from_freelist: the head item (0) must always remain free.")); + uintptr_t id; + + id = (uintptr_t)list[0]; + KASSERT(id != 0, + ("%s: the head item (0) must always remain free.", __func__)); list[0] = list[id]; return (id); } @@ -333,8 +358,7 @@ xennet_rxidx(RING_IDX idx) } static inline struct mbuf * -xennet_get_rx_mbuf(struct netfront_info *np, - RING_IDX ri) +xennet_get_rx_mbuf(struct netfront_info *np, RING_IDX ri) { int i = xennet_rxidx(ri); struct mbuf *m; @@ -355,9 +379,13 @@ xennet_get_rx_ref(struct netfront_info * #define IPRINTK(fmt, args...) \ printf("[XEN] " fmt, ##args) +#ifdef INVARIANTS #define WPRINTK(fmt, args...) \ printf("[XEN] " fmt, ##args) -#if 0 +#else +#define WPRINTK(fmt, args...) +#endif +#ifdef DEBUG #define DPRINTK(fmt, args...) \ printf("[XEN] %s: " fmt, __func__, ##args) #else @@ -477,25 +505,25 @@ talk_to_backend(device_t dev, struct net goto destroy_ring; } err = xenbus_printf(xbt, node, "tx-ring-ref","%u", - info->tx_ring_ref); + info->tx_ring_ref); if (err) { message = "writing tx ring-ref"; goto abort_transaction; } err = xenbus_printf(xbt, node, "rx-ring-ref","%u", - info->rx_ring_ref); + info->rx_ring_ref); if (err) { message = "writing rx ring-ref"; goto abort_transaction; } err = xenbus_printf(xbt, node, - "event-channel", "%u", irq_to_evtchn_port(info->irq)); + "event-channel", "%u", irq_to_evtchn_port(info->irq)); if (err) { message = "writing event-channel"; goto abort_transaction; } err = xenbus_printf(xbt, node, "request-rx-copy", "%u", - info->copying_receiver); + info->copying_receiver); if (err) { message = "writing request-rx-copy"; goto abort_transaction; @@ -656,9 +684,9 @@ xn_free_rx_ring(struct netfront_info *sc int i; for (i = 0; i < NET_RX_RING_SIZE; i++) { - if (sc->xn_cdata.xn_rx_chain[i] != NULL) { - m_freem(sc->xn_cdata.xn_rx_chain[i]); - sc->xn_cdata.xn_rx_chain[i] = NULL; + if (sc->xn_cdata.rx_mbufs[i] != NULL) { + m_freem(sc->rx_mbufs[i]); + sc->rx_mbufs[i] = NULL; } } @@ -675,8 +703,8 @@ xn_free_tx_ring(struct netfront_info *sc int i; for (i = 0; i < NET_TX_RING_SIZE; i++) { - if (sc->xn_cdata.xn_tx_chain[i] != NULL) { - m_freem(sc->xn_cdata.xn_tx_chain[i]); + if (sc->tx_mbufs[i] != NULL) { + m_freem(sc->tx_mbufs[i]); sc->xn_cdata.xn_tx_chain[i] = NULL; } } @@ -685,39 +713,36 @@ xn_free_tx_ring(struct netfront_info *sc #endif } -/* - * Do some brief math on the number of descriptors available to - * determine how many slots are available. - * - * Firstly - wouldn't something with RING_FREE_REQUESTS() be more applicable? - * Secondly - MAX_SKB_FRAGS is a Linux construct which may not apply here. - * Thirdly - it isn't used here anyway; the magic constant '24' is possibly - * wrong? - * The "2" is presumably to ensure there are also enough slots available for - * the ring entries used for "options" (eg, the TSO entry before a packet - * is queued); I'm not sure why its 2 and not 1. Perhaps to make sure there's - * a "free" node in the tx mbuf list (node 0) to represent the freelist? +/** + * \brief Verify that there is sufficient space in the Tx ring + * buffer for a maximally sized request to be enqueued. * - * This only figures out whether any xenbus ring descriptors are available; - * it doesn't at all reflect how many tx mbuf ring descriptors are also - * available. + * A transmit request requires a transmit descriptor for each packet + * fragment, plus up to 2 entries for "options" (e.g. TSO). */ static inline int -netfront_tx_slot_available(struct netfront_info *np) +xn_tx_slot_available(struct netfront_info *np) { - return ((np->tx.req_prod_pvt - np->tx.rsp_cons) < - (TX_MAX_TARGET - /* MAX_SKB_FRAGS */ 24 - 2)); + return (RING_FREE_REQUESTS(&np->tx) > (MAX_TX_REQ_FRAGS + 2)); } + static void netif_release_tx_bufs(struct netfront_info *np) { - struct mbuf *m; int i; for (i = 1; i <= NET_TX_RING_SIZE; i++) { - m = np->xn_cdata.xn_tx_chain[i]; + struct mbuf *m; + + m = np->tx_mbufs[i]; - if (((u_long)m) < KERNBASE) + /* + * We assume that no kernel addresses are + * less than NET_TX_RING_SIZE. Any entry + * in the table that is below this number + * must be an index from free-list tracking. + */ + if (((uintptr_t)m) <= NET_TX_RING_SIZE) continue; gnttab_grant_foreign_access_ref(np->grant_tx_ref[i], xenbus_get_otherend_id(np->xbdev), @@ -756,19 +781,25 @@ network_alloc_rx_buffers(struct netfront return; /* - * Allocate skbuffs greedily, even though we batch updates to the + * Allocate mbufs greedily, even though we batch updates to the * receive ring. This creates a less bursty demand on the memory - * allocator, so should reduce the chance of failed allocation + * allocator, and so should reduce the chance of failed allocation * requests both for ourself and for other kernel subsystems. + * + * Here we attempt to maintain rx_target buffers in flight, counting + * buffers that we have yet to process in the receive ring. */ batch_target = sc->rx_target - (req_prod - sc->rx.rsp_cons); for (i = mbufq_len(&sc->xn_rx_batch); i < batch_target; i++) { MGETHDR(m_new, M_DONTWAIT, MT_DATA); - if (m_new == NULL) + if (m_new == NULL) { + printf("%s: MGETHDR failed\n", __func__); goto no_mbuf; + } m_cljget(m_new, M_DONTWAIT, MJUMPAGESIZE); if ((m_new->m_flags & M_EXT) == 0) { + printf("%s: m_cljget failed\n", __func__); m_freem(m_new); no_mbuf: @@ -785,16 +816,29 @@ no_mbuf: mbufq_tail(&sc->xn_rx_batch, m_new); } - /* Is the batch large enough to be worthwhile? */ + /* + * If we've allocated at least half of our target number of entries, + * submit them to the backend - we have enough to make the overhead + * of submission worthwhile. Otherwise wait for more mbufs and + * request entries to become available. + */ if (i < (sc->rx_target/2)) { if (req_prod >sc->rx.sring->req_prod) goto push; return; } - /* Adjust floating fill target if we risked running out of buffers. */ - if ( ((req_prod - sc->rx.sring->rsp_prod) < (sc->rx_target / 4)) && - ((sc->rx_target *= 2) > sc->rx_max_target) ) - sc->rx_target = sc->rx_max_target; + + /* + * Double floating fill target if we risked having the backend + * run out of empty buffers for receive traffic. We define "running + * low" as having less than a fourth of our target buffers free + * at the time we refilled the queue. + */ + if ((req_prod - sc->rx.sring->rsp_prod) < (sc->rx_target / 4)) { + sc->rx_target *= 2; + if (sc->rx_target > sc->rx_max_target) + sc->rx_target = sc->rx_max_target; + } refill: for (nr_flips = i = 0; ; i++) { @@ -806,9 +850,8 @@ refill: id = xennet_rxidx(req_prod + i); - KASSERT(sc->xn_cdata.xn_rx_chain[id] == NULL, - ("non-NULL xm_rx_chain")); - sc->xn_cdata.xn_rx_chain[id] = m_new; + KASSERT(sc->rx_mbufs[id] == NULL, ("non-NULL xm_rx_chain")); + sc->rx_mbufs[id] = m_new; ref = gnttab_claim_grant_reference(&sc->gref_rx_head); KASSERT((short)ref >= 0, ("negative ref")); @@ -932,14 +975,13 @@ xn_rxeof(struct netfront_info *np) memset(extras, 0, sizeof(rinfo.extras)); m = NULL; - err = xennet_get_responses(np, &rinfo, rp, &m, + err = xennet_get_responses(np, &rinfo, rp, &i, &m, &pages_flipped); if (unlikely(err)) { if (m) mbufq_tail(&errq, m); np->stats.rx_errors++; - i = np->rx.rsp_cons; continue; } @@ -961,7 +1003,7 @@ xn_rxeof(struct netfront_info *np) np->stats.rx_bytes += m->m_pkthdr.len; mbufq_tail(&rxq, m); - np->rx.rsp_cons = ++i; + np->rx.rsp_cons = i; } if (pages_flipped) { @@ -1056,7 +1098,6 @@ xn_txeof(struct netfront_info *np) return; ifp = np->xn_ifp; - ifp->if_timer = 0; do { prod = np->tx.sring->rsp_prod; @@ -1067,9 +1108,16 @@ xn_txeof(struct netfront_info *np) if (txr->status == NETIF_RSP_NULL) continue; + if (txr->status != NETIF_RSP_OKAY) { + printf("%s: WARNING: response is %d!\n", + __func__, txr->status); + } id = txr->id; - m = np->xn_cdata.xn_tx_chain[id]; + m = np->tx_mbufs[id]; KASSERT(m != NULL, ("mbuf not found in xn_tx_chain")); + KASSERT((uintptr_t)m > NET_TX_RING_SIZE, + ("mbuf already on the free list, but we're " + "trying to free it again!")); M_ASSERTVALID(m); /* @@ -1080,10 +1128,8 @@ xn_txeof(struct netfront_info *np) ifp->if_opackets++; if (unlikely(gnttab_query_foreign_access( np->grant_tx_ref[id]) != 0)) { - printf("network_tx_buf_gc: warning " - "-- grant still in use by backend " - "domain.\n"); - goto out; + panic("grant id %u still in use by the backend", + id); } gnttab_end_foreign_access_ref( np->grant_tx_ref[id]); @@ -1091,12 +1137,9 @@ xn_txeof(struct netfront_info *np) &np->gref_tx_head, np->grant_tx_ref[id]); np->grant_tx_ref[id] = GRANT_INVALID_REF; - np->xn_cdata.xn_tx_chain[id] = NULL; - add_id_to_freelist(np->xn_cdata.xn_tx_chain, id); + np->tx_mbufs[id] = NULL; + add_id_to_freelist(np->tx_mbufs, id); np->xn_cdata.xn_tx_chain_cnt--; - if (np->xn_cdata.xn_tx_chain_cnt < 0) { - panic("netif_release_tx_bufs: tx_chain_cnt must be >= 0"); - } m_free(m); /* Only mark the queue active if we've freed up at least one slot to try */ ifp->if_drv_flags &= ~IFF_DRV_OACTIVE; @@ -1118,7 +1161,6 @@ xn_txeof(struct netfront_info *np) mb(); } while (prod != np->tx.sring->rsp_prod); - out: if (np->tx_full && ((np->tx.sring->req_prod - prod) < NET_TX_RING_SIZE)) { np->tx_full = 0; @@ -1142,7 +1184,7 @@ xn_intr(void *xsc) ifp->if_drv_flags & IFF_DRV_RUNNING)) return; #endif - if (np->tx.rsp_cons != np->tx.sring->rsp_prod) { + if (RING_HAS_UNCONSUMED_RESPONSES(&np->tx)) { XN_TX_LOCK(np); xn_txeof(np); XN_TX_UNLOCK(np); @@ -1174,10 +1216,9 @@ xennet_move_rx_slot(struct netfront_info static int xennet_get_extras(struct netfront_info *np, - struct netif_extra_info *extras, RING_IDX rp) + struct netif_extra_info *extras, RING_IDX rp, RING_IDX *cons) { struct netif_extra_info *extra; - RING_IDX cons = np->rx.rsp_cons; int err = 0; @@ -1185,17 +1226,17 @@ xennet_get_extras(struct netfront_info * struct mbuf *m; grant_ref_t ref; - if (unlikely(cons + 1 == rp)) { + if (unlikely(*cons + 1 == rp)) { #if 0 if (net_ratelimit()) WPRINTK("Missing extra info\n"); #endif - err = -EINVAL; + err = EINVAL; break; } extra = (struct netif_extra_info *) - RING_GET_RESPONSE(&np->rx, ++cons); + RING_GET_RESPONSE(&np->rx, ++(*cons)); if (unlikely(!extra->type || extra->type >= XEN_NETIF_EXTRA_TYPE_MAX)) { @@ -1204,23 +1245,22 @@ xennet_get_extras(struct netfront_info * WPRINTK("Invalid extra type: %d\n", extra->type); #endif - err = -EINVAL; + err = EINVAL; } else { memcpy(&extras[extra->type - 1], extra, sizeof(*extra)); } - m = xennet_get_rx_mbuf(np, cons); - ref = xennet_get_rx_ref(np, cons); + m = xennet_get_rx_mbuf(np, *cons); + ref = xennet_get_rx_ref(np, *cons); xennet_move_rx_slot(np, m, ref); } while (extra->flags & XEN_NETIF_EXTRA_FLAG_MORE); - np->rx.rsp_cons = cons; return err; } static int xennet_get_responses(struct netfront_info *np, - struct netfront_rx_info *rinfo, RING_IDX rp, + struct netfront_rx_info *rinfo, RING_IDX rp, RING_IDX *cons, struct mbuf **list, int *pages_flipped_p) { @@ -1229,45 +1269,48 @@ xennet_get_responses(struct netfront_inf struct multicall_entry *mcl; struct netif_rx_response *rx = &rinfo->rx; struct netif_extra_info *extras = rinfo->extras; - RING_IDX cons = np->rx.rsp_cons; struct mbuf *m, *m0, *m_prev; - grant_ref_t ref = xennet_get_rx_ref(np, cons); - int max = 5 /* MAX_SKB_FRAGS + (rx->status <= RX_COPY_THRESHOLD) */; + grant_ref_t ref = xennet_get_rx_ref(np, *cons); + RING_IDX ref_cons = *cons; + int max = 5 /* MAX_TX_REQ_FRAGS + (rx->status <= RX_COPY_THRESHOLD) */; int frags = 1; int err = 0; u_long ret; - m0 = m = m_prev = xennet_get_rx_mbuf(np, cons); + m0 = m = m_prev = xennet_get_rx_mbuf(np, *cons); if (rx->flags & NETRXF_extra_info) { - err = xennet_get_extras(np, extras, rp); - cons = np->rx.rsp_cons; + err = xennet_get_extras(np, extras, rp, cons); } if (m0 != NULL) { - m0->m_pkthdr.len = 0; - m0->m_next = NULL; + m0->m_pkthdr.len = 0; + m0->m_next = NULL; } for (;;) { u_long mfn; #if 0 - printf("rx->status=%hd rx->offset=%hu frags=%u\n", + DPRINTK("rx->status=%hd rx->offset=%hu frags=%u\n", rx->status, rx->offset, frags); #endif if (unlikely(rx->status < 0 || rx->offset + rx->status > PAGE_SIZE)) { + #if 0 if (net_ratelimit()) WPRINTK("rx->offset: %x, size: %u\n", rx->offset, rx->status); #endif xennet_move_rx_slot(np, m, ref); - err = -EINVAL; - goto next; + if (m0 == m) + m0 = NULL; + m = NULL; + err = EINVAL; + goto next_skip_queue; } /* @@ -1276,11 +1319,12 @@ xennet_get_responses(struct netfront_inf * situation to the system controller to reboot the backed. */ if (ref == GRANT_INVALID_REF) { + #if 0 if (net_ratelimit()) WPRINTK("Bad rx response id %d.\n", rx->id); #endif - err = -EINVAL; + err = EINVAL; goto next; } @@ -1289,12 +1333,10 @@ xennet_get_responses(struct netfront_inf * headroom, ... */ if (!(mfn = gnttab_end_foreign_transfer_ref(ref))) { - if (net_ratelimit()) - WPRINTK("Unfulfilled rx req " - "(id=%d, st=%d).\n", - rx->id, rx->status); + WPRINTK("Unfulfilled rx req (id=%d, st=%d).\n", + rx->id, rx->status); xennet_move_rx_slot(np, m, ref); - err = -ENOMEM; + err = ENOMEM; goto next; } @@ -1332,23 +1374,43 @@ next: m->m_data += rx->offset; m0->m_pkthdr.len += rx->status; +next_skip_queue: if (!(rx->flags & NETRXF_more_data)) break; - if (cons + frags == rp) { + if (*cons + frags == rp) { if (net_ratelimit()) WPRINTK("Need more frags\n"); - err = -ENOENT; + err = ENOENT; + printf("%s: cons %u frags %u rp %u, not enough frags\n", + __func__, *cons, frags, rp); break; } + /* + * Note that m can be NULL, if rx->status < 0 or if + * rx->offset + rx->status > PAGE_SIZE above. + */ m_prev = m; - rx = RING_GET_RESPONSE(&np->rx, cons + frags); - m = xennet_get_rx_mbuf(np, cons + frags); + rx = RING_GET_RESPONSE(&np->rx, *cons + frags); + m = xennet_get_rx_mbuf(np, *cons + frags); - m_prev->m_next = m; + /* + * m_prev == NULL can happen if rx->status < 0 or if + * rx->offset + * rx->status > PAGE_SIZE above. + */ + if (m_prev != NULL) + m_prev->m_next = m; + + /* + * m0 can be NULL if rx->status < 0 or if * rx->offset + + * rx->status > PAGE_SIZE above. + */ + if (m0 == NULL) + m0 = m; m->m_next = NULL; - ref = xennet_get_rx_ref(np, cons + frags); + ref = xennet_get_rx_ref(np, *cons + frags); + ref_cons = *cons + frags; frags++; } *list = m0; @@ -1356,11 +1418,12 @@ next: if (unlikely(frags > max)) { if (net_ratelimit()) WPRINTK("Too many frags\n"); - err = -E2BIG; + printf("%s: too many frags %d > max %d\n", __func__, frags, + max); + err = E2BIG; } - if (unlikely(err)) - np->rx.rsp_cons = cons + frags; + *cons += frags; *pages_flipped_p = pages_flipped; @@ -1389,209 +1452,228 @@ xn_tick(void *xsc) XN_RX_UNLOCK(sc); } -static void -xn_start_locked(struct ifnet *ifp) + +/** + * \brief Count the number of fragments in an mbuf chain. + * + * Surprisingly, there isn't an M* macro for this. + */ +static inline int +xn_count_frags(struct mbuf *m) { - int otherend_id; - unsigned short id; - struct mbuf *m_head, *m; - struct netfront_info *sc; - netif_tx_request_t *tx; + int nfrags; + + for (nfrags = 0; m != NULL; m = m->m_next) + nfrags++; + + return (nfrags); +} + +/** + * Given an mbuf chain, make sure we have enough room and then push + * it onto the transmit ring. + */ +static int +xn_assemble_tx_request(struct netfront_info *sc, struct mbuf *m_head) +{ + struct ifnet *ifp; + struct mbuf *m; + u_int nfrags; netif_extra_info_t *extra; - RING_IDX i; - grant_ref_t ref; - u_long mfn, tx_bytes; - int notify, nfrags; + int otherend_id; - sc = ifp->if_softc; - otherend_id = xenbus_get_otherend_id(sc->xbdev); - tx_bytes = 0; + ifp = sc->xn_ifp; - if (!netfront_carrier_ok(sc)) - return; - - for (i = sc->tx.req_prod_pvt; TRUE; i++) { *** DIFF OUTPUT TRUNCATED AT 1000 LINES ***