From owner-svn-src-all@freebsd.org Thu Aug 11 21:13:59 2016 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7F414BB63B6; Thu, 11 Aug 2016 21:13:59 +0000 (UTC) (envelope-from smh@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5A77C1C6F; Thu, 11 Aug 2016 21:13:59 +0000 (UTC) (envelope-from smh@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id u7BLDwoc087675; Thu, 11 Aug 2016 21:13:58 GMT (envelope-from smh@FreeBSD.org) Received: (from smh@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id u7BLDw0c087673; Thu, 11 Aug 2016 21:13:58 GMT (envelope-from smh@FreeBSD.org) Message-Id: <201608112113.u7BLDw0c087673@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: smh set sender to smh@FreeBSD.org using -f From: Steven Hartland Date: Thu, 11 Aug 2016 21:13:58 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r303971 - head/sys/dev/virtio/network X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Aug 2016 21:13:59 -0000 Author: smh Date: Thu Aug 11 21:13:58 2016 New Revision: 303971 URL: https://svnweb.freebsd.org/changeset/base/303971 Log: Fix vtnet hang with max_virtqueue_pairs > VTNET_MAX_QUEUE_PAIRS Correctly limit npairs passed to vtnet_ctrl_mq_cmd. This ensures that VQ_ALLOC_INFO_INIT is called with the correct value, preventing the system from hanging when max_virtqueue_pairs > VTNET_MAX_QUEUE_PAIRS. Add new sysctl requested_vq_pairs which allow the user to configure the requested number of virtqueue pairs. The actual value will still take into account the system limits. Also missing sysctls for the current tunables so their values can be seen. PR: 207446 Reported by: Andy Carrel MFC after: 3 days Relnotes: Yes Sponsored by: Multiplay Modified: head/sys/dev/virtio/network/if_vtnet.c head/sys/dev/virtio/network/if_vtnetvar.h Modified: head/sys/dev/virtio/network/if_vtnet.c ============================================================================== --- head/sys/dev/virtio/network/if_vtnet.c Thu Aug 11 20:48:03 2016 (r303970) +++ head/sys/dev/virtio/network/if_vtnet.c Thu Aug 11 21:13:58 2016 (r303971) @@ -230,18 +230,32 @@ static void vtnet_disable_interrupts(str static int vtnet_tunable_int(struct vtnet_softc *, const char *, int); /* Tunables. */ +static SYSCTL_NODE(_hw, OID_AUTO, vtnet, CTLFLAG_RD, 0, "VNET driver parameters"); static int vtnet_csum_disable = 0; TUNABLE_INT("hw.vtnet.csum_disable", &vtnet_csum_disable); +SYSCTL_INT(_hw_vtnet, OID_AUTO, csum_disable, CTLFLAG_RDTUN, + &vtnet_csum_disable, 0, "Disables receive and send checksum offload"); static int vtnet_tso_disable = 0; TUNABLE_INT("hw.vtnet.tso_disable", &vtnet_tso_disable); +SYSCTL_INT(_hw_vtnet, OID_AUTO, tso_disable, CTLFLAG_RDTUN, &vtnet_tso_disable, + 0, "Disables TCP Segmentation Offload"); static int vtnet_lro_disable = 0; TUNABLE_INT("hw.vtnet.lro_disable", &vtnet_lro_disable); +SYSCTL_INT(_hw_vtnet, OID_AUTO, lro_disable, CTLFLAG_RDTUN, &vtnet_lro_disable, + 0, "Disables TCP Large Receive Offload"); static int vtnet_mq_disable = 0; TUNABLE_INT("hw.vtnet.mq_disable", &vtnet_mq_disable); -static int vtnet_mq_max_pairs = 0; +SYSCTL_INT(_hw_vtnet, OID_AUTO, mq_disable, CTLFLAG_RDTUN, &vtnet_mq_disable, + 0, "Disables Multi Queue support"); +static int vtnet_mq_max_pairs = VTNET_MAX_QUEUE_PAIRS; TUNABLE_INT("hw.vtnet.mq_max_pairs", &vtnet_mq_max_pairs); +SYSCTL_INT(_hw_vtnet, OID_AUTO, mq_max_pairs, CTLFLAG_RDTUN, + &vtnet_mq_max_pairs, 0, "Sets the maximum number of Multi Queue pairs"); static int vtnet_rx_process_limit = 512; TUNABLE_INT("hw.vtnet.rx_process_limit", &vtnet_rx_process_limit); +SYSCTL_INT(_hw_vtnet, OID_AUTO, rx_process_limit, CTLFLAG_RDTUN, + &vtnet_rx_process_limit, 0, + "Limits the number RX segments processed in a single pass"); static uma_zone_t vtnet_tx_header_zone; @@ -597,7 +611,6 @@ static void vtnet_setup_features(struct vtnet_softc *sc) { device_t dev; - int max_pairs, max; dev = sc->vtnet_dev; @@ -646,32 +659,31 @@ vtnet_setup_features(struct vtnet_softc if (virtio_with_feature(dev, VIRTIO_NET_F_MQ) && sc->vtnet_flags & VTNET_FLAG_CTRL_VQ) { - max_pairs = virtio_read_dev_config_2(dev, + sc->vtnet_max_vq_pairs = virtio_read_dev_config_2(dev, offsetof(struct virtio_net_config, max_virtqueue_pairs)); - if (max_pairs < VIRTIO_NET_CTRL_MQ_VQ_PAIRS_MIN || - max_pairs > VIRTIO_NET_CTRL_MQ_VQ_PAIRS_MAX) - max_pairs = 1; } else - max_pairs = 1; + sc->vtnet_max_vq_pairs = 1; - if (max_pairs > 1) { + if (sc->vtnet_max_vq_pairs > 1) { /* - * Limit the maximum number of queue pairs to the number of - * CPUs or the configured maximum. The actual number of - * queues that get used may be less. + * Limit the maximum number of queue pairs to the lower of + * the number of CPUs and the configured maximum. + * The actual number of queues that get used may be less. */ + int max; + max = vtnet_tunable_int(sc, "mq_max_pairs", vtnet_mq_max_pairs); - if (max > 0 && max_pairs > max) - max_pairs = max; - if (max_pairs > mp_ncpus) - max_pairs = mp_ncpus; - if (max_pairs > VTNET_MAX_QUEUE_PAIRS) - max_pairs = VTNET_MAX_QUEUE_PAIRS; - if (max_pairs > 1) - sc->vtnet_flags |= VTNET_FLAG_MULTIQ; + if (max > VIRTIO_NET_CTRL_MQ_VQ_PAIRS_MIN) { + if (max > mp_ncpus) + max = mp_ncpus; + if (max > VIRTIO_NET_CTRL_MQ_VQ_PAIRS_MAX) + max = VIRTIO_NET_CTRL_MQ_VQ_PAIRS_MAX; + if (max > 1) { + sc->vtnet_requested_vq_pairs = max; + sc->vtnet_flags |= VTNET_FLAG_MULTIQ; + } + } } - - sc->vtnet_max_vq_pairs = max_pairs; } static int @@ -2982,13 +2994,11 @@ vtnet_set_active_vq_pairs(struct vtnet_s dev = sc->vtnet_dev; if ((sc->vtnet_flags & VTNET_FLAG_MULTIQ) == 0) { - MPASS(sc->vtnet_max_vq_pairs == 1); sc->vtnet_act_vq_pairs = 1; return; } - /* BMV: Just use the maximum configured for now. */ - npairs = sc->vtnet_max_vq_pairs; + npairs = sc->vtnet_requested_vq_pairs; if (vtnet_ctrl_mq_cmd(sc, npairs) != 0) { device_printf(dev, @@ -3852,6 +3862,9 @@ vtnet_setup_sysctl(struct vtnet_softc *s SYSCTL_ADD_INT(ctx, child, OID_AUTO, "max_vq_pairs", CTLFLAG_RD, &sc->vtnet_max_vq_pairs, 0, "Maximum number of supported virtqueue pairs"); + SYSCTL_ADD_INT(ctx, child, OID_AUTO, "requested_vq_pairs", + CTLFLAG_RD, &sc->vtnet_requested_vq_pairs, 0, + "Requested number of virtqueue pairs"); SYSCTL_ADD_INT(ctx, child, OID_AUTO, "act_vq_pairs", CTLFLAG_RD, &sc->vtnet_act_vq_pairs, 0, "Number of active virtqueue pairs"); Modified: head/sys/dev/virtio/network/if_vtnetvar.h ============================================================================== --- head/sys/dev/virtio/network/if_vtnetvar.h Thu Aug 11 20:48:03 2016 (r303970) +++ head/sys/dev/virtio/network/if_vtnetvar.h Thu Aug 11 21:13:58 2016 (r303971) @@ -155,6 +155,7 @@ struct vtnet_softc { int vtnet_if_flags; int vtnet_act_vq_pairs; int vtnet_max_vq_pairs; + int vtnet_requested_vq_pairs; struct virtqueue *vtnet_ctrl_vq; struct vtnet_mac_filter *vtnet_mac_filter;