From owner-freebsd-net@FreeBSD.ORG Mon Apr 8 18:34:51 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 9B939851 for ; Mon, 8 Apr 2013 18:34:51 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wi0-x22a.google.com (mail-wi0-x22a.google.com [IPv6:2a00:1450:400c:c05::22a]) by mx1.freebsd.org (Postfix) with ESMTP id 38CD3FE6 for ; Mon, 8 Apr 2013 18:34:51 +0000 (UTC) Received: by mail-wi0-f170.google.com with SMTP id hm11so4173913wib.5 for ; Mon, 08 Apr 2013 11:34:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=mYZIN9jWAin2kLz1dt4BgN87EVTfubZZt5TPfzD3kds=; b=beO/wEBnmRizs0YjW9NRphqTR5kPhIG4wDxVbvoPp/d/Qj9YgG5pSUjnqwfJ7se/Ot 9A2Q6cF5SYSHYThym/YUehvl1Jtj9j0keAVdFEsEQz6G4plkeJLIAL2a+MAcJLuE73Yl gD9/8YS1f8mFWGoi6IV+Fzdqxs3LhQEQHXyefIklT2MJKAafiAuZAeZ0js8qdTvq1ysJ sgie2CfjwRTsf2rx09Rg5RFTrhRN+9pEZAGj5zWG1qHgcsEgwIh5Fai/lkgyRo43qtHv ggKnRbeFFwwfoKS1WPivnpx4lWz1J4B4ZhhW9VEqxodPPB/mFDXTHrELkNZ5wthVaDtk UljA== MIME-Version: 1.0 X-Received: by 10.180.19.39 with SMTP id b7mr14983009wie.15.1365446090451; Mon, 08 Apr 2013 11:34:50 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.217.121.136 with HTTP; Mon, 8 Apr 2013 11:34:50 -0700 (PDT) In-Reply-To: References: Date: Mon, 8 Apr 2013 11:34:50 -0700 X-Google-Sender-Auth: H5KWXE5WLsIIJXVYePl3-ok0rXA Message-ID: Subject: Re: drbr_enqueue - OCE driver - Freebsd 9.1 From: Adrian Chadd To: "Duvvuru,Venkat Kumar" Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Apr 2013 18:34:51 -0000 On 8 April 2013 10:14, Duvvuru,Venkat Kumar wrote: > Hi, > In the transmit path, if enqueue mechanism is used instead of blocking on the lock, the throughput is not good in some scenarios (especially single queue, multiple connections). > For example: > if (TRY_LOCK(&wq->tx_lock)) { > status = oce_multiq_transmit(ifp, m, wq); > UNLOCK(&wq->tx_lock); > } else { > status = drbr_enqueue(ifp, wq->br, m); > } > > Instead of the above code where the request is enqueued if I use a normal LOCK and block on it, it is giving good performance. > > Any suggestions on why the throughput is low in case of enqueue mechanism. You'll have to do a little more digging: * Is your application spewing TCP or UDP traffic? Or sometihng else? * are frames filling up the drbr enqueue and being dropped? having a lock means that the caller will block until the transmit completes, so traffic isn't immediately dropped as a side effect; * is the driver correctly checking the drbr queue when it finishes a transmit? In fact - oce_multiq_transmit should not be transmitting that mbuf, it should be enqueueing to the end of the drbr, then transmitting the head item in the drbr queue. Otherwise you may get out of order packets. adrian