From owner-freebsd-hackers@freebsd.org Thu Oct 4 09:51:10 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 676B310C686C for ; Thu, 4 Oct 2018 09:51:10 +0000 (UTC) (envelope-from steevanxperia@gmail.com) Received: from mail-ot1-x330.google.com (mail-ot1-x330.google.com [IPv6:2607:f8b0:4864:20::330]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D580E8D0F9 for ; Thu, 4 Oct 2018 09:51:09 +0000 (UTC) (envelope-from steevanxperia@gmail.com) Received: by mail-ot1-x330.google.com with SMTP id e18-v6so8544909oti.8 for ; Thu, 04 Oct 2018 02:51:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=rxKJL8Lb/cngJHapEbLnRKNNIWW1AeKt/+zIgsp0wyI=; b=G+MLCGLKVjEkgohMDoJnCcmcbK51n2s7f4LXkxG//qcYNWuWDDvpv8qiKgN6/VK+it DdClcB1+uErN0yAANvKKQj9Yf7IzMZb7hfGhgr0n4+EwcCIJx5gN0cnx+cPtwSnrYZrR bTlJUEul3SjTa0qgsJmzJzHOb4NfLBq+G+QRN774HIO4RHS+6iDPLRTytt1lcV+wmnWK s7YqvHyasfxsJoBQCqo9IIYD+rfo+BJQEgN4385H6iiyrCNv0aUgzIkSfJb0bgonmRVM Q0JCl0gW9RNUl1QVoUCYqMr7tWrI3ZYXMvmIOKpnmG6L6NGwws0UQW0iZWkRTAlnX06L gbgw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=rxKJL8Lb/cngJHapEbLnRKNNIWW1AeKt/+zIgsp0wyI=; b=OmgwACCVZ3nmEQ3rK9uplYWryivEG0El/IjbZ42W6ZuaRrBaygvZs5ahZ7PO9k/XgN YyZ6Td31AThrHZBC+LoXaxBBOBaw6t5BxUp6eyUWpN3GQOVQ8S/B833hK/lZZT2JrvgI wVrN097RRzFcsu0wSmHqWZG4s1jbzoDbt9iuFcIsF/Aa9MV92jjV6HD42OldCAX1U0qV W1XT745ATH4KN5DkEyaVHkD6mAcHgSzJ5402JbgJ53WM/CSyJtAp5Z6rIqXFV6eK3jtM hXCl36o0zJ5YdURuwNj/qtnzMJJR206P5w6DvZWN0aQ9H1MdMr6nCrCJj6yWUdjZ/om5 3TZA== X-Gm-Message-State: ABuFfoil+PDIAmdL5TfKJH/N+UBW+adgZNr0WS6apNMcg9hlFq2UCriJ rANL34BVywjK8zMG6r5BseztcXoIQouuXMOTM8ZRoF+th2g= X-Google-Smtp-Source: ACcGV60zg9DgoTSSqd+wvWmCFo4jgIQZduk7T5d3LH4CRzwRUwg8uXjBVKMRArQepgqmHPOkAYDtMT0QXA7pD/RK8Cg= X-Received: by 2002:a9d:216e:: with SMTP id l43mr3314009otd.29.1538646668746; Thu, 04 Oct 2018 02:51:08 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Steevan Rodrigues Date: Thu, 4 Oct 2018 15:20:21 +0530 Message-ID: Subject: Re: Contigfree takes too much time that cuases PCIe driver unload to take up to 30 minutes To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Oct 2018 09:51:10 -0000 Hi , After some investigation I came across following email thread: http://freebsd.1045724.x6.nabble.com/FreeBSD-11-1-contigfree-performance-issue-td6247284.html Now I see that the issue seen by me is same as the one described in the above email thread. We found that our driver too takes too much time during configfree and contigmalloc calls. This happens only in server with xeon dual CPU with 10 core each ( total 20 cores / 40 threads). This issue is not seen in server with Xeon Dual CPU with 6 core each ( total 12 cores / 24 threads) Any suggestions about how to overcome this problem with contigmalloc and configfree ? Thanks Steevan On Fri, Sep 21, 2018 at 9:32 AM Steevan Rodrigues wrote: > Hello Folks, > > We have a PCI express card for data processing to achieve 25 to 30 Gbps . > Recently we have been facing a issue in one of the server at customer > site. > > System information: > Supermicro motherboard-X11DPH-TQ motherboard > hw.model: Intel(R) Xeon(R) Gold 5115 CPU @ 2.40GHz ( Dual CPU with total > 20 cores) > 16 GB RAM > > In this system the same PCIe card works fine in RHEL 7.5 and the driver > load and unload also works fine. Our driver usually takes about 20 seconds > to load and approx 20 seconds to unload. > > However, on this particular system when FreeBSD ( tried with 11.1 and > 11.2 Release) is used it takes about 2 to 4 minutes to load and about 8 to > 30 minutes to unload. > During unload it looks like the system freezes completely and I can not > run any commands to find out what is happening . > > The same driver works fine in our lab servers Dell T620 ( Xeon 12 core > CPUs) and desktops with FreeBSD 11.1 and 11.2 and 10.4 . > > Also I ran couple of Phoronix tests on this SuperMicro server. In one > particular test I see that the time taken is too much. > The test suite is OsBench. In this, the thread creation test shows average > time taken to create threads is 9000 usec . On the other hand we have > servers and desktops in our lab and in that this same thread creation takes > only 20 to 30 usec . > > Any pointers about what could be wrong with the system or our PCIe card > driver ? > > Thanks > Steevan > > > >