From owner-freebsd-hackers@freebsd.org Mon Nov 19 12:49:12 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9C6FF1102AF6 for ; Mon, 19 Nov 2018 12:49:12 +0000 (UTC) (envelope-from steevanxperia@gmail.com) Received: from mail-oi1-x229.google.com (mail-oi1-x229.google.com [IPv6:2607:f8b0:4864:20::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D7CC874351 for ; Mon, 19 Nov 2018 12:49:11 +0000 (UTC) (envelope-from steevanxperia@gmail.com) Received: by mail-oi1-x229.google.com with SMTP id 192-v6so25131445oii.6 for ; Mon, 19 Nov 2018 04:49:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=8/IA1eCjJ+axOhBXJYYMPx7MB4rsevTdSLN+SZ54pFQ=; b=veNxxBsyQnjxoNkHwBNygKp1e2jeVxiJVvtEqciy/o2HtLGDSiDhCsZLpm5+1RIDGC z8yc8Twghfqdp9QNKoI92MEuxV1acIPcWou/jEhBayw9V8pQnkDP4hc3EKpO7pVTjmCD o+56hfgXg7C6nK+5crxRFSHdVMaKMn3ZdAJnxVG6lWyK46RaC241GU6lc2Wf/y/3mMYp s7KalKHOK2UQ9sQ5xgkbh/kAfMZLb/ZKVBNG4QVhuHW7s258+o7SYamOHnTWFaDa1xIL uWmvjO5UAXjEi7uGQX1avk9zQtYRd/ThlKfNAjnFPDhNJ0i2BDMLoif94ieVtpbee36I fBcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=8/IA1eCjJ+axOhBXJYYMPx7MB4rsevTdSLN+SZ54pFQ=; b=rzIPFPLKEjaECmjrbk04Q1Ro/X4zsXWGlUta5k3eYbzcS8xb1pjd0NPVnFwSY95GNi Qiyw8JmgQIAM7mrTIgVl6Px6pVu/5nJquyXRpWMOn+ZYt3QuacyQ4iNGY1/R59OqQm/V iNVbRtSJNy65cmHFp9+Frer8VtS0rBxYKAHNVkdZ7A9Qg6Wt29lJdVPQmIm+jgOxeeKD kAoisGzvTbC3vchfp1KPMxY6RpCoGS1tDGDA77hvJWzj2zmzamn4TxPE56KpXyNObnr1 qoyP58N4ZBZgX4axRw1yCaeWyuHzAYeVkIT6999bhE93jCq8X9ZXq8xPffxQl9mPSvaw 8xQQ== X-Gm-Message-State: AGRZ1gKSFxEGfZVZIzYcg8iuDRHtiARtFe0u6LI6MYbfTY3I5W+7TJpB Fjk2Emp/Gd3KMLdj9emYLzutJyIbObhAbiXsBfM= X-Google-Smtp-Source: AJdET5eNUdkNIjhs9OGioVGXGJA+7/c7ilGQ04vchonTasqU2lbIH/1B5MO4j57XvWJtkfAZzMPhj5BQK8fUnHLqXhU= X-Received: by 2002:aca:a57:: with SMTP id 84mr7344715oik.197.1542631751103; Mon, 19 Nov 2018 04:49:11 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Steevan Rodrigues Date: Mon, 19 Nov 2018 18:18:59 +0530 Message-ID: Subject: Re: Contigfree takes too much time that cuases PCIe driver unload to take up to 30 minutes To: rysto32@gmail.com Cc: freebsd-hackers@freebsd.org X-Rspamd-Queue-Id: D7CC874351 X-Spamd-Result: default: False [-3.83 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.994,0]; R_DKIM_ALLOW(-0.20)[gmail.com]; FROM_HAS_DN(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_DN_NONE(0.00)[]; NEURAL_HAM_SHORT(-0.96)[-0.963,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; RCVD_IN_DNSWL_NONE(0.00)[9.2.2.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; IP_SCORE(-0.86)[ipnet: 2607:f8b0::/32(-2.52), asn: 15169(-1.70), country: US(-0.09)] X-Rspamd-Server: mx1.freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Nov 2018 12:49:12 -0000 I tried FreeBSD 12 Beta 3 version on this server with Xeon Gold 5115 CPU. All these problems have disappeared. Actually contigfree was taking too much time on FreeBSD 11.x on my server with Xeon Gold 5115 CPU. In FreeBSD 12 Beta3 also contigfree still takes much more time compared to contigmalloc. However when I compare the values to FreeBSD 11.x number I can see huge improvement in FreeBSD 12 Beta 3 . Because of this contigfree issue my driver unload used to take 5 to 20 minutes in FreeBSD 11.x. Now my driver takes only a few seconds to load and a few seconds to unload in FreeBSD 12. BEta 3. Hence it looks like the problem was specific FreeBSD 11.x. Regards, Steevan On Sat, Oct 27, 2018 at 12:22 PM Steevan Rodrigues wrote: > > Yes , it is contigfree that is taking too much time. ( Contigfree is > taking almost 5x to 10x more time than contigmalloc ) > However, I see this issue only on a SuperMicro Server which has Xeon Gold > CPU with 10 cores ( dual CPU i.e total 20 cores). > > So I am wondering whether FreeBSD has performance issues on mutlicore ( > more than 16 cores) servers ? > Has anyone come across issues like this? > > Thanks > Steevan > > > > > On Fri, Oct 5, 2018 at 11:31 PM Ryan Stone wrote: > >> I'm not sure why configfree() would be taking a long time, but >> configmalloc() can be extraordinarily expensive when it needs to >> defragment memory to meet the request. Does your application really >> require a lot of physically contiguous memory? if you can restructure >> to not require contigmalloc() all -- maybe by using S/G DMA -- you may >> find your life significantly easier. >> >