From owner-freebsd-hackers@freebsd.org Tue Apr 26 12:54:24 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AE5BEB1C9E5 for ; Tue, 26 Apr 2016 12:54:24 +0000 (UTC) (envelope-from sebastian.huber@embedded-brains.de) Received: from mail.embedded-brains.de (mail.embedded-brains.de [82.135.62.35]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 61C38127C for ; Tue, 26 Apr 2016 12:54:24 +0000 (UTC) (envelope-from sebastian.huber@embedded-brains.de) Received: from localhost (localhost.localhost [127.0.0.1]) by mail.embedded-brains.de (Postfix) with ESMTP id 3918E2A1816 for ; Tue, 26 Apr 2016 14:49:39 +0200 (CEST) Received: from mail.embedded-brains.de ([127.0.0.1]) by localhost (zimbra.eb.localhost [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id OQ6rqbLXBv8d for ; Tue, 26 Apr 2016 14:49:37 +0200 (CEST) Received: from localhost (localhost.localhost [127.0.0.1]) by mail.embedded-brains.de (Postfix) with ESMTP id EF1682A183C for ; Tue, 26 Apr 2016 14:49:36 +0200 (CEST) X-Virus-Scanned: amavisd-new at zimbra.eb.localhost Received: from mail.embedded-brains.de ([127.0.0.1]) by localhost (zimbra.eb.localhost [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id bzjJhQhKQSHX for ; Tue, 26 Apr 2016 14:49:36 +0200 (CEST) Received: from [192.168.96.129] (unknown [192.168.96.129]) by mail.embedded-brains.de (Postfix) with ESMTPSA id B14BA2A1816 for ; Tue, 26 Apr 2016 14:49:36 +0200 (CEST) To: freebsd-hackers@freebsd.org From: Sebastian Huber Subject: UMA alloc with max items < potential items in per processor buckets Message-ID: <571F63BD.1090001@embedded-brains.de> Date: Tue, 26 Apr 2016 14:49:01 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.5.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Apr 2016 12:54:24 -0000 Hello, I don't use FreeBSD directly, but instead a port of the FreeBSD network=20 stack to RTEMS. So, this problem may not apply to FreeBSD. I observed=20 the following problem during an UDP socket create. They are allocated=20 from the following zone: void udp_init(void) { in_pcbinfo_init(&V_udbinfo, "udp", &V_udb, UDBHASHSIZE, UDBHASHSIZE, "udp_inpcb", udp_inpcb_init, NULL, UMA_ZONE_NOFREE, IPI_HASHFIELDS_2TUPLE); V_udpcb_zone =3D uma_zcreate("udpcb", sizeof(struct udpcb), NULL, NULL, NULL, NULL, UMA_ALIGN_PTR, UMA_ZONE_NOFREE); uma_zone_set_max(V_udpcb_zone, maxsockets); EVENTHANDLER_REGISTER(maxsockets_change, udp_zone_change, NULL, EVENTHANDLER_PRI_ANY); } In my setup maxsockets is 32. This is probably artificially small=20 compared to a real FreeBSD machine. The system has 24 processors, so we=20 need 128 * 24 items for the per processor cache buckets. This is=20 considerably lager than the single keg of the zone can deliver. Thus in=20 case a processor without a per processor bucket tries to do a=20 uma_zalloc(V_udpcb_zone, M_NOWAIT | M_ZERO), then it will get no item if=20 other processors already consumed all items for their per processor=20 cache buckets. I adjusted the uma_zone_set_max() like this diff --git a/freebsd/sys/vm/uma_core.c b/freebsd/sys/vm/uma_core.c index 845c433..f88d3d3 100644 --- a/freebsd/sys/vm/uma_core.c +++ b/freebsd/sys/vm/uma_core.c @@ -2903,6 +2903,10 @@ uma_zone_set_max(uma_zone_t zone, int nitems) uma_keg_t keg; ZONE_LOCK(zone); +#ifdef SMP + if (nitems < zone->uz_count * mp_maxid) + nitems =3D zone->uz_count * mp_maxid; +#endif keg =3D zone_first_keg(zone); keg->uk_maxpages =3D (nitems / keg->uk_ipers) * keg->uk_ppera; if (keg->uk_maxpages * keg->uk_ipers < nitems) and this seems to fix the problem. Is there an implicit assumption in=20 FreeBSD that the zone max is always large enough to provide items for=20 the per processor cache buckets? --=20 Sebastian Huber, embedded brains GmbH Address : Dornierstr. 4, D-82178 Puchheim, Germany Phone : +49 89 189 47 41-16 Fax : +49 89 189 47 41-09 E-Mail : sebastian.huber@embedded-brains.de PGP : Public key available on request. Diese Nachricht ist keine gesch=C3=A4ftliche Mitteilung im Sinne des EHUG= .