From owner-freebsd-arch@freebsd.org Wed Nov 18 21:38:12 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F01E5A32218 for ; Wed, 18 Nov 2015 21:38:11 +0000 (UTC) (envelope-from onwahe@gmail.com) Received: from mail-io0-f175.google.com (mail-io0-f175.google.com [209.85.223.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BE59B189A for ; Wed, 18 Nov 2015 21:38:11 +0000 (UTC) (envelope-from onwahe@gmail.com) Received: by iofh3 with SMTP id h3so68830074iof.3 for ; Wed, 18 Nov 2015 13:38:10 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=mime-version:date:message-id:subject:from:to:content-type; bh=BKW84/cUNzJi0CRehHT3eMIgvtCQ1r2HrjCg6amX7DA=; b=f9uEEQFoaDHTe+uMsJb7U5CcZaxh4rOL25bHmOXdCqR9dG8CQFoLT/gg5xASz1EMeC YYRdfMd5e9VkyUmOGN+fR3HPJIy5HCRf0cBKRQKKr0jJK5H7aON/pWaTeRp0rPRN1aEW F/FI/uKIu777sqbW6x2tZoGFZjgpeLElp1SbJe+9ysgS+b1RnmMH3v4KAh2gwiqab10O S5Nhimea0fer6tkMctp7oyH+5BlwB5ELlkYQCR/wu+MxWuyXoak8+cXGDKaaedjqBIBQ meDANvpdbca9M4JYk7wT+4eZNt0D37tD992Ok89PO0woN9ad7Hkv/56WYSdUmM8p+6wB a2eQ== X-Received: by 10.107.4.213 with SMTP id 204mr4103368ioe.195.1447862450668; Wed, 18 Nov 2015 08:00:50 -0800 (PST) Received: from mail-ig0-f174.google.com (mail-ig0-f174.google.com. [209.85.213.174]) by smtp.gmail.com with ESMTPSA id wc5sm11840774igb.1.2015.11.18.08.00.49 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 18 Nov 2015 08:00:50 -0800 (PST) Received: by igvg19 with SMTP id g19so120479115igv.1 for ; Wed, 18 Nov 2015 08:00:49 -0800 (PST) MIME-Version: 1.0 X-Received: by 10.50.128.194 with SMTP id nq2mr3534476igb.19.1447862449706; Wed, 18 Nov 2015 08:00:49 -0800 (PST) Received: by 10.64.130.38 with HTTP; Wed, 18 Nov 2015 08:00:49 -0800 (PST) Date: Wed, 18 Nov 2015 17:00:49 +0100 X-Gmail-Original-Message-ID: Message-ID: Subject: a question about BUS_DMA_MIN_ALLOC_COMP flag meaning From: Svatopluk Kraus To: FreeBSD Arch , Konstantin Belousov Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Nov 2015 21:38:12 -0000 Hi, I have fallen to some problem with inconsistent use of BUS_DMA_MIN_ALLOC_COMP flag. This flag was introduced in x86 MD code very very long ago and so, the problem covers all archs which came out from it. However, it's only about bus_dma_tag_t with BUS_DMA_COULD_BOUNCE flag set. (1) When bus_dma_tag_t is being created with BUS_DMA_ALLOCNOW flag specified, some bounce pages could be allocated in advance and BUS_DMA_MIN_ALLOC_COMP flag is set to the tag. The bounce pages are allocated only if the tag's maxsize property is higher than size of all bounce pages already allocated in a bounce zone. (2) When bus_dmamap_t is being created, then if BUS_DMA_MIN_ALLOC_COMP is not set on associated tag, some bounce pages are ALWAYS allocated and BUS_DMA_MIN_ALLOC_COMP is set afterwards, (3) else some bounce pages could be allocated if there is not enough pages in a bounce zone and BUS_DMA_MIN_ALLOC_COMP is set afterwards. The problem is the following. Due to case (2), the number of pages in bounce zone can grow infinitely, as bounce pages once allocated are never freed. It can happen when a big number of bus_dma_tag_t together with bus_dmamap_t are created, or they are created dynamically either because of a loadable module or by design. The inconsistency is that when bus_dma_tag_t is being created, there is no limit for how much pages could be allocated. On the other hand, when bus_dmamap_t is being created, there is MAX_BPAGES limitation. I think that fix for case (2) presented as x86 fix is the following: diff --git a/sys/x86/x86/busdma_bounce.c b/sys/x86/x86/busdma_bounce.c index 4826a2b..a15139f 100644 --- a/sys/x86/x86/busdma_bounce.c +++ b/sys/x86/x86/busdma_bounce.c @@ -308,7 +308,7 @@ bounce_bus_dmamap_create(bus_dma_tag_t dmat, int flags, bus_dmamap_t *mapp) else maxpages = MIN(MAX_BPAGES, Maxmem - atop(dmat->common.lowaddr)); - if ((dmat->bounce_flags & BUS_DMA_MIN_ALLOC_COMP) == 0 || + if ((dmat->bounce_flags & BUS_DMA_MIN_ALLOC_COMP) == 0 && (bz->map_count > 0 && bz->total_bpages < maxpages)) { pages = MAX(atop(dmat->common.maxsize), 1); pages = MIN(maxpages - bz->total_bpages, pages); IMO, it also fixes logic by making it same as in bus_dma_tag_t case. The next question is, if case (1) should be limited by MAX_BPAGES as in case (3) or maybe better if there should be some internal limitation for bounce zone itself. As this is quite old code which covers many archs, I really would appreciate your comments. Thanks, Svatopluk Kraus