From owner-freebsd-hackers@freebsd.org Sat May 26 18:24:39 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B1595EF1CE6 for ; Sat, 26 May 2018 18:24:39 +0000 (UTC) (envelope-from pratiy0100@gmail.com) Received: from mail-wm0-f46.google.com (mail-wm0-f46.google.com [74.125.82.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 31E327C405; Sat, 26 May 2018 18:24:39 +0000 (UTC) (envelope-from pratiy0100@gmail.com) Received: by mail-wm0-f46.google.com with SMTP id v131-v6so3178869wma.1; Sat, 26 May 2018 11:24:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=bErhvcNqtS1EUw6Lx+1iQsCl/gZ2hzXzPpiOQp8/3b8=; b=Aixvftn2/swPEEo5GYRZe3fN3SFkp7C69xggQfjgdWZcxFsHoqB1r+TxkheSLuPH+M 7K/4/JmkjWzxB5rbilcWFfWvvpqwmNxY5BbTmFwZbykJ7JstKRtAhMaeQZ+yejgRW6+i 7QK8gd+IJLyL4ktGsNvRTu2sYWzbm2bWci2Z75B4V8VZzv62JMd5KDxcHr2Y/6bzfeNX HwvLwUVikV4A3aWGJBeu2Zpj9qLm0GH5k88hQb7HkOKAAmtncJvHaHVh6jbMmTeeE5a2 +mFc4YhBNtA6wBLUEY4iR8jmVZM5OCOxyU2+41vHEcOvQnCE9JMRf/HCexCv5LYBQtq8 hoIA== X-Gm-Message-State: ALKqPwdAshRqnIrcn4TZHRcBkqGKEt7/ZOF/y3wLQWF28f4uAMw0Vn3t PSe6mhnz+5WHAcmke3fRrHkO4X9A X-Google-Smtp-Source: AB8JxZoPTNhde+0FHRKRRkEbS7nTLAwxls/oLS9Ob8tpgKmQWPWYNSBZxnyXqj2UVAJK34vYEC7qxQ== X-Received: by 2002:aa7:d34e:: with SMTP id m14-v6mr7959965edr.108.1527359071600; Sat, 26 May 2018 11:24:31 -0700 (PDT) Received: from mail-wm0-f54.google.com (mail-wm0-f54.google.com. [74.125.82.54]) by smtp.gmail.com with ESMTPSA id h38-v6sm1612567eda.43.2018.05.26.11.24.31 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 26 May 2018 11:24:31 -0700 (PDT) Received: by mail-wm0-f54.google.com with SMTP id t11-v6so22147806wmt.0; Sat, 26 May 2018 11:24:31 -0700 (PDT) X-Received: by 2002:a50:ec94:: with SMTP id e20-v6mr1641266edr.119.1527359070933; Sat, 26 May 2018 11:24:30 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Pratyush Yadav Date: Sat, 26 May 2018 23:54:17 +0530 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Why is there no bus_dmamap_load_sg()? To: Warner Losh Cc: freebsd-hackers@freebsd.org, Akshay Jaggi , Edward Napierala Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.26 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 May 2018 18:24:39 -0000 Hi Warner, On Sat 26 May, 2018, 7:40 PM Warner Losh, wrote: > > > On Sat, May 26, 2018 at 3:07 AM, Pratyush Yadav > wrote: > >> I was looking at the man page and I notice the function >> bus_dmamap_load_mbuf_sg() which directly returns the segment array to >> the client. But a similar variant of bus_dmamap_load() is missing. >> This variant would be useful to me and help me in my work. >> >> Is there a reason the sg variant was only implemented for mbufs? >> >> I was looking at the code in sys/kern/subr_bus_dma.c and a >> bus_dmamap_load_sg() does not look too difficult to implement. Is it >> ok if I submit a patch implementing it? >> > > The biggest reason is because of the failure modes. If there's not > sufficient memory or other resources to honor the request, we defer until > there is such memory / resources and then call the callback. There's no > convenient hook the drivers can connect to so they can retry the allocation > when memory becomes available (busdma has one since eventually a request > will complete and unload). The original port was for CAM SIMs which had > stringent requirements to always make progress on I/Os to avoid deadlock. > Rather than put this tricky code in all the CAM SIMs, it's centralized to > retry as soon and as robustly as it can. > > Network drivers, however, don't have such requirements and often don't > care. If they can't load a map, they can put the packet back on the > transmit queue (or whatever), set a flag and return without putting it into > the NIC's buffer rings. Eventually, a packet will finish transmitting and > there will be a chance to send it again (or one of a number of other > events, I'm glossing heavily here). It's a better fit for network drivers > than storage drivers. Since bus_dmamap_load_mbuf* is used only for network > drivers, it makes sense. > > So, it would only make sense to add if you can cope with BUS_DMA_NOWAIT > always, and that error handling is simpler than letting busdma call you > back when the shortage is over. Is that the case for your port? > I'm not sure. I looked at the OpenBSD code and they always call with a BUS_DMA_NOWAIT flag, but I'm not sure what the answer is for FreeBSD. Or is it just because OpenBSD doesn't have this sort of interface (the > callback difference dates back to the initial port to FreeBSD by Justin > Gibbs because it never got folded back into NetBSD for reasons too long, > too old and only half remembered by me for this post)? Can the code you are > porting cope with error returns and retry again later? > The other problem with using callbacks is that because I pass the xen-specific callback to the dma load, the driver using the xen specific dma operation can't pass it's own callback. I thought using the _sg() variant would allow the driver to use a callback. But the way you put it, that kind of defeats the purpose of the callback, which is to use it when allocation is defered. The bottom line is, I need to re-evaluate. I will spend some more time on this and discuss it with my mentors. I'll let you (and everyone else) know if I do decide we need it. Thanks for the enlightening answer. Regards, Pratyush Yadav