From owner-freebsd-drivers@freebsd.org Thu Dec 13 10:04:16 2018 Return-Path: Delivered-To: freebsd-drivers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 13D5D1314FA0; Thu, 13 Dec 2018 10:04:16 +0000 (UTC) (envelope-from rajfbsd@gmail.com) Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 626BB80444; Thu, 13 Dec 2018 10:04:15 +0000 (UTC) (envelope-from rajfbsd@gmail.com) Received: by mail-wr1-x42b.google.com with SMTP id z5so1319328wrt.11; Thu, 13 Dec 2018 02:04:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=LxHKRq0TG85/hYg4YVoEoF6YTc33Usep0g1fYHrUsHc=; b=FnSPY8XDJKlzk3RdxrJVRgLwzy0iX8YXHSO27BT/1QMeLcTEgHqeGFUWF5KtKxbnhq A/F7PX/5feakv1kGbbnkJEyEt+tc8rO+fnp+5MYcnZ2cBt27Hka5JPgjPtwdU7kggWTh 4u5dRSzvK7khXj9BL7pC6GMspheEpl7zFFZ2ieXhCk0rX6Yl3X7fBQ9H82fOrw0fI8tV OFqvFi6jTLBdv5lEHR3UczQejkIjoUpeJrdbZUofDA6Kgs0JIbRa+6FbOr0CiCR7d+0Q 57moXO5OXxjth279AGmJozRc5776ZOas1sSw10M6Bp71ubzr+bzuPbsn8B7kivNVrkLg DKBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=LxHKRq0TG85/hYg4YVoEoF6YTc33Usep0g1fYHrUsHc=; b=nmtqULGyCrToSwO1471T8OPJswehlhNmGU6M7qzLU4OHdvBSeZJkw1Ui8KPxHNmpY6 AqKb8mcr2xyrqlIwHJzhB/bF5RJVGJYHQ1iEKj/PHiSeOeEJbGj9dbTwTP6OU4rHXwQ5 HxX55v6xl8ouhFEYLU0HPSOMh3AWn5gnGjE8CmdZVl/JPgFINT7fXINQNWgYsirp7d8g kKcD6cGQ4BNEh3NvXNUc6R2yRld8g+K7bHTQlIMedg1rx0XbwLEVn6irUGQaavslouud qForGnZDBpgzJnYb8UqWYWAWvSZTQq+EzMlPLayBuYTC/G0PmOCPmFnXJmIe2MtZyoOU GklA== X-Gm-Message-State: AA+aEWZe0IXoD4bH8kTRATzCdStbPeR+9ep1F/NeRwEUHRNtjEHKAHpC mf6SH+RtOCnGf05HrwQIgOuYp/pjC3DydEWC5ldnaA== X-Google-Smtp-Source: AFSGD/UO6W0P9fuPYyf4jjJMVgUOKp55RCoK4VjxKvbA+mbcvWkUxVhdnqBAmI5VJ/v+++mrybzHRa/JFwHE6O9vGIA= X-Received: by 2002:adf:fd03:: with SMTP id e3mr19654008wrr.280.1544695453984; Thu, 13 Dec 2018 02:04:13 -0800 (PST) MIME-Version: 1.0 From: Rajesh Kumar Date: Thu, 13 Dec 2018 15:34:02 +0530 Message-ID: Subject: How to use the DMA Engine in FreeBSD? To: freebsd-drivers@freebsd.org, freebsd-hackers@freebsd.org X-Rspamd-Queue-Id: 626BB80444 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=FnSPY8XD; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of rajfbsd@gmail.com designates 2a00:1450:4864:20::42b as permitted sender) smtp.mailfrom=rajfbsd@gmail.com X-Spamd-Result: default: False [-2.41 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.97)[-0.972,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_DN_NONE(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; RCVD_IN_DNSWL_NONE(0.00)[b.2.4.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-0.85)[-0.847,0]; RCPT_COUNT_TWO(0.00)[2]; IP_SCORE(-0.58)[ipnet: 2a00:1450::/32(-1.49), asn: 15169(-1.30), country: US(-0.09)]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; MIME_TRACE(0.00)[0:+,1:+]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Dec 2018 10:04:16 -0000 Hi, Is there any good documentation available to understand the existing support, API's and how-to use the DMA Engine in FreeBSD? I am trying to write a test driver which will use DMA Engine to do the data transfer (rather than plain memcpy which involves cpu). Can anyone point to any driver implementation which has similar functions implemented? I see references to SYS_RES_DRQ to allocate DMA channels and play around. But that seems to be specific to ISA. Can it be used for PCI drivers as well? Thanks, Rajesh. From owner-freebsd-drivers@freebsd.org Thu Dec 13 10:41:20 2018 Return-Path: Delivered-To: freebsd-drivers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 84DED1316561; Thu, 13 Dec 2018 10:41:20 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (turbocat.net [IPv6:2a01:4f8:c17:6c4b::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9CA0A81D64; Thu, 13 Dec 2018 10:41:19 +0000 (UTC) (envelope-from hps@selasky.org) Received: from hps2016.home.selasky.org (unknown [178.17.145.230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id B060C26019D; Thu, 13 Dec 2018 11:41:10 +0100 (CET) Subject: Re: How to use the DMA Engine in FreeBSD? To: Rajesh Kumar , freebsd-drivers@freebsd.org, freebsd-hackers@freebsd.org References: From: Hans Petter Selasky Message-ID: <434ed3ba-6613-b5eb-5a68-f69936bcda21@selasky.org> Date: Thu, 13 Dec 2018 11:39:17 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.0.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 9CA0A81D64 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of hps@selasky.org designates 2a01:4f8:c17:6c4b::2 as permitted sender) smtp.mailfrom=hps@selasky.org X-Spamd-Result: default: False [-4.17 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+a:mail.turbocat.net]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; DMARC_NA(0.00)[selasky.org]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[mail.turbocat.net]; NEURAL_HAM_SHORT(-0.96)[-0.956,0]; IP_SCORE(-2.91)[ip: (-8.71), ipnet: 2a01:4f8::/29(-3.30), asn: 24940(-2.52), country: DE(-0.01)]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Dec 2018 10:41:20 -0000 On 12/13/18 11:04 AM, Rajesh Kumar wrote: > Hi, > > Is there any good documentation available to understand the existing > support, API's and how-to use the DMA Engine in FreeBSD? > > I am trying to write a test driver which will use DMA Engine to do the data > transfer (rather than plain memcpy which involves cpu). Can anyone point > to any driver implementation which has similar functions implemented? I > see references to SYS_RES_DRQ to allocate DMA channels and play around. But > that seems to be specific to ISA. Can it be used for PCI drivers as well? Hi, What you are looking for is called PCI busmaster. --HPS From owner-freebsd-drivers@freebsd.org Thu Dec 13 11:16:32 2018 Return-Path: Delivered-To: freebsd-drivers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B89551317937; Thu, 13 Dec 2018 11:16:32 +0000 (UTC) (envelope-from johalun0@gmail.com) Received: from mail-oi1-x22d.google.com (mail-oi1-x22d.google.com [IPv6:2607:f8b0:4864:20::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 01ED983366; Thu, 13 Dec 2018 11:16:32 +0000 (UTC) (envelope-from johalun0@gmail.com) Received: by mail-oi1-x22d.google.com with SMTP id x202so1284095oif.13; Thu, 13 Dec 2018 03:16:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-transfer-encoding:content-language; bh=8qW5LlRMkcuhT9loX/aiMkvfpazq/y7Z/PX2nBuHS7g=; b=JMWQs9XvG2qBL2bAPdxL6Wv2L0y7sXt7Ecg+7LZtDLXwZQ5B2ViPJmnMf69gTQVeYO PrPHdNH7BKaVHBZOboDpXymmSVTgNbCElezVnJbunY/UkKto1bkC0amFe4oSwARKmbXz HE7uxjKAQqcWFmc9oqhL+E7i7hqHyndILoLjBVbXniI0wn0HHj1bSK4zdJadwGT259wM c4qXIh6dzI0zmuJvzQ50HK0I5AB9YqYBEL9g610Bu3GUFm9wnDRGvT/D5CUgy5o2kC5P k4jS1GbvlnOp+EnRsA/DIo+Yg44vS1eEG6yItrRbwLi6g6cM5a47RcEITRAXdNZMfyGJ OYOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=8qW5LlRMkcuhT9loX/aiMkvfpazq/y7Z/PX2nBuHS7g=; b=U6HzK0Jy+ixenQDnm6+hXhGN8h0/aObK6lRW6pLHt8WpvAooZG/d4KcpKpedZFuTfz 8NW5hPalJ6umfq/jhKEGbCW4FrySK/9Mt4z5zuzaOJP+sX7g1xQ9r3ibFNmEtaakNg0Z r1PvXl9BnfLE1V+ofZvJ+7jKv2J0eeOz4dSo0fndO/KPaoyMeIae/KLvGBsx4I+ejDn9 3IU6x1z7zr38ItaJDaINtg9Xty2IB1tp/PpJ2pqlJoEN+uZAWT2WBB1uHygO0loPOaZr JdHTJSaujOhavKpQtaV+A1vOpMX/WT5sXrpJwdbdtvRp8DwGsee7IwiGMzJdsa38tiKk rmMA== X-Gm-Message-State: AA+aEWYP3VO6wu3dWk6B9dtWp2YzQwmZffiVulwwBVHDOEFRVqAZDusz dHs7VdAnFEzE9mWFns5EalP+vMIa X-Google-Smtp-Source: AFSGD/VDPu6q0wJHAOUDne2nbVnrbT+aCJHYK5+vcomoC0UWafTTkH3iWlBNsanmLutaWTmMUylBQg== X-Received: by 2002:aca:b104:: with SMTP id a4mr2471872oif.133.1544699790824; Thu, 13 Dec 2018 03:16:30 -0800 (PST) Received: from [192.168.1.33] ([81.174.250.12]) by smtp.gmail.com with ESMTPSA id j23sm679877oih.22.2018.12.13.03.16.29 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 13 Dec 2018 03:16:30 -0800 (PST) Subject: Re: How to use the DMA Engine in FreeBSD? To: Rajesh Kumar , freebsd-drivers@freebsd.org, freebsd-hackers@freebsd.org References: From: Johannes Lundberg Message-ID: <90ce7684-6a8f-c398-5a46-a7f6b03015e8@gmail.com> Date: Thu, 13 Dec 2018 11:16:27 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.3.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-Rspamd-Queue-Id: 01ED983366 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=JMWQs9Xv; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of johalun0@gmail.com designates 2607:f8b0:4864:20::22d as permitted sender) smtp.mailfrom=johalun0@gmail.com X-Spamd-Result: default: False [-2.55 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.996,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_SHORT(-0.95)[-0.949,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(-0.59)[ipnet: 2607:f8b0::/32(-1.58), asn: 15169(-1.30), country: US(-0.09)]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[d.2.2.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; SUBJECT_ENDS_QUESTION(1.00)[] X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Dec 2018 11:16:33 -0000 On 12/13/18 10:04 AM, Rajesh Kumar wrote: > Hi, > > Is there any good documentation available to understand the existing > support, API's and how-to use the DMA Engine in FreeBSD? > > I am trying to write a test driver which will use DMA Engine to do the data > transfer (rather than plain memcpy which involves cpu). Can anyone point > to any driver implementation which has similar functions implemented? I > see references to SYS_RES_DRQ to allocate DMA channels and play around. But > that seems to be specific to ISA. Can it be used for PCI drivers as well? Hi The book "FreeBSD Device Drivers" by Joseph Kong has an example. Not sure it matches exactly what you are looking for but it might help more than the FreeBSD source code which tends to lack documentation. > Thanks, > Rajesh. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-drivers@freebsd.org Thu Dec 13 15:53:41 2018 Return-Path: Delivered-To: freebsd-drivers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ECD8F1329A0A for ; Thu, 13 Dec 2018 15:53:40 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qt1-x841.google.com (mail-qt1-x841.google.com [IPv6:2607:f8b0:4864:20::841]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 008F88D996 for ; Thu, 13 Dec 2018 15:53:40 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qt1-x841.google.com with SMTP id z16so2645888qtq.4 for ; Thu, 13 Dec 2018 07:53:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=fzccQgKTpAFs4ytrOSOCXM6BAxqxFNZ9e4SObqusW7c=; b=yHR7qIMAKmY7D60/gyWTftxnB3QktcYcUf8JMLWBgoWVEo6LzFoH1T5Rl7UKrCSbs9 L/Wr9mRONca/vdhWZxwzjSX5Isgq2MmwrakP+dgZgmb4UVSQOhDRSSSCOTY9JzxZROKo 11FHHg/I4ZPseRyX9loErwbmAF8FiCAbimhibjF4opzXe+61iWd58FDf1xk9CMor7efU 5m4xC/OknDF/bQr4t33AMx6DxPY7RzPLk2Ak/EQvCe6GQ9nBM1mcoHte1zR9+FaAQRdz 5j6HTUIZ0Eszm/JFZ52reVhUOo2cwn//0w8E0rPU7Cky/wooQI8tFKLipHlMfWZJVqpX B7iw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=fzccQgKTpAFs4ytrOSOCXM6BAxqxFNZ9e4SObqusW7c=; b=JcOEyH4UItYsWx/l7spw1d0lGkqx3ZIVJ8BZPN7O5VyIKIZn4bPXYQIU5qz6/AY6nI cWiCmkTGlQiZ8m2rCe0noCBFUssULcY/sjabeOOmUVNiKB9d9AgVOJXF9SKB3EwlIB0r rGpl5XJ+Y62zyotagz41lUmB6nyBbsP2aKQudm75AuiNNcN5T7NuTpSRkAfnuE2U8O4X DaEzJH6n78lHdNk9zjwPeijDzxkRJeES0Is1cAgfMZGxnziPjODch58yTVJT8b8jgWo+ DPcr0pBgNk1CI9/zjWDaJpV5HDqJ8+EnOqQoYieJD2svRaTYCaQUYuVOmmMwvCgGxbQA 07ww== X-Gm-Message-State: AA+aEWYWk4kSBxjBFTyhfuljxORm5vWJZCe47f9CWB0u2du7crh16NJL 128NMTjCKTUJdlDHC0Sw+TZfv2EX6OmuLsQzyheEfg== X-Google-Smtp-Source: AFSGD/UZ/ZHRQ+Vx8fR93m4D6LW7NJpTFxgTfJzv6+bk3qZ6+SeW0iLrMX/UYA5VRHiyEEj4wN+9Y2ma25WvHh3Nb60= X-Received: by 2002:a0c:f143:: with SMTP id y3mr23963534qvl.21.1544716419308; Thu, 13 Dec 2018 07:53:39 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Thu, 13 Dec 2018 08:53:15 -0700 Message-ID: Subject: Re: How to use the DMA Engine in FreeBSD? To: Rajesh Kumar Cc: freebsd-drivers@freebsd.org, FreeBSD Hackers X-Rspamd-Queue-Id: 008F88D996 X-Spamd-Bar: +++ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=yHR7qIMA X-Spamd-Result: default: False [3.25 / 15.00]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; IP_SCORE(0.54)[ip: (5.65), ipnet: 2607:f8b0::/32(-1.58), asn: 15169(-1.30), country: US(-0.09)]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-drivers@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; NEURAL_SPAM_MEDIUM(1.00)[0.995,0]; NEURAL_SPAM_SHORT(0.73)[0.730,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; MX_GOOD(-0.01)[cached: ALT1.aspmx.l.google.com]; RCVD_IN_DNSWL_NONE(0.00)[1.4.8.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; FREEMAIL_TO(0.00)[gmail.com]; SUBJECT_ENDS_QUESTION(1.00)[]; MIME_TRACE(0.00)[0:+,1:+]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; RCVD_TLS_LAST(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Dec 2018 15:53:41 -0000 On Thu, Dec 13, 2018, 3:04 AM Rajesh Kumar Hi, > > Is there any good documentation available to understand the existing > support, API's and how-to use the DMA Engine in FreeBSD? > Usually you just use pci busmastering and it just works. I am trying to write a test driver which will use DMA Engine to do the data > transfer (rather than plain memcpy which involves cpu). Can anyone point > to any driver implementation which has similar functions implemented? I > see references to SYS_RES_DRQ to allocate DMA channels and play around. But > that seems to be specific to ISA. Can it be used for PCI drivers as well? > No. ISA DMA is only for really old hardware without it's own DMA engine. Look at the busdma api/man page. Warner > Thanks, > Rajesh. > _______________________________________________ > freebsd-drivers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-drivers > To unsubscribe, send any mail to "freebsd-drivers-unsubscribe@freebsd.org" > From owner-freebsd-drivers@freebsd.org Fri Dec 14 15:35:19 2018 Return-Path: Delivered-To: freebsd-drivers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B0C6F1337E0C; Fri, 14 Dec 2018 15:35:19 +0000 (UTC) (envelope-from lidl@pix.net) Received: from hydra.pix.net (hydra.pix.net [IPv6:2001:470:e254:11::4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.pix.net", Issuer "Pix.Com Technologies LLC CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 2D3DA81FE3; Fri, 14 Dec 2018 15:35:19 +0000 (UTC) (envelope-from lidl@pix.net) Received: from torb.pix.net ([IPv6:2001:470:e254:10:702f:25e8:7b4f:59c6]) (authenticated bits=0) by hydra.pix.net (8.15.2/8.15.2) with ESMTPA id wBEFZ9B2072892; Fri, 14 Dec 2018 10:35:16 -0500 (EST) (envelope-from lidl@pix.net) X-Authentication-Warning: hydra.pix.net: Host [IPv6:2001:470:e254:10:702f:25e8:7b4f:59c6] claimed to be torb.pix.net Subject: Re: How to use the DMA Engine in FreeBSD? To: Warner Losh , Rajesh Kumar Cc: FreeBSD Hackers , freebsd-drivers@freebsd.org References: From: Kurt Lidl Message-ID: <26df913b-a2f8-2709-1cec-d11ad7d113a8@pix.net> Date: Fri, 14 Dec 2018 10:35:09 -0500 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:60.0) Gecko/20100101 Thunderbird/60.3.3 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Dec 2018 15:35:19 -0000 On 12/13/18 10:53 AM, Warner Losh wrote: > On Thu, Dec 13, 2018, 3:04 AM Rajesh Kumar >> Hi, >> >> Is there any good documentation available to understand the existing >> support, API's and how-to use the DMA Engine in FreeBSD? >> > > > Usually you just use pci busmastering and it just works. > > I am trying to write a test driver which will use DMA Engine to do the data >> transfer (rather than plain memcpy which involves cpu). Can anyone point >> to any driver implementation which has similar functions implemented? I >> see references to SYS_RES_DRQ to allocate DMA channels and play around. But >> that seems to be specific to ISA. Can it be used for PCI drivers as well? >> > > No. ISA DMA is only for really old hardware without it's own DMA engine. > > Look at the busdma api/man page. For some Intel based server hardware, there is the "ioat" driver, which allows for user code to schedule DMA operations. See ioat(4) for details, including a pointer to the test program. -Kurt From owner-freebsd-drivers@freebsd.org Fri Dec 14 16:16:29 2018 Return-Path: Delivered-To: freebsd-drivers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C984A133A823; Fri, 14 Dec 2018 16:16:29 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-lj1-f194.google.com (mail-lj1-f194.google.com [209.85.208.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48CD384A2F; Fri, 14 Dec 2018 16:16:29 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-lj1-f194.google.com with SMTP id v1-v6so5395173ljd.0; Fri, 14 Dec 2018 08:16:29 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Jejr1c7PhunWb6ZkdwVBhQmtzlD9MaCe/Ck1w0V5Dwg=; b=jiSu97OWKztqzjVVqx7NwM9M6G8hknqWcJjta33V+gUqGWhwwpcagn3l+w50xDSE1Q 1YjKsVYTioKf7v+jfwHgpOTQMzjBXpTWlMAF0tc0Hn3vQeEFR+hP5TggJL9JrpKyW5/J wd7Beccgr5i1Lml2WgT949IcuMuAY+E5LliR1LPAmMALuv8dQjLGQIxlAvgjh3bJzNS5 43+yBhPcuATvUkm1UrGFuKQVcZvb2IldQI02fVIp3RtQbTIIHM9+GGkM1RcuOvbCxNJI IovRtpljcyLV8QqG5e9FSCU+rVVkh+h6Y2JdoXzVqwOWQu272VxjBl8l1VcgZ1f+g7AV bryA== X-Gm-Message-State: AA+aEWaj88PM60PNqBQQVbtEsqMffIQEjAWsJ0XMbA+aPHafXWUWzZ/D ySZQvrO8LFf4Qd6sTrY35O29UxqMPxT0oJBQ6B4= X-Google-Smtp-Source: AFSGD/WsMu2bPw9w1+XIV72/rjnbYssNNFytmXEiqxqw/uhw5rhUCOgaDuuPkibDlI+EMVgIoVaN/lUuVrjiCW3Bog4= X-Received: by 2002:a2e:5418:: with SMTP id i24-v6mr2414937ljb.51.1544803769469; Fri, 14 Dec 2018 08:09:29 -0800 (PST) MIME-Version: 1.0 References: <26df913b-a2f8-2709-1cec-d11ad7d113a8@pix.net> In-Reply-To: <26df913b-a2f8-2709-1cec-d11ad7d113a8@pix.net> From: Alan Somers Date: Fri, 14 Dec 2018 09:09:17 -0700 Message-ID: Subject: Re: How to use the DMA Engine in FreeBSD? To: Kurt Lidl Cc: Warner Losh , rajfbsd@gmail.com, "freebsd-hackers@freebsd.org" , freebsd-drivers@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 48CD384A2F X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-5.90 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.995,0]; NEURAL_HAM_SHORT(-0.90)[-0.902,0]; REPLY(-4.00)[] X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Dec 2018 16:16:30 -0000 On Fri, Dec 14, 2018 at 8:36 AM Kurt Lidl wrote: > > On 12/13/18 10:53 AM, Warner Losh wrote: > > On Thu, Dec 13, 2018, 3:04 AM Rajesh Kumar > > >> Hi, > >> > >> Is there any good documentation available to understand the existing > >> support, API's and how-to use the DMA Engine in FreeBSD? > >> > > > > > > Usually you just use pci busmastering and it just works. > > > > I am trying to write a test driver which will use DMA Engine to do the data > >> transfer (rather than plain memcpy which involves cpu). Can anyone point > >> to any driver implementation which has similar functions implemented? I > >> see references to SYS_RES_DRQ to allocate DMA channels and play around. But > >> that seems to be specific to ISA. Can it be used for PCI drivers as well? > >> > > > > No. ISA DMA is only for really old hardware without it's own DMA engine. > > > > Look at the busdma api/man page. > > For some Intel based server hardware, there is the "ioat" driver, which > allows for user code to schedule DMA operations. See ioat(4) for > details, including a pointer to the test program. > > -Kurt ioat(4) looks cool. But the man page is vague on a few points. Do you know the answers to these questions? * What happened to ioatcontrol(8)? It's reference by the man page, but doesn't exist anywhere. * In what context are callbacks called? Are they called from a signal handler, or in a separate thread, or something else? * Why isn't ioat.h installed? * Are "interrupts" synonymous with callbacks? * Do you have a rough idea for about the minimum buffer size that makes sense to use with ioat? -Alan From owner-freebsd-drivers@freebsd.org Fri Dec 14 23:04:03 2018 Return-Path: Delivered-To: freebsd-drivers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2007C132381D; Fri, 14 Dec 2018 23:04:03 +0000 (UTC) (envelope-from lidl@pix.net) Received: from hydra.pix.net (hydra.pix.net [IPv6:2001:470:e254:11::4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.pix.net", Issuer "Pix.Com Technologies LLC CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 737FC6F8F0; Fri, 14 Dec 2018 23:04:02 +0000 (UTC) (envelope-from lidl@pix.net) Received: from torb.pix.net ([IPv6:2001:470:e254:10:702f:25e8:7b4f:59c6]) (authenticated bits=0) by hydra.pix.net (8.15.2/8.15.2) with ESMTPA id wBEN3pBB083640; Fri, 14 Dec 2018 18:03:58 -0500 (EST) (envelope-from lidl@pix.net) X-Authentication-Warning: hydra.pix.net: Host [IPv6:2001:470:e254:10:702f:25e8:7b4f:59c6] claimed to be torb.pix.net Subject: Re: How to use the DMA Engine in FreeBSD? To: Alan Somers Cc: Warner Losh , rajfbsd@gmail.com, "freebsd-hackers@freebsd.org" , freebsd-drivers@freebsd.org References: <26df913b-a2f8-2709-1cec-d11ad7d113a8@pix.net> From: Kurt Lidl Message-ID: Date: Fri, 14 Dec 2018 18:03:51 -0500 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:60.0) Gecko/20100101 Thunderbird/60.3.3 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Dec 2018 23:04:03 -0000 On 12/14/18 11:09 AM, Alan Somers wrote: > On Fri, Dec 14, 2018 at 8:36 AM Kurt Lidl wrote: >> >> On 12/13/18 10:53 AM, Warner Losh wrote: >>> On Thu, Dec 13, 2018, 3:04 AM Rajesh Kumar >> >>>> Hi, >>>> >>>> Is there any good documentation available to understand the existing >>>> support, API's and how-to use the DMA Engine in FreeBSD? >>>> >>> >>> >>> Usually you just use pci busmastering and it just works. >>> >>> I am trying to write a test driver which will use DMA Engine to do the data >>>> transfer (rather than plain memcpy which involves cpu). Can anyone point >>>> to any driver implementation which has similar functions implemented? I >>>> see references to SYS_RES_DRQ to allocate DMA channels and play around. But >>>> that seems to be specific to ISA. Can it be used for PCI drivers as well? >>>> >>> >>> No. ISA DMA is only for really old hardware without it's own DMA engine. >>> >>> Look at the busdma api/man page. >> >> For some Intel based server hardware, there is the "ioat" driver, which >> allows for user code to schedule DMA operations. See ioat(4) for >> details, including a pointer to the test program. >> >> -Kurt > > ioat(4) looks cool. But the man page is vague on a few points. Do > you know the answers to these questions? > * What happened to ioatcontrol(8)? It's reference by the man page, > but doesn't exist anywhere. root@busybox: locate ioatcontrol /usr/src/tools/tools/ioat/ioatcontrol.8 /usr/src/tools/tools/ioat/ioatcontrol.c > * In what context are callbacks called? Are they called from a signal > handler, or in a separate thread, or something else? I don't know. > * Why isn't ioat.h installed? I don't know that either, but it is in the source tree: root@busybox: locate ioat.h /usr/src/sys/dev/ioat/ioat.h > * Are "interrupts" synonymous with callbacks? > * Do you have a rough idea for about the minimum buffer size that > makes sense to use with ioat? I don't know that either -- I was mostly just pointing out that the facility and driver existed for (some) Intel server hardware. Mostly I'm aware of this because I was surprised when a machine I using this summer started reporting this hardware, and I wasn't familiar with it. Good luck. -Kurt From owner-freebsd-drivers@freebsd.org Sat Dec 15 00:59:56 2018 Return-Path: Delivered-To: freebsd-drivers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9382A13278DB; Sat, 15 Dec 2018 00:59:56 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: from mail-it1-f179.google.com (mail-it1-f179.google.com [209.85.166.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9366B73870; Sat, 15 Dec 2018 00:59:55 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: by mail-it1-f179.google.com with SMTP id c9so12298907itj.1; Fri, 14 Dec 2018 16:59:55 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:reply-to :from:date:message-id:subject:to:cc; bh=EU5UMKETp3I2V730UX34nZFrsQLTwDvGGqEmFD4No18=; b=nFyfrZSodB8TGGdP4cdVuI44Wax0iYG3DUe611GXw2Y3l5b/1Q72SMrM6vRU4o4TaZ cZLwlM+/Djb7WVYL44fRBC4mgQPByFON1jN+RIoHmVRdCyKdDqluMvZyjcsLUdWtJo6f tuYBdC5M4FPhl1ZEfeKTSlepUxU7d8S1AtKQm5qZd5J6TIDFCv3eOnObqu/6OVgn8481 6eoDFAzkDM1ufuOzwIQxTusYFwl2BHPod/HURYsx2IC1KgTR/lpPwaYF+jdEXAlwVr9a k06ribbhJ2tzp0x+R40kw2vX825zM6cAlXRApmU2yK2IuE9srSnoRiOWEW86TKj3rGNa UWKw== X-Gm-Message-State: AA+aEWbOOW/aqyXmoX+l7XgmKzY1rpK+Q4e0wyB916blsRCRHRpUyPSU 7aDyxzkMg0Chmo4M6jP9jSEaFfnS X-Google-Smtp-Source: AFSGD/XUX3GrM+fBKa+bw8roDrvBqNMlKlvrkvbC79tNWb9nI696Zz6ZOOvL5qEmnSirJqgx2V/auA== X-Received: by 2002:a24:4ac3:: with SMTP id k186mr4514863itb.76.1544835588913; Fri, 14 Dec 2018 16:59:48 -0800 (PST) Received: from mail-io1-f51.google.com (mail-io1-f51.google.com. [209.85.166.51]) by smtp.gmail.com with ESMTPSA id 196sm3419524itu.37.2018.12.14.16.59.48 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 14 Dec 2018 16:59:48 -0800 (PST) Received: by mail-io1-f51.google.com with SMTP id o13so2406987iom.4; Fri, 14 Dec 2018 16:59:48 -0800 (PST) X-Received: by 2002:a6b:6111:: with SMTP id v17mr4620029iob.107.1544835588458; Fri, 14 Dec 2018 16:59:48 -0800 (PST) MIME-Version: 1.0 References: <26df913b-a2f8-2709-1cec-d11ad7d113a8@pix.net> In-Reply-To: Reply-To: cem@freebsd.org From: Conrad Meyer Date: Fri, 14 Dec 2018 16:59:37 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: How to use the DMA Engine in FreeBSD? To: Alan Somers Cc: "freebsd-hackers@freebsd.org" , freebsd-drivers@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 9366B73870 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of csecem@gmail.com designates 209.85.166.179 as permitted sender) smtp.mailfrom=csecem@gmail.com X-Spamd-Result: default: False [-4.83 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_REPLYTO(0.00)[cem@freebsd.org]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; REPLYTO_ADDR_EQ_FROM(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; NEURAL_HAM_SHORT(-0.95)[-0.948,0]; FORGED_SENDER(0.30)[cem@freebsd.org,csecem@gmail.com]; SUBJECT_ENDS_QUESTION(1.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; R_DKIM_NA(0.00)[]; TAGGED_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_NEQ_ENVFROM(0.00)[cem@freebsd.org,csecem@gmail.com]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; DMARC_NA(0.00)[freebsd.org]; RCVD_TLS_LAST(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[179.166.85.209.list.dnswl.org : 127.0.5.0]; IP_SCORE(-2.87)[ip: (-9.42), ipnet: 209.85.128.0/17(-3.53), asn: 15169(-1.33), country: US(-0.09)]; RWL_MAILSPIKE_POSSIBLE(0.00)[179.166.85.209.rep.mailspike.net : 127.0.0.17] X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Dec 2018 00:59:56 -0000 On Fri, Dec 14, 2018 at 8:17 AM Alan Somers wrote: > > For some Intel based server hardware, there is the "ioat" driver, which > > allows for user code to schedule DMA operations. See ioat(4) for > > details, including a pointer to the test program. This isn't true (or at least, only for testing). It's only usable by the kernel at the moment. > * In what context are callbacks called? Are they called from a signal > handler, or in a separate thread, or something else? Directly from the ithread, mostly. Callers can't take sleep locks or do very much besides set a flag and wakeup, or kick off a callout or taskqueue. > * Why isn't ioat.h installed? Why would it be? It's a kernel interface. > * Are "interrupts" synonymous with callbacks? I don't understand the question. Interrupts are synonymous with interrupts. Like, MSI-X. > * Do you have a rough idea for about the minimum buffer size that > makes sense to use with ioat? What makes sense will depend on your use case. It's not necessarily faster than CPU memcpy, but it may be worth some slowdown to offload latency-insensitive bulk copies. (It may make a lot of sense for asynchronous page zero, if someone wants to bring that back.) It's worth benchmarking your specific model. We use it for >= 8kB copies on Broadwell, although that's largely an artifact of our use case (write sizes are exactly 8 bytes, 512 bytes, or multiples of 8kB). IIRC, it shows up as a gen3 PCIe device with some small number of lanes on Broadwell, although I may be mistaken; and it may not communicate with RAM over the PCIe bus anyway, given the tight integration with the CPU. Best, Conrad From owner-freebsd-drivers@freebsd.org Sat Dec 15 01:02:24 2018 Return-Path: Delivered-To: freebsd-drivers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0EDF81327C11; Sat, 15 Dec 2018 01:02:24 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-lj1-f196.google.com (mail-lj1-f196.google.com [209.85.208.196]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7C9DD73C45; Sat, 15 Dec 2018 01:02:23 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-lj1-f196.google.com with SMTP id e5-v6so6408691lja.4; Fri, 14 Dec 2018 17:02:23 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=YksFTx6dp36biHe3ZqVMIxRi4GshjX65ibeFAbvHzfY=; b=h567i4G4KICLYY+A4tASOncpzX/1BwFyCPyMa2N1MG/wnfMfDvPmS9cJQUik5ZEAlI ryP4FurOyNzZKjKQnpWZ4oPpe7mL8Pu65Z2VX0GEB4ZYgy+RiJQxYrXgCJ++RJlJAfRy ee4vqOr5BW3xd7/SD0LdKnrdC1Cs+I7K/gt14hwu6vy9jmWx2jaEPZlBtM5p3pACStH8 HV12bdhxXaKpepNBMzLHdIdHfNLPDDFr2l/GLqThLEXch2GkYBj2JIxpNquhH4sP29oB vXEBLQZW02vTD+eyue6F5n+V/1J/VuE8EG+D9IJkNHKLEw+N0Vt0YIOvehqvQ8XhTPko bRhA== X-Gm-Message-State: AA+aEWaNTy8x3R5p5e5CnyqgJ1cHg1ksYwJK4qzTSPXtNFsjgkre9HmJ E7FWCp8WPsAIVG2nFk+uf3sOaF5da2hRcQxOwvmOsA== X-Google-Smtp-Source: AFSGD/WCE+5K7BcFi8Cyl8coRf9Ci1dJ3movDX6EU4eW5/DExTIryXBDKYKIp0ZTlnqJelU5MVUGiHNyhnpofchi1QA= X-Received: by 2002:a2e:94ce:: with SMTP id r14-v6mr2635221ljh.34.1544831841853; Fri, 14 Dec 2018 15:57:21 -0800 (PST) MIME-Version: 1.0 References: <26df913b-a2f8-2709-1cec-d11ad7d113a8@pix.net> In-Reply-To: From: Alan Somers Date: Fri, 14 Dec 2018 16:57:09 -0700 Message-ID: Subject: Re: How to use the DMA Engine in FreeBSD? To: Kurt Lidl Cc: Warner Losh , Rajesh Kumar , "freebsd-hackers@freebsd.org" , freebsd-drivers@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 7C9DD73C45 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.99 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.99)[-0.993,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-0.997,0] X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Dec 2018 01:02:24 -0000 On Fri, Dec 14, 2018 at 4:04 PM Kurt Lidl wrote: > > On 12/14/18 11:09 AM, Alan Somers wrote: > > On Fri, Dec 14, 2018 at 8:36 AM Kurt Lidl wrote: > >> > >> On 12/13/18 10:53 AM, Warner Losh wrote: > >>> On Thu, Dec 13, 2018, 3:04 AM Rajesh Kumar >>> > >>>> Hi, > >>>> > >>>> Is there any good documentation available to understand the existing > >>>> support, API's and how-to use the DMA Engine in FreeBSD? > >>>> > >>> > >>> > >>> Usually you just use pci busmastering and it just works. > >>> > >>> I am trying to write a test driver which will use DMA Engine to do the data > >>>> transfer (rather than plain memcpy which involves cpu). Can anyone point > >>>> to any driver implementation which has similar functions implemented? I > >>>> see references to SYS_RES_DRQ to allocate DMA channels and play around. But > >>>> that seems to be specific to ISA. Can it be used for PCI drivers as well? > >>>> > >>> > >>> No. ISA DMA is only for really old hardware without it's own DMA engine. > >>> > >>> Look at the busdma api/man page. > >> > >> For some Intel based server hardware, there is the "ioat" driver, which > >> allows for user code to schedule DMA operations. See ioat(4) for > >> details, including a pointer to the test program. > >> > >> -Kurt > > > > ioat(4) looks cool. But the man page is vague on a few points. Do > > you know the answers to these questions? > > * What happened to ioatcontrol(8)? It's reference by the man page, > > but doesn't exist anywhere. > > root@busybox: locate ioatcontrol > /usr/src/tools/tools/ioat/ioatcontrol.8 > /usr/src/tools/tools/ioat/ioatcontrol.c > > > * In what context are callbacks called? Are they called from a signal > > handler, or in a separate thread, or something else? > > I don't know. > > > * Why isn't ioat.h installed? > > I don't know that either, but it is in the source tree: > > root@busybox: locate ioat.h > /usr/src/sys/dev/ioat/ioat.h > > > * Are "interrupts" synonymous with callbacks? > > * Do you have a rough idea for about the minimum buffer size that > > makes sense to use with ioat? > > I don't know that either -- I was mostly just pointing out that the > facility and driver existed for (some) Intel server hardware. Mostly > I'm aware of this because I was surprised when a machine I using this > summer started reporting this hardware, and I wasn't familiar with it. > > Good luck. > > -Kurt Oh, I see. ioat(4) can't be used from userland at all. It's strictly an in-kernel API. The man page should probably be moved to section 9. I'll take it up with cem. -Alan