From owner-freebsd-hackers@freebsd.org Sun Mar 1 09:55:38 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AC33526B845 for ; Sun, 1 Mar 2020 09:55:38 +0000 (UTC) (envelope-from salvadore@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48Vdty3WJkz43X5 for ; Sun, 1 Mar 2020 09:55:38 +0000 (UTC) (envelope-from salvadore@freebsd.org) Received: by freefall.freebsd.org (Postfix, from userid 1472) id EE6EB168BE; Sun, 1 Mar 2020 09:55:36 +0000 (UTC) To: freebsd-hackers@FreeBSD.org Subject: Call for 2020Q1 quarterly status reports Message-Id: <20200301095537.EE6EB168BE@freefall.freebsd.org> Date: Sun, 1 Mar 2020 09:55:36 +0000 (UTC) From: Lorenzo Salvadore X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Mar 2020 09:55:38 -0000 Dear FreeBSD Community, The deadline for the next FreeBSD Quarterly Status update is April 1, 2020, for work done since the last round of Quarterly Reports: January, 2020 - March, 2020. I would like to remind you that reports are collected during the last month of every quarter. Status report submissions do not need to be very long. They may be about anything happening in the FreeBSD project and community, and they provide a great way to inform FreeBSD users and developers about work that is underway or has been completed. Report submissions are not limited to committers; anyone doing anything interesting and FreeBSD related can -- and should -- write one! The preferred method is to follow the guidelines at the Quarterly GitHub repository: https://github.com/freebsd/freebsd-quarterly Alternatively you can fetch the Markdown template, fill it in, and email it to quarterly@FreeBSD.org. The template can be found at: https://raw.githubusercontent.com/freebsd/freebsd-quarterly/master/report-sample.md We look forward to seeing your 2020Q1 reports! Thanks, Lorenzo Salvadore (on behalf of quarterly@) From owner-freebsd-hackers@freebsd.org Sun Mar 1 23:08:06 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 38C3C2516F1; Sun, 1 Mar 2020 23:08:06 +0000 (UTC) (envelope-from rajfbsd@gmail.com) Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [IPv6:2a00:1450:4864:20::332]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48VzTH5nHWz4CBn; Sun, 1 Mar 2020 23:08:03 +0000 (UTC) (envelope-from rajfbsd@gmail.com) Received: by mail-wm1-x332.google.com with SMTP id 6so709247wmi.5; Sun, 01 Mar 2020 15:08:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=TfanTrN2Blo9Kozjc1qqPPtADXhQbomL9tGQ1PXhYNY=; b=Few4htTciWeYGC5FU34iyc/ZxqBOEGn/KhXeeuiNj1NGHl++zqmD7gTHPBmRG34IwM PyQ70ZZxloyBRbYtl6L6FvzRuT+lCpPmd+vw4wQ84PMVrfnN7gYCtF2IQLfy78YFs1Ig 0WFL6JdooZINdMALI6CBG2+l2ZveLsHWvHlgQqEXI3JmmNhTkL27EheUpkep/xARb8Ij 2PqGTZSDSiwLheOyG5wlyi+4v6Hl3oH0+aZh2NPVIf3kun02m6RuBzjW1knpdBUK4jva 6aAuj7K51tl8lZxNglMJk+Bmw2NuppLGil+GWd9wRkrr8I1M9ypZg9ywgS4ajkAkoCHx c8qw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=TfanTrN2Blo9Kozjc1qqPPtADXhQbomL9tGQ1PXhYNY=; b=AxFH28x1kf6cazOfLKrquc5zty5/GpL/yTkcPZdLNBzUf6OdJMtfLfsjipilWL4IhS WKQT2J5CjGtmHqh10HHjKM9QG9xLbZNbmskRbOXIEyYdufyvGX5ZziYIpE3x/3ZAen14 xLv78ixoHTOVWDHJkV/qPT+75sL9fluj4xbiMrurSuXXoixbdvWZHrePr+ZoHVPPwPmc 3HDBpHIMyjYhWWziyANyyi9xU6du49qUOPdZBs/dtIQ9QRRAWsXSCS7EltxyQjgUX0ax ly+lFcUPrqRTarF0OmJ/kghMBSnJ4bLZFiKIQYYGPcHz1NhT6JQi5cb4Qn7VBV+tAqf1 xZJw== X-Gm-Message-State: APjAAAWmVf7vn828aL9dTuf85+x3dq+YW+zDgEL5mscHeA5akU4t4xft z7Q+GukqeNmjw05T3MHrCXAyrrsg4avM/3JjUWIZWA== X-Google-Smtp-Source: APXvYqyHpm46QhtP6+AlsSJEN/Mx8X57UcPTO5tjMeVMghFxdOAZ5KPUsOQHuWld29UjleTNpMh1dIzxXKodQYKhaPw= X-Received: by 2002:a1c:6a15:: with SMTP id f21mr15676114wmc.126.1583104079815; Sun, 01 Mar 2020 15:07:59 -0800 (PST) MIME-Version: 1.0 References: <7cfc7c52-b548-19bd-343b-899aca45c654@selasky.org> In-Reply-To: From: Rajesh Kumar Date: Mon, 2 Mar 2020 04:37:48 +0530 Message-ID: Subject: Re: Network throughput not reaching line rate. Need clarification on iflib. To: Daniel Ebdrup Jensen Cc: freebsd-drivers@freebsd.org, FreeBSD Hackers X-Rspamd-Queue-Id: 48VzTH5nHWz4CBn X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=Few4htTc; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of rajfbsd@gmail.com designates 2a00:1450:4864:20::332 as permitted sender) smtp.mailfrom=rajfbsd@gmail.com X-Spamd-Result: default: False [-2.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_DN_SOME(0.00)[]; URI_COUNT_ODD(1.00)[9]; IP_SCORE_FREEMAIL(0.00)[]; IP_SCORE(0.00)[ip: (-9.06), ipnet: 2a00:1450::/32(-2.41), asn: 15169(-1.67), country: US(-0.05)]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[2.3.3.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Mar 2020 23:08:06 -0000 Hi Guys, Thanks for your time in responding. As you people stated, the problem here doesn't seem to be single threaded vs multi threaded. I am experience similar behavior with "iperf". It's mostly to do with iflib and my driver(mostly my driver). It looks like the receiver is slow in reading the packets than the sender sending the packets. In this case, iflib drives the packet read through the exposed interfaces (rxd_available, rxd_pkt_get etc.,). So, how to make the receiver side read the packet faster with iflib? Is there anything that I should take care in my driver in this regard? Thanks, Rajesh. On Sat, Feb 29, 2020 at 3:47 AM Daniel Ebdrup Jensen wrote: > On Fri, Feb 28, 2020 at 7:39 PM Bruce A. Mah wrote: > > > [Resending with a From: address that hopefully works better.] > > > > If memory serves me right, Daniel Ebdrup Jensen wrote: > > > Yes, iperf3 will default to single-threaded packet generation, et al. > > which > > > favours fast cores with frequency boosting facilities. > > > You might want to use iperf2 as that's properly multi-threaded, or you > > can > > > use pkt-gen out of src/tools/tools/netmap/ or ports/net/pkt-gen. > > > > While it's true that iperf3 is single-threaded, it should be capable of > > saturating a 10GE link with a single TCP connection, given proper > > command-line arguments (in particular, specifying a sufficiently large > > socket-buffer size with the -w option). > > > > But based on the symptom of packet loss, I'd say the single-threaded vs. > > multi-threaded argument might not be relevant to the problem that the OP > > has. > > > > Bruce. > > > > > On Fri, Feb 28, 2020 at 10:35 AM Hans Petter Selasky > > > wrote: > > > > > >> On 2020-02-28 10:03, Rajesh Kumar wrote: > > >>> Hi FreeBSD team, > > >>> > > >>> I am writing a network driver using iflib framework and using > "iperf3" > > >> tool > > >>> for performance testing. > > >>> > > >> > > >> Is there any difference with "iperf" tool and using multiple threads? > I > > >> think iperf3 is single threaded ??? > > >> > > >> --HPS > > >> > > >> _______________________________________________ > > >> freebsd-hackers@freebsd.org mailing list > > >> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > >> To unsubscribe, send any mail to " > > freebsd-hackers-unsubscribe@freebsd.org" > > >> > > > _______________________________________________ > > > freebsd-hackers@freebsd.org mailing list > > > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > > To unsubscribe, send any mail to " > > freebsd-hackers-unsubscribe@freebsd.org" > > > > > > > Oh, I didn't mean to imply that that wasn't part of the issue - I'm sorry > if I made it sound like that. > I was just confirming what Hans was asking, and possibly using the excuse > to mention some things in base/ports that I think are also pretty neat. :) > > Also no longer top-posting, which was rather ghastly of me. I apologise. > _______________________________________________ > freebsd-drivers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-drivers > To unsubscribe, send any mail to "freebsd-drivers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@freebsd.org Mon Mar 2 00:46:38 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8F9042542FF for ; Mon, 2 Mar 2020 00:46:38 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from smtp.digiware.nl (smtp.digiware.nl [IPv6:2001:4cb8:90:ffff::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 48W1g04RYqz45yj for ; Mon, 2 Mar 2020 00:46:36 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from router.digiware.nl (localhost.digiware.nl [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 3A2A673395; Mon, 2 Mar 2020 01:46:27 +0100 (CET) X-Virus-Scanned: amavisd-new at digiware.com Received: from smtp.digiware.nl ([127.0.0.1]) by router.digiware.nl (router.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GyDUa8jw8M4C; Mon, 2 Mar 2020 01:46:26 +0100 (CET) Received: from [192.168.10.9] (vaio [192.168.10.9]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id AB2C673391 for ; Mon, 2 Mar 2020 01:46:26 +0100 (CET) To: FreeBSD Hackers From: Willem Jan Withagen Subject: What is the exact function of the offset parameter in preadv?? Message-ID: Date: Mon, 2 Mar 2020 01:46:25 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Firefox/68.0 Thunderbird/68.5.0 MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Content-Language: nl X-Rspamd-Queue-Id: 48W1g04RYqz45yj X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of wjw@digiware.nl designates 2001:4cb8:90:ffff::3 as permitted sender) smtp.mailfrom=wjw@digiware.nl X-Spamd-Result: default: False [-4.36 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.996,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[4]; RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[]; DMARC_NA(0.00)[digiware.nl]; IP_SCORE(-3.07)[ip: (-9.44), ipnet: 2001:4cb8::/29(-4.62), asn: 28878(-1.30), country: NL(0.03)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:28878, ipnet:2001:4cb8::/29, country:NL]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2020 00:46:38 -0000 Hi, Tried looking in the manual page, but it does not really describes what off_t offset does with a preadv... I suspect that is the offset in the file to start reading from? Next question is: What if the value is negative?     Are we then reading from the end of the file/device if it is     seekable? Manual page says:  The pread() and preadv() system calls may also return the following      errors:      [EINVAL]           The offset value was negative. But then I have this behyve code that gets calls like: block_proc_read: buf=0x0, cnt=1, len=512, off=-1024(0xfffffc00). block_proc_read: buf=0x0, cnt=1, len=512, off=-512(0xfffffe00). block_proc_read: buf=0x0, cnt=1, len=512, off=-1024(0xfffffc00). block_proc_read: buf=0x0, cnt=1, len=512, off=-512(0xfffffe00). block_proc_read: buf=0x0, cnt=1, len=512, off=-512(0xfffffe00). block_proc_read: buf=0x0, cnt=1, len=2048, off=-32256(0xffff8200). block_proc_read: buf=0x0, cnt=1, len=512, off=512(0x200). block_proc_read: buf=0x0, cnt=1, len=512, off=0(0x0). block_proc_read: buf=0x0, cnt=1, len=512, off=512(0x200). block_proc_read: buf=0x0, cnt=1, len=512, off=0(0x0). block_proc_read: buf=0x0, cnt=1, len=512, off=512(0x200). block_proc_read: buf=0x0, cnt=1, len=16384, off=1024(0x400). block_proc_read: buf=0x0, cnt=1, len=512, off=-512(0xfffffe00). block_proc_read: buf=0x0, cnt=1, len=16384, off=-16896(0xffffbe00). block_proc_read: buf=0x0, cnt=1, len=512, off=-512(0xfffffe00). Where these are the very first calls that a bhyve loaded image executes. which I guess is for probing the bootdisk to see where/what it needs to mount a root disk. Below bhyve boot output of a not working boot, and a working boot. The read trace above is from the working bhyve boot. So what are the negative offsets? --WjW Compare Using block_rbd_read for reading from a Rados device ============= vtblk0: on virtio_pci1 vtblk0: 134217727MB (34359738366 4096 byte sectors) ahci0: mem 0xc0004000-0xc00043ff irq 18 at device 4.0 on pci0 ahci0: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] driver bug: Unable to set devclass (class: atkbdc devname: (unknown)) psm0: irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: model Generic PS/2 mouse, device ID 0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart0: console (9600,n,8,1) uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 vga0: at port 0x3b0-0x3bb iomem 0xb0000-0xb7fff pnpid PNP0900 on isa0 Timecounters tick every 10.000 msec usb_needs_explore_all: no devclass Trying to mount root from ufs:/dev/ada0p2 [rw]... mountroot: waiting for device /dev/ada0p2... Mounting from ufs:/dev/ada0p2 failed with error 19. Loader variables:   vfs.root.mountfrom=ufs:/dev/ada0p2   vfs.root.mountfrom.options=rw Manual root filesystem specification:   : [options]       Mount using filesystem       and with the specified (optional) option list.     eg. ufs:/dev/da0s1a         zfs:zroot/ROOT/default         cd9660:/dev/cd0 ro           (which is equivalent to: mount -t cd9660 -o ro /dev/cd0 /)   ?               List valid disk boot devices   .               Yield 1 second (for background tasks)       Abort manual input ============= With booting from a normal file device: atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] driver bug: Unable to set devclass (class: atkbdc devname: (unknown)) psm0: irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: model Generic PS/2 mouse, device ID 0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart0: console (9600,n,8,1) uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 vga0: at port 0x3b0-0x3bb iomem 0xb0000-0xb7fff pnpid PNP0900 on isa0 Timecounters tick every 10.000 msec usb_needs_explore_all: no devclass Trying to mount root from ufs:/dev/ada0p2 [rw]... Root mount waiting for: CAM ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 ada0: ACS-2 ATA SATA 3.x device ada0: Serial Number BHYVE-8C7D-D922-AD47 ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 20480MB (41943040 512 byte sectors) ada1 at ahcich6 bus 0 scbus1 target 0 lun 0 ada1: ACS-2 ATA SATA 3.x device ada1: Serial Number BHYVE-4318-7904-5D51 ada1: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) ada1: Command Queueing enabled ada1: 20480MB (41943040 512 byte sectors) mountroot: waiting for device /dev/ada0p2... WARNING: / was not properly dismounted WARNING: /: mount pending error: blocks 168 files 1 ========== From owner-freebsd-hackers@freebsd.org Mon Mar 2 12:09:53 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 631061AC8FB for ; Mon, 2 Mar 2020 12:09:53 +0000 (UTC) (envelope-from ben.rubson@gmx.com) Received: from mout.gmx.net (mout.gmx.net [212.227.17.20]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass Class 2 CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48WJqM06jbz4GwY for ; Mon, 2 Mar 2020 12:09:50 +0000 (UTC) (envelope-from ben.rubson@gmx.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1583150988; bh=HyRL5/XhKoohaKdCYKh3kvjs0mHdBI4zxhVGi3KgyyI=; h=X-UI-Sender-Class:From:Subject:Date:To; b=RfXhphtwuy/oFBlqbUnA0VmOARImTx3yPPForaGuuk9Z7BgOhXz3hB3CLB0kf0IaS POlDefFPnjw37iCJgGPs67reWwElp9uONYOweHlMCLq4FLLEsbL6CDkQVwdmUwxMSt RRqBrsGekSsJLfBqpJgUCTa9YypPma9rJYs/3r8k= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [192.168.0.101] ([82.64.198.151]) by mail.gmx.com (mrgmx104 [212.227.17.174]) with ESMTPSA (Nemesis) id 1MAfUe-1jFJTQ1VJQ-00B7EI for ; Mon, 02 Mar 2020 13:09:48 +0100 From: Ben RUBSON Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 13.0 \(3608.60.0.2.5\)) Subject: Allow to run SSHd in Installer (12.2 patch) Message-Id: <2352A2A0-999C-453F-92A1-D067E4C05712@gmx.com> Date: Mon, 2 Mar 2020 13:09:47 +0100 To: freebsd-hackers@freebsd.org X-Mailer: Apple Mail (2.3608.60.0.2.5) X-Provags-ID: V03:K1:xFdvcDSk0b9jnv5hKcog6Qd/Iv75PVMfGiPa3XzjSkpPfJvMgXK uVd+pdbgcLbsfljkVJXecNhU+04DchryE4qaR3Qj7NXgNM47Fyge6oVE9CxoIBBfWEg4Gmh qE56Ob69tt9snTZi6Nxv3UJ46FswrTGIq3yc/BLCgJUqJrF0z8fCRfltumMD5LHGnJckTnW T993Caraq521xkpj9NWVw== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:oe3E7M4vE+Y=:RdN6FMdjIoQR+vWx3Pb1+l heQbw5a8DTZnAruZT7Ppl9/ppS6oqJFxJGvdwl4A96jT9NV6cocBCmRCwJJ/iIuAMWAMjKs9i GNY3HHfNX7saI7Yk0C3DAXv+SbVV/lMqGqdx/p5z4e1FsW5cLUoDHvMIHmpgaBjChLiytC58h B2W2wzriXLr0nm+A5f03gKyqeJ4HYfeY2wBWuAv4uRdA+S8NEaYIcXziNiE3FWOZI2akYrwRU g8F28BAuf5wUnm9LJaEC52jxwJEZnPVmlnzOp9KNbDbuNOQUcyZKHo6R5ux5CS33MuDK2f/4i YwLbjfWpiXCbJW1mTKqhUZtDOFoM44frg2gZHPqaT+8E6fdtX4Ite/nQnRf8Bwe47v339cRvF 1bGAjMX6oO8AMpMo+dWviLkQaUcv1wbCcZaU4oHI+x1W2cuGzEfvGD65t+vtM6O0oLkOHiv/D QucKL89rMt5SHHJqqTG2tblMUOfm2g6UzlBFub1u7l9TwhAiCkDzb3zFlwDWsJ3vHBG9IZUTn naxniqHH0GQq+FHaa7ytOJy0pwJLITsfSf9Bb7cJhcXaV1aM8ey/luKKMoczcDxWmypjwnUDl Rci02gT+DfPAR0oiJLQJ/IpGRDSGUwZwNXYZDadH19o+IhWTXtgb40KUhkqQRQIzrQMuEU9ms l17Eu9KM0l2LRXj7lYx5Zx7eZ3L8q1nHQuxBetQMyxDJkwpXiVRVyY5Qgugm0OaVLGPFzuoUy 15DZQLyoGh6bxrVKYiUc+SKZmv8LSGml0xvuF+xbRHXUaSVKrWJs75oWJyjwozKlZp9Zqjyrg h3gSnbpW22/dbQBKDyakhIuYLWnGzSRXRyAHSjnCQNx6/0U8QI+dTeQ81wZm6zAu5iAbUzEp/ kXS9F1mwk/UNSn/pYc9WE3cXUjOaPzyYf0+D/SyKXXTcpqf1nf9uWLceNVe0jcQkopxDBQ7JG hXanxApx3rfQrQpn35povg61ru7QWSk/fzvuuXoLGZKICz8pZzgYBbrp9mSWG3R6CJ3majl23 hLpz+NUqBBNM5bwklfsP72qOrdM/fUvU5Dwt605moQU/+ueuYr1Bao4aJXiNMnIXt/Ni91tz/ wRQXDbrKakE/zeScnmbQPQMPVUO68z1XoG1vWf8at+as7y5Jvq7Nmmp9tnlpGvZ76pkj4lKu1 RHlDSVyltvlfq4xyr7QcNUgIFZlexDsC44JeuToAzoAwMMfcf9Ctb8Ogm69ej0mzCOGqVy6Be lwid7WII6Ti9lNGFk X-Rspamd-Queue-Id: 48WJqM06jbz4GwY X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmx.net header.s=badeba3b8450 header.b=RfXhphtw; dmarc=none; spf=pass (mx1.freebsd.org: domain of ben.rubson@gmx.com designates 212.227.17.20 as permitted sender) smtp.mailfrom=ben.rubson@gmx.com X-Spamd-Result: default: False [-2.09 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:212.227.17.0/27]; FREEMAIL_FROM(0.00)[gmx.com]; TO_DN_NONE(0.00)[]; MV_CASE(0.50)[]; DKIM_TRACE(0.00)[gmx.net:+]; RECEIVED_SPAMHAUS_PBL(0.00)[151.198.64.82.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.11]; RCVD_IN_DNSWL_LOW(-0.10)[20.17.227.212.list.dnswl.org : 127.0.3.1]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:8560, ipnet:212.227.0.0/16, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmx.com]; R_DKIM_ALLOW(-0.20)[gmx.net:s=badeba3b8450]; NEURAL_HAM_MEDIUM(-0.99)[-0.995,0]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[gmx.com]; IP_SCORE_FREEMAIL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(0.00)[ip: (-6.57), ipnet: 212.227.0.0/16(-1.15), asn: 8560(2.17), country: DE(-0.02)]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2020 12:09:53 -0000 Hi, I've done some work to allow to connect to FreeBSD installer through = SSH. It can be useful for example if we have specific tasks to perform before = installation, such as disks configuration etc... Working through a SSH connection is much more convenient than in front = of a console. FreeBSD installer can then also be used as a rescue disk. To achieve this, I've modified FreeBSD installer, so that after having = installed SSHd, if performs following configuration modifications : - generate host keys into /var/ssh (as default /etc/ssh is not writable) = ; - only allow keys authentication ; - allow root authentication ; - read authorized_keys file from /var/ssh (as default homedirs are not = writable). SSHd can then be started thanks to the installer shell : service sshd = start And a public key put into for example = /var/ssh-keys/root/authorized_keys, thanks to fetch or whatever. Work is here : https://github.com/freebsd/freebsd/pull/156 Rather simple, and ready to be merged. This job is more than 2 years old, I would then really be glad if we = could see this in 12.2 installation ISOs. It would prevent me from having to modify the new ISO files to implement = this patch. Many thanks ! Best regards, Ben From owner-freebsd-hackers@freebsd.org Tue Mar 3 00:35:40 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D86762602D7 for ; Tue, 3 Mar 2020 00:35:40 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: from mail-ot1-f67.google.com (mail-ot1-f67.google.com [209.85.210.67]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48WdMv1gC7z3FZ8 for ; Tue, 3 Mar 2020 00:35:38 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: by mail-ot1-f67.google.com with SMTP id j5so1262630otn.10 for ; Mon, 02 Mar 2020 16:35:38 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:reply-to :from:date:message-id:subject:to:cc; bh=VVIzGndlgdEh3gZyvaoQ/53wbUzlLhqnmE6LnA2ZSY0=; b=HiL5eOdjZINzFExK9xbxtAEbJj3G6XHoxp4Rx6uhUH7J5g2w4WJYBvMXwKvvaShQRA jtVNZXpnxnwHHOHBXo9jmkzZZl5I0JcsYK2wBxsUHZpHA+5aU3l+q58qFIuuMs1xzABf d1bDs59V5dOdU1bFjxBUvxFhYo5HA/berIgE8+ABvAVNcZX1P9hbysB3A8OfvBnsOjtD hRm0UaMjuLLSxw/qO6w3x1b5nJPGJpt+0dP3nfeq6YR5E33OuSo62lmySe64g5IW9hzP lfDQw1uxbu3KepVZRZe1tc4ho8c7TctMtoyEkj3SkdBT7Yj1W96CCdEV87vsEXT4157E gTBQ== X-Gm-Message-State: ANhLgQ0sh1AM87Xn4kbIDS9B2mzXfV5NckcEjl9dr3wdday7IszlhlFT 1Y3AFD08ZNP3bY4o+fGQU4Jx6gtv X-Google-Smtp-Source: ADFU+vu/RXHNRZacafWocOBhij7yvOcOLXRPVdhJ7safcjmSFrU1Hy+4KESy4Nw7N8qJjiW48qxWGw== X-Received: by 2002:a9d:2028:: with SMTP id n37mr1486790ota.127.1583195737254; Mon, 02 Mar 2020 16:35:37 -0800 (PST) Received: from mail-oi1-f174.google.com (mail-oi1-f174.google.com. [209.85.167.174]) by smtp.gmail.com with ESMTPSA id o15sm4824530ote.2.2020.03.02.16.35.36 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 02 Mar 2020 16:35:37 -0800 (PST) Received: by mail-oi1-f174.google.com with SMTP id i1so1214219oie.8 for ; Mon, 02 Mar 2020 16:35:36 -0800 (PST) X-Received: by 2002:a05:6808:289:: with SMTP id z9mr745220oic.48.1583195736660; Mon, 02 Mar 2020 16:35:36 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: Reply-To: cem@freebsd.org From: Conrad Meyer Date: Mon, 2 Mar 2020 16:35:25 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: What is the exact function of the offset parameter in preadv?? To: Willem Jan Withagen Cc: FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 48WdMv1gC7z3FZ8 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of csecem@gmail.com designates 209.85.210.67 as permitted sender) smtp.mailfrom=csecem@gmail.com X-Spamd-Result: default: False [-1.99 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_REPLYTO(0.00)[cem@freebsd.org]; RWL_MAILSPIKE_GOOD(0.00)[67.210.85.209.rep.mailspike.net : 127.0.0.18]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; REPLYTO_ADDR_EQ_FROM(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; TO_DN_ALL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; FORGED_SENDER(0.30)[cem@freebsd.org,csecem@gmail.com]; SUBJECT_ENDS_QUESTION(1.00)[]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; FROM_NEQ_ENVFROM(0.00)[cem@freebsd.org,csecem@gmail.com]; TAGGED_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[67.210.85.209.list.dnswl.org : 127.0.5.0]; IP_SCORE(-1.00)[ip: (-0.28), ipnet: 209.85.128.0/17(-2.99), asn: 15169(-1.66), country: US(-0.05)]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2020 00:35:41 -0000 Hi Willem, The offset parameter to the p{read,write}{,v} family of functions indeed indicates a file offset. For ordinary files, negative offsets are invalid (EINVAL). Bhyve is probably accessing a special kernel device file, kmem(4), and those negative "offsets" instead represent addresses in kernel virtual memory. Best, Conrad On Sun, Mar 1, 2020 at 4:47 PM Willem Jan Withagen wrote: > > Hi, > > Tried looking in the manual page, but it does not really describes what > off_t offset does with a preadv... > > I suspect that is the offset in the file to start reading from? > > Next question is: What if the value is negative? > Are we then reading from the end of the file/device if it is > seekable? > > Manual page says: > The pread() and preadv() system calls may also return the following > errors: > [EINVAL] The offset value was negative. > > But then I have this behyve code that gets calls like: > block_proc_read: buf=0x0, cnt=1, len=512, off=-1024(0xfffffc00). > block_proc_read: buf=0x0, cnt=1, len=512, off=-512(0xfffffe00). > block_proc_read: buf=0x0, cnt=1, len=512, off=-1024(0xfffffc00). > block_proc_read: buf=0x0, cnt=1, len=512, off=-512(0xfffffe00). > block_proc_read: buf=0x0, cnt=1, len=512, off=-512(0xfffffe00). > block_proc_read: buf=0x0, cnt=1, len=2048, off=-32256(0xffff8200). > block_proc_read: buf=0x0, cnt=1, len=512, off=512(0x200). > block_proc_read: buf=0x0, cnt=1, len=512, off=0(0x0). > block_proc_read: buf=0x0, cnt=1, len=512, off=512(0x200). > block_proc_read: buf=0x0, cnt=1, len=512, off=0(0x0). > block_proc_read: buf=0x0, cnt=1, len=512, off=512(0x200). > block_proc_read: buf=0x0, cnt=1, len=16384, off=1024(0x400). > block_proc_read: buf=0x0, cnt=1, len=512, off=-512(0xfffffe00). > block_proc_read: buf=0x0, cnt=1, len=16384, off=-16896(0xffffbe00). > block_proc_read: buf=0x0, cnt=1, len=512, off=-512(0xfffffe00). > > Where these are the very first calls that a bhyve loaded image executes. > which I guess is for probing the bootdisk to see where/what it needs to > mount a root disk. > > Below bhyve boot output of a not working boot, and a working boot. > The read trace above is from the working bhyve boot. > > So what are the negative offsets? > > --WjW > > > Compare > Using block_rbd_read for reading from a Rados device > > ============= > vtblk0: on virtio_pci1 > vtblk0: 134217727MB (34359738366 4096 byte sectors) > ahci0: mem 0xc0004000-0xc00043ff irq > 18 at device 4.0 on pci0 > ahci0: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported > atkbdc0: port 0x60,0x64 irq 1 on acpi0 > atkbd0: irq 1 on atkbdc0 > kbd0 at atkbd0 > atkbd0: [GIANT-LOCKED] > driver bug: Unable to set devclass (class: atkbdc devname: (unknown)) > psm0: irq 12 on atkbdc0 > psm0: [GIANT-LOCKED] > psm0: model Generic PS/2 mouse, device ID 0 > uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 > uart0: console (9600,n,8,1) > uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 > vga0: at port 0x3b0-0x3bb iomem 0xb0000-0xb7fff pnpid > PNP0900 on isa0 > Timecounters tick every 10.000 msec > usb_needs_explore_all: no devclass > Trying to mount root from ufs:/dev/ada0p2 [rw]... > mountroot: waiting for device /dev/ada0p2... > Mounting from ufs:/dev/ada0p2 failed with error 19. > > Loader variables: > vfs.root.mountfrom=ufs:/dev/ada0p2 > vfs.root.mountfrom.options=rw > > Manual root filesystem specification: > : [options] > Mount using filesystem > and with the specified (optional) option list. > > eg. ufs:/dev/da0s1a > zfs:zroot/ROOT/default > cd9660:/dev/cd0 ro > (which is equivalent to: mount -t cd9660 -o ro /dev/cd0 /) > > ? List valid disk boot devices > . Yield 1 second (for background tasks) > Abort manual input > > ============= > > With booting from a normal file device: > atkbdc0: port 0x60,0x64 irq 1 on acpi0 > atkbd0: irq 1 on atkbdc0 > kbd0 at atkbd0 > atkbd0: [GIANT-LOCKED] > driver bug: Unable to set devclass (class: atkbdc devname: (unknown)) > psm0: irq 12 on atkbdc0 > psm0: [GIANT-LOCKED] > psm0: model Generic PS/2 mouse, device ID 0 > uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 > uart0: console (9600,n,8,1) > uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 > vga0: at port 0x3b0-0x3bb iomem 0xb0000-0xb7fff pnpid > PNP0900 on isa0 > Timecounters tick every 10.000 msec > usb_needs_explore_all: no devclass > Trying to mount root from ufs:/dev/ada0p2 [rw]... > Root mount waiting for: CAM > ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 > ada0: ACS-2 ATA SATA 3.x device > ada0: Serial Number BHYVE-8C7D-D922-AD47 > ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) > ada0: Command Queueing enabled > ada0: 20480MB (41943040 512 byte sectors) > ada1 at ahcich6 bus 0 scbus1 target 0 lun 0 > ada1: ACS-2 ATA SATA 3.x device > ada1: Serial Number BHYVE-4318-7904-5D51 > ada1: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) > ada1: Command Queueing enabled > ada1: 20480MB (41943040 512 byte sectors) > mountroot: waiting for device /dev/ada0p2... > WARNING: / was not properly dismounted > WARNING: /: mount pending error: blocks 168 files 1 > ========== > > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@freebsd.org Tue Mar 3 09:11:31 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 87268268F81 for ; Tue, 3 Mar 2020 09:11:31 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from smtp.digiware.nl (smtp.digiware.nl [176.74.240.9]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 48Wrq72Pm8z4Fb2; Tue, 3 Mar 2020 09:11:31 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from router.digiware.nl (localhost.digiware.nl [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id CF9D725B07; Tue, 3 Mar 2020 10:11:28 +0100 (CET) X-Virus-Scanned: amavisd-new at digiware.com Received: from smtp.digiware.nl ([127.0.0.1]) by router.digiware.nl (router.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hRF70qu_H0zk; Tue, 3 Mar 2020 10:11:28 +0100 (CET) Received: from [192.168.10.9] (vaio [192.168.10.9]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id 1AEFE25B06; Tue, 3 Mar 2020 10:11:28 +0100 (CET) Subject: Re: What is the exact function of the offset parameter in preadv?? To: cem@freebsd.org Cc: FreeBSD Hackers References: From: Willem Jan Withagen Message-ID: <896f3360-f4df-391a-3b08-47fd8de6eabb@digiware.nl> Date: Tue, 3 Mar 2020 10:11:28 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Firefox/68.0 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: nl X-Rspamd-Queue-Id: 48Wrq72Pm8z4Fb2 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-5.92 / 15.00]; NEURAL_HAM_MEDIUM(-0.92)[-0.921,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2020 09:11:31 -0000 On 3-3-2020 01:35, Conrad Meyer wrote: > Hi Willem, > > The offset parameter to the p{read,write}{,v} family of functions > indeed indicates a file offset. For ordinary files, negative offsets > are invalid (EINVAL). > > Bhyve is probably accessing a special kernel device file, kmem(4), and > those negative "offsets" instead represent addresses in kernel virtual > memory. Conrad, Right, I have annotated the code that does block_local writes, and I'm in the process of adding block_rbd to it. And it is from those that I get these values. So they are actual accesses to regular files. But I think I got caught in 32->64 sign extention while printing the values. And we are actually accessing things like `size(file) - 1024` . I'll see if I can get a bit of extra doc in the manpage. Thanx, --WjW > Best, > Conrad > > On Sun, Mar 1, 2020 at 4:47 PM Willem Jan Withagen wrote: >> Hi, >> >> Tried looking in the manual page, but it does not really describes what >> off_t offset does with a preadv... >> >> I suspect that is the offset in the file to start reading from? >> >> Next question is: What if the value is negative? >> Are we then reading from the end of the file/device if it is >> seekable? >> >> Manual page says: >> The pread() and preadv() system calls may also return the following >> errors: >> [EINVAL] The offset value was negative. >> >> But then I have this behyve code that gets calls like: >> block_proc_read: buf=0x0, cnt=1, len=512, off=-1024(0xfffffc00). >> block_proc_read: buf=0x0, cnt=1, len=512, off=-512(0xfffffe00). >> block_proc_read: buf=0x0, cnt=1, len=512, off=-1024(0xfffffc00). >> block_proc_read: buf=0x0, cnt=1, len=512, off=-512(0xfffffe00). >> block_proc_read: buf=0x0, cnt=1, len=512, off=-512(0xfffffe00). >> block_proc_read: buf=0x0, cnt=1, len=2048, off=-32256(0xffff8200). >> block_proc_read: buf=0x0, cnt=1, len=512, off=512(0x200). >> block_proc_read: buf=0x0, cnt=1, len=512, off=0(0x0). >> block_proc_read: buf=0x0, cnt=1, len=512, off=512(0x200). >> block_proc_read: buf=0x0, cnt=1, len=512, off=0(0x0). >> block_proc_read: buf=0x0, cnt=1, len=512, off=512(0x200). >> block_proc_read: buf=0x0, cnt=1, len=16384, off=1024(0x400). >> block_proc_read: buf=0x0, cnt=1, len=512, off=-512(0xfffffe00). >> block_proc_read: buf=0x0, cnt=1, len=16384, off=-16896(0xffffbe00). >> block_proc_read: buf=0x0, cnt=1, len=512, off=-512(0xfffffe00). >> >> Where these are the very first calls that a bhyve loaded image executes. >> which I guess is for probing the bootdisk to see where/what it needs to >> mount a root disk. >> >> Below bhyve boot output of a not working boot, and a working boot. >> The read trace above is from the working bhyve boot. >> >> So what are the negative offsets? >> >> --WjW >> >> >> Compare >> Using block_rbd_read for reading from a Rados device >> >> ============= >> vtblk0: on virtio_pci1 >> vtblk0: 134217727MB (34359738366 4096 byte sectors) >> ahci0: mem 0xc0004000-0xc00043ff irq >> 18 at device 4.0 on pci0 >> ahci0: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported >> atkbdc0: port 0x60,0x64 irq 1 on acpi0 >> atkbd0: irq 1 on atkbdc0 >> kbd0 at atkbd0 >> atkbd0: [GIANT-LOCKED] >> driver bug: Unable to set devclass (class: atkbdc devname: (unknown)) >> psm0: irq 12 on atkbdc0 >> psm0: [GIANT-LOCKED] >> psm0: model Generic PS/2 mouse, device ID 0 >> uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 >> uart0: console (9600,n,8,1) >> uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 >> vga0: at port 0x3b0-0x3bb iomem 0xb0000-0xb7fff pnpid >> PNP0900 on isa0 >> Timecounters tick every 10.000 msec >> usb_needs_explore_all: no devclass >> Trying to mount root from ufs:/dev/ada0p2 [rw]... >> mountroot: waiting for device /dev/ada0p2... >> Mounting from ufs:/dev/ada0p2 failed with error 19. >> >> Loader variables: >> vfs.root.mountfrom=ufs:/dev/ada0p2 >> vfs.root.mountfrom.options=rw >> >> Manual root filesystem specification: >> : [options] >> Mount using filesystem >> and with the specified (optional) option list. >> >> eg. ufs:/dev/da0s1a >> zfs:zroot/ROOT/default >> cd9660:/dev/cd0 ro >> (which is equivalent to: mount -t cd9660 -o ro /dev/cd0 /) >> >> ? List valid disk boot devices >> . Yield 1 second (for background tasks) >> Abort manual input >> >> ============= >> >> With booting from a normal file device: >> atkbdc0: port 0x60,0x64 irq 1 on acpi0 >> atkbd0: irq 1 on atkbdc0 >> kbd0 at atkbd0 >> atkbd0: [GIANT-LOCKED] >> driver bug: Unable to set devclass (class: atkbdc devname: (unknown)) >> psm0: irq 12 on atkbdc0 >> psm0: [GIANT-LOCKED] >> psm0: model Generic PS/2 mouse, device ID 0 >> uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 >> uart0: console (9600,n,8,1) >> uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 >> vga0: at port 0x3b0-0x3bb iomem 0xb0000-0xb7fff pnpid >> PNP0900 on isa0 >> Timecounters tick every 10.000 msec >> usb_needs_explore_all: no devclass >> Trying to mount root from ufs:/dev/ada0p2 [rw]... >> Root mount waiting for: CAM >> ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 >> ada0: ACS-2 ATA SATA 3.x device >> ada0: Serial Number BHYVE-8C7D-D922-AD47 >> ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) >> ada0: Command Queueing enabled >> ada0: 20480MB (41943040 512 byte sectors) >> ada1 at ahcich6 bus 0 scbus1 target 0 lun 0 >> ada1: ACS-2 ATA SATA 3.x device >> ada1: Serial Number BHYVE-4318-7904-5D51 >> ada1: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) >> ada1: Command Queueing enabled >> ada1: 20480MB (41943040 512 byte sectors) >> mountroot: waiting for device /dev/ada0p2... >> WARNING: / was not properly dismounted >> WARNING: /: mount pending error: blocks 168 files 1 >> ========== >> >> >> >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@freebsd.org Wed Mar 4 21:43:36 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4C3CD252AA2 for ; Wed, 4 Mar 2020 21:43:36 +0000 (UTC) (envelope-from keno@juliacomputing.com) Received: from mail-il1-x143.google.com (mail-il1-x143.google.com [IPv6:2607:f8b0:4864:20::143]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48XnSQ0LFbz4b4F for ; Wed, 4 Mar 2020 21:43:33 +0000 (UTC) (envelope-from keno@juliacomputing.com) Received: by mail-il1-x143.google.com with SMTP id g126so3183406ilh.2 for ; Wed, 04 Mar 2020 13:43:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=juliacomputing-com.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to:cc; bh=mP8tDChjczQfoaVT7BtcMixrtSz+HS/j/2DN2oIXxL8=; b=UwK5mv+NgTPOrG+VSnOZIOV5kAOrF6lozUMRX0jAFJ8EzEg/nD2DsgugSeeDYroGSk geAAcP4fhD8EZ4SwYob/4RmnMKrP4fIjtDtOsZWWt8owp1DLcW0/ExxXpVBSpMQ+UPaC WBf0483aN9HYVOYOJDwx9tZNR7ITZwicHcGAWTVjZYYke6HwpHE1fN9Eb3wWj/bMTLHk WjyG6xKpoj3jvyE0eQkTrfcgURUCFhZit6cainJJMAccmiTIs7l0Q9Fzo1LzzAi/aeWp X+xneZ+m0ttVCLp7e01/4Tv6Ps4mihsbotqIXk2lbfy2p5EN27gGwlZq00lT5h7369fQ +CKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=mP8tDChjczQfoaVT7BtcMixrtSz+HS/j/2DN2oIXxL8=; b=Qrd+2nNFaDfEYsBXuX5c/AByf3gbsehYEOR8fRRMND9H4pzrziCEEt0mbelhpux7Bl QEUewQpyv4QJWMgRgDIgtCDwZaZB76iTvnJ55shQxp8WdFUibBw3vS/3DUOkaI7+nDg6 tMXXxovX+RaC0poOZyVdAiBhRawazMBHm0jKv1lqgy9gS4D92fgyICd1CttzNfI+V4K5 SN8cEfN8oefX04nO6481pjyxj+GV3/VpX3ec0e2/wcSmItFYFvqco7TU9EInlIoX5NQX /VBXZLktkeBpwgJLFSxw0TVUtcILLeJi4834WHSEpbRLuBZ6v88PcUwvFXoKKKt4892X ZAnQ== X-Gm-Message-State: ANhLgQ1m1z/cjpKLhp0XSaokTNqOHRRg1dxgOvwLLV/c1SRhrL74AIcW a6gEThppMRvJYSHLKuvRz+a0CSzUZtow8rlAHTGKCnWfDIw= X-Google-Smtp-Source: ADFU+vsXFcQQQcGRvM+sK+OGB3Uk55uxON19Wki3BGl/WmjWuKil8CeWkw+WU+4oedvLhDp/z4rsFaIZN4D4VN/2AWc= X-Received: by 2002:a92:6b03:: with SMTP id g3mr4735809ilc.201.1583358212512; Wed, 04 Mar 2020 13:43:32 -0800 (PST) MIME-Version: 1.0 From: Keno Fischer Date: Wed, 4 Mar 2020 16:42:56 -0500 Message-ID: Subject: FreeBSD Pipe behavior in pipe OOM situations To: freebsd-hackers@freebsd.org Cc: Elliot Saba X-Rspamd-Queue-Id: 48XnSQ0LFbz4b4F X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=juliacomputing-com.20150623.gappssmtp.com header.s=20150623 header.b=UwK5mv+N; dmarc=none; spf=none (mx1.freebsd.org: domain of keno@juliacomputing.com has no SPF policy when checking 2607:f8b0:4864:20::143) smtp.mailfrom=keno@juliacomputing.com X-Spamd-Result: default: False [-1.60 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.993,0]; R_DKIM_ALLOW(-0.20)[juliacomputing-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[juliacomputing.com]; URI_COUNT_ODD(1.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[juliacomputing-com.20150623.gappssmtp.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[3.4.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-0.30)[ip: (2.04), ipnet: 2607:f8b0::/32(-1.86), asn: 15169(-1.66), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Mar 2020 21:43:36 -0000 Greetings, I am debugging intermittent failures we see on the CI system for the Julia programming language on FreeBSD, but not elsewhere. The Julia ticket for this issue can be found at https://github.com/JuliaLang/julia/issues/23143. The symptom is an ENOMEM error on a write to a pipe, together with the following message in dmesg: kern.ipc.maxpipekva exceeded; see tuning(7) Now, as far as I understand it, what's happening here is that FreeBSD has a hard limit on the amount of kernel memory that can be used for pipe buffers, which we are exceeding by creating too many pipes (not entirely surprising, our test suites spawns many processes and uses lots of pipes). I understand that we can likely work around this issue by increasing the referenced sysctl. However, I am a bit puzzled by the ENOMEM behavior. I don't have very much experience with the FreeBSD kernel, but from my experience from working on other operating systems, I would have expected that either: 1) Some minimal buffer is allocated anyway and exempt from such pipe-specific memory limits (e.g. a few bytes of the pipe struct), or, 2) The writing process is blocked until pipe buffer space becomes available (e.g. by a different pipe draining and freeing up space), or, 3) The writing process is blocked until a reader comes along, at which point the write is performed directly without intermediate kernel buffer. I.e. I would have expected such an OOM situation for pipe buffers to degrade pipe performance, but not to have it exposed to the user. Indeed, a cursory read of the FreeBSD kernel source seems to reinforce this notion. In pipe_create, we see the following comment: ``` /* * Note that these functions can fail if pipe map is exhausted * (as a result of too many pipes created), but we ignore the * error as it is not fatal and could be provoked by * unprivileged users. The only consequence is worse performance * with given pipe. */ if (amountpipekva > maxpipekva / 2) (void)pipespace_new(pipe, SMALL_PIPE_SIZE); else (void)pipespace_new(pipe, PIPE_SIZE); ``` But then later, in pipe_write, we see: ``` if (wpipe->pipe_buffer.size == 0) { /* * This can only happen for reverse direction use of pipes * in a complete OOM situation. */ error = ENOMEM; ``` >From my (admittedly limited) understanding of the code, it doesn't seem that either comment is accurate. If the pipe buffer allocation fails, then `write`s will return `ENOMEM`, even in the forward direction (the buffer for the reverse direction isn't allocated by default, but as indicated by the first comment, the allocation for the forward direction can certainly fail). I was hoping a FreeBSD kernel developer could shed some light on whether the kernel behavior we're experiencing here is indeed expected on FreeBSD, or whether it would be expected that the kernel would try harder to service the pipe request in such a situation. Thanks, Keno From owner-freebsd-hackers@freebsd.org Wed Mar 4 23:39:23 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0459825550D for ; Wed, 4 Mar 2020 23:39:23 +0000 (UTC) (envelope-from kib@freebsd.org) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 48Xr216p0Fz4RVL for ; Wed, 4 Mar 2020 23:39:21 +0000 (UTC) (envelope-from kib@freebsd.org) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id 024Nd75x099764 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Thu, 5 Mar 2020 01:39:10 +0200 (EET) (envelope-from kib@freebsd.org) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 024Nd75x099764 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id 024Nd6Pw099763; Thu, 5 Mar 2020 01:39:06 +0200 (EET) (envelope-from kib@freebsd.org) X-Authentication-Warning: tom.home: kostik set sender to kib@freebsd.org using -f Date: Thu, 5 Mar 2020 01:39:06 +0200 From: Konstantin Belousov To: Keno Fischer Cc: freebsd-hackers@freebsd.org, Elliot Saba Subject: Re: FreeBSD Pipe behavior in pipe OOM situations Message-ID: <20200304233906.GB98340@kib.kiev.ua> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on tom.home X-Rspamd-Queue-Id: 48Xr216p0Fz4RVL X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-0.08 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_LONG(-1.00)[-0.998,0]; NEURAL_SPAM_MEDIUM(0.92)[0.915,0]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Mar 2020 23:39:23 -0000 On Wed, Mar 04, 2020 at 04:42:56PM -0500, Keno Fischer wrote: > Greetings, > > I am debugging intermittent failures we see on the CI system for the Julia > programming language on FreeBSD, but not elsewhere. The Julia ticket > for this issue can be found at > https://github.com/JuliaLang/julia/issues/23143. > > The symptom is an ENOMEM error on a write to a pipe, > together with the following message in dmesg: > > kern.ipc.maxpipekva exceeded; see tuning(7) > > Now, as far as I understand it, what's happening here is that FreeBSD has a > hard limit on the amount of kernel memory that can be used for pipe buffers, > which we are exceeding by creating too many pipes (not entirely surprising, > our test suites spawns many processes and uses lots of pipes). > > I understand that we can likely work around this issue by increasing the > referenced sysctl. However, I am a bit puzzled by the ENOMEM behavior. > I don't have very much experience with the FreeBSD kernel, but from my > experience from working on other operating systems, > I would have expected that either: > > 1) Some minimal buffer is allocated anyway and exempt from such > pipe-specific memory limits (e.g. a few bytes of the pipe struct), or, No. > 2) The writing process is blocked until pipe buffer space becomes available > (e.g. by a different pipe draining and freeing up space), or, Yes, but only as the space inside the allocated buffer, i.e. the bytes that are consumed by reader, not as a space that is provided for buffer. > 3) The writing process is blocked until a reader comes along, at which point > the write is performed directly without intermediate kernel buffer. First, there is a requirement that an atomic write size exists, i.e. writes less than SC_PIPE_BUF are guaranteed to not interleave if succeeded. Our PIPE_BUF is 512 bytes. We pre-allocate some buffers on the pipe creation, and then might adjust it at start of the write. The buffers initially consume only kernel virtual address space (KVA). Physical memory is instantiated when touched and can be swapped out (this is somewhat simplified, but details are not important). The atomicity requirement means that we must not allocate less than PIPE_BUF, but since we are using VM interfaces, we make the lowest limit 4K (actually page size). When there is enough space, we might go to up to 64K per pipe, but retract down when pipe KVA is filled. The KVA used for pipe buffers is shared by all pipes in system among all users. Currently allocation of pipe buffers does not wait for space, if there is no space it fails with ENOMEM. Waiting for the space means that the writer is blocked until some unrelated process does some action that frees pipe buffer, perhaps closes its pipe. I think that unexplained blocking (it is very hard to track down such state) is worse then ENOMEM outcome. > > I.e. I would have expected such an OOM situation for pipe buffers to > degrade pipe performance, but not to have it exposed to the user. Indeed, a > cursory > read of the FreeBSD kernel source seems to reinforce this notion. > In pipe_create, we see the following comment: > > ``` > /* > * Note that these functions can fail if pipe map is exhausted > * (as a result of too many pipes created), but we ignore the > * error as it is not fatal and could be provoked by > * unprivileged users. The only consequence is worse performance > * with given pipe. > */ > if (amountpipekva > maxpipekva / 2) > (void)pipespace_new(pipe, SMALL_PIPE_SIZE); > else > (void)pipespace_new(pipe, PIPE_SIZE); > ``` This happens at pipe open. As you see, we might preallocate only SMALL_PIPE_SIZE (4K) if low on KVA, or not preallocate at all if KVA is exhausted, hoping that at the time of write(2) the situation changes. > > But then later, in pipe_write, we see: > ``` > if (wpipe->pipe_buffer.size == 0) { > /* > * This can only happen for reverse direction use of pipes > * in a complete OOM situation. > */ > error = ENOMEM; > ``` > > >From my (admittedly limited) understanding of the code, it doesn't > seem that either comment is accurate. If the pipe buffer allocation > fails, then `write`s will return `ENOMEM`, even in the forward direction > (the buffer for the reverse direction isn't allocated by default, but > as indicated by the first comment, the allocation for the forward > direction can certainly fail). Yes, this comment is confusing and if both preallocation at the pipe creation time, and then allocation at first write both failed, we return ENOMEM. We reserve 1/64 of the physical memory for pipekva. It costs nothing to increase this number initially for 64bit systems because it is only KVA, but note that eventually this memory will be instantiated with physical backing pages. E.g. on my workstation with 32G RAM I see kern.ipc.maxpipekva: 534261760 (512M) and I do not want to make it larger. What is the amount of memory on the machine where you see ENOMEM ? > > I was hoping a FreeBSD kernel developer could shed some light on > whether the kernel behavior we're experiencing here is indeed expected > on FreeBSD, or whether it would be expected that the kernel would try > harder to service the pipe request in such a situation. > > Thanks, > Keno > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@freebsd.org Thu Mar 5 23:35:11 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 83676250336 for ; Thu, 5 Mar 2020 23:35:11 +0000 (UTC) (envelope-from nonesuch@longcount.org) Received: from mail-qk1-x72b.google.com (mail-qk1-x72b.google.com [IPv6:2607:f8b0:4864:20::72b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48YRtj6vZfz4KDX for ; Thu, 5 Mar 2020 23:35:09 +0000 (UTC) (envelope-from nonesuch@longcount.org) Received: by mail-qk1-x72b.google.com with SMTP id m2so622686qka.7 for ; Thu, 05 Mar 2020 15:35:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=longcount-org.20150623.gappssmtp.com; s=20150623; h=content-transfer-encoding:from:mime-version:subject:message-id:date :to; bh=MsArC53SO14aq1FVfLL0jG3fo/N5Eb/tLqYB2jnB4eg=; b=LHgiJAw53a+9B5Q/dz+aqhnrUN8ic4k3xgRTNyvbtAX1VXLWesEic7dhvTsaDZB1B1 g2/1CaCffXslUKNVtOU+VFqW2WKWPzyG7iPn+2Dkbf1sqXIDwoEAmYju7XbfYbIsRGv4 EGOM5P/SHaFIf03k1dkCaIf6TfNoauI4GQT8uwYyg4Z1UdIrxl74E6yglEnfPtpTFSpx HSHmlPwDiPMjE7el/y5vUCOL9x0IQKkvRHQpAuJvTKSi2yL+UOVchPK0sEL1UUMPBb9Z WcoDHRmYMB7acbCfCQ7y3ajOV5u3kTqkc9VeFAyUdkFqEmYgPv+1w+eCJF5s7ZXShS9E Kt6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:content-transfer-encoding:from:mime-version :subject:message-id:date:to; bh=MsArC53SO14aq1FVfLL0jG3fo/N5Eb/tLqYB2jnB4eg=; b=kTzLceiiJoPril0Oqa6r75+VsFvAuyNv2S/eiJFTkon+siWP6EZ9w+DNrIkxcEQA0C Auec+KZZM+/d9WC26J556IVfbDFeno2U75Ylgt/7gvTs0d6Kxs3B+Bzk4+z+CouTvIVN XdLngsTbZU/yn0qVoMMmbZpEJ5p08OiVQV5YPaKdc5EppxS9PGdS0XwNSim/MF3XNQpB wjkKAs7hV7SRHmPcRRf9IxAWQolGFivyBppTJWlWJ/HQmk7K8+UyzeNyS8TYmS6ik+o1 C5T4Cm5bv5Sd7FIIgTtMkfxKsBPllOc7D30O9/lE6s/sSNT4Q/35J47ZzLagaSOxwyjN fGLg== X-Gm-Message-State: ANhLgQ1n4E1/ch23u0YP+6aCuRyMQjvRX+MemQiB+ixMtkhGZOYw4hkk tfo0MIQVg8oOYvgeSQKWbeLWUeveNbY= X-Google-Smtp-Source: ADFU+vsAALX8qgvS9TvhjqTxjL0FjWFbvN9QRD3t2LPAQJwmZ71TPQwn8Cn7W7H9eThPFbOTVEV33w== X-Received: by 2002:a37:aa06:: with SMTP id t6mr413501qke.495.1583451308677; Thu, 05 Mar 2020 15:35:08 -0800 (PST) Received: from ?IPv6:2600:1017:b819:6dc8:fd79:2dd:c75a:9f15? ([2600:1017:b819:6dc8:fd79:2dd:c75a:9f15]) by smtp.gmail.com with ESMTPSA id t6sm16530173qke.57.2020.03.05.15.35.07 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 05 Mar 2020 15:35:08 -0800 (PST) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable From: Mark Saad Mime-Version: 1.0 (1.0) Subject: Dell DPAT Message-Id: <1F938730-E7DE-4323-9A0D-778A9C14555B@longcount.org> Date: Thu, 5 Mar 2020 18:35:05 -0500 To: FreeBSD Hackers X-Mailer: iPhone Mail (17D50) X-Rspamd-Queue-Id: 48YRtj6vZfz4KDX X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=longcount-org.20150623.gappssmtp.com header.s=20150623 header.b=LHgiJAw5; dmarc=none; spf=none (mx1.freebsd.org: domain of nonesuch@longcount.org has no SPF policy when checking 2607:f8b0:4864:20::72b) smtp.mailfrom=nonesuch@longcount.org X-Spamd-Result: default: False [-4.38 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[longcount-org.20150623.gappssmtp.com:s=20150623]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[longcount.org]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[longcount-org.20150623.gappssmtp.com:+]; RCVD_IN_DNSWL_NONE(0.00)[b.2.7.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; IP_SCORE(-2.58)[ip: (-9.33), ipnet: 2607:f8b0::/32(-1.86), asn: 15169(-1.65), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Mar 2020 23:35:11 -0000 =EF=BB=BFAll Does anyone know of a way to get FreeBSD 12 or newer to turn on dell assist= ed turbo a/k/a DPAT ?=20 --- Mark Saad | nonesuch@longcount.org= From owner-freebsd-hackers@freebsd.org Fri Mar 6 11:25:35 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CBEF7265A87 for ; Fri, 6 Mar 2020 11:25:35 +0000 (UTC) (envelope-from ivlevsr@yandex.ru) Received: from forward400o.mail.yandex.net (forward400o.mail.yandex.net [37.140.190.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 48YlfQ2q6jz4ZyR for ; Fri, 6 Mar 2020 11:25:33 +0000 (UTC) (envelope-from ivlevsr@yandex.ru) Received: from mxback26g.mail.yandex.net (mxback26g.mail.yandex.net [IPv6:2a02:6b8:0:1472:2741:0:8b7:325]) by forward400o.mail.yandex.net (Yandex) with ESMTP id 2881B14C10D3 for ; Fri, 6 Mar 2020 14:25:31 +0300 (MSK) Received: from localhost (localhost [::1]) by mxback26g.mail.yandex.net (mxback/Yandex) with ESMTP id E2R8jTGEDi-PUxu7p4N; Fri, 06 Mar 2020 14:25:30 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1583493930; bh=HGiEpxezmonVOhZAsB29cg8pkdOKoo1NJ9TtOEkddY4=; h=Message-Id:Date:Subject:To:From; b=v2PqrD6oyfSqUb/1gkIXGQjYFZB2lXQ64V0/nrCKEJhEGsinTGJQXqpD0h+Sjc9uz yteSkFJu4CamhFXWnXjs52tz70/aFDYlu0Ae5iZuFjjUb3Y/mKDRtT8gFBSWh7PzjT +4z7bM+ouHeFHSSvhbf1T+MALI6c7Vyu4mdzA65g= Received: by iva8-5e86d95f65ab.qloud-c.yandex.net with HTTP; Fri, 06 Mar 2020 14:25:30 +0300 From: Sergey Ivlev To: "freebsd-hackers@freebsd.org" Subject: Cross-building ports with release(7) MIME-Version: 1.0 X-Mailer: Yamail [ http://yandex.ru ] 5.0 Date: Fri, 06 Mar 2020 16:25:30 +0500 Message-Id: <2228511583493903@vla3-bebe75876e15.qloud-c.yandex.net> Content-Transfer-Encoding: 7bit Content-Type: text/plain X-Rspamd-Queue-Id: 48YlfQ2q6jz4ZyR X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yandex.ru header.s=mail header.b=v2PqrD6o; dmarc=pass (policy=none) header.from=yandex.ru; spf=pass (mx1.freebsd.org: domain of ivlevsr@yandex.ru designates 37.140.190.176 as permitted sender) smtp.mailfrom=ivlevsr@yandex.ru X-Spamd-Result: default: False [-3.08 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.98)[-0.984,0]; R_DKIM_ALLOW(-0.20)[yandex.ru:s=mail]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[yandex.ru.dwl.dnswl.org : 127.0.5.0]; R_SPF_ALLOW(-0.20)[+ip4:37.140.128.0/18:c]; FREEMAIL_FROM(0.00)[yandex.ru]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE_FREEMAIL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[3]; IP_SCORE(0.00)[ipnet: 37.140.128.0/18(-4.89), asn: 13238(-3.84), country: RU(0.01)]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[yandex.ru:+]; DMARC_POLICY_ALLOW(-0.50)[yandex.ru,none]; TO_DN_EQ_ADDR_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yandex.ru]; ASN(0.00)[asn:13238, ipnet:37.140.128.0/18, country:RU]; RCVD_TLS_LAST(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[176.190.140.37.list.dnswl.org : 127.0.5.1] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2020 11:25:35 -0000 Hello! I've successfully built FreeBSD 13.0-CURRENT (r358609) image for ARMv7 on AMD64 host using release(7). I wonder is it possible for AMD64 host to build/install ports into the image? I've defined arm_do_quirk function in my configuration file, where I'm calling `make DESTDIR=${CHROOTDIR}/${DESTDIR} TARGET=arm TARGET_ARCH=armv7 ... build install clean`. But it ended up with error, of course, because that make(1) chroots into ${CHROOTDIR}/${DESTDIR} and runs executables from there, but ${CHROOTDIR}/${DESTDIR} is where **built target** located (i.e. where files for ARMv7; ${CHROOTDIR}/${DESTDIR} is {my chrrot dir}/usr/obj/usr/src/arm.armv7/release/{kernel name}). Altering the PATH variable so it will specify ${CHROOTDIR} as the first entry, doesn't help either -- I end up with the same ``exec format error'' (env PATH="${CHROOTDIR}:${PATH}" make DESTDIR=...). I had also tried to `chroot ${CHROOTDIR} make DESTDIR=${DESTDIR} TARGET=arm TARGET_ARCH=armv7 ... build install clean`, but mount_nullfs(8) failed under chroot(8). Therefore, my question is: is it possible while building an image for ARMv7 on AMD64 host with release(7) also building and installing ports into the result image? Thanks in advance! From owner-freebsd-hackers@freebsd.org Fri Mar 6 17:09:25 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8102026C49D for ; Fri, 6 Mar 2020 17:09:25 +0000 (UTC) (envelope-from theron.tarigo@gmail.com) Received: from mail-qk1-x734.google.com (mail-qk1-x734.google.com [IPv6:2607:f8b0:4864:20::734]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48YvH81z66z3DHV for ; Fri, 6 Mar 2020 17:09:23 +0000 (UTC) (envelope-from theron.tarigo@gmail.com) Received: by mail-qk1-x734.google.com with SMTP id f3so3003465qkh.1 for ; Fri, 06 Mar 2020 09:09:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:to:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding:content-language; bh=2j/WO+Nzbh7sFoOLQRfS0jFS3hdhgzDC2vGdlXzOEAc=; b=djco01RawXb1B/jqk6sQlqxbTbvCzHVUAUn+BSdK6mqjbu30Higoq4ZccY1oJSH6z1 3RsAeo6lm3VymxJJNcWShN61GdIOUoJ4kBgyH+j4EYs2W+2JfBF012VBNeK1d+qKAhRu eSbobEX0mg5RuHY6zjURvtYY1505tNvuMVJ27R0giJD7Gxpj5AN6L7myipBummKjoRg5 2EgRxoYgkPEbQlpRiVag6Qj4vDVMaEPSLWTTjiUzijA+4C1TVnr7qRh6q2MZwmpFGtfA IStMlNRKW1Bahu9lkUpkAxQvW2vuJE7G85NCoXQs/FXI1OFDJsDuR2Hx/BcBAgKiVXcA K+Ag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:to:references:from:message-id :date:user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=2j/WO+Nzbh7sFoOLQRfS0jFS3hdhgzDC2vGdlXzOEAc=; b=IsiQy/6whs9xf6xCI4fKqOZGC+2EXGl7+loGcb6QRHH33NjCGSyhDa0ZnJF7E0DAS7 PszXs6IreJK9RH04Bpaxc5X/1L0KNQ1W02aCaEYRdv763sXJgNJPoGWX3aO+4Oj0LRlZ gla+Ilx+ZWoHVej1FZn3VSbk+gRq5mWxCniYLVyTWAbMtQUn94Q9HRKzFPseNYsv5Vkg JLldjXkKfgq3jaffGlprAdYZsc4I+ywagwRfg3HM6MptneP71hUc1JjzbBv30DSkow79 0fhYi/IBWJSCOfHd1NMx7hamHgIM8SP2mnWE2oVJvDzKj4HvQnoqwfHIhQtk5JkUe1Yb OXCg== X-Gm-Message-State: ANhLgQ2GlYlM1hEeButoS0jrKyjC15S0/F/zYSCz16FL2pOo8O9tqA+1 sCHk0FMs4RxkZefVyiFr3JtW5Rlu X-Google-Smtp-Source: ADFU+vswYw0apBHGolEuxOYrO0o5/EL6n6lNFZhr/lAUU+DZjS30xWoXOMCIgo+1YXwPibDTJcPBew== X-Received: by 2002:a37:a38d:: with SMTP id m135mr3618277qke.303.1583514562871; Fri, 06 Mar 2020 09:09:22 -0800 (PST) Received: from [168.122.152.42] (dhcp-resnet-168-122-152-42.bu.edu. [168.122.152.42]) by smtp.gmail.com with ESMTPSA id s27sm9496825qtj.89.2020.03.06.09.09.21 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 06 Mar 2020 09:09:21 -0800 (PST) Sender: Theron Tarigo Subject: Re: Cross-building ports with release(7) To: Sergey Ivlev , "freebsd-hackers@freebsd.org" References: <2228511583493903@vla3-bebe75876e15.qloud-c.yandex.net> From: Theron Message-ID: Date: Fri, 6 Mar 2020 12:09:21 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: <2228511583493903@vla3-bebe75876e15.qloud-c.yandex.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-Rspamd-Queue-Id: 48YvH81z66z3DHV X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=djco01Ra; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of therontarigo@gmail.com designates 2607:f8b0:4864:20::734 as permitted sender) smtp.mailfrom=therontarigo@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FREEMAIL_TO(0.00)[yandex.ru]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[4.3.7.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; IP_SCORE(0.00)[ip: (-9.17), ipnet: 2607:f8b0::/32(-1.86), asn: 15169(-1.65), country: US(-0.05)]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2020 17:09:25 -0000 On 2020-03-06 06:25, Sergey Ivlev wrote: > Therefore, my question is: is it possible while building an image for ARMv7 on AMD64 host with release(7) also building and installing ports into the result image? > > Thanks in advance! This will likely require much work, either as local workarounds or as some reworking of the ports building framework: https://wiki.freebsd.org/CrossBuildingPorts Theron From owner-freebsd-hackers@freebsd.org Fri Mar 6 17:15:52 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A30DB26C8B0 for ; Fri, 6 Mar 2020 17:15:52 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound2m.ore.mailhop.org (outbound2m.ore.mailhop.org [54.149.155.156]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 48YvQb6b5Nz3yWS for ; Fri, 6 Mar 2020 17:15:51 +0000 (UTC) (envelope-from ian@freebsd.org) ARC-Seal: i=1; a=rsa-sha256; t=1583514950; cv=none; d=outbound.mailhop.org; s=arc-outbound20181012; b=rolx7/IzpOwqZaXMAib7XRBoCw6Q3mHTuZhiSSbh1nkpp6iN9vlQA0/Hzx1wvbnjr48NcK51Gr79e kjhRVeu+1f2AybVX8Yyzvog/I39DkPfQwMFZWKLr0JXbI59O+sUgj+L6kvIdMckoW/JCvH3jRv5rfJ H2RGLfEMA4peXkaxqVSZWAc0FEG5M11UCSEtXPmDkkNwQU0fG9xxvU2sb6OJvtUsplbom8bMcCx4XA kGQHYFbiIXF6VB0XUyiy+eROlpjRJXTyovKlfMlfJNReqtAWbVijChqomSeLnHHFmumarJXf2NpCDV PKsvrXOpMKH1h5Q6s1Fgl5d4axuvhiA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=arc-outbound20181012; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:to:from:subject:message-id:dkim-signature:from; bh=CiZIgykwn4GOVmimZU7eQ/naMYSo5zy/UBps3BomjxQ=; b=M4E67p3p3AMxQ5I/Vmyt8wQvA9dHMF3xZ5voT0xcfEd9nVCcx01EVQErrcdRK33PyiFTBJNbpmMOV Uc/GwVITx8vD/GUNwyM7FFrwHGfyJQ1zJqGqtu237BbyjGfkhxb8Sdu1WRS3dy0J2n2VBT/TqiTWod nJP/5TrBYpA2CwqIHn+0We5edHdNPqxr3uW5dgcHdNHeO3hv/F2lNBKpslAnlvT4vZjrZpf0zrSEOZ M83SpZicG6WNMw607N/kq54JdchExuQUvgjv5CcBCto1v/pvCt1PulJCPRfbzIiidNhC71bixcduhp 9GEvNLpwK9LmImeZmW96gg0iQYxYVjw== ARC-Authentication-Results: i=1; outbound4.ore.mailhop.org; spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60; dmarc=none header.from=freebsd.org; arc=none header.oldest-pass=0; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=dkim-high; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:to:from:subject:message-id:from; bh=CiZIgykwn4GOVmimZU7eQ/naMYSo5zy/UBps3BomjxQ=; b=EYUbBGcsRG5vZGTE2FNO0vXPjD6LDoPq/WRqd9ec88ddyxtXgmRmWYpgokvXIZ6siuOOe202mQCJg Fcg6bnrMxPrcEPBfnat4ugMi/8j+FB6WYQuUg2uUajjq7gLJe3JHQIHIBBFCXX9SXc48eeOnZtqT/l w0qmlkip1TOk9dlWImt8IXdjp79JCQ7Kz+658dfIMyz4h3DrBINnGEWZ393jNy06dooWe3dOzgjr6g uiWPbrYP934wn6OUn2MrhN+mLwf2LdirT7tD/ga3lFROaa9n/NAKTF8K2tnb7ZvOWOVOfylFwULULg nEcQON2GE0S76lqG/2eNFVHmKlOJx0A== X-MHO-RoutePath: aGlwcGll X-MHO-User: 1f178112-5fce-11ea-9eb3-25e2dfa9fa8d X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [67.177.211.60]) by outbound4.ore.mailhop.org (Halon) with ESMTPSA id 1f178112-5fce-11ea-9eb3-25e2dfa9fa8d; Fri, 06 Mar 2020 17:15:48 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id 026HFlH2028688; Fri, 6 Mar 2020 10:15:47 -0700 (MST) (envelope-from ian@freebsd.org) Message-ID: <76e66ea321b8d7480af75cfbd38fa55f5ac40afd.camel@freebsd.org> Subject: Re: Cross-building ports with release(7) From: Ian Lepore To: Sergey Ivlev , "freebsd-hackers@freebsd.org" , "freebsd-arm@FreeBSD.org" Date: Fri, 06 Mar 2020 10:15:47 -0700 In-Reply-To: <2228511583493903@vla3-bebe75876e15.qloud-c.yandex.net> References: <2228511583493903@vla3-bebe75876e15.qloud-c.yandex.net> Content-Type: text/plain; charset="ASCII" X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 48YvQb6b5Nz3yWS X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-0.38 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_SPAM_MEDIUM(0.60)[0.602,0]; ASN(0.00)[asn:16509, ipnet:54.148.0.0/15, country:US]; NEURAL_HAM_LONG(-0.98)[-0.982,0] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2020 17:15:52 -0000 On Fri, 2020-03-06 at 16:25 +0500, Sergey Ivlev wrote: > Hello! > > I've successfully built FreeBSD 13.0-CURRENT (r358609) image for > ARMv7 on AMD64 host using release(7). > > I wonder is it possible for AMD64 host to build/install ports into > the image? > > I've defined arm_do_quirk function in my configuration file, where > I'm calling `make DESTDIR=${CHROOTDIR}/${DESTDIR} TARGET=arm > TARGET_ARCH=armv7 ... build install clean`. But it ended up with > error, of course, because that make(1) chroots into > ${CHROOTDIR}/${DESTDIR} and runs executables from there, but > ${CHROOTDIR}/${DESTDIR} is where **built target** located (i.e. where > files for ARMv7; ${CHROOTDIR}/${DESTDIR} is {my chrrot > dir}/usr/obj/usr/src/arm.armv7/release/{kernel name}). Altering the > PATH variable so it will specify ${CHROOTDIR} as the first entry, > doesn't help either -- I end up with the same ``exec format error'' > (env PATH="${CHROOTDIR}:${PATH}" make DESTDIR=...). > > I had also tried to `chroot ${CHROOTDIR} make DESTDIR=${DESTDIR} > TARGET=arm TARGET_ARCH=armv7 ... build install clean`, but > mount_nullfs(8) failed under chroot(8). > > Therefore, my question is: is it possible while building an image for > ARMv7 on AMD64 host with release(7) also building and installing > ports into the result image? > > Thanks in advance! > Cross-building ports is done with poudriere and qemu. There is some info on setting it up in https://wiki.freebsd.org/QemuUserModeHowTo but I think that's pretty out of date. There may be newer how-to info that I'm not aware of. I'm going to CC the freebsd-arm@ list on this reply, because the folks there probably know of better information resources for all this. I also have no idea how to integrate poudriere-based building with release(7) stuff. It may be a matter of building a local package repo using poudriere, then configuring the release script stuff to use it. -- Ian From owner-freebsd-hackers@freebsd.org Sat Mar 7 05:33:42 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5152325BD29 for ; Sat, 7 Mar 2020 05:33:42 +0000 (UTC) (envelope-from keno@juliacomputing.com) Received: from mail-io1-xd36.google.com (mail-io1-xd36.google.com [IPv6:2607:f8b0:4864:20::d36]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48ZCnw5Ysbz3P8c for ; Sat, 7 Mar 2020 05:33:40 +0000 (UTC) (envelope-from keno@juliacomputing.com) Received: by mail-io1-xd36.google.com with SMTP id r15so4260859iog.0 for ; Fri, 06 Mar 2020 21:33:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=juliacomputing-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=xzb6DzfVJJ/BlLl2W/P+AjQD+rB0jCaZeEszsIY29Ro=; b=oxPGlfwkQPOaE+vmaz/e/DiEapR0RYG5A6Njms11f+6IAWKvZdLoe5hamyGHz+BORf Nh+JVGgu9V48u0RehIrNSiFghkXol4shWxLrZeZ3BcbZfo5y4vzbwtaASzhXoTTPMXZ7 cUvx8PXMKAoD+bScFVyzkVlV2QqHJ1cPJUCpR2Lotk998nEojyzlOFM+4D7rwU25KSzF nWxZNPsDmttzYuO7E/JqLIrj+v/NTX2w/PRJ59x0T+aPuif4sJW8FBxR4AloRI3bO7gf L2yY9hLGOQXaldhf9zSaimswNRyW/nU9k4ziF9jE2WVGnEh+0WG9UW4yiFsvFbXm1QJU xqRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=xzb6DzfVJJ/BlLl2W/P+AjQD+rB0jCaZeEszsIY29Ro=; b=piCC2AD7rNfQ50c5Lz1PNBs6UVj6y6EVPiJnhk3O1gPLG4scGSUg9yEiiVHIeScngq 9LvtcU7E3oWfAOBO7qDxywcJgiAm3NATtwM01yF4gcPiV5YsKaq6SUORscx5cHNN5SWk D1SMzCwmtQ0NN6e3gC+5esKvy3ATSF6i9oFtmOMnLXuXYhuzwVheBKBsI0CvKPNgjjeo /AYBI/w+cwicDxCyzn4s4/MmQ1sc4z4xsng/xEhk9i0/g2PByRn01XILVV8TicOIp4BD lk7Jm4IEidfJYcHJVocRdvpOspO6439xRKJC4N1ITu+dXpYku8Gq4pfXvIwVJK2c3781 rEwQ== X-Gm-Message-State: ANhLgQ2xQlzusD1pnu06RYMba3qBUyTRk3LsIXzT+wkxWGuWrcZDrNO3 XtdQ8ndgm5f7WkR+OiNIsYPWFwJ29TnzwkrmnrPi/A== X-Google-Smtp-Source: ADFU+vsYR77FxUR39DLZYJUI6Fr+liDFCIQ7Q0AyJ8X55uowBZ2/TrN2617Li6f+h7WMRXu5lSrNwgbEceiRvZkEauU= X-Received: by 2002:a5e:8f45:: with SMTP id x5mr5743663iop.19.1583559218192; Fri, 06 Mar 2020 21:33:38 -0800 (PST) MIME-Version: 1.0 References: <20200304233906.GB98340@kib.kiev.ua> In-Reply-To: <20200304233906.GB98340@kib.kiev.ua> From: Keno Fischer Date: Sat, 7 Mar 2020 00:33:02 -0500 Message-ID: Subject: Re: FreeBSD Pipe behavior in pipe OOM situations To: Konstantin Belousov Cc: freebsd-hackers@freebsd.org, Elliot Saba X-Rspamd-Queue-Id: 48ZCnw5Ysbz3P8c X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=juliacomputing-com.20150623.gappssmtp.com header.s=20150623 header.b=oxPGlfwk; dmarc=none; spf=none (mx1.freebsd.org: domain of keno@juliacomputing.com has no SPF policy when checking 2607:f8b0:4864:20::d36) smtp.mailfrom=keno@juliacomputing.com X-Spamd-Result: default: False [-4.30 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[juliacomputing-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[juliacomputing.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[juliacomputing-com.20150623.gappssmtp.com:+]; RCVD_IN_DNSWL_NONE(0.00)[6.3.d.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-2.00)[ip: (-6.43), ipnet: 2607:f8b0::/32(-1.86), asn: 15169(-1.65), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Mar 2020 05:33:42 -0000 Hi Konstantin, thanks for getting back to me, First, there is a requirement that an atomic write size exists, i.e. writes > less than SC_PIPE_BUF are guaranteed to not interleave if succeeded. Our > PIPE_BUF is 512 bytes. > Useful to know, thanks. > I think that unexplained blocking (it is very hard to track down such > state) is worse then ENOMEM outcome. > This is probably true, but the ENOMEM behavior is by no means benign. If a process exhausts all pipe kva, the next process trying to allocate and write to a pipe will probably crash. That could basically be anything in the system. From the ssh server to (in our case) the infrastructure that runs jobs. Of course arguable the same thing happens in a regular ENOMEM situation also, but between paging and user space monitoring of memory usage, such situations seem easier to manage. I guess there may also be a concern about dos-style attacks. It seems pretty easy to allocate enough pipes to exhaust that limit. Regardless, I just wanted to raise this, since I considered the behavior odd and we didn't see it elsewhere. We have since found the culprit for the OOM condition: One of our tests tries to provoke an EMFILE condition to test our handling of this corner case, so it just fills every unallocated fd with pipes. However, since it doesn't write to them, we never actually see a failure there. Instead some random other process in the system will crash. Sometimes another test run (where we saw the error), but occasionally also the CI system itself. I suspect this is responsible for a fair number of mysterious failures we observed. From our perspective this issue should be resolved - I guess I'll leave it to you to decide whether there's anything to be done about the denial of service concern. I don't know what guarantees FreeBSD makes for kernel resource usage particularly in the context of jails, so I don't know if this is of concern at all. In regular usage, without a malicious program (or well-it's-malicious-but-it's-a-test-script), the admin probably would just bump the sysctl and everything would keep running nicely. Thanks again for your detailed reply. Keno From owner-freebsd-hackers@freebsd.org Sat Mar 7 11:42:25 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2AE5E262958 for ; Sat, 7 Mar 2020 11:42:25 +0000 (UTC) (envelope-from kib@freebsd.org) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 48ZMzM5vjqz455W for ; Sat, 7 Mar 2020 11:42:23 +0000 (UTC) (envelope-from kib@freebsd.org) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id 027BgEo3045292 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Sat, 7 Mar 2020 13:42:17 +0200 (EET) (envelope-from kib@freebsd.org) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 027BgEo3045292 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id 027BgERS045291; Sat, 7 Mar 2020 13:42:14 +0200 (EET) (envelope-from kib@freebsd.org) X-Authentication-Warning: tom.home: kostik set sender to kib@freebsd.org using -f Date: Sat, 7 Mar 2020 13:42:14 +0200 From: Konstantin Belousov To: Keno Fischer Cc: freebsd-hackers@freebsd.org, Elliot Saba Subject: Re: FreeBSD Pipe behavior in pipe OOM situations Message-ID: <20200307114214.GH98340@kib.kiev.ua> References: <20200304233906.GB98340@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on tom.home X-Rspamd-Queue-Id: 48ZMzM5vjqz455W X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-0.00 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_SPAM_MEDIUM(0.97)[0.968,0]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; NEURAL_HAM_LONG(-0.97)[-0.971,0] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Mar 2020 11:42:25 -0000 On Sat, Mar 07, 2020 at 12:33:02AM -0500, Keno Fischer wrote: > Hi Konstantin, > > thanks for getting back to me, > > First, there is a requirement that an atomic write size exists, i.e. writes > > less than SC_PIPE_BUF are guaranteed to not interleave if succeeded. Our > > PIPE_BUF is 512 bytes. > > > > Useful to know, thanks. > > > > I think that unexplained blocking (it is very hard to track down such > > state) is worse then ENOMEM outcome. > > > > This is probably true, but the ENOMEM behavior is by no means benign. > If a process exhausts all pipe kva, the next process trying to allocate and > write to a pipe will probably crash. That could basically be anything in the > system. From the ssh server to (in our case) the infrastructure that runs > jobs. > Of course arguable the same thing happens in a regular ENOMEM situation > also, but between paging and user space monitoring of memory usage, such > situations seem easier to manage. I guess there may also be a concern about > dos-style attacks. It seems pretty easy to allocate enough pipes to exhaust > that limit. I do not disagree, but we do not have a good way out if the pipe KVA is exhausted. I can propose a different approach to it: allocate at least the minimal sized buffer on pipe creation, and return ENOMEM if allocation failed. Then userspace can get ENOMEM on e.g. pipe(2) or open of the named fifo, but if the pipe creation failed, then write(2) is guaranteed to not return with ENOMEM. Prototype of the change is at https://reviews.freebsd.org/D23993, I did not yet even booted with it. > > Regardless, I just wanted to raise this, since I considered the behavior > odd and > we didn't see it elsewhere. We have since found the culprit for the OOM > condition: > One of our tests tries to provoke an EMFILE condition to test our handling > of this > corner case, so it just fills every unallocated fd with pipes. However, > since it doesn't > write to them, we never actually see a failure there. Instead some random > other process > in the system will crash. Sometimes another test run (where we saw the > error), but > occasionally also the CI system itself. I suspect this is responsible for a > fair number > of mysterious failures we observed. From our perspective this issue should > be resolved - > I guess I'll leave it to you to decide whether there's anything to be done > about the denial > of service concern. I don't know what guarantees FreeBSD makes for kernel > resource > usage particularly in the context of jails, so I don't know if this is of > concern at all. In > regular usage, without a malicious program (or > well-it's-malicious-but-it's-a-test-script), > the admin probably would just bump the sysctl and everything would keep > running nicely. With that change in place you are more likely to get ENOMEM than EMFILE. > > Thanks again for your detailed reply. > > Keno