From owner-freebsd-arch@FreeBSD.ORG Sun Feb 19 00:22:27 2012 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx2.freebsd.org (mx2.freebsd.org [IPv6:2001:4f8:fff6::35]) by hub.freebsd.org (Postfix) with ESMTP id 9A94A106564A; Sun, 19 Feb 2012 00:22:27 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from 172-17-197-151.globalsuite.net (hub.freebsd.org [IPv6:2001:4f8:fff6::36]) by mx2.freebsd.org (Postfix) with ESMTP id 13D501768B0; Sun, 19 Feb 2012 00:22:27 +0000 (UTC) Message-ID: <4F4040C2.3060604@FreeBSD.org> Date: Sat, 18 Feb 2012 16:22:26 -0800 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:10.0.2) Gecko/20120218 Thunderbird/10.0.2 MIME-Version: 1.0 To: Don Lewis References: <201202171845.q1HIj1vP011584@gw.catspoiler.org> In-Reply-To: <201202171845.q1HIj1vP011584@gw.catspoiler.org> X-Enigmail-Version: 1.3.5 OpenPGP: id=1A1ABC84 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: arch@FreeBSD.org Subject: Re: [patch] allow crash dumps to Linux swap partitions X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Feb 2012 00:22:27 -0000 On 02/17/2012 10:45, Don Lewis wrote: > I made the same change to g_part_ebr.c that I made to g_part_mbr.c but I > didn't have a way of testing it because I don't have any drives with > extended partitions. Is the partition type 0x82? Maybe there's another > problem in g_part_ebr_dumpto(). It's probably never been exercised > before. The data for partition 4 is: sysid 15 (0x0f),(Extended DOS (LBA)) -- It's always a long day; 86400 doesn't fit into a short. Breadth of IT experience, and depth of knowledge in the DNS. Yours for the right price. :) http://SupersetSolutions.com/ From owner-freebsd-arch@FreeBSD.ORG Sun Feb 19 00:46:23 2012 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx2.freebsd.org (mx2.freebsd.org [IPv6:2001:4f8:fff6::35]) by hub.freebsd.org (Postfix) with ESMTP id C6B75106566B; Sun, 19 Feb 2012 00:46:23 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from 172-17-197-151.globalsuite.net (hub.freebsd.org [IPv6:2001:4f8:fff6::36]) by mx2.freebsd.org (Postfix) with ESMTP id A8CF5150312; Sun, 19 Feb 2012 00:46:22 +0000 (UTC) Message-ID: <4F40465E.4030909@FreeBSD.org> Date: Sat, 18 Feb 2012 16:46:22 -0800 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:10.0.2) Gecko/20120218 Thunderbird/10.0.2 MIME-Version: 1.0 To: "Andrey V. Elsukov" References: <201202171857.q1HIvZcC011615@gw.catspoiler.org> <4F3FE4D2.4090803@FreeBSD.org> In-Reply-To: <4F3FE4D2.4090803@FreeBSD.org> X-Enigmail-Version: 1.3.5 OpenPGP: id=1A1ABC84 Content-Type: multipart/mixed; boundary="------------020708030107030008020108" Cc: arch@FreeBSD.org, Don Lewis Subject: Re: [patch] allow crash dumps to Linux swap partitions X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Feb 2012 00:46:23 -0000 This is a multi-part message in MIME format. --------------020708030107030008020108 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit On 02/18/2012 09:50, Andrey V. Elsukov wrote: > I think we can check bp->bio_from field, something like that (not tested): That made dumpon happy anyway, thanks! I'm a bit rusty on "how to cause a dump on purpose" nowadays .... I'm running 8-stable right now, if someone wants to give me a hint I'll test this. I attached a version of the patch with your updated src change, and a tweak to the comment (still not sure I'm right with that, but I didn't understand yours, sorry). Doug -- It's always a long day; 86400 doesn't fit into a short. Breadth of IT experience, and depth of knowledge in the DNS. Yours for the right price. :) http://SupersetSolutions.com/ --------------020708030107030008020108 Content-Type: text/plain; name="kerneldump.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="kerneldump.diff" Index: geom/part/g_part.c =================================================================== --- geom/part/g_part.c (revision 231902) +++ geom/part/g_part.c (working copy) @@ -2042,9 +2042,14 @@ /* * Check that the partition is suitable for kernel * dumps. Typically only swap partitions should be - * used. + * used. If the request comes from the nested scheme + * we allow dumping there as well. */ - if (!G_PART_DUMPTO(table, entry)) { + if (table->gpt_depth == 0 && + bp->bio_from != NULL && + bp->bio_from->geom->class == &g_part_class) { + /* FALLTHROUGH */ + } else if (G_PART_DUMPTO(table, entry) == 0) { g_io_deliver(bp, ENODEV); printf("GEOM_PART: Partition '%s' not suitable" " for kernel dumps (wrong type?)\n", --------------020708030107030008020108-- From owner-freebsd-arch@FreeBSD.ORG Sun Feb 19 00:54:29 2012 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9F130106564A; Sun, 19 Feb 2012 00:54:29 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from gw.catspoiler.org (gw.catspoiler.org [75.1.14.242]) by mx1.freebsd.org (Postfix) with ESMTP id 6969E8FC12; Sun, 19 Feb 2012 00:54:29 +0000 (UTC) Received: from FreeBSD.org (mousie.catspoiler.org [192.168.101.2]) by gw.catspoiler.org (8.13.3/8.13.3) with ESMTP id q1J0sK60017096; Sat, 18 Feb 2012 16:54:24 -0800 (PST) (envelope-from truckman@FreeBSD.org) Message-Id: <201202190054.q1J0sK60017096@gw.catspoiler.org> Date: Sat, 18 Feb 2012 16:54:20 -0800 (PST) From: Don Lewis To: dougb@FreeBSD.org In-Reply-To: <4F40465E.4030909@FreeBSD.org> MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Cc: ae@FreeBSD.org, arch@FreeBSD.org Subject: Re: [patch] allow crash dumps to Linux swap partitions X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Feb 2012 00:54:29 -0000 On 18 Feb, Doug Barton wrote: > On 02/18/2012 09:50, Andrey V. Elsukov wrote: >> I think we can check bp->bio_from field, something like that (not tested): > > That made dumpon happy anyway, thanks! I'm a bit rusty on "how to cause > a dump on purpose" nowadays .... I'm running 8-stable right now, if > someone wants to give me a hint I'll test this. Break into DDB and type "call doadump". From owner-freebsd-arch@FreeBSD.ORG Sun Feb 19 00:57:16 2012 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx2.freebsd.org (mx2.freebsd.org [IPv6:2001:4f8:fff6::35]) by hub.freebsd.org (Postfix) with ESMTP id 16C231065670; Sun, 19 Feb 2012 00:57:16 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from 172-17-197-151.globalsuite.net (hub.freebsd.org [IPv6:2001:4f8:fff6::36]) by mx2.freebsd.org (Postfix) with ESMTP id 73D7514F680; Sun, 19 Feb 2012 00:57:15 +0000 (UTC) Message-ID: <4F4048EB.1070402@FreeBSD.org> Date: Sat, 18 Feb 2012 16:57:15 -0800 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:10.0.2) Gecko/20120218 Thunderbird/10.0.2 MIME-Version: 1.0 To: Don Lewis References: <201202190054.q1J0sK60017096@gw.catspoiler.org> In-Reply-To: <201202190054.q1J0sK60017096@gw.catspoiler.org> X-Enigmail-Version: 1.3.5 OpenPGP: id=1A1ABC84 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: ae@FreeBSD.org, arch@FreeBSD.org Subject: Re: [patch] allow crash dumps to Linux swap partitions X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Feb 2012 00:57:16 -0000 On 02/18/2012 16:54, Don Lewis wrote: > On 18 Feb, Doug Barton wrote: >> On 02/18/2012 09:50, Andrey V. Elsukov wrote: >>> I think we can check bp->bio_from field, something like that (not tested): >> >> That made dumpon happy anyway, thanks! I'm a bit rusty on "how to cause >> a dump on purpose" nowadays .... I'm running 8-stable right now, if >> someone wants to give me a hint I'll test this. > > Break into DDB and type "call doadump". Yeah, that's what I used to do, but Ctrl-Alt-Esc didn't do anything for me. Do I have to compile in options BREAK_TO_DEBUGGER even on a desktop? Doug -- It's always a long day; 86400 doesn't fit into a short. Breadth of IT experience, and depth of knowledge in the DNS. Yours for the right price. :) http://SupersetSolutions.com/ From owner-freebsd-arch@FreeBSD.ORG Sun Feb 19 01:00:55 2012 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx2.freebsd.org (mx2.freebsd.org [IPv6:2001:4f8:fff6::35]) by hub.freebsd.org (Postfix) with ESMTP id 23E971065675; Sun, 19 Feb 2012 01:00:55 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from 172-17-197-151.globalsuite.net (hub.freebsd.org [IPv6:2001:4f8:fff6::36]) by mx2.freebsd.org (Postfix) with ESMTP id BE404150A2A; Sun, 19 Feb 2012 01:00:53 +0000 (UTC) Message-ID: <4F4049C5.6040605@FreeBSD.org> Date: Sat, 18 Feb 2012 17:00:53 -0800 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:10.0.2) Gecko/20120218 Thunderbird/10.0.2 MIME-Version: 1.0 To: Don Lewis References: <201202190054.q1J0sK60017096@gw.catspoiler.org> <4F4048EB.1070402@FreeBSD.org> In-Reply-To: <4F4048EB.1070402@FreeBSD.org> X-Enigmail-Version: 1.3.5 OpenPGP: id=1A1ABC84 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: ae@FreeBSD.org, arch@FreeBSD.org Subject: Re: [patch] allow crash dumps to Linux swap partitions X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Feb 2012 01:00:55 -0000 On 02/18/2012 16:57, Doug Barton wrote: > On 02/18/2012 16:54, Don Lewis wrote: >> On 18 Feb, Doug Barton wrote: >>> On 02/18/2012 09:50, Andrey V. Elsukov wrote: >>>> I think we can check bp->bio_from field, something like that (not tested): >>> >>> That made dumpon happy anyway, thanks! I'm a bit rusty on "how to cause >>> a dump on purpose" nowadays .... I'm running 8-stable right now, if >>> someone wants to give me a hint I'll test this. >> >> Break into DDB and type "call doadump". > > Yeah, that's what I used to do, but Ctrl-Alt-Esc didn't do anything for > me. Do I have to compile in options BREAK_TO_DEBUGGER even on a desktop? Gah, PEBKAC! -stable != current .... I have been running current so long I forgot that DDB isn't in the default -stable kernel. Will fix and test the dump. Doug -- It's always a long day; 86400 doesn't fit into a short. Breadth of IT experience, and depth of knowledge in the DNS. Yours for the right price. :) http://SupersetSolutions.com/ From owner-freebsd-arch@FreeBSD.ORG Sun Feb 19 01:01:45 2012 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0D8AC1065670; Sun, 19 Feb 2012 01:01:45 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from gw.catspoiler.org (gw.catspoiler.org [75.1.14.242]) by mx1.freebsd.org (Postfix) with ESMTP id C9E0A8FC17; Sun, 19 Feb 2012 01:01:44 +0000 (UTC) Received: from FreeBSD.org (mousie.catspoiler.org [192.168.101.2]) by gw.catspoiler.org (8.13.3/8.13.3) with ESMTP id q1J11cnJ017127; Sat, 18 Feb 2012 17:01:41 -0800 (PST) (envelope-from truckman@FreeBSD.org) Message-Id: <201202190101.q1J11cnJ017127@gw.catspoiler.org> Date: Sat, 18 Feb 2012 17:01:38 -0800 (PST) From: Don Lewis To: dougb@FreeBSD.org In-Reply-To: <4F4048EB.1070402@FreeBSD.org> MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Cc: ae@FreeBSD.org, arch@FreeBSD.org Subject: Re: [patch] allow crash dumps to Linux swap partitions X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Feb 2012 01:01:45 -0000 On 18 Feb, Doug Barton wrote: > On 02/18/2012 16:54, Don Lewis wrote: >> On 18 Feb, Doug Barton wrote: >>> On 02/18/2012 09:50, Andrey V. Elsukov wrote: >>>> I think we can check bp->bio_from field, something like that (not tested): >>> >>> That made dumpon happy anyway, thanks! I'm a bit rusty on "how to cause >>> a dump on purpose" nowadays .... I'm running 8-stable right now, if >>> someone wants to give me a hint I'll test this. >> >> Break into DDB and type "call doadump". > > Yeah, that's what I used to do, but Ctrl-Alt-Esc didn't do anything for > me. Do I have to compile in options BREAK_TO_DEBUGGER even on a desktop? Looks like it. I've got that option in my custom kernel config files that start with "include GENERIC". From owner-freebsd-arch@FreeBSD.ORG Sun Feb 19 01:33:18 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B676C106566B; Sun, 19 Feb 2012 01:33:18 +0000 (UTC) (envelope-from nparhar@gmail.com) Received: from mail-bk0-f54.google.com (mail-bk0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id AD7F78FC0C; Sun, 19 Feb 2012 01:33:17 +0000 (UTC) Received: by bkcjg1 with SMTP id jg1so5252237bkc.13 for ; Sat, 18 Feb 2012 17:33:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=qh/VdV7luniIjDIj91X9MJXLNAJI+6D6Ny5xwE5wGv8=; b=p1Omcfk9/JJhkrbxxSlM6zSjoFFRxketTBEMvtwIlSQeBQTOjLqpMbvvi4qZs6AY9E 5UMTuIp4WJ5I/bmgGsAF0y15zOx0dtAgTGfn2h5LNWXGdIv1ukjcWe4tJrfM41P+YDtN WDeJCF23+RuCsrJb0Mhi3KaflJtfuvM7Ccacg= MIME-Version: 1.0 Received: by 10.204.154.92 with SMTP id n28mr6736247bkw.30.1329613398136; Sat, 18 Feb 2012 17:03:18 -0800 (PST) Received: by 10.204.190.143 with HTTP; Sat, 18 Feb 2012 17:03:18 -0800 (PST) In-Reply-To: <4F4048EB.1070402@FreeBSD.org> References: <201202190054.q1J0sK60017096@gw.catspoiler.org> <4F4048EB.1070402@FreeBSD.org> Date: Sat, 18 Feb 2012 17:03:18 -0800 Message-ID: From: Navdeep Parhar To: Doug Barton Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: ae@freebsd.org, Don Lewis , arch@freebsd.org Subject: Re: [patch] allow crash dumps to Linux swap partitions X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Feb 2012 01:33:18 -0000 On Sat, Feb 18, 2012 at 4:57 PM, Doug Barton wrote: > On 02/18/2012 16:54, Don Lewis wrote: >> On 18 Feb, Doug Barton wrote: >>> On 02/18/2012 09:50, Andrey V. Elsukov wrote: >>>> I think we can check bp->bio_from field, something like that (not test= ed): >>> >>> That made dumpon happy anyway, thanks! =A0I'm a bit rusty on "how to ca= use >>> a dump on purpose" nowadays .... I'm running 8-stable right now, if >>> someone wants to give me a hint I'll test this. >> >> Break into DDB and type "call doadump". > > Yeah, that's what I used to do, but Ctrl-Alt-Esc didn't do anything for > me. Do I have to compile in options BREAK_TO_DEBUGGER even on a desktop? You could try a "reboot -d" if all you want is a crash dump. Navdeep > > > Doug > > -- > > =A0 =A0 =A0 =A0It's always a long day; 86400 doesn't fit into a short. > > =A0 =A0 =A0 =A0Breadth of IT experience, and depth of knowledge in the DN= S. > =A0 =A0 =A0 =A0Yours for the right price. =A0:) =A0http://SupersetSolutio= ns.com/ > > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Sun Feb 19 02:09:24 2012 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx2.freebsd.org (mx2.freebsd.org [IPv6:2001:4f8:fff6::35]) by hub.freebsd.org (Postfix) with ESMTP id DA61E1065673; Sun, 19 Feb 2012 02:09:24 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from 172-17-197-151.globalsuite.net (hub.freebsd.org [IPv6:2001:4f8:fff6::36]) by mx2.freebsd.org (Postfix) with ESMTP id BFE5114D9B1; Sun, 19 Feb 2012 02:09:22 +0000 (UTC) Message-ID: <4F4059D2.30302@FreeBSD.org> Date: Sat, 18 Feb 2012 18:09:22 -0800 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:10.0.2) Gecko/20120218 Thunderbird/10.0.2 MIME-Version: 1.0 To: "Andrey V. Elsukov" References: <201202171857.q1HIvZcC011615@gw.catspoiler.org> <4F3FE4D2.4090803@FreeBSD.org> <4F40465E.4030909@FreeBSD.org> In-Reply-To: <4F40465E.4030909@FreeBSD.org> X-Enigmail-Version: 1.3.5 OpenPGP: id=1A1ABC84 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: arch@FreeBSD.org, Don Lewis Subject: Re: [patch] allow crash dumps to Linux swap partitions X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Feb 2012 02:09:24 -0000 On 02/18/2012 16:46, Doug Barton wrote: > That made dumpon happy anyway, thanks! Ok, Andrey's patch works for both artificial dumps and (tragically) real-world dumping scenarios that I experienced moments later. :) And for the record, with just DDB actually in the kernel Ctrl-Alt-Esc works as expected. Doug -- It's always a long day; 86400 doesn't fit into a short. Breadth of IT experience, and depth of knowledge in the DNS. Yours for the right price. :) http://SupersetSolutions.com/ From owner-freebsd-arch@FreeBSD.ORG Sun Feb 19 09:55:28 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5D8591065672; Sun, 19 Feb 2012 09:55:28 +0000 (UTC) (envelope-from to.my.trociny@gmail.com) Received: from mail-bk0-f54.google.com (mail-bk0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 7A8CE8FC0A; Sun, 19 Feb 2012 09:55:27 +0000 (UTC) Received: by bkcjg1 with SMTP id jg1so5396686bkc.13 for ; Sun, 19 Feb 2012 01:55:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=from:to:cc:subject:references:x-comment-to:sender:date:in-reply-to :message-id:user-agent:mime-version:content-type; bh=tl5yeoBmbCvJmqfIcdvrMpqlFmiztvar4PmPvp2PgdQ=; b=OfDaafVMzwJerozqarEsl9Sw8Vx3m3XeD54tN4UoP8LP53PYmIDF/zJtIpb5iahr7y Qe7PmsCLWl5gSXf+VfDBdJCPFMQAHFZfQR7q0GXjmMzVrlTAlPfQaeRwKbHBrKsNCMMn KM4dPShPl00geK1ni8AeSLoDQPelhkYgUhZJI= Received: by 10.204.141.11 with SMTP id k11mr7186204bku.5.1329645326254; Sun, 19 Feb 2012 01:55:26 -0800 (PST) Received: from localhost ([95.69.173.122]) by mx.google.com with ESMTPS id jd17sm32575564bkb.4.2012.02.19.01.55.22 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 19 Feb 2012 01:55:24 -0800 (PST) From: Mikolaj Golub To: Konstantin Belousov References: <86sjjobzmn.fsf@kopusha.home.net> <86fwfnti5t.fsf@kopusha.home.net> <20120112215106.GC31224@deviant.kiev.zoral.com.ua> <86hazntwmu.fsf@kopusha.home.net> <20120123031238.GL31224@deviant.kiev.zoral.com.ua> <86zkcfu9ac.fsf@kopusha.home.net> <20120218215003.GM3283@deviant.kiev.zoral.com.ua> X-Comment-To: Konstantin Belousov Sender: Mikolaj Golub Date: Sun, 19 Feb 2012 11:55:20 +0200 In-Reply-To: <20120218215003.GM3283@deviant.kiev.zoral.com.ua> (Konstantin Belousov's message of "Sat, 18 Feb 2012 23:50:03 +0200") Message-ID: <86ehtrf8qf.fsf@kopusha.home.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: "Robert N. M. Watson" , freebsd-arch@freebsd.org Subject: Re: unix domain sockets on nullfs(5) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Feb 2012 09:55:28 -0000 On Sat, 18 Feb 2012 23:50:03 +0200 Konstantin Belousov wrote: KB> On Sat, Feb 18, 2012 at 11:22:03PM +0200, Mikolaj Golub wrote: >> After collecting all suggestions and additional testing I have got this patch >> set: >> >> http://people.freebsd.org/~trociny/unp_prepare_reclaim.1.patch KB> Including unpcb.h into vfs_subr.c looks too extreme. Put the prototype KB> into vnode.h, possibly renaming the function to vfs_unp_reclaim. Done. >> http://people.freebsd.org/~trociny/unp_connect.LOCKSHARED.1.patch >> http://people.freebsd.org/~trociny/VOP_UNP.3.patch KB> I has a painting suggestion there, call the vops VOP_UNP_DETACH etc, KB> otherwise it takes too much reading to understand that it is not undetach. Unfortunately, vnode_if.awk does not support underscores in VOP names. It failed with "Invalid %% construction" due to the restriction: $2 !~ /^[a-z]+$/. So I reverted back to the old naming. The updated patches: http://people.freebsd.org/~trociny/unp_prepare_reclaim.2.patch http://people.freebsd.org/~trociny/unp_connect.LOCKSHARED.1.patch http://people.freebsd.org/~trociny/VOP_UNP.4.patch -- Mikolaj Golub From owner-freebsd-arch@FreeBSD.ORG Sun Feb 19 10:49:06 2012 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0957E106566C; Sun, 19 Feb 2012 10:49:06 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from fallbackmx06.syd.optusnet.com.au (fallbackmx06.syd.optusnet.com.au [211.29.132.8]) by mx1.freebsd.org (Postfix) with ESMTP id 4FC2C8FC08; Sun, 19 Feb 2012 10:49:04 +0000 (UTC) Received: from mail08.syd.optusnet.com.au (mail08.syd.optusnet.com.au [211.29.132.189]) by fallbackmx06.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id q1J7hGUl022890; Sun, 19 Feb 2012 18:43:16 +1100 Received: from c211-30-171-136.carlnfd1.nsw.optusnet.com.au (c211-30-171-136.carlnfd1.nsw.optusnet.com.au [211.30.171.136]) by mail08.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id q1J7hCpT011087 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 19 Feb 2012 18:43:13 +1100 Date: Sun, 19 Feb 2012 18:43:12 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Don Lewis In-Reply-To: <201202190101.q1J11cnJ017127@gw.catspoiler.org> Message-ID: <20120219170935.I873@besplex.bde.org> References: <201202190101.q1J11cnJ017127@gw.catspoiler.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: ae@FreeBSD.org, dougb@FreeBSD.org, arch@FreeBSD.org Subject: Re: [patch] allow crash dumps to Linux swap partitions X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Feb 2012 10:49:06 -0000 On Sat, 18 Feb 2012, Don Lewis wrote: > On 18 Feb, Doug Barton wrote: >> On 02/18/2012 16:54, Don Lewis wrote: >>> On 18 Feb, Doug Barton wrote: >>>> On 02/18/2012 09:50, Andrey V. Elsukov wrote: >>>>> I think we can check bp->bio_from field, something like that (not tested): >>>> >>>> That made dumpon happy anyway, thanks! I'm a bit rusty on "how to cause >>>> a dump on purpose" nowadays .... I'm running 8-stable right now, if >>>> someone wants to give me a hint I'll test this. >>> >>> Break into DDB and type "call doadump". >> >> Yeah, that's what I used to do, but Ctrl-Alt-Esc didn't do anything for >> me. Do I have to compile in options BREAK_TO_DEBUGGER even on a desktop? > Looks like it. I've got that option in my custom kernel config files > that start with "include GENERIC". This is not needed, since there is a sysctl (debug.kdb_break_to_debugger) and a tunable for it. But everything for configuration of this is broken. I reported the following bugs in more detail to the breaker, but received no reply. 1. BREAK_TO_DEBUGGER means "enter the debugger on a serial line break", not "use a breakpoint trap to enter the debugger". That was just confusing, but now BREAK_TO_DEBUGGER is abused for the latter (only as the default for the static configuration, but that is bad enough). The sysctl and the tunable repeat the naming error in lower case. The function name kdb_break() adds to the confusion (this function does the breakpoint trap, using 2 layers of wrappers to add bugs). 2. Serial line breaks to enter the debugger are too painful to use in practice, since they occur when you unplug the cable, which you often do when you are not looking at the console so you don't notice that the system stopped in ddb. So someone implemented an alternative way to enter the debugger from serial consoles. This was and is misnamed ALT_BREAK_TO_DEBUGGER. This increases the confusion -- it gave an alternative way to a serial line break of entering the debugger (it wasn't an alternative to BREAK_TO_DEBUGGER, since both could be configured and they worked mostly independently). Now it is abused as (the default for control of) a general alternative way of entering the debugger. 3. Since BREAK_TO_DEBUGGER is too hard to use for its correct purpose if you _might_ have a serial console (which is in all configurations that have a serial driver that support consoles, since you can enable this support dynamically), you have to leave it unconfigured, at least statically. This breaks debugger entry on the syscons console. 4. Sysons has its own static and dynamic configuration corresponding to serial drivers' previously semi-private BREAK_TO_DEBUGGER: - SC_DISABLE_KDBKEY gives static configuration (default enabled) - the hw.syscons_kbd_debug sysctl (default enabled) gives dynamic configuration. There is no tunable for this, and though I hate tunables, it makes sense to have one for this -- it may need to be used before booting to close a security hole. It doesn't help that the sysctls and tunables are undocumented (?), and in different namespaces, yet kbd looks too much like kdb. 5. The BREAK_TO_DEBUGGER/debug.kdb_break_to_debugger controls are worse than the old sysctl controls, since they are too global. You can view them as global controls for extra security, with all subsystems that support consoles being required to have local controls, with the access control being the AND of all the enables, but this is too complicated and only supported accidentally by 1 set of local controls in 1 driver (syscons). 6. To work around some of the above bugs, I keep leaving BREAK_TO_DEBUGGER out of static configurations, and set debug.kdb_break_to_debugger in rc. The latter doesn't break serial consoles since I only do this on systems with no serial consoles. This works, except when the system crashes early (can't run sysctl) or when booting to the single user shell (have to do the sysctl manually). I always configure ALT_BREAK_TO_DEBUGGER, since it causes no problems for serial consoles, but it is harder to use. 7. Next, there are the configuration bugs and security holes from configuring ALT_BREAK_TO_DEBUGGER. It used to be just for serial consoles. Now it is for all consoles. If you turn it on so that it works for one console, then it works for all, since are no local controls for it. Syscons' old local controls corresponding to the serial line break controls are defeated in another way by this. It doesn't help security to disable debugger entries via the syscons hotkeys if the escape sequence can be used. For complicated configurations, with multiple serial consoles and only some trusted for input, there must be local controls on every device. Even per-driver is too coarse, and we only had per-subsystem-group (all tty drivers together, with syscons special). The global controls just let you quickly close any security holes in misconfigured consoles, at the cost of more confusion and complexity in both simple and complicated configurations. Bruce From owner-freebsd-arch@FreeBSD.ORG Sun Feb 19 17:01:59 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CA1EC106564A; Sun, 19 Feb 2012 17:01:59 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 4A8DF8FC1B; Sun, 19 Feb 2012 17:01:57 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q1JH1pFC035035 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 19 Feb 2012 19:01:51 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q1JH1pN4049581; Sun, 19 Feb 2012 19:01:51 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q1JH1pKC049580; Sun, 19 Feb 2012 19:01:51 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 19 Feb 2012 19:01:51 +0200 From: Konstantin Belousov To: Mikolaj Golub Message-ID: <20120219170151.GS3283@deviant.kiev.zoral.com.ua> References: <86fwfnti5t.fsf@kopusha.home.net> <20120112215106.GC31224@deviant.kiev.zoral.com.ua> <86hazntwmu.fsf@kopusha.home.net> <20120123031238.GL31224@deviant.kiev.zoral.com.ua> <86zkcfu9ac.fsf@kopusha.home.net> <20120218215003.GM3283@deviant.kiev.zoral.com.ua> <86ehtrf8qf.fsf@kopusha.home.net> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="uNi3qDZqkobVgwr4" Content-Disposition: inline In-Reply-To: <86ehtrf8qf.fsf@kopusha.home.net> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.9 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: "Robert N. M. Watson" , freebsd-arch@freebsd.org Subject: Re: unix domain sockets on nullfs(5) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Feb 2012 17:02:00 -0000 --uNi3qDZqkobVgwr4 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Feb 19, 2012 at 11:55:20AM +0200, Mikolaj Golub wrote: >=20 > On Sat, 18 Feb 2012 23:50:03 +0200 Konstantin Belousov wrote: >=20 > KB> On Sat, Feb 18, 2012 at 11:22:03PM +0200, Mikolaj Golub wrote: > >> After collecting all suggestions and additional testing I have got th= is patch > >> set: > >>=20 > >> http://people.freebsd.org/~trociny/unp_prepare_reclaim.1.patch > KB> Including unpcb.h into vfs_subr.c looks too extreme. Put the prototy= pe > KB> into vnode.h, possibly renaming the function to vfs_unp_reclaim. >=20 > Done. >=20 > >> http://people.freebsd.org/~trociny/unp_connect.LOCKSHARED.1.patch > >> http://people.freebsd.org/~trociny/VOP_UNP.3.patch > KB> I has a painting suggestion there, call the vops VOP_UNP_DETACH etc, > KB> otherwise it takes too much reading to understand that it is not und= etach. >=20 > Unfortunately, vnode_if.awk does not support underscores in VOP names. It > failed with "Invalid %% construction" due to the restriction: $2 !~ /^[a-= z]+$/. > So I reverted back to the old naming. Ok, might be it makes sense to try to add underscore to the set of allowed symbols. >=20 > The updated patches: >=20 > http://people.freebsd.org/~trociny/unp_prepare_reclaim.2.patch > http://people.freebsd.org/~trociny/unp_connect.LOCKSHARED.1.patch > http://people.freebsd.org/~trociny/VOP_UNP.4.patch In the comment above vfs_unp_reclaim(), say "before socket-type vnode reclamation". I have no further comments. --uNi3qDZqkobVgwr4 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAk9BKv4ACgkQC3+MBN1Mb4hcigCeKs+nrQrCGOZhf1o6TFGa8YFK La0AoLWjyUvLlg5fZUYlgrZdayZjHsK1 =C0Nd -----END PGP SIGNATURE----- --uNi3qDZqkobVgwr4-- From owner-freebsd-arch@FreeBSD.ORG Mon Feb 20 09:03:08 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5A9A51065670; Mon, 20 Feb 2012 09:03:08 +0000 (UTC) (envelope-from to.my.trociny@gmail.com) Received: from mail-ee0-f54.google.com (mail-ee0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id 896138FC08; Mon, 20 Feb 2012 09:03:07 +0000 (UTC) Received: by eekb47 with SMTP id b47so2390848eek.13 for ; Mon, 20 Feb 2012 01:03:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=from:to:cc:subject:organization:references:sender:date:in-reply-to :message-id:user-agent:mime-version:content-type; bh=EZvtxh//YN97GwGk1A91N1n0s2orMdsTPN8vUS65uO4=; b=IluSlaN1ly6fm5Bf3GhxzzSdDdlj1tvo0JM1ZVNkwonx0buEcvJv4OUGzkQVaqDq+J SjK+zwkSrCclPrHSWfWmUCXB5KUCdQaQ82r0Jspihn+CLodmWakymKPedGsZC0NKWEpS XTkJ8TdVOzmTFByVyNrBMbEtn8z4yDVxagZ7I= Received: by 10.14.94.66 with SMTP id m42mr7916915eef.66.1329728586478; Mon, 20 Feb 2012 01:03:06 -0800 (PST) Received: from localhost ([94.27.39.186]) by mx.google.com with ESMTPS id c16sm75494186eei.1.2012.02.20.01.03.02 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 20 Feb 2012 01:03:04 -0800 (PST) From: Mikolaj Golub To: Konstantin Belousov Organization: TOA Ukraine References: <86fwfnti5t.fsf@kopusha.home.net> <20120112215106.GC31224@deviant.kiev.zoral.com.ua> <86hazntwmu.fsf@kopusha.home.net> <20120123031238.GL31224@deviant.kiev.zoral.com.ua> <86zkcfu9ac.fsf@kopusha.home.net> <20120218215003.GM3283@deviant.kiev.zoral.com.ua> <86ehtrf8qf.fsf@kopusha.home.net> <20120219170151.GS3283@deviant.kiev.zoral.com.ua> Sender: Mikolaj Golub Date: Mon, 20 Feb 2012 11:02:59 +0200 In-Reply-To: <20120219170151.GS3283@deviant.kiev.zoral.com.ua> (Konstantin Belousov's message of "Sun, 19 Feb 2012 19:01:51 +0200") Message-ID: <8662f1q3lo.fsf@in138.ua3> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: "Robert N. M. Watson" , freebsd-arch@freebsd.org Subject: Re: unix domain sockets on nullfs(5) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Feb 2012 09:03:08 -0000 On Sun, 19 Feb 2012 19:01:51 +0200 Konstantin Belousov wrote: KB> On Sun, Feb 19, 2012 at 11:55:20AM +0200, Mikolaj Golub wrote: >> >> On Sat, 18 Feb 2012 23:50:03 +0200 Konstantin Belousov wrote: >> >> KB> On Sat, Feb 18, 2012 at 11:22:03PM +0200, Mikolaj Golub wrote: >> >> After collecting all suggestions and additional testing I have got this patch >> >> set: >> >> >> >> http://people.freebsd.org/~trociny/unp_prepare_reclaim.1.patch >> KB> Including unpcb.h into vfs_subr.c looks too extreme. Put the prototype >> KB> into vnode.h, possibly renaming the function to vfs_unp_reclaim. >> >> Done. >> >> >> http://people.freebsd.org/~trociny/unp_connect.LOCKSHARED.1.patch >> >> http://people.freebsd.org/~trociny/VOP_UNP.3.patch >> KB> I has a painting suggestion there, call the vops VOP_UNP_DETACH etc, >> KB> otherwise it takes too much reading to understand that it is not undetach. >> >> Unfortunately, vnode_if.awk does not support underscores in VOP names. It >> failed with "Invalid %% construction" due to the restriction: $2 !~ /^[a-z]+$/. >> So I reverted back to the old naming. KB> Ok, might be it makes sense to try to add underscore to the set of KB> allowed symbols. It does not look difficult and the patch is trivial: http://people.freebsd.org/~trociny/vnode_if.awk.patch I built and booted the kernel with VOP_UNP_* using it. I could commit if other people are ok with this and don't see issues. >> >> The updated patches: >> >> http://people.freebsd.org/~trociny/unp_prepare_reclaim.2.patch >> http://people.freebsd.org/~trociny/unp_connect.LOCKSHARED.1.patch >> http://people.freebsd.org/~trociny/VOP_UNP.4.patch KB> In the comment above vfs_unp_reclaim(), say "before socket-type vnode KB> reclamation". KB> I have no further comments. Thanks. -- Mikolaj Golub From owner-freebsd-arch@FreeBSD.ORG Mon Feb 20 10:09:02 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C2296106566B; Mon, 20 Feb 2012 10:09:02 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 2D5948FC14; Mon, 20 Feb 2012 10:09:01 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q1KA8ccm039540 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 20 Feb 2012 12:08:42 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q1KA8cRg054821; Mon, 20 Feb 2012 12:08:38 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q1KA8chi054820; Mon, 20 Feb 2012 12:08:38 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 20 Feb 2012 12:08:38 +0200 From: Konstantin Belousov To: Mikolaj Golub Message-ID: <20120220100837.GA3283@deviant.kiev.zoral.com.ua> References: <20120112215106.GC31224@deviant.kiev.zoral.com.ua> <86hazntwmu.fsf@kopusha.home.net> <20120123031238.GL31224@deviant.kiev.zoral.com.ua> <86zkcfu9ac.fsf@kopusha.home.net> <20120218215003.GM3283@deviant.kiev.zoral.com.ua> <86ehtrf8qf.fsf@kopusha.home.net> <20120219170151.GS3283@deviant.kiev.zoral.com.ua> <8662f1q3lo.fsf@in138.ua3> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ce/x92YNq88pcHEK" Content-Disposition: inline In-Reply-To: <8662f1q3lo.fsf@in138.ua3> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.9 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: "Robert N. M. Watson" , freebsd-arch@freebsd.org Subject: Re: unix domain sockets on nullfs(5) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Feb 2012 10:09:02 -0000 --ce/x92YNq88pcHEK Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Feb 20, 2012 at 11:02:59AM +0200, Mikolaj Golub wrote: >=20 > On Sun, 19 Feb 2012 19:01:51 +0200 Konstantin Belousov wrote: >=20 > KB> On Sun, Feb 19, 2012 at 11:55:20AM +0200, Mikolaj Golub wrote: > >>=20 > >> Unfortunately, vnode_if.awk does not support underscores in VOP names= . It > >> failed with "Invalid %% construction" due to the restriction: $2 !~ /= ^[a-z]+$/. > >> So I reverted back to the old naming. > KB> Ok, might be it makes sense to try to add underscore to the set of > KB> allowed symbols. >=20 > It does not look difficult and the patch is trivial: >=20 > http://people.freebsd.org/~trociny/vnode_if.awk.patch >=20 > I built and booted the kernel with VOP_UNP_* using it. >=20 > I could commit if other people are ok with this and don't see issues. I cannot imagine a failure from this change. --ce/x92YNq88pcHEK Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAk9CG6UACgkQC3+MBN1Mb4h3uwCg8PodgFkal5XIGlq2UQu11kIn uGMAoNlKBATuEZWaedhdZB7YVynsl3ZX =gNmd -----END PGP SIGNATURE----- --ce/x92YNq88pcHEK-- From owner-freebsd-arch@FreeBSD.ORG Tue Feb 21 17:50:21 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E5F551065670 for ; Tue, 21 Feb 2012 17:50:20 +0000 (UTC) (envelope-from matthewstory@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 911C68FC16 for ; Tue, 21 Feb 2012 17:50:20 +0000 (UTC) Received: by vbbfa15 with SMTP id fa15so6320601vbb.13 for ; Tue, 21 Feb 2012 09:50:19 -0800 (PST) Received-SPF: pass (google.com: domain of matthewstory@gmail.com designates 10.52.76.196 as permitted sender) client-ip=10.52.76.196; Authentication-Results: mr.google.com; spf=pass (google.com: domain of matthewstory@gmail.com designates 10.52.76.196 as permitted sender) smtp.mail=matthewstory@gmail.com; dkim=pass header.i=matthewstory@gmail.com Received: from mr.google.com ([10.52.76.196]) by 10.52.76.196 with SMTP id m4mr12448935vdw.112.1329846619647 (num_hops = 1); Tue, 21 Feb 2012 09:50:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=bAgCxWAU6mFpzn2icorH+LLd96Px6DGjdD1vAZBWWtE=; b=yHwhEX0WSEVozNdHohZtjzBa2WaMbJj1SPab8AwuomqUGPDLoZIEtzcSvjhXLRnIke n7wgnIfVp1rfotXZlI11Dz/8pDdlk9sX3O7HV8mPBIsxiPMH5rt2HPr1VuC6QAg1imim 0Y8AgV44RCNnSPgNvWXNyZNtw2RfWCv1rAqH8= MIME-Version: 1.0 Received: by 10.52.76.196 with SMTP id m4mr10106767vdw.112.1329846619584; Tue, 21 Feb 2012 09:50:19 -0800 (PST) Received: by 10.52.21.84 with HTTP; Tue, 21 Feb 2012 09:50:19 -0800 (PST) In-Reply-To: References: Date: Tue, 21 Feb 2012 12:50:19 -0500 Message-ID: From: Matthew Story To: freebsd-arch@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: Change to xargs to avoid orphaning of utility processes on signal|exit 255 from child X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Feb 2012 17:50:21 -0000 On Thu, Feb 16, 2012 at 7:09 PM, Matthew Story wrote: > Apologies if this is the wrong list, I would like to submit a patch that > changes the behavior of xargs(1) on signal to child utility process or > child utility process exiting 255. The patch(es) is|are available here: > > > http://axe0.blackskyresearch.net/patches/matt/xargs.no_orphan.patch.txt-- this version will apply to current xargs, and adds diagnostic > information for exit 255|signal to utility, as required by POSIX (see > PR165155). > > http://axe0.blackskyresearch.net/patches/matt/xargs.no_orphan.PR165155.patch.txt-- this version will apply on top of the patch in PR165155, as the errx > calls in that patch need to be modified to warnx calls. > I have updated these 2 patches above to branch correctly (the change from errx to warnx requires else'ing ... may have to purge browser catch to pick up the change), wondering if this patch should be expanded to include all of the cases mentioned in the ``Consequences of Errors'' section of the POSIX specification: > [... snip] > If a command line meeting the specified requirements cannot be assembled, > the utility cannot be invoked, an invocation of the utility is terminated > by a signal, or an invocation of the utility exits with exit status 255, > the *xargs* utility shall write a diagnostic message and exit without > processing any remaining input. > This would cause xargs to wait on children if the command line cannot be assembled, or utility cannot be invoked, in addition to the 2 cases covered by the patch. This should leave termination via signal to the xargs process itself as the only outstanding case where xargs orphans utilities, which is congruent with sh(1) behavior, and still allows for signaling all processes via signal to the process group, if you actually desire to signal all utility processes, along with xargs itself. Thoughts? > > The patch preserves orphaning when the xargs process itself is terminated > by signal, but augments the behavior when a child utility process is > terminated by signal or exits 255 to wait for other existing child > utilities until exiting 1. My reasoning for this (beyond orphaning > nastiness) is that I always want to fail as soon as I know an operation is > fatal, and then clean-up. By orphaning children, there is no reliable way > to clean-up following use of the 255 exit code (or signal termination): > > $ # a contrived example forcing a race-condition, with a clean-up function. > $ mkdir -p foo; jot - 1 10 | xargs -P5 -n1 sh -c 'sleep $1; touch foo/$1; > exit 255;' worker || find foo -type f -delete > $ # demonstration that cleanup is not possible > $ sleep 5 && ls -l foo > total 2 > -rw-r--r-- 1 matt matt 0 Feb 16 19:01 2 > -rw-r--r-- 1 matt matt 0 Feb 16 19:01 3 > -rw-r--r-- 1 matt matt 0 Feb 16 19:01 4 > -rw-r--r-- 1 matt matt 0 Feb 16 19:01 5 > > Following the patch, we get a nice and reliable cleanup, as we have no > orphans: > $ mkdir -p foo; jot - 1 10 | usr.bin/xargs/xargs -P5 -n1 sh -c 'sleep $1; > touch foo/$1; exit 255;' worker || find foo -type f -delete > xargs: sh: exited with status 255, aborting > xargs: sh: exited with status 255, aborting > xargs: sh: exited with status 255, aborting > xargs: sh: exited with status 255, aborting > xargs: sh: exited with status 255, aborting > $ ls -l foo/ > total 0 > > Please let me know what you think, I would very much like to see this > patch make it's way into xargs as I find this short-circuit behavior nearly > very usable, and it's only failing is that it orphans children > unnecessarily, resulting in unpredictable behavior. > > -- > regards, > matt > -- regards, matt From owner-freebsd-arch@FreeBSD.ORG Thu Feb 23 17:10:43 2012 Return-Path: Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8B39C106566C; Thu, 23 Feb 2012 17:10:43 +0000 (UTC) (envelope-from das@FreeBSD.ORG) Received: from zim.MIT.EDU (ZIM.MIT.EDU [18.95.3.101]) by mx1.freebsd.org (Postfix) with ESMTP id 47DCA8FC18; Thu, 23 Feb 2012 17:10:36 +0000 (UTC) Received: from zim.MIT.EDU (localhost [127.0.0.1]) by zim.MIT.EDU (8.14.5/8.14.2) with ESMTP id q1NH9JOb079069; Thu, 23 Feb 2012 12:09:19 -0500 (EST) (envelope-from das@FreeBSD.ORG) Received: (from das@localhost) by zim.MIT.EDU (8.14.5/8.14.2/Submit) id q1NH9IIE079068; Thu, 23 Feb 2012 12:09:18 -0500 (EST) (envelope-from das@FreeBSD.ORG) Date: Thu, 23 Feb 2012 12:09:18 -0500 From: David Schultz To: Gabor Kovesdan Message-ID: <20120223170918.GA79013@zim.MIT.EDU> Mail-Followup-To: Gabor Kovesdan , Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?= , Andriy Gapon , freebsd-arch@freebsd.org References: <4F3C28DD.1020003@FreeBSD.org> <4F3C2D2D.5000402@FreeBSD.org> <4F3E78BA.4060203@FreeBSD.org> <864nupcuvl.fsf@ds4.des.no> <4F3E7B5A.20103@FreeBSD.org> <86zkchbff6.fsf@ds4.des.no> <4F3EADB5.7060008@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <4F3EADB5.7060008@FreeBSD.org> Cc: Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?= , Andriy Gapon , freebsd-arch@FreeBSD.ORG Subject: Re: bsd/citrus iconv X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Feb 2012 17:10:43 -0000 On Fri, Feb 17, 2012, Gabor Kovesdan wrote: > Em 17-02-2012 17:19, Dag-Erling Smørgrav escreveu: > >Andriy Gapon writes: > >>But nothing precludes it from being a separate library in principle? > >>There is even src/lib/libiconv, it's just not connected to the build. > >You're right - for some reason, I thought src/lib/libiconv was just a > >Makefile for GNU libiconv. > POSIX requires iconv() to be in libc. However, a glue Makefile was > provided there for testing purposes. Where do you see that? I believe it's optional. Providing it as a separate library is more maintainable, and makes it easier for people to drop in a different iconv library if they need it. From owner-freebsd-arch@FreeBSD.ORG Thu Feb 23 17:12:14 2012 Return-Path: Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 04D30106564A for ; Thu, 23 Feb 2012 17:12:14 +0000 (UTC) (envelope-from das@FreeBSD.ORG) Received: from zim.MIT.EDU (ZIM.MIT.EDU [18.95.3.101]) by mx1.freebsd.org (Postfix) with ESMTP id A371F8FC17 for ; Thu, 23 Feb 2012 17:12:13 +0000 (UTC) Received: from zim.MIT.EDU (localhost [127.0.0.1]) by zim.MIT.EDU (8.14.5/8.14.2) with ESMTP id q1NHCBcq079103; Thu, 23 Feb 2012 12:12:11 -0500 (EST) (envelope-from das@FreeBSD.ORG) Received: (from das@localhost) by zim.MIT.EDU (8.14.5/8.14.2/Submit) id q1NHCAmK079102; Thu, 23 Feb 2012 12:12:11 -0500 (EST) (envelope-from das@FreeBSD.ORG) Date: Thu, 23 Feb 2012 12:12:10 -0500 From: David Schultz To: Tijl Coosemans Message-ID: <20120223171210.GB79013@zim.MIT.EDU> Mail-Followup-To: Tijl Coosemans , freebsd-arch@freebsd.org, Damjan Jovanovic References: <201202061916.45998.tijl@coosemans.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201202061916.45998.tijl@coosemans.org> Cc: Damjan Jovanovic , freebsd-arch@FreeBSD.ORG Subject: Re: amd64 cc -m32 support and headers project branch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Feb 2012 17:12:14 -0000 On Mon, Feb 06, 2012, Tijl Coosemans wrote: > After a hiatus of almost a year I'd like to continue with merging i386 > and amd64 headers to get "cc -m32" working on amd64. Previously I tried > to fix any problems with the headers first before merging, but this turned > out to be a long tedious process. So, because lack of -m32 support is > blocking other development I'd like to just merge a minimal set of > headers without any macro/type definition changes or other > reorganisations. The result will be similar to existing powerpc and mips > headers which already support both 32 and 64 bit. > > When that's done I can create a development branch to work on the > problems that have come up. Possible goals for such a branch are: > > - Fix inconsistencies such as types defined in sys/ and their limits > in machine/. Also, sometimes the limits don't have the correct type. > - For types/limits defined under machine/ there is a lot of duplication > between architectures. Try to move some to sys/. > - Try to make headers compiler agnostic; limit use of __attribute__, > __builtin_* to a single file. > - Maybe remove support for gcc 2.*. The oldest version in ports is 3.4 > and I suspect some headers don't compile anymore with gcc 2.*. > - Add support for new compiler attributes/built-ins. > - Add missing POSIX prototypes, maybe commented out with /* UNIMPL ... */ > or similar so they can be grepped. > - The gcc ports currently patch some system headers where they think > something is non-standard. These patched headers override the system > headers which means you have to rebuild these ports whenever you do > installworld to make sure they contain the latest changes. Some headers > are trivial to fix, others less so. > > > If there are no objections, I'd like to start with the -m32 work in > a week or so. This sounds like it could degenerate into an error-prone ifdef hell. Wouldn't it be much easier to just have a /usr/include32 directory? From owner-freebsd-arch@FreeBSD.ORG Thu Feb 23 18:06:42 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 276BA106564A; Thu, 23 Feb 2012 18:06:42 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id D8DDB8FC0C; Thu, 23 Feb 2012 18:06:41 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id 8374265DE; Thu, 23 Feb 2012 18:06:40 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id 437528B5C; Thu, 23 Feb 2012 19:06:40 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Gabor Kovesdan References: <4F3C28DD.1020003@FreeBSD.org> <4F3C2D2D.5000402@FreeBSD.org> <4F3E78BA.4060203@FreeBSD.org> <864nupcuvl.fsf@ds4.des.no> <4F3E7B5A.20103@FreeBSD.org> <86zkchbff6.fsf@ds4.des.no> <4F3EADB5.7060008@FreeBSD.org> <20120223170918.GA79013@zim.MIT.EDU> Date: Thu, 23 Feb 2012 19:06:39 +0100 In-Reply-To: <20120223170918.GA79013@zim.MIT.EDU> (David Schultz's message of "Thu, 23 Feb 2012 12:09:18 -0500") Message-ID: <86r4xl8lw0.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Andriy Gapon , freebsd-arch@freebsd.org Subject: Re: bsd/citrus iconv X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Feb 2012 18:06:42 -0000 David Schultz writes: > Gabor Kovesdan writes: > > POSIX requires iconv() to be in libc. However, a glue Makefile was > > provided there for testing purposes. > Where do you see that? I believe it's optional. Providing it as > a separate library is more maintainable, and makes it easier for > people to drop in a different iconv library if they need it. It's a question of semantics... it's an extension (part of the _XOPEN_UNIX option group), but if supported, it's supposed to be part of libc. I don't think the SUS states this explicitly, but it's implied by the lack of mention of a separate iconv library in the c99 man page. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Thu Feb 23 18:23:45 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9046A106566C for ; Thu, 23 Feb 2012 18:23:45 +0000 (UTC) (envelope-from wollman@hergotha.csail.mit.edu) Received: from hergotha.csail.mit.edu (wollman-1-pt.tunnel.tserv4.nyc4.ipv6.he.net [IPv6:2001:470:1f06:ccb::2]) by mx1.freebsd.org (Postfix) with ESMTP id 3EF258FC1A for ; Thu, 23 Feb 2012 18:23:45 +0000 (UTC) Received: from hergotha.csail.mit.edu (localhost [127.0.0.1]) by hergotha.csail.mit.edu (8.14.4/8.14.4) with ESMTP id q1NIMQcm020805; Thu, 23 Feb 2012 13:22:26 -0500 (EST) (envelope-from wollman@hergotha.csail.mit.edu) Received: (from wollman@localhost) by hergotha.csail.mit.edu (8.14.4/8.14.4/Submit) id q1NIMQOd020804; Thu, 23 Feb 2012 13:22:26 -0500 (EST) (envelope-from wollman) Date: Thu, 23 Feb 2012 13:22:26 -0500 (EST) From: Garrett Wollman Message-Id: <201202231822.q1NIMQOd020804@hergotha.csail.mit.edu> To: des@des.no X-Newsgroups: mit.lcs.mail.freebsd-arch In-Reply-To: <86r4xl8lw0.fsf@ds4.des.no> References: <4F3C28DD.1020003@FreeBSD.org> <4F3C2D2D.5000402@FreeBSD.org> <4F3E78BA.4060203@FreeBSD.org> <864nupcuvl.fsf@ds4.des.no> <4F3E7B5A.20103@FreeBSD.org> <86zkchbff6.fsf@ds4.des.no> <4F3EADB5.7060008@FreeBSD.org> <20120223170918.GA79013@zim.MIT.EDU> Organization: none X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (hergotha.csail.mit.edu [127.0.0.1]); Thu, 23 Feb 2012 13:22:26 -0500 (EST) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=disabled version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on hergotha.csail.mit.edu Cc: arch@freebsd.org Subject: Re: bsd/citrus iconv X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Feb 2012 18:23:45 -0000 In article <86r4xl8lw0.fsf@ds4.des.no>, des@des.no writes: >It's a question of semantics... it's an extension (part of the >_XOPEN_UNIX option group), but if supported, it's supposed to be part of >libc. I don't think the SUS states this explicitly, but it's implied by >the lack of mention of a separate iconv library in the c99 man page. POSIX doesn't specify libc or any other library. It specifies (by omission) that the function shall be available when the "c99" command is used without any special "-l" arguments (such as "-l rt"). It can be located in any library, so long as "c99" always includes that library when building an executable. (Of course, we don't support _XOPEN_UNIX and probably never will, but that would be the requirement if we did.) -GAWollman From owner-freebsd-arch@FreeBSD.ORG Thu Feb 23 18:46:55 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4DC71106566B for ; Thu, 23 Feb 2012 18:46:55 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 0675F8FC13 for ; Thu, 23 Feb 2012 18:46:54 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id 3A19B6607; Thu, 23 Feb 2012 18:46:54 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id 281F78B62; Thu, 23 Feb 2012 19:46:54 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Garrett Wollman References: <4F3C28DD.1020003@FreeBSD.org> <4F3C2D2D.5000402@FreeBSD.org> <4F3E78BA.4060203@FreeBSD.org> <864nupcuvl.fsf@ds4.des.no> <4F3E7B5A.20103@FreeBSD.org> <86zkchbff6.fsf@ds4.des.no> <4F3EADB5.7060008@FreeBSD.org> <20120223170918.GA79013@zim.MIT.EDU> <201202231822.q1NIMQOd020804@hergotha.csail.mit.edu> Date: Thu, 23 Feb 2012 19:46:54 +0100 In-Reply-To: <201202231822.q1NIMQOd020804@hergotha.csail.mit.edu> (Garrett Wollman's message of "Thu, 23 Feb 2012 13:22:26 -0500 (EST)") Message-ID: <86lint8k0x.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: arch@freebsd.org Subject: Re: bsd/citrus iconv X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Feb 2012 18:46:55 -0000 Garrett Wollman writes: > POSIX doesn't specify libc or any other library. It specifies (by > omission) that the function shall be available when the "c99" command > is used without any special "-l" arguments (such as "-l rt"). It can > be located in any library, so long as "c99" always includes that > library when building an executable. I beg to differ. The c99 man page explicitly lists a number of libraries (libc, libi, libpthread, libm, librt, libtrace, libxnet and liby), and states that libc will contain all functions *except* those defined in a short list of headers. This list does not include ; therefore, iconv must be in libc, cf. the original meaning of "the exception that proves the rule". DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Thu Feb 23 19:27:44 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 241171065670 for ; Thu, 23 Feb 2012 19:27:44 +0000 (UTC) (envelope-from wollman@hergotha.csail.mit.edu) Received: from hergotha.csail.mit.edu (wollman-1-pt.tunnel.tserv4.nyc4.ipv6.he.net [IPv6:2001:470:1f06:ccb::2]) by mx1.freebsd.org (Postfix) with ESMTP id AACA58FC0C for ; Thu, 23 Feb 2012 19:27:43 +0000 (UTC) Received: from hergotha.csail.mit.edu (localhost [127.0.0.1]) by hergotha.csail.mit.edu (8.14.4/8.14.4) with ESMTP id q1NJQPEr021655; Thu, 23 Feb 2012 14:26:25 -0500 (EST) (envelope-from wollman@hergotha.csail.mit.edu) Received: (from wollman@localhost) by hergotha.csail.mit.edu (8.14.4/8.14.4/Submit) id q1NJQPFa021654; Thu, 23 Feb 2012 14:26:25 -0500 (EST) (envelope-from wollman) Date: Thu, 23 Feb 2012 14:26:25 -0500 (EST) From: Garrett Wollman Message-Id: <201202231926.q1NJQPFa021654@hergotha.csail.mit.edu> To: des@des.no X-Newsgroups: mit.lcs.mail.freebsd-arch In-Reply-To: <86lint8k0x.fsf@ds4.des.no> References: <4F3C28DD.1020003@FreeBSD.org> <4F3C2D2D.5000402@FreeBSD.org> <4F3E78BA.4060203@FreeBSD.org> <864nupcuvl.fsf@ds4.des.no> <4F3E7B5A.20103@FreeBSD.org> <86zkchbff6.fsf@ds4.des.no> <4F3EADB5.7060008@FreeBSD.org> <20120223170918.GA79013@zim.MIT.EDU> <201202231822.q1NIMQOd020804@hergotha.csail.mit.edu> Organization: none X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (hergotha.csail.mit.edu [127.0.0.1]); Thu, 23 Feb 2012 14:26:25 -0500 (EST) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=disabled version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on hergotha.csail.mit.edu Cc: arch@freebsd.org Subject: Re: bsd/citrus iconv X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Feb 2012 19:27:44 -0000 In article <86lint8k0x.fsf@ds4.des.no>, des@des.no writes: >I beg to differ. The c99 man page explicitly lists a number of >libraries (libc, libi, libpthread, libm, librt, libtrace, libxnet and >liby), and states that libc will contain all functions *except* those >defined in a short list of headers. This list does not include >; therefore, iconv must be in libc, cf. the original meaning of >"the exception that proves the rule". You missed the bit on the next page: It is unspecified whether the libraries libc.a, libm.a, librt.a, libpthread.a, libl.a, liby.a, or libxnet exist as regular files. The implementation may accept as -l operands names of objects that do not exist as regular files. (This is from the 2001 approved standard; the text that you're looking at may differ slightly.) The "c99" implementation may implement the "-l c" option, whether implied or explicit, any way it wants, provided that the symbols described are defined during linking. In particular, it may realize those symbols through any set of actual underlying archive or shared libraries, regardless of the mechanics of how they are bound to the executable. This is also independent of what arguments are required for any other compiler invocation, as POSIX only specifies "c99". (The next POSIX revision will specify only "c11".) -GAWollman From owner-freebsd-arch@FreeBSD.ORG Thu Feb 23 21:24:23 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 059EA106566B for ; Thu, 23 Feb 2012 21:24:23 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id B95B58FC14 for ; Thu, 23 Feb 2012 21:24:22 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id ACF1E66A0; Thu, 23 Feb 2012 21:24:21 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id 8118D8B7A; Thu, 23 Feb 2012 22:24:21 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Garrett Wollman References: <4F3C28DD.1020003@FreeBSD.org> <4F3C2D2D.5000402@FreeBSD.org> <4F3E78BA.4060203@FreeBSD.org> <864nupcuvl.fsf@ds4.des.no> <4F3E7B5A.20103@FreeBSD.org> <86zkchbff6.fsf@ds4.des.no> <4F3EADB5.7060008@FreeBSD.org> <20120223170918.GA79013@zim.MIT.EDU> <201202231822.q1NIMQOd020804@hergotha.csail.mit.edu> <201202231926.q1NJQPFa021654@hergotha.csail.mit.edu> Date: Thu, 23 Feb 2012 22:24:21 +0100 In-Reply-To: <201202231926.q1NJQPFa021654@hergotha.csail.mit.edu> (Garrett Wollman's message of "Thu, 23 Feb 2012 14:26:25 -0500 (EST)") Message-ID: <86d3958cqi.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: arch@freebsd.org Subject: Re: bsd/citrus iconv X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Feb 2012 21:24:23 -0000 Garrett Wollman writes: > You missed the bit on the next page: > > It is unspecified whether the libraries libc.a, libm.a, > librt.a, libpthread.a, libl.a, liby.a, or libxnet exist as > regular files. The implementation may accept as -l operands > names of objects that do not exist as regular files. That's entirely academic unless you want to modify gcc and clang to automatically pull in libiconv. The point is that if the iconv extension is implemented, it must be available without requiring additional -l options. It all boils down to this: do we aspire to SUS conformance? However, considering that we currently don't have iconv in base at all, providing it in a separate library is still an improvement on the status quo. So long as it's a temporary measure, and we move it into libc when we deem it production-ready... > (This is from the 2001 approved standard; the text that you're looking > at may differ slightly.) The latest version is at http://pubs.opengroup.org/onlinepubs/009695399/ DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Thu Feb 23 21:59:41 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 67B981065673 for ; Thu, 23 Feb 2012 21:59:41 +0000 (UTC) (envelope-from wollman@hergotha.csail.mit.edu) Received: from hergotha.csail.mit.edu (wollman-1-pt.tunnel.tserv4.nyc4.ipv6.he.net [IPv6:2001:470:1f06:ccb::2]) by mx1.freebsd.org (Postfix) with ESMTP id 165148FC12 for ; Thu, 23 Feb 2012 21:59:40 +0000 (UTC) Received: from hergotha.csail.mit.edu (localhost [127.0.0.1]) by hergotha.csail.mit.edu (8.14.4/8.14.4) with ESMTP id q1NLwMG8023508; Thu, 23 Feb 2012 16:58:22 -0500 (EST) (envelope-from wollman@hergotha.csail.mit.edu) Received: (from wollman@localhost) by hergotha.csail.mit.edu (8.14.4/8.14.4/Submit) id q1NLwMSc023505; Thu, 23 Feb 2012 16:58:22 -0500 (EST) (envelope-from wollman) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Message-ID: <20294.46718.356434.248475@hergotha.csail.mit.edu> Date: Thu, 23 Feb 2012 16:58:22 -0500 From: Garrett Wollman To: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= In-Reply-To: <86d3958cqi.fsf@ds4.des.no> References: <4F3C28DD.1020003@FreeBSD.org> <4F3C2D2D.5000402@FreeBSD.org> <4F3E78BA.4060203@FreeBSD.org> <864nupcuvl.fsf@ds4.des.no> <4F3E7B5A.20103@FreeBSD.org> <86zkchbff6.fsf@ds4.des.no> <4F3EADB5.7060008@FreeBSD.org> <20120223170918.GA79013@zim.MIT.EDU> <201202231822.q1NIMQOd020804@hergotha.csail.mit.edu> <201202231926.q1NJQPFa021654@hergotha.csail.mit.edu> <86d3958cqi.fsf@ds4.des.no> X-Mailer: VM 7.17 under 21.4 (patch 22) "Instant Classic" XEmacs Lucid X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (hergotha.csail.mit.edu [127.0.0.1]); Thu, 23 Feb 2012 16:58:22 -0500 (EST) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=disabled version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on hergotha.csail.mit.edu Cc: arch@freebsd.org Subject: Re: bsd/citrus iconv X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Feb 2012 21:59:41 -0000 < said: > That's entirely academic unless you want to modify gcc and clang to > automatically pull in libiconv. It doesn't matter what gcc and clang do. POSIX ONLY specifies the behavior of the "c99" utility. It does not specify the behavior of any other compiler. -GAWollman From owner-freebsd-arch@FreeBSD.ORG Fri Feb 24 12:41:19 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D21E61065670 for ; Fri, 24 Feb 2012 12:41:19 +0000 (UTC) (envelope-from phk@phk.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 8D52B8FC12 for ; Fri, 24 Feb 2012 12:41:19 +0000 (UTC) Received: from critter.freebsd.dk (critter.freebsd.dk [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 0C5785DE7; Fri, 24 Feb 2012 12:41:18 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.5/8.14.5) with ESMTP id q1OCfG4v006740; Fri, 24 Feb 2012 12:41:16 GMT (envelope-from phk@phk.freebsd.dk) To: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= From: "Poul-Henning Kamp" In-Reply-To: Your message of "Thu, 23 Feb 2012 22:24:21 +0100." <86d3958cqi.fsf@ds4.des.no> Content-Type: text/plain; charset=ISO-8859-1 Date: Fri, 24 Feb 2012 12:41:16 +0000 Message-ID: <6739.1330087276@critter.freebsd.dk> Cc: arch@freebsd.org, Garrett Wollman Subject: Re: bsd/citrus iconv X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Feb 2012 12:41:19 -0000 In message <86d3958cqi.fsf@ds4.des.no>, =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= wr ites: >It all boils down to this: do we aspire to SUS conformance? Why would we aspire to be a mediocre 1990-vintage commercial UNIX ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Fri Feb 24 18:04:31 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BAF5E106564A for ; Fri, 24 Feb 2012 18:04:31 +0000 (UTC) (envelope-from matthewstory@gmail.com) Received: from mail-qw0-f47.google.com (mail-qw0-f47.google.com [209.85.216.47]) by mx1.freebsd.org (Postfix) with ESMTP id 65D1F8FC0C for ; Fri, 24 Feb 2012 18:04:31 +0000 (UTC) Received: by qadz30 with SMTP id z30so116124qad.13 for ; Fri, 24 Feb 2012 10:04:30 -0800 (PST) Received-SPF: pass (google.com: domain of matthewstory@gmail.com designates 10.229.102.79 as permitted sender) client-ip=10.229.102.79; Authentication-Results: mr.google.com; spf=pass (google.com: domain of matthewstory@gmail.com designates 10.229.102.79 as permitted sender) smtp.mail=matthewstory@gmail.com; dkim=pass header.i=matthewstory@gmail.com Received: from mr.google.com ([10.229.102.79]) by 10.229.102.79 with SMTP id f15mr2620217qco.94.1330106670824 (num_hops = 1); Fri, 24 Feb 2012 10:04:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=BJtX81+ZyoT7o/iWRZQ/TBEEHJ6RztkKM1dEvcH0X2w=; b=REXmK80D5U9KgHuExZhqmaMYgpavB5ojg/Qf7k3JvAxByZkf3vUgRoVuNf5RL1T46P WBUlu5Fpwtj/hGzy6Bo52D/vw0niwlf/hRAACno9TSaQKimNVhrAzQsWLgxgYjfmOADl M3T5YKHBEGdd5m6JYWHL92SPdIZCLU6jCZbzw= MIME-Version: 1.0 Received: by 10.229.102.79 with SMTP id f15mr2164583qco.94.1330106670664; Fri, 24 Feb 2012 10:04:30 -0800 (PST) Received: by 10.229.95.74 with HTTP; Fri, 24 Feb 2012 10:04:22 -0800 (PST) In-Reply-To: References: Date: Fri, 24 Feb 2012 13:04:22 -0500 Message-ID: From: Matthew Story To: freebsd-arch@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: Change to xargs to avoid orphaning of utility processes on signal|exit 255 from child X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Feb 2012 18:04:31 -0000 On Tue, Feb 21, 2012 at 12:50 PM, Matthew Story wrote: > On Thu, Feb 16, 2012 at 7:09 PM, Matthew Story wrote: > >> Apologies if this is the wrong list, I would like to submit a patch that >> changes the behavior of xargs(1) on signal to child utility process or >> child utility process exiting 255. The patch(es) is|are available here: >> >> >> http://axe0.blackskyresearch.net/patches/matt/xargs.no_orphan.patch.txt-- this version will apply to current xargs, and adds diagnostic >> information for exit 255|signal to utility, as required by POSIX (see >> PR165155). >> >> http://axe0.blackskyresearch.net/patches/matt/xargs.no_orphan.PR165155.patch.txt-- this version will apply on top of the patch in PR165155, as the errx >> calls in that patch need to be modified to warnx calls. >> > My HTTP logs are indicating that these links are being hit with trailing --'s, so just cleaning these up ... time to switch back to a real mail client, i guess ... http://axe0.blackskyresearch.net/patches/matt/xargs.no_orphan.patch.txt http://axe0.blackskyresearch.net/patches/matt/xargs.no_orphan.PR165155.patch.txt Sorry for any confusion. > I have updated these 2 patches above to branch correctly (the change from > errx to warnx requires else'ing ... may have to purge browser catch to pick > up the change), wondering if this patch should be expanded to include all > of the cases mentioned in the ``Consequences of Errors'' section of the > POSIX specification: > > >> [... snip] >> If a command line meeting the specified requirements cannot be assembled, >> the utility cannot be invoked, an invocation of the utility is terminated >> by a signal, or an invocation of the utility exits with exit status 255, >> the *xargs* utility shall write a diagnostic message and exit without >> processing any remaining input. >> > > This would cause xargs to wait on children if the command line cannot be > assembled, or utility cannot be invoked, in addition to the 2 cases covered > by the patch. This should leave termination via signal to the xargs > process itself as the only outstanding case where xargs orphans utilities, > which is congruent with sh(1) behavior, and still allows for signaling all > processes via signal to the process group, if you actually desire to signal > all utility processes, along with xargs itself. Thoughts? > > >> >> The patch preserves orphaning when the xargs process itself is terminated >> by signal, but augments the behavior when a child utility process is >> terminated by signal or exits 255 to wait for other existing child >> utilities until exiting 1. My reasoning for this (beyond orphaning >> nastiness) is that I always want to fail as soon as I know an operation is >> fatal, and then clean-up. By orphaning children, there is no reliable way >> to clean-up following use of the 255 exit code (or signal termination): >> >> $ # a contrived example forcing a race-condition, with a clean-up >> function. >> $ mkdir -p foo; jot - 1 10 | xargs -P5 -n1 sh -c 'sleep $1; touch foo/$1; >> exit 255;' worker || find foo -type f -delete >> $ # demonstration that cleanup is not possible >> $ sleep 5 && ls -l foo >> total 2 >> -rw-r--r-- 1 matt matt 0 Feb 16 19:01 2 >> -rw-r--r-- 1 matt matt 0 Feb 16 19:01 3 >> -rw-r--r-- 1 matt matt 0 Feb 16 19:01 4 >> -rw-r--r-- 1 matt matt 0 Feb 16 19:01 5 >> >> Following the patch, we get a nice and reliable cleanup, as we have no >> orphans: >> $ mkdir -p foo; jot - 1 10 | usr.bin/xargs/xargs -P5 -n1 sh -c 'sleep $1; >> touch foo/$1; exit 255;' worker || find foo -type f -delete >> xargs: sh: exited with status 255, aborting >> xargs: sh: exited with status 255, aborting >> xargs: sh: exited with status 255, aborting >> xargs: sh: exited with status 255, aborting >> xargs: sh: exited with status 255, aborting >> $ ls -l foo/ >> total 0 >> >> Please let me know what you think, I would very much like to see this >> patch make it's way into xargs as I find this short-circuit behavior nearly >> very usable, and it's only failing is that it orphans children >> unnecessarily, resulting in unpredictable behavior. >> >> -- >> regards, >> matt >> > > > > -- > regards, > matt > -- regards, matt From owner-freebsd-arch@FreeBSD.ORG Fri Feb 24 20:47:20 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 847AF1065675 for ; Fri, 24 Feb 2012 20:47:20 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail11.syd.optusnet.com.au (mail11.syd.optusnet.com.au [211.29.132.192]) by mx1.freebsd.org (Postfix) with ESMTP id 193798FC12 for ; Fri, 24 Feb 2012 20:47:19 +0000 (UTC) Received: from c211-30-171-136.carlnfd1.nsw.optusnet.com.au (c211-30-171-136.carlnfd1.nsw.optusnet.com.au [211.30.171.136]) by mail11.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id q1OKk3wR024729 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 25 Feb 2012 07:46:05 +1100 Date: Sat, 25 Feb 2012 07:46:03 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Garrett Wollman In-Reply-To: <20294.46718.356434.248475@hergotha.csail.mit.edu> Message-ID: <20120225073707.H2864@besplex.bde.org> References: <4F3C28DD.1020003@FreeBSD.org> <4F3C2D2D.5000402@FreeBSD.org> <4F3E78BA.4060203@FreeBSD.org> <864nupcuvl.fsf@ds4.des.no> <4F3E7B5A.20103@FreeBSD.org> <86zkchbff6.fsf@ds4.des.no> <4F3EADB5.7060008@FreeBSD.org> <20120223170918.GA79013@zim.MIT.EDU> <201202231822.q1NIMQOd020804@hergotha.csail.mit.edu> <201202231926.q1NJQPFa021654@hergotha.csail.mit.edu> <86d3958cqi.fsf@ds4.des.no> <20294.46718.356434.248475@hergotha.csail.mit.edu> MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="0-1569451402-1330116363=:2864" Cc: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= , arch@freebsd.org Subject: Re: bsd/citrus iconv X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Feb 2012 20:47:20 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --0-1569451402-1330116363=:2864 Content-Type: TEXT/PLAIN; charset=X-UNKNOWN; format=flowed Content-Transfer-Encoding: QUOTED-PRINTABLE On Thu, 23 Feb 2012, Garrett Wollman wrote: > < said: > >> That's entirely academic unless you want to modify gcc and clang to >> automatically pull in libiconv. > > It doesn't matter what gcc and clang do. POSIX ONLY specifies the > behavior of the "c99" utility. It does not specify the behavior of > any other compiler. FreeBSD's c99 doesn't actually work, since it doesn't know about libm. It also doesn't seem to provide a way to pass flags through to gcc, so it just breaks on -v and won't tell me which libraries it knows about :-). Any Standard C compiler for a hosted implementation needs to know about libm, so gcc -std=3Dc99 [-pedantic -fno-freestanding] is similarly broken (or just not a C compiler despite -std=3Dc99). Similarly for c89 and -std=3Dc89. I've never used a C compiler :-). c89 is closer to being one since C was simpler. Bruce --0-1569451402-1330116363=:2864-- From owner-freebsd-arch@FreeBSD.ORG Sat Feb 25 10:59:32 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ED89C106566C; Sat, 25 Feb 2012 10:59:31 +0000 (UTC) (envelope-from to.my.trociny@gmail.com) Received: from mail-bk0-f54.google.com (mail-bk0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 213A08FC08; Sat, 25 Feb 2012 10:59:30 +0000 (UTC) Received: by bkcjg1 with SMTP id jg1so3547351bkc.13 for ; Sat, 25 Feb 2012 02:59:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=from:to:cc:subject:references:x-comment-to:sender:date:in-reply-to :message-id:user-agent:mime-version:content-type; bh=pRH6U6w2RB9A9ts8fB5YLUurIPD34RfEtGlwuYKI1ao=; b=vydcDTrolSnjM9m92zH82nrN2afamzaHEzme/mom+oP+YqeT+2bht33aONMrVdcl8B W6HcSPICL6cO+6FMH5kJI3kAFnYyx71NXOhPwJlS37Cx98Fb8axYs43eRSwzz2jEKcHH fScUpTzaCAa8Gb3tmMEoaMPW49LGWDt0qXWlM= Received: by 10.204.150.91 with SMTP id x27mr2333915bkv.65.1330167569933; Sat, 25 Feb 2012 02:59:29 -0800 (PST) Received: from localhost ([95.69.173.122]) by mx.google.com with ESMTPS id i2sm13487883bkd.10.2012.02.25.02.59.26 (version=TLSv1/SSLv3 cipher=OTHER); Sat, 25 Feb 2012 02:59:28 -0800 (PST) From: Mikolaj Golub To: Konstantin Belousov References: <86fwfnti5t.fsf@kopusha.home.net> <20120112215106.GC31224@deviant.kiev.zoral.com.ua> <86hazntwmu.fsf@kopusha.home.net> <20120123031238.GL31224@deviant.kiev.zoral.com.ua> <86zkcfu9ac.fsf@kopusha.home.net> <20120218215003.GM3283@deviant.kiev.zoral.com.ua> <86ehtrf8qf.fsf@kopusha.home.net> <20120219170151.GS3283@deviant.kiev.zoral.com.ua> X-Comment-To: Konstantin Belousov Sender: Mikolaj Golub Date: Sat, 25 Feb 2012 12:59:24 +0200 In-Reply-To: <20120219170151.GS3283@deviant.kiev.zoral.com.ua> (Konstantin Belousov's message of "Sun, 19 Feb 2012 19:01:51 +0200") Message-ID: <86d393dvqr.fsf@kopusha.home.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: "Robert N. M. Watson" , freebsd-arch@freebsd.org Subject: Re: unix domain sockets on nullfs(5) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Feb 2012 10:59:32 -0000 On Sun, 19 Feb 2012 19:01:51 +0200 Konstantin Belousov wrote: KB> On Sun, Feb 19, 2012 at 11:55:20AM +0200, Mikolaj Golub wrote: >> >> On Sat, 18 Feb 2012 23:50:03 +0200 Konstantin Belousov wrote: >> >> KB> On Sat, Feb 18, 2012 at 11:22:03PM +0200, Mikolaj Golub wrote: >> >> After collecting all suggestions and additional testing I have got this patch >> >> set: >> >> >> >> http://people.freebsd.org/~trociny/unp_prepare_reclaim.1.patch >> KB> Including unpcb.h into vfs_subr.c looks too extreme. Put the prototype >> KB> into vnode.h, possibly renaming the function to vfs_unp_reclaim. >> >> Done. >> >> >> http://people.freebsd.org/~trociny/unp_connect.LOCKSHARED.1.patch >> >> http://people.freebsd.org/~trociny/VOP_UNP.3.patch >> KB> I has a painting suggestion there, call the vops VOP_UNP_DETACH etc, >> KB> otherwise it takes too much reading to understand that it is not undetach. >> >> Unfortunately, vnode_if.awk does not support underscores in VOP names. It >> failed with "Invalid %% construction" due to the restriction: $2 !~ /^[a-z]+$/. >> So I reverted back to the old naming. KB> Ok, might be it makes sense to try to add underscore to the set of KB> allowed symbols. >> >> The updated patches: >> >> http://people.freebsd.org/~trociny/unp_prepare_reclaim.2.patch >> http://people.freebsd.org/~trociny/unp_connect.LOCKSHARED.1.patch >> http://people.freebsd.org/~trociny/VOP_UNP.4.patch KB> In the comment above vfs_unp_reclaim(), say "before socket-type vnode KB> reclamation". KB> I have no further comments. I have committed all preparation patches and going to commit the VOP_UNP_* bits. The latest version: http://people.freebsd.org/~trociny/VOP_UNP.6.patch (in this version I added a note to UPDATING). I am thinking now about 'MFC after'. Thanks to jhb's spare vops it looks like it is possible to merge it in stable/9 without breaking the KBI, but then I will consume 3 of 5 available spare vops. So is it worth doing? -- Mikolaj Golub From owner-freebsd-arch@FreeBSD.ORG Sat Feb 25 13:01:34 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 81AB0106564A; Sat, 25 Feb 2012 13:01:34 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-lpp01m010-f54.google.com (mail-lpp01m010-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id BF4928FC0C; Sat, 25 Feb 2012 13:01:33 +0000 (UTC) Received: by lagz14 with SMTP id z14so5424826lag.13 for ; Sat, 25 Feb 2012 05:01:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=SEoxNxGnZT1w8j87qoxtutrOoOj9OePdjTJ2I+vL6F4=; b=sSipndwfEl0UjGJ2cSgxSX3m8O1ZJ46dedB1IAwRScAFFjpmiq+0wXjxWmwz4hD9t2 CYbi1WsOEZ2xjMCVoGeQdpG9vdEzGrnwFk6xvnNI7Q4hBri5GCM9KM2t1PqG8ZHYar4T G5L2OqadxtwqvgMA24aa9yel+YrbKjIxEqULw= MIME-Version: 1.0 Received: by 10.112.10.41 with SMTP id f9mr2200298lbb.8.1330174892478; Sat, 25 Feb 2012 05:01:32 -0800 (PST) Sender: asmrookie@gmail.com Received: by 10.112.41.5 with HTTP; Sat, 25 Feb 2012 05:01:32 -0800 (PST) In-Reply-To: <20120203193719.GB3283@deviant.kiev.zoral.com.ua> References: <20120203193719.GB3283@deviant.kiev.zoral.com.ua> Date: Sat, 25 Feb 2012 13:01:32 +0000 X-Google-Sender-Auth: AIz5d3RDFZnnVRbqo_8cufNFnfs Message-ID: From: Attilio Rao To: Konstantin Belousov , Pawel Jakub Dawidek Content-Type: text/plain; charset=UTF-8 Cc: arch@freebsd.org Subject: Re: Prefaulting for i/o buffers X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Feb 2012 13:01:34 -0000 Il 03 febbraio 2012 19:37, Konstantin Belousov ha scritto: > FreeBSD I/O infrastructure has well known issue with deadlock caused > by vnode lock order reversal when buffers supplied to read(2) or > write(2) syscalls are backed by mmaped file. > > I previously published the patches to convert i/o path to use VMIO, > based on the Jeff Roberson proposal, see > http://wiki.freebsd.org/VM6. As a side effect, the VM6 fixed the > deadlock. Since that work is very intrusive and did not got any > follow-up, it get stalled. > > Below is very lightweight patch which only goal is to fix deadlock in > the least intrusive way. This is possible after FreeBSD got the > vm_fault_quick_hold_pages(9) and vm_fault_disable_pagefaults(9) KPIs. > http://people.freebsd.org/~kib/misc/vm1.3.patch Hi, I was reviewing: http://people.freebsd.org/~kib/misc/vm1.11.patch and I think it is great. It is simple enough and I don't have further comments on it. However, as a side note, I was thinking if we could get one day at the point to integrate rangelocks into vnodes lockmgr directly. It would be a huge patch, rewrtiting the locking of several members of vnodes likely, but I think it would be worth it in terms of cleaness of the interface and less overhead. Also, it would be interesting to consider merging rangelock implementation in ZFS' one, at some point. BTW, nice job! Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-arch@FreeBSD.ORG Sat Feb 25 15:14:55 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2C3941065675 for ; Sat, 25 Feb 2012 15:14:55 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 8824A8FC13 for ; Sat, 25 Feb 2012 15:14:54 +0000 (UTC) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) by mail.dawidek.net (Postfix) with ESMTPSA id CC9F89DD; Sat, 25 Feb 2012 16:14:51 +0100 (CET) Date: Sat, 25 Feb 2012 16:13:34 +0100 From: Pawel Jakub Dawidek To: Attilio Rao Message-ID: <20120225151334.GH1344@garage.freebsd.pl> References: <20120203193719.GB3283@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="dDnEQgWzhgf+8aPe" Content-Disposition: inline In-Reply-To: X-OS: FreeBSD 10.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Konstantin Belousov , arch@freebsd.org Subject: Re: Prefaulting for i/o buffers X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Feb 2012 15:14:55 -0000 --dDnEQgWzhgf+8aPe Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Feb 25, 2012 at 01:01:32PM +0000, Attilio Rao wrote: > Il 03 febbraio 2012 19:37, Konstantin Belousov ha s= critto: > > FreeBSD I/O infrastructure has well known issue with deadlock caused > > by vnode lock order reversal when buffers supplied to read(2) or > > write(2) syscalls are backed by mmaped file. > > > > I previously published the patches to convert i/o path to use VMIO, > > based on the Jeff Roberson proposal, see > > http://wiki.freebsd.org/VM6. As a side effect, the VM6 fixed the > > deadlock. Since that work is very intrusive and did not got any > > follow-up, it get stalled. > > > > Below is very lightweight patch which only goal is to fix deadlock in > > the least intrusive way. This is possible after FreeBSD got the > > vm_fault_quick_hold_pages(9) and vm_fault_disable_pagefaults(9) KPIs. > > http://people.freebsd.org/~kib/misc/vm1.3.patch >=20 > Hi, > I was reviewing: > http://people.freebsd.org/~kib/misc/vm1.11.patch >=20 > and I think it is great. It is simple enough and I don't have further > comments on it. >=20 > However, as a side note, I was thinking if we could get one day at the > point to integrate rangelocks into vnodes lockmgr directly. > It would be a huge patch, rewrtiting the locking of several members of > vnodes likely, but I think it would be worth it in terms of cleaness > of the interface and less overhead. Also, it would be interesting to > consider merging rangelock implementation in ZFS' one, at some point. I personal opinion about rangelocks and many other VFS features we currently have is that it is good idea in theory, but in practise it tends to overcomplicate VFS. I'm in opinion that we should move as much stuff as we can to individual file systems. We try to implement everything in VFS itself in hope that this will simplify file systems we have. It then turns out only one file system is really using this stuff (most of the time it is UFS) and this is PITA for all the other file systems as well as maintaining VFS. VFS became so complicated over the years that there are maybe few people that can understand it, and every single change to VFS is a huge risk of potentially breaking some unrelated parts. File systems most of the time know much better how they work and what should be done to make them optimal. For example ZFS had range locking =66rom day one, but we can't take advantage of this, because our VFS "knows better" how ZFS locking should be done. There plenty of examples: - range vnode locking, - shared vnode locking, - quota (which I believe is still part of UFS, but I remember ideas of moving it to VFS), - suspend/resume fs, - buffer cache, - vnodes reclamation. I'm sure there are other examples. In my opinion we should do whatever we can to simplify VFS. Having complex VFS makes it harder, _not_ easier to develop file systems for and port file systems to FreeBSD. Interaction with VFS was definiately the hardest part of my work to port ZFS to FreeBSD. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://tupytaj.pl --dDnEQgWzhgf+8aPe Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk9I+p4ACgkQForvXbEpPzQgHACfZaE1YXFSq/O7ry49e6kZ+dMr Fq0An28wA1tET0i4dA1GbvqYx0GhGPMS =oWR/ -----END PGP SIGNATURE----- --dDnEQgWzhgf+8aPe-- From owner-freebsd-arch@FreeBSD.ORG Sat Feb 25 16:34:07 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 164F4106564A; Sat, 25 Feb 2012 16:34:07 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id D61338FC1B; Sat, 25 Feb 2012 16:34:06 +0000 (UTC) Received: by daec6 with SMTP id c6so1586943dae.13 for ; Sat, 25 Feb 2012 08:34:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=DTSKe0TcQUv2zKy6CEa1uN3ClRosV2obtPxInZPMnto=; b=JSZeKfjb38BHMW8c/zBerI/aoyvyjtwGXJk2gPOTotXh7nTJbhthcLf5ZWVnYbAKjn R0Qe7NF2zgxD2sCIixKfqWoRmFdfegvuXBbsifePQ/WtN/B6jy00TuBlOkHp9N5248fX z2hYXWjeqejI2MxP55JEJ+mit8unWs+zNi7qo= MIME-Version: 1.0 Received: by 10.68.216.4 with SMTP id om4mr17410512pbc.19.1330186047552; Sat, 25 Feb 2012 08:07:27 -0800 (PST) Sender: mdf356@gmail.com Received: by 10.68.238.70 with HTTP; Sat, 25 Feb 2012 08:07:27 -0800 (PST) In-Reply-To: <20120225151334.GH1344@garage.freebsd.pl> References: <20120203193719.GB3283@deviant.kiev.zoral.com.ua> <20120225151334.GH1344@garage.freebsd.pl> Date: Sat, 25 Feb 2012 08:07:27 -0800 X-Google-Sender-Auth: uLCdUPVjp7kTn2rWXksb2b_FbP8 Message-ID: From: mdf@FreeBSD.org To: Pawel Jakub Dawidek Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Attilio Rao , Konstantin Belousov , arch@freebsd.org Subject: Re: Prefaulting for i/o buffers X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Feb 2012 16:34:07 -0000 On Sat, Feb 25, 2012 at 7:13 AM, Pawel Jakub Dawidek wrot= e: > On Sat, Feb 25, 2012 at 01:01:32PM +0000, Attilio Rao wrote: >> Il 03 febbraio 2012 19:37, Konstantin Belousov ha = scritto: >> > FreeBSD I/O infrastructure has well known issue with deadlock caused >> > by vnode lock order reversal when buffers supplied to read(2) or >> > write(2) syscalls are backed by mmaped file. >> > >> > I previously published the patches to convert i/o path to use VMIO, >> > based on the Jeff Roberson proposal, see >> > http://wiki.freebsd.org/VM6. As a side effect, the VM6 fixed the >> > deadlock. Since that work is very intrusive and did not got any >> > follow-up, it get stalled. >> > >> > Below is very lightweight patch which only goal is to fix deadlock in >> > the least intrusive way. This is possible after FreeBSD got the >> > vm_fault_quick_hold_pages(9) and vm_fault_disable_pagefaults(9) KPIs. >> > http://people.freebsd.org/~kib/misc/vm1.3.patch >> >> Hi, >> I was reviewing: >> http://people.freebsd.org/~kib/misc/vm1.11.patch >> >> and I think it is great. It is simple enough and I don't have further >> comments on it. >> >> However, as a side note, I was thinking if we could get one day at the >> point to integrate rangelocks into vnodes lockmgr directly. >> It would be a huge patch, rewrtiting the locking of several members of >> vnodes likely, but I think it would be worth it in terms of cleaness >> of the interface and less overhead. Also, it would be interesting to >> consider merging rangelock implementation in ZFS' one, at some point. > > I personal opinion about rangelocks and many other VFS features we > currently have is that it is good idea in theory, but in practise it > tends to overcomplicate VFS. > > I'm in opinion that we should move as much stuff as we can to individual > file systems. We try to implement everything in VFS itself in hope that > this will simplify file systems we have. It then turns out only one file > system is really using this stuff (most of the time it is UFS) and this > is PITA for all the other file systems as well as maintaining VFS. VFS > became so complicated over the years that there are maybe few people > that can understand it, and every single change to VFS is a huge risk of > potentially breaking some unrelated parts. > > File systems most of the time know much better how they work and what > should be done to make them optimal. For example ZFS had range locking > from day one, but we can't take advantage of this, because our VFS "knows > better" how ZFS locking should be done. There plenty of examples: > - range vnode locking, > - shared vnode locking, > - quota (which I believe is still part of UFS, but I remember ideas of > =A0moving it to VFS), > - suspend/resume fs, > - buffer cache, > - vnodes reclamation. > I'm sure there are other examples. > > In my opinion we should do whatever we can to simplify VFS. Having > complex VFS makes it harder, _not_ easier to develop file systems for > and port file systems to FreeBSD. Interaction with VFS was definiately > the hardest part of my work to port ZFS to FreeBSD. It's also a large hassle when maintaining a custom VFS for a commercial pro= duct. When we implemented multi-writer (either multiple nodes or multiple threads on a node writing to a file) we had to add a new lockmgr(9) lock type and implement range locks on the file data. But the details of my file system's range locking needs are not the same as yours. And maybe your filesystem will never allow multi-writer since it doesn't have the same performance concerns. But then we need a bunch of new VOPs to attempt a DELTA locked operation, and fail back to a SHARED or EXCLUSIVE when the filesystem doesn't support it. This becomes a large patch on vfs_* files to maintain, making it harder to stay in sync with other FreeBSD changes. So it's not just ZFS, that's just the most accessible example. Cheers, matthew From owner-freebsd-arch@FreeBSD.ORG Sat Feb 25 17:45:02 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EF5BB106566B; Sat, 25 Feb 2012 17:45:01 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 8F4A78FC12; Sat, 25 Feb 2012 17:45:01 +0000 (UTC) Received: by vcge1 with SMTP id e1so965933vcg.13 for ; Sat, 25 Feb 2012 09:45:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=nzKdFKlpGVQm/tk7uzQX2cCDaf1QX/Hi/Ko4Jxzv+Pg=; b=mTGJ8Cn416IgY3Z+1WnOQF+p1/rkXim5oAEKTqZCZFUEZ6ssA+YcAoihlFAFE9B7rT 06fVdvZrrvT87VjmGVG5lI/mSkST/Ry4RqEv6aks4RLj4iM/SBWHjFT5xPc82eODlEOH cjFk08xdcISX4SX5UjmV3Qpzi1FzPPhM1/ihs= MIME-Version: 1.0 Received: by 10.52.77.101 with SMTP id r5mr2974872vdw.109.1330191900926; Sat, 25 Feb 2012 09:45:00 -0800 (PST) Sender: asmrookie@gmail.com Received: by 10.220.189.2 with HTTP; Sat, 25 Feb 2012 09:45:00 -0800 (PST) In-Reply-To: <20120225151334.GH1344@garage.freebsd.pl> References: <20120203193719.GB3283@deviant.kiev.zoral.com.ua> <20120225151334.GH1344@garage.freebsd.pl> Date: Sat, 25 Feb 2012 18:45:00 +0100 X-Google-Sender-Auth: tjy6w1h0-KKAAhFteSUAQwEo-58 Message-ID: From: Attilio Rao To: Pawel Jakub Dawidek Content-Type: text/plain; charset=UTF-8 Cc: Konstantin Belousov , arch@freebsd.org Subject: Re: Prefaulting for i/o buffers X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Feb 2012 17:45:02 -0000 Il 25 febbraio 2012 16:13, Pawel Jakub Dawidek ha scritto: > On Sat, Feb 25, 2012 at 01:01:32PM +0000, Attilio Rao wrote: >> Il 03 febbraio 2012 19:37, Konstantin Belousov ha scritto: >> > FreeBSD I/O infrastructure has well known issue with deadlock caused >> > by vnode lock order reversal when buffers supplied to read(2) or >> > write(2) syscalls are backed by mmaped file. >> > >> > I previously published the patches to convert i/o path to use VMIO, >> > based on the Jeff Roberson proposal, see >> > http://wiki.freebsd.org/VM6. As a side effect, the VM6 fixed the >> > deadlock. Since that work is very intrusive and did not got any >> > follow-up, it get stalled. >> > >> > Below is very lightweight patch which only goal is to fix deadlock in >> > the least intrusive way. This is possible after FreeBSD got the >> > vm_fault_quick_hold_pages(9) and vm_fault_disable_pagefaults(9) KPIs. >> > http://people.freebsd.org/~kib/misc/vm1.3.patch >> >> Hi, >> I was reviewing: >> http://people.freebsd.org/~kib/misc/vm1.11.patch >> >> and I think it is great. It is simple enough and I don't have further >> comments on it. >> >> However, as a side note, I was thinking if we could get one day at the >> point to integrate rangelocks into vnodes lockmgr directly. >> It would be a huge patch, rewrtiting the locking of several members of >> vnodes likely, but I think it would be worth it in terms of cleaness >> of the interface and less overhead. Also, it would be interesting to >> consider merging rangelock implementation in ZFS' one, at some point. > > I personal opinion about rangelocks and many other VFS features we > currently have is that it is good idea in theory, but in practise it > tends to overcomplicate VFS. > > I'm in opinion that we should move as much stuff as we can to individual > file systems. We try to implement everything in VFS itself in hope that > this will simplify file systems we have. It then turns out only one file > system is really using this stuff (most of the time it is UFS) and this > is PITA for all the other file systems as well as maintaining VFS. VFS > became so complicated over the years that there are maybe few people > that can understand it, and every single change to VFS is a huge risk of > potentially breaking some unrelated parts. I think this is questionable due to the following assets: - If the problem is filesystems writers having trouble in understanding the necessary locking we should really provide cleaner and more complete documentation. One would think the same with our VM subsystem, but at least in that case there is plenty of comments that help understanding how to deal with vm_object, vm_pages locking during their lifelines. - Our primitives may be more complicated than the 'all-in-the-filesystem' one, but at least they offer a complete and centralized view over the resources we have allocated in the whole system and they allow building better policies about how to manage them. One problem I see here, is that those policies are not fully implemented, tuned or just got outdated, removing one of the highest beneficial that we have by making vnodes so generic About the thing I mentioned myself: - As long as the same path now has both range-locking and vnode locking I don't see as a good idea to keep both separated forever. Merging them seems to me an important evolution (not only helping shrinking the number of primitives themselves but also introducing less overhead and likely rewamped scalability for vnodes (but I think this needs a deep investigation). - About ZFS rangelocks absorbing the VFS ones, I think this is a minor point, but still, if you think it can be done efficiently and without loosing performance I don't see why not do that. You already wrote rangelocks for ZFS, so you are have earned a big experience in this area and can comment on fallouts, etc., but I don't see a good reason to not do that, unless it is just too difficult. This is not about generalizing a new mechanism, it is using a general mechanism in a specific implementation, if possible. Thanks, Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-arch@FreeBSD.ORG Sat Feb 25 17:48:24 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7950E106566C; Sat, 25 Feb 2012 17:48:24 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 085C38FC0A; Sat, 25 Feb 2012 17:48:23 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q1PHmIUP086383; Sat, 25 Feb 2012 19:48:18 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q1PHmIxq020059; Sat, 25 Feb 2012 19:48:18 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q1PHmIeu020058; Sat, 25 Feb 2012 19:48:18 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 25 Feb 2012 19:48:18 +0200 From: Konstantin Belousov To: Mikolaj Golub Message-ID: <20120225174818.GC55074@deviant.kiev.zoral.com.ua> References: <20120112215106.GC31224@deviant.kiev.zoral.com.ua> <86hazntwmu.fsf@kopusha.home.net> <20120123031238.GL31224@deviant.kiev.zoral.com.ua> <86zkcfu9ac.fsf@kopusha.home.net> <20120218215003.GM3283@deviant.kiev.zoral.com.ua> <86ehtrf8qf.fsf@kopusha.home.net> <20120219170151.GS3283@deviant.kiev.zoral.com.ua> <86d393dvqr.fsf@kopusha.home.net> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="KdsnslqZnlo4hNtV" Content-Disposition: inline In-Reply-To: <86d393dvqr.fsf@kopusha.home.net> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: "Robert N. M. Watson" , freebsd-arch@freebsd.org Subject: Re: unix domain sockets on nullfs(5) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Feb 2012 17:48:24 -0000 --KdsnslqZnlo4hNtV Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Feb 25, 2012 at 12:59:24PM +0200, Mikolaj Golub wrote: >=20 > On Sun, 19 Feb 2012 19:01:51 +0200 Konstantin Belousov wrote: >=20 > KB> On Sun, Feb 19, 2012 at 11:55:20AM +0200, Mikolaj Golub wrote: > >>=20 > >> On Sat, 18 Feb 2012 23:50:03 +0200 Konstantin Belousov wrote: > >>=20 > >> KB> On Sat, Feb 18, 2012 at 11:22:03PM +0200, Mikolaj Golub wrote: > >> >> After collecting all suggestions and additional testing I have go= t this patch > >> >> set: > >> >>=20 > >> >> http://people.freebsd.org/~trociny/unp_prepare_reclaim.1.patch > >> KB> Including unpcb.h into vfs_subr.c looks too extreme. Put the pro= totype > >> KB> into vnode.h, possibly renaming the function to vfs_unp_reclaim. > >>=20 > >> Done. > >>=20 > >> >> http://people.freebsd.org/~trociny/unp_connect.LOCKSHARED.1.patch > >> >> http://people.freebsd.org/~trociny/VOP_UNP.3.patch > >> KB> I has a painting suggestion there, call the vops VOP_UNP_DETACH = etc, > >> KB> otherwise it takes too much reading to understand that it is not= undetach. > >>=20 > >> Unfortunately, vnode_if.awk does not support underscores in VOP names= . It > >> failed with "Invalid %% construction" due to the restriction: $2 !~ /= ^[a-z]+$/. > >> So I reverted back to the old naming. > KB> Ok, might be it makes sense to try to add underscore to the set of > KB> allowed symbols. >=20 > >>=20 > >> The updated patches: > >>=20 > >> http://people.freebsd.org/~trociny/unp_prepare_reclaim.2.patch > >> http://people.freebsd.org/~trociny/unp_connect.LOCKSHARED.1.patch > >> http://people.freebsd.org/~trociny/VOP_UNP.4.patch >=20 > KB> In the comment above vfs_unp_reclaim(), say "before socket-type vnode > KB> reclamation". >=20 > KB> I have no further comments. >=20 > I have committed all preparation patches and going to commit the VOP_UNP_* > bits. The latest version: >=20 > http://people.freebsd.org/~trociny/VOP_UNP.6.patch >=20 > (in this version I added a note to UPDATING). Ok. >=20 > I am thinking now about 'MFC after'. Thanks to jhb's spare vops it looks = like > it is possible to merge it in stable/9 without breaking the KBI, but then= I > will consume 3 of 5 available spare vops. So is it worth doing? I do not think that we shall be so restrained on the use of spare VOPs. --KdsnslqZnlo4hNtV Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAk9JHuIACgkQC3+MBN1Mb4ix9QCfYo5dPvO7xhCXCCPwq89miWYX SwQAn2vKifyWasEIR4E8zmD5lb7F4voM =5vZu -----END PGP SIGNATURE----- --KdsnslqZnlo4hNtV-- From owner-freebsd-arch@FreeBSD.ORG Sat Feb 25 19:16:12 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 241071065670; Sat, 25 Feb 2012 19:16:12 +0000 (UTC) (envelope-from rwatson@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id E92458FC1B; Sat, 25 Feb 2012 19:16:11 +0000 (UTC) Received: from [192.168.2.111] (host86-163-246-207.range86-163.btcentralplus.com [86.163.246.207]) by cyrus.watson.org (Postfix) with ESMTPSA id DD54446B2A; Sat, 25 Feb 2012 14:16:10 -0500 (EST) Mime-Version: 1.0 (Apple Message framework v1257) Content-Type: text/plain; charset=us-ascii From: "Robert N. M. Watson" In-Reply-To: <20120225174818.GC55074@deviant.kiev.zoral.com.ua> Date: Sat, 25 Feb 2012 19:16:08 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: <8B0B7D86-1EE6-4901-9E36-EA1918315163@freebsd.org> References: <20120112215106.GC31224@deviant.kiev.zoral.com.ua> <86hazntwmu.fsf@kopusha.home.net> <20120123031238.GL31224@deviant.kiev.zoral.com.ua> <86zkcfu9ac.fsf@kopusha.home.net> <20120218215003.GM3283@deviant.kiev.zoral.com.ua> <86ehtrf8qf.fsf@kopusha.home.net> <20120219170151.GS3283@deviant.kiev.zoral.com.ua> <86d393dvqr.fsf@kopusha.home.net> <20120225174818.GC55074@deviant.kiev.zoral.com.ua> To: Konstantin Belousov X-Mailer: Apple Mail (2.1257) Cc: Mikolaj Golub , freebsd-arch@freebsd.org Subject: Re: unix domain sockets on nullfs(5) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Feb 2012 19:16:12 -0000 On 25 Feb 2012, at 17:48, Konstantin Belousov wrote: >> I am thinking now about 'MFC after'. Thanks to jhb's spare vops it = looks like >> it is possible to merge it in stable/9 without breaking the KBI, but = then I >> will consume 3 of 5 available spare vops. So is it worth doing? > I do not think that we shall be so restrained on the use of spare = VOPs. I think the only real question here is whether John had specific things = in mind when he added the spares, or whether they are open for general = use. Assuming they are open for general use, this use seems a quite = beneficial one :-). Robert= From owner-freebsd-arch@FreeBSD.ORG Sat Feb 25 19:47:51 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A2C53106564A; Sat, 25 Feb 2012 19:47:51 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id E76478FC08; Sat, 25 Feb 2012 19:47:50 +0000 (UTC) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) by mail.dawidek.net (Postfix) with ESMTPSA id A00D5A9E; Sat, 25 Feb 2012 20:47:48 +0100 (CET) Date: Sat, 25 Feb 2012 20:46:31 +0100 From: Pawel Jakub Dawidek To: Attilio Rao Message-ID: <20120225194630.GI1344@garage.freebsd.pl> References: <20120203193719.GB3283@deviant.kiev.zoral.com.ua> <20120225151334.GH1344@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="IbA9xpzOQlG26JSn" Content-Disposition: inline In-Reply-To: X-OS: FreeBSD 10.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Konstantin Belousov , arch@freebsd.org Subject: Re: Prefaulting for i/o buffers X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Feb 2012 19:47:51 -0000 --IbA9xpzOQlG26JSn Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Feb 25, 2012 at 06:45:00PM +0100, Attilio Rao wrote: > Il 25 febbraio 2012 16:13, Pawel Jakub Dawidek ha scrit= to: > > I personal opinion about rangelocks and many other VFS features we > > currently have is that it is good idea in theory, but in practise it > > tends to overcomplicate VFS. > > > > I'm in opinion that we should move as much stuff as we can to individual > > file systems. We try to implement everything in VFS itself in hope that > > this will simplify file systems we have. It then turns out only one file > > system is really using this stuff (most of the time it is UFS) and this > > is PITA for all the other file systems as well as maintaining VFS. VFS > > became so complicated over the years that there are maybe few people > > that can understand it, and every single change to VFS is a huge risk of > > potentially breaking some unrelated parts. >=20 > I think this is questionable due to the following assets: > - If the problem is filesystems writers having trouble in > understanding the necessary locking we should really provide cleaner > and more complete documentation. One would think the same with our VM > subsystem, but at least in that case there is plenty of comments that > help understanding how to deal with vm_object, vm_pages locking during > their lifelines. Documentation is not the answer here. If the code is so complex it is harder to learn, no matter how good the documentation is, it makes less people willing to learn it in the first place and it makes the code more buggy, because there are more edge/special cases you can forget about. > - Our primitives may be more complicated than the > 'all-in-the-filesystem' one, but at least they offer a complete and > centralized view over the resources we have allocated in the whole > system and they allow building better policies about how to manage > them. One problem I see here, is that those policies are not fully > implemented, tuned or just got outdated, removing one of the highest > beneficial that we have by making vnodes so generic Again, this is only nice theory, that is far from being the reality. You will never be able to have control on all the resources allocated by file systems. > About the thing I mentioned myself: > - As long as the same path now has both range-locking and vnode > locking I don't see as a good idea to keep both separated forever. > Merging them seems to me an important evolution (not only helping > shrinking the number of primitives themselves but also introducing > less overhead and likely rewamped scalability for vnodes (but I think > this needs a deep investigation). > - About ZFS rangelocks absorbing the VFS ones, I think this is a minor > point, but still, if you think it can be done efficiently and without > loosing performance I don't see why not do that. You already wrote > rangelocks for ZFS, so you are have earned a big experience in this > area and can comment on fallouts, etc., but I don't see a good reason > to not do that, unless it is just too difficult. This is not about > generalizing a new mechanism, it is using a general mechanism in a > specific implementation, if possible. I did not implement rangelocking for ZFS. It came with ZFS when I ported it. Until we want to merge changes from upstream (which is now IllumOS) we don't want to make huge changes just for the sake of proving that this is general purpose mechanism used by more than one file system. Attilio, don't get me wrong. In 99% cases it is good to make code more general and more universal and reusable, but we can't ignore reality. There are reasons why file systems like XFS, ReiserFS and others where never fully ported. I'm not saying VFS complexity was the only reason, but I'm sure it was one of them. Our VFS is very UFS-centric. We make so many assumptions that sounds fine only for UFS. I saw plenty of those while working on ZFS, like: - "Every file system needs cache. Let's make it general, so that all file systems can use it!" Well, for VFS each file system is a separate entity, which is not the case for ZFS. ZFS can cache one block only once that is used by one file system, 10 clones and 100 snapshots, which all are separate mount points from VFS perspective. The same block would be cached 111 times by the buffer cache. - "rmdir(2) on a mountpoint is bad idea, let's deny it at VFS level." It is bad idea, indeed, but in ZFS it is a nice way to remove snapshot by rmdiring .zfs/snapshot/ directory. - Noone implemented rangelocking in VFS, so no file system can use it. Even if the given file system has all the code to do it. etc. I'm also sure it will be way easier for Jeff to make VFS MP-safe if it was less complex. When looking at the big picture, it would be nice to have all this general stuff like rangelocking, quota, buffer cache, etc. as some kind of libraries for file systems to use and not something that is mandatory. If I develop a file system for FreeBSD only and I don't want to reinvent the wheel, I can use those libraries. If I port file system to FreeBSD or develop a file system that doesn't really need those libraries I'm not forced to use them. All this might make a good working group subject at BSDCan devsummit. We could cross swords there:) --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://tupytaj.pl --IbA9xpzOQlG26JSn Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk9JOpYACgkQForvXbEpPzR1vgCgva2ltveZ/GgGBjfFj6J741hq IWIAn3WtFCQ7GjKfS6OwyiZTTKXQbfTG =/aPq -----END PGP SIGNATURE----- --IbA9xpzOQlG26JSn-- From owner-freebsd-arch@FreeBSD.ORG Sat Feb 25 21:03:46 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4E2C41065670; Sat, 25 Feb 2012 21:03:46 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 8300E8FC0A; Sat, 25 Feb 2012 21:03:45 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q1PL3dj2099366; Sat, 25 Feb 2012 23:03:39 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q1PL3dTu021205; Sat, 25 Feb 2012 23:03:39 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q1PL3d5v021204; Sat, 25 Feb 2012 23:03:39 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 25 Feb 2012 23:03:39 +0200 From: Konstantin Belousov To: Attilio Rao Message-ID: <20120225210339.GM55074@deviant.kiev.zoral.com.ua> References: <20120203193719.GB3283@deviant.kiev.zoral.com.ua> <20120225151334.GH1344@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="96Fzkco6h7mx3UO9" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: arch@freebsd.org, Pawel Jakub Dawidek Subject: Re: Prefaulting for i/o buffers X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Feb 2012 21:03:46 -0000 --96Fzkco6h7mx3UO9 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Feb 25, 2012 at 06:45:00PM +0100, Attilio Rao wrote: > Il 25 febbraio 2012 16:13, Pawel Jakub Dawidek ha scrit= to: > > On Sat, Feb 25, 2012 at 01:01:32PM +0000, Attilio Rao wrote: > >> Il 03 febbraio 2012 19:37, Konstantin Belousov h= a scritto: > >> > FreeBSD I/O infrastructure has well known issue with deadlock caused > >> > by vnode lock order reversal when buffers supplied to read(2) or > >> > write(2) syscalls are backed by mmaped file. > >> > > >> > I previously published the patches to convert i/o path to use VMIO, > >> > based on the Jeff Roberson proposal, see > >> > http://wiki.freebsd.org/VM6. As a side effect, the VM6 fixed the > >> > deadlock. Since that work is very intrusive and did not got any > >> > follow-up, it get stalled. > >> > > >> > Below is very lightweight patch which only goal is to fix deadlock in > >> > the least intrusive way. This is possible after FreeBSD got the > >> > vm_fault_quick_hold_pages(9) and vm_fault_disable_pagefaults(9) KPIs. > >> > http://people.freebsd.org/~kib/misc/vm1.3.patch > >> > >> Hi, > >> I was reviewing: > >> http://people.freebsd.org/~kib/misc/vm1.11.patch > >> > >> and I think it is great. It is simple enough and I don't have further > >> comments on it. Thank you. This spoiled an announce I intended to send this weekend :) > >> > >> However, as a side note, I was thinking if we could get one day at the > >> point to integrate rangelocks into vnodes lockmgr directly. > >> It would be a huge patch, rewrtiting the locking of several members of > >> vnodes likely, but I think it would be worth it in terms of cleaness > >> of the interface and less overhead. Also, it would be interesting to > >> consider merging rangelock implementation in ZFS' one, at some point. > > > > I personal opinion about rangelocks and many other VFS features we > > currently have is that it is good idea in theory, but in practise it > > tends to overcomplicate VFS. > > > > I'm in opinion that we should move as much stuff as we can to individual > > file systems. We try to implement everything in VFS itself in hope that > > this will simplify file systems we have. It then turns out only one file > > system is really using this stuff (most of the time it is UFS) and this > > is PITA for all the other file systems as well as maintaining VFS. VFS > > became so complicated over the years that there are maybe few people > > that can understand it, and every single change to VFS is a huge risk of > > potentially breaking some unrelated parts. >=20 > I think this is questionable due to the following assets: > - If the problem is filesystems writers having trouble in > understanding the necessary locking we should really provide cleaner > and more complete documentation. One would think the same with our VM > subsystem, but at least in that case there is plenty of comments that > help understanding how to deal with vm_object, vm_pages locking during > their lifelines. > - Our primitives may be more complicated than the > 'all-in-the-filesystem' one, but at least they offer a complete and > centralized view over the resources we have allocated in the whole > system and they allow building better policies about how to manage > them. One problem I see here, is that those policies are not fully > implemented, tuned or just got outdated, removing one of the highest > beneficial that we have by making vnodes so generic >=20 > About the thing I mentioned myself: > - As long as the same path now has both range-locking and vnode > locking I don't see as a good idea to keep both separated forever. > Merging them seems to me an important evolution (not only helping > shrinking the number of primitives themselves but also introducing > less overhead and likely rewamped scalability for vnodes (but I think > this needs a deep investigation). The proper direction to move there is to designate the vnode lock for the vnode structure protection, and have the range lock protect the i/o atomicity. This is somewhat done in the proposed patch (since now vnode lock does not protect the i/o operation, but only chunked i/o transactions inside the operation). The Jeff idea of using page cache as the source of i/o data (implemented in the VM6 patchset) pushes the idea much further. E.g., the write does not obtain the write vnode lock typically (but sometimes it had, to extend the vnode). Probably, I will revive VM6 after this change is landed. > - About ZFS rangelocks absorbing the VFS ones, I think this is a minor > point, but still, if you think it can be done efficiently and without > loosing performance I don't see why not do that. You already wrote > rangelocks for ZFS, so you are have earned a big experience in this > area and can comment on fallouts, etc., but I don't see a good reason > to not do that, unless it is just too difficult. This is not about > generalizing a new mechanism, it is using a general mechanism in a > specific implementation, if possible. ZFS rangelocks, as I understand from the cursory look, have completely different purpose. Or rather, they protect a different object then the rangelocks added in the proposed patch. Completely different question is the merge of _implementation_ of rangelocks, e.g. throwing out the naive code in kern_rangelock.c is fine with me. But we do not take CDDL-licensed implementation in kern/. --96Fzkco6h7mx3UO9 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAk9JTKsACgkQC3+MBN1Mb4h3AACdG6Y+lo4HO/up1IuOAv/kEHpD lisAoK6DVcaOuQT1HAhnEh44fzTLkikv =1+zm -----END PGP SIGNATURE----- --96Fzkco6h7mx3UO9--