From owner-freebsd-hackers@FreeBSD.ORG Sun Dec 9 01:05:35 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 47E3BB87 for ; Sun, 9 Dec 2012 01:05:35 +0000 (UTC) (envelope-from rsharpe@richardsharpe.com) Received: from zmail.servaris.com (zmail.servaris.com [107.6.51.160]) by mx1.freebsd.org (Postfix) with ESMTP id ECB768FC1C for ; Sun, 9 Dec 2012 01:05:34 +0000 (UTC) Received: (qmail 70620 invoked by uid 89); 9 Dec 2012 01:05:33 -0000 Received: from unknown (HELO ?192.168.2.23?) (rsharpe@richardsharpe.com@108.225.16.199) by mail.richardsharpe.com with ESMTPA; 9 Dec 2012 01:05:33 -0000 Subject: Re: Possible obscure socket leak when system under load and listener is slow to accept From: Richard Sharpe To: Andre Oppermann In-Reply-To: <50C3D22D.3060008@freebsd.org> References: <50C3D22D.3060008@freebsd.org> Content-Type: text/plain; charset="UTF-8" Date: Sat, 08 Dec 2012 17:05:31 -0800 Message-ID: <1355015131.6752.12.camel@localhost.localdomain> Mime-Version: 1.0 X-Mailer: Evolution 2.32.3 (2.32.3-1.fc14) Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Dec 2012 01:05:35 -0000 On Sun, 2012-12-09 at 00:50 +0100, Andre Oppermann wrote: > > Hi folks, > > > > Our QA group (at xxx) using Samba and smbtorture has been seeing a > > lot of cases where accept returns ECONNABORTED because the system load > > is high and Samba has a large listen backlog. > > > > Every now and then we get a crash in smbd or in winbindd and winbindd > > complains of too many open files in the system. > > > > In looking at kern_accept, it seems to me that FreeBSD can leak a socket > > when kern_accept calls soaccept on it but gets ECONNABORTED. This error > > is the only error returned from tcp_usr_accept. > > > > It seems like the socket taken off so_comp is never freed in this case > > and that there has been a call on soref on it as well, so that something > > like the following is needed in the error path: > > > > ==== //some-path/freebsd/sys/kern/uipc_syscalls.c#1 > > - /home/rsharpe/dev-src/packages/freebsd/sys/kern/uipc_syscalls.c ==== > > @@ -433,6 +433,14 @@ > > */ > > if (name) > > *namelen = 0; > > + /* > > + * We need to close the socket we unlinked > > + * so we do not leak it. > > + */ > > + ACCEPT_LOCK(); > > + SOCK_LOCK(so); > > + soclose(so); > > goto noconnection; > > } > > if (sa == NULL) { > > > > I think an soclose is needed at this point because soisconnected has > > been called on the socket. > > > > Do you think this analysis is reasonable? > > > > We are using FreeBSD 8.0 but it seems the same is true for 9.0. However, > > maybe I am wrong since I am not sure if the fdclose call would free the > > socket, but a quick look suggested that it doesn't. > > The fdclose should properly tear down the file descriptor. The call > graph is: fdclose() -> fdrop() -> _fdrop() -> fo_close()/soo_close() -> > soclose() -> sorele() -> sofree() -> sodealloc(). > > A socket leak would not count against "kern.maxfiles" unless the file > descriptor leaks as well. So it is unlikely that this is the problem. OK, thanks for the feedback. I will keep looking. > Samba may open a large number of files (real files and sockets) and > you may run into the maxfiles limit. You can check the limit with > "sysctl kern.maxfiles" and increase it at boot time in boot/loader.conf > with "kern.maxfiles=100000" for example. Well, some of the smbds are dying, but it is possible that there is a file leak in Samba or our VFS that we are tripping as well. From owner-freebsd-hackers@FreeBSD.ORG Sun Dec 9 01:45:57 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 47D2028D for ; Sun, 9 Dec 2012 01:45:57 +0000 (UTC) (envelope-from dieterbsd@engineer.com) Received: from mailout-us.gmx.com (mailout-us.gmx.com [74.208.5.67]) by mx1.freebsd.org (Postfix) with SMTP id DB3A08FC15 for ; Sun, 9 Dec 2012 01:45:56 +0000 (UTC) Received: (qmail 16547 invoked by uid 0); 9 Dec 2012 01:45:50 -0000 Received: from 67.206.183.28 by rms-us001 with HTTP Content-Type: text/plain; charset="utf-8" Date: Sat, 08 Dec 2012 20:45:46 -0500 From: "Dieter BSD" Message-ID: <20121209014547.238070@gmx.com> MIME-Version: 1.0 Subject: FreeBSD for serious performance? (was: Re: 9.x -- New Install -- serious partition misalignment) To: freebsd-hackers@freebsd.org X-Authenticated: #74169980 X-Flags: 0001 X-Mailer: GMX.com Web Mailer x-registered: 0 Content-Transfer-Encoding: 8bit X-GMX-UID: HIixcJs93zOlNR3dAHAh06F+IGRvb0B2 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Dec 2012 01:45:57 -0000 Ronald writes: > This probably wouldn't be such a big deal if we were just talking about > Linux.  But FreeBSD has always prided itself on being a serious OS for > serious people with serious work to do... like major server farms and > such.  In the context of high-end applications on high-end hardware where > people are often trying to squeeze out that last drop of performance, Linux is certainly a steaming pile of crap. BSD is orders of magnitude better, but hey, that doesn't take much. But don't brag about high-end hardware.  But FreeBSD has dropped support for even semi-high-end hardware (DEC Alpha). So I'm stuck running it on AMD64. Nothing against AMD, they did what they could to try and make a silk purse (amd64) out of a sow's ear (x86). But even getting what passes for a high quality board in amd64/x86 land with good reviews doesn't compare. The firmware is absolute crap, and it's not like it is something you can ignore. BTW, real high end hardware is redundant, better than mil-spec, and provides better than 5-9s uptime. Been there, done that. Several chips/features aren't supported properly. PRs sit for years on end. > Performance has been degraded by a whopping 75% ! Having a 4KiB misalignment is nothing compared with not having NCQ support. (Which even linux has, btw.) 25% performance would be a massive upgrade. Or even worse, having the disk driver go into an infinite loop with interrupts blocked, so *nothing* happens and all your incoming data is lost until you manually intervene. Speaking of alignment, I still get "partition 1 does not end on a track boundary" messages. FreeBSD has no clue where the track boundaries are and neither do I. Disks have used varying numbers of sectors/track for longer than FreeBSD has existed. This is your idea of serious? From owner-freebsd-hackers@FreeBSD.ORG Sun Dec 9 03:52:41 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E1C5F864 for ; Sun, 9 Dec 2012 03:52:41 +0000 (UTC) (envelope-from rfg@tristatelogic.com) Received: from outgoing.tristatelogic.com (segfault.tristatelogic.com [69.62.255.118]) by mx1.freebsd.org (Postfix) with ESMTP id B45C38FC08 for ; Sun, 9 Dec 2012 03:52:41 +0000 (UTC) Received: from segfault-nmh-helo.tristatelogic.com (localhost [127.0.0.1]) by segfault.tristatelogic.com (Postfix) with ESMTP id 4BB495081D for ; Sat, 8 Dec 2012 19:52:34 -0800 (PST) cc: freebsd-hackers@freebsd.org Subject: Re: FreeBSD for serious performance? (was: Re: 9.x -- New Install -- serious partition misalignment) In-Reply-To: <20121209014547.238070@gmx.com> Date: Sat, 08 Dec 2012 19:52:34 -0800 Message-ID: <1830.1355025154@tristatelogic.com> From: "Ronald F. Guilmette" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Dec 2012 03:52:41 -0000 In message <20121209014547.238070@gmx.com>, "Dieter BSD" wrote: >But don't brag about high-end hardware.  But FreeBSD has dropped support >for even semi-high-end hardware (DEC Alpha). So I'm stuck running it on >AMD64. Nothing against AMD, they did what they could to try and make a silk >purse (amd64) out of a sow's ear (x86). But even getting what passes for >a high quality board in amd64/x86 land with good reviews doesn't compare. >The firmware is absolute crap, and it's not like it is something you can >ignore. BTW, real high end hardware is redundant, better than mil-spec, >and provides better than 5-9s uptime. Been there, done that. > >Several chips/features aren't supported properly. PRs sit for years on end. > >> Performance has been degraded by a whopping 75% ! > >Having a 4KiB misalignment is nothing compared with not having NCQ >support. (Which even linux has, btw.) 25% performance would be a massive >upgrade. Or even worse, having the disk driver go into an infinite loop >with interrupts blocked, so *nothing* happens and all your incoming data >is lost until you manually intervene. > >Speaking of alignment, I still get "partition 1 does not end on a >track boundary" messages. FreeBSD has no clue where the track boundaries >are and neither do I. Disks have used varying numbers of sectors/track >for longer than FreeBSD has existed. > >This is your idea of serious? In general, I agree with the point(s) you are making, but I do have a few minor quibbles... 1) According to: http://en.wikipedia.org/wiki/DEC_Alpha#Model_history the last Alpha to be produced was shipped way back in 2004... eight years ago... with a top speed of 1.3 GHz. I now have a cheap little media player thingy sitting on my desk, and _each_ of its two cores runs faster than that. In short, Alphas hardly constitute high-end hardware in this day and age. 2) Although I don't know a lot about boards, it is my understanding that ASUS makes pretty good ones, and they have always worked for me (and the firmware is typically quite good). As regards to the Native Command Queuing.... all I can say is "Crap!" I wasn't aware...until now... that FreeBSD did not support that. That really is a rather entirely serious issue. But I do think that the performance hit from that would be dwarfed by the performance hit that could be caused by the AF misaligment problem. And finally, yes, _nobody_ (except maybe the manufacturers) has known where the actual physical track boundaries have been for a long long time now. Regards, rfg From owner-freebsd-hackers@FreeBSD.ORG Sun Dec 9 04:04:41 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 188D7B61 for ; Sun, 9 Dec 2012 04:04:41 +0000 (UTC) (envelope-from kabaev@gmail.com) Received: from mail-vc0-f182.google.com (mail-vc0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id B61438FC16 for ; Sun, 9 Dec 2012 04:04:40 +0000 (UTC) Received: by mail-vc0-f182.google.com with SMTP id fo14so2017518vcb.13 for ; Sat, 08 Dec 2012 20:04:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:in-reply-to:references:x-mailer :mime-version:content-type; bh=UU46dKdHC1T3oL1/oK9rZgyE+aJrh5MaMpimA5aVDu4=; b=uKCAx0dOmcHB29fF3T1Ua7XZiIKFvu+cZEJcvOo+bYRPY9E7ykhypLyQ5iHMFkQAFB VfvhVrK19dk3ZG81W9Z51NpvpIv0q2ihe308LiiFt30toQKK+mcMDx1fAH/96X+xWQgX F6WbHqod9d1+QRKBOyjjDKnV1gZI9MAxcywEobLKIg6JEIRvMsPbVSkDCxUntZcW0AdN xRUcVBDNBJAI9jfkSndBW8QN6ob8dm644i3ozpqamm+4JBK7iTjjJRveiMv6mwPIjyGP xkTG9s0UI7Ou4nlGYSbhjc2UhpeXQKn3VzRs7pKHrdlxtEsQg/a+Hu2ShwLz2mVfyxO4 oI3A== Received: by 10.52.35.20 with SMTP id d20mr5792855vdj.50.1355025880254; Sat, 08 Dec 2012 20:04:40 -0800 (PST) Received: from kan.dyndns.org (c-24-63-226-98.hsd1.ma.comcast.net. [24.63.226.98]) by mx.google.com with ESMTPS id x17sm1711454vdi.1.2012.12.08.20.04.38 (version=SSLv3 cipher=OTHER); Sat, 08 Dec 2012 20:04:39 -0800 (PST) Date: Sat, 8 Dec 2012 23:04:33 -0500 From: Alexander Kabaev To: "Ronald F. Guilmette" Subject: Re: FreeBSD for serious performance? (was: Re: 9.x -- New Install -- serious partition misalignment) Message-ID: <20121208230433.2c54f44a@kan.dyndns.org> In-Reply-To: <1830.1355025154@tristatelogic.com> References: <20121209014547.238070@gmx.com> <1830.1355025154@tristatelogic.com> X-Mailer: Claws Mail 3.8.1 (GTK+ 2.24.6; amd64-portbld-freebsd10.0) Mime-Version: 1.0 Content-Type: multipart/signed; micalg=PGP-SHA1; boundary="Sig_/fPOp/Lv6L2AbPx+pizu/K++"; protocol="application/pgp-signature" Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Dec 2012 04:04:41 -0000 --Sig_/fPOp/Lv6L2AbPx+pizu/K++ Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Sat, 08 Dec 2012 19:52:34 -0800 "Ronald F. Guilmette" wrote: "analysis" skipped. >=20 > As regards to the Native Command Queuing.... all I can say is "Crap!" > I wasn't aware...until now... that FreeBSD did not support that. That > really is a rather entirely serious issue. But I do think that the > performance hit from that would be dwarfed by the performance hit that > could be caused by the AF misaligment problem. >=20 ada0 at ahcich1 bus 0 scbus2 target 0 lun 0 ada0: ATA-8 SATA 2.x device ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 715404MB (1465149168 512 byte sectors: 16H 63S/T 16383C) ada0: Previously was known as ad0 ada1 at ahcich3 bus 0 scbus4 target 0 lun 0 ada1: ATA-8 SATA 2.x device ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada1: Command Queueing enabled ada1: 715404MB (1465149168 512 byte sectors: 16H 63S/T 16383C) Crap! I must be dreaming then. --=20 Alexander Kabaev --Sig_/fPOp/Lv6L2AbPx+pizu/K++ Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iD8DBQFQxA3WQ6z1jMm+XZYRAseaAKDPlHKCrzVgKx/UbDTwwg7g5cajWACfYiXf 0YQzgKzM9RPluTHpUHWZooo= =/66g -----END PGP SIGNATURE----- --Sig_/fPOp/Lv6L2AbPx+pizu/K++-- From owner-freebsd-hackers@FreeBSD.ORG Sun Dec 9 04:08:13 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D1CC0C64 for ; Sun, 9 Dec 2012 04:08:13 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-vb0-f54.google.com (mail-vb0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 7935C8FC12 for ; Sun, 9 Dec 2012 04:08:13 +0000 (UTC) Received: by mail-vb0-f54.google.com with SMTP id l1so2049005vba.13 for ; Sat, 08 Dec 2012 20:08:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=2yKq6dW8Ah2pt4wkNXuruV6DVPYtfo3r1/WHgG+ttWE=; b=k81D4fSqosqqnGbZGS507h2MgLjAmnvZi8KFPndfMbXHqv9sAH7qj02AVRq36gI2U/ L2QJRbS+87TM1HJCLnPh5XFmX+Hoiiq8JNudwp5oca8phc8rx5bNC6/OkQ0b+sED+XuQ 5FmTNnch6REDV4VKrIolM9JINg4+eDP9znrn0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:x-gm-message-state; bh=2yKq6dW8Ah2pt4wkNXuruV6DVPYtfo3r1/WHgG+ttWE=; b=joWerqovGS4WJ8FD7t3ykrw9gZXbQ8Z4vF3C6EDmn/+tXgzLzjhXhlAH3t7uUGoHaR G7fY8R/SyBBVWeqSyS2cpDDNSBkWgN8jJGuXZsCgYiMybnoj2/m3qMXg5B8y6b/y0X0m l4HlrhxgRWgtSqWjPk5pR7p6F+Q635RnIIqQLY9RpFFwHIIYlpPM2mJT4/etMIRxRHf5 De5FT97Yr7cA4fxIbPcWhO5/BoGiLG3r7EEz4Bu3JCss2E6UGG6NzHuNLT3PirzqM6I0 afAZ9YWNQvCJvSLuZn7lvUXnzDpA5k8EkEPmXzUbQatSstKHHZGm6M/+znxVwlg+w0x4 wFzA== Received: by 10.58.252.72 with SMTP id zq8mr6881750vec.20.1355026091744; Sat, 08 Dec 2012 20:08:11 -0800 (PST) MIME-Version: 1.0 Received: by 10.220.113.137 with HTTP; Sat, 8 Dec 2012 20:07:40 -0800 (PST) In-Reply-To: <20121208230433.2c54f44a@kan.dyndns.org> References: <20121209014547.238070@gmx.com> <1830.1355025154@tristatelogic.com> <20121208230433.2c54f44a@kan.dyndns.org> From: Eitan Adler Date: Sat, 8 Dec 2012 23:07:40 -0500 Message-ID: Subject: Re: FreeBSD for serious performance? (was: Re: 9.x -- New Install -- serious partition misalignment) To: Alexander Kabaev Content-Type: text/plain; charset=UTF-8 X-Gm-Message-State: ALoCoQmuQs8hp2N2Gh08uf5e37EtRFiyfrMv65sKEQ8/cC7OZTUSJcJSS5eu5OBs0jDziy5Eh5v1 Cc: freebsd-hackers@freebsd.org, "Ronald F. Guilmette" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Dec 2012 04:08:13 -0000 On 8 December 2012 23:04, Alexander Kabaev wrote: > On Sat, 08 Dec 2012 19:52:34 -0800 > "Ronald F. Guilmette" wrote: > > "analysis" skipped. > >> >> As regards to the Native Command Queuing.... all I can say is "Crap!" >> I wasn't aware...until now... that FreeBSD did not support that. That >> really is a rather entirely serious issue. But I do think that the >> performance hit from that would be dwarfed by the performance hit that >> could be caused by the AF misaligment problem. >From my memory, if you use ahci(4) instead of the old ata(4) you have NCQ. If you still use ata(4), you don't. -- Eitan Adler From owner-freebsd-hackers@FreeBSD.ORG Sun Dec 9 04:19:09 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 02814F12 for ; Sun, 9 Dec 2012 04:19:09 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-vc0-f182.google.com (mail-vc0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id A2DD78FC0C for ; Sun, 9 Dec 2012 04:19:08 +0000 (UTC) Received: by mail-vc0-f182.google.com with SMTP id fo14so2022945vcb.13 for ; Sat, 08 Dec 2012 20:19:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=riEmCS03Q2u44KoM5oDb9wSHjw9MbdJH6utdSTNmTTs=; b=m0ebBYe2GmIdfgiaQsvf0wobykkRv32HMozXdXxhRv8B152mKp6hDEaO/JHWsZgB1d 1QRvTWCD5XBzE/aLO7XdVRfSDQZ6o7aVzd4ZAHBs2p68MD00+rXLp2zS1jgZe19Jek+2 nr1ljsbDq2KVOTK4TTxfb24n95ZDvt42hUEPpLPiPnFlSUMi382UGL+O54gOV+sC2iwH +yfAQdrkXocN0vzFbZpfdyKsEYJRi2L5O00DGWFQLiUDu0pBcNoLbHU2I1ZvBszvXSIe w8DuKpMQa5mhIKKHFulBYe0JryXTeSd9qRCstZV5KuX5TZ3U3WKJWx9nMtrRVEQhKE7L gL6Q== MIME-Version: 1.0 Received: by 10.52.67.133 with SMTP id n5mr5793481vdt.24.1355026747806; Sat, 08 Dec 2012 20:19:07 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.58.201.202 with HTTP; Sat, 8 Dec 2012 20:19:07 -0800 (PST) In-Reply-To: References: <20121209014547.238070@gmx.com> <1830.1355025154@tristatelogic.com> <20121208230433.2c54f44a@kan.dyndns.org> Date: Sat, 8 Dec 2012 20:19:07 -0800 X-Google-Sender-Auth: dsk0GICkOAfo3TbhC9Hy3xLLPwQ Message-ID: Subject: Re: FreeBSD for serious performance? (was: Re: 9.x -- New Install -- serious partition misalignment) From: Adrian Chadd To: Eitan Adler Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org, "Ronald F. Guilmette" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Dec 2012 04:19:09 -0000 Hi, Yes. atacam supports NCQ. The older IDE/ATA code doesn't support NCQ. The CAM ATA code (ie, atacam) supports it if the drive supports it. So, the "FreeBSD doesn't do NCQ" point is incorrect. If you don't believe me - look in sys/cam/ata/ata_da.c, look for ata_ncq_cmd(). Adrian From owner-freebsd-hackers@FreeBSD.ORG Sun Dec 9 04:21:36 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 18813132 for ; Sun, 9 Dec 2012 04:21:36 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-vb0-f54.google.com (mail-vb0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id AF6E98FC0C for ; Sun, 9 Dec 2012 04:21:35 +0000 (UTC) Received: by mail-vb0-f54.google.com with SMTP id l1so2053822vba.13 for ; Sat, 08 Dec 2012 20:21:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=5D9O3WuM9XDejlULhCsC6dXtoC06M8+5JVUtThcZVfI=; b=Y4uKM/gMEN50QzmW1dIxFfX9kQ6m2JJ5h1igF2cyeKj60oigWxPW696CFnhYDQb74h zO9KrgP4vFaXOqOzUxoyI063uOTtDVOhnf/mHHgc7rL2YqwSj71mgB5+JKmOTC7KeEYV 62efSBmLEkMlXBgmkHaQKNn5cmNobsouS2xhY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:x-gm-message-state; bh=5D9O3WuM9XDejlULhCsC6dXtoC06M8+5JVUtThcZVfI=; b=YAMKUdh+DsZrBLX5uwE4yKBjNtNQrhW7nyYd8fx8KhBSIOH2cas8+SvIx/rqxfvKj3 02zQT7Qo3TUXThVy44P059LTlrn0pFOKzvxpbWSwkNsLJdsJluN7N3WillV4+v2vHzwq KzFR9H8BJkxyQ7Ddb+mqaz+FZZxROhrrZSAWijdkhC02yxLRl9WH/elZaE6BK7YML/5v kj1KWymAvXjt4ksrdIkE1vOyeWu9p9wjgaF+4issz0OPmgSfkfvZ4LreWSCyjzqbHEN7 n3jP0kigjlotyDUWUle4cmcYSKOxoAfR0SCO7WH6o494JdK0xRkRySkdHgYv/M1+f2cb oMsg== Received: by 10.220.220.18 with SMTP id hw18mr6572367vcb.59.1355026895002; Sat, 08 Dec 2012 20:21:35 -0800 (PST) MIME-Version: 1.0 Received: by 10.220.113.137 with HTTP; Sat, 8 Dec 2012 20:21:04 -0800 (PST) In-Reply-To: References: <20121209014547.238070@gmx.com> <1830.1355025154@tristatelogic.com> <20121208230433.2c54f44a@kan.dyndns.org> From: Eitan Adler Date: Sat, 8 Dec 2012 23:21:04 -0500 Message-ID: Subject: Re: FreeBSD for serious performance? (was: Re: 9.x -- New Install -- serious partition misalignment) To: Adrian Chadd Content-Type: text/plain; charset=UTF-8 X-Gm-Message-State: ALoCoQlDnKPT5gEcHETLY6A/pJs1LFP5DwKv6rFaG+3QX427d2WCJAflM4iW3I8mPYyk2O+avwI/ Cc: freebsd-hackers@freebsd.org, "Ronald F. Guilmette" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Dec 2012 04:21:36 -0000 On 8 December 2012 23:19, Adrian Chadd wrote: > Hi, > > Yes. atacam supports NCQ. > > The older IDE/ATA code doesn't support NCQ. The CAM ATA code (ie, > atacam) supports it if the drive supports it. > > So, the "FreeBSD doesn't do NCQ" point is incorrect. > > If you don't believe me - look in sys/cam/ata/ata_da.c, look for ata_ncq_cmd(). Thanks for the correction - my memory was wrong. -- Eitan Adler From owner-freebsd-hackers@FreeBSD.ORG Sun Dec 9 05:51:06 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CF62CA81 for ; Sun, 9 Dec 2012 05:51:06 +0000 (UTC) (envelope-from wblock@wonkity.com) Received: from wonkity.com (wonkity.com [67.158.26.137]) by mx1.freebsd.org (Postfix) with ESMTP id 857C08FC08 for ; Sun, 9 Dec 2012 05:51:06 +0000 (UTC) Received: from wonkity.com (localhost [127.0.0.1]) by wonkity.com (8.14.5/8.14.5) with ESMTP id qB95p2T8002486; Sat, 8 Dec 2012 22:51:02 -0700 (MST) (envelope-from wblock@wonkity.com) Received: from localhost (wblock@localhost) by wonkity.com (8.14.5/8.14.5/Submit) with ESMTP id qB95p26R002483; Sat, 8 Dec 2012 22:51:02 -0700 (MST) (envelope-from wblock@wonkity.com) Date: Sat, 8 Dec 2012 22:51:02 -0700 (MST) From: Warren Block To: Dieter BSD Subject: Re: FreeBSD for serious performance? (was: Re: 9.x -- New Install -- serious partition misalignment) In-Reply-To: <20121209014547.238070@gmx.com> Message-ID: References: <20121209014547.238070@gmx.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (wonkity.com [127.0.0.1]); Sat, 08 Dec 2012 22:51:02 -0700 (MST) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Dec 2012 05:51:06 -0000 On Sat, 8 Dec 2012, Dieter BSD wrote: > Having a 4KiB misalignment is nothing compared with not having NCQ > support. ... > Speaking of alignment, I still get "partition 1 does not end on a > track boundary" messages. FreeBSD has no clue where the track boundaries > are and neither do I. Disks have used varying numbers of sectors/track > for longer than FreeBSD has existed. A. Use ahci(4): "The ada device driver takes full advantage of NCQ, when supported." Use 'camcontrol identify ada0' to check for NCQ support. B. Use GPT, which does not have the CHS baggage. It is easier and more versatile. My systems with GPT disks don't complain about track alignment. Or maybe that's ahci(4)'s doing. From owner-freebsd-hackers@FreeBSD.ORG Sun Dec 9 08:10:06 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CD9D29C0; Sun, 9 Dec 2012 08:10:06 +0000 (UTC) (envelope-from bright@mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id AE0DB8FC16; Sun, 9 Dec 2012 08:10:06 +0000 (UTC) Received: from Alfreds-MacBook-Pro-6.local (c-67-180-208-218.hsd1.ca.comcast.net [67.180.208.218]) by elvis.mu.org (Postfix) with ESMTPSA id 285521A3C78; Sun, 9 Dec 2012 00:10:05 -0800 (PST) Message-ID: <50C4475D.9020300@mu.org> Date: Sun, 09 Dec 2012 00:10:05 -0800 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/17.0 Thunderbird/17.0 MIME-Version: 1.0 To: Richard Sharpe Subject: Re: Possible obscure socket leak when system under load and listener is slow to accept References: <50C3D22D.3060008@freebsd.org> <1355015131.6752.12.camel@localhost.localdomain> In-Reply-To: <1355015131.6752.12.camel@localhost.localdomain> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, Andre Oppermann X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Dec 2012 08:10:06 -0000 On 12/8/12 5:05 PM, Richard Sharpe wrote: > On Sun, 2012-12-09 at 00:50 +0100, Andre Oppermann wrote: >>> Hi folks, >>> >>> Our QA group (at xxx) using Samba and smbtorture has been seeing a >>> lot of cases where accept returns ECONNABORTED because the system load >>> is high and Samba has a large listen backlog. >>> >>> Every now and then we get a crash in smbd or in winbindd and winbindd >>> complains of too many open files in the system. >>> >>> In looking at kern_accept, it seems to me that FreeBSD can leak a socket >>> when kern_accept calls soaccept on it but gets ECONNABORTED. This error >>> is the only error returned from tcp_usr_accept. >>> >>> It seems like the socket taken off so_comp is never freed in this case >>> and that there has been a call on soref on it as well, so that something >>> like the following is needed in the error path: >>> >>> ==== //some-path/freebsd/sys/kern/uipc_syscalls.c#1 >>> - /home/rsharpe/dev-src/packages/freebsd/sys/kern/uipc_syscalls.c ==== >>> @@ -433,6 +433,14 @@ >>> */ >>> if (name) >>> *namelen = 0; >>> + /* >>> + * We need to close the socket we unlinked >>> + * so we do not leak it. >>> + */ >>> + ACCEPT_LOCK(); >>> + SOCK_LOCK(so); >>> + soclose(so); >>> goto noconnection; >>> } >>> if (sa == NULL) { >>> >>> I think an soclose is needed at this point because soisconnected has >>> been called on the socket. >>> >>> Do you think this analysis is reasonable? >> > >>> We are using FreeBSD 8.0 but it seems the same is true for 9.0. However, >>> maybe I am wrong since I am not sure if the fdclose call would free the >>> socket, but a quick look suggested that it doesn't. >> The fdclose should properly tear down the file descriptor. The call >> graph is: fdclose() -> fdrop() -> _fdrop() -> fo_close()/soo_close() -> >> soclose() -> sorele() -> sofree() -> sodealloc(). >> >> A socket leak would not count against "kern.maxfiles" unless the file >> descriptor leaks as well. So it is unlikely that this is the problem. > OK, thanks for the feedback. I will keep looking. > >> Samba may open a large number of files (real files and sockets) and >> you may run into the maxfiles limit. You can check the limit with >> "sysctl kern.maxfiles" and increase it at boot time in boot/loader.conf >> with "kern.maxfiles=100000" for example. > Well, some of the smbds are dying, but it is possible that there is a > file leak in Samba or our VFS that we are tripping as well. lsof and sockstat can be helpful. lsof may be able to help determine if there's a leak because it MAY will find sockets not associated with a process. Hope this helps. -Alfred From owner-freebsd-hackers@FreeBSD.ORG Sun Dec 9 09:13:08 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1FB03FDD for ; Sun, 9 Dec 2012 09:13:08 +0000 (UTC) (envelope-from dieterbsd@engineer.com) Received: from mailout-us.gmx.com (mailout-us.gmx.com [74.208.5.67]) by mx1.freebsd.org (Postfix) with SMTP id CD2D78FC1B for ; Sun, 9 Dec 2012 09:13:07 +0000 (UTC) Received: (qmail 19038 invoked by uid 0); 9 Dec 2012 09:13:07 -0000 Received: from 67.206.183.93 by rms-us016 with HTTP Content-Type: text/plain; charset="utf-8" Date: Sun, 09 Dec 2012 04:13:03 -0500 From: "Dieter BSD" Message-ID: <20121209091305.238100@gmx.com> MIME-Version: 1.0 Subject: Re: FreeBSD for serious performance? To: freebsd-hackers@freebsd.org X-Authenticated: #74169980 X-Flags: 0001 X-Mailer: GMX.com Web Mailer x-registered: 0 Content-Transfer-Encoding: 8bit X-GMX-UID: djO2cGY83zOlNR3dAHAh1Cx+IGRvb8CF X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Dec 2012 09:13:08 -0000 Ronald writes: > the last Alpha to be produced was shipped way back in 2004... eight years > ago... with a top speed of 1.3 GHz.  I now have a cheap little media player > thingy sitting on my desk, and _each_ of its two cores runs faster than that. > In short, Alphas hardly constitute high-end hardware in this day and age. So clock rate is the only thing that matters in your world? In my world, high-end means high quality. It doesn't necessarily mean fast, or recent. Warren writes: > A. Use ahci(4): >    "The ada device driver takes full advantage of NCQ, when supported." Achi and siis support NCQ, but neither attach to the nforce4-ultra, which does support NCQ. I knew that NCQ would be required for acceptable performance and gave up other useful features to get it. Silly me, assuming that the performance orientated version of BSD would support such an essential performance feature on a very popular chipset. Linux has had NCQ on nforce4 since Oct 2006. Without NCQ, writes are slower than USB2, even with purely sequential writes (to the raw drive, no filesystem, so no seeking other then to the next track): device    r/s    w/s     kr/s     kw/s wait  svc_t  %b  controller   ad6 1726.5    0.0 217535.0      0.0    1    0.6  96  ata3-master   ad6    0.0  109.5      0.0  13794.8    1    9.1 100  ata3-master   da0  249.9    0.0  15741.4      0.0    1    6.0 100  umass-sim0 ad6 is ST3000DM001-9YN166 on nforce4-ultra chipset da0 is ST3000DM001-9YN166 USB3 disk on a USB2 port (from nforce) Sorry no write performance data for da0, I yanked the USB-to-SATA bridge off after doing some testing. The bridges they're currently shipping are slightly less crappy than they used to be, but they are still crap. > B. Use GPT, which does not have the CHS baggage.  It is easier and more >    versatile.  My systems with GPT disks don't complain about track >    alignment.  Or maybe that's ahci(4)'s doing. I never found a way to boot from different partitions, much less different disks with GPT. So I use NetBSD's MBR for disks I want to boot from. FreeBSD's and NetBSD's fdisk are both broken. Building MBRs by hand is such fun. Non-boot disks with multiple partitions use GPT. Big data disks get newfs-ed directly, no partitioning. The useless CHS baggage hangs around for decades, but useful hardware loses all support 5 nanoseconds after the last machine is sold. Other useful hardware waits years hoping to get support. From owner-freebsd-hackers@FreeBSD.ORG Sun Dec 9 09:52:22 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 40F49497 for ; Sun, 9 Dec 2012 09:52:22 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wi0-f170.google.com (mail-wi0-f170.google.com [209.85.212.170]) by mx1.freebsd.org (Postfix) with ESMTP id C30DD8FC14 for ; Sun, 9 Dec 2012 09:52:21 +0000 (UTC) Received: by mail-wi0-f170.google.com with SMTP id hq7so514191wib.1 for ; Sun, 09 Dec 2012 01:52:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=RSev5BNZsaVybnL0Uq4G3T/oj8b0GCKqvVO0NRqfVdw=; b=X+k5gvUtH+fZu6jfLxqmntWyhzzHf8ZfwxDmugpd/xxgnNWhUANiMG7RHpNQGCz5BS 4HCIsy8c5HOmzdSBg7tCu0+Zz1vI8+OlbrOjcACMM2de+zoa1Z0P8Xl0LrG2+MWIri1A HOQRjFfwho8Mug8KWJppzUpUPcxYKYWsC2P4uImRPSZ2doijS7V5xCNFdMArMyWgGF+M GqanAUuCsjqPn+TaeqY7wyF23kyxHazOqarf1/FfiHBu4f5YkPmMjRQ6ovzJlJdBff+C pnmwqe7ojAU+7ovPPZicmlHxqWHJyyWjy3VLrwaUHrk69bbfS6hTQ3/JUGobKfaguaej hu0g== MIME-Version: 1.0 Received: by 10.180.102.234 with SMTP id fr10mr5807868wib.17.1355046740782; Sun, 09 Dec 2012 01:52:20 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.217.57.9 with HTTP; Sun, 9 Dec 2012 01:52:20 -0800 (PST) In-Reply-To: <20121209091305.238100@gmx.com> References: <20121209091305.238100@gmx.com> Date: Sun, 9 Dec 2012 01:52:20 -0800 X-Google-Sender-Auth: VqG-SDziBHCMl4OLEE__5iPcm0w Message-ID: Subject: Re: FreeBSD for serious performance? From: Adrian Chadd To: Dieter BSD Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Dec 2012 09:52:22 -0000 Right, so the bug here is "why isn't atacam attaching to the nforce4 ultra chipset." So this has changed from "FreeBSD doesn't do NCQ" to "FreeBSD doesn't do NCQ on my particular desktop-aimed motherboard chipset." They're slightly different in scope, wouldn't you agree? Please file a PR and see if that can get resolved. I personally have no idea about the storage side of things so I don't know if it's a device id or whether there's something more complicated than that. Thanks, Adrian From owner-freebsd-hackers@FreeBSD.ORG Sun Dec 9 12:33:23 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 066B2E13; Sun, 9 Dec 2012 12:33:23 +0000 (UTC) (envelope-from bf1783@googlemail.com) Received: from mail-ie0-f170.google.com (mail-ie0-f170.google.com [209.85.223.170]) by mx1.freebsd.org (Postfix) with ESMTP id B27198FC08; Sun, 9 Dec 2012 12:33:22 +0000 (UTC) Received: by mail-ie0-f170.google.com with SMTP id k10so5787653iea.15 for ; Sun, 09 Dec 2012 04:33:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:date:message-id:subject:from:to:content-type; bh=XmjDjleXhpbuU3/geuxUSYjj2mGZ39LG5750vLqq+4Y=; b=BVL6u1epZr885xOphWmavgevvswNelT+H4OotD2OE8aR6BgoKfRLrIq4X3a2tzyfhK vL3X+LKFzmcPATgVZjamGuy5jwPPrGUM5Ag6sBEn250BcoGO6j12EgHnDoBxNeTWqPqr pTcA+go0TCHzTA71e6SQWWOqc7iRnL5XA4yUn9nb8sHv5uhZfLEtTypiCCjPtlW7PFrW 2k8c5v1oCFIKAyeqqT0BDrqgENzk1Xm+45RleDGORFcbPhSRf50AYUNoNHMf52EkAGAv UyNWY3xLq249kkbUYvxXgG0md1hgzi7zYonCY0JGbX449GLXEte7OFsEjJGEKqxYInGn XbHg== MIME-Version: 1.0 Received: by 10.50.161.169 with SMTP id xt9mr3715569igb.62.1355056396087; Sun, 09 Dec 2012 04:33:16 -0800 (PST) Received: by 10.50.88.137 with HTTP; Sun, 9 Dec 2012 04:33:16 -0800 (PST) Date: Sun, 9 Dec 2012 12:33:16 +0000 Message-ID: Subject: Re: FreeBSD for serious performance? From: "b.f." To: Adrian Chadd , freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: bf1783@gmail.com List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Dec 2012 12:33:23 -0000 >Right, so the bug here is "why isn't atacam attaching to the nforce4 >ultra chipset." > >So this has changed from "FreeBSD doesn't do NCQ" to "FreeBSD doesn't >do NCQ on my particular desktop-aimed motherboard chipset." They're >slightly different in scope, wouldn't you agree? >Please file a PR and see if that can get resolved. I personally have >no idea about the storage side of things so I don't know if it's a >device id or whether there's something more complicated than that. Both the old and the new ata(4) have basic support for most of the older Nvidia chipsets, but lack some SATA-specific features like NCQ. Before AHCI was widely adopted, there were a variety of different interfaces on early SATA controllers. When Alexander and Scott were rewriting parts of the ATA and CAM code in FreeBSD, they had a limited amount of time and resources, so they concentrated on the newer ahci(4), mvs(4), and siis(4), not the many different interfaces of the earlier SATA controllers, although they invited people to add some of these on their own. The situation is also complicated by the fact that some of these earlier controllers had bugs, so even in Linux, where a larger number of the earlier controllers are more fully supported, some of the drivers disable NCQ and some other features by default, although the controllers are supposed to support them, e.g.: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=commitdiff;h=e2e031eb09760c36099ac127eeb175e06d257aef Of course, it is now possible to buy relatively inexpensive add-in SATA controller cards that are fully supported by one of the newer drivers and perform better than most of the old controllers. b. From owner-freebsd-hackers@FreeBSD.ORG Sun Dec 9 13:41:10 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E9FDE5E6; Sun, 9 Dec 2012 13:41:10 +0000 (UTC) (envelope-from rank1seeker@gmail.com) Received: from mail-ee0-f54.google.com (mail-ee0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id 03B8E8FC08; Sun, 9 Dec 2012 13:41:09 +0000 (UTC) Received: by mail-ee0-f54.google.com with SMTP id c13so1306618eek.13 for ; Sun, 09 Dec 2012 05:41:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:from:to:subject:date:content-type :content-transfer-encoding:in-reply-to:references:x-mailer; bh=OtVI3WYqgnD11J1R2dcQaduYsmP9pvJlYuBwh7SezyQ=; b=J7tGQC/tw3bObX1GnMCkMEnHkw/zrJQJpG8Aln4U1YwFnCmsWYKRUHxqgZIvz8u9EF QiWC15WUq02ed6u0ICKXQX2OVXK8uFvjjR/iX/FQfAEYMdHWVn8jgzOYBA+rp73ij8yE pUBczOXLlmapAK0AvoX44KXtTAVklUwWBsBa98XhEFummM4nveop3gG5wdqEZqmkvGfz UEINAFBd0x0Dvos+GOKfQHcqqt3/dc4B8Xj1ACIj6jDzcgCBF4MYoSswqSXovCpkQMl2 vrm8KngE/heONYZ3eJBZdBthlz1+bY2XeKalwGOFgHXJ8Ig7A/SNqIKWCIyioyeI/CY/ WeUQ== Received: by 10.14.0.3 with SMTP id 3mr38572626eea.16.1355060468882; Sun, 09 Dec 2012 05:41:08 -0800 (PST) Received: from DOMYPC (93-138-86-170.adsl.net.t-com.hr. [93.138.86.170]) by mx.google.com with ESMTPS id f49sm36669901eep.12.2012.12.09.05.41.06 (version=SSLv3 cipher=OTHER); Sun, 09 Dec 2012 05:41:07 -0800 (PST) Message-ID: <20121209.134114.135.1@DOMY-PC> From: rank1seeker@gmail.com To: hackers@freebsd.org, alc@freebsd.org, avg@freebsd.org, davide@freebsd.org Subject: Re: huge ktr buffer Date: Sun, 09 Dec 2012 14:41:14 +0100 Content-Type: text/plain; charset="Windows-1250" Content-Transfer-Encoding: quoted-printable In-Reply-To: References: <50C0B763.8000004@FreeBSD.org> X-Mailer: POP Peeper (3.8.1.0) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Dec 2012 13:41:11 -0000 > As also Alan suggested, a way to workaround the problem is to = increase=0D=0A> NKPT value (e.g. from 32 to 64). Obviously, this is not a = proper fix.=0D=0A> For a proper fix the kernel needs to be able to = dynamically set the=0D=0A> size of NKPT. In this particular case, this = wouldn't be too hard, but=0D=0A> there is a different case, where people = preload a large memory disk=0D=0A> image at boot time that isn't so easy = to fix.=0D=0A> =0D=0A> Thanks,=0D=0A> =0D=0A> Davide=0D=0A=0D=0A=0D=0AHad = a same issue.=0D=0AI use very big preloaded images, with full world + = many compiled ports in it.=0D=0A=0D=0AFix:=0D=0A'sh' code snip = ...=0D=0A----=0D=0A# Get default NKTP value=0D=0Ankpt=3D`cat = "/sys/$arch/include/pmap.h" | sed -En = 's/.+NKPT[[:blank:]]+([0-9]{2})$/\1/p'`=0D=0A=0D=0A# How many additional = NKPT (4 Mb each), for our image, added to amount of NKPT?=0D=0A# = Calculated in Kb=0D=0A: $((nkpt +=3D "$img_size" / = 4096))=0D=0A----=0D=0A=0D=0ABut it loads sooooo slow into the = RAM.=0D=0AThat should be enhanced, too.=0D=0A=0D=0A=0D=0ADomagoj = Smol=E8i=E6 From owner-freebsd-hackers@FreeBSD.ORG Sun Dec 9 15:32:23 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2AD63B06 for ; Sun, 9 Dec 2012 15:32:23 +0000 (UTC) (envelope-from wblock@wonkity.com) Received: from wonkity.com (wonkity.com [67.158.26.137]) by mx1.freebsd.org (Postfix) with ESMTP id D6BD88FC18 for ; Sun, 9 Dec 2012 15:32:22 +0000 (UTC) Received: from wonkity.com (localhost [127.0.0.1]) by wonkity.com (8.14.5/8.14.5) with ESMTP id qB9FWIj9005618; Sun, 9 Dec 2012 08:32:18 -0700 (MST) (envelope-from wblock@wonkity.com) Received: from localhost (wblock@localhost) by wonkity.com (8.14.5/8.14.5/Submit) with ESMTP id qB9FWIs9005615; Sun, 9 Dec 2012 08:32:18 -0700 (MST) (envelope-from wblock@wonkity.com) Date: Sun, 9 Dec 2012 08:32:18 -0700 (MST) From: Warren Block To: Dieter BSD Subject: Re: FreeBSD for serious performance? In-Reply-To: <20121209091305.238100@gmx.com> Message-ID: References: <20121209091305.238100@gmx.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="3512871622-1204479703-1355067138=:5523" X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (wonkity.com [127.0.0.1]); Sun, 09 Dec 2012 08:32:19 -0700 (MST) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Dec 2012 15:32:23 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --3512871622-1204479703-1355067138=:5523 Content-Type: TEXT/PLAIN; charset=utf-8; format=flowed Content-Transfer-Encoding: 8BIT On Sun, 9 Dec 2012, Dieter BSD wrote: >> B. Use GPT, which does not have the CHS baggage.  It is easier and more >>    versatile.  My systems with GPT disks don't complain about track >>    alignment.  Or maybe that's ahci(4)'s doing. > > I never found a way to boot from different partitions, much less > different disks with GPT. Grub (or grub2) can do it. I haven't tried it because other operating systems I multiboot do not support GPT anyway. If you're booting multiple versions of FreeBSD, see gpart(8) for some partition attributes that may help. Or even consider ZFS boot environments. --3512871622-1204479703-1355067138=:5523-- From owner-freebsd-hackers@FreeBSD.ORG Sun Dec 9 17:57:34 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A0B4219F for ; Sun, 9 Dec 2012 17:57:34 +0000 (UTC) (envelope-from rsharpe@richardsharpe.com) Received: from zmail.servaris.com (zmail.servaris.com [107.6.51.160]) by mx1.freebsd.org (Postfix) with ESMTP id 5E63A8FC0C for ; Sun, 9 Dec 2012 17:57:34 +0000 (UTC) Received: (qmail 47081 invoked by uid 89); 9 Dec 2012 17:57:32 -0000 Received: from unknown (HELO ?192.168.2.23?) (rsharpe@richardsharpe.com@108.225.16.199) by mail.richardsharpe.com with ESMTPA; 9 Dec 2012 17:57:32 -0000 Subject: Re: Possible obscure socket leak when system under load and listener is slow to accept From: Richard Sharpe To: Alfred Perlstein In-Reply-To: <50C4475D.9020300@mu.org> References: <50C3D22D.3060008@freebsd.org> <1355015131.6752.12.camel@localhost.localdomain> <50C4475D.9020300@mu.org> Content-Type: text/plain; charset="UTF-8" Date: Sun, 09 Dec 2012 09:57:30 -0800 Message-ID: <1355075850.6752.15.camel@localhost.localdomain> Mime-Version: 1.0 X-Mailer: Evolution 2.32.3 (2.32.3-1.fc14) Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, Andre Oppermann X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Dec 2012 17:57:34 -0000 On Sun, 2012-12-09 at 00:10 -0800, Alfred Perlstein wrote: > On 12/8/12 5:05 PM, Richard Sharpe wrote: > > On Sun, 2012-12-09 at 00:50 +0100, Andre Oppermann wrote: > >>> Hi folks, > >>> > >>> Our QA group (at xxx) using Samba and smbtorture has been seeing a > >>> lot of cases where accept returns ECONNABORTED because the system load > >>> is high and Samba has a large listen backlog. > >>> > >>> Every now and then we get a crash in smbd or in winbindd and winbindd > >>> complains of too many open files in the system. > >>> > >>> In looking at kern_accept, it seems to me that FreeBSD can leak a socket > >>> when kern_accept calls soaccept on it but gets ECONNABORTED. This error > >>> is the only error returned from tcp_usr_accept. > >>> > >>> It seems like the socket taken off so_comp is never freed in this case > >>> and that there has been a call on soref on it as well, so that something > >>> like the following is needed in the error path: > >>> > >>> ==== //some-path/freebsd/sys/kern/uipc_syscalls.c#1 > >>> - /home/rsharpe/dev-src/packages/freebsd/sys/kern/uipc_syscalls.c ==== > >>> @@ -433,6 +433,14 @@ > >>> */ > >>> if (name) > >>> *namelen = 0; > >>> + /* > >>> + * We need to close the socket we unlinked > >>> + * so we do not leak it. > >>> + */ > >>> + ACCEPT_LOCK(); > >>> + SOCK_LOCK(so); > >>> + soclose(so); > >>> goto noconnection; > >>> } > >>> if (sa == NULL) { > >>> > >>> I think an soclose is needed at this point because soisconnected has > >>> been called on the socket. > >>> > >>> Do you think this analysis is reasonable? > >> > > >>> We are using FreeBSD 8.0 but it seems the same is true for 9.0. However, > >>> maybe I am wrong since I am not sure if the fdclose call would free the > >>> socket, but a quick look suggested that it doesn't. > >> The fdclose should properly tear down the file descriptor. The call > >> graph is: fdclose() -> fdrop() -> _fdrop() -> fo_close()/soo_close() -> > >> soclose() -> sorele() -> sofree() -> sodealloc(). > >> > >> A socket leak would not count against "kern.maxfiles" unless the file > >> descriptor leaks as well. So it is unlikely that this is the problem. > > OK, thanks for the feedback. I will keep looking. > > > >> Samba may open a large number of files (real files and sockets) and > >> you may run into the maxfiles limit. You can check the limit with > >> "sysctl kern.maxfiles" and increase it at boot time in boot/loader.conf > >> with "kern.maxfiles=100000" for example. > > Well, some of the smbds are dying, but it is possible that there is a > > file leak in Samba or our VFS that we are tripping as well. > > lsof and sockstat can be helpful. lsof may be able to help determine if > there's a leak because it MAY will find sockets not associated with a > process. > > Hope this helps. Thanks Alfred. After following through the call graph and confirming (with the code) that it was correct, I am now pretty convinced that I was wrong in assuming that it was a socket leak. However, lsof will be useful in allowing me to see how many FDs each smdb in this test is using. We have, I am told, kern.maxfiles set to 65536, which I think might be a little low for the test they are running. From owner-freebsd-hackers@FreeBSD.ORG Sun Dec 9 22:02:11 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9E7CF949 for ; Sun, 9 Dec 2012 22:02:11 +0000 (UTC) (envelope-from rfg@tristatelogic.com) Received: from outgoing.tristatelogic.com (segfault.tristatelogic.com [69.62.255.118]) by mx1.freebsd.org (Postfix) with ESMTP id 777738FC12 for ; Sun, 9 Dec 2012 22:02:11 +0000 (UTC) Received: from segfault-nmh-helo.tristatelogic.com (localhost [127.0.0.1]) by segfault.tristatelogic.com (Postfix) with ESMTP id D97AC5081A for ; Sun, 9 Dec 2012 14:02:10 -0800 (PST) To: freebsd-hackers@freebsd.org Subject: Re: FreeBSD for serious performance? In-Reply-To: <20121209091305.238100@gmx.com> Date: Sun, 09 Dec 2012 14:02:10 -0800 Message-ID: <20594.1355090530@tristatelogic.com> From: "Ronald F. Guilmette" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Dec 2012 22:02:11 -0000 In message <20121209091305.238100@gmx.com>, "Dieter BSD" wrote: >Ronald writes: >> the last Alpha to be produced was shipped way back in 2004... eight years >> ago... with a top speed of 1.3 GHz.  I now have a cheap little media player >> thingy sitting on my desk, and _each_ of its two cores runs faster than that. >> In short, Alphas hardly constitute high-end hardware in this day and age. > >So clock rate is the only thing that matters in your world? Yea, pretty much. As regards to reliability, except for the occasional low-level quirk (which is usually taken care of for me by the kernel guys) I've never had a processor fail on me. Once, about five or six years ago I accidentally burnt up an Athlon XP (by not having the heatsink properly seated) but that was entirely my fault. >I never found a way to boot from different partitions, much less >different disks with GPT. Having just been recently convinced to switch over to GPT (from MBR) I do most sincerly hope that you are either joking or mistaken about this. >The useless CHS baggage hangs around for decades, but useful >hardware loses all support 5 nanoseconds after the last machine >is sold. Other useful hardware waits years hoping to get support. Yeabut on the bright side, you can't beat the price! Regards, rfg From owner-freebsd-hackers@FreeBSD.ORG Sun Dec 9 22:15:34 2012 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3394BAF1; Sun, 9 Dec 2012 22:15:34 +0000 (UTC) (envelope-from grog@lemis.com) Received: from w3.lemis.com (w3.lemis.com [208.86.224.149]) by mx1.freebsd.org (Postfix) with ESMTP id EFB748FC12; Sun, 9 Dec 2012 22:15:33 +0000 (UTC) Received: from eureka.lemis.com (1032.x.rootbsd.net [208.86.224.149]) by w3.lemis.com (Postfix) with ESMTP id 74DEE3B751; Sun, 9 Dec 2012 22:15:26 +0000 (UTC) Received: by eureka.lemis.com (Postfix, from userid 1004) id 0EECBF794D; Mon, 10 Dec 2012 09:15:25 +1100 (EST) Date: Mon, 10 Dec 2012 09:15:25 +1100 From: Greg 'groggy' Lehey To: Jaakko Heinonen Subject: Re: calendar(1) regressions Message-ID: <20121209221525.GC36593@eureka.lemis.com> References: <20121206092202.GA2182@a91-153-116-96.elisa-laajakaista.fi> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="FkmkrVfFsRoUs1wW" Content-Disposition: inline In-Reply-To: <20121206092202.GA2182@a91-153-116-96.elisa-laajakaista.fi> User-Agent: Mutt/1.4.2.3i Organization: The FreeBSD Project Phone: +61-3-5346-1370 Mobile: +61-418-838-708 WWW-Home-Page: http://www.FreeBSD.org/ X-PGP-Fingerprint: 9A1B 8202 BCCE B846 F92F 09AC 22E6 F290 507A 4223 Cc: freebsd-hackers@FreeBSD.org, edwin@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Dec 2012 22:15:34 -0000 --FkmkrVfFsRoUs1wW Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Thursday, 6 December 2012 at 11:22:03 +0200, Jaakko Heinonen wrote: > > Hi, > > Unfortunately r205821 [1] has caused several regressions to calendar(1). > Relevant PRs: > > bin/157718 > bin/162211 > bin/168785 > bin/170930 > > Some regressions were fixed in summer 2011 but they are still lacking > MFCs. > > Is anyone aware of possible problems that reverting calendar(1) to > pre-r205821 version would cause? (Of course excluding the actual > calendar files.) I think we fix bugs rather than revert the commits. I've had a chat with edwin@, and I'll look at the issues Real Soon Now. Greg -- Sent from my desktop computer. Finger grog@FreeBSD.org for PGP public key. See complete headers for address and phone numbers. This message is digitally signed. If your Microsoft MUA reports problems, please read http://tinyurl.com/broken-mua --FkmkrVfFsRoUs1wW Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlDFDXwACgkQIubykFB6QiPZcwCdELqCkobZCMuuFQa7K0Dbczvt xiUAoIxUOrSnoUIpyxuzXQucNpnnQhnQ =IC+t -----END PGP SIGNATURE----- --FkmkrVfFsRoUs1wW-- From owner-freebsd-hackers@FreeBSD.ORG Sun Dec 9 22:37:09 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 814CC14F for ; Sun, 9 Dec 2012 22:37:09 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-we0-f182.google.com (mail-we0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 104BA8FC1B for ; Sun, 9 Dec 2012 22:37:08 +0000 (UTC) Received: by mail-we0-f182.google.com with SMTP id u54so1204084wey.13 for ; Sun, 09 Dec 2012 14:37:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=9U3onKW5ce5JmxOeMXQErygfHfYDDpifO0tpbVbspKs=; b=z9Sqw7F+FafYc7Jh/hm7aUYyDUx3eSmb/co3OOS8a+2uvQVeH9U7WXuZTInuyE/sc5 xHM1PYfpLZ6AJHGAshT+cB75yKcOqq80ac4qW6FaSBZ0M1sedCs6ILIsa7j3uU6FK13X /8prC76vfg4mEiSVLMpCyzwPMDKgInjsCTCiCSw7NdW5zfgaLq2M96Dulyg1O2Ivivmu t0CG9p3SksuLtgiHcejt+jdf1Gkm96bMiQ/5e8uAZOYtCXpWsW6ieUKgE94nG2EEERsf AJQNQFywXLQXUtwmexuqeqwDIB9eTdVgYqKoZM6VZEb5xDClYstXBkvGEpWA6nseOdPt wCCA== MIME-Version: 1.0 Received: by 10.180.97.137 with SMTP id ea9mr7802587wib.13.1355092627792; Sun, 09 Dec 2012 14:37:07 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.217.57.9 with HTTP; Sun, 9 Dec 2012 14:37:07 -0800 (PST) In-Reply-To: <20594.1355090530@tristatelogic.com> References: <20121209091305.238100@gmx.com> <20594.1355090530@tristatelogic.com> Date: Sun, 9 Dec 2012 14:37:07 -0800 X-Google-Sender-Auth: 3rpdrTJxNHSxFJgcSZ8miqbYINo Message-ID: Subject: Re: FreeBSD for serious performance? From: Adrian Chadd To: "Ronald F. Guilmette" Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Dec 2012 22:37:09 -0000 .. the problem with Alpha is that there wasn't anyone who wanted to support it any longer. If someone wanted to stand up and resurrect it, support it, etc; I doubt the FreeBSD project would complain. The same thing is happening with ia64. Marcel still cares and he still does a lot of ia64 heavy lifting. Adrian From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 10 04:42:27 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9627489E for ; Mon, 10 Dec 2012 04:42:27 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-lb0-f182.google.com (mail-lb0-f182.google.com [209.85.217.182]) by mx1.freebsd.org (Postfix) with ESMTP id F27798FC12 for ; Mon, 10 Dec 2012 04:42:26 +0000 (UTC) Received: by mail-lb0-f182.google.com with SMTP id go10so2180739lbb.13 for ; Sun, 09 Dec 2012 20:42:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=0zdEFE/m7GprczwdGf9SEwLc7mEY0sQslEmx3Brsaks=; b=TyfO1ZknlmS8IN9UcAhhK2xZQQbQy26piOAq4BJMiSxs3j0+au2laka3fKOBXnFxfw umCVuN4053UrCJ/+aOLoiOPlLK7f2QkXd1HgYKJpp+HtYA4WjzCf6YdnLZUD3p9i3/dv +AkS2Kc4sewq6hBwXb1ledx/iWW/00Om7jiKs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:x-gm-message-state; bh=0zdEFE/m7GprczwdGf9SEwLc7mEY0sQslEmx3Brsaks=; b=C6xhehmXFcO+qVhNGklXfblrAqRgerjQLl5JzoQ/TrSPlhQ0bF3hvQJDL4nPqApTLE JNaQs+RCozEhxJ0mDBVhSKIxOUpbDkisc/3gLJhFOYK6PkDCmGIQDkNbVCpt6iZ9AZ7u JTy3kppnuK9KppAAfeLtuN/sgRGAUg+CMLoRMzxH1W9bzxtdDwP23Ot/8ugTxI3vlq1g xmChp3+DLRYPxP+ldtGKLO5VU9NBgfP2QPB6onviraiyrhEJDuZStrCTJ0APAs9Oq7Po fRm9wtGDiXXlRfHFM5x0dvoP1xkK2ZDwvJfvytlEsODZz9i4OoFgnWsbklZHttClei+w WLNA== Received: by 10.152.145.162 with SMTP id sv2mr4226134lab.41.1355114545839; Sun, 09 Dec 2012 20:42:25 -0800 (PST) MIME-Version: 1.0 Received: by 10.112.43.229 with HTTP; Sun, 9 Dec 2012 20:41:55 -0800 (PST) In-Reply-To: References: From: Eitan Adler Date: Sun, 9 Dec 2012 23:41:55 -0500 Message-ID: Subject: Re: GELI on USB To: Ali Mashtizadeh Content-Type: text/plain; charset=UTF-8 X-Gm-Message-State: ALoCoQkpONyAM7YmbQJfYvb/SwRitpQ0uYjo+Y4OFLhP0RZeIUPD+jnzptj7l1LIOdDunYy7585T Cc: freebsd-fs@freebsd.org, FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Dec 2012 04:42:27 -0000 On 5 December 2012 04:54, Ali Mashtizadeh wrote: > I think I found a possible bug in 9.1 where I configured an encrypted > root partition on a USB key and I have trouble entering the password > from what seems like a race. Could you please submit a PR about this so it doesn't get lost in the archives of the mailing list? -- Eitan Adler From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 10 04:48:13 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 8A75FC37 for ; Mon, 10 Dec 2012 04:48:13 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: from mail-oa0-f54.google.com (mail-oa0-f54.google.com [209.85.219.54]) by mx1.freebsd.org (Postfix) with ESMTP id 4BB6F8FC12 for ; Mon, 10 Dec 2012 04:48:13 +0000 (UTC) Received: by mail-oa0-f54.google.com with SMTP id n9so2754141oag.13 for ; Sun, 09 Dec 2012 20:48:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=k7OiPlXmEVDTUYsMOo6Z5cRkZ7MbjkfAVlK+3Nb3cik=; b=uGLx76NLtwZrqVvnAulG/tPlkvpyDqfJ9YZs48s8g7JbMtlEeRMz9mGg/oj0z1yySS qwW/5YM7LUh3kAsdPNM26xJfU0YBTCmuzCQLnEk71gogzA98UUst5SurIj39YZo2YSdI MM6e7Eo1vWTRGAmHQDcqwEcBrjzxI+LMZXm5+GIFV8OIsYRKDsytT2sD/hA9RJdj/qVY D36hX3MA+AifJHSuOfJ4XYATKACs/cwEW9mpJiY50L121T/GOZZ74N+Ld4P6EweOc0cb nYqx+1RXOR/1NKDSmAlehmD0ZRgwQI03I8h6uAcqjUYRW+ZiHF1G2Aw3ACJjzznkSFX+ dMyg== MIME-Version: 1.0 Received: by 10.182.207.70 with SMTP id lu6mr6662670obc.78.1355114892499; Sun, 09 Dec 2012 20:48:12 -0800 (PST) Received: by 10.182.158.137 with HTTP; Sun, 9 Dec 2012 20:48:12 -0800 (PST) Date: Sun, 9 Dec 2012 23:48:12 -0500 Message-ID: Subject: using FreeBSD to create a completely new OS From: Aryeh Friedman To: FreeBSD Mailing List Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Dec 2012 04:48:13 -0000 For personal hobby reasons I want to write an OS completely from scratch (due to some aspects of the design no existing OS is a suitable starting place)... what I mean is I want to start with the MBR (boot0) and go on from there... I only have one *REAL* machine to work with which means I need to work with something like emulators/virtualbox-ose... I also want to do as many automated tests as possible (for example seeing if the installer copied the MBR [and later other stuff] correctly to the virtual HDD).... for this reason I have a few questions on vb (or perhaps QEMU if not possible in vb): 1. Can it be scripted? 2. Is there any documentation on the various virtual HDD formats and such (that way I can check the "physical" drive and not by indirect query)? Also can people give me some idea of a good general development/testing framework.... the one I have in mind so far is: 1. Write enough of the OS in FreeBSD to boot and give a command prompt and then develop using it (assume that there is a working compilor [note I am doing it completely in Java (note 1) and will be using either gjava (gcc) or writing my own compiler]) Notes: 1. Due to OS's needing to address physical RAM directly (DMA mapped I/O) I will be introducing some form of ptr's into java with the compiler or native ASM From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 10 05:17:55 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 04E385D1 for ; Mon, 10 Dec 2012 05:17:55 +0000 (UTC) (envelope-from dieterbsd@engineer.com) Received: from mailout-us.gmx.com (mailout-us.gmx.com [74.208.5.67]) by mx1.freebsd.org (Postfix) with SMTP id 9A1B48FC0C for ; Mon, 10 Dec 2012 05:17:54 +0000 (UTC) Received: (qmail 11673 invoked by uid 0); 10 Dec 2012 05:17:47 -0000 Received: from 67.206.185.224 by rms-us007 with HTTP Content-Type: text/plain; charset="utf-8" Date: Mon, 10 Dec 2012 00:17:43 -0500 From: "Dieter BSD" Message-ID: <20121210051745.238070@gmx.com> MIME-Version: 1.0 Subject: Re: FreeBSD for serious performance? To: freebsd-hackers@freebsd.org X-Authenticated: #74169980 X-Flags: 0001 X-Mailer: GMX.com Web Mailer x-registered: 0 Content-Transfer-Encoding: 8bit X-GMX-UID: LhW3cJs93zOlNR3dAHAh+eJ+IGRvb8AG X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Dec 2012 05:17:55 -0000 [ lack of SATA NCQ support for nforce4-ultra ] Adrian writes: > http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=commitdiff;h=e2e031eb09760c36099ac127eeb175e06d257aef which is: The mcp61 has bug with ncq. - { PCI_VDEVICE(NVIDIA, PCI_DEVICE_ID_NVIDIA_NFORCE_MCP61_SATA), SWNCQ }, - { PCI_VDEVICE(NVIDIA, PCI_DEVICE_ID_NVIDIA_NFORCE_MCP61_SATA2), SWNCQ }, - { PCI_VDEVICE(NVIDIA, PCI_DEVICE_ID_NVIDIA_NFORCE_MCP61_SATA3), SWNCQ }, + { PCI_VDEVICE(NVIDIA, PCI_DEVICE_ID_NVIDIA_NFORCE_MCP61_SATA), GENERIC }, + { PCI_VDEVICE(NVIDIA, PCI_DEVICE_ID_NVIDIA_NFORCE_MCP61_SATA2), GENERIC }, + { PCI_VDEVICE(NVIDIA, PCI_DEVICE_ID_NVIDIA_NFORCE_MCP61_SATA3), GENERIC }, http://en.wikipedia.org/wiki/GeForce_6_Series says: "MCP61 introduced a bug in the SATA NCQ implementation. As a result, Nvidia employees have contributed code to disable NCQ operations under Linux." I have not found a description of the bug. But FreeBSD says I have: port 0x9f0-0x9f7,0xbf0-0xbf3,0x970-0x977,0xb70-0xb73,0xcc00-0xcc0f mem 0xfebfb000-0xfebfbfff irq 21 at device 7.0 on pci0 port 0x9e0-0x9e7,0xbe0-0xbe3,0x960-0x967,0xb60-0xb63,0xb800-0xb80f mem 0xfebfa000-0xfebfafff irq 22 at device 8.0 on pci0 The CK804 (a few lines higher in the source file) did not get changed to GENERIC. As far as I can figure out, the bug (whatever it is) is limited to the MCP61. From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 10 06:15:58 2012 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 78EFEEA1 for ; Mon, 10 Dec 2012 06:15:58 +0000 (UTC) (envelope-from bf1783@googlemail.com) Received: from mail-ie0-f182.google.com (mail-ie0-f182.google.com [209.85.223.182]) by mx1.freebsd.org (Postfix) with ESMTP id 3F47B8FC14 for ; Mon, 10 Dec 2012 06:15:57 +0000 (UTC) Received: by mail-ie0-f182.google.com with SMTP id s9so8884657iec.13 for ; Sun, 09 Dec 2012 22:15:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:date:message-id:subject:from:to:content-type; bh=oegZVOf/49Ib2gl/JLubCVqVuGrg28rbx5CUMOQwgtQ=; b=HsobUJy/hC01Dj43u6wRCui3kNYn+jOCv/0JNvgPdVmVnsWaRJxeI6zCncHbsNjgdc Wo8KpusAW7mMm21CEJ3KJwX8xyim1IFl3khWY//zqCeeTPVd+4A3CWY7Wpe69b1F7OI4 EgdgKmP97tiIC1g3N1ShCQMI3/sNBa14wPfz09O1PwZIhTnA9mV+UF8P63w3XilQzmg+ C08vfwsx29iNrqNuM/GXQ1NFg5XYYg3J5OomrUEPbUJjlFDtHymqRXvXga5DC2f5wMdW uKTBCJgUqHPKm+Yo04QNwIY8wKksCmUW4iAAFm9rgqJNccnVwfvNKXLx6IdbRYLsbUxh kmCw== MIME-Version: 1.0 Received: by 10.43.62.133 with SMTP id xa5mr10327248icb.28.1355120157342; Sun, 09 Dec 2012 22:15:57 -0800 (PST) Received: by 10.50.88.137 with HTTP; Sun, 9 Dec 2012 22:15:57 -0800 (PST) Date: Mon, 10 Dec 2012 06:15:57 +0000 Message-ID: Subject: Re: FreeBSD for serious performance? From: "b.f." To: Dieter BSD , freebsd-hackers@FreeBSD.org Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: bf1783@gmail.com List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Dec 2012 06:15:58 -0000 >[ lack of SATA NCQ support for nforce4-ultra ] > > > http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=commitdiff;h=e2e031eb09760c36099ac127eeb175e06d257aef > >which is: > >The mcp61 has bug with ncq. >- { PCI_VDEVICE(NVIDIA, PCI_DEVICE_ID_NVIDIA_NFORCE_MCP61_SATA), SWNCQ }, >- { PCI_VDEVICE(NVIDIA, PCI_DEVICE_ID_NVIDIA_NFORCE_MCP61_SATA2), SWNCQ }, >- { PCI_VDEVICE(NVIDIA, PCI_DEVICE_ID_NVIDIA_NFORCE_MCP61_SATA3), SWNCQ }, >+ { PCI_VDEVICE(NVIDIA, PCI_DEVICE_ID_NVIDIA_NFORCE_MCP61_SATA), GENERIC }, >+ { PCI_VDEVICE(NVIDIA, PCI_DEVICE_ID_NVIDIA_NFORCE_MCP61_SATA2), GENERIC }, >+ { PCI_VDEVICE(NVIDIA, PCI_DEVICE_ID_NVIDIA_NFORCE_MCP61_SATA3), GENERIC }, > >http://en.wikipedia.org/wiki/GeForce_6_Series >says: "MCP61 introduced a bug in the SATA NCQ implementation. >As a result, Nvidia employees have contributed code to disable NCQ operations under Linux." >I have not found a description of the bug. > >But FreeBSD says I have: > > port 0x9f0-0x9f7,0xbf0-0xbf3,0x970-0x977,0xb70-0xb73,0xcc00-0xcc0f >mem 0xfebfb000-0xfebfbfff irq 21 at device 7.0 on pci0 > port 0x9e0-0x9e7,0xbe0-0xbe3,0x960-0x967,0xb60-0xb63,0xb800-0xb80f >mem 0xfebfa000-0xfebfafff irq 22 at device 8.0 on pci0 > >The CK804 (a few lines higher in the source file) did not get changed to GENERIC. >As far as I can figure out, the bug (whatever it is) is limited to the MCP61. Sure, I wasn't suggesting that the bug affected your particular case -- I was just mentioning the bug as an example that some of the older SATA controllers have problems with useful features like MSI, power management, port multiplication, or NCQ that the controllers are supposed to support -- so that if anyone were considering creating a FreeBSD driver for one of these controllers based on our existing FreeBSD drivers, the product datasheets, and the Linux drivers at: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=tree;f=drivers/ata;h=32325a5432d90c22dd1f19ee6c811af2145fc031;hb=HEAD in order to get more than legacy mode support for the controller, then they would be wise to look at the Linux drivers to see if the features that they want actually work -- and weigh the investment of time and effort against the alternative of buying a new storage card that works well with one of our newer drivers. b. From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 10 07:38:11 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9EBC7C7E for ; Mon, 10 Dec 2012 07:38:11 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: from mail-la0-f54.google.com (mail-la0-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id 160088FC12 for ; Mon, 10 Dec 2012 07:38:10 +0000 (UTC) Received: by mail-la0-f54.google.com with SMTP id j13so2194109lah.13 for ; Sun, 09 Dec 2012 23:38:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=yVRcBAa7XulQDYTZ2tUYOn/n1Z3hZNDwD6z7QtGkK8U=; b=rHDMNugta6MVtGeRfwVBeJafEUf+cuY8zaHk6kC3ity/0KolEzyWBedvVVZRLlGjCH jMh0Qb3VGpJJkwO3fUR/ysFri7wfjWFDSzQrP9b8VFPoOFPktRjhcFL6d9GaXbaxUDrQ 0V5EtyC2EdOjm3RTylTFBiIFrLMakcPW7R4mIzKCSAnbyYkoFUIyoRtW4KqI1E8rwUwh ILMNX71AD7tNpO39xKdS5k//m4SFOpjA1AlDFsl/KUPv0HBtlWM5Fp4973zkjeU3YLe4 bbr8afAiHTZ+DlLtCSSWdrdPr7CZXmxgaJJMf7vt4LuA7RaYTj33Q5fq2mClesOmW8lY 1hbQ== MIME-Version: 1.0 Received: by 10.152.148.129 with SMTP id ts1mr5710833lab.19.1355125083589; Sun, 09 Dec 2012 23:38:03 -0800 (PST) Received: by 10.112.61.33 with HTTP; Sun, 9 Dec 2012 23:38:03 -0800 (PST) In-Reply-To: References: Date: Mon, 10 Dec 2012 02:38:03 -0500 Message-ID: Subject: Re: using FreeBSD to create a completely new OS From: Zaphod Beeblebrox To: Aryeh Friedman Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Mailing List X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Dec 2012 07:38:11 -0000 On Sun, Dec 9, 2012 at 11:48 PM, Aryeh Friedman wrote: > For personal hobby reasons I want to write an OS completely from > scratch (due to some aspects of the design no existing OS is a > suitable starting place)... what I mean is I want to start with the > MBR (boot0) and go on from there... I only have one *REAL* machine to Well... most would say that the interesting part of an OS has very little to do with the loader, but who am I to judge: maybe a loader is interesting for you. There are quite a few things that can be loaded and useful from an MBR. Pretty much any machine will provide you with a boot environment that has lived longer than any other. It's pretty much DOS without a filesystem, tho. In freebsd, you want to look at /boot/mbr and maybe (later) /boot/loader (or one of it's variants). > work with which means I need to work with something like > emulators/virtualbox-ose... I also want to do as many automated tests > as possible (for example seeing if the installer copied the MBR [and > later other stuff] correctly to the virtual HDD).... for this reason I > have a few questions on vb (or perhaps QEMU if not possible in vb): > > 1. Can it be scripted? > 2. Is there any documentation on the various virtual HDD formats and > such (that way I can check the "physical" drive and not by indirect > query)? The format of an MBR (master boot record) is documented everywhere. I'm pretty sure wikipedia has a good article on it. This mailing list is not google, BTW. > Also can people give me some idea of a good general > development/testing framework.... the one I have in mind so far is: > > 1. Write enough of the OS in FreeBSD to boot and give a command prompt > and then develop using it (assume that there is a working compilor > [note I am doing it completely in Java (note 1) and will be using > either gjava (gcc) or writing my own compiler]) > > Notes: > > 1. Due to OS's needing to address physical RAM directly (DMA mapped > I/O) I will be introducing some form of ptr's into java with the > compiler or native My first thought was the humor of getting java to deal with segment registers. It's hard enough in C. But in all seriousness, most systems go from the firmware to something like a bootblock (often called a stage one boot) to a smarter bootloader (stage two boot) to activating the OS. In a PC-ish world for FreeBSD, that's BIOS->/boot/mbr->/boot/loader->kernel. Specifically locore.s (note: assembly language file). locore.s is one of a few regions of kernels that are generally _not_ written in high level languages. Other's include the very lowest level of the scheduler and the virtual memory manager. In locore's case, you can't express the special instructions (for a PC) to exit real mode and enter virtual mode (or some such transformation --- I really haven't been there in awhile). Note that FreeBSD boots on a number of things and that each boot process is a little bit different. If you're going to write an OS from scratch, you might start with something _way_ simpler than FreeBSD. Minix would be a good choice. I think it still comes as a book you buy with bootable code. If you haven't done OS work before, the book part might get you going faster than diving into the FreeBSD codebase. From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 10 07:56:39 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5855CF1E for ; Mon, 10 Dec 2012 07:56:39 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: from mail-oa0-f54.google.com (mail-oa0-f54.google.com [209.85.219.54]) by mx1.freebsd.org (Postfix) with ESMTP id 14EDB8FC14 for ; Mon, 10 Dec 2012 07:56:38 +0000 (UTC) Received: by mail-oa0-f54.google.com with SMTP id n9so2870565oag.13 for ; Sun, 09 Dec 2012 23:56:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=1o/0sqg5E/URvNlbRob/oSgIsFPPOm7rZ8dUiz2q7pQ=; b=OYnj3nGI1XUg2XYXJrvi/8OrJ9FuBfwtoGQ4GEPom5yniHxLCZEADyALa1Txu7LT0X Dq2XJTwHOFvRHgrFgr0x4apUfiNwqpru0MwOma1g+G6oWtyaM0BSxuK5Nq6FzUe23dTT vJXYLu67HyC4svVRt5g9jKA3wbbSMqMs0+9aniPkvDxATB7cEJpz1uZXxWtQMky8mum2 4TNR9LQhlq5dgcM4dOS9pqJe7yF8HEwwOR+qfb5Xf5PzO+1hchG1eeoFiKiBnFUQlZtR ZCj1O6aVPN+/LEZp3DV29XrXPd2zd3T/XqrqRWSyfOgDe54Hv+r2TEL2+14UH5IpjDiU nSdQ== MIME-Version: 1.0 Received: by 10.60.7.129 with SMTP id j1mr6953827oea.54.1355126198256; Sun, 09 Dec 2012 23:56:38 -0800 (PST) Received: by 10.182.158.137 with HTTP; Sun, 9 Dec 2012 23:56:38 -0800 (PST) In-Reply-To: References: Date: Mon, 10 Dec 2012 02:56:38 -0500 Message-ID: Subject: Re: using FreeBSD to create a completely new OS From: Aryeh Friedman To: Zaphod Beeblebrox Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Mailing List X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Dec 2012 07:56:39 -0000 On Mon, Dec 10, 2012 at 2:38 AM, Zaphod Beeblebrox wrote: > On Sun, Dec 9, 2012 at 11:48 PM, Aryeh Friedman > wrote: >> For personal hobby reasons I want to write an OS completely from >> scratch (due to some aspects of the design no existing OS is a >> suitable starting place)... what I mean is I want to start with the >> MBR (boot0) and go on from there... I only have one *REAL* machine to > > Well... most would say that the interesting part of an OS has very > little to do with the loader, but who am I to judge: maybe a loader is > interesting for you. There are quite a few things that can be loaded > and useful from an MBR. Pretty much any machine will provide you with > a boot environment that has lived longer than any other. It's pretty > much DOS without a filesystem, tho. In freebsd, you want to look at > /boot/mbr and maybe (later) /boot/loader (or one of it's variants). I was asking how to extract it from the virtual HDD made by vbox > >> work with which means I need to work with something like >> emulators/virtualbox-ose... I also want to do as many automated tests >> as possible (for example seeing if the installer copied the MBR [and >> later other stuff] correctly to the virtual HDD).... for this reason I >> have a few questions on vb (or perhaps QEMU if not possible in vb): >> >> 1. Can it be scripted? >> 2. Is there any documentation on the various virtual HDD formats and >> such (that way I can check the "physical" drive and not by indirect >> query)? > > The format of an MBR (master boot record) is documented everywhere. > I'm pretty sure wikipedia has a good article on it. This mailing list > is not google, BTW. I know (about the list not being google) and have seen the formats for MBR's (even wrote a few by hand) the question was how to extract it from the virtual HDD (see above).... namely is I have a prototype MBR that should be made I want to check if it was actually written to the virtual HDD correctly > >> Also can people give me some idea of a good general >> development/testing framework.... the one I have in mind so far is: >> >> 1. Write enough of the OS in FreeBSD to boot and give a command prompt >> and then develop using it (assume that there is a working compilor >> [note I am doing it completely in Java (note 1) and will be using >> either gjava (gcc) or writing my own compiler]) >> >> Notes: >> >> 1. Due to OS's needing to address physical RAM directly (DMA mapped >> I/O) I will be introducing some form of ptr's into java with the >> compiler or native > > My first thought was the humor of getting java to deal with segment > registers. It's hard enough in C. > > But in all seriousness, most systems go from the firmware to something > like a bootblock (often called a stage one boot) to a smarter > bootloader (stage two boot) to activating the OS. In a PC-ish world > for FreeBSD, that's BIOS->/boot/mbr->/boot/loader->kernel. > Specifically locore.s (note: assembly language file). locore.s is one > of a few regions of kernels that are generally _not_ written in high > level languages. Other's include the very lowest level of the > scheduler and the virtual memory manager. In locore's case, you can't > express the special instructions (for a PC) to exit real mode and > enter virtual mode (or some such transformation --- I really haven't > been there in awhile). Note that FreeBSD boots on a number of things > and that each boot process is a little bit different. The idea is not replicate *ANY* existing OS but write a completely new one (completely new architecture for an OS even). For that reason starting with the MBR is the right thing to do for reasons that would take several pages to explain. >From my initial research there is almost nothing that depends on actual ASM except for a very small number of things that create the environment needed to run the equiv of the JVM (no garbage collection at this level) natively on an x86. My guess the total amount of ASM needed is less then a thousand instructions (99% are macro's mostly for performance reasons). The other 1% is just to get the protected mode and the native JVM loaded and started. Note when I say JVM I actually mean a collection of VM's that model real HW (not some abstract HW like the sun JVM does) and if the VM just happens to be identical to the native HW then it skips the emulation part of the VM (i.e. direct execution on the actual HW). The first such VM I am planning is x86 (64-bit). Namely if your running on a PC then it runs natively but on anything else it runs on a VM identical to a PC. > > If you're going to write an OS from scratch, you might start with > something _way_ simpler than FreeBSD. Minix would be a good choice. > I think it still comes as a book you buy with bootable code. If you > haven't done OS work before, the book part might get you going faster > than diving into the FreeBSD codebase. I have done some trivial OS stuff before like getting to the point where I can type the equiv of ls and cd on a trivial virtual FS. From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 10 08:08:17 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D899817D for ; Mon, 10 Dec 2012 08:08:17 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 91AFA8FC08 for ; Mon, 10 Dec 2012 08:08:17 +0000 (UTC) Received: by mail-ob0-f182.google.com with SMTP id 16so2756916obc.13 for ; Mon, 10 Dec 2012 00:08:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=PFp51ibj9jIzp7j4BI5nwVBrXo00pjTZtEOierPU+iI=; b=nxr+ZtjkN+F0AfUjFqc+y5H/A6z8So6y/Gn/LVdg3z7e4XtF5x3DoD8RNv1MCsuNep ltX1Jb4v/vU8GzD5QjU/IrB8OvvB1Kv9zzvQT0HzTS9w3NdPH86OL5FefUavFoiGorno nkgTbserc/ZGxSjVRa0r0nknPvYBWJyo8+ldtJLV+qSjM06m9k8nINp6dxjdoOzyCmEU teNRb+K3GXNo/cu7J9IO0URbxCwfua6qd/EO6AG0Yzrxn6xWPA/66vVbZmMZPAh7eST1 mvSkLVA1QLBf8ibEYIBThLwJfHEdG94wrXihu/RVguHweeYib3RmgvPYYjO0rFUC2ct3 su7w== MIME-Version: 1.0 Received: by 10.60.30.42 with SMTP id p10mr7026492oeh.59.1355126891467; Mon, 10 Dec 2012 00:08:11 -0800 (PST) Received: by 10.182.158.137 with HTTP; Mon, 10 Dec 2012 00:08:11 -0800 (PST) In-Reply-To: References: Date: Mon, 10 Dec 2012 03:08:11 -0500 Message-ID: Subject: Re: using FreeBSD to create a completely new OS From: Aryeh Friedman To: Zaphod Beeblebrox Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Mailing List X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Dec 2012 08:08:17 -0000 Forgot to mention this project vary well turn into my PhD thesis From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 10 08:12:10 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 93CA03CC for ; Mon, 10 Dec 2012 08:12:10 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: from mail-lb0-f182.google.com (mail-lb0-f182.google.com [209.85.217.182]) by mx1.freebsd.org (Postfix) with ESMTP id 124658FC0C for ; Mon, 10 Dec 2012 08:12:09 +0000 (UTC) Received: by mail-lb0-f182.google.com with SMTP id go10so2297270lbb.13 for ; Mon, 10 Dec 2012 00:12:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=opYEARHgf4Z/Ya/s4ku2tAiAYCN8nlrnLaQnSJIZBaQ=; b=PEsqBk4J1Ljx2kuiGEfFl/lmSOZAyGWBowFaz0Cj+6gIYfK1aK8N8Zh8CcWeBsCzDe Tl0XeR4JYjGYbOfTta5yw4TnqYjj1mZNUE31FysudsnY/TSRi2PcwnHOo3bN5kQP8A2y +cny+on05P4eQ01Ogb/tQ//l8ntQyNlvqd+vyKJfgGd/aoVK064Xrty4e9oa7wE8eCP2 oKT39zBWjYzqKMv1LLULNQoAl6su5AdXsjW5MdZyvmwG12NFShO9k3VMyticwkUKXtFD TJero3mpURNTCxrz6sh5ytRzLQuKaBPPeEC8BE7o+oR7g2c/ki/zG5ziV4jfwJYFK75U +W+Q== MIME-Version: 1.0 Received: by 10.112.101.232 with SMTP id fj8mr5359029lbb.83.1355127128568; Mon, 10 Dec 2012 00:12:08 -0800 (PST) Received: by 10.112.61.33 with HTTP; Mon, 10 Dec 2012 00:12:08 -0800 (PST) In-Reply-To: References: Date: Mon, 10 Dec 2012 03:12:08 -0500 Message-ID: Subject: Re: using FreeBSD to create a completely new OS From: Zaphod Beeblebrox To: Aryeh Friedman Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Mailing List X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Dec 2012 08:12:10 -0000 On Mon, Dec 10, 2012 at 2:56 AM, Aryeh Friedman wrote: > On Mon, Dec 10, 2012 at 2:38 AM, Zaphod Beeblebrox wrote: > I know (about the list not being google) and have seen the formats for > MBR's (even wrote a few by hand) the question was how to extract it > from the virtual HDD (see above).... namely is I have a prototype MBR > that should be made I want to check if it was actually written to the > virtual HDD correctly The simplest thing I can think of is to attach said virtual disk to a FreeBSD guest and use dd. Sure ... I'm pretty sure the virtual disk you're using is documented, but follow the path of least resistance. > From my initial research there is almost nothing that depends on > actual ASM except for a very small number of things that create the > environment needed to run the equiv of the JVM (no garbage collection > at this level) natively on an x86. My guess the total amount of ASM > needed is less then a thousand instructions (99% are macro's mostly > for performance reasons). The other 1% is just to get the protected > mode and the native JVM loaded and started. If you mean swtch.s here, I suppose you're interpreting what I said rightly. > Note when I say JVM I actually mean a collection of VM's that model > real HW (not some abstract HW like the sun JVM does) and if the VM > just happens to be identical to the native HW then it skips the > emulation part of the VM (i.e. direct execution on the actual HW). > The first such VM I am planning is x86 (64-bit). Namely if your > running on a PC then it runs natively but on anything else it runs on > a VM identical to a PC. Note that this pretty much makes java pretty much worthless. I say that carefully as the seemingly "good" thing that makes the extra effort in java "worth it" is the guarantee against the halting problem that makes the security madness possible. If your "vm" is i386, you can't offer that guarantee. The JVM is very specially crafted for this purpose. From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 10 08:16:18 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3073D603 for ; Mon, 10 Dec 2012 08:16:18 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from mail.bitblocks.com (ns1.bitblocks.com [173.228.5.8]) by mx1.freebsd.org (Postfix) with ESMTP id 0BD2A8FC08 for ; Mon, 10 Dec 2012 08:16:17 +0000 (UTC) Received: from bitblocks.com (localhost [127.0.0.1]) by mail.bitblocks.com (Postfix) with ESMTP id 92F23B827; Mon, 10 Dec 2012 00:11:05 -0800 (PST) To: Aryeh Friedman Subject: Re: using FreeBSD to create a completely new OS In-reply-to: Your message of "Sun, 09 Dec 2012 23:48:12 EST." References: Comments: In-reply-to Aryeh Friedman message dated "Sun, 09 Dec 2012 23:48:12 -0500." Date: Mon, 10 Dec 2012 00:11:05 -0800 From: Bakul Shah Message-Id: <20121210081105.92F23B827@mail.bitblocks.com> Cc: FreeBSD Mailing List X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Dec 2012 08:16:18 -0000 On Sun, 09 Dec 2012 23:48:12 EST Aryeh Friedman wrote: > For personal hobby reasons I want to write an OS completely from > scratch (due to some aspects of the design no existing OS is a > suitable starting place)... what I mean is I want to start with the > MBR (boot0) and go on from there... I only have one *REAL* machine to > work with which means I need to work with something like > emulators/virtualbox-ose... I also want to do as many automated tests > as possible (for example seeing if the installer copied the MBR [and > later other stuff] correctly to the virtual HDD).... for this reason I > have a few questions on vb (or perhaps QEMU if not possible in vb): > > 1. Can it be scripted? > 2. Is there any documentation on the various virtual HDD formats and > such (that way I can check the "physical" drive and not by indirect > query)? > > Also can people give me some idea of a good general > development/testing framework.... the one I have in mind so far is: You may wish to check out http://wiki.osdev.org/Expanded_Main_Page http://wiki.osdev.org/Projects From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 10 08:21:04 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B45F7828 for ; Mon, 10 Dec 2012 08:21:04 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 6E8A48FC0C for ; Mon, 10 Dec 2012 08:21:04 +0000 (UTC) Received: by mail-ob0-f182.google.com with SMTP id 16so2766754obc.13 for ; Mon, 10 Dec 2012 00:21:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=nDwLO6Dex3SsauDxA5/jXq9qGfHGESrPidLYebU1/+c=; b=0PIUVpFPDvsGJJ7dXZyRbiYWIOfB97uKQUQRFAbrJnJss0lFz9FO5KMbNqnDaBSu31 ZNhMVQ+LtudE+0OocfpVduUlpqnL6CfOOomsatBx+fezkgNyL/ZwH6yfjoBKUX4e/gGl 5LWLXUCkuZk9ulTysgConM6+pNyFr4InQ4meumN2082oPi5AZJxcAEjYU+PKNqY5Hq4i kd8EVzd2TIzkGr2/FLQqW1GZDgFiH0Hfpss0IEknDfQ6OKf482ydbTKMqrncruAAbfdg pD0thEBWnNSD4jeMtl7uP6+hIZaGmRlixdLqeJwtnTY96zHqVSCHUG6FV3MKndpQr969 wCSQ== MIME-Version: 1.0 Received: by 10.182.31.33 with SMTP id x1mr2356941obh.29.1355127663881; Mon, 10 Dec 2012 00:21:03 -0800 (PST) Received: by 10.182.158.137 with HTTP; Mon, 10 Dec 2012 00:21:03 -0800 (PST) In-Reply-To: References: Date: Mon, 10 Dec 2012 03:21:03 -0500 Message-ID: Subject: Re: using FreeBSD to create a completely new OS From: Aryeh Friedman To: Zaphod Beeblebrox , FreeBSD Mailing List Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Dec 2012 08:21:04 -0000 The reason for Java has nothing to do with VM's per se but with some of the networking features I have in mind. Also from a theoretical point of view any turning complete vm would do (the actual HW differences are fairly easy to abstract in the model I have in mind and polymorphism is a must for various other reasons that would take far to long to explain so that means either C++ or Java... from a pure CS point of view C++ is the wrong choice (in my mind the only arg for it is it has ptr 's which in reality often are too easy to misuse anyways and that's the reason for wrapping them in Java like I described in the OP). From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 10 08:38:58 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 8AF49CBA for ; Mon, 10 Dec 2012 08:38:58 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: from mail-oa0-f54.google.com (mail-oa0-f54.google.com [209.85.219.54]) by mx1.freebsd.org (Postfix) with ESMTP id 3E0A08FC13 for ; Mon, 10 Dec 2012 08:38:57 +0000 (UTC) Received: by mail-oa0-f54.google.com with SMTP id n9so2903737oag.13 for ; Mon, 10 Dec 2012 00:38:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=mLcP71JHLXpQeerKheR9JyMJ4zQlaG7zNjNNkhIRlUk=; b=l9pgBYfjOHyJePz8WtDcSCcMotm9/U03Nysz55Ti1Jwxbp8an3eEYxf32RTucmj0H/ ho0QH+wIo2TJtraHQA+0gxtnjftzM44iyTW3/KLHEBmjZ/y7YtGtMq5QhTKoWFSI4NqF 6CQ5zrkTE/66iEb9teZsyvVbpiqlneCNkcceyahSuRNLpoPi4XjS6IG5r7C+XvP2tGFx PfIOpOKLBvVP4fSvoAcEGEIYP1fhlNO0ONqZXNhfGipnGfVN8HmQUVSVdYcoOxgARm5z U+YnOuZbnLnPvsV8bRuipFlU5Hn9pJ4B5z69O1mTymSmv+Gsl2djaH9KIeOTRi9EZLFj C2VQ== MIME-Version: 1.0 Received: by 10.182.64.14 with SMTP id k14mr7296755obs.72.1355128737667; Mon, 10 Dec 2012 00:38:57 -0800 (PST) Received: by 10.182.158.137 with HTTP; Mon, 10 Dec 2012 00:38:57 -0800 (PST) In-Reply-To: References: Date: Mon, 10 Dec 2012 03:38:57 -0500 Message-ID: Subject: Re: using FreeBSD to create a completely new OS From: Aryeh Friedman To: Zaphod Beeblebrox Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Mailing List X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Dec 2012 08:38:58 -0000 > Note that this pretty much makes java pretty much worthless. I say > that carefully as the seemingly "good" thing that makes the extra > effort in java "worth it" is the guarantee against the halting problem > that makes the security madness possible. If your "vm" is i386, you > can't offer that guarantee. The JVM is very specially crafted for > this purpose. On the halting problem unless my basic theory of computation is very out of date it is not possible for any turing complete machine to guerntee against the halting problem.... btw I think the AI crowd would love to hear that Java is a oracle (pun intended) which is likely more then any human can do (as far I know the current thinking is humans are somewhere between UTM's and oracles) From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 10 08:45:37 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2C2DE220 for ; Mon, 10 Dec 2012 08:45:37 +0000 (UTC) (envelope-from erichsfreebsdlist@alogt.com) Received: from alogt.com (alogt.com [69.36.191.58]) by mx1.freebsd.org (Postfix) with ESMTP id D76C58FC12 for ; Mon, 10 Dec 2012 08:45:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=alogt.com; s=default; h=Content-Transfer-Encoding:Content-Type:Mime-Version:References:In-Reply-To:Message-ID:Subject:Cc:To:From:Date; bh=aE6Ifr3aPnUNacXjajj7l0zS4BiYwM8ADsHsVb+Rl6g=; b=pvkdVyvkJWqcP1fNSY5r+ZDbjGmlKUodD607XxmqlPFtn/EM95MNMcs4/O0Tg0o2jiem3FcWUbbUSVxx6kJqVg5bY7L6AL3CupNXaiUY/9c2vq1FWpFIh2Lxydk6A944; Received: from [122.129.203.50] (port=17761 helo=X220.ovitrap.com) by sl-508-2.slc.westdc.net with esmtpsa (TLSv1:DHE-RSA-AES128-SHA:128) (Exim 4.80) (envelope-from ) id 1ThyzP-003KGv-Uk; Mon, 10 Dec 2012 01:45:29 -0700 Date: Mon, 10 Dec 2012 15:44:51 +0700 From: Erich Dollansky To: Aryeh Friedman Subject: Re: using FreeBSD to create a completely new OS Message-ID: <20121210154451.605d7bd1@X220.ovitrap.com> In-Reply-To: References: X-Mailer: Claws Mail 3.8.0 (GTK+ 2.24.6; amd64-portbld-freebsd10.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - sl-508-2.slc.westdc.net X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - alogt.com X-Get-Message-Sender-Via: sl-508-2.slc.westdc.net: authenticated_id: erichsfreebsdlist@alogt.com X-Source: X-Source-Args: X-Source-Dir: Cc: FreeBSD Mailing List , Zaphod Beeblebrox X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Dec 2012 08:45:37 -0000 Hi, On Mon, 10 Dec 2012 03:08:11 -0500 Aryeh Friedman wrote: > Forgot to mention this project vary well turn into my PhD thesis I would then extremely careful with Java. There are many reasons why assembler and C are still the languages used for OS programming. Erich > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to > "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 10 08:48:31 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id DB1BD3F5 for ; Mon, 10 Dec 2012 08:48:31 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: from mail-oa0-f54.google.com (mail-oa0-f54.google.com [209.85.219.54]) by mx1.freebsd.org (Postfix) with ESMTP id 937068FC0C for ; Mon, 10 Dec 2012 08:48:31 +0000 (UTC) Received: by mail-oa0-f54.google.com with SMTP id n9so2911969oag.13 for ; Mon, 10 Dec 2012 00:48:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=+z/rmLQoEgn2r7sYd1ZdrMPDoeAhX/kRqV8PumNQkyQ=; b=klMbrKkXtl3RmPy6FrVBXcLVnXj8aEiBT90TSzYxQk5tQFW79vmy2utsblGe+W72j5 188Xezl/WTq25WqRmcErHjBbPmDCFOicTmLOtOgF3D8yRqZqkdNbelRSNji6w/9MI08B +Sp45+c7DETWaqH58NXMCGngfV1Lskrv3aXYssHfqPrfdDSFGMzbMG2mcof/IMJABQsH ELDF8cE/c9UCsJE2LGvJnVeSEdqPAJmZAvGcxIvWqDMbUiZvUs6r/Zvvp0tB7vuEtc2U 2pGCmhqmTn7+nKybhmMuPwkHT+T76OrxZJdCpfz3Xc9/FuZFJundiMpf4QkNOetIcKmm VKow== MIME-Version: 1.0 Received: by 10.60.171.11 with SMTP id aq11mr7020818oec.104.1355129311164; Mon, 10 Dec 2012 00:48:31 -0800 (PST) Received: by 10.182.158.137 with HTTP; Mon, 10 Dec 2012 00:48:31 -0800 (PST) In-Reply-To: <20121210154451.605d7bd1@X220.ovitrap.com> References: <20121210154451.605d7bd1@X220.ovitrap.com> Date: Mon, 10 Dec 2012 03:48:31 -0500 Message-ID: Subject: Re: using FreeBSD to create a completely new OS From: Aryeh Friedman To: Erich Dollansky Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Mailing List , Zaphod Beeblebrox X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Dec 2012 08:48:31 -0000 On Mon, Dec 10, 2012 at 3:44 AM, Erich Dollansky wrote: > Hi, > > On Mon, 10 Dec 2012 03:08:11 -0500 > Aryeh Friedman wrote: > >> Forgot to mention this project vary well turn into my PhD thesis > > I would then extremely careful with Java. > > There are many reasons why assembler and C are still the languages used > for OS programming. Thats why I have spent almost 5 years doing pencil and paper designs to make sure it fits into the Java computational model but still has full HW access From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 10 09:11:38 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 52372B9D for ; Mon, 10 Dec 2012 09:11:38 +0000 (UTC) (envelope-from perryh@pluto.rain.com) Received: from agora.rdrop.com (agora.rdrop.com [IPv6:2607:f678:1010::34]) by mx1.freebsd.org (Postfix) with ESMTP id 2C92C8FC0C for ; Mon, 10 Dec 2012 09:11:38 +0000 (UTC) Received: from agora.rdrop.com (66@localhost [127.0.0.1]) by agora.rdrop.com (8.13.1/8.12.7) with ESMTP id qBA9BblW031909 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Mon, 10 Dec 2012 01:11:37 -0800 (PST) (envelope-from perryh@pluto.rain.com) Received: (from uucp@localhost) by agora.rdrop.com (8.13.1/8.14.2/Submit) with UUCP id qBA9BbNZ031908; Mon, 10 Dec 2012 01:11:37 -0800 (PST) (envelope-from perryh@pluto.rain.com) Received: by pluto.rain.com (4.1/SMI-4.1-pluto-M2060407) id AA17194; Mon, 10 Dec 12 01:02:22 PST Date: Mon, 10 Dec 12 01:02:22 PST From: perryh@pluto.rain.com (Perry Hutchison) Message-Id: <11212100902.AA17194@pluto.rain.com> To: aryeh.friedman@gmail.com Subject: Re: using FreeBSD to create a completely new OS Cc: freebsd-hackers@freebsd.org, zbeeble@gmail.com X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Dec 2012 09:11:38 -0000 > Forgot to mention this project vary well turn into my PhD thesis That's a rather different context than "personal hobby reasons". I was going to suggest this anyway, but for a dissertation project I would _very strongly_ suggest that you thoroughly research Linus Torvalds' experiences in implementing Linux, as well as any other low-level OS implementation projects that you can find in the literature (Multics, Unix, and OS/360 at least; Symbol 2R also comes to mind). Much of what those earlier implementors learned about "building an OS from scratch" will be directly applicable. (Granted Linus started with the SVID and you'll be writing your own specification, but the process of getting from {specification} to {code that actually works} is almost guaranteed to involve many of the same issues. PhD work is supposed to advance the state of the art, not rediscover the hard way what could have been learned in the library.) Perhaps equally important, I hope you're planning to figure out early on what "significant contribution to knowledge[1]" you will be making, and consider with your advisor whether it's necessary to go to all the work of building an entirely new OS to make that contribution. Again, you want to be standing on the shoulders of those who have gone before, not retracing their footsteps. [1] For those not familiar, that's the phrase typically used to describe the level of work that a PhD dissertation is supposed to report -- or at least it was when I was in graduate school. P.S. Just for general interest, what university? From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 10 09:15:19 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BC382D11 for ; Mon, 10 Dec 2012 09:15:19 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: from mail-oa0-f54.google.com (mail-oa0-f54.google.com [209.85.219.54]) by mx1.freebsd.org (Postfix) with ESMTP id 799038FC15 for ; Mon, 10 Dec 2012 09:15:19 +0000 (UTC) Received: by mail-oa0-f54.google.com with SMTP id n9so2934777oag.13 for ; Mon, 10 Dec 2012 01:15:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=XTzvBc5VY7swA4+EQTHpy2EQZOed61AOSllod1kUxE8=; b=en3mreE1toD8wEQEfl5o9QsVdw5JknM1ZUp2++TVrxsNFRFG0jtHm2F+aMD+oW4fJO js3FWbgYPUa/S+l/6vyHizM7cMz2QXPpgQ7cm7cDfTy5TMkSjIo9EpxXR+/xq56fXKiY rYxf+6ozPdjtjwk9GinD86+c+WdSPYqkSReQTsnOEeh1weApaAmPHz55CAVhxmbiYnH4 WoBYWoaLlT5NMsVmvQ7CAn7qpLJ7hfRMYACS1X6Sy3v9wJdeJU3KBMri+BxLgp4OquPf tn+SozmbiNW8StvlxISV+rOVbc+kKDI2vD/tVJvadznY0cnfjG0O0uOTyrIOlI3Z7AZ3 boHg== MIME-Version: 1.0 Received: by 10.60.4.35 with SMTP id h3mr2835152oeh.123.1355130918880; Mon, 10 Dec 2012 01:15:18 -0800 (PST) Received: by 10.182.158.137 with HTTP; Mon, 10 Dec 2012 01:15:18 -0800 (PST) In-Reply-To: <11212100902.AA17194@pluto.rain.com> References: <11212100902.AA17194@pluto.rain.com> Date: Mon, 10 Dec 2012 04:15:18 -0500 Message-ID: Subject: Re: using FreeBSD to create a completely new OS From: Aryeh Friedman To: Perry Hutchison Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org, zbeeble@gmail.com X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Dec 2012 09:15:19 -0000 > P.S. Just for general interest, what university? CUNY grad center (under a special arrangement that allows me to skip the course work part of grad school) [I have not formally started yet] As to your complexity issues I do not plan to do the whole OS just the stuff up to mid-level I/O (for FreeBSD this is also known as user land I/O) From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 10 09:31:28 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 884515CE for ; Mon, 10 Dec 2012 09:31:28 +0000 (UTC) (envelope-from bert@xs4all.net) Received: from smtp-vbr11.xs4all.nl (smtp-vbr11.xs4all.nl [194.109.24.31]) by mx1.freebsd.org (Postfix) with ESMTP id 18CBF8FC19 for ; Mon, 10 Dec 2012 09:31:27 +0000 (UTC) Received: from peyote.boppelans.net (original.xs4all.nl [82.95.229.134]) (authenticated bits=0) by smtp-vbr11.xs4all.nl (8.13.8/8.13.8) with ESMTP id qBA9Uorp042243 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 10 Dec 2012 10:30:50 +0100 (CET) (envelope-from bert@xs4all.net) Message-ID: <50C5ABE1.5030003@xs4all.net> Date: Mon, 10 Dec 2012 10:31:13 +0100 From: Bert Kiers Organization: XS4ALL Internet User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:17.0) Gecko/17.0 Thunderbird/17.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: using FreeBSD to create a completely new OS References: <11212100902.AA17194@pluto.rain.com> In-Reply-To: <11212100902.AA17194@pluto.rain.com> X-Enigmail-Version: 1.4.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Virus-Scanned: by XS4ALL Virus Scanner X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Dec 2012 09:31:28 -0000 On 12/10/12 10:02 AM, Perry Hutchison wrote: Hi, >> Forgot to mention this project vary well turn into my PhD thesis > > That's a rather different context than "personal hobby reasons". > > I was going to suggest this anyway, but for a dissertation project > I would _very strongly_ suggest that you thoroughly research Linus > Torvalds' experiences in implementing Linux, as well as any other > low-level OS implementation projects that you can find in the > literature (Multics, Unix, and OS/360 at least; Symbol 2R also > comes to mind). Much of what those earlier implementors learned Also Plan9 (http://plan9.bell-labs.com/plan9/) and Amoeba (http://en.wikipedia.org/wiki/Amoeba_distributed_operating_system) Grtnx, -- Bert Kiers XS4ALL UNIX systeembeheerder, suspected terrorist 1984 was not meant as a manual From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 10 10:50:48 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A07EB90E for ; Mon, 10 Dec 2012 10:50:48 +0000 (UTC) (envelope-from perryh@pluto.rain.com) Received: from agora.rdrop.com (agora.rdrop.com [IPv6:2607:f678:1010::34]) by mx1.freebsd.org (Postfix) with ESMTP id 7B0B68FC17 for ; Mon, 10 Dec 2012 10:50:48 +0000 (UTC) Received: from agora.rdrop.com (66@localhost [127.0.0.1]) by agora.rdrop.com (8.13.1/8.12.7) with ESMTP id qBAAolZD034906 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Mon, 10 Dec 2012 02:50:47 -0800 (PST) (envelope-from perryh@pluto.rain.com) Received: (from uucp@localhost) by agora.rdrop.com (8.13.1/8.14.2/Submit) with UUCP id qBAAol85034905; Mon, 10 Dec 2012 02:50:47 -0800 (PST) (envelope-from perryh@pluto.rain.com) Received: by pluto.rain.com (4.1/SMI-4.1-pluto-M2060407) id AA17688; Mon, 10 Dec 12 02:48:59 PST Date: Mon, 10 Dec 12 02:48:59 PST From: perryh@pluto.rain.com (Perry Hutchison) Message-Id: <11212101048.AA17688@pluto.rain.com> To: aryeh.friedman@gmail.com Subject: Re: using FreeBSD to create a completely new OS Cc: freebsd-hackers@freebsd.org, zbeeble@gmail.com X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Dec 2012 10:50:48 -0000 > ... a special arrangement that allows me to skip > the course work part of grad school ... [shudder] I hope that "special arrangement" includes passing the final exams, or otherwise demonstrating that you already know the content, of at least the minimum course work that would ordinarily be required :) > [I have not formally started yet] So you don't yet have a formally-assigned advisor :( I hope you at least have had some substantial (although necessarily informal) discussions with the department head, and/or the professor whom you anticipate will be your advisor, before doing a lot of work that might turn out not to be useful in your pursuit. > I do not plan to do the whole OS just the stuff up to mid-level I/O > (for FreeBSD this is also known as user land I/O) While the userland is by far the _largest_ part, the low-level kernel is likely to be the _trickiest_ part to get right ;) Another OS for your research list: Mach (which had been around for a while before being adopted as the basis of Darwin, the MacOS X kernel). Darwin is highly modular; depending on what you're doing, you may be able to write less code from scratch by reusing or adapting some parts of it. From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 10 11:24:25 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id EFAC9DD4 for ; Mon, 10 Dec 2012 11:24:25 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-wi0-f170.google.com (mail-wi0-f170.google.com [209.85.212.170]) by mx1.freebsd.org (Postfix) with ESMTP id 67BDE8FC15 for ; Mon, 10 Dec 2012 11:24:24 +0000 (UTC) Received: by mail-wi0-f170.google.com with SMTP id hq7so1066839wib.1 for ; Mon, 10 Dec 2012 03:24:23 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=2b7MyPy1b7/Y+RLQ1tLw/MJckjQ5pDyLPvI7tfaZo8Y=; b=jzJCFa3Qso82VRsmREHn55+qyWGvdB3HU0QpTgS9XmlaG7etSABo/B9mzQLEisqFtZ YpPohMmRxV8qlfX4jp5PCyij6nZzDGilMBpwZzf+vt37xrhxO/DtXETxXh5bTqW7gLLk 3vwVflTzkjZ57wZb6se+xJlvSA/VtlBlh6WKs5uzGesnG5TiEhMZlA4J5uYGf3xnDtup hy9go6F1bZ/gW0VeL+heLjp2eOVq9wLO7xCamaXgq7NrwDye3cZCYESFzOdVSNkSH16h kKc9dSt9FgjRwijDiMxnfSN10hXoBZBLKCkdAypvFq6T6tHsSA668aAvgOOaCNkJM5xL b1uA== Received: by 10.216.209.130 with SMTP id s2mr5436789weo.86.1355138663668; Mon, 10 Dec 2012 03:24:23 -0800 (PST) Received: from dfleuriot-at-hi-media.com ([83.167.62.196]) by mx.google.com with ESMTPS id h19sm9624152wiv.7.2012.12.10.03.24.22 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 10 Dec 2012 03:24:22 -0800 (PST) Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: kernel module parallel build? From: Fleuriot Damien In-Reply-To: Date: Mon, 10 Dec 2012 12:24:21 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <6C48F39A-83D9-4EA0-A7C5-99484FF00D3F@my.gd> References: <5083D84E.50903@freebsd.org> <201212041052.51476.jhb@freebsd.org> <201212051142.18361.jhb@freebsd.org> <0EC857C9-4C1B-467D-8499-B493401B64BC@bsdimp.com> <20E25F79-2C53-45FD-BB7F-060AC9B26245@my.gd> To: Garrett Cooper X-Mailer: Apple Mail (2.1499) X-Gm-Message-State: ALoCoQmIs1sdOZdRgVg/IBS143CSB7Wx1OsdLtxCHVdJZYW+tNaaByhtykHbd5xyggTPgHXz0px9 Cc: "freebsd-hackers@freebsd.org" , AndreOppermann , FreeBSD Current , Ryan Stone X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Dec 2012 11:24:26 -0000 On Dec 6, 2012, at 1:28 AM, Garrett Cooper wrote: > On Wed, Dec 5, 2012 at 3:51 PM, Damien Fleuriot wrote: >=20 > ... >=20 >> All trolling aside, I believe an awesome fix to be setting module = override in /etc/make.conf to only build the 4-5 specific modules one = needs. >>=20 >> To be honest I think this configuration tweak should be advertised a = bit more as it definitely speeds up kernel builds. >>=20 >> I would be happy to check if this is advertised in the handbook in = the "rebuilding kernel" section and enhance its visibility if required. >>=20 >> I can provide en_US and fr_FR. >=20 > +1. Please write it up if you can; it's much quicker/better than the > kitchen sink approach if you know what you're doing and don't have to > build for a large set of platforms. > Thanks! > -Garrett I've had a look at the handbook and the MODULES_OVERRIDE bit is already = well written and definitely visible: = http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig-bui= lding.html I believe no rewrite is necessary. However, I fail to see any mention of nextboot. It might be a good idea to let people know they can boot their new = kernel just once to see if it works, and revert to their old kernel if = boot should fail. From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 10 14:27:44 2012 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B82D925F; Mon, 10 Dec 2012 14:27:44 +0000 (UTC) (envelope-from jh@FreeBSD.org) Received: from gw02.mail.saunalahti.fi (gw02.mail.saunalahti.fi [195.197.172.116]) by mx1.freebsd.org (Postfix) with ESMTP id 685218FC13; Mon, 10 Dec 2012 14:27:44 +0000 (UTC) Received: from jh (a91-153-115-208.elisa-laajakaista.fi [91.153.115.208]) by gw02.mail.saunalahti.fi (Postfix) with SMTP id 8A5E913951C; Mon, 10 Dec 2012 16:27:39 +0200 (EET) Date: Mon, 10 Dec 2012 16:27:39 +0200 From: Jaakko Heinonen To: Greg 'groggy' Lehey Subject: Re: calendar(1) regressions Message-ID: <20121210142739.GA38913@jh> References: <20121206092202.GA2182@a91-153-116-96.elisa-laajakaista.fi> <20121209221525.GC36593@eureka.lemis.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20121209221525.GC36593@eureka.lemis.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-hackers@FreeBSD.org, edwin@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Dec 2012 14:27:44 -0000 Hi! On 2012-12-10, Greg 'groggy' Lehey wrote: > > Unfortunately r205821 [1] has caused several regressions to calendar(1). > > Relevant PRs: > > > > bin/157718 > > bin/162211 > > bin/168785 > > bin/170930 > > I think we fix bugs rather than revert the commits. Of course it's preferred but I didn't see any progress happening. As a result some of the basic functionality is broken (especially in stable/8). Maybe I should have debugged and fixed those bugs but I wasn't really keen to debug a large commit adding mostly features which I don't care much. > I've had a chat with edwin@, and I'll look at the issues Real Soon > Now. This is great news! Thanks. -- Jaakko From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 10 15:34:26 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A00B8624 for ; Mon, 10 Dec 2012 15:34:26 +0000 (UTC) (envelope-from yerenkow@gmail.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 5C96A8FC1C for ; Mon, 10 Dec 2012 15:34:25 +0000 (UTC) Received: by mail-ob0-f182.google.com with SMTP id 16so3215801obc.13 for ; Mon, 10 Dec 2012 07:34:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=649bbkX5Fw4Bb3AVmqs4Nyo+RjPIbqo4fuaL6eu4d7E=; b=vxf/1XBFZjXU+49MkZ/ss6Sy1yTyfXrmjwAZYYp2upkIkP0HhT6UhkKNJBhv/f+YHA iZclEL8jNjbai6RiUNVZTcpuBUnYlz5UyV4JxwIcqq9gYwPlQDTslh7GmYcHn7kX/MfG w94U4A3qkNOehjP+NZGJs6j6xWMzsC95sJTJA4rIWNv2sb5Y99eCICqrXsmCxaZTUWHJ CjyKSTpPfKu4yTL9f5A0tJ/ZK5f6ehKljUkFI2spk/IMHMLqroppm64FwjoojecT6s3E cb5ZlYcCyBJRsqR9A6fXX+WodGle6M/bt+1Rqg9pJtUsmLyt+yIxnA4Y7dHnegFu6F68 imnw== MIME-Version: 1.0 Received: by 10.182.43.104 with SMTP id v8mr7507149obl.98.1355153665348; Mon, 10 Dec 2012 07:34:25 -0800 (PST) Received: by 10.60.170.167 with HTTP; Mon, 10 Dec 2012 07:34:25 -0800 (PST) In-Reply-To: <11212101048.AA17688@pluto.rain.com> References: <11212101048.AA17688@pluto.rain.com> Date: Mon, 10 Dec 2012 17:34:25 +0200 Message-ID: Subject: Re: using FreeBSD to create a completely new OS From: Alexander Yerenkow To: Perry Hutchison Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-hackers@freebsd.org, zbeeble@gmail.com, aryeh.friedman@gmail.com X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Dec 2012 15:34:26 -0000 If you consider VirtualBox for automated tests, you should make some testing life cycle. I'd recommend to you: 1. create sparse file (truncate -s 1G for example) 2. mdconfig -af file (produced md0) 3. work with md0 as if it was real hw disk (write boot sectors, fs, etc), then destroy md0 (mdconfig -du 0) 4. create vmdk for file ( https://github.com/yerenkow/freebsd-vm-image/blob/master/freebsd-firmware/create-scsi-vmdk.sh -- this helps me create vmdk for raw file and feed them to VmWare/Virtualbox just fine) 5. boot VirtualBox using cli (and run some tests). I had some experience in running in-virtualbox-os tests a while ago, but there was real OS there, and not test one. Hope this helps. -- Regards, Alexander Yerenkow From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 10 18:07:35 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 20D6BCC1 for ; Mon, 10 Dec 2012 18:07:35 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [89.206.35.99]) by mx1.freebsd.org (Postfix) with ESMTP id 6CB8B8FC0C for ; Mon, 10 Dec 2012 18:07:33 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5) with ESMTP id qBAI4CCX013457; Mon, 10 Dec 2012 19:04:12 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5/Submit) with ESMTP id qBAI4BSO013454; Mon, 10 Dec 2012 19:04:12 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Mon, 10 Dec 2012 19:04:11 +0100 (CET) From: Wojciech Puchar To: Aryeh Friedman Subject: Re: using FreeBSD to create a completely new OS In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Mon, 10 Dec 2012 19:04:12 +0100 (CET) Cc: FreeBSD Mailing List X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Dec 2012 18:07:35 -0000 > suitable starting place)... what I mean is I want to start with the > MBR (boot0) and go on from there... I only have one *REAL* machine to > work with which means I need to work with something like > emulators/virtualbox-ose... I also want to do as many automated tests > as possible (for example seeing if the installer copied the MBR [and > later other stuff] correctly to the virtual HDD).... for this reason I > have a few questions on vb (or perhaps QEMU if not possible in vb): > > 1. Can it be scripted? for such usage i would recommend qemu. Virtualbox is for production use with common OSes, and is very ineffective in batch usage and inflexible, but fast. qemu is slow but you can have anything. including precise definition of your virtual hardware > such (that way I can check the "physical" drive and not by indirect > query)? just use "raw" format. From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 10 18:30:13 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1F8EA500 for ; Mon, 10 Dec 2012 18:30:13 +0000 (UTC) (envelope-from rsimmons0@gmail.com) Received: from mail-vc0-f182.google.com (mail-vc0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id CCA318FC17 for ; Mon, 10 Dec 2012 18:30:11 +0000 (UTC) Received: by mail-vc0-f182.google.com with SMTP id fo14so3646178vcb.13 for ; Mon, 10 Dec 2012 10:30:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=vJlIc+NJDCXz1fG8oW9Av2h/4HPrAo4m+sVCJ5rjnHA=; b=RBIOKoF6jntqd/xrTOIIvGx1bmy2TagTWWqgU5LKQoUlLkMnZyv5bWXzeQ2AZbBvc9 EsNNVU1Iy+2pfDvjimVeUHV6laMj8SOo+87qTFHg7eqEdY+WmHRkuvMOY7E1pJkAs02H tybar5ZSMGxtRv68PpOvLvNOaBGtBLgBw67PwVkxf+zEmG6vDHOZpsJvLmDE/z8rZJAv pHfdwSIOlT1FFoLYOdnaZwZ+FQo5s/JpC37JHwRUeYj2FNC1iYul/Tw5XFOr76e0C2Mg Wkev8LaCxr38Vo7r/v6ZstVHntbwJ1crXMjx9m5Nb9mSjJXAkDG5gihFgvTtA38obdrG n7ug== MIME-Version: 1.0 Received: by 10.220.115.19 with SMTP id g19mr9455014vcq.69.1355164211294; Mon, 10 Dec 2012 10:30:11 -0800 (PST) Received: by 10.52.26.48 with HTTP; Mon, 10 Dec 2012 10:30:11 -0800 (PST) Date: Mon, 10 Dec 2012 13:30:11 -0500 Message-ID: Subject: Building v8 on FreeBSD with clang From: Robert Simmons To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Dec 2012 18:30:13 -0000 I'm having a problem building google v8 on FreeBSD with clang. I get this error: /usr/bin/ld: final link failed: Nonrepresentable section on output clang++: error: linker command failed with exit code 1 (use -v to see invocation) gmake[1]: *** [/root/v8/out/x64.release/cctest] Error 1 gmake[1]: Leaving directory `/root/v8/out' gmake: *** [x64.release] Error 2 I needed to use gmake rather than make, and I added the following to use clang: setenv CC /usr/bin/clang setenv CXX /usr/bin/clang++ setenv GYP_DEFINES "clang=1" Some version info: # clang -v FreeBSD clang version 3.1 (branches/release_31 156863) 20120523 Target: x86_64-unknown-freebsd9.0 Thread model: posix # gmake -v GNU Make 3.82 Built for amd64-portbld-freebsd9.1 command used to checkout v8 code: # svn checkout http://v8.googlecode.com/svn/trunk/ v8 # uname -a FreeBSD thug 9.1-RC3 FreeBSD 9.1-RC3 #0 r242324: Tue Oct 30 00:58:57 UTC 2012 root@farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 10 23:05:01 2012 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2D5637A5; Mon, 10 Dec 2012 23:05:01 +0000 (UTC) (envelope-from grog@lemis.com) Received: from w3.lemis.com (w3.lemis.com [208.86.224.149]) by mx1.freebsd.org (Postfix) with ESMTP id E95FC8FC08; Mon, 10 Dec 2012 23:05:00 +0000 (UTC) Received: from eureka.lemis.com (1032.x.rootbsd.net [208.86.224.149]) by w3.lemis.com (Postfix) with ESMTP id 991CB3B767; Mon, 10 Dec 2012 23:04:53 +0000 (UTC) Received: by eureka.lemis.com (Postfix, from userid 1004) id 29EE2F794F; Tue, 11 Dec 2012 10:04:52 +1100 (EST) Date: Tue, 11 Dec 2012 10:04:52 +1100 From: Greg 'groggy' Lehey To: Jaakko Heinonen Subject: Re: calendar(1) regressions Message-ID: <20121210230452.GE36593@eureka.lemis.com> References: <20121206092202.GA2182@a91-153-116-96.elisa-laajakaista.fi> <20121209221525.GC36593@eureka.lemis.com> <20121210142739.GA38913@jh> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="cHMo6Wbp1wrKhbfi" Content-Disposition: inline In-Reply-To: <20121210142739.GA38913@jh> User-Agent: Mutt/1.4.2.3i Organization: The FreeBSD Project Phone: +61-3-5346-1370 Mobile: +61-418-838-708 WWW-Home-Page: http://www.FreeBSD.org/ X-PGP-Fingerprint: 9A1B 8202 BCCE B846 F92F 09AC 22E6 F290 507A 4223 Cc: freebsd-hackers@FreeBSD.org, edwin@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Dec 2012 23:05:01 -0000 --cHMo6Wbp1wrKhbfi Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Monday, 10 December 2012 at 16:27:39 +0200, Jaakko Heinonen wrote: > > Hi! > > On 2012-12-10, Greg 'groggy' Lehey wrote: >>> Unfortunately r205821 [1] has caused several regressions to calendar(1). >>> Relevant PRs: >>> >>> bin/157718 >>> bin/162211 >>> bin/168785 >>> bin/170930 >> >> I think we fix bugs rather than revert the commits. > > Of course it's preferred but I didn't see any progress happening. As a > result some of the basic functionality is broken (especially in > stable/8). Maybe I should have debugged and fixed those bugs but I > wasn't really keen to debug a large commit adding mostly features which > I don't care much. IIRC the main issue in 8 was that debug output was appearing as a matter of course. It would be relatively trivial to MF9 (i.e. copy the 9-STABLE version), but of course that would only solve part of the problem. Do you want an MFC to 8 when I'm done? Greg -- Sent from my desktop computer. Finger grog@FreeBSD.org for PGP public key. See complete headers for address and phone numbers. This message is digitally signed. If your Microsoft MUA reports problems, please read http://tinyurl.com/broken-mua --cHMo6Wbp1wrKhbfi Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlDGapMACgkQIubykFB6QiNraQCeNvQvqeK2NKpsw+KnkMdr/DiM QjkAnjNwojbVIdHlPOy0kgx0pDNUxOw1 =KIbV -----END PGP SIGNATURE----- --cHMo6Wbp1wrKhbfi-- From owner-freebsd-hackers@FreeBSD.ORG Tue Dec 11 00:11:44 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 81950353; Tue, 11 Dec 2012 00:11:44 +0000 (UTC) (envelope-from rfg@tristatelogic.com) Received: from outgoing.tristatelogic.com (segfault.tristatelogic.com [69.62.255.118]) by mx1.freebsd.org (Postfix) with ESMTP id 5A9BA8FC1C; Tue, 11 Dec 2012 00:11:44 +0000 (UTC) Received: from segfault-nmh-helo.tristatelogic.com (localhost [127.0.0.1]) by segfault.tristatelogic.com (Postfix) with ESMTP id 01CDE5081D; Mon, 10 Dec 2012 16:11:44 -0800 (PST) To: freebsd-hackers@freebsd.org, Nathan Whitehorn Subject: Re: 9.x -- New Install -- serious partition misalignment Date: Mon, 10 Dec 2012 16:11:44 -0800 Message-ID: <2474.1355184704@tristatelogic.com> From: "Ronald F. Guilmette" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Dec 2012 00:11:44 -0000 Apparently, I owe everyone an apology. I found out (a bit too late) that the WD 1 TB "black" drive that had come to me in sealed anti-static bag with a warning on it about "Advanced Format" drives (and possible OS incompatability) was not in fact itself an AF drive. Rather it is a traditional drive with 512 byte sectors. I tried doing a test install of FreeBSD 9.1-RC3 onto a different drive... one that I am quite sure had the new AF 4KB physical blocks and sure enough, all partitions were properly and automagically aligned on 4KB boundaries. I have put in a request to cancel the PR that I had filed regarding this issue. My thanks for Nathan Whitehorn for his input and forbearance. From owner-freebsd-hackers@FreeBSD.ORG Tue Dec 11 02:52:51 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 40F12C1B; Tue, 11 Dec 2012 02:52:51 +0000 (UTC) (envelope-from jhs@berklix.com) Received: from slim.berklix.org (slim.berklix.org [94.185.90.68]) by mx1.freebsd.org (Postfix) with ESMTP id AF95C8FC15; Tue, 11 Dec 2012 02:52:49 +0000 (UTC) Received: from mart.js.berklix.net (pD9FBFB06.dip.t-dialin.net [217.251.251.6]) (authenticated bits=128) by slim.berklix.org (8.14.5/8.14.5) with ESMTP id qBB2qfT7061374; Tue, 11 Dec 2012 03:52:42 +0100 (CET) (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (fire.js.berklix.net [192.168.91.41]) by mart.js.berklix.net (8.14.3/8.14.3) with ESMTP id qBB2qen9019919; Tue, 11 Dec 2012 03:52:41 +0100 (CET) (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (localhost [127.0.0.1]) by fire.js.berklix.net (8.14.4/8.14.4) with ESMTP id qBB2qXZS041825; Tue, 11 Dec 2012 03:52:39 +0100 (CET) (envelope-from jhs@fire.js.berklix.net) Message-Id: <201212110252.qBB2qXZS041825@fire.js.berklix.net> To: "Greg 'groggy' Lehey" Subject: Re: calendar(1) regressions From: "Julian H. Stacey" Organization: http://berklix.com BSD Unix Linux Consultancy, Munich Germany User-agent: EXMH on FreeBSD http://berklix.com/free/ X-URL: http://www.berklix.com In-reply-to: Your message "Tue, 11 Dec 2012 10:04:52 +1100." <20121210230452.GE36593@eureka.lemis.com> Date: Tue, 11 Dec 2012 03:52:32 +0100 Sender: jhs@berklix.com Cc: Jaakko Heinonen , edwin@freebsd.org, freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Dec 2012 02:52:51 -0000 "Greg 'groggy' Lehey" wrote: > > --cHMo6Wbp1wrKhbfi > Content-Type: text/plain; charset=us-ascii > Content-Disposition: inline > > On Monday, 10 December 2012 at 16:27:39 +0200, Jaakko Heinonen wrote: > > > > Hi! > > > > On 2012-12-10, Greg 'groggy' Lehey wrote: > >>> Unfortunately r205821 [1] has caused several regressions to calendar(1). > >>> Relevant PRs: > >>> > >>> bin/157718 > >>> bin/162211 > >>> bin/168785 > >>> bin/170930 > >> > >> I think we fix bugs rather than revert the commits. > > > > Of course it's preferred but I didn't see any progress happening. As a > > result some of the basic functionality is broken (especially in > > stable/8). Maybe I should have debugged and fixed those bugs but I > > wasn't really keen to debug a large commit adding mostly features which > > I don't care much. > > IIRC the main issue in 8 was that debug output was appearing as a > matter of course. It would be relatively trivial to MF9 (i.e. copy > the 9-STABLE version), but of course that would only solve part of the > problem. Do you want an MFC to 8 when I'm done? > > Greg > -- > Sent from my desktop computer. > Finger grog@FreeBSD.org for PGP public key. > See complete headers for address and phone numbers. > This message is digitally signed. If your Microsoft MUA reports > problems, please read http://tinyurl.com/broken-mua > Hi Greg I too filed send-prs re. rgeression bugs in recent broken calendar, hopefully you can look/ fix please. http://www.berklix.com/~jhs/src/bsd/fixes/FreeBSD/src/gen/usr.bin/calendar/REAME.JHS bin/15182 http://www.berklix.com/~jhs/src/bsd/fixes/FreeBSD/src/gen/usr.bin/calendar/send-pr.impossible_date.no_customise bin/157718 http://www.berklix.com/~jhs/src/bsd/fixes/FreeBSD/src/gen/usr.bin/calendar/core_dump_shell.no_customise bin/162211 http://www.berklix.com/~jhs/src/bsd/fixes/FreeBSD/src/gen/usr.bin/calendar/send-pr.first_last_in_month.no_customise -------------- PS for Anyone, doesnt need to be Greg misc/157748 This PR is NOT a regression coding error, but corrections & additions for a lot of badly sorted & incorrect src/ data in wrong calendar files etc. It doesn't need a programmer to fix it, any commiter could make corrections & also someone with a sense of orthogonality could shuffle data around, as some current calendar data in src/ is in wrong files, & also in some cases date is wrong. http://www.berklix.com/~jhs/src/bsd/fixes/FreeBSD/src/gen/usr.bin/calendar/calendars/calendar.holiday.send-pr http://www.berklix.com/~jhs/src/bsd/fixes/FreeBSD/src/gen/usr.bin/calendar/calendars/ Cheers, Julian -- Julian Stacey, BSD Unix Linux C Sys Eng Consultant, Munich http://berklix.com Reply below not above, like a play script. Indent old text with "> ". Send plain text. Not: HTML, multipart/alternative, base64, quoted-printable. From owner-freebsd-hackers@FreeBSD.ORG Tue Dec 11 03:25:07 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 673C4251; Tue, 11 Dec 2012 03:25:07 +0000 (UTC) (envelope-from grog@lemis.com) Received: from w3.lemis.com (w3.lemis.com [208.86.224.149]) by mx1.freebsd.org (Postfix) with ESMTP id 2E88A8FC12; Tue, 11 Dec 2012 03:25:06 +0000 (UTC) Received: from eureka.lemis.com (1032.x.rootbsd.net [208.86.224.149]) by w3.lemis.com (Postfix) with ESMTP id 6B7A33B752; Tue, 11 Dec 2012 03:25:05 +0000 (UTC) Received: by eureka.lemis.com (Postfix, from userid 1004) id 27DE2F794F; Tue, 11 Dec 2012 14:25:04 +1100 (EST) Date: Tue, 11 Dec 2012 14:25:04 +1100 From: Greg 'groggy' Lehey To: "Julian H. Stacey" Subject: Re: calendar(1) regressions Message-ID: <20121211032504.GF36593@eureka.lemis.com> References: <20121210230452.GE36593@eureka.lemis.com> <201212110252.qBB2qXZS041825@fire.js.berklix.net> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ryJZkp9/svQ58syV" Content-Disposition: inline In-Reply-To: <201212110252.qBB2qXZS041825@fire.js.berklix.net> User-Agent: Mutt/1.4.2.3i Organization: The FreeBSD Project Phone: +61-3-5346-1370 Mobile: +61-418-838-708 WWW-Home-Page: http://www.FreeBSD.org/ X-PGP-Fingerprint: 9A1B 8202 BCCE B846 F92F 09AC 22E6 F290 507A 4223 Cc: Jaakko Heinonen , edwin@freebsd.org, freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Dec 2012 03:25:07 -0000 --ryJZkp9/svQ58syV Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Tuesday, 11 December 2012 at 3:52:32 +0100, Julian H. Stacey wrote: > "Greg 'groggy' Lehey" wrote: >> On Monday, 10 December 2012 at 16:27:39 +0200, Jaakko Heinonen wrote: >>> >>> Hi! >>> >>> On 2012-12-10, Greg 'groggy' Lehey wrote: >>>>> Unfortunately r205821 [1] has caused several regressions to calendar(1). >>>>> Relevant PRs: >>>>> >>>>> bin/157718 >>>>> bin/162211 >>>>> bin/168785 >>>>> bin/170930 > > I too filed send-prs re. rgeression bugs in recent broken calendar, > hopefully you can look/ fix please. > > bin/157718 > bin/162211 These two are mentioned above. > bin/15182 > http://www.berklix.com/~jhs/src/bsd/fixes/FreeBSD/src/gen/usr.bin/calendar/send-pr.impossible_date.no_customise This one was closed a little over 11 years ago. Since then, the code base has changed completely, but it seems that another bug (one that I'm currently scratching my head about) has overtaken it. I'll include your example in my test cases. > PS for Anyone, doesnt need to be Greg > > misc/157748 OK, I'll take a look when I'm done here. Greg -- Sent from my desktop computer. Finger grog@FreeBSD.org for PGP public key. See complete headers for address and phone numbers. This message is digitally signed. If your Microsoft MUA reports problems, please read http://tinyurl.com/broken-mua --ryJZkp9/svQ58syV Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlDGp48ACgkQIubykFB6QiOLZACfauc8cM4do/r7GQh03DtP3w3L P2cAoKv/uuzhRGagU9jeO+AXTLSqCG7e =TbDk -----END PGP SIGNATURE----- --ryJZkp9/svQ58syV-- From owner-freebsd-hackers@FreeBSD.ORG Tue Dec 11 03:35:13 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B8B0D4FC; Tue, 11 Dec 2012 03:35:13 +0000 (UTC) (envelope-from jhs@berklix.com) Received: from slim.berklix.org (slim.berklix.org [94.185.90.68]) by mx1.freebsd.org (Postfix) with ESMTP id 302AD8FC18; Tue, 11 Dec 2012 03:35:12 +0000 (UTC) Received: from mart.js.berklix.net (pD9FBFB06.dip.t-dialin.net [217.251.251.6]) (authenticated bits=128) by slim.berklix.org (8.14.5/8.14.5) with ESMTP id qBB3ZBD4048186; Tue, 11 Dec 2012 04:35:11 +0100 (CET) (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (fire.js.berklix.net [192.168.91.41]) by mart.js.berklix.net (8.14.3/8.14.3) with ESMTP id qBB3ZBJr020164; Tue, 11 Dec 2012 04:35:11 +0100 (CET) (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (localhost [127.0.0.1]) by fire.js.berklix.net (8.14.4/8.14.4) with ESMTP id qBB3Z58j042602; Tue, 11 Dec 2012 04:35:11 +0100 (CET) (envelope-from jhs@fire.js.berklix.net) Message-Id: <201212110335.qBB3Z58j042602@fire.js.berklix.net> To: "Greg 'groggy' Lehey" Subject: Re: calendar(1) regressions From: "Julian H. Stacey" Organization: http://berklix.com BSD Unix Linux Consultancy, Munich Germany User-agent: EXMH on FreeBSD http://berklix.com/free/ X-URL: http://www.berklix.com In-reply-to: Your message "Tue, 11 Dec 2012 14:25:04 +1100." <20121211032504.GF36593@eureka.lemis.com> Date: Tue, 11 Dec 2012 04:35:05 +0100 Sender: jhs@berklix.com Cc: Jaakko Heinonen , edwin@freebsd.org, freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Dec 2012 03:35:13 -0000 "Greg 'groggy' Lehey" wrote: > > --ryJZkp9/svQ58syV > Content-Type: text/plain; charset=us-ascii > Content-Disposition: inline > > On Tuesday, 11 December 2012 at 3:52:32 +0100, Julian H. Stacey wrote: > > "Greg 'groggy' Lehey" wrote: > >> On Monday, 10 December 2012 at 16:27:39 +0200, Jaakko Heinonen wrote: > >>> > >>> Hi! > >>> > >>> On 2012-12-10, Greg 'groggy' Lehey wrote: > >>>>> Unfortunately r205821 [1] has caused several regressions to calendar(1). > >>>>> Relevant PRs: > >>>>> > >>>>> bin/157718 > >>>>> bin/162211 > >>>>> bin/168785 > >>>>> bin/170930 > > > > I too filed send-prs re. rgeression bugs in recent broken calendar, > > hopefully you can look/ fix please. > > > > bin/157718 > > bin/162211 > > These two are mentioned above. Ah ! > > bin/15182 > > http://www.berklix.com/~jhs/src/bsd/fixes/FreeBSD/src/gen/usr.bin/calendar/send-pr.impossible_date.no_customise > > This one was closed a little over 11 years ago. Since then, the code Whoops :-) I should sleep ! > base has changed completely, but it seems that another bug (one that > I'm currently scratching my head about) has overtaken it. I'll > include your example in my test cases. Cheers :-) > > PS for Anyone, doesnt need to be Greg > > > > misc/157748 > > OK, I'll take a look when I'm done here. > > Greg Thanks Greg :-) Cheers, Julian -- Julian Stacey, BSD Unix Linux C Sys Eng Consultant, Munich http://berklix.com Reply below not above, like a play script. Indent old text with "> ". Send plain text. Not: HTML, multipart/alternative, base64, quoted-printable. From owner-freebsd-hackers@FreeBSD.ORG Tue Dec 11 10:55:42 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id EC62983E for ; Tue, 11 Dec 2012 10:55:42 +0000 (UTC) (envelope-from tevans.uk@googlemail.com) Received: from mail-qc0-f173.google.com (mail-qc0-f173.google.com [209.85.216.173]) by mx1.freebsd.org (Postfix) with ESMTP id 998F58FC1C for ; Tue, 11 Dec 2012 10:55:42 +0000 (UTC) Received: by mail-qc0-f173.google.com with SMTP id b12so2502076qca.18 for ; Tue, 11 Dec 2012 02:55:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=4YyhMkgfiY2zIA0dBOPNJ8UkCfB5aVazO1ZPurWLn7U=; b=sa/0fZXVsYC83JilJSQHCzKB3d+aDbUxGlWliT3at6nD9bPbz25bh5bVdDIkKvhHya dyWJP92Om2094Pep1nO4koBlScVvDMvItfbEuJh6NMWF5SdFyuVkxsx31GtR/4ZVBkSN cUH10aA92w+5u63hDq2sTH3ojGVDDW6FBuIKqIdUQ237HvQJb76dzAKUvDo4w2jixZ/Y zalMRzKoQHivSgQsH4oLORkE6EjcoRWocJ+yADqVGtU+MwlnuqlO1b8mqj59FaKRTL+d GM2X+BLcILY9Y/uYxe5h/cLDVuwZzpo27OqjqiiZ7xUxnQsHUGpZbO6CqT7spFZAG1Fg qeFw== MIME-Version: 1.0 Received: by 10.224.180.205 with SMTP id bv13mr32034070qab.7.1355223336736; Tue, 11 Dec 2012 02:55:36 -0800 (PST) Received: by 10.49.116.129 with HTTP; Tue, 11 Dec 2012 02:55:36 -0800 (PST) In-Reply-To: References: Date: Tue, 11 Dec 2012 10:55:36 +0000 Message-ID: Subject: Re: Building v8 on FreeBSD with clang From: Tom Evans To: Robert Simmons Content-Type: text/plain; charset=UTF-8 Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Dec 2012 10:55:43 -0000 On Mon, Dec 10, 2012 at 6:30 PM, Robert Simmons wrote: > I'm having a problem building google v8 on FreeBSD with clang. I get > this error: > > /usr/bin/ld: final link failed: Nonrepresentable section on output > clang++: error: linker command failed with exit code 1 (use -v to see > invocation) > gmake[1]: *** [/root/v8/out/x64.release/cctest] Error 1 > gmake[1]: Leaving directory `/root/v8/out' > gmake: *** [x64.release] Error 2 > > I needed to use gmake rather than make, and I added the following to use clang: > setenv CC /usr/bin/clang > setenv CXX /usr/bin/clang++ > setenv GYP_DEFINES "clang=1" setenv LINK /usr/bin/clang++ (as grokked from the v8 ports makefile - there are also patches in there to fix building on fbsd10 and adding pthread to the libraries to link to, which may help you next...) Cheers Tom From owner-freebsd-hackers@FreeBSD.ORG Tue Dec 11 15:57:23 2012 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3B3A9F10; Tue, 11 Dec 2012 15:57:23 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) by mx1.freebsd.org (Postfix) with ESMTP id A87B18FC08; Tue, 11 Dec 2012 15:57:21 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.5/8.14.5) with ESMTP id qBBFvJtI093063; Tue, 11 Dec 2012 19:57:19 +0400 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.5/8.14.5/Submit) id qBBFvIHM093062; Tue, 11 Dec 2012 19:57:18 +0400 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Tue, 11 Dec 2012 19:57:18 +0400 From: Gleb Smirnoff To: Richard Sharpe Subject: Re: Possible obscure socket leak when system under load and listener is slow to accept Message-ID: <20121211155718.GR48639@FreeBSD.org> References: <50C3D22D.3060008@freebsd.org> <1355015131.6752.12.camel@localhost.localdomain> <50C4475D.9020300@mu.org> <1355075850.6752.15.camel@localhost.localdomain> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: <1355075850.6752.15.camel@localhost.localdomain> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-hackers@FreeBSD.org, Alfred Perlstein , Andre Oppermann X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Dec 2012 15:57:23 -0000 On Sun, Dec 09, 2012 at 09:57:30AM -0800, Richard Sharpe wrote: R> > lsof and sockstat can be helpful. lsof may be able to help determine if R> > there's a leak because it MAY will find sockets not associated with a R> > process. R> > R> > Hope this helps. R> R> Thanks Alfred. After following through the call graph and confirming R> (with the code) that it was correct, I am now pretty convinced that I R> was wrong in assuming that it was a socket leak. You can always check number of socket allocations in kernel via: vmstat -z | grep ^socket | awk '{print $4}' If you can't establish a scenario when the number infinitely grows, then there is no leak. -- Totus tuus, Glebius. From owner-freebsd-hackers@FreeBSD.ORG Tue Dec 11 20:43:24 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7052C505 for ; Tue, 11 Dec 2012 20:43:24 +0000 (UTC) (envelope-from dieterbsd@engineer.com) Received: from mailout-us.gmx.com (mailout-us.gmx.com [74.208.5.67]) by mx1.freebsd.org (Postfix) with SMTP id 2AD3B8FC12 for ; Tue, 11 Dec 2012 20:43:24 +0000 (UTC) Received: (qmail 21454 invoked by uid 0); 11 Dec 2012 20:43:23 -0000 Received: from 67.206.186.114 by rms-us005 with HTTP Content-Type: text/plain; charset="utf-8" Date: Tue, 11 Dec 2012 15:43:21 -0500 From: "Dieter BSD" Message-ID: <20121211204323.310760@gmx.com> MIME-Version: 1.0 Subject: Re: FreeBSD for serious performance? To: freebsd-hackers@freebsd.org X-Authenticated: #74169980 X-Flags: 0001 X-Mailer: GMX.com Web Mailer x-registered: 0 Content-Transfer-Encoding: 8bit X-GMX-UID: vP+1cAAY3zOlNR3dAHAh09N+IGRvb8DE X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Dec 2012 20:43:24 -0000 >>Ronald writes: >>> the last Alpha to be produced was shipped way back in 2004... eight years >>> ago... with a top speed of 1.3 GHz I now have a cheap little media player >>> thingy sitting on my desk, and _each_ of its two cores runs faster than tha\ t. >>> In short, Alphas hardly constitute high-end hardware in this day and age. >> >> So clock rate is the only thing that matters in your world? > > Yea, pretty much. > > As regards to reliability, except for the occasional low-level quirk (which > is usually taken care of for me by the kernel guys) I've never had a processo\ r > fail on me.  Once, about five or six years ago I accidentally burnt up an > Athlon XP (by not having the heatsink properly seated) but that was entirely > my fault. I care about data integrity, so things like ECC are on my must-have list. I suspect that your "cheap little media player thingy" doesn't have ECC. If you don't care about getting the correct answer you can have infinite speed. A high clock rate doesn't help when some device driver does block_all_interrupts(); while(1) DELAY(MIGHT_AS_WELL_BE_FOREVER); At least four device drivers have caused me to lose data this way. Not what I call high performance. >> In my world, high-end means high quality. It doesn't necessarily >> mean fast, or recent. Data integrity, and yes, reliability, that sort of thing. Performance-wise, in most cases I don't expect to get 99.9999% of the theoretical best case, I'm usually happy with 90-95%. But without NCQ I'm only getting ~6% of what I should be getting. Pretty pathetic for an OS that claims to be all about performance. All the more so when the crappy OSes do support NCQ on that chip. It's not some rare, obscure chip. Lots of boxes have it. >> I never found a way to boot from different partitions, much less >> different disks with GPT. > > Having just been recently convinced to switch over to GPT (from MBR) I do > most sincerly hope that you are either joking or mistaken about this. I am neither joking nor mistaken. I looked but could not find a way. I am not claiming that a way does not exist, merely that I couldn't find it. Perhaps there wasn't a way when I looked (it was awhile ago) but does exist now? I have never been a fan of MBRs, they are for pee-cees with the expected ugly kludges and limitations, real machines don't use them. GPT isn't perfect, but it seems much nicer than MBRs. Warren writes: > Grub (or grub2) can do it. Back when I was triple-booting FreeBSD, NetBSD and Linux I used grub (rev number forgotten). It was supposed to be able to boot BSD from a partition but I never got that to work. I had to have it boot the MBR of a different disk which then booted BSD. I wrote 3 little shell scripts that edited grub's menu to change the default. So I could be running FreeBSD and type "boot_netbsd", and go have lunch while it rebooted. Other than not booting BSD from a partition it wasn't that bad for something that smells of penguin. > If you're booting multiple versions of FreeBSD, see gpart(8) > for some partition attributes that may help. You mean the bootme bootonce stuff? That looks promising, assuming I can manage to decode the man page, and figure out what it actually does. Mostly multiple versions of FreeBSD. (for example 7.0 had a couple of bugs preventing it from booting, so having 6 still available was essential.) I no long need Linux (YEA!!!), and after a certain fubar incident have declared a Linux Free Zone. It would be nice if I could also boot Net/Open/Dragonfly. I don't see a way to boot multiple disks, but GPT allows enough partitions that I probably won't care. Not sure if the bootme bootonce stuff wasn't there yet when I looked, or if I just missed it. Thanks for the pointer. > Or even consider ZFS boot environments. I plan to stick with FFS w/softdeps. From owner-freebsd-hackers@FreeBSD.ORG Tue Dec 11 21:40:51 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A289E513 for ; Tue, 11 Dec 2012 21:40:51 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-we0-f182.google.com (mail-we0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 331D98FC17 for ; Tue, 11 Dec 2012 21:40:50 +0000 (UTC) Received: by mail-we0-f182.google.com with SMTP id u54so2394321wey.13 for ; Tue, 11 Dec 2012 13:40:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=vxEydevgk261S0czjRQXMconZvTNE7HdJTafXLJK1Z0=; b=kCvms+OCzDGWPk/ZRyXD9NVQOvg2wlHr5oS9NShSDADqv/Xodb1yG064GpXhAE0MNa 9NiDX5eti6WyWux2YRYDPLqPeVSpBJ2mhYGmsKh998LZPkFOWyJtv/FfriLymrXgffFf 6C5ni4bEL763a3HFU+KElRqZxPmHWVn1u3c6oRCOc4pNayp5jF5y7fBumBEBbgj0hT2+ vWoTIfZB43WCId0sjPYys+dF7MsGIKdwJqmmQF9K1FbLrB5iDxKoE5on19BTgmSLzxd/ 9JOrlv46irCVjX8LBY/61KUzWAudck5lyLOQpng7lxc3Hey21Ix44i07fA/nkdc0wHXO q77w== MIME-Version: 1.0 Received: by 10.194.93.40 with SMTP id cr8mr2982922wjb.16.1355262050053; Tue, 11 Dec 2012 13:40:50 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.217.57.9 with HTTP; Tue, 11 Dec 2012 13:40:49 -0800 (PST) In-Reply-To: <20121211204323.310760@gmx.com> References: <20121211204323.310760@gmx.com> Date: Tue, 11 Dec 2012 13:40:49 -0800 X-Google-Sender-Auth: I7g6Z8BWxIpNot1aQhx2Wo9LDIc Message-ID: Subject: Re: FreeBSD for serious performance? From: Adrian Chadd To: Dieter BSD Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Dec 2012 21:40:51 -0000 .... I'm not seeing: * any references to driver code that exhibits that very broken behaviour; * any patches from you to implement NCQ on your nforce chipset; * any offer of incentive to any developer to add that support. Now, (1) is definitely worrying but as you've not provided any actual information, the level of helpfulness of this comment is 0. (2) and (3) would go a long way to making FreeBSD "better". FreeBSD is only as much as the community of contributors and developers have the time and energy to make it be. If you want FreeBSD to be different/better/something, you have to step up. Adrian On 11 December 2012 12:43, Dieter BSD wrote: >>>Ronald writes: >>>> the last Alpha to be produced was shipped way back in 2004... eight years >>>> ago... with a top speed of 1.3 GHz I now have a cheap little media player >>>> thingy sitting on my desk, and _each_ of its two cores runs faster than tha\ > t. >>>> In short, Alphas hardly constitute high-end hardware in this day and age. >>> >>> So clock rate is the only thing that matters in your world? >> >> Yea, pretty much. >> >> As regards to reliability, except for the occasional low-level quirk (which >> is usually taken care of for me by the kernel guys) I've never had a processo\ > r >> fail on me. Once, about five or six years ago I accidentally burnt up an >> Athlon XP (by not having the heatsink properly seated) but that was entirely >> my fault. > > I care about data integrity, so things like ECC are on my must-have list. > I suspect that your "cheap little media player thingy" doesn't have ECC. > If you don't care about getting the correct answer you can have infinite > speed. > > A high clock rate doesn't help when some device driver does > > block_all_interrupts(); > while(1) > DELAY(MIGHT_AS_WELL_BE_FOREVER); > > At least four device drivers have caused me to lose data this way. > Not what I call high performance. > >>> In my world, high-end means high quality. It doesn't necessarily >>> mean fast, or recent. > > Data integrity, and yes, reliability, that sort of thing. > > Performance-wise, in most cases I don't expect to get 99.9999% > of the theoretical best case, I'm usually happy with 90-95%. > But without NCQ I'm only getting ~6% of what I should be getting. > Pretty pathetic for an OS that claims to be all about performance. > All the more so when the crappy OSes do support NCQ on that chip. > It's not some rare, obscure chip. Lots of boxes have it. > >>> I never found a way to boot from different partitions, much less >>> different disks with GPT. >> >> Having just been recently convinced to switch over to GPT (from MBR) I do >> most sincerly hope that you are either joking or mistaken about this. > > I am neither joking nor mistaken. I looked but could not find a way. > I am not claiming that a way does not exist, merely that I couldn't > find it. Perhaps there wasn't a way when I looked (it was awhile ago) > but does exist now? I have never been a fan of MBRs, they are for > pee-cees with the expected ugly kludges and limitations, real machines > don't use them. GPT isn't perfect, but it seems much nicer than MBRs. > > Warren writes: >> Grub (or grub2) can do it. > > Back when I was triple-booting FreeBSD, NetBSD and Linux I used > grub (rev number forgotten). It was supposed to be able to boot > BSD from a partition but I never got that to work. I had to have it > boot the MBR of a different disk which then booted BSD. I wrote > 3 little shell scripts that edited grub's menu to change the default. > So I could be running FreeBSD and type "boot_netbsd", and go have lunch > while it rebooted. Other than not booting BSD from a partition it > wasn't that bad for something that smells of penguin. > >> If you're booting multiple versions of FreeBSD, see gpart(8) >> for some partition attributes that may help. > > You mean the bootme bootonce stuff? That looks promising, assuming > I can manage to decode the man page, and figure out what it actually > does. Mostly multiple versions of FreeBSD. (for example 7.0 had a couple > of bugs preventing it from booting, so having 6 still available was > essential.) I no long need Linux (YEA!!!), and after a certain fubar > incident have declared a Linux Free Zone. It would be nice if I could > also boot Net/Open/Dragonfly. I don't see a way to boot multiple disks, > but GPT allows enough partitions that I probably won't care. > > Not sure if the bootme bootonce stuff wasn't there yet when I looked, > or if I just missed it. Thanks for the pointer. > >> Or even consider ZFS boot environments. > > I plan to stick with FFS w/softdeps. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Tue Dec 11 22:43:24 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 43003FDB for ; Tue, 11 Dec 2012 22:43:24 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from vps.rulingia.com (host-122-100-2-194.octopus.com.au [122.100.2.194]) by mx1.freebsd.org (Postfix) with ESMTP id A52338FC19 for ; Tue, 11 Dec 2012 22:43:23 +0000 (UTC) Received: from server.rulingia.com (c220-239-237-85.belrs5.nsw.optusnet.com.au [220.239.237.85]) by vps.rulingia.com (8.14.5/8.14.5) with ESMTP id qBBMhGKA092191 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Wed, 12 Dec 2012 09:43:16 +1100 (EST) (envelope-from peter@rulingia.com) X-Bogosity: Ham, spamicity=0.000000 Received: from server.rulingia.com (localhost.rulingia.com [127.0.0.1]) by server.rulingia.com (8.14.5/8.14.5) with ESMTP id qBBMhAMh061757 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 12 Dec 2012 09:43:10 +1100 (EST) (envelope-from peter@server.rulingia.com) Received: (from peter@localhost) by server.rulingia.com (8.14.5/8.14.5/Submit) id qBBMhAdC061754; Wed, 12 Dec 2012 09:43:10 +1100 (EST) (envelope-from peter) Date: Wed, 12 Dec 2012 09:43:10 +1100 From: Peter Jeremy To: Dieter BSD Subject: Re: FreeBSD for serious performance? Message-ID: <20121211224310.GB35245@server.rulingia.com> References: <20121211204323.310760@gmx.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="7AUc2qLy4jB3hD7Z" Content-Disposition: inline In-Reply-To: <20121211204323.310760@gmx.com> X-PGP-Key: http://www.rulingia.com/keys/peter.pgp User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Dec 2012 22:43:24 -0000 --7AUc2qLy4jB3hD7Z Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2012-Dec-11 15:43:21 -0500, Dieter BSD wrote: >I care about data integrity, so things like ECC are on my must-have list. Well, that's supported by all server CPUs (AMD Opteron, Intel Itanium, Intel Xeon, Oracle/Sun SPARC) and some desktop CPUs (most AMD x86 chips). >A high clock rate doesn't help when some device driver does > >block_all_interrupts(); >while(1) > DELAY(MIGHT_AS_WELL_BE_FOREVER); > >At least four device drivers have caused me to lose data this way. Which device drivers? We can't fix problems we don't know about. >Data integrity, and yes, reliability, that sort of thing. Virtually everything except some embedded and consumer-grade x86 systems manage that. >But without NCQ I'm only getting ~6% of what I should be getting. So, in one sentence you state that ECC is a "must have" and then you complain that that FreeBSD doesn't support NCQ on an old, low-end (consumer-grade) chipset that doesn't support ECC. >It's not some rare, obscure chip. Lots of boxes have it. None that support ECC, so you wouldn't be interested in any of them. >>> I never found a way to boot from different partitions, much less >>> different disks with GPT. Yes, this is a limitation of FreeBSD's GPT loader. So far, no-one has written the code to support multiple boot partitions or disks. Note that most BIOS's allow you to select the boot disk - which is a workaround. --=20 Peter Jeremy --7AUc2qLy4jB3hD7Z Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlDHtv4ACgkQ/opHv/APuIcm0ACfTErO5FtLBDmMQsIB0ESDgZmH oAgAoLWhNvilwlJ3jO1pE7V/vObi4tZ9 =y975 -----END PGP SIGNATURE----- --7AUc2qLy4jB3hD7Z-- From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 12 06:20:48 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5C0EEBB; Wed, 12 Dec 2012 06:20:48 +0000 (UTC) (envelope-from bright@mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id E8C568FC12; Wed, 12 Dec 2012 06:20:47 +0000 (UTC) Received: from Alfreds-MacBook-Pro-6.local (c-67-180-208-218.hsd1.ca.comcast.net [67.180.208.218]) by elvis.mu.org (Postfix) with ESMTPSA id 4E5441A3C3D; Tue, 11 Dec 2012 22:20:41 -0800 (PST) Message-ID: <50C82238.80008@mu.org> Date: Tue, 11 Dec 2012 22:20:40 -0800 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/17.0 Thunderbird/17.0 MIME-Version: 1.0 To: hackers@freebsd.org, Eitan Adler Subject: why is kern.maxproc not read/write? Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Dec 2012 06:20:48 -0000 Eitan was asking me to update the FAQ section 5.7: > *5.7.* Why do I get the error "kernel: proc: table is full"? > That error is no longer relevant, but I also seemed to find out something else interesting.. Been grepping through the code, and it seems like the only side-effect of maxproc changing would be overcrowding the hash table tidhashtbl and pidhashtbl. I can't see anything that's statically allocated any longer. The only bad thing is that the procs seem to be taken from a UMA_ZONE_NOFREE zone, so if the user makes an insanely high value, it could be end of days. Even the MD code seems to use it to size the number of pv entries. I'm wondering if making this a runtime tunable that has a SYSCTL_PROC attached that doesn't allow it to go below some PROC_MIN would be OK. Am I missing something? As far as Eitan's question about the FAQ section, the new message printed is: "maxproc limit exceeded by uid %i, please see tuning(7) and login.conf(5)" The faq is wrong, and tells the user to change sysctl.conf, where it should say to update loader.conf. -Alfred From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 12 12:13:35 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1DEA5C31 for ; Wed, 12 Dec 2012 12:13:35 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [89.206.35.99]) by mx1.freebsd.org (Postfix) with ESMTP id 56EFA8FC0C for ; Wed, 12 Dec 2012 12:13:33 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5) with ESMTP id qBCCDIEU031688; Wed, 12 Dec 2012 13:13:18 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5/Submit) with ESMTP id qBCCDIsr031685; Wed, 12 Dec 2012 13:13:18 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Wed, 12 Dec 2012 13:13:18 +0100 (CET) From: Wojciech Puchar To: Dieter BSD Subject: Re: FreeBSD for serious performance? (was: Re: 9.x -- New Install -- serious partition misalignment) In-Reply-To: <20121209014547.238070@gmx.com> Message-ID: References: <20121209014547.238070@gmx.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="2456600518-825042995-1355314398=:1751" X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Wed, 12 Dec 2012 13:13:18 +0100 (CET) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Dec 2012 12:13:35 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --2456600518-825042995-1355314398=:1751 Content-Type: TEXT/PLAIN; charset=utf-8; format=flowed Content-Transfer-Encoding: 8BIT >> people are often trying to squeeze out that last drop of performance, > > Linux is certainly a steaming pile of crap. BSD is orders of magnitude > better, but hey, that doesn't take much. just pray FreeBSD will not incorporate too much "modern technologies" if you know what i mean. > > But don't brag about high-end hardware.  But FreeBSD has dropped support > for even semi-high-end hardware (DEC Alpha). So I'm stuck running it on Because new ones no longer exist and new AMD64 hardware easily beats alpha or SPARC hardware, with SPARC T3 being only serious competitor in total throughput, but with enormous price and being in hands of oracle. > AMD64. Nothing against AMD, they did what they could to try and make a silk AMD64 is instruction set standard, made first by AMD but now all intels have it too. > a high quality board in amd64/x86 land with good reviews doesn't compare. i would say that Dell servers are actually good in compatibility with standards and performance. the "low end" (single socket xeon) ones are cheap, others are not. > The firmware is absolute crap, and it's not like it is something you can true. > >> Performance has been degraded by a whopping 75% ! > > Having a 4KiB misalignment is nothing compared with not having NCQ 4kB misalignment is HUGE performance loss. > Speaking of alignment, I still get "partition 1 does not end on a so why you create windows style slices at all? Why ada0s1a not just ada0a? --2456600518-825042995-1355314398=:1751-- From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 12 12:16:27 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D9F93EE5 for ; Wed, 12 Dec 2012 12:16:27 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [89.206.35.99]) by mx1.freebsd.org (Postfix) with ESMTP id 39B658FC08 for ; Wed, 12 Dec 2012 12:16:26 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5) with ESMTP id qBCCGOf1038889; Wed, 12 Dec 2012 13:16:24 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5/Submit) with ESMTP id qBCCGMYU038370; Wed, 12 Dec 2012 13:16:24 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Wed, 12 Dec 2012 13:16:22 +0100 (CET) From: Wojciech Puchar To: "Ronald F. Guilmette" Subject: Re: FreeBSD for serious performance? (was: Re: 9.x -- New Install -- serious partition misalignment) In-Reply-To: <1830.1355025154@tristatelogic.com> Message-ID: References: <1830.1355025154@tristatelogic.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Wed, 12 Dec 2012 13:16:24 +0100 (CET) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Dec 2012 12:16:27 -0000 > > 2) Although I don't know a lot about boards, it is my understanding that > ASUS makes pretty good ones, and they have always worked for me (and the > firmware is typically quite good). maybe it's strange for for a long time i NEVER really succeeded making stable configuration from popular parts. Yes it works fast, but when running under heavy load with CPU quite loaded, lots of I/O from multiple disks and over ethernet in the same time none are stable. AHCI timeout is the most common message... or just random reboots. Now getting low end Dell server (which isn't really low end) for not much more money is what i do. it works. From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 12 12:16:59 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 21DFCFE4; Wed, 12 Dec 2012 12:16:59 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [89.206.35.99]) by mx1.freebsd.org (Postfix) with ESMTP id 7021A8FC08; Wed, 12 Dec 2012 12:16:58 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5) with ESMTP id qBCCGrAU043745; Wed, 12 Dec 2012 13:16:53 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5/Submit) with ESMTP id qBCCGrRF043726; Wed, 12 Dec 2012 13:16:53 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Wed, 12 Dec 2012 13:16:53 +0100 (CET) From: Wojciech Puchar To: Adrian Chadd Subject: Re: FreeBSD for serious performance? (was: Re: 9.x -- New Install -- serious partition misalignment) In-Reply-To: Message-ID: References: <20121209014547.238070@gmx.com> <1830.1355025154@tristatelogic.com> <20121208230433.2c54f44a@kan.dyndns.org> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Wed, 12 Dec 2012 13:16:53 +0100 (CET) Cc: Eitan Adler , "Ronald F. Guilmette" , freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Dec 2012 12:16:59 -0000 what's a point of not using ahci(4)? On Sat, 8 Dec 2012, Adrian Chadd wrote: > Hi, > > Yes. atacam supports NCQ. > > The older IDE/ATA code doesn't support NCQ. The CAM ATA code (ie, > atacam) supports it if the drive supports it. > > So, the "FreeBSD doesn't do NCQ" point is incorrect. > > If you don't believe me - look in sys/cam/ata/ata_da.c, look for ata_ncq_cmd(). > > > Adrian > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 12 12:17:22 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 35ACE164 for ; Wed, 12 Dec 2012 12:17:22 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [89.206.35.99]) by mx1.freebsd.org (Postfix) with ESMTP id 8AC2D8FC12 for ; Wed, 12 Dec 2012 12:17:21 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5) with ESMTP id qBCCHGd7045796; Wed, 12 Dec 2012 13:17:16 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5/Submit) with ESMTP id qBCCHGhS045761; Wed, 12 Dec 2012 13:17:16 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Wed, 12 Dec 2012 13:17:16 +0100 (CET) From: Wojciech Puchar To: Warren Block Subject: Re: FreeBSD for serious performance? (was: Re: 9.x -- New Install -- serious partition misalignment) In-Reply-To: Message-ID: References: <20121209014547.238070@gmx.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Wed, 12 Dec 2012 13:17:16 +0100 (CET) Cc: freebsd-hackers@freebsd.org, Dieter BSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Dec 2012 12:17:22 -0000 > B. Use GPT, which does not have the CHS baggage. It is easier and more bsd labels doesn't have too > versatile. My systems with GPT disks don't complain about track > alignment. Or maybe that's ahci(4)'s doing. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 12 12:20:10 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 8B44C2A7 for ; Wed, 12 Dec 2012 12:20:10 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [89.206.35.99]) by mx1.freebsd.org (Postfix) with ESMTP id CBBFF8FC14 for ; Wed, 12 Dec 2012 12:20:09 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5) with ESMTP id qBCCK7vT062093; Wed, 12 Dec 2012 13:20:07 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5/Submit) with ESMTP id qBCCK6Rg062053; Wed, 12 Dec 2012 13:20:07 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Wed, 12 Dec 2012 13:20:06 +0100 (CET) From: Wojciech Puchar To: "Ronald F. Guilmette" Subject: Re: FreeBSD for serious performance? In-Reply-To: <20594.1355090530@tristatelogic.com> Message-ID: References: <20594.1355090530@tristatelogic.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Wed, 12 Dec 2012 13:20:07 +0100 (CET) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Dec 2012 12:20:10 -0000 >> >> So clock rate is the only thing that matters in your world? > > Yea, pretty much. operations per second do matter. compared to latest x86 hardware alpha lose even at the same clock speed. From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 12 13:48:52 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 53B9933B for ; Wed, 12 Dec 2012 13:48:52 +0000 (UTC) (envelope-from se@freebsd.org) Received: from nm15-vm0.bullet.mail.ukl.yahoo.com (nm15-vm0.bullet.mail.ukl.yahoo.com [217.146.183.252]) by mx1.freebsd.org (Postfix) with ESMTP id 9A5988FC12 for ; Wed, 12 Dec 2012 13:48:51 +0000 (UTC) Received: from [217.12.10.82] by nm15.bullet.mail.ukl.yahoo.com with NNFMP; 12 Dec 2012 13:42:50 -0000 Received: from [77.238.184.56] by tm16.bullet.mail.ukl.yahoo.com with NNFMP; 12 Dec 2012 13:42:50 -0000 Received: from [127.0.0.1] by smtp125.mail.ukl.yahoo.com with NNFMP; 12 Dec 2012 13:42:50 -0000 X-Yahoo-Newman-Id: 808341.54573.bm@smtp125.mail.ukl.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: g4fy27wVM1lhV4a9mBeNszvsa0p3KOsPyDJbDDsZXHm4t8C 3yRmaItUs3gHKzfVsjey..USsHi6ayjwR3NuEIYVcO1xfK71yv.3VBAj55TK F5gm_vQ05np4bYIGkdxv4.0amzE0Sxwpy0upBbZqADsVPpS7Dsk8pT.rdrcT df.w.aaFWkGW1e5b1w_o_2494lSVbUZmvpSrS52.vqULj_P4UeIg6NQlJ1EP 7DvfMbJb9Q1Mqcds.OX8KMfDEMpUYs.xt1a5g6WlAZUn_2d0jLEMsypeoLrS TCpd915U8Dm4YOhdtfMqgLBCBEgDrE4hBVThubf4hEnVFLecA9Jt5tlJbD_v iZTucOP9gs2nhzzWIrKNcGLW8E0kRdn_VlPyx35m9N752mTKpdR37RmLs.TY 1tNUs9COLbmVzqpOWGo5UnwzD0nUUUryFUMh_6XOq3EdC X-Yahoo-SMTP: iDf2N9.swBDAhYEh7VHfpgq0lnq. Received: from [192.168.119.18] (se@87.158.11.102 with plain) by smtp125.mail.ukl.yahoo.com with SMTP; 12 Dec 2012 13:42:50 +0000 GMT Message-ID: <50C889D3.1050404@freebsd.org> Date: Wed, 12 Dec 2012 14:42:43 +0100 From: Stefan Esser User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/17.0 Thunderbird/17.0 MIME-Version: 1.0 CC: freebsd-hackers@freebsd.org Subject: Re: FreeBSD for serious performance? References: <20121211204323.310760@gmx.com> In-Reply-To: X-Enigmail-Version: 1.4.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Dec 2012 13:48:52 -0000 Am 11.12.2012 22:40, schrieb Adrian Chadd: > .... I'm not seeing: > > * any references to driver code that exhibits that very broken behaviour; > * any patches from you to implement NCQ on your nforce chipset; > * any offer of incentive to any developer to add that support. > > Now, (1) is definitely worrying but as you've not provided any actual > information, the level of helpfulness of this comment is 0. > (2) and (3) would go a long way to making FreeBSD "better". Yes, but the answer to (1) seems to be very simple: The cause of the low write performance is the disabled write cache. Enabling the write cache is unsafe on SATA drives (with or without NCQ), since they do not make any guarantees that nearby data is not lost if power fails during a disk write. It never happened to me, but there is a reason that SAS drives have less capacity, much lower BER (one to two magnitudes) and are more expensive than SATA drives. The solution to the performance problem is simple: Turn on the write cache. If the data is valuable, then SAS is the solution to both the performance and the inherent reliability problems. If SATA with NCQ really provides acceptable reliability, then a cheap SATA controller with NCQ support in FreeBSD might also be an option, at a cost that would pay one developer hour. Asking Nvidia to release the confidential documentation for their chip-set might help, but I doubt that there is much interest to add support for NCQ to an obsolete chip-set, today, unless you pay a developer (and even then ...). If all these don't work for you, then you may really be better served by Linux with the drivers donated by Nvidia. And if you buy a new system, you may consider choosing components from vendors that do not provide binary blobs for selected operating systems, but publish the necessary documentation for driver writers (or which develop open source drivers for FreeBSD, not only for Linux). There are so many ways to solve your problem From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 12 16:50:22 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 34DBE270 for ; Wed, 12 Dec 2012 16:50:22 +0000 (UTC) (envelope-from nonesuch@longcount.org) Received: from mail-vb0-f54.google.com (mail-vb0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id CB6B48FC12 for ; Wed, 12 Dec 2012 16:50:21 +0000 (UTC) Received: by mail-vb0-f54.google.com with SMTP id l1so1041578vba.13 for ; Wed, 12 Dec 2012 08:50:20 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-originating-ip:date:message-id:subject:from:to :content-type:x-gm-message-state; bh=Pk+FViX/XOz7ue/TKg+OQgu34YmCjGkfduiQKypgb28=; b=o79hdy/M+vildTLv0JjDkflIs/4wHpGSVprDonflwH9H9RU2Kmpg0GRtAiJNiDJhjH GnwQEJuSWiJQmr13XCc/5IrG7Z1hC4h8EA1kKFK8Anx/4qLZGv92k+mlO2NVggiMZ6iM IaIpAsizXcRFa3G7Q6goBEhJCUoAOwn1RG2IbUb/+Mcod/7ZACESzkmFNRLnIKWS+7hk I7OL39mHhlwLIorspA+0J3D+DNuuIBqTKRNL/sasabpvdlT53855tTNnqrrgzZlmndfI jLoMQxXz1GdmZPdcNxGSgvBxOE1/om1dDu9l6cCfxN9iTDCylRfAypjDamq5uUErA1zF GIpw== MIME-Version: 1.0 Received: by 10.52.37.103 with SMTP id x7mr769116vdj.61.1355331020627; Wed, 12 Dec 2012 08:50:20 -0800 (PST) Received: by 10.58.210.34 with HTTP; Wed, 12 Dec 2012 08:50:20 -0800 (PST) X-Originating-IP: [216.223.13.216] Date: Wed, 12 Dec 2012 11:50:20 -0500 Message-ID: Subject: Syslog-ng weirdness on 9.1-RELEASE From: Mark Saad To: "freebsd-hackers@freebsd.org" X-Gm-Message-State: ALoCoQnhJgywBbS2YZcbcfq2LV2BgT5zSNX2Kf9afAAPn7LMh6dgDnns//ezZ0FWJNRnlNkG49Ml Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Dec 2012 16:50:22 -0000 All I am wondering if anyone has seen this. I pulled down the 9.1-RELEASE install media and did a clean install . After installing some ports from the packages on ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-9-stable I noticed that syslog-ng is spinning out of control with a stock config. ps shows I have two syslog-ng pids the lower numbed pid is idle and the higher numbed pid is spinning out of control Here is a truss of each # truss -p 20979 SIGNAL 17 (SIGSTOP # truss -p 20980 kevent(3,{},0,{0xb,EVFILT_READ,0x0,0,0x56fb,0x80251b5f0},8,{10.861307840 }) = 1 (0x1) clock_gettime(4,{2199.064214776 }) = 0 (0x0) kevent(3,{},0,{0xb,EVFILT_READ,0x0,0,0x56fb,0x80251b5f0},8,{10.860672308 }) = 1 (0x1) clock_gettime(4,{2199.064899380 }) = 0 (0x0) kevent(3,{},0,{0xb,EVFILT_READ,0x0,0,0x56fb,0x80251b5f0},8,{10.859987704 }) = 1 (0x1) clock_gettime(4,{2199.065608622 }) = 0 (0x0) kevent(3,{},0,{0xb,EVFILT_READ,0x0,0,0x56fb,0x80251b5f0},8,{10.859278462 }) = 1 (0x1) clock_gettime(4,{2199.066240031 }) = 0 (0x0) kevent(3,{},0,{0xb,EVFILT_READ,0x0,0,0x56fb,0x80251b5f0},8,{10.858647053 }) = 1 (0x1) clock_gettime(4,{2199.066876796 }) = 0 (0x0) kevent(3,{},0,{0xb,EVFILT_READ,0x0,0,0x56fb,0x80251b5f0},8,{10.858010288 }) = 1 (0x1 # procstat -f 20979 PID COMM FD T V FLAGS REF OFFSET PRO NAME 20979 syslog-ng text v r r-------- - - - /usr/local/sbin/syslog-ng 20979 syslog-ng cwd v d r-------- - - - / 20979 syslog-ng root v d r-------- - - - / 20979 syslog-ng 0 v c r-------- 1 0 - /dev/null 20979 syslog-ng 1 v c -w------- 2 0 - /dev/null 20979 syslog-ng 2 v c -w------- 2 0 - /dev/null # procstat -f 20980 PID COMM FD T V FLAGS REF OFFSET PRO NAME 20980 syslog-ng text v r r-------- - - - /usr/local/sbin/syslog-ng 20980 syslog-ng cwd v d r-------- - - - /var/db 20980 syslog-ng root v d r-------- - - - / 20980 syslog-ng 0 v c r-------- 1 0 - /dev/null 20980 syslog-ng 1 v c -w------- 2 0 - /dev/null 20980 syslog-ng 2 v c -w------- 2 0 - /dev/null 20980 syslog-ng 3 k - rw------- 1 0 - - 20980 syslog-ng 4 p - rw------- 1 0 - - 20980 syslog-ng 5 s - rw---n--- 2 0 UDS /var/db/syslog-ng.ctl 20980 syslog-ng 6 p - rw---n--- 2 0 - - 20980 syslog-ng 7 p - rw---n--- 1 0 - - 20980 syslog-ng 8 v r rw------- 1 16384 - - 20980 syslog-ng 9 s - rw---n--- 2 0 UDD /var/run/log 20980 syslog-ng 10 s - rw---n--- 2 0 UDD /var/run/logpriv 20980 syslog-ng 11 v c r----n--- 2 0 - /dev/klog 20980 syslog-ng 12 p - rw---n--- 2 0 - - 20980 syslog-ng 13 p - rw---n--- 1 0 - - 20980 syslog-ng 14 p - rw---n--- 2 0 - - 20980 syslog-ng 15 p - rw---n--- 1 0 - - 20980 syslog-ng 16 p - rw---n--- 2 0 - - 20980 syslog-ng 17 p - rw---n--- 1 0 - - 20980 syslog-ng 18 p - rw---n--- 2 0 - - 20980 syslog-ng 19 p - rw---n--- 1 0 - - Here is the config # cat /usr/local/etc/syslog-ng.conf ############################################################################# # Default syslog-ng.conf file which collects all local logs into a # single file called /var/log/messages. # @version: 3.3 @include "scl.conf" source s_local { system(); internal(); }; source s_network { udp(); }; destination d_local { file("/var/log/messages"); }; log { source(s_local); # uncomment this line to open port 514 to receive messages #source(s_network); destination(d_local); }; Here are the packages I installed. bash-4.2.37 ca_root_nss-3.13.6 cciss_vol_status-1.10 compat6x-amd64-6.4.604000.200810_3 compat7x-amd64-7.3.703000.201008_1 curl-7.24.0_1 db41-4.1.25_4 dejavu-2.33 dmidecode-2.11 eventlog-0.2.12 expat-2.0.1_2 figlet-2.2.4 fontconfig-2.9.0,1 freetype2-2.4.9_1 gamin-0.1.10_4 gettext-0.18.1.1 gio-fam-backend-2.28.8_1 glib-2.28.8_4 gmake-3.82_1 inputproto-2.0.2 java-zoneinfo-2012.f javavmwrapper-2.4 jpeg-8_3 kbproto-1.0.5 libICE-1.0.7,1 libSM-1.2.0,1 libX11-1.4.4,1 libXau-1.0.6 libXaw-1.0.9,2 libXdmcp-1.1.0 libXext-1.3.0_1,1 libXft-2.1.14 libXi-1.4.5,1 libXmu-1.1.0,1 libXp-1.0.1,1 libXpm-3.5.9 libXrender-0.9.6 libXt-1.1.1,1 libXtst-1.2.0 libiconv-1.14 libpthread-stubs-0.3_3 libxcb-1.7 mcelog-1.0.p3 net-snmp-5.7.2 nspr-4.9.2 nss-3.13.6_1 open-motif-2.3.3_2 openjdk6-b26_1 openldap-client-2.4.33 openssl-1.0.1_4 pam_ldap-1.8.6_2 pam_mkhomedir-0.2 pcre-8.31_1 perl-5.14.2_2 pkgconf-0.8.9 png-1.5.12 portaudit-0.6.0 portupgrade-2.4.10,2 printproto-1.0.5 python27-2.7.3_3 recordproto-1.14.1 renderproto-0.11.1 rsync-3.0.9_2 ruby-1.8.7.370,1 ruby18-bdb-0.6.6 sqlite3-3.7.14.1 sudo-1.8.6.p3_1 syslog-ng-3.3.6_3 unzip-6.0_1 vim-lite-7.3.669 xbitmaps-1.1.1 xextproto-7.2.0 xproto-7.0.22 Anyone have any ideas ? -- mark saad | nonesuch@longcount.org From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 12 16:53:50 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D50F24C2 for ; Wed, 12 Dec 2012 16:53:50 +0000 (UTC) (envelope-from nonesuch@longcount.org) Received: from mail-vc0-f182.google.com (mail-vc0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 69FA48FC08 for ; Wed, 12 Dec 2012 16:53:50 +0000 (UTC) Received: by mail-vc0-f182.google.com with SMTP id fo14so1069168vcb.13 for ; Wed, 12 Dec 2012 08:53:49 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-originating-ip:in-reply-to:references:date :message-id:subject:from:to:content-type:x-gm-message-state; bh=f/a6ViVyDoSQ6se65jRRN2GaDch9ypGcsS6PJrgdf3o=; b=eccWvIafa7gABq50NWexjJgGMxYDLo7HLd85eP1PijnpAvXyxNcl5+ZH+wZAN9OvzQ HxTtcAHjHP/wDchZzkGO6h9K9Injmn2GgvJb3LzSm9tkgZcxDVnABnbT7YCK8Nq7qmZN KzfFhldejeZoYl63bmExhxEGArqL7kiHnXOtP1kkBU8KceGb1oYC3ZDNCsRVbqGBhdl0 UMPDi28i9xHUgfDTvqw5LDIG7MswnitkbPMy6+fCp69YLYu3N11zPUk65dKwuGLMmKjH ObsQOEUUvAvTcosW16MxZnKt26S+/0jHR4Vqv59Y1rmsW5j8BlYdR6f7Mr+xmXaUoLpj jQNw== MIME-Version: 1.0 Received: by 10.220.240.141 with SMTP id la13mr866758vcb.39.1355331229018; Wed, 12 Dec 2012 08:53:49 -0800 (PST) Received: by 10.58.210.34 with HTTP; Wed, 12 Dec 2012 08:53:48 -0800 (PST) X-Originating-IP: [216.223.13.216] In-Reply-To: References: Date: Wed, 12 Dec 2012 11:53:48 -0500 Message-ID: Subject: Re: Syslog-ng weirdness on 9.1-RELEASE From: Mark Saad To: "freebsd-hackers@freebsd.org" X-Gm-Message-State: ALoCoQlCNwybm2FRnCdaoBsu0pQ1faHYVfduS3Yj7yjtze+JWGQM7Cn0RYXlgJaPfpXeZa+AFTFp Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Dec 2012 16:53:50 -0000 looks like i jumped the gun its a bug in syslog-ng https://bugzilla.balabit.com/show_bug.cgi?id=193 On Wed, Dec 12, 2012 at 11:50 AM, Mark Saad wrote: > All > I am wondering if anyone has seen this. I pulled down the 9.1-RELEASE > install media and did a clean install . After installing some ports from > the packages on > ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-9-stable > > I noticed that syslog-ng is spinning out of control with a stock config. > > ps shows I have two syslog-ng pids > > the lower numbed pid is idle and the higher numbed pid is spinning out of > control > > Here is a truss of each > > # truss -p 20979 > SIGNAL 17 (SIGSTOP > > # truss -p 20980 > kevent(3,{},0,{0xb,EVFILT_READ,0x0,0,0x56fb,0x80251b5f0},8,{10.861307840 > }) = 1 (0x1) > clock_gettime(4,{2199.064214776 }) = 0 (0x0) > kevent(3,{},0,{0xb,EVFILT_READ,0x0,0,0x56fb,0x80251b5f0},8,{10.860672308 > }) = 1 (0x1) > clock_gettime(4,{2199.064899380 }) = 0 (0x0) > kevent(3,{},0,{0xb,EVFILT_READ,0x0,0,0x56fb,0x80251b5f0},8,{10.859987704 > }) = 1 (0x1) > clock_gettime(4,{2199.065608622 }) = 0 (0x0) > kevent(3,{},0,{0xb,EVFILT_READ,0x0,0,0x56fb,0x80251b5f0},8,{10.859278462 > }) = 1 (0x1) > clock_gettime(4,{2199.066240031 }) = 0 (0x0) > kevent(3,{},0,{0xb,EVFILT_READ,0x0,0,0x56fb,0x80251b5f0},8,{10.858647053 > }) = 1 (0x1) > clock_gettime(4,{2199.066876796 }) = 0 (0x0) > kevent(3,{},0,{0xb,EVFILT_READ,0x0,0,0x56fb,0x80251b5f0},8,{10.858010288 > }) = 1 (0x1 > > # procstat -f 20979 > PID COMM FD T V FLAGS REF OFFSET PRO NAME > 20979 syslog-ng text v r r-------- - - - > /usr/local/sbin/syslog-ng > 20979 syslog-ng cwd v d r-------- - - - > / > 20979 syslog-ng root v d r-------- - - - > / > 20979 syslog-ng 0 v c r-------- 1 0 - > /dev/null > 20979 syslog-ng 1 v c -w------- 2 0 - > /dev/null > 20979 syslog-ng 2 v c -w------- 2 0 - > /dev/null > > > # procstat -f 20980 > PID COMM FD T V FLAGS REF OFFSET PRO NAME > 20980 syslog-ng text v r r-------- - - - > /usr/local/sbin/syslog-ng > 20980 syslog-ng cwd v d r-------- - - - > /var/db > 20980 syslog-ng root v d r-------- - - - > / > 20980 syslog-ng 0 v c r-------- 1 0 - > /dev/null > 20980 syslog-ng 1 v c -w------- 2 0 - > /dev/null > 20980 syslog-ng 2 v c -w------- 2 0 - > /dev/null > 20980 syslog-ng 3 k - rw------- 1 0 - > - > 20980 syslog-ng 4 p - rw------- 1 0 - > - > 20980 syslog-ng 5 s - rw---n--- 2 0 UDS > /var/db/syslog-ng.ctl > 20980 syslog-ng 6 p - rw---n--- 2 0 - > - > 20980 syslog-ng 7 p - rw---n--- 1 0 - > - > 20980 syslog-ng 8 v r rw------- 1 16384 - > - > 20980 syslog-ng 9 s - rw---n--- 2 0 UDD /var/run/log > 20980 syslog-ng 10 s - rw---n--- 2 0 UDD /var/run/logpriv > 20980 syslog-ng 11 v c r----n--- 2 0 - > /dev/klog > 20980 syslog-ng 12 p - rw---n--- 2 0 - > - > 20980 syslog-ng 13 p - rw---n--- 1 0 - > - > 20980 syslog-ng 14 p - rw---n--- 2 0 - > - > 20980 syslog-ng 15 p - rw---n--- 1 0 - > - > 20980 syslog-ng 16 p - rw---n--- 2 0 - > - > 20980 syslog-ng 17 p - rw---n--- 1 0 - > - > 20980 syslog-ng 18 p - rw---n--- 2 0 - > - > 20980 syslog-ng 19 p - rw---n--- 1 0 - - > > Here is the config > > # cat /usr/local/etc/syslog-ng.conf > > > ############################################################################# > # Default syslog-ng.conf file which collects all local logs into a > # single file called /var/log/messages. > # > > @version: 3.3 > @include "scl.conf" > > source s_local { > system(); > internal(); > }; > > source s_network { > udp(); > }; > > destination d_local { > file("/var/log/messages"); > }; > > log { > source(s_local); > > # uncomment this line to open port 514 to receive messages > #source(s_network); > destination(d_local); > }; > > > Here are the packages I installed. > > bash-4.2.37 > ca_root_nss-3.13.6 > cciss_vol_status-1.10 > compat6x-amd64-6.4.604000.200810_3 > compat7x-amd64-7.3.703000.201008_1 > curl-7.24.0_1 > db41-4.1.25_4 > dejavu-2.33 > dmidecode-2.11 > eventlog-0.2.12 > expat-2.0.1_2 > figlet-2.2.4 > fontconfig-2.9.0,1 > freetype2-2.4.9_1 > gamin-0.1.10_4 > gettext-0.18.1.1 > gio-fam-backend-2.28.8_1 > glib-2.28.8_4 > gmake-3.82_1 > inputproto-2.0.2 > java-zoneinfo-2012.f > javavmwrapper-2.4 > jpeg-8_3 > kbproto-1.0.5 > libICE-1.0.7,1 > libSM-1.2.0,1 > libX11-1.4.4,1 > libXau-1.0.6 > libXaw-1.0.9,2 > libXdmcp-1.1.0 > libXext-1.3.0_1,1 > libXft-2.1.14 > libXi-1.4.5,1 > libXmu-1.1.0,1 > libXp-1.0.1,1 > libXpm-3.5.9 > libXrender-0.9.6 > libXt-1.1.1,1 > libXtst-1.2.0 > libiconv-1.14 > libpthread-stubs-0.3_3 > libxcb-1.7 > mcelog-1.0.p3 > net-snmp-5.7.2 > nspr-4.9.2 > nss-3.13.6_1 > open-motif-2.3.3_2 > openjdk6-b26_1 > openldap-client-2.4.33 > openssl-1.0.1_4 > pam_ldap-1.8.6_2 > pam_mkhomedir-0.2 > pcre-8.31_1 > perl-5.14.2_2 > pkgconf-0.8.9 > png-1.5.12 > portaudit-0.6.0 > portupgrade-2.4.10,2 > printproto-1.0.5 > python27-2.7.3_3 > recordproto-1.14.1 > renderproto-0.11.1 > rsync-3.0.9_2 > ruby-1.8.7.370,1 > ruby18-bdb-0.6.6 > sqlite3-3.7.14.1 > sudo-1.8.6.p3_1 > syslog-ng-3.3.6_3 > unzip-6.0_1 > vim-lite-7.3.669 > xbitmaps-1.1.1 > xextproto-7.2.0 > xproto-7.0.22 > > > Anyone have any ideas ? > > -- > mark saad | nonesuch@longcount.org > > -- mark saad | nonesuch@longcount.org From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 12 16:57:05 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3F541807 for ; Wed, 12 Dec 2012 16:57:05 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: from mail-la0-f54.google.com (mail-la0-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id A9D5A8FC0A for ; Wed, 12 Dec 2012 16:57:04 +0000 (UTC) Received: by mail-la0-f54.google.com with SMTP id j13so859441lah.13 for ; Wed, 12 Dec 2012 08:57:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=K1rIU7iP6L4+Amh0a4OEVJeG6a05QAt/zFx2cUxT918=; b=ZLJkuA8sYN3OWa4r5yR98h8qMEOtDxaUkmEbHnXT2JJ6sOCDYAt+YGx7fyK3VKTHff Hb1m4ROKXoRgTI9EPxSrsdA74NdZKnz9RwaXpL9K8VXxIn2HdKfhoXtYFG6KIjnZYDQu PwkN7OVyxn4+uaKtaQhfx3nIPmBek30WBBzyWMttV59om/A0mZh/n+SscEIBDxuvfDFT fsVwVyK4S45LoK1acMGUCxlcSt/JQjtmqA89kdk2xGdIVPib7DGzyNBbbKNaMRalQ5kt /NqkZDukKsQM/7jLgudxw8zRDlZvwxl1d0KxPF7qxP/N5zAQ3zfYzaW/7BRGUAO+ucq7 9tTw== MIME-Version: 1.0 Received: by 10.152.148.129 with SMTP id ts1mr1741387lab.19.1355331423533; Wed, 12 Dec 2012 08:57:03 -0800 (PST) Received: by 10.112.61.33 with HTTP; Wed, 12 Dec 2012 08:57:03 -0800 (PST) Date: Wed, 12 Dec 2012 11:57:03 -0500 Message-ID: Subject: iSCSI vs. SMB with ZFS. From: Zaphod Beeblebrox To: FreeBSD Hackers Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Dec 2012 16:57:05 -0000 So... I have two machines. My Fileserver is a core-2-duo machine with FreeBSD-9.1-ish ZFS, istgt and samba 3.6. My workstation is windows 7 on an i7. Both have GigE and are connected directly via a managed switch with jumbo packets (specifically 9016) enabled. Both are using tagged vlan packets to the switch (if that matters at all). Some time ago, I created a 2T iSCSI disk on ZFS to serve the "Steam" directory (games) on my C drive as it was growing rather large. I've been quite happy with this. The performance of the iSCSI disk is about the same as the local disk for some operations --- faster for some, slower for others. The workstation has 12G of memory and it's my perception that iSCSI is heavily cached and that this enhances it's performance. The second launch of a game ... or the second switch into an area (ie: loading a specific piece of geometry again) is very fast. But this is imperfect. The iSCSI disk reserves all of it's space and the files on the disk are only accessible to the computer that mounts it. The most recent Steam update supported an easy way to put steam folders on other disks and partitions. I created another Steam folder on an SMB share from the same server and proceeded to move one of my games there. The performance on the SMB share is abysmal compared to the performance on the iSCSI share. At the very least, there seems to be little benifit to launching the same application twice --- which is most likely windows fault. I haven't done any major amount of tuning on the SMB share lately, but the last time I cared, it was setup reasonably... with TCPNODELAY and whatnot. I also notice that my copy of smbd runs with 1 thread (according to top) rather than the 11 threads that istgt uses. Does this breakdown of performance square with other's experiences? Will SMB always have significantly less performance than iSCSI coming from ZFS? From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 12 17:05:27 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 979FDE34; Wed, 12 Dec 2012 17:05:27 +0000 (UTC) (envelope-from prabhpal@digital-infotech.net) Received: from mail.digital-infotech.net (mail.digital-infotech.net [41.211.25.193]) by mx1.freebsd.org (Postfix) with ESMTP id 34BFF8FC13; Wed, 12 Dec 2012 17:05:27 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mail.digital-infotech.net (Postfix) with ESMTP id 4B0302E4103; Wed, 12 Dec 2012 17:05:26 +0000 (GMT) Received: from mail.digital-infotech.net ([127.0.0.1]) by localhost (mail.digital-infotech.net [127.0.0.1]) (maiad, port 10024) with ESMTP id 00193-02; Wed, 12 Dec 2012 17:05:26 +0000 (GMT) Received: from mail.digital-infotech.net (localhost [127.0.0.1]) by mail.digital-infotech.net (Postfix) with ESMTP id 2CFD32E4102; Wed, 12 Dec 2012 17:05:26 +0000 (GMT) X-DKIM: OpenDKIM Filter v2.5.0 mail.digital-infotech.net 2CFD32E4102 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=digital-infotech.net; s=digital; t=1355331926; bh=bmxGCCIA9RYZm7LYuE0iH7j56pE0kos62LHndmEWfJc=; h=Date:Subject:From:To:Reply-To; b=JjDuvbKLKy+B3LUclzwbyO7Cl/W6av+NpmQxaYYOI9yYVIBQ95QmrrV6LQ/ZPDQ+I XFf2xysJ4Vh+gy7ktN3PIczuSUpGJWncu07gzZMDHzwE5u+KmnXVNpPcWY+nijW3yf /f04B9TN/aHHR0bZ25l6rmOwtYAuRovZ+qpIe96g= Received: from 41.211.28.7 (SquirrelMail authenticated user prabhpal@digital-infotech.net) by mail.digital-infotech.net with HTTP; Wed, 12 Dec 2012 17:05:26 -0000 Message-ID: Date: Wed, 12 Dec 2012 17:05:26 -0000 Subject: download quota of 200MB per voucher From: "Prabhpal S. Mavi" To: freebsd-hackers@freebsd.org, freebsd-stable@freebsd.org, list@lists.pfsense.org User-Agent: SquirrelMail/1.4.22 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: prabhpal@digital-infotech.net List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Dec 2012 17:05:27 -0000 Dear Friends, i have a question for you, i am sure someone can help. The pfsense captive portal is up and running. Time countdown vouchers are working without issue, such as 30m, 45m, 1h & so on. However, I'd like to set up a download quota of 200MB per voucher. but then you need to login with a username and password, instead of vouchers. but I haven't found a way to generate username & password when generating vouchers. is there someone who managed to get this working? At the moment vouchers are only for time based login. any clue, little information or document reference would be greatly appreciated. Thanks From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 12 16:51:56 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 37DBA37E for ; Wed, 12 Dec 2012 16:51:56 +0000 (UTC) (envelope-from prabhpal@digital-infotech.net) Received: from mail.digital-infotech.net (mail.digital-infotech.net [41.211.25.193]) by mx1.freebsd.org (Postfix) with ESMTP id C7B9A8FC12 for ; Wed, 12 Dec 2012 16:51:55 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mail.digital-infotech.net (Postfix) with ESMTP id 6173A2E4103 for ; Wed, 12 Dec 2012 16:51:53 +0000 (GMT) Received: from mail.digital-infotech.net ([127.0.0.1]) by localhost (mail.digital-infotech.net [127.0.0.1]) (maiad, port 10024) with ESMTP id 95976-09 for ; Wed, 12 Dec 2012 16:51:53 +0000 (GMT) Received: from mail.digital-infotech.net (localhost [127.0.0.1]) by mail.digital-infotech.net (Postfix) with ESMTP id 437622E4102 for ; Wed, 12 Dec 2012 16:51:53 +0000 (GMT) X-DKIM: OpenDKIM Filter v2.5.0 mail.digital-infotech.net 437622E4102 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=digital-infotech.net; s=digital; t=1355331113; bh=DxVI2myHhx5hpwGBZlvF8DKJzjI8GJmT9393jolM0XY=; h=Date:Subject:From:To:Reply-To; b=dlQyFvDEBRu63mvOjf6KbX9CqWX0B8IeKFLv9pupVUWZuwtWnXCIYkpzdYimesG6O BGoMhCUx3q6umJlXqZBMRxymk5xNZwd7MXIwwBYRgj4Hq1msAzV1eV9ZDcN1GZNZNf cLyjRDU1vJdQdyHzpVtx4keAzBYDdwnlVZKolsaM= Received: from 41.211.28.7 (SquirrelMail authenticated user prabhpal@digital-infotech.net) by mail.digital-infotech.net with HTTP; Wed, 12 Dec 2012 16:51:53 -0000 Message-ID: <2689bf8086e9f8e3dff7255031ca4a75.squirrel@mail.digital-infotech.net> Date: Wed, 12 Dec 2012 16:51:53 -0000 Subject: Captive Portal & FreeRadius to use MySQL From: "Prabhpal S. Mavi" To: freebsd-hackers@freebsd.org User-Agent: SquirrelMail/1.4.22 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal X-Mailman-Approved-At: Wed, 12 Dec 2012 17:07:39 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: prabhpal@digital-infotech.net List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Dec 2012 16:51:56 -0000 Dear Users, i want Captive Portal & FreeRadius to use MySQL for authentication but am not sure how to install MySQL on PfSense Box. Can anyone advice how do we do when using MySQL with FreeRadius PfSense? Thanks From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 12 22:04:33 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AE735A0E for ; Wed, 12 Dec 2012 22:04:33 +0000 (UTC) (envelope-from rfg@tristatelogic.com) Received: from outgoing.tristatelogic.com (segfault.tristatelogic.com [69.62.255.118]) by mx1.freebsd.org (Postfix) with ESMTP id 7ABEB8FC13 for ; Wed, 12 Dec 2012 22:04:32 +0000 (UTC) Received: from segfault-nmh-helo.tristatelogic.com (localhost [127.0.0.1]) by segfault.tristatelogic.com (Postfix) with ESMTP id 574D250821; Wed, 12 Dec 2012 14:04:31 -0800 (PST) To: Mark Saad Subject: Re: Syslog-ng weirdness on 9.1-RELEASE In-Reply-To: Date: Wed, 12 Dec 2012 14:04:31 -0800 Message-ID: <43441.1355349871@tristatelogic.com> From: "Ronald F. Guilmette" Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Dec 2012 22:04:33 -0000 In message Mark Saad wrote: > I am wondering if anyone has seen this. I pulled down the 9.1-RELEASE >install media... That's very interesting. Where did you obtain that? >From where I am sitting, the freebsd web site is still only offering version 9.1-RC3. From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 12 22:11:50 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 386DFD52 for ; Wed, 12 Dec 2012 22:11:50 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wi0-f180.google.com (mail-wi0-f180.google.com [209.85.212.180]) by mx1.freebsd.org (Postfix) with ESMTP id B351D8FC13 for ; Wed, 12 Dec 2012 22:11:49 +0000 (UTC) Received: by mail-wi0-f180.google.com with SMTP id hj13so1038532wib.13 for ; Wed, 12 Dec 2012 14:11:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=Xs/yjLFYGhRukaq5UhsefCZjGYDXFaEMUWr+xKJj1iA=; b=0VhDlo/UWH1cZPpeADzfgMuIG1m8uYiZZw6nDKL5WpmqoSBRp027R10I1U4OIHUirj 0pDGqNLCwkhcc4AhZL8SBGKjagKKwCShZk4BOkmMxqt9xpYlfBF26fMuodImXWsjD3XJ L9X98cY6o4N6qOMkJgJN2zORk23s0hNOhYh7BPF+nWx1cyObzavuc6EKTK0INDdO9ddt kI09FtjhsxnGx9YuixV6DOW1TCcIUR4u5OJ8aThjEfxqbj5jjx+q2MzIAsoj4d8aygnJ HfvpJnjlwRGYcrfV25nsnLCv9GPoMa2eS7ChaXpzehn6eUyeWgMtBxd0SEWaGoP0/Npf J+2A== MIME-Version: 1.0 Received: by 10.180.102.101 with SMTP id fn5mr4349224wib.19.1355350308462; Wed, 12 Dec 2012 14:11:48 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.217.57.9 with HTTP; Wed, 12 Dec 2012 14:11:48 -0800 (PST) In-Reply-To: <43441.1355349871@tristatelogic.com> References: <43441.1355349871@tristatelogic.com> Date: Wed, 12 Dec 2012 14:11:48 -0800 X-Google-Sender-Auth: twKEXVI3h-ro1sCqyeAf4YCtNrw Message-ID: Subject: Re: Syslog-ng weirdness on 9.1-RELEASE From: Adrian Chadd To: "Ronald F. Guilmette" Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-hackers@freebsd.org" , Mark Saad X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Dec 2012 22:11:50 -0000 On 12 December 2012 14:04, Ronald F. Guilmette wrote: > > In message > Mark Saad wrote: > >> I am wondering if anyone has seen this. I pulled down the 9.1-RELEASE >>install media... > > That's very interesting. Where did you obtain that? > > From where I am sitting, the freebsd web site is still only offering > version 9.1-RC3. The images have to be pushed out to ftp-master and over to the mirrors before the release announcement goes out. Now, we've had to pull images at the last minute due to QA/testing issues; hence why we don't announce that images are available once they're uploaded. otherwise you may install an image that is actually pulled and replaced before the release.. adrian From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 12 22:17:03 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 08E36175 for ; Wed, 12 Dec 2012 22:17:03 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [89.206.35.99]) by mx1.freebsd.org (Postfix) with ESMTP id 134DC8FC1B for ; Wed, 12 Dec 2012 22:17:01 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5) with ESMTP id qBCMGuma001568; Wed, 12 Dec 2012 23:16:57 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5/Submit) with ESMTP id qBCMGuV6001565; Wed, 12 Dec 2012 23:16:56 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Wed, 12 Dec 2012 23:16:56 +0100 (CET) From: Wojciech Puchar To: Zaphod Beeblebrox Subject: Re: iSCSI vs. SMB with ZFS. In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Wed, 12 Dec 2012 23:16:57 +0100 (CET) Cc: FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Dec 2012 22:17:03 -0000 > about the same as the local disk for some operations --- faster for > some, slower for others. The workstation has 12G of memory and it's > my perception that iSCSI is heavily cached and that this enhances it's any REAL test means doing something that will not fit in cache. > But this is imperfect. The iSCSI disk reserves all of it's space and > the files on the disk are only accessible to the computer that mounts > it. it is even more imperfect. you layout one filesystem over another filesystem. not only degraded performance but you don't have parallel access to files on this "disk". > The performance on the SMB share is abysmal compared to the > performance on the iSCSI share. At the very least, there seems to be > little benifit to launching the same application twice --- which is > most likely windows fault. This is SMB protocol. sorry it is stupid. And it doesn't make real use of cache. This is how windows file sharing works. Fine if you just want to copy files. not fine if you work on them. > Will SMB always have significantly less performance than iSCSI coming depends what you do but yes SMB is not efficient. i am happy with SMB as it is enough to store users or shared documents. And it is quite fast on large file copy etc. but terrible on randomly accessing files. From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 12 22:26:19 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4A1B76FC; Wed, 12 Dec 2012 22:26:19 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [89.206.35.99]) by mx1.freebsd.org (Postfix) with ESMTP id 77C138FC0A; Wed, 12 Dec 2012 22:26:18 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5) with ESMTP id qBCMQGNF001473; Wed, 12 Dec 2012 23:26:16 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5/Submit) with ESMTP id qBCMQGXp001470; Wed, 12 Dec 2012 23:26:16 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Wed, 12 Dec 2012 23:26:16 +0100 (CET) From: Wojciech Puchar To: Stefan Esser Subject: Re: FreeBSD for serious performance? In-Reply-To: <50C889D3.1050404@freebsd.org> Message-ID: References: <20121211204323.310760@gmx.com> <50C889D3.1050404@freebsd.org> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Wed, 12 Dec 2012 23:26:16 +0100 (CET) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Dec 2012 22:26:19 -0000 > The cause of the low write performance is the disabled write cache. > Enabling the write cache is unsafe on SATA drives (with or without > NCQ), since they do not make any guarantees that nearby data is not > lost if power fails during a disk write. It never happened to me, > but there is a reason that SAS drives have less capacity, much lower > BER (one to two magnitudes) and are more expensive than SATA drives. interface have nothing to do. Both allows you to force writes now and then. > The solution to the performance problem is simple: Turn on the write > cache. If the data is valuable, then SAS is the solution to both the If data is valuable, regular and well done backup practice is the only solution. > would pay one developer hour. Asking Nvidia to release the confidential > documentation for their chip-set might help, but I doubt that there is > much interest to add support for NCQ to an obsolete chip-set, today, > unless you pay a developer (and even then ...). Even without this, i've never seen properly working NVidia hardware. ANY From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 12 22:28:03 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 19E69972 for ; Wed, 12 Dec 2012 22:28:03 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [89.206.35.99]) by mx1.freebsd.org (Postfix) with ESMTP id 53A608FC0C for ; Wed, 12 Dec 2012 22:28:01 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5) with ESMTP id qBCMRxDs001483; Wed, 12 Dec 2012 23:27:59 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5/Submit) with ESMTP id qBCMRx9p001480; Wed, 12 Dec 2012 23:27:59 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Wed, 12 Dec 2012 23:27:59 +0100 (CET) From: Wojciech Puchar To: Zaphod Beeblebrox Subject: Re: iSCSI vs. SMB with ZFS. In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Wed, 12 Dec 2012 23:27:59 +0100 (CET) Cc: FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Dec 2012 22:28:03 -0000 as you show your needs for unshared data for single workstation is in order of single large hard drive. reducing drive count on file server by one and connecting this one drive directly to workstation is the best solution From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 12 22:44:57 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2963E5EB; Wed, 12 Dec 2012 22:44:57 +0000 (UTC) (envelope-from rfg@tristatelogic.com) Received: from outgoing.tristatelogic.com (segfault.tristatelogic.com [69.62.255.118]) by mx1.freebsd.org (Postfix) with ESMTP id E91B08FC0A; Wed, 12 Dec 2012 22:44:56 +0000 (UTC) Received: from segfault-nmh-helo.tristatelogic.com (localhost [127.0.0.1]) by segfault.tristatelogic.com (Postfix) with ESMTP id 3A06A5081D; Wed, 12 Dec 2012 14:44:56 -0800 (PST) To: Adrian Chadd Subject: Re: Syslog-ng weirdness on 9.1-RELEASE In-Reply-To: Date: Wed, 12 Dec 2012 14:44:56 -0800 Message-ID: <43868.1355352296@tristatelogic.com> From: "Ronald F. Guilmette" Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Dec 2012 22:44:57 -0000 In message , Adrian Chadd wrote: >On 12 December 2012 14:04, Ronald F. Guilmette wrote: >> From where I am sitting, the freebsd web site is still only offering >> version 9.1-RC3. > >The images have to be pushed out to ftp-master and over to the mirrors >before the release announcement goes out. OK. I understand now. >Now, we've had to pull images at the last minute due to QA/testing >issues; hence why we don't announce that images are available once >they're uploaded. otherwise you may install an image that is actually >pulled and replaced before the release.. I understand. Anyway, this is a Good Thing... that you pulled back the release. Perhaps it will give some time to address ANOTHER bug that I think might be considered critical by some. Am I the only one for whom trying to install from a burnt 9.1-RC3 CD results in multiple irrecoverable READ ERRORS on the CD? At first when this happened I thought that I had just gotten a bad CD (media) so I burnt another and then got the exact same errors. I got those errors when trying to install from the 9.1-RC3 CD while using _three_ different CD/DVD/BluRay drives on _two_ different systems. In one of these three cases, the errors... which were limited to a half dozen or so in the other test cases... became an infinite loop of the same CD read error being reported to the console over and over and over again endlessly. In the two cases where there was just a sequence of a half dozen or so of these CD read errors (at one specific point in the install process) the install process _did_ seem to complete, despite the CD read errors reported on the console, but with one of the three CD/DVD drives I used to simply try to do a 9.1(-RC3) install, as I say, the thing when into an infinite loop of printing CD read errors and thus the install could not even be completed when using that particular CD/DVD drive. (And the drive in question is really _not_ that old, and continues to work well for everything else I ask it to do.) So, am I the only one on the planet who still installs FreeBSD the old- fashioned way, i.e. from an actual physical CD? That's the only plausible explanation that I can think of for why nobody (else) noticed this apparently serious problem in the release. Regards, rfg P.S. I am just going to download the install CD ISO for 9.0-RELEASE now and see if the same thing happens with that. From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 12 23:49:14 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B588A993; Wed, 12 Dec 2012 23:49:14 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [89.206.35.99]) by mx1.freebsd.org (Postfix) with ESMTP id 177898FC08; Wed, 12 Dec 2012 23:49:13 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5) with ESMTP id qBCNnBaP001793; Thu, 13 Dec 2012 00:49:11 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5/Submit) with ESMTP id qBCNnBWG001790; Thu, 13 Dec 2012 00:49:11 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Thu, 13 Dec 2012 00:49:10 +0100 (CET) From: Wojciech Puchar To: "Ronald F. Guilmette" Subject: Re: Syslog-ng weirdness on 9.1-RELEASE In-Reply-To: <43868.1355352296@tristatelogic.com> Message-ID: References: <43868.1355352296@tristatelogic.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Thu, 13 Dec 2012 00:49:11 +0100 (CET) Cc: "freebsd-hackers@freebsd.org" , Adrian Chadd X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Dec 2012 23:49:14 -0000 > results in multiple irrecoverable READ ERRORS on the CD? not just 9.* but with some machines there are problems with CD/DVD drive switch to loader prompt and type set hw.ata.atapi_dma=0 worked with 8.* > So, am I the only one on the planet who still installs FreeBSD the old- > fashioned way, i.e. from an actual physical CD? That's the only plausible maybe. i don't use installer at all. i boot working FreeBSD system by any means (live CD, pendrive, NFS) and unpack my builds from tarballs. From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 13 00:33:57 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 218CF8AA for ; Thu, 13 Dec 2012 00:33:57 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 7D89C8FC12 for ; Thu, 13 Dec 2012 00:33:55 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ap8EABMiyVCDaFvO/2dsb2JhbABFFoYiuEtzgiUjBIEWGQIEVYgqDJwtjlGSe4xLgzCBEwOIYIYlhwSBHI8sgxGCBA X-IronPort-AV: E=Sophos;i="4.84,269,1355115600"; d="scan'208";a="4591900" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu.net.uoguelph.ca with ESMTP; 12 Dec 2012 19:33:41 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 5D738B406B for ; Wed, 12 Dec 2012 19:33:41 -0500 (EST) Date: Wed, 12 Dec 2012 19:33:41 -0500 (EST) From: Rick Macklem To: freebsd-hackers@freebsd.org Message-ID: <1860364947.1365748.1355358821312.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <503182717.1365734.1355358788463.JavaMail.root@erie.cs.uoguelph.ca> Subject: request for review: gssd patch for alternate cred cache files MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_1365747_1486623735.1355358821309" X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Linux)/6.0.10_GA_2692) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Dec 2012 00:33:57 -0000 ------=_Part_1365747_1486623735.1355358821309 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Hi, A couple of people have reported problems using NFS mounts with sec=krb5 because the version of sshd they use doesn't use credential cache files named /tmp/krb5cc_. The attached patch modifies the gssd so that it roughly emulates what the gssd used by most Linux distros does when a new "-s" option is used. This has been tested by the reporters and fixed their issue. Would someone like to review this? rick ps: The patch can also be found at http://people.freebsd.org/~rmacklem/gssd-ccache.patch ------=_Part_1365747_1486623735.1355358821309 Content-Type: text/x-patch; name=gssd-ccache.patch Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename=gssd-ccache.patch LS0tIHVzci5zYmluL2dzc2QvZ3NzZC5jLnNhdjIJMjAxMi0xMC0wOCAxNjo0OTo1MC4wMDAwMDAw MDAgLTA0MDAKKysrIHVzci5zYmluL2dzc2QvZ3NzZC5jCTIwMTItMTItMTIgMTk6MTk6NTEuMDAw MDAwMDAwIC0wNTAwCkBAIC0zNSw3ICszNSw5IEBAIF9fRkJTRElEKCIkRnJlZUJTRDogaGVhZC91 c3Iuc2Jpbi9nc3NkL2cKICNpbmNsdWRlIDxzeXMvcXVldWUuaD4KICNpbmNsdWRlIDxzeXMvc3lz bG9nLmg+CiAjaW5jbHVkZSA8Y3R5cGUuaD4KKyNpbmNsdWRlIDxkaXJlbnQuaD4KICNpbmNsdWRl IDxlcnIuaD4KKyNpbmNsdWRlIDxrcmI1Lmg+CiAjaW5jbHVkZSA8cHdkLmg+CiAjaW5jbHVkZSA8 c3RkaW8uaD4KICNpbmNsdWRlIDxzdGRsaWIuaD4KQEAgLTY0LDggKzY2LDEyIEBAIGludCBnc3Nf cmVzb3VyY2VfY291bnQ7CiB1aW50MzJfdCBnc3NfbmV4dF9pZDsKIHVpbnQzMl90IGdzc19zdGFy dF90aW1lOwogaW50IGRlYnVnX2xldmVsOworc3RhdGljIGNoYXIgY2NmaWxlX2Rpcmxpc3RbUEFU SF9NQVggKyAxXSwgY2NmaWxlX3N1YnN0cmluZ1tOQU1FX01BWCArIDFdOworc3RhdGljIGNoYXIg cHJlZl9yZWFsbVsxMDI0XTsKIAogc3RhdGljIHZvaWQgZ3NzZF9sb2FkX21lY2godm9pZCk7Citz dGF0aWMgaW50IGZpbmRfY2NhY2hlX2ZpbGUoY29uc3QgY2hhciAqLCB1aWRfdCwgY2hhciAqKTsK K3N0YXRpYyBpbnQgaXNfYV92YWxpZF90Z3RfY2FjaGUoY29uc3QgY2hhciAqLCB1aWRfdCwgaW50 ICosIHRpbWVfdCAqKTsKIAogZXh0ZXJuIHZvaWQgZ3NzZF8xKHN0cnVjdCBzdmNfcmVxICpycXN0 cCwgU1ZDWFBSVCAqdHJhbnNwKTsKIGV4dGVybiBpbnQgZ3NzZF9zeXNjYWxsKGNoYXIgKnBhdGgp OwpAQCAtODIsMTQgKzg4LDQ1IEBAIG1haW4oaW50IGFyZ2MsIGNoYXIgKiphcmd2KQogCWludCBm ZCwgb2xkbWFzaywgY2gsIGRlYnVnOwogCVNWQ1hQUlQgKnhwcnQ7CiAKKwkvKgorCSAqIEluaXRp YWxpemUgdGhlIGNyZWRlbnRpYWwgY2FjaGUgZmlsZSBuYW1lIHN1YnN0cmluZyBhbmQgdGhlCisJ ICogc2VhcmNoIGRpcmVjdG9yeSBsaXN0LgorCSAqLworCXN0cmxjcHkoY2NmaWxlX3N1YnN0cmlu ZywgImtyYjVjY18iLCBzaXplb2YoY2NmaWxlX3N1YnN0cmluZykpOworCWNjZmlsZV9kaXJsaXN0 WzBdID0gJ1wwJzsKKwlwcmVmX3JlYWxtWzBdID0gJ1wwJzsKIAlkZWJ1ZyA9IDA7Ci0Jd2hpbGUg KChjaCA9IGdldG9wdChhcmdjLCBhcmd2LCAiZCIpKSAhPSAtMSkgeworCXdoaWxlICgoY2ggPSBn ZXRvcHQoYXJnYywgYXJndiwgImRzOmM6cjoiKSkgIT0gLTEpIHsKIAkJc3dpdGNoIChjaCkgewog CQljYXNlICdkJzoKIAkJCWRlYnVnX2xldmVsKys7CiAJCQlicmVhazsKKwkJY2FzZSAncyc6CisJ CQkvKgorCQkJICogU2V0IHRoZSBkaXJlY3Rvcnkgc2VhcmNoIGxpc3QuIFRoaXMgZW5hYmxlcyB1 c2Ugb2YKKwkJCSAqIGZpbmRfY2NhY2hlX2ZpbGUoKSB0byBzZWFyY2ggdGhlIGRpcmVjdG9yaWVz IGZvciBhCisJCQkgKiBzdWl0YWJsZSBjcmVkZW50aWFscyBjYWNoZSBmaWxlLgorCQkJICovCisJ CQlzdHJsY3B5KGNjZmlsZV9kaXJsaXN0LCBvcHRhcmcsIHNpemVvZihjY2ZpbGVfZGlybGlzdCkp OworCQkJYnJlYWs7CisJCWNhc2UgJ2MnOgorCQkJLyoKKwkJCSAqIFNwZWNpZnkgYSBub24tZGVm YXVsdCBjcmVkZW50aWFsIGNhY2hlIGZpbGUKKwkJCSAqIHN1YnN0cmluZy4KKwkJCSAqLworCQkJ c3RybGNweShjY2ZpbGVfc3Vic3RyaW5nLCBvcHRhcmcsCisJCQkgICAgc2l6ZW9mKGNjZmlsZV9z dWJzdHJpbmcpKTsKKwkJCWJyZWFrOworCQljYXNlICdyJzoKKwkJCS8qCisJCQkgKiBTZXQgdGhl IHByZWZlcnJlZCByZWFsbSBmb3IgdGhlIGNyZWRlbnRpYWwgY2FjaGUgdGd0LgorCQkJICovCisJ CQlzdHJsY3B5KHByZWZfcmVhbG0sIG9wdGFyZywgc2l6ZW9mKHByZWZfcmVhbG0pKTsKKwkJCWJy ZWFrOwogCQlkZWZhdWx0OgotCQkJZnByaW50ZihzdGRlcnIsICJ1c2FnZTogJXMgWy1kXVxuIiwg YXJndlswXSk7CisJCQlmcHJpbnRmKHN0ZGVyciwKKwkJCSAgICAidXNhZ2U6ICVzIFstZF0gWy1z IGRpci1saXN0XSBbLWMgZmlsZS1zdWJzdHJpbmddIgorCQkJICAgICIgWy1yIHByZWZlcnJlZC1y ZWFsbV1cbiIsIGFyZ3ZbMF0pOwogCQkJZXhpdCgxKTsKIAkJCWJyZWFrOwogCQl9CkBAIC0yNjcs MTMgKzMwNCwzNiBAQCBnc3NkX2luaXRfc2VjX2NvbnRleHRfMV9zdmMoaW5pdF9zZWNfY29uCiAJ Z3NzX2NyZWRfaWRfdCBjcmVkID0gR1NTX0NfTk9fQ1JFREVOVElBTDsKIAlnc3NfY3R4X2lkX3Qg Y3R4ID0gR1NTX0NfTk9fQ09OVEVYVDsKIAlnc3NfbmFtZV90IG5hbWUgPSBHU1NfQ19OT19OQU1F OwotCWNoYXIgY2NuYW1lW3N0cmxlbigiRklMRTovdG1wL2tyYjVjY18iKSArIDYgKyAxXTsKKwlj aGFyIGNjbmFtZVtQQVRIX01BWCArIDUgKyAxXSwgKmNwLCAqY3AyOworCWludCBnb3RvbmU7CiAK LQlzbnByaW50ZihjY25hbWUsIHNpemVvZihjY25hbWUpLCAiRklMRTovdG1wL2tyYjVjY18lZCIs Ci0JICAgIChpbnQpIGFyZ3AtPnVpZCk7CisJbWVtc2V0KHJlc3VsdCwgMCwgc2l6ZW9mKCpyZXN1 bHQpKTsKKwlpZiAoY2NmaWxlX2Rpcmxpc3RbMF0gIT0gJ1wwJyAmJiBhcmdwLT5jcmVkID09IDAp IHsKKwkJZ290b25lID0gMDsKKwkJY3AgPSBjY2ZpbGVfZGlybGlzdDsKKwkJZG8geworCQkJY3Ay ID0gc3RyY2hyKGNwLCAnOicpOworCQkJaWYgKGNwMiAhPSBOVUxMKQorCQkJCSpjcDIgPSAnXDAn OworCQkJZ290b25lID0gZmluZF9jY2FjaGVfZmlsZShjcCwgYXJncC0+dWlkLCBjY25hbWUpOwor CQkJaWYgKGdvdG9uZSAhPSAwKQorCQkJCWJyZWFrOworCQkJaWYgKGNwMiAhPSBOVUxMKQorCQkJ CSpjcDIrKyA9ICc6JzsKKwkJCWNwID0gY3AyOworCQl9IHdoaWxlIChjcCAhPSBOVUxMICYmICpj cCAhPSAnXDAnKTsKKwkJaWYgKGdvdG9uZSA9PSAwKQorCQkJc25wcmludGYoY2NuYW1lLCBzaXpl b2YoY2NuYW1lKSwgIkZJTEU6L3RtcC9rcmI1Y2NfJWQiLAorCQkJICAgIChpbnQpIGFyZ3AtPnVp ZCk7CisJfSBlbHNlCisJCS8qCisJCSAqIElmIHRoZXJlIHdhc24ndCBhICItcyIgb3B0aW9uIG9y IHRoZSBjcmVkZW50aWFscyBoYXZlCisJCSAqIGJlZW4gcHJvdmlkZWQgYXMgYW4gYXJndW1lbnQs IGRvIGl0IHRoZSBvbGQgd2F5LgorCQkgKi8KKwkJc25wcmludGYoY2NuYW1lLCBzaXplb2YoY2Nu YW1lKSwgIkZJTEU6L3RtcC9rcmI1Y2NfJWQiLAorCQkgICAgKGludCkgYXJncC0+dWlkKTsKIAlz ZXRlbnYoIktSQjVDQ05BTUUiLCBjY25hbWUsIFRSVUUpOwogCi0JbWVtc2V0KHJlc3VsdCwgMCwg c2l6ZW9mKCpyZXN1bHQpKTsKIAlpZiAoYXJncC0+Y3JlZCkgewogCQljcmVkID0gZ3NzZF9maW5k X3Jlc291cmNlKGFyZ3AtPmNyZWQpOwogCQlpZiAoIWNyZWQpIHsKQEAgLTUxNiwxMyArNTc2LDM3 IEBAIGdzc2RfYWNxdWlyZV9jcmVkXzFfc3ZjKGFjcXVpcmVfY3JlZF9hcmcKIHsKIAlnc3NfbmFt ZV90IGRlc2lyZWRfbmFtZSA9IEdTU19DX05PX05BTUU7CiAJZ3NzX2NyZWRfaWRfdCBjcmVkOwot CWNoYXIgY2NuYW1lW3N0cmxlbigiRklMRTovdG1wL2tyYjVjY18iKSArIDYgKyAxXTsKKwljaGFy IGNjbmFtZVtQQVRIX01BWCArIDUgKyAxXSwgKmNwLCAqY3AyOworCWludCBnb3RvbmU7CiAKLQlz bnByaW50ZihjY25hbWUsIHNpemVvZihjY25hbWUpLCAiRklMRTovdG1wL2tyYjVjY18lZCIsCi0J ICAgIChpbnQpIGFyZ3AtPnVpZCk7CisJbWVtc2V0KHJlc3VsdCwgMCwgc2l6ZW9mKCpyZXN1bHQp KTsKKwlpZiAoY2NmaWxlX2Rpcmxpc3RbMF0gIT0gJ1wwJyAmJiBhcmdwLT5kZXNpcmVkX25hbWUg PT0gMCkgeworCQlnb3RvbmUgPSAwOworCQljcCA9IGNjZmlsZV9kaXJsaXN0OworCQlkbyB7CisJ CQljcDIgPSBzdHJjaHIoY3AsICc6Jyk7CisJCQlpZiAoY3AyICE9IE5VTEwpCisJCQkJKmNwMiA9 ICdcMCc7CisJCQlnb3RvbmUgPSBmaW5kX2NjYWNoZV9maWxlKGNwLCBhcmdwLT51aWQsIGNjbmFt ZSk7CisJCQlpZiAoZ290b25lICE9IDApCisJCQkJYnJlYWs7CisJCQlpZiAoY3AyICE9IE5VTEwp CisJCQkJKmNwMisrID0gJzonOworCQkJY3AgPSBjcDI7CisJCX0gd2hpbGUgKGNwICE9IE5VTEwg JiYgKmNwICE9ICdcMCcpOworCQlpZiAoZ290b25lID09IDApCisJCQlzbnByaW50ZihjY25hbWUs IHNpemVvZihjY25hbWUpLCAiRklMRTovdG1wL2tyYjVjY18lZCIsCisJCQkgICAgKGludCkgYXJn cC0+dWlkKTsKKwl9IGVsc2UKKwkJLyoKKwkJICogSWYgdGhlcmUgd2Fzbid0IGEgIi1zIiBvcHRp b24gb3IgdGhlIG5hbWUgaGFzCisJCSAqIGJlZW4gcHJvdmlkZWQgYXMgYW4gYXJndW1lbnQsIGRv IGl0IHRoZSBvbGQgd2F5LgorCQkgKiAoVGhlIG5hbWUgaXMgcHJvdmlkZWQgZm9yIGhvc3QgYmFz ZWQgaW5pdGlhdG9yIGNyZWRlbnRpYWxzLikKKwkJICovCisJCXNucHJpbnRmKGNjbmFtZSwgc2l6 ZW9mKGNjbmFtZSksICJGSUxFOi90bXAva3JiNWNjXyVkIiwKKwkJICAgIChpbnQpIGFyZ3AtPnVp ZCk7CiAJc2V0ZW52KCJLUkI1Q0NOQU1FIiwgY2NuYW1lLCBUUlVFKTsKIAotCW1lbXNldChyZXN1 bHQsIDAsIHNpemVvZigqcmVzdWx0KSk7CiAJaWYgKGFyZ3AtPmRlc2lyZWRfbmFtZSkgewogCQlk ZXNpcmVkX25hbWUgPSBnc3NkX2ZpbmRfcmVzb3VyY2UoYXJncC0+ZGVzaXJlZF9uYW1lKTsKIAkJ aWYgKCFkZXNpcmVkX25hbWUpIHsKQEAgLTYzMSwzICs3MTUsMTcyIEBAIGdzc2RfMV9mcmVlcmVz dWx0KFNWQ1hQUlQgKnRyYW5zcCwgeGRycHIKIAogCXJldHVybiAoVFJVRSk7CiB9CisKKy8qCisg KiBTZWFyY2ggYSBkaXJlY3RvcnkgZm9yIHRoZSBtb3N0IGxpa2VseSBjYW5kaWRhdGUgdG8gYmUg dXNlZCBhcyB0aGUKKyAqIGNyZWRlbnRpYWwgY2FjaGUgZm9yIGEgdWlkLiBJZiBzdWNjZXNzZnVs LCByZXR1cm4gMSBhbmQgZmlsbCB0aGUKKyAqIGZpbGUncyBwYXRoIGlkIGludG8gInJwYXRoIi4g T3RoZXJ3aXNlLCByZXR1cm4gMC4KKyAqLworc3RhdGljIGludAorZmluZF9jY2FjaGVfZmlsZShj b25zdCBjaGFyICpkaXJwYXRoLCB1aWRfdCB1aWQsIGNoYXIgKnJwYXRoKQoreworCURJUiAqZGly cDsKKwlzdHJ1Y3QgZGlyZW50ICpkcDsKKwlzdHJ1Y3Qgc3RhdCBzYjsKKwl0aW1lX3QgZXhwdGlt ZSwgb2V4cHRpbWU7CisJaW50IGdvdG9uZSwgbGVuLCByYXRpbmcsIG9yYXRpbmc7CisJY2hhciBu YW1lcGF0aFtQQVRIX01BWCArIDUgKyAxXTsKKwljaGFyIHJldHBhdGhbUEFUSF9NQVggKyA1ICsg MV07CisKKwlkaXJwID0gb3BlbmRpcihkaXJwYXRoKTsKKwlpZiAoZGlycCA9PSBOVUxMKQorCQly ZXR1cm4gKDApOworCWdvdG9uZSA9IDA7CisJb3JhdGluZyA9IDA7CisJb2V4cHRpbWUgPSAwOwor CXdoaWxlICgoZHAgPSByZWFkZGlyKGRpcnApKSAhPSBOVUxMKSB7CisJCWxlbiA9IHNucHJpbnRm KG5hbWVwYXRoLCBzaXplb2YobmFtZXBhdGgpLCAiJXMvJXMiLCBkaXJwYXRoLAorCQkgICAgZHAt PmRfbmFtZSk7CisJCWlmIChsZW4gPCBzaXplb2YobmFtZXBhdGgpICYmCisJCSAgICBzdHJzdHIo ZHAtPmRfbmFtZSwgY2NmaWxlX3N1YnN0cmluZykgIT0gTlVMTCAmJgorCQkgICAgbHN0YXQobmFt ZXBhdGgsICZzYikgPj0gMCAmJgorCQkgICAgc2Iuc3RfdWlkID09IHVpZCAmJgorCQkgICAgU19J U1JFRyhzYi5zdF9tb2RlKSkgeworCQkJbGVuID0gc25wcmludGYobmFtZXBhdGgsIHNpemVvZihu YW1lcGF0aCksICJGSUxFOiVzLyVzIiwKKwkJCSAgICBkaXJwYXRoLCBkcC0+ZF9uYW1lKTsKKwkJ CWlmIChsZW4gPCBzaXplb2YobmFtZXBhdGgpICYmCisJCQkgICAgaXNfYV92YWxpZF90Z3RfY2Fj aGUobmFtZXBhdGgsIHVpZCwgJnJhdGluZywKKwkJCSAgICAmZXhwdGltZSkgIT0gMCkgeworCQkJ CWlmIChnb3RvbmUgPT0gMCB8fCByYXRpbmcgPiBvcmF0aW5nIHx8CisJCQkJICAgIChyYXRpbmcg PT0gb3JhdGluZyAmJiBleHB0aW1lID4gb2V4cHRpbWUpKSB7CisJCQkJCW9yYXRpbmcgPSByYXRp bmc7CisJCQkJCW9leHB0aW1lID0gZXhwdGltZTsKKwkJCQkJc3RyY3B5KHJldHBhdGgsIG5hbWVw YXRoKTsKKwkJCQkJZ290b25lID0gMTsKKwkJCQl9CisJCQl9CisJCX0KKwl9CisJY2xvc2VkaXIo ZGlycCk7CisJaWYgKGdvdG9uZSAhPSAwKSB7CisJCXN0cmNweShycGF0aCwgcmV0cGF0aCk7CisJ CXJldHVybiAoMSk7CisJfQorCXJldHVybiAoMCk7Cit9CisKKy8qCisgKiBUcnkgdG8gZGV0ZXJt aW5lIGlmIHRoZSBmaWxlIGlzIGEgdmFsaWQgdGd0IGNhY2hlIGZpbGUuCisgKiBDaGVjayB0aGF0 IHRoZSBmaWxlIGhhcyBhIHZhbGlkIHRndCBmb3IgYSBwcmluY2lwYWwuCisgKiBJZiBpdCBkb2Vz LCByZXR1cm4gMSwgb3RoZXJ3aXNlIHJldHVybiAwLgorICogSXQgYWxzbyByZXR1cm5zIGEgInJh dGluZyIgYW5kIHRoZSBleHBpcnkgdGltZSBmb3IgdGhlIFRHVCwgd2hlbiBmb3VuZC4KKyAqIFRo aXMgInJhdGluZyIgaXMgaGlnaGVyIGJhc2VkIG9uIGhldXJpc3RpY3MgdGhhdCBtYWtlIGl0IG1v cmUKKyAqIGxpa2VseSB0byBiZSB0aGUgY29ycmVjdCBjcmVkZW50aWFsIGNhY2hlIGZpbGUgdG8g dXNlLiBJdCBjYW4KKyAqIGJlIHVzZWQgYnkgdGhlIGNhbGxlciwgYWxvbmcgd2l0aCBleHBpcnkg dGltZSwgdG8gc2VsZWN0IGZyb20KKyAqIG11bHRpcGxlIGNyZWRlbnRpYWwgY2FjaGUgZmlsZXMu CisgKi8KK3N0YXRpYyBpbnQKK2lzX2FfdmFsaWRfdGd0X2NhY2hlKGNvbnN0IGNoYXIgKmZpbGVw YXRoLCB1aWRfdCB1aWQsIGludCAqcmV0cmF0aW5nLAorICAgIHRpbWVfdCAqcmV0ZXhwdGltZSkK K3sKKwlrcmI1X2NvbnRleHQgY29udGV4dDsKKwlrcmI1X3ByaW5jaXBhbCBwcmluYzsKKwlrcmI1 X2NjYWNoZSBjY2FjaGU7CisJa3JiNV9lcnJvcl9jb2RlIHJldHZhbDsKKwlrcmI1X2NjX2N1cnNv ciBjdXJzZTsKKwlrcmI1X2NyZWRzIGtyYmNyZWQ7CisJaW50IGdvdG9uZSwgb3JhdGluZywgcmF0 aW5nLCByZXQ7CisJc3RydWN0IHBhc3N3ZCAqcHc7CisJY2hhciAqY3AsICpjcDIsICpwbmFtZTsK Kwl0aW1lX3QgZXhwdGltZTsKKworCS8qIEZpbmQgYSBsaWtlbHkgbmFtZSBmb3IgdGhlIHVpZCBw cmluY2lwYWwuICovCisJcHcgPSBnZXRwd3VpZCh1aWQpOworCisJLyoKKwkgKiBEbyBhIGJ1bmNo IG9mIGtyYjUgbGlicmFyeSBzdHVmZiB0byB0cnkgYW5kIGRldGVybWluZSBpZgorCSAqIHRoaXMg ZmlsZSBpcyBhIGNyZWRlbnRpYWxzIGNhY2hlIHdpdGggYW4gYXBwcm9wcmlhdGUgVEdUCisJICog aW4gaXQuCisJICovCisJcmV0dmFsID0ga3JiNV9pbml0X2NvbnRleHQoJmNvbnRleHQpOworCWlm IChyZXR2YWwgIT0gMCkKKwkJcmV0dXJuICgwKTsKKwlyZXR2YWwgPSBrcmI1X2NjX3Jlc29sdmUo Y29udGV4dCwgZmlsZXBhdGgsICZjY2FjaGUpOworCWlmIChyZXR2YWwgIT0gMCkgeworCQlrcmI1 X2ZyZWVfY29udGV4dChjb250ZXh0KTsKKwkJcmV0dXJuICgwKTsKKwl9CisJcmV0ID0gMDsKKwlv cmF0aW5nID0gMDsKKwlleHB0aW1lID0gMDsKKwlyZXR2YWwgPSBrcmI1X2NjX3N0YXJ0X3NlcV9n ZXQoY29udGV4dCwgY2NhY2hlLCAmY3Vyc2UpOworCWlmIChyZXR2YWwgPT0gMCkgeworCQl3aGls ZSAoKHJldHZhbCA9IGtyYjVfY2NfbmV4dF9jcmVkKGNvbnRleHQsIGNjYWNoZSwgJmN1cnNlLAor CQkgICAgJmtyYmNyZWQpKSA9PSAwKSB7CisJCQlnb3RvbmUgPSAwOworCQkJcmF0aW5nID0gMDsK KwkJCXJldHZhbCA9IGtyYjVfdW5wYXJzZV9uYW1lKGNvbnRleHQsIGtyYmNyZWQuc2VydmVyLAor CQkJICAgICZwbmFtZSk7CisJCQlpZiAocmV0dmFsID09IDApIHsKKwkJCQljcCA9IHN0cmNocihw bmFtZSwgJy8nKTsKKwkJCQlpZiAoY3AgIT0gTlVMTCkgeworCQkJCQkqY3ArKyA9ICdcMCc7CisJ CQkJCWlmIChzdHJjbXAocG5hbWUsICJrcmJ0Z3QiKSA9PSAwICYmCisJCQkJCSAgICBrcmJjcmVk LnRpbWVzLmVuZHRpbWUgPiB0aW1lKE5VTEwpCisJCQkJCSAgICApIHsKKwkJCQkJCWdvdG9uZSA9 IDE7CisJCQkJCQkvKgorCQkJCQkJICogVGVzdCB0byBzZWUgaWYgdGhpcyBpcyBhCisJCQkJCQkg KiB0Z3QgZm9yIGNyb3NzLXJlYWxtIGF1dGguCisJCQkJCQkgKiBSYXRlIGl0IGhpZ2hlciwgaWYg aXQgaXMgbm90LgorCQkJCQkJICovCisJCQkJCQljcDIgPSBzdHJjaHIoY3AsICdAJyk7CisJCQkJ CQlpZiAoY3AyICE9IE5VTEwpIHsKKwkJCQkJCQkqY3AyKysgPSAnXDAnOworCQkJCQkJCWlmIChz dHJjbXAoY3AsIGNwMikgPT0KKwkJCQkJCQkgICAgMCkKKwkJCQkJCQkJcmF0aW5nKys7CisJCQkJ CQl9CisJCQkJCX0KKwkJCQl9CisJCQkJZnJlZShwbmFtZSk7CisJCQl9CisJCQlpZiAoZ290b25l ICE9IDApIHsKKwkJCQlyZXR2YWwgPSBrcmI1X3VucGFyc2VfbmFtZShjb250ZXh0LAorCQkJCSAg ICBrcmJjcmVkLmNsaWVudCwgJnBuYW1lKTsKKwkJCQlpZiAocmV0dmFsID09IDApIHsKKwkJCQkJ Y3AgPSBzdHJjaHIocG5hbWUsICdAJyk7CisJCQkJCWlmIChjcCAhPSBOVUxMKSB7CisJCQkJCQkq Y3ArKyA9ICdcMCc7CisJCQkJCQlpZiAocHcgIT0gTlVMTCAmJiBzdHJjbXAocG5hbWUsCisJCQkJ CQkgICAgcHctPnB3X25hbWUpID09IDApCisJCQkJCQkJcmF0aW5nKys7CisJCQkJCQlpZiAoc3Ry Y2hyKHBuYW1lLCAnLycpID09IE5VTEwpCisJCQkJCQkJcmF0aW5nKys7CisJCQkJCQlpZiAocHJl Zl9yZWFsbVswXSAhPSAnXDAnICYmCisJCQkJCQkgICAgc3RyY21wKGNwLCBwcmVmX3JlYWxtKSA9 PSAwKQorCQkJCQkJCXJhdGluZysrOworCQkJCQl9CisJCQkJfQorCQkJCWZyZWUocG5hbWUpOwor CQkJCWlmIChyYXRpbmcgPiBvcmF0aW5nKSB7CisJCQkJCW9yYXRpbmcgPSByYXRpbmc7CisJCQkJ CWV4cHRpbWUgPSBrcmJjcmVkLnRpbWVzLmVuZHRpbWU7CisJCQkJfSBlbHNlIGlmIChyYXRpbmcg PT0gb3JhdGluZyAmJgorCQkJCSAgICBrcmJjcmVkLnRpbWVzLmVuZHRpbWUgPiBleHB0aW1lKQor CQkJCQlleHB0aW1lID0ga3JiY3JlZC50aW1lcy5lbmR0aW1lOworCQkJCXJldCA9IDE7CisJCQl9 CisJCQlrcmI1X2ZyZWVfY3JlZF9jb250ZW50cyhjb250ZXh0LCAma3JiY3JlZCk7CisJCX0KKwkJ a3JiNV9jY19lbmRfc2VxX2dldChjb250ZXh0LCBjY2FjaGUsICZjdXJzZSk7CisJfQorCWtyYjVf Y2NfY2xvc2UoY29udGV4dCwgY2NhY2hlKTsKKwlrcmI1X2ZyZWVfY29udGV4dChjb250ZXh0KTsK KwlpZiAocmV0ICE9IDApIHsKKwkJKnJldHJhdGluZyA9IG9yYXRpbmc7CisJCSpyZXRleHB0aW1l ID0gZXhwdGltZTsKKwl9CisJcmV0dXJuIChyZXQpOworfQorCi0tLSB1c3Iuc2Jpbi9nc3NkL2dz c2QuOC5zYXYJMjAxMi0xMC0wOCAxNzo1NDowMi4wMDAwMDAwMDAgLTA0MDAKKysrIHVzci5zYmlu L2dzc2QvZ3NzZC44CTIwMTItMTItMTIgMTk6MDc6MTEuMDAwMDAwMDAwIC0wNTAwCkBAIC0yNSw3 ICsyNSw3IEBACiAuXCIKIC5cIiAkRnJlZUJTRDogaGVhZC91c3Iuc2Jpbi9nc3NkL2dzc2QuOCAy MzY1MDAgMjAxMi0wNi0wMyAwNjo1Nzo0N1ogam9lbCAkCiAuXCIKLS5EZCBOb3ZlbWJlciA1LCAy MDA4CisuRGQgT2N0IDgsIDIwMTIKIC5EdCBHU1NEIDgKIC5PcwogLlNoIE5BTUUKQEAgLTM0LDYg KzM0LDkgQEAKIC5TaCBTWU5PUFNJUwogLk5tCiAuT3AgRmwgZAorLk9wIEZsIHMgQXIgZGlyLWxp c3QKKy5PcCBGbCBjIEFyIGZpbGUtc3Vic3RyaW5nCisuT3AgRmwgciBBciBwcmVmZXJyZWQtcmVh bG0KIC5TaCBERVNDUklQVElPTgogVGhlCiAuTm0KQEAgLTQ2LDYgKzQ5LDI1IEBAIFJ1biBpbiBk ZWJ1ZyBtb2RlLgogSW4gdGhpcyBtb2RlLAogLk5tCiB3aWxsIG5vdCBmb3JrIHdoZW4gaXQgc3Rh cnRzLgorLkl0IEZsIHMgQXIgZGlyLWxpc3QKK0xvb2sgZm9yIGFuIGFwcHJvcHJpYXRlIGNyZWRl bnRpYWwgY2FjaGUgZmlsZSBpbiB0aGlzIGxpc3Qgb2YgZGlyZWN0b3JpZXMuCitUaGUgbGlzdCBz aG91bGQgYmUgZnVsbCBwYXRobmFtZXMgZnJvbSByb290LCBzZXBhcmF0ZWQgYnkgJzonIGNoYXJh Y3RlcnMuCitVc3VhbGx5IHRoaXMgbGlzdCB3aWxsIHNpbXBseSBiZSAiL3RtcCIuCisuSXQgRmwg YyBBciBmaWxlLXN1YnN0cmluZworU2V0IGEgZmlsZS1zdWJzdHJpbmcgZm9yIHRoZSBjcmVkZW50 aWFsIGNhY2hlIGZpbGUgbmFtZXMuCitPbmx5IGZpbGVzIHdpdGggdGhpcyBzdWJzdHJpbmcgZW1i ZWRkZWQgaW4gdGhlaXIgbmFtZXMgd2lsbCBiZQorc2VsZWN0ZWQgYXMgY2FuZGlkYXRlcyB3aGVu IHRoZQorLkZsIHMKK2hhcyBiZWVuIHNwZWNpZmllZC4KK0lmIG5vdCBzcGVjaWZpZWQsIGl0IGRl ZmF1bHRzIHRvICJrcmI1Y2NfIi4KKy5JdCBGbCByIEFyIHByZWZlcnJlZC1yZWFsbQorU2V0IGEg cHJlZmVycmVkIEtlcmJlcm9zIHJlYWxtIGZvciB0aGUgc2VhcmNoIG9mIHRoZSBkaXJlY3Rvcnkg bGlzdCBmb3IKK2EgY3JlZGVudGlhbHMgY2FjaGUgZmlsZS4KK1doZW4gc2V0LCBmaWxlcyB3aXRo IFRHVCBjcmVkZW50aWFscyBmb3IgdGhpcyByZWFsbSB3aWxsIGJlIHNlbGVjdGVkIG92ZXIKK290 aGVyIGNyZWRlbnRpYWwgZmlsZXMuCitUaGlzIG9wdGlvbiBpcyBvbmx5IG1lYW5pbmdmdWwgd2l0 aCB0aGUKKy5GbCBzCitvcHRpb24gaGFzIGJlZW4gc3BlY2lmaWVkLgogLkVsCiAuU2ggRklMRVMK IC5CbCAtdGFnIC13aWR0aCAiLlBhIC9ldGMva3JiNS5rZXl0YWIiIC1jb21wYWN0Cg== ------=_Part_1365747_1486623735.1355358821309-- From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 13 01:14:25 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B8200B2A for ; Thu, 13 Dec 2012 01:14:25 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: from mail-lb0-f182.google.com (mail-lb0-f182.google.com [209.85.217.182]) by mx1.freebsd.org (Postfix) with ESMTP id 3650D8FC16 for ; Thu, 13 Dec 2012 01:14:24 +0000 (UTC) Received: by mail-lb0-f182.google.com with SMTP id go10so1281873lbb.13 for ; Wed, 12 Dec 2012 17:14:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Jv9cV9+79eibADQZR2lh5DlO5HvnFXrtbqh2t+aogoA=; b=0rSCmbI1WH2PyQMcU8bKxK5TOOOXY+GrlsMRuQ3pvN1uN8AYpXRCNIyNH4Yhua2cKZ coHybqOyXrjcN2u5qJy8/OZBfWnJtI78rMdzqQvxLBSqH0ug/WG/7q6HLoZTxbDyeFfJ ocTB19GhsQEoGNZKrQhh++aUS4+DyKtwFxEWtRxOeFtNbLTaEsxQl10a/C+3Xa+QceFk KmbUah1q2lGBPNmyv9daf9qITGRiHaqkl1dSveiv2oz+xkIh19d4z5wypq7y8rbr1Qx0 pTnRS8EcJ+GEVzOWE7ZqZjj0ttHEQwUuq4XVwNwJG+nSiPd2Yv46QkO9NQekPSzeiduz OwDQ== MIME-Version: 1.0 Received: by 10.112.37.40 with SMTP id v8mr159694lbj.112.1355361263786; Wed, 12 Dec 2012 17:14:23 -0800 (PST) Received: by 10.112.61.33 with HTTP; Wed, 12 Dec 2012 17:14:23 -0800 (PST) In-Reply-To: References: Date: Wed, 12 Dec 2012 20:14:23 -0500 Message-ID: Subject: Re: iSCSI vs. SMB with ZFS. From: Zaphod Beeblebrox To: Wojciech Puchar Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Dec 2012 01:14:25 -0000 On Wed, Dec 12, 2012 at 5:16 PM, Wojciech Puchar wrote: >> about the same as the local disk for some operations --- faster for >> some, slower for others. The workstation has 12G of memory and it's >> my perception that iSCSI is heavily cached and that this enhances it's > any REAL test means doing something that will not fit in cache. That's plainly not true at all on it's face. It depends on what you're testing. In this particular test, I'm looking at the performance of the components on a singular common task --- that of running a game. It's common to run a game more than once and it's common to move from area to area in the game loading, unloading and reloading the same data. My test is a valid comparison of the two modes of loading the game ... from iSCSI and from SMB. You cold criticize me for several things --- I only tested two games or I have unrealistically large and powerful hardware, but really... consider what you are testing before you pontificate on test design. And even in the case where you want to look at the "enterprise" performance of a system, knowing both the cache performance and the disk performance is better than only knowing one or other. Throughput is a combination of these features. Pure disk performance serves as a lower bound, but cache performance (especially on some of the ZFS systems people are creating these days ... with 100's of gigs of RAM) is an equally valid statistic and optimization. From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 13 01:21:54 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C3898CA5 for ; Thu, 13 Dec 2012 01:21:54 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [89.206.35.99]) by mx1.freebsd.org (Postfix) with ESMTP id 219118FC12 for ; Thu, 13 Dec 2012 01:21:53 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5) with ESMTP id qBD1Lo1W002261; Thu, 13 Dec 2012 02:21:50 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5/Submit) with ESMTP id qBD1LnUD002258; Thu, 13 Dec 2012 02:21:50 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Thu, 13 Dec 2012 02:21:49 +0100 (CET) From: Wojciech Puchar To: Zaphod Beeblebrox Subject: Re: iSCSI vs. SMB with ZFS. In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Thu, 13 Dec 2012 02:21:50 +0100 (CET) Cc: FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Dec 2012 01:21:54 -0000 > common to move from area to area in the game loading, unloading and > reloading the same data. My test is a valid comparison of the two > modes of loading the game ... from iSCSI and from SMB. i don't know how windows cache network shares (iSCSI is treated as local not network). Here is a main problem IMHO. From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 13 06:52:41 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 43EF19E5 for ; Thu, 13 Dec 2012 06:52:41 +0000 (UTC) (envelope-from reko.turja@liukuma.net) Received: from www.liukuma.net (www.liukuma.net [IPv6:2001:470:28:38a::1]) by mx1.freebsd.org (Postfix) with ESMTP id D411D8FC08 for ; Thu, 13 Dec 2012 06:52:40 +0000 (UTC) Received: from www.liukuma.net (localhost [127.0.0.1]) by www.liukuma.net (Postfix) with ESMTP id 809521CCBB; Thu, 13 Dec 2012 08:52:31 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=liukuma.net; s=liukudkim; t=1355381551; bh=sWBx2kuBROn765megj+SJNwbx/c/r+hM3joS3AyrPK0=; h=From:To:References:In-Reply-To:Subject:Date; b=bQ4lA2ngqVSYHQO1acfidF6gPZ43lUeS4OULeFJYuLQPJn7MvPcHiTrg/4ZtiHpzg XS6+X+IWdel7zUnLQEtQxqYa+OEV6Z7fUFNXvwntRjHOxNAAFAxJvm5lDbxyH+FhSs A2cjrxKkG15rgG9yX9pSJkrfkr5Z94TzzGeyqF1k= X-Virus-Scanned: amavisd-new at liukuma.net Received: from www.liukuma.net ([127.0.0.1]) by www.liukuma.net (www.liukuma.net [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 8eDJO41qhgGx; Thu, 13 Dec 2012 08:52:09 +0200 (EET) Received: from Rivendell (dsl-kmibrasgw1-fe47de00-175.dhcp.inet.fi [80.222.71.175]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client did not present a certificate) (Authenticated sender: ignatz@www.liukuma.net) by www.liukuma.net (Postfix) with ESMTPSA id C12251CCB5; Thu, 13 Dec 2012 08:52:08 +0200 (EET) Message-ID: From: "Reko Turja" To: "Zaphod Beeblebrox" , "FreeBSD Hackers" References: In-Reply-To: Subject: Re: iSCSI vs. SMB with ZFS. Date: Thu, 13 Dec 2012 08:52:05 +0200 MIME-Version: 1.0 X-Priority: 3 X-MSMail-Priority: Normal Importance: Normal X-Mailer: Microsoft Windows Live Mail 15.4.3555.308 X-MimeOLE: Produced By Microsoft MimeOLE V15.4.3555.308 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Dec 2012 06:52:41 -0000 -----Original Message-----=20 From: Zaphod Beeblebrox=20 Sent: Wednesday, December 12, 2012 6:57 PM=20 To: FreeBSD Hackers=20 Subject: iSCSI vs. SMB with ZFS.=20 > So... I have two machines. My Fileserver is a core-2-duo machine with > FreeBSD-9.1-ish ZFS, istgt and samba 3.6. My workstation is windows 7 > on an i7. Both have GigE and are connected directly via a managed > switch with jumbo packets (specifically 9016) enabled. Both are using > tagged vlan packets to the switch (if that matters at all). My experience on samba has been that it=E2=80=99s slow whatever one does = to tweak it (probably just too linux-centric code to start with or = whatever...) Just as another datapoint =E2=80=93 do you have tried NFS = yet? Win7 has NFS available as OS component, although not installed by = default? -Reko From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 13 10:35:24 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CEBE3E40 for ; Thu, 13 Dec 2012 10:35:24 +0000 (UTC) (envelope-from pczanik@fang.fa.gau.hu) Received: from intmail-smtp-out.integrity.hu (intmail-smtp-out.integrity.hu [212.52.165.188]) by mx1.freebsd.org (Postfix) with ESMTP id 81A3C8FC13 for ; Thu, 13 Dec 2012 10:35:24 +0000 (UTC) Received: by intmail-smtp-out.integrity.hu (Postfix, from userid 10000) id BB31413468B7; Thu, 13 Dec 2012 11:26:02 +0100 (CET) Received: from linux-dbr3.site (unknown [10.254.0.13]) (Authenticated sender: czanik@integrity.hu) by intmail-smtp-out.integrity.hu (Postfix) with ESMTPSA id 4A5A3134689D for ; Thu, 13 Dec 2012 11:26:02 +0100 (CET) Message-ID: <50C9AD39.2010103@fang.fa.gau.hu> Date: Thu, 13 Dec 2012 11:26:01 +0100 From: Peter Czanik User-Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/17.0 Thunderbird/17.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: Syslog-ng weirdness on 9.1-RELEASE Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: clamav-milter 0.97.5 at mail-autosubmit X-Virus-Status: Clean X-Mailman-Approved-At: Thu, 13 Dec 2012 12:52:51 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Dec 2012 10:35:24 -0000 Hello, Mark Saad longcount.org> writes: > > All > I am wondering if anyone has seen this. I pulled down the 9.1-RELEASE > install media and did a clean install . After installing some ports from > the packages on > ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-9-stable > > I noticed that syslog-ng is spinning out of control with a stock config. > > ps shows I have two syslog-ng pids That is normal, one is the supervisor, the other one does the actual logging part: http://www.balabit.com/wiki/syslog-ng-faq-supervisor > the lower numbed pid is idle and the higher numbed pid is spinning out of > control As it turned out, 3.3.6 was buggy on FreeBSD when reading kernel messages (the bundled ivykis library was changed from the BalaBit maintained fork to upstream). You should use 3.3.7 from ports and read the related entry from /usr/ports/UPDATING Bye, CzP From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 13 13:40:22 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 960882C8 for ; Thu, 13 Dec 2012 13:40:22 +0000 (UTC) (envelope-from se@freebsd.org) Received: from nm15-vm0.bullet.mail.ird.yahoo.com (nm15-vm0.bullet.mail.ird.yahoo.com [77.238.189.216]) by mx1.freebsd.org (Postfix) with ESMTP id C20118FC08 for ; Thu, 13 Dec 2012 13:40:20 +0000 (UTC) Received: from [77.238.189.231] by nm15.bullet.mail.ird.yahoo.com with NNFMP; 13 Dec 2012 13:40:19 -0000 Received: from [217.146.188.152] by tm12.bullet.mail.ird.yahoo.com with NNFMP; 13 Dec 2012 13:40:18 -0000 Received: from [127.0.0.1] by smtp110.mail.ird.yahoo.com with NNFMP; 13 Dec 2012 13:40:18 -0000 X-Yahoo-Newman-Id: 751952.63222.bm@smtp110.mail.ird.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: epfF7OMVM1m05MZgDXGcLRvqdgCdyJYMM14rVDVq0p9M9IE 68ubckfY_dWC_CGaCSfjCJSU3OOWnAK5rNRIWk6SXorBgb.GAU2u_20RHtIt .jDklAHEztKqNN0Pd9K59cbbJpETnC7JAtv7iA3QU0ZE5hs4J3xa6OaxBT4A A8MuMfaqmswTIk0.7lYOK95tdbogMJ5Kb6KwIcWHSyxlqWTUn7LirAziQjdH DTX8b0E7NaO.fYozBMpeeG1J5jQaDdGCdfYZ_UxfJ6EMqPMwPM.pn_LROqWs ePZURf_vggcRbCg_aa2OiB3dw_r.ZpfO.X8_GzcGR4VZUjtjbZD_Rf3GGhob eLtZ7mnWdnzPqtgYdS5GWiGMqs__eb0VZ1G7c16oaEb8DGJxIclqQdW2FekY KjS_pZ1esDPE7eJsat8Tf6eahdFnlJa5ZCe2lnHq0xvY- X-Yahoo-SMTP: iDf2N9.swBDAhYEh7VHfpgq0lnq. Received: from [192.168.119.18] (se@87.158.2.148 with plain) by smtp110.mail.ird.yahoo.com with SMTP; 13 Dec 2012 05:40:18 -0800 PST Message-ID: <50C9DABA.1020008@freebsd.org> Date: Thu, 13 Dec 2012 14:40:10 +0100 From: Stefan Esser User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/17.0 Thunderbird/17.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: FreeBSD for serious performance? References: <20121211204323.310760@gmx.com> <50C889D3.1050404@freebsd.org> In-Reply-To: X-Enigmail-Version: 1.4.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Wojciech Puchar X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Dec 2012 13:40:22 -0000 Am 12.12.2012 23:26, schrieb Wojciech Puchar: >> The cause of the low write performance is the disabled write cache. >> Enabling the write cache is unsafe on SATA drives (with or without >> NCQ), since they do not make any guarantees that nearby data is not >> lost if power fails during a disk write. It never happened to me, >> but there is a reason that SAS drives have less capacity, much lower >> BER (one to two magnitudes) and are more expensive than SATA drives. > > interface have nothing to do. Both allows you to force writes now and then. Yes and no. SATA drives could be built to the same standards as SAS drives, but with the exception of the WD Raptor they hardly ever are. SATA drives are optimized for storage density at low cost. SAS drives have completely different design targets, in general (or their higher price could not be satisfied). SATA drives have been reported to lie about the completion not only of normal writes, but also of flush commands. And with "advanced format" (4K sector) drives, data is at risk in (logical) sectors that are not even being written to. There is no technical reason that SATA drives are less reliable with regard to hardware (bit density, BER, ...) and firmware (less strict conformance testing compared to SAS drives), but there are market forces that have this effect. >> The solution to the performance problem is simple: Turn on the write >> cache. If the data is valuable, then SAS is the solution to both the > > If data is valuable, regular and well done backup practice is the only > solution. I was referring to the higher quality firmware (with verified support for TCQ) of SAS drives. There were a number of consumer grade drives that supported tags, but where data was at risk due to firmware bugs. Backups are a way to recover from loss of data. Use of devices that are magnitudes more reliable improves short-term availability of the data and reduces the risk that a recovery will be needed. Don't mix immediate availability (as required by processing requirements) with availability of a recovery path. >> would pay one developer hour. Asking Nvidia to release the confidential >> documentation for their chip-set might help, but I doubt that there is >> much interest to add support for NCQ to an obsolete chip-set, today, >> unless you pay a developer (and even then ...). > > Even without this, i've never seen properly working NVidia hardware. ANY Yes, I've also had more problems with Nvidia hardware than with most other vendors' products. I guess this is due to the market segment they target (not sure whether this still applies for their expensive GPUs targeted at high performance computing). Regards, STefan From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 13 14:37:41 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 008374A7; Thu, 13 Dec 2012 14:37:40 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [89.206.35.99]) by mx1.freebsd.org (Postfix) with ESMTP id BCE818FC13; Thu, 13 Dec 2012 14:37:39 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5) with ESMTP id qBDEbU2I003834; Thu, 13 Dec 2012 15:37:31 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5/Submit) with ESMTP id qBDEbUvg003831; Thu, 13 Dec 2012 15:37:30 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Thu, 13 Dec 2012 15:37:30 +0100 (CET) From: Wojciech Puchar To: Stefan Esser Subject: Re: FreeBSD for serious performance? In-Reply-To: <50C9DABA.1020008@freebsd.org> Message-ID: References: <20121211204323.310760@gmx.com> <50C889D3.1050404@freebsd.org> <50C9DABA.1020008@freebsd.org> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Thu, 13 Dec 2012 15:37:31 +0100 (CET) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Dec 2012 14:37:41 -0000 > There is no technical reason that SATA drives are less reliable with > regard to hardware (bit density, BER, ...) and firmware (less strict > conformance testing compared to SAS drives), but there are market > forces that have this effect. The only sentence i could agree. But actually it is mostly propaganda to sell nearly same thing at 5x price. From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 13 16:08:48 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 274D810C for ; Thu, 13 Dec 2012 16:08:48 +0000 (UTC) (envelope-from emaste@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 0180F8FC18 for ; Thu, 13 Dec 2012 16:08:48 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id qBDG8l3A078469 for ; Thu, 13 Dec 2012 16:08:47 GMT (envelope-from emaste@freebsd.org) Received: (from emaste@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id qBDG8l5K078468 for freebsd-hackers@freebsd.org; Thu, 13 Dec 2012 16:08:47 GMT (envelope-from emaste@freebsd.org) X-Authentication-Warning: freefall.freebsd.org: emaste set sender to emaste@freebsd.org using -f Date: Thu, 13 Dec 2012 16:08:47 +0000 From: Ed Maste To: freebsd-hackers@freebsd.org Subject: [PATCH] Shared library debug .symbols files Message-ID: <20121213160847.GA78448@sandvine.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Dec 2012 16:08:48 -0000 I've been working generating userland debugging symbols, with the goal that we'll build them for each release. The user could install them along with the system, or later on when needed for debugging. The symbols files will also be useful for profiling and tools such as Valgrind, without needing to build and install world first. This patch enables .symbols files for shared libraries when DEBUG_FLAGS is set. Future changes will be needed to address static libraries and base system binaries, and the release build bits. This is a different approach to the patches Mark Johnston posted to -hackers about two years ago. I've followed the example of kmod.mk in generating a .debug file which is split into the two components with objcopy at build time. (Mark's patch overloaded strip to do it at install time.) Note that I used --strip-all and not --strip-debug, as the latter results in duplication in the symtab and strtab between the shared lib and its .symbols file. diff --git a/share/mk/bsd.lib.mk b/share/mk/bsd.lib.mk index 2c96df1..6a1b476 100644 --- a/share/mk/bsd.lib.mk +++ b/share/mk/bsd.lib.mk @@ -34,14 +34,13 @@ NO_WERROR= .endif .if defined(DEBUG_FLAGS) +OBJCOPY?= objcopy CFLAGS+= ${DEBUG_FLAGS} .if ${MK_CTF} != "no" && ${DEBUG_FLAGS:M-g} != "" CTFFLAGS+= -g .endif -.endif - -.if !defined(DEBUG_FLAGS) +.else STRIP?= -s .endif @@ -173,14 +172,17 @@ SOLINKOPTS+= -Wl,--fatal-warnings -Wl,--warn-shared-textrel .endif .if target(beforelinking) -${SHLIB_NAME}: ${SOBJS} beforelinking +${SHLIB_NAME}: beforelinking +.endif +.if defined(DEBUG_FLAGS) +${SHLIB_NAME}.debug: ${SOBJS} .else ${SHLIB_NAME}: ${SOBJS} .endif @${ECHO} building shared library ${SHLIB_NAME} - @rm -f ${.TARGET} ${SHLIB_LINK} + @rm -f ${.SHLIB_NAME} ${SHLIB_LINK} .if defined(SHLIB_LINK) - @ln -fs ${.TARGET} ${SHLIB_LINK} + @ln -fs ${.SHLIB_NAME} ${SHLIB_LINK} .endif .if !defined(NM) @${CC} ${LDFLAGS} ${SSP_CFLAGS} ${SOLINKOPTS} \ @@ -194,6 +196,15 @@ ${SHLIB_NAME}: ${SOBJS} .if ${MK_CTF} != "no" ${CTFMERGE} ${CTFFLAGS} -o ${.TARGET} ${SOBJS} .endif + +.if defined(DEBUG_FLAGS) +${SHLIB_NAME}: ${SHLIB_NAME}.debug ${SHLIB_NAME}.symbols + ${OBJCOPY} --strip-all --add-gnu-debuglink=${SHLIB_NAME}.symbols\ + ${SHLIB_NAME}.debug ${.TARGET} + +${SHLIB_NAME}.symbols: + ${OBJCOPY} --only-keep-debug ${SHLIB_NAME}.debug ${.TARGET} +.endif .endif .if defined(INSTALL_PIC_ARCHIVE) && defined(LIB) && !empty(LIB) && ${MK_TOOLCHAIN} != "no" @@ -270,6 +281,11 @@ _libinstall: ${INSTALL} ${STRIP} -o ${LIBOWN} -g ${LIBGRP} -m ${LIBMODE} \ ${_INSTALLFLAGS} ${_SHLINSTALLFLAGS} \ ${SHLIB_NAME} ${DESTDIR}${SHLIBDIR} +.if defined(DEBUG_FLAGS) + ${INSTALL} -o ${LIBOWN} -g ${LIBGRP} -m ${LIBMODE} \ + ${_INSTALLFLAGS} ${_SHLINSTALLFLAGS} \ + ${SHLIB_NAME}.symbols ${DESTDIR}${SHLIBDIR} +.endif .if defined(SHLIB_LINK) # ${_SHLIBDIRPREFIX} and ${_LDSCRIPTROOT} are both needed when cross-building # and when building 32 bits library shims. ${_SHLIBDIRPREFIX} is the directory From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 13 18:54:38 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E618C841; Thu, 13 Dec 2012 18:54:38 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id 4D3DF8FC12; Thu, 13 Dec 2012 18:54:38 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.5/8.14.5) with ESMTP id qBDIsV14057738; Thu, 13 Dec 2012 20:54:31 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.7.1 kib.kiev.ua qBDIsV14057738 Received: (from kostik@localhost) by tom.home (8.14.5/8.14.5/Submit) id qBDIsV1J057737; Thu, 13 Dec 2012 20:54:31 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 13 Dec 2012 20:54:31 +0200 From: Konstantin Belousov To: Ed Maste Subject: Re: [PATCH] Shared library debug .symbols files Message-ID: <20121213185431.GH71906@kib.kiev.ua> References: <20121213160847.GA78448@sandvine.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="4C6bbPZ6c/S1npyF" Content-Disposition: inline In-Reply-To: <20121213160847.GA78448@sandvine.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Dec 2012 18:54:39 -0000 --4C6bbPZ6c/S1npyF Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Dec 13, 2012 at 04:08:47PM +0000, Ed Maste wrote: > I've been working generating userland debugging symbols, with the goal > that we'll build them for each release. The user could install them > along with the system, or later on when needed for debugging. The > symbols files will also be useful for profiling and tools such as > Valgrind, without needing to build and install world first. >=20 > This patch enables .symbols files for shared libraries when DEBUG_FLAGS > is set. Future changes will be needed to address static libraries and > base system binaries, and the release build bits. You cannot strip static libraries, at least if you want the result to be useful for the consequent use. --4C6bbPZ6c/S1npyF Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlDKJGcACgkQC3+MBN1Mb4jStgCfXILAUJ9+yO05myFUzjgQq5TE 3KIAoPeP1d5um+wGopFJ+sjBtGPd+7Kt =Nydp -----END PGP SIGNATURE----- --4C6bbPZ6c/S1npyF-- From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 13 20:05:15 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5768DB88 for ; Thu, 13 Dec 2012 20:05:15 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-ie0-f171.google.com (mail-ie0-f171.google.com [209.85.223.171]) by mx1.freebsd.org (Postfix) with ESMTP id 184918FC0A for ; Thu, 13 Dec 2012 20:05:13 +0000 (UTC) Received: by mail-ie0-f171.google.com with SMTP id 17so4648224iea.16 for ; Thu, 13 Dec 2012 12:05:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=wx0AcZ1lCTvGyx13RVM1s0n9xKoI8Op/m6RiKI6gRmE=; b=Es0et+e93bUWyD7boNlcEVURcMAW/+d6cWzMKuKfcAWa/7aWw7QZr0x8CxvxVULBnF 5QtSDASgePkvvAL/rFY7N+6OHSYFfFKAzpk+ZMR5BI5Q/eYalaqknIArRFjh2d/wGdFW CeRg740kNdW2CVPePfaikelc7FHNXR02P3UjsJPRvzObg7tNoihGDJNo+Ve7Y9E8k98+ gw4WXfq2EFgu87XHDEeAqhhXDEVo9CI85OcMLWsdvfchD2RNATfk/sEcvwvunnpx8TpE fX69Sd8K8KrWNV7LXaqoUOWvyGgwMNwSOVGFHklqrWRy70rk6toTEUYR5FHHErmC/OgA lqMQ== MIME-Version: 1.0 Received: by 10.50.15.138 with SMTP id x10mr18180041igc.16.1355429107013; Thu, 13 Dec 2012 12:05:07 -0800 (PST) Sender: carpeddiem@gmail.com Received: by 10.50.151.135 with HTTP; Thu, 13 Dec 2012 12:05:06 -0800 (PST) In-Reply-To: <20121213185431.GH71906@kib.kiev.ua> References: <20121213160847.GA78448@sandvine.com> <20121213185431.GH71906@kib.kiev.ua> Date: Thu, 13 Dec 2012 15:05:06 -0500 X-Google-Sender-Auth: l7nYcOoYJm7ekcyIXQNkUPzQU54 Message-ID: Subject: Re: [PATCH] Shared library debug .symbols files From: Ed Maste To: Konstantin Belousov Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Dec 2012 20:05:15 -0000 On 13 December 2012 13:54, Konstantin Belousov wrote: > On Thu, Dec 13, 2012 at 04:08:47PM +0000, Ed Maste wrote: ... >> This patch enables .symbols files for shared libraries when DEBUG_FLAGS >> is set. Future changes will be needed to address static libraries and >> base system binaries, and the release build bits. > You cannot strip static libraries, at least if you want the result > to be useful for the consequent use. Right, presumably I can use --strip-debug on static libraries and not --strip-all (assuming that objcopy still leaves the required parts). From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 13 22:08:48 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id ABB91BC8; Thu, 13 Dec 2012 22:08:48 +0000 (UTC) (envelope-from eric@vangyzen.net) Received: from aussmtpmrkps320.us.dell.com (aussmtpmrkps320.us.dell.com [143.166.224.254]) by mx1.freebsd.org (Postfix) with ESMTP id 66C488FC08; Thu, 13 Dec 2012 22:08:48 +0000 (UTC) X-Loopcount0: from 64.238.244.148 X-IronPort-AV: E=Sophos;i="4.84,276,1355119200"; d="scan'208";a="12491570" Message-ID: <50CA51AB.6060801@vangyzen.net> Date: Thu, 13 Dec 2012 16:07:39 -0600 From: Eric van Gyzen User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:14.0) Gecko/20120822 Thunderbird/14.0 MIME-Version: 1.0 To: Ed Maste Subject: Re: [PATCH] Shared library debug .symbols files References: <20121213160847.GA78448@sandvine.com> In-Reply-To: <20121213160847.GA78448@sandvine.com> Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Dec 2012 22:08:48 -0000 On 12/13/2012 10:08, Ed Maste wrote: > I've been working generating userland debugging symbols, with the goal > that we'll build them for each release. The user could install them > along with the system, or later on when needed for debugging. The > symbols files will also be useful for profiling and tools such as > Valgrind, without needing to build and install world first. > > This patch enables .symbols files for shared libraries when DEBUG_FLAGS > is set. Future changes will be needed to address static libraries and > base system binaries, and the release build bits. > > This is a different approach to the patches Mark Johnston posted to > -hackers about two years ago. I've followed the example of kmod.mk in > generating a .debug file which is split into the two components with > objcopy at build time. (Mark's patch overloaded strip to do it at > install time.) > > Note that I used --strip-all and not --strip-debug, as the latter > results in duplication in the symtab and strtab between the shared lib > and its .symbols file. Excellent. I've wanted this for a while, and I like your approach. I have one comment below; otherwise, the patch looks fine (not that I have any authority on the subject...). > diff --git a/share/mk/bsd.lib.mk b/share/mk/bsd.lib.mk > index 2c96df1..6a1b476 100644 > --- a/share/mk/bsd.lib.mk > +++ b/share/mk/bsd.lib.mk > @@ -34,14 +34,13 @@ NO_WERROR= > .endif > > .if defined(DEBUG_FLAGS) > +OBJCOPY?= objcopy > CFLAGS+= ${DEBUG_FLAGS} > > .if ${MK_CTF} != "no" && ${DEBUG_FLAGS:M-g} != "" > CTFFLAGS+= -g > .endif > -.endif > - > -.if !defined(DEBUG_FLAGS) > +.else > STRIP?= -s > .endif > > @@ -173,14 +172,17 @@ SOLINKOPTS+= -Wl,--fatal-warnings -Wl,--warn-shared-textrel > .endif > > .if target(beforelinking) > -${SHLIB_NAME}: ${SOBJS} beforelinking > +${SHLIB_NAME}: beforelinking > +.endif > +.if defined(DEBUG_FLAGS) > +${SHLIB_NAME}.debug: ${SOBJS} > .else > ${SHLIB_NAME}: ${SOBJS} > .endif > @${ECHO} building shared library ${SHLIB_NAME} > - @rm -f ${.TARGET} ${SHLIB_LINK} > + @rm -f ${.SHLIB_NAME} ${SHLIB_LINK} dot-SHLIB_NAME? I'm not aware of this magic. > .if defined(SHLIB_LINK) > - @ln -fs ${.TARGET} ${SHLIB_LINK} > + @ln -fs ${.SHLIB_NAME} ${SHLIB_LINK} > .endif > .if !defined(NM) > @${CC} ${LDFLAGS} ${SSP_CFLAGS} ${SOLINKOPTS} \ > @@ -194,6 +196,15 @@ ${SHLIB_NAME}: ${SOBJS} > .if ${MK_CTF} != "no" > ${CTFMERGE} ${CTFFLAGS} -o ${.TARGET} ${SOBJS} > .endif > + > +.if defined(DEBUG_FLAGS) > +${SHLIB_NAME}: ${SHLIB_NAME}.debug ${SHLIB_NAME}.symbols > + ${OBJCOPY} --strip-all --add-gnu-debuglink=${SHLIB_NAME}.symbols\ > + ${SHLIB_NAME}.debug ${.TARGET} > + > +${SHLIB_NAME}.symbols: > + ${OBJCOPY} --only-keep-debug ${SHLIB_NAME}.debug ${.TARGET} > +.endif > .endif > > .if defined(INSTALL_PIC_ARCHIVE) && defined(LIB) && !empty(LIB) && ${MK_TOOLCHAIN} != "no" > @@ -270,6 +281,11 @@ _libinstall: > ${INSTALL} ${STRIP} -o ${LIBOWN} -g ${LIBGRP} -m ${LIBMODE} \ > ${_INSTALLFLAGS} ${_SHLINSTALLFLAGS} \ > ${SHLIB_NAME} ${DESTDIR}${SHLIBDIR} > +.if defined(DEBUG_FLAGS) > + ${INSTALL} -o ${LIBOWN} -g ${LIBGRP} -m ${LIBMODE} \ > + ${_INSTALLFLAGS} ${_SHLINSTALLFLAGS} \ > + ${SHLIB_NAME}.symbols ${DESTDIR}${SHLIBDIR} > +.endif > .if defined(SHLIB_LINK) > # ${_SHLIBDIRPREFIX} and ${_LDSCRIPTROOT} are both needed when cross-building > # and when building 32 bits library shims. ${_SHLIBDIRPREFIX} is the directory > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 13 22:11:19 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A3446D82; Thu, 13 Dec 2012 22:11:19 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 52D5B8FC16; Thu, 13 Dec 2012 22:11:19 +0000 (UTC) Received: by mail-ob0-f182.google.com with SMTP id 16so2671137obc.13 for ; Thu, 13 Dec 2012 14:11:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=VKn+54wtL19q/4nGJFHp0qj1SB1++7r8R/zj+bWin1g=; b=TCoffNMived9kPEF/Htl7YLxfHhzZ8KlMIBvNn+MvTY6keQV03UXphh6dKGxmF7Xtc 33TgRHuGBpjjYJlh8K5tNAiMci4Gz16e6XfNY3wzixRzP1XR+pIs9ZWbHs4prWl25Zjy WyCpGhRxL/p7I2ARpj1DxQ9e4DpEPR84srtiwWDreYs+Gc745t+yuQENF8ROFuAkkprh l3I3YMYSKhixusWDQZG2p2SBovD25kVxB9rs6V1hAURdWvVIJa8HM/r1hDnebLDsAcTs DnRj6DuiKvhR6GSS7dEB7qvrFckFD0ujy7HL9tqLJOhyb7/3lm/TA3cELM675xD29zNq aIqQ== MIME-Version: 1.0 Received: by 10.60.172.164 with SMTP id bd4mr2918278oec.51.1355436678604; Thu, 13 Dec 2012 14:11:18 -0800 (PST) Received: by 10.76.143.33 with HTTP; Thu, 13 Dec 2012 14:11:18 -0800 (PST) In-Reply-To: <50CA51AB.6060801@vangyzen.net> References: <20121213160847.GA78448@sandvine.com> <50CA51AB.6060801@vangyzen.net> Date: Thu, 13 Dec 2012 14:11:18 -0800 Message-ID: Subject: Re: [PATCH] Shared library debug .symbols files From: Garrett Cooper To: Eric van Gyzen Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org, Ed Maste X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Dec 2012 22:11:19 -0000 On Thu, Dec 13, 2012 at 2:07 PM, Eric van Gyzen wrote: ... > dot-SHLIB_NAME? I'm not aware of this magic. Looks like a typo: $ make -V.SHLIB_NAME $ make -VSHLIB_NAME libc.so.7 Thanks! -Garrett From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 13 22:11:55 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5BC1EEBE for ; Thu, 13 Dec 2012 22:11:55 +0000 (UTC) (envelope-from dieterbsd@engineer.com) Received: from mailout-us.gmx.com (mailout-us.gmx.com [74.208.5.67]) by mx1.freebsd.org (Postfix) with SMTP id 179E38FC14 for ; Thu, 13 Dec 2012 22:11:55 +0000 (UTC) Received: (qmail 31818 invoked by uid 0); 13 Dec 2012 22:11:48 -0000 Received: from 67.206.186.87 by rms-us006 with HTTP Content-Type: text/plain; charset="utf-8" Date: Thu, 13 Dec 2012 17:11:44 -0500 From: "Dieter BSD" Message-ID: <20121213221145.310760@gmx.com> MIME-Version: 1.0 Subject: NetBSD's boot select MBR To: freebsd-hackers@freebsd.org X-Authenticated: #74169980 X-Flags: 0001 X-Mailer: GMX.com Web Mailer x-registered: 0 Content-Transfer-Encoding: 8bit X-GMX-UID: 9je4cAAY3zOlNR3dAHAhKHR+IGRvb8A7 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Dec 2012 22:11:55 -0000 [ from the FreeBSD for serious performance? thread ] > > So I use NetBSD's MBR for disks I want > > to boot from. > > Can I have a CMD sequence? > > First would be ... > # fetch ... Read NetBSD's fdisk(8) and mbr(8). The MBR is only 512 bytes, and must contain the code and data. This is very limiting, so there are multiple versions depending on which features you need. IIRC, /usr/mdec/mbr_bootsel is the one I use. You can specify a label (up to 7 characters) for each bootable partition ("slice" in FreeBSD-speak). When the system executes the MBR code, it presents a menu with these labels. You can type '1' for the first partition/slice, '2' for the second, and so on. You can also type a function key to run the MBR of a different disk. The disk numbering can be rather insane, so it must be using the bios disk numbering. If you don't type anything it times out and boots the active partition/slice. Sadly, the FreeBSD bootstrap insists on booting the partition/slice marked active in the MBR. (The MBR code should look at the active bit, the later bootstraps should not.) So if you want to have multiple FreeBSD versions, it works best to put each on its own disk. The last time I used fdisk, large disks triggered a bug where some variable overflowed, giving negative numbers. Hopefully this is fixed by now. FreeBSD's fdisk also had some bug, so I had great fun handcrafting a MBR. It's been a long time since I installed NetBSD from scratch, (I usually just unpack the tar files, run installboot, and edit some config files.) I assume you can just boot a CD and get a shell, and experiment with fdisk on a scratch disk without actually installing NetBSD. To avoid a disaster, do something like dd if=$DISK of=/boot/MBR_backup bs=512 count=1 ; sync beforehand, in case you accidently mess up your boot drive. man pages: http://netbsd.gw.com/cgi-bin/man-cgi?++NetBSD-current isos: http://www.netbsd.org/mirrors/ for example: ftp://iso.netbsd.org/pub/NetBSD/iso/5.2/amd64cd-5.2.iso or ftp://iso.netbsd.org/pub/NetBSD/iso/5.2/i386cd-5.2.iso From owner-freebsd-hackers@FreeBSD.ORG Fri Dec 14 10:42:24 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 13572DC5 for ; Fri, 14 Dec 2012 10:42:24 +0000 (UTC) (envelope-from rank1seeker@gmail.com) Received: from mail-bk0-f54.google.com (mail-bk0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 891E48FC18 for ; Fri, 14 Dec 2012 10:42:23 +0000 (UTC) Received: by mail-bk0-f54.google.com with SMTP id je9so1565104bkc.13 for ; Fri, 14 Dec 2012 02:42:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:from:to:subject:date:content-type :content-transfer-encoding:in-reply-to:references:x-mailer; bh=wF9MakPlq3f3LAMsjNkHe/VLW6JMZsYmER55VEkmmzM=; b=z1dwSDuZP1shGDFAQUFZdzW2EXHCcnjAcAr9w3ooAAVIayKcKs86VmgzR407LMBw2C ljOktVbiE9ZBjQZIf92mLyEJ9nh8hCDoQghlgWpgKiYzFLSJkziiPuZtJ/Hxa8FNJYeg Wiw2eyy8/i8WPoOSz254M4egpAC2w/WK8af1nbZPqe2TtrfCjySoQv8AlQ4nBkl3xiWZ zONS+vkecA7hlIdsgYAKCSfvVUKzzf1dPOSkVDprZjYeYjSAdxUrwgD2qJ8a5PKToU8G hvGf/xAbFFU7RRigDegIuLmjBLZDH2cWvIEzLNvQywmdIOlW5QWJpKd+mXwG4Mp+qTM9 CEvQ== Received: by 10.204.5.145 with SMTP id 17mr2491784bkv.98.1355481742174; Fri, 14 Dec 2012 02:42:22 -0800 (PST) Received: from DOMYPC ([82.193.208.225]) by mx.google.com with ESMTPS id q22sm3908488bkv.16.2012.12.14.02.42.18 (version=SSLv3 cipher=OTHER); Fri, 14 Dec 2012 02:42:20 -0800 (PST) Message-ID: <20121214.104219.547.1@DOMY-PC> From: rank1seeker@gmail.com To: "Dieter BSD" , hackers@freebsd.org Subject: Re: NetBSD's boot select MBR Date: Fri, 14 Dec 2012 11:42:19 +0100 Content-Type: text/plain; charset="Windows-1250" Content-Transfer-Encoding: quoted-printable In-Reply-To: <20121213221145.310760@gmx.com> References: <20121213221145.310760@gmx.com> X-Mailer: POP Peeper (3.8.1.0) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Dec 2012 10:42:24 -0000 > > > So I use NetBSD's MBR for disks I want=0D=0A> > > to boot = from.=0D=0A> >=0D=0A> > Can I have a CMD sequence?=0D=0A> >=0D=0A> > = First would be ...=0D=0A> > # fetch ...=0D=0A> =0D=0A> Read NetBSD's = fdisk(8) and mbr(8).=0D=0A> =0D=0A> The MBR is only 512 bytes, and must = contain the code and data.=0D=0A> This is very limiting, so there are = multiple versions depending=0D=0A> on which features you need. IIRC, = /usr/mdec/mbr_bootsel is the one I use.=0D=0A> You can specify a label = (up to 7 characters) for each bootable partition=0D=0A> ("slice" in = FreeBSD-speak). When the system executes the MBR code,=0D=0A> it presents = a menu with these labels. You can type '1' for the first=0D=0A> = partition/slice, '2' for the second, and so on. You can also type = a=0D=0A> function key to run the MBR of a different disk. The disk = numbering=0D=0A> can be rather insane, so it must be using the bios disk = numbering. If=0D=0A> you don't type anything it times out and boots the = active partition/slice.=0D=0A> =0D=0A> Sadly, the FreeBSD bootstrap = insists on booting the partition/slice=0D=0A> marked active in the MBR. = (The MBR code should look at the active=0D=0A> bit, the later bootstraps = should not.) So if you want to have=0D=0A> multiple FreeBSD versions, it = works best to put each on its own disk.=0D=0A> =0D=0A> The last time I = used fdisk, large disks triggered a bug where some=0D=0A> variable = overflowed, giving negative numbers. Hopefully this is fixed=0D=0A> by = now. FreeBSD's fdisk also had some bug, so I had great fun=0D=0A> = handcrafting a MBR.=0D=0A> =0D=0A> It's been a long time since I = installed NetBSD from scratch, (I usually=0D=0A> just unpack the tar = files, run installboot, and edit some config files.)=0D=0A> I assume you = can just boot a CD and get a shell, and experiment with=0D=0A> fdisk on a = scratch disk without actually installing NetBSD.=0D=0A> To avoid a = disaster, do something like=0D=0A> dd if=3D$DISK of=3D/boot/MBR_backup = bs=3D512 count=3D1 ; sync=0D=0A> beforehand, in case you accidently mess = up your boot drive.=0D=0A> =0D=0A> man pages:=0D=0A> = http://netbsd.gw.com/cgi-bin/man-cgi?++NetBSD-current=0D=0A> =0D=0A> = isos:=0D=0A> http://www.netbsd.org/mirrors/=0D=0A> for example:=0D=0A> = ftp://iso.netbsd.org/pub/NetBSD/iso/5.2/amd64cd-5.2.iso=0D=0A> or=0D=0A> = ftp://iso.netbsd.org/pub/NetBSD/iso/5.2/i386cd-5.2.iso=0D=0A=0D=0A=0D=0AMBR = supports max of 4 slices/partitions.=0D=0A=0D=0ALooing = at:=0D=0Ahttp://netbsd.gw.com/cgi-bin/man-cgi?mbr++NetBSD-6.0=0D=0A=0D=0AThe = '/usr/mdec/mbr_bootsel', which you've mentioned, is equivalent to = FreeBSD's '/boot/boot0', which is tuned via 'boot0cfg' util.=0D=0AIt will = also show menu, to choose from, a slice to boot or another drive OR =3D> = 'F6' to choose PXE boot.=0D=0AIf nothing has been selected, it will = simply boot an active slice.=0D=0ASo I see FreeBSD's, as superior = (additionally supports PXE) =3D> 8.0 onward ...=0D=0A=0D=0AI don't use = fdisk at all. Gpart is here ...=0D=0A;)=0D=0A=0D=0AAdditionally ..., when = I said '# fetch ...', I meant to get only 1 file, from ftp pub, which is = installed into MBR ('/usr/mdec/mbr_$SOMETHING')=0D=0AThen I would use = FreeBSD's:=0D=0A# gpart bootcode -b $path_to_netbsd_mbr_code = ...=0D=0A=0D=0ABut this won't happen, as I thought NetBSD's had a MBR = code, which could be told via a tool, from fully booted and running = FreeBSD:=0D=0A"Active slice is 2, but please, boot slice 4 at a next = boot".=0D=0AAfter that, I would '# reboot' and without any menu, slice 4 = would be booted, for that one time, instaed of an active slice = 2.=0D=0A=0D=0AAm I missing something?=0D=0AThanks in = advance.=0D=0A=0D=0A=0D=0ADomagoj Smol=E8i=E6=0D=0A=0D=0A From owner-freebsd-hackers@FreeBSD.ORG Fri Dec 14 19:10:52 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (unknown [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 791966A8 for ; Fri, 14 Dec 2012 19:10:52 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-ie0-f182.google.com (mail-ie0-f182.google.com [209.85.223.182]) by mx1.freebsd.org (Postfix) with ESMTP id 1E3528FC0C for ; Fri, 14 Dec 2012 19:10:51 +0000 (UTC) Received: by mail-ie0-f182.google.com with SMTP id s9so6501561iec.13 for ; Fri, 14 Dec 2012 11:10:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=RDQ4ARBjk1bN/8Ud8liijidKc0w4oQ5QK4ZzxGnJFSE=; b=XkF44UfUVo7wcFOAWLhBEKFpWzIFTCYd8NNrxjbsiBo3raufysoaHx/PX4LRrghpMd qMotyJ3tKDuTPU1L1KBrj5jtTkLA43iYrlQ78Z0g+WdmWngs77knIVoZcjIai+An1+BX A4z7cD05btwb400NS51IB4ivszhVGBDqyngVzW2P/mFDzQ5EutflxTh+mqTiELAqijfM xI3CDJZ6+RhMAbvCQN5KZ/bEZMPzeDbGBMZUgR8WOMF8EPYDE6fYIzAlTK9BCQV3nMMV kDgazvTgDMBaGd8Fu2NqZRx3qecpXpOW0omlghmsy/yufBzD0Ue5AxwgRAF+Ogw3vxD5 1Y8g== MIME-Version: 1.0 Received: by 10.50.37.168 with SMTP id z8mr2734139igj.1.1355512251580; Fri, 14 Dec 2012 11:10:51 -0800 (PST) Sender: carpeddiem@gmail.com Received: by 10.50.112.134 with HTTP; Fri, 14 Dec 2012 11:10:51 -0800 (PST) In-Reply-To: <50CA51AB.6060801@vangyzen.net> References: <20121213160847.GA78448@sandvine.com> <50CA51AB.6060801@vangyzen.net> Date: Fri, 14 Dec 2012 14:10:51 -0500 X-Google-Sender-Auth: Y6UqfNGZhZrb3wrASXbovB1pies Message-ID: Subject: Re: [PATCH] Shared library debug .symbols files From: Ed Maste To: Eric van Gyzen Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Dec 2012 19:10:52 -0000 On 13 December 2012 17:07, Eric van Gyzen wrote: >> @${ECHO} building shared library ${SHLIB_NAME} >> - @rm -f ${.TARGET} ${SHLIB_LINK} >> + @rm -f ${.SHLIB_NAME} ${SHLIB_LINK} > > > dot-SHLIB_NAME? I'm not aware of this magic. No magic, just a typo introduced while transcribing from an isolated tree. Thanks for spotting. >> +${SHLIB_NAME}: ${SHLIB_NAME}.debug ${SHLIB_NAME}.symbols >> + ${OBJCOPY} --strip-all --add-gnu-debuglink=${SHLIB_NAME}.symbols\ >> + ${SHLIB_NAME}.debug ${.TARGET} >> + >> +${SHLIB_NAME}.symbols: Mark Johnston also pointed out I'm missing the ${SHLIB_NAME}.debug dependency here. From owner-freebsd-hackers@FreeBSD.ORG Fri Dec 14 23:54:24 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id F26AAA66 for ; Fri, 14 Dec 2012 23:54:23 +0000 (UTC) (envelope-from baptiste.daroussin@gmail.com) Received: from mail-ea0-f182.google.com (mail-ea0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id 759028FC14 for ; Fri, 14 Dec 2012 23:54:22 +0000 (UTC) Received: by mail-ea0-f182.google.com with SMTP id a14so1537930eaa.13 for ; Fri, 14 Dec 2012 15:54:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:subject:message-id:mime-version:content-type :content-disposition:user-agent; bh=mpx8txW9OpN2NKNwn6PM4uvI6/HHuafQDIBjBfEV6wo=; b=FQxpl3A/knzzhaQAd7hO4sOQZYqFMjLVE3u9QJUeL7doZ9DynMTPsXRLfZRs2LGbkn nVrwqf+KPwUeHReqt0O6YUM/5+UsvNwTOfee4C4GrLQTxJWmh1FLsl+7eECjRSvQ9pQ+ 2EwYDuHHK1e4sp1HzVMPVhKHzhCw9M25bWyWu6n5BbjqqC8IxUuG5oXCcPL4/qlVV9h6 stFnYIwu+6naiUJ9BKJwxe8TZmJ8Ln7yDHyIaOT40n4BBvGpqSjF90XmACIkQJcUMOsH FuG3QsKBKOoLNshzSJdIapSfU+gZPUjRTfgBl9CDS1yauX0M4K9n2i7KDJY+Vxu0Ct0/ olAQ== Received: by 10.14.223.200 with SMTP id v48mr18968525eep.24.1355529262020; Fri, 14 Dec 2012 15:54:22 -0800 (PST) Received: from ithaqua.etoilebsd.net (ithaqua.etoilebsd.net. [37.59.37.188]) by mx.google.com with ESMTPS id f49sm12275548eep.12.2012.12.14.15.54.20 (version=TLSv1/SSLv3 cipher=OTHER); Fri, 14 Dec 2012 15:54:21 -0800 (PST) Sender: Baptiste Daroussin Date: Sat, 15 Dec 2012 00:54:19 +0100 From: Baptiste Daroussin To: hackers@FreeBSD.org Subject: Fix overlinking in base aka import pkgconf Message-ID: <20121214235418.GF18884@ithaqua.etoilebsd.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ni93GHxFvA+th69W" Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Dec 2012 23:54:24 -0000 --ni93GHxFvA+th69W Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi, Some of our binary are overlinked, the way we handle the linking doesn't he= lp for that. On proposition could be to use pkgconf https://github.com/pkgconf/pkgconf w= hich is BSD license pkg-config implementation 100% compatible with pkg-config. What I propose is to create a new PCADD variable for the Makefiles. PCADD will invoke pkgconf to gather the libraries and the cflags for a given project. The second thing would be to create .pc files for all of our libraries. for example: usr.bin/fstat dynamic build is overlinked With the following simple patch we can solve the problem: Index: Makefile =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- Makefile (revision 243899) +++ Makefile (working copy) @@ -5,7 +5,7 @@ SRCS=3D fstat.c fuser.c main.c LINKS=3D ${BINDIR}/fstat ${BINDIR}/fuser DPADD=3D ${LIBKVM} ${LIBUTIL} ${LIBPROCSTAT} -LDADD=3D -lkvm -lutil -lprocstat +PCADD=3D procstat =20 MAN1=3D fuser.1 fstat.1 =20 This requires the following .pc files quick and dirty ones: - util.pc: prefix=3D/usr libdir=3D${prefix}/lib includedir=3D${prefix}/include Name: util Libs: -L${libdir} -lutil Cflags: -I${includedir} - kvm.pc: prefix=3D/usr libdir=3D${prefix}/lib includedir=3D${prefix}/include Name: kvm Libs: -L${libdir} -lkvm Cflags: -I${includedir} - procstat.pc: prefix=3D/usr libdir=3D${prefix}/lib includedir=3D${prefix}/include Name: procstat Requires.private: kvm util Libs: -L${libdir} -lprocstat Cflags: -I${includedir} The quick and dirty patch for our framework is: Index: bsd.prog.mk =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- bsd.prog.mk (revision 243899) +++ bsd.prog.mk (working copy) @@ -36,6 +36,18 @@ LDFLAGS+=3D -static .endif =20 +.if defined(PCADD) +PCFLAGS!=3D ${PKGCONF} --cflags ${PCADD} +.if defined(NO_SHARED) && (${NO_SHARED} !=3D "no" && ${NO_SHARED} !=3D "NO= ") +PCLIBS!=3D ${PKGCONF} --libs --static ${PCADD} +.else +PCLIBS!=3D ${PKGCONF} --libs ${PCADD} +.endif +CFLAGS+=3D ${PCFLAGS} +CXXFLAGS+=3D ${PCFLAGS} +LDADD+=3D ${PCLIBS} +.endif + .if defined(PROG_CXX) PROG=3D ${PROG_CXX} .endif Index: sys.mk =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- sys.mk (revision 243899) +++ sys.mk (working copy) @@ -137,6 +137,8 @@ PC ?=3D pc PFLAGS ?=3D =20 +PKGCONF ?=3D pkgconf + RC ?=3D f77 RFLAGS ?=3D =20 Of course a lot of work is needed to get something really well integrated b= ut I wanted feedback first before working on anything like this :) regards, Bapt --ni93GHxFvA+th69W Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlDLvCoACgkQ8kTtMUmk6EzbPACbB6cR+36w0Gv8sFW93J7EJTbi k5QAnRpPMLDTjadalIT32dhOTjvjl7PX =nloN -----END PGP SIGNATURE----- --ni93GHxFvA+th69W-- From owner-freebsd-hackers@FreeBSD.ORG Sat Dec 15 01:22:38 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A6546DEE; Sat, 15 Dec 2012 01:22:38 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id 0DDAE8FC13; Sat, 15 Dec 2012 01:22:37 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.5/8.14.5) with ESMTP id qBF1MYqF075014; Sat, 15 Dec 2012 03:22:34 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.7.3 kib.kiev.ua qBF1MYqF075014 Received: (from kostik@localhost) by tom.home (8.14.5/8.14.5/Submit) id qBF1MYoW075013; Sat, 15 Dec 2012 03:22:34 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 15 Dec 2012 03:22:34 +0200 From: Konstantin Belousov To: Baptiste Daroussin Subject: Re: Fix overlinking in base aka import pkgconf Message-ID: <20121215012233.GP71906@kib.kiev.ua> References: <20121214235418.GF18884@ithaqua.etoilebsd.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="nVYOjVWOcH+Ezkzp" Content-Disposition: inline In-Reply-To: <20121214235418.GF18884@ithaqua.etoilebsd.net> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Dec 2012 01:22:38 -0000 --nVYOjVWOcH+Ezkzp Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Dec 15, 2012 at 12:54:19AM +0100, Baptiste Daroussin wrote: > Hi, >=20 > Some of our binary are overlinked, the way we handle the linking doesn't = help > for that. What do you mean there ? Do you mean that some libraries specified for the linking stage of the final binary are not needed for the execution ? >=20 > On proposition could be to use pkgconf https://github.com/pkgconf/pkgconf= which > is BSD license pkg-config implementation 100% compatible with pkg-config. >=20 > What I propose is to create a new PCADD variable for the Makefiles. >=20 > PCADD will invoke pkgconf to gather the libraries and the cflags for a gi= ven > project. >=20 > The second thing would be to create .pc files for all of our libraries. >=20 > for example: > usr.bin/fstat dynamic build is overlinked And how this is better than just removing the unneeded library from the Makefile ? For the port consumption, I believe that the better solution is to provide a pack of the .pc files describing base libraries, most likely as port. Using .pc for the base system build is overkill, it does not add anything that cannot be accomplished by our existing build system. IMO. --nVYOjVWOcH+Ezkzp Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJQy9DYAAoJEJDCuSvBvK1BiCAP/3zRx2QSKw5BRn5TQzdJnjQi Tnq//KcfGrtD2fbjdICriYIrWePINSskxQIZsTXvGxLK0QhPFtPIsiqwkFLdUg8G 5H2WPp1NdV5eMwy/y8S3XfZRlj7HX0UMOH3AFa6qfGp0CURDRgsD6ixA5Cc5oJPj WjhytEKk3XM9dvI5+Usj0L7Q1IDtGVVVctCdbAL+6SVq2AT7q4DeCgfR2ZY7KUD2 lkVJwXGX8EwWHMU31vqR0ty0oL0aN8tdLTwsoT6u2VjFpTSJn6YdlgEuuJIKfckR a2nnV0fZ7ja8t2RDzXbUCIbD6BvDwaeyNBSnT4w7dBCsKHSlBpbUDF8a9RvFbKtb 6N+F6GRWqeuoICW/D5JmOUTWk5sn5cr4rrtVG3/EKzmXTb7nxEfD2H1x1fuQ9EV9 JqJRkfjwHqJXXAyBZMs6+mv9/aFK4dj7cB5tZLXzerk/tlLmkjBvCn3M8loT6D4p 9A3a2OBhw9ytjulaxntNyAFo2Y1sYUXet+hHZ3qrsngny5ZdspsfJRaet8L7bbKr h2V91CEsZ0yP5UZ9zkZ/4c2TS+LE/QEPDSamUWgGLSTByCZ8+qGOTE5/eVNevERm hzdaIC+4nRW9LXyVjEQL6dVTQhhE5Ka+skaKP3Fze1wmFFIO/vX8pmbeUngLbbOi ZmJayDL/44C9dGwmlu1I =HTJx -----END PGP SIGNATURE----- --nVYOjVWOcH+Ezkzp-- From owner-freebsd-hackers@FreeBSD.ORG Sat Dec 15 04:43:07 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 75E53A88 for ; Sat, 15 Dec 2012 04:43:07 +0000 (UTC) (envelope-from david.i.noel@gmail.com) Received: from mail-oa0-f54.google.com (mail-oa0-f54.google.com [209.85.219.54]) by mx1.freebsd.org (Postfix) with ESMTP id 359B38FC0A for ; Sat, 15 Dec 2012 04:43:06 +0000 (UTC) Received: by mail-oa0-f54.google.com with SMTP id n9so4316343oag.13 for ; Fri, 14 Dec 2012 20:43:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:date:message-id:subject:from:to:content-type; bh=paNwiAmqp3Gk8Kpoms9zqCMuSMGzVPM2/z7/CX81coY=; b=TAWJA1FHh6W9aCPemkXwEgzincT8Dx+tQtoOFgaXW9aRJvd62S1hzTzFMZX7kHN+R1 MrdcuOUZ7uQkiO9Kz2p3UMRnyM0jMjm6xp23u/2ZlqtAuPsfVLYJqQZx9wzKHNMwjh6z M4+pqOlddfKFGDpiWkSz+2qA/e9jRgP/saz01e8+T3IRgbgtt5sDhLOUFb9Xza/wssyd 9wYh0d+PEHQXGanbYZlB/ma/Vcr1LFDKEmIKF0XFR8axBZpELWodbf1NoDAOSMSEZeYO 3vn27BJL0USF3HWMifdPjjxwdh6M79HFu/kTPluWb83NriODlhFHYRSgeB7qQtl0KTHu PW3Q== MIME-Version: 1.0 Received: by 10.60.31.68 with SMTP id y4mr6566081oeh.121.1355546580184; Fri, 14 Dec 2012 20:43:00 -0800 (PST) Received: by 10.76.172.98 with HTTP; Fri, 14 Dec 2012 20:43:00 -0800 (PST) Date: Fri, 14 Dec 2012 22:43:00 -0600 Message-ID: Subject: postgres, initdb, FreeBSD bug? From: David Noel To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: David.I.Noel@gmail.com List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Dec 2012 04:43:07 -0000 I've been fighting with a bug I can't quite seem to figure out and was told that this might be the place to come. I'm running postgresql-9.2.2 on FreeBSD 8.3-RELEASE-p5 and am having things break down when I try to run initdb. I got in contact with the pgsql-general mailing list and we debugged the issue to the point where it seemed that this might be a FreeBSD-related error. Relevant excerpts from several email are below that piece together the error: I'm running into the following error message when running initdb (FreeBSD host): ygg# /usr/local/etc/rc.d/postgresql initdb -D /zdb/pgsql/data --debug The files belonging to this database system will be owned by user "pgsql". This user must also own the server process. The database cluster will be initialized with locales COLLATE: C CTYPE: en_US.UTF-8 MESSAGES: en_US.UTF-8 MONETARY: en_US.UTF-8 NUMERIC: en_US.UTF-8 TIME: en_US.UTF-8 The default text search configuration will be set to "english". creating directory /zdb/pgsql/data ... ok creating subdirectories ... ok selecting default max_connections ... 100 selecting default shared_buffers ... 32MB creating configuration files ... ok creating template1 database in /zdb/pgsql/data/base/1 ... FATAL: could not open file "pg_xlog/000000010000000000000001" (log file 0, segment 1): No such file or directory child process exited with exit code 1 initdb: removing data directory "/zdb/pgsql/data" ... Interestingly, I have a second--virtually identical--server that I just tried initdb on. FreeBSD 8.3-RELEASE-p5, postgresql-server-9.2.2. Exact same "FATAL: could not open file pg_xlog" error. So it is reproducible. ... The relevant part of the ktrace output is 71502 postgres CALL unlink(0x7fffffffc130) 71502 postgres NAMI "pg_xlog/xlogtemp.71502" 71502 postgres RET unlink -1 errno 2 No such file or directory 71502 postgres CALL open(0x7fffffffc130,O_RDWR|O_CREAT|O_EXCL,S_IRUSR|S_IWUSR) 71502 postgres NAMI "pg_xlog/xlogtemp.71502" 71502 postgres RET open 3 71502 postgres CALL write(0x3,0x801a56030,0x2000) 71502 postgres GIO fd 3 wrote 4096 bytes .... a lot of uninteresting write() calls snipped ... 71502 postgres RET write 8192/0x2000 71502 postgres CALL close(0x3) 71502 postgres RET close 0 71502 postgres CALL unlink(0x7fffffffbc60) 71502 postgres NAMI "pg_xlog/000000010000000000000001" 71502 postgres RET unlink -1 errno 2 No such file or directory 71502 postgres CALL link(0x7fffffffc130,0x7fffffffbc60) 71502 postgres NAMI "pg_xlog/xlogtemp.71502" 71502 postgres NAMI "pg_xlog/000000010000000000000001" 71502 postgres RET link -1 errno 1 Operation not permitted 71502 postgres CALL unlink(0x7fffffffc130) 71502 postgres NAMI "pg_xlog/xlogtemp.71502" 71502 postgres RET unlink 0 71502 postgres CALL open(0x7fffffffc530,O_RDWR,0x180) 71502 postgres NAMI "pg_xlog/000000010000000000000001" 71502 postgres RET open -1 errno 2 No such file or directory This corresponds to the execution of XLogFileInit(), and what's evidently happening is that we successfully create and zero-fill the first xlog segment file under a temporary name, but then the attempt to rename it into place with link() fails with EPERM. This is really a WTF kind of failure, I think. The directory is certainly writable --- it was made under our own UID, and what's more we just managed to create the file there under its temp name. So how can we get an EPERM failure from link()? I think this is a kernel bug. regards, tom lane PS: one odd thing here is that the ereport(LOG) in InstallXLogFileSegment isn't doing anything; otherwise we'd have gotten a much more helpful error report about "could not link file". I don't think we run the bootstrap mode with log_min_messages set high enough to disable LOG messages, so why isn't it printing? Nonetheless, this error shouldn't have occurred. ... Where to from here? The freebsd-database@freebsd.org mailing list? The postgresql port maintainer? Who should I be in touch with? ... You need to talk to some FreeBSD kernel hackers about why link() might be failing here. Since you see it on UFS too, we can probably exonerate the ZFS filesystem-specific code. I did some googling and found that EPERM can be issued if the filesystem doesn't support hard links (which shouldn't apply to ZFS I trust). Also, Linux has a "protected_hardlinks" option that causes certain attempts at creating hard links to fail --- but our use-case here doesn't fall foul of any of those restrictions AFAICS, and of course FreeBSD isn't Linux. Still, I wonder if you're running into some misdesigned or misimplemented security restriction. You might want to look at your kernel parameters and see if any of them look like they might have to do with restricting hard-link operations. Also, since Amitabh failed to duplicate the failure on both earlier and later FreeBSD kernels, and we've not heard reports of this from anybody else either, it seems more than possible that it's a plain old bug in the specific kernel version you're using. As a short-term workaround, I'd suggest rebuilding with HAVE_WORKING_LINK disabled. (Just remove that #define from src/include/pg_config_manual.h and rebuild.) regards, tom lane ... Does this make any sense to anyone? -David From owner-freebsd-hackers@FreeBSD.ORG Sat Dec 15 04:55:33 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CB0C8C22 for ; Sat, 15 Dec 2012 04:55:33 +0000 (UTC) (envelope-from david.i.noel@gmail.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 8A1D28FC1C for ; Sat, 15 Dec 2012 04:55:33 +0000 (UTC) Received: by mail-ob0-f182.google.com with SMTP id 16so4116873obc.13 for ; Fri, 14 Dec 2012 20:55:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:content-type; bh=d1Mh0op93HoFebMz/sdPFXDGBLom1rf7+mEr2pN4zYE=; b=jLfT+Wh7robeiuihBeKBTHM1puTdfPN+BKhXYYMA2fv4Vsgdp2OX+54Wf74stq6I+9 IRu3hog1GuQcUrFmDVo6+4cN6p00akTSB3MZ55Y5Lv/RN0T6LH6V8Qg0iB3T/MzlKB/X F9LOg52a1PJf28IwuMnyt3m+T5Kweb7HvOGrlq73Nck71Aj6ANBFAmy9LT967dVGcgzz oem6LjLtptA5LFWGr1JgAGo63wjrLlMCSUxdt6duOOzyeeivVstZTfC73t+TNYVpRqLy YCKNPz+fBpW9a7O/SKWgLlKH3cd4QvhO2CNVNcNchyxXpxtliU3pfi5OP2xpazT/SQyQ eZmA== MIME-Version: 1.0 Received: by 10.60.31.68 with SMTP id y4mr6582361oeh.121.1355547332983; Fri, 14 Dec 2012 20:55:32 -0800 (PST) Received: by 10.76.172.98 with HTTP; Fri, 14 Dec 2012 20:55:32 -0800 (PST) In-Reply-To: References: Date: Fri, 14 Dec 2012 22:55:32 -0600 Message-ID: Subject: Re: postgres, initdb, FreeBSD bug? From: David Noel To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: David.I.Noel@gmail.com List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Dec 2012 04:55:33 -0000 I just compiled a generic kernel and tried it again.. same error. So no funny business going on with my kernel settings it seems. From owner-freebsd-hackers@FreeBSD.ORG Sat Dec 15 04:27:14 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4A22E9A8 for ; Sat, 15 Dec 2012 04:27:14 +0000 (UTC) (envelope-from david.i.noel@gmail.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 0B2388FC19 for ; Sat, 15 Dec 2012 04:27:13 +0000 (UTC) Received: by mail-ob0-f182.google.com with SMTP id 16so4106294obc.13 for ; Fri, 14 Dec 2012 20:27:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:date:message-id:subject:from:to:content-type; bh=ZThbgbeejs+J6X09dd8BuSUbZucvlNqgaA7isIZfU2k=; b=mQMvpm17+mFAmco/S4R1doDRqFpCWpYK6rEDZaytOOd30GySq7RfkPd0on5pIh6dcd 8oO2LNKUUM6EfEY2mkesWJlYozbAHGyuYV06eMgo2GdUyTYkCNduxrQ/8EZZUJfNDtxH nRF7nxGufPMfmS9EbtSRJiG+kG5J1MpykFgZEtIn6NrQh5fSV7sUIedlpI/ncPrtsqdq 6qp7wpbczFOqvdc56b01Gn1swcHiyXJwSJxACREkZkMKOgXtAB6hhpnl6tXnTdxiOlkJ s7s4wGQDyVQ1DcxLK5VxRNxAS/+zhNGLCHrFfWcVq+1PqujOPDMXbcQj5VLLdZo/pciZ Gj8g== MIME-Version: 1.0 Received: by 10.60.32.50 with SMTP id f18mr6609321oei.8.1355545633148; Fri, 14 Dec 2012 20:27:13 -0800 (PST) Received: by 10.76.172.98 with HTTP; Fri, 14 Dec 2012 20:27:13 -0800 (PST) Date: Fri, 14 Dec 2012 22:27:13 -0600 Message-ID: Subject: postgres, initdb, FreeBSD bug? From: David Noel To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Mailman-Approved-At: Sat, 15 Dec 2012 05:08:30 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: David.I.Noel@gmail.com List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Dec 2012 04:27:14 -0000 I've been fighting with a bug I can't quite seem to figure out and was told that this might be the place to come. I'm running postgresql-9.2.2 on FreeBSD 8.3-RELEASE-p5 and am having things break down when I try to run initdb. I got in contact with the pgsql-general mailing list and we debugged the issue to the point where it seemed that this might be a FreeBSD-related error. Relevant excerpts from several email are below that piece together the error: I'm running into the following error message when running initdb (FreeBSD host): ygg# /usr/local/etc/rc.d/postgresql initdb -D /zdb/pgsql/data --debug The files belonging to this database system will be owned by user "pgsql". This user must also own the server process. The database cluster will be initialized with locales COLLATE: C CTYPE: en_US.UTF-8 MESSAGES: en_US.UTF-8 MONETARY: en_US.UTF-8 NUMERIC: en_US.UTF-8 TIME: en_US.UTF-8 The default text search configuration will be set to "english". creating directory /zdb/pgsql/data ... ok creating subdirectories ... ok selecting default max_connections ... 100 selecting default shared_buffers ... 32MB creating configuration files ... ok creating template1 database in /zdb/pgsql/data/base/1 ... FATAL: could not open file "pg_xlog/000000010000000000000001" (log file 0, segment 1): No such file or directory child process exited with exit code 1 initdb: removing data directory "/zdb/pgsql/data" ... Interestingly, I have a second--virtually identical--server that I just tried initdb on. FreeBSD 8.3-RELEASE-p5, postgresql-server-9.2.2. Exact same "FATAL: could not open file pg_xlog" error. So it is reproducible. ... The relevant part of the ktrace output is 71502 postgres CALL unlink(0x7fffffffc130) 71502 postgres NAMI "pg_xlog/xlogtemp.71502" 71502 postgres RET unlink -1 errno 2 No such file or directory 71502 postgres CALL open(0x7fffffffc130,O_RDWR|O_CREAT|O_EXCL,S_IRUSR|S_IWUSR) 71502 postgres NAMI "pg_xlog/xlogtemp.71502" 71502 postgres RET open 3 71502 postgres CALL write(0x3,0x801a56030,0x2000) 71502 postgres GIO fd 3 wrote 4096 bytes .... a lot of uninteresting write() calls snipped ... 71502 postgres RET write 8192/0x2000 71502 postgres CALL close(0x3) 71502 postgres RET close 0 71502 postgres CALL unlink(0x7fffffffbc60) 71502 postgres NAMI "pg_xlog/000000010000000000000001" 71502 postgres RET unlink -1 errno 2 No such file or directory 71502 postgres CALL link(0x7fffffffc130,0x7fffffffbc60) 71502 postgres NAMI "pg_xlog/xlogtemp.71502" 71502 postgres NAMI "pg_xlog/000000010000000000000001" 71502 postgres RET link -1 errno 1 Operation not permitted 71502 postgres CALL unlink(0x7fffffffc130) 71502 postgres NAMI "pg_xlog/xlogtemp.71502" 71502 postgres RET unlink 0 71502 postgres CALL open(0x7fffffffc530,O_RDWR,0x180) 71502 postgres NAMI "pg_xlog/000000010000000000000001" 71502 postgres RET open -1 errno 2 No such file or directory This corresponds to the execution of XLogFileInit(), and what's evidently happening is that we successfully create and zero-fill the first xlog segment file under a temporary name, but then the attempt to rename it into place with link() fails with EPERM. This is really a WTF kind of failure, I think. The directory is certainly writable --- it was made under our own UID, and what's more we just managed to create the file there under its temp name. So how can we get an EPERM failure from link()? I think this is a kernel bug. regards, tom lane PS: one odd thing here is that the ereport(LOG) in InstallXLogFileSegment isn't doing anything; otherwise we'd have gotten a much more helpful error report about "could not link file". I don't think we run the bootstrap mode with log_min_messages set high enough to disable LOG messages, so why isn't it printing? Nonetheless, this error shouldn't have occurred. ... Where to from here? The freebsd-database@freebsd.org mailing list? The postgresql port maintainer? Who should I be in touch with? ... You need to talk to some FreeBSD kernel hackers about why link() might be failing here. Since you see it on UFS too, we can probably exonerate the ZFS filesystem-specific code. I did some googling and found that EPERM can be issued if the filesystem doesn't support hard links (which shouldn't apply to ZFS I trust). Also, Linux has a "protected_hardlinks" option that causes certain attempts at creating hard links to fail --- but our use-case here doesn't fall foul of any of those restrictions AFAICS, and of course FreeBSD isn't Linux. Still, I wonder if you're running into some misdesigned or misimplemented security restriction. You might want to look at your kernel parameters and see if any of them look like they might have to do with restricting hard-link operations. Also, since Amitabh failed to duplicate the failure on both earlier and later FreeBSD kernels, and we've not heard reports of this from anybody else either, it seems more than possible that it's a plain old bug in the specific kernel version you're using. As a short-term workaround, I'd suggest rebuilding with HAVE_WORKING_LINK disabled. (Just remove that #define from src/include/pg_config_manual.h and rebuild.) regards, tom lane ... Does this make any sense to anyone? -David From owner-freebsd-hackers@FreeBSD.ORG Sat Dec 15 09:15:53 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9D30680F; Sat, 15 Dec 2012 09:15:53 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id 38F348FC15; Sat, 15 Dec 2012 09:15:53 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.5/8.14.5) with ESMTP id qBF9FmhU025900; Sat, 15 Dec 2012 11:15:48 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.7.3 kib.kiev.ua qBF9FmhU025900 Received: (from kostik@localhost) by tom.home (8.14.5/8.14.5/Submit) id qBF9Fmut025899; Sat, 15 Dec 2012 11:15:48 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 15 Dec 2012 11:15:48 +0200 From: Konstantin Belousov To: Bernhard Fr?hlich Subject: Re: Fix overlinking in base aka import pkgconf Message-ID: <20121215091548.GT71906@kib.kiev.ua> References: <20121214235418.GF18884@ithaqua.etoilebsd.net> <20121215012233.GP71906@kib.kiev.ua> <1355558294.7933.2.camel@Nokia-N900-42-11> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="h/77pM/XNBmEJr8g" Content-Disposition: inline In-Reply-To: <1355558294.7933.2.camel@Nokia-N900-42-11> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: Baptiste Daroussin , hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Dec 2012 09:15:53 -0000 --h/77pM/XNBmEJr8g Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Dec 15, 2012 at 08:58:14AM +0100, Bernhard Fr?hlich wrote: > On Sa., 15. Dez. 2012 02:22:34 CET, Konstantin Belousov wrote: >=20 > > On Sat, Dec 15, 2012 at 12:54:19AM +0100, Baptiste Daroussin wrote: > > > Hi, > > >=20 > > > Some of our binary are overlinked, the way we handle the linking > > > doesn't help for that. > > What do you mean there ? Do you mean that some libraries specified for > > the linking stage of the final binary are not needed for the execution ? > >=20 > > >=20 > > > On proposition could be to use pkgconf > > > https://github.com/pkgconf/pkgconf which is BSD license pkg-config > > > implementation 100% compatible with pkg-config. > > >=20 > > > What I propose is to create a new PCADD variable for the Makefiles. > > >=20 > > > PCADD will invoke pkgconf to gather the libraries and the cflags for a > > > given project. > > >=20 > > > The second thing would be to create .pc files for all of our librarie= s. > > >=20 > > > for example: > > > usr.bin/fstat dynamic build is overlinked > > And how this is better than just removing the unneeded library from > > the Makefile ? > >=20 > > For the port consumption, I believe that the better solution is to > > provide a pack of the .pc files describing base libraries, most likely > > as port. >=20 > We should definitely generate some pc files for our base libraries. > We already have quite a few ports that need to hack around because of > missing pc files for ssl for example. This is fine. But making the base build depend on the pkgconf does not give us anything except complications and overhead, in my opinion. --h/77pM/XNBmEJr8g Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJQzD/CAAoJEJDCuSvBvK1B7NwP+gOtvkXZoYL8nYTJGtaokGHU s35gG1/7y7biksaYBp4c36mNy+MtZ1l2xSeot17su8mf4o53cnh2DPFKOXfJvSx9 1aeEp2QXKn+Etr97n5rUsS+XtlSQeZBOIM+uUtuveuwqmNSNGECELB2+vHvXb0lS rLDsOsQ1rxoki8fmyr7XTDG1hwz7S12fsC/cytt5vbr3mTuR2WrSDryqek6fOcwd gc0O6DRn8vRcMJlAsYTxkNunxmb4fg9Seo4NlNTiajwl0juE1g1wHBPN4v8gDT+J hzTRsNlJX17CWKp1RxSIEMFNOMYYbHz3GyU6cyHe3y708xnTLp8ASvDza8TPATi2 7ekB+ssHrIM8Ntm/Fc0W2iEx3H8Zs/KrTqEAQZqRnQjydt48RHg5/RL0Qvu2TqGZ pCEDrGbTiNw5vUb4kCkJ1DB6PAQR11vZGHCOVDf7xPpwV5kpJtk5KDRTROhpRlGg 1fOAj2Z+GqZrMDapFu4GW7gPg/YyPH6/xlbtsgZe2NqGgWrJdQ5PKb5zxbQeujdj G7SMc3fp1lv9YoV/11sJlcBJIJGb5uBT5zephr2i2EMR1X16BrHYewG/YgjU9AME nNigsfpYmajn8i0geboF7VUHiEji50kQrg/lYbo11PiFg//aKmkzR2BK6wyx0Geb 5Aqj17p7RvEnR+tVsAux =vDzD -----END PGP SIGNATURE----- --h/77pM/XNBmEJr8g-- From owner-freebsd-hackers@FreeBSD.ORG Sat Dec 15 09:53:34 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4430431B for ; Sat, 15 Dec 2012 09:53:34 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id A82328FC0A for ; Sat, 15 Dec 2012 09:53:33 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.5/8.14.5) with ESMTP id qBF9rONT029314; Sat, 15 Dec 2012 11:53:24 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.7.3 kib.kiev.ua qBF9rONT029314 Received: (from kostik@localhost) by tom.home (8.14.5/8.14.5/Submit) id qBF9rOSq029313; Sat, 15 Dec 2012 11:53:24 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 15 Dec 2012 11:53:24 +0200 From: Konstantin Belousov To: David Noel Subject: Re: postgres, initdb, FreeBSD bug? Message-ID: <20121215095324.GU71906@kib.kiev.ua> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="wu6d9FdQ4ohoCGf7" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Dec 2012 09:53:34 -0000 --wu6d9FdQ4ohoCGf7 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Dec 14, 2012 at 10:43:00PM -0600, David Noel wrote: > I've been fighting with a bug I can't quite seem to figure out and was > told that this might be the place to come. I'm running > postgresql-9.2.2 on FreeBSD 8.3-RELEASE-p5 and am having things break > down when I try to run initdb. I got in contact with the pgsql-general > mailing list and we debugged the issue to the point where it seemed > that this might be a FreeBSD-related error. Relevant excerpts from > several email are below that piece together the error: >=20 > I'm running into the following error message when running initdb (FreeBSD= host): >=20 > ygg# /usr/local/etc/rc.d/postgresql initdb -D /zdb/pgsql/data --debug > The files belonging to this database system will be owned by user "pgs= ql". > This user must also own the server process. >=20 > The database cluster will be initialized with locales > COLLATE: C > CTYPE: en_US.UTF-8 > MESSAGES: en_US.UTF-8 > MONETARY: en_US.UTF-8 > NUMERIC: en_US.UTF-8 > TIME: en_US.UTF-8 > The default text search configuration will be set to "english". >=20 > creating directory /zdb/pgsql/data ... ok > creating subdirectories ... ok > selecting default max_connections ... 100 > selecting default shared_buffers ... 32MB > creating configuration files ... ok > creating template1 database in /zdb/pgsql/data/base/1 ... FATAL: > could not open file "pg_xlog/000000010000000000000001" (log file 0, > segment 1): No such file or directory > child process exited with exit code 1 > initdb: removing data directory "/zdb/pgsql/data" >=20 > ... >=20 > Interestingly, I have a second--virtually identical--server that I > just tried initdb on. FreeBSD 8.3-RELEASE-p5, postgresql-server-9.2.2. > Exact same "FATAL: could not open file pg_xlog" error. So it is > reproducible. >=20 > ... >=20 > The relevant part of the ktrace output is >=20 > 71502 postgres CALL unlink(0x7fffffffc130) > 71502 postgres NAMI "pg_xlog/xlogtemp.71502" > 71502 postgres RET unlink -1 errno 2 No such file or directory > 71502 postgres CALL > open(0x7fffffffc130,O_RDWR|O_CREAT|O_EXCL,S_IRUSR|S_IWUSR) > 71502 postgres NAMI "pg_xlog/xlogtemp.71502" > 71502 postgres RET open 3 > 71502 postgres CALL write(0x3,0x801a56030,0x2000) > 71502 postgres GIO fd 3 wrote 4096 bytes > .... a lot of uninteresting write() calls snipped ... > 71502 postgres RET write 8192/0x2000 > 71502 postgres CALL close(0x3) > 71502 postgres RET close 0 > 71502 postgres CALL unlink(0x7fffffffbc60) > 71502 postgres NAMI "pg_xlog/000000010000000000000001" > 71502 postgres RET unlink -1 errno 2 No such file or directory > 71502 postgres CALL link(0x7fffffffc130,0x7fffffffbc60) > 71502 postgres NAMI "pg_xlog/xlogtemp.71502" > 71502 postgres NAMI "pg_xlog/000000010000000000000001" > 71502 postgres RET link -1 errno 1 Operation not permitted > 71502 postgres CALL unlink(0x7fffffffc130) > 71502 postgres NAMI "pg_xlog/xlogtemp.71502" > 71502 postgres RET unlink 0 > 71502 postgres CALL open(0x7fffffffc530,O_RDWR,0x180) > 71502 postgres NAMI "pg_xlog/000000010000000000000001" > 71502 postgres RET open -1 errno 2 No such file or directory >=20 > This corresponds to the execution of XLogFileInit(), and what's > evidently happening is that we successfully create and zero-fill > the first xlog segment file under a temporary name, but then > the attempt to rename it into place with link() fails with EPERM. >=20 > This is really a WTF kind of failure, I think. The directory is > certainly writable --- it was made under our own UID, and what's > more we just managed to create the file there under its temp name. > So how can we get an EPERM failure from link()? >=20 > I think this is a kernel bug. >=20 > regards, tom lane >=20 > PS: one odd thing here is that the ereport(LOG) in > InstallXLogFileSegment isn't doing anything; otherwise we'd have gotten > a much more helpful error report about "could not link file". I don't > think we run the bootstrap mode with log_min_messages set high enough to > disable LOG messages, so why isn't it printing? Nonetheless, this error > shouldn't have occurred. >=20 > ... >=20 > Where to from here? The freebsd-database@freebsd.org mailing list? The > postgresql port maintainer? Who should I be in touch with? >=20 > ... >=20 > You need to talk to some FreeBSD kernel hackers about why link() > might be failing here. Since you see it on UFS too, we can probably > exonerate the ZFS filesystem-specific code. >=20 > I did some googling and found that EPERM can be issued if the filesystem > doesn't support hard links (which shouldn't apply to ZFS I trust). > Also, Linux has a "protected_hardlinks" option that causes certain > attempts at creating hard links to fail --- but our use-case here > doesn't fall foul of any of those restrictions AFAICS, and of course > FreeBSD isn't Linux. Still, I wonder if you're running into some > misdesigned or misimplemented security restriction. You might want > to look at your kernel parameters and see if any of them look like > they might have to do with restricting hard-link operations. >=20 > Also, since Amitabh failed to duplicate the failure on both earlier > and later FreeBSD kernels, and we've not heard reports of this from > anybody else either, it seems more than possible that it's a plain > old bug in the specific kernel version you're using. >=20 > As a short-term workaround, I'd suggest rebuilding with > HAVE_WORKING_LINK disabled. (Just remove that #define from > src/include/pg_config_manual.h and rebuild.) >=20 > regards, tom lane >=20 > ... >=20 > Does this make any sense to anyone? Show the ktrace from the same error on UFS. Show the security.bsd sysctl settings, in particular, harlink_check_{u,g}id. Show the ls -la output for the pg_xlog directory. --wu6d9FdQ4ohoCGf7 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJQzEiTAAoJEJDCuSvBvK1BUucP/ieTa+3u54nQ6WOpQ696Kjym jj9wdgonQtUDc1/AR45xKTmq+VaIMox2G7edxI51kO+XAhYQJXf9t3a15UX3FqN5 dzPpnaDbn8bwYg1ppKMeOMF8txi7ERdfzZIcE0ScNwIDJxOyH66TXHV8eEvBg1g/ HMWM8HtWnB9eSjPtIRB6NpMIo/GqIxeCL4QJMDIBoZ2GBuf1JdUdaHcjDkXu/2J3 a7DuvXLIDJ77FnIKzg98V92NFv7rMr/EJSPJGdDWIW/hKfonIgnApcRiYreyjBb4 GnsHRPqTIX/OnjS3pyvfwPypzlK5Dfn2GEvkuKpaPoOotz2OdeKDni5gbDNbBSzx YhmIsrZOZqTAC7RGufSwFJzoFWuqI0SoFtpU+mNmB4uatea09IIyc4Kkmt2bsj0e N+L5CrcmIuM1Oj3IJHZQ18GELHbe3LhAWtaeE5XgVivPbR9z4uzwgjO1dUnDR4nU gdLC//5aUIqQjRDgnY+Gb4F8nusrEhACnwsAJtERWOudAPel3TBTsDcLSp1qyluf 8J/hOmEEhlFs76sZPIz13YV1QjtLDcrxDksifu5jjNFgwez5DJI2uU8qq9OlQAXg vOZga1YDpcwmUvWUmVuFWc5lg983T3yqC7dQoPZQ8Nq02BQWmQ51rN20ChkEJosq WpNMNS5IgwbXVuoatKco =3Mjl -----END PGP SIGNATURE----- --wu6d9FdQ4ohoCGf7-- From owner-freebsd-hackers@FreeBSD.ORG Sat Dec 15 10:56:47 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id DC3FC85A for ; Sat, 15 Dec 2012 10:56:47 +0000 (UTC) (envelope-from baptiste.daroussin@gmail.com) Received: from mail-we0-f182.google.com (mail-we0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 63EA08FC18 for ; Sat, 15 Dec 2012 10:56:47 +0000 (UTC) Received: by mail-we0-f182.google.com with SMTP id u54so1960980wey.13 for ; Sat, 15 Dec 2012 02:56:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=oGk43HTgH+2KYcU+4vdoXWMsp0J7ZUmrq12nWM7kAPo=; b=SvsRtMW5fcRjoBQ1HCthSrHhutR4AOXeYQ2Jy3IO+h4+Vf10CbGqTQ4BZxxMyq1dHm aqUGZDHF9clCkzgCZrUZtnVXmmrnNkrmddpoDPT1sMuWDRrDuGqKRyp+c0WyB3ITJWPW KLNK6G9vI1SOaSKfdLFqcMTQs9tXCahHDUdVqOM6viXdWh2PRyiVOl2I/ZfEFxBQx0NS aPnzxyXZ+E8m2UC4DlA8OUgJVbhI0q8TEiyLrua6bWWNHU+Y3cJeFWpgtzdxEKkAxa95 oGPLmIYaduA2xuBxDcNVr/5VvZwWsKsRtbIhFK1dTIii8nCb2Ko3pndHKhGmgWe+2fPG xu/A== Received: by 10.180.80.168 with SMTP id s8mr7023233wix.8.1355569005409; Sat, 15 Dec 2012 02:56:45 -0800 (PST) Received: from ithaqua.etoilebsd.net (ithaqua.etoilebsd.net. [37.59.37.188]) by mx.google.com with ESMTPS id i2sm1544449wiw.3.2012.12.15.02.56.44 (version=TLSv1/SSLv3 cipher=OTHER); Sat, 15 Dec 2012 02:56:44 -0800 (PST) Sender: Baptiste Daroussin Date: Sat, 15 Dec 2012 11:56:43 +0100 From: Baptiste Daroussin To: Konstantin Belousov Subject: Re: Fix overlinking in base aka import pkgconf Message-ID: <20121215105643.GG18884@ithaqua.etoilebsd.net> References: <20121214235418.GF18884@ithaqua.etoilebsd.net> <20121215012233.GP71906@kib.kiev.ua> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="HKEL+t8MFpg/ASTE" Content-Disposition: inline In-Reply-To: <20121215012233.GP71906@kib.kiev.ua> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Dec 2012 10:56:47 -0000 --HKEL+t8MFpg/ASTE Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Dec 15, 2012 at 03:22:34AM +0200, Konstantin Belousov wrote: > On Sat, Dec 15, 2012 at 12:54:19AM +0100, Baptiste Daroussin wrote: > > Hi, > >=20 > > Some of our binary are overlinked, the way we handle the linking doesn'= t help > > for that. > What do you mean there ? Do you mean that some libraries specified for the > linking stage of the final binary are not needed for the execution ? I mean some library are registrer in the binary with DT_NEEDED tag where th= ey don't need to. >=20 > >=20 > > On proposition could be to use pkgconf https://github.com/pkgconf/pkgco= nf which > > is BSD license pkg-config implementation 100% compatible with pkg-confi= g. > >=20 > > What I propose is to create a new PCADD variable for the Makefiles. > >=20 > > PCADD will invoke pkgconf to gather the libraries and the cflags for a = given > > project. > >=20 > > The second thing would be to create .pc files for all of our libraries. > >=20 > > for example: > > usr.bin/fstat dynamic build is overlinked > And how this is better than just removing the unneeded library from > the Makefile ? In that case to statically build you need -lkvm -lutil -lprocstat but if you build dynamically you will only need -lproctast meaning you don't need to be directly linked to libutil and libkvm. This means a breakage of libutil will only have inpact on procstat and no more on fstat for example. >=20 > For the port consumption, I believe that the better solution is to provide > a pack of the .pc files describing base libraries, most likely as port. Yeah the port is another thing which yes can probably be done that way. >=20 > Using .pc for the base system build is overkill, it does not add anything > that cannot be accomplished by our existing build system. IMO. Probably. The thing is with pkgconf, fstat does not need to know that procstat needs libkvm and libutil for static link, it just has to know it needs proc= stat and pkgconf does all the magic. and pkgconf is really small (only 48k on an amd64 box) Other solution would be to reinvent the same thing using our framework? Maybe a LDSADD (LD STATIC ADD) to differenciate both? regards, Bapt --HKEL+t8MFpg/ASTE Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlDMV2sACgkQ8kTtMUmk6EywggCfcYWzp3/daYFKOwX3YD1gFgDK rLIAn1iujMVzzxGDTw5iqpQfFlZVuiLe =xxMg -----END PGP SIGNATURE----- --HKEL+t8MFpg/ASTE-- From owner-freebsd-hackers@FreeBSD.ORG Sat Dec 15 11:07:59 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A22A4A3F; Sat, 15 Dec 2012 11:07:59 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id EB8E18FC14; Sat, 15 Dec 2012 11:07:58 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.5/8.14.5) with ESMTP id qBFB7skH036614; Sat, 15 Dec 2012 13:07:54 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.7.3 kib.kiev.ua qBFB7skH036614 Received: (from kostik@localhost) by tom.home (8.14.5/8.14.5/Submit) id qBFB7sKx036613; Sat, 15 Dec 2012 13:07:54 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 15 Dec 2012 13:07:54 +0200 From: Konstantin Belousov To: Baptiste Daroussin Subject: Re: Fix overlinking in base aka import pkgconf Message-ID: <20121215110754.GV71906@kib.kiev.ua> References: <20121214235418.GF18884@ithaqua.etoilebsd.net> <20121215012233.GP71906@kib.kiev.ua> <20121215105643.GG18884@ithaqua.etoilebsd.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="9xA8aadJAx1hWuKz" Content-Disposition: inline In-Reply-To: <20121215105643.GG18884@ithaqua.etoilebsd.net> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Dec 2012 11:07:59 -0000 --9xA8aadJAx1hWuKz Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Dec 15, 2012 at 11:56:43AM +0100, Baptiste Daroussin wrote: > On Sat, Dec 15, 2012 at 03:22:34AM +0200, Konstantin Belousov wrote: > > On Sat, Dec 15, 2012 at 12:54:19AM +0100, Baptiste Daroussin wrote: > > > Hi, > > >=20 > > > Some of our binary are overlinked, the way we handle the linking does= n't help > > > for that. > > What do you mean there ? Do you mean that some libraries specified for = the > > linking stage of the final binary are not needed for the execution ? >=20 > I mean some library are registrer in the binary with DT_NEEDED tag where = they > don't need to. >=20 > >=20 > > >=20 > > > On proposition could be to use pkgconf https://github.com/pkgconf/pkg= conf which > > > is BSD license pkg-config implementation 100% compatible with pkg-con= fig. > > >=20 > > > What I propose is to create a new PCADD variable for the Makefiles. > > >=20 > > > PCADD will invoke pkgconf to gather the libraries and the cflags for = a given > > > project. > > >=20 > > > The second thing would be to create .pc files for all of our librarie= s. > > >=20 > > > for example: > > > usr.bin/fstat dynamic build is overlinked > > And how this is better than just removing the unneeded library from > > the Makefile ? >=20 > In that case to statically build you need -lkvm -lutil -lprocstat but if = you > build dynamically you will only need -lproctast meaning you don't need to= be > directly linked to libutil and libkvm. This means a breakage of libutil w= ill > only have inpact on procstat and no more on fstat for example. >=20 > >=20 > > For the port consumption, I believe that the better solution is to prov= ide > > a pack of the .pc files describing base libraries, most likely as port. >=20 > Yeah the port is another thing which yes can probably be done that way. >=20 > >=20 > > Using .pc for the base system build is overkill, it does not add anythi= ng > > that cannot be accomplished by our existing build system. IMO. >=20 > Probably. > The thing is with pkgconf, fstat does not need to know that procstat > needs libkvm and libutil for static link, it just has to know it needs pr= ocstat > and pkgconf does all the magic. and pkgconf is really small (only 48k on = an > amd64 box) >=20 > Other solution would be to reinvent the same thing using our framework? > Maybe a LDSADD (LD STATIC ADD) to differenciate both? First, there is a linker flag, --as-needed, which does exactly what you want: it only registers the dso as a dependency using DT_NEEDED dynamic tag when there are real references from the linked object to the symbols in the dso. On the other hand, I am not a big fun of this feature. Dynamic linker is permissive for the main binary to reference a symbols in implicit dependencies of the explicit dependencies. If sudden change causes the main binary to start using a symbol from 'second order' dsos, you cause unneccessary complications there. More, recently there was a trend in the Linux world to disable the static linker from resolving symbols from the second order dsos. See the --no-add-needed and its recent switch in the newer ld. Overall, I like the fact that we explicitely list all depended base librari= es in the link command, for the base system. --9xA8aadJAx1hWuKz Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJQzFoJAAoJEJDCuSvBvK1B06QP+wfKvb/2Rt4Oaf7IA8HgOu28 B9ichNuUasraRhi4Kk9O+J2Q96O9jXr3m/dBv+MP6S+iJrMrfZaW9kEc/e4rWCeo hp0KIgb5Cq0/MnOOWXl/IVXu8ZDTFCJtVr/iQSBdOZINxYumImPSM9QuD6d0yttP tXzHY/aF/iaqgba0cY6esAz0Drf9TWTdg1HJtAx1uUInbM3Q3NpoEa/8xy7zEgUr 5WKoaFZbhBi0HUT1j9Xme6nC3wFpjj2lTC00gVIgtg+lYU6VxbKywuHtt1NOaRlX jXzN2x55KZ6Gq1lhN50Ku0ElubEA7F7WRO3NO+Ree5bxI1oX2Y4Oryo0AVfoGI9J GsHff+1of2MZlHP3XeQEczYAjjEn9kVSoi5NOn9uavZL7EEUe5nHxqyxbKouAaqF Oham0/et38i3RhfC7n2KkxGDRPSrRqWq3P2hQS1fr9LEMI4YoP7zlWh7JTiZIWdN RPDPGrcMwDqvUgqZA5QENBuzcQnrNVP1smHuggSWXutEfuBlyX+xaYjTUfcNYNqu WVElBratCGWAUlQYB9sdjKA1jV0XreCJ6LoFIK6VUBPUBmqi9X8IgXXfeZ7SgcYY kmeljICrP+Khu/2sXJoNEc69dmMe9qodOCqTrXjlyvBz1vbBBJwEBysqeaZcGNKF w1JN1JLdlC5oHbIi+7N7 =/Oaj -----END PGP SIGNATURE----- --9xA8aadJAx1hWuKz-- From owner-freebsd-hackers@FreeBSD.ORG Sat Dec 15 07:58:15 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4B6F28C9 for ; Sat, 15 Dec 2012 07:58:15 +0000 (UTC) (envelope-from decke@bluelife.at) Received: from mail-ea0-f182.google.com (mail-ea0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id BEB9B8FC14 for ; Sat, 15 Dec 2012 07:58:14 +0000 (UTC) Received: by mail-ea0-f182.google.com with SMTP id a14so1615395eaa.13 for ; Fri, 14 Dec 2012 23:58:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bluelife.at; s=google; h=from:reply-to:to:cc:subject:x-mailer:references:in-reply-to :content-type:content-id:date:message-id:mime-version :content-transfer-encoding; bh=MGXXSwaWBqlQg4obHfgP7bwwSRWldPOkJSYOvK8HAKY=; b=iXB589UQsMJrF0Qg5JRhVYQk/qLYeG6UkGIgPLtS8QkrGwr7CWE6n4e6HStdMmMupU fjADMcd44svk8IPFGn4GlrywpZinjM3sKSnUEpdaKRTkF4GEMes0mR4eULfkLXf6/KJQ ENlw/qX9EzZZSXvPJ+54wt0puas9X4h2LLkUg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=from:reply-to:to:cc:subject:x-mailer:references:in-reply-to :content-type:content-id:date:message-id:mime-version :content-transfer-encoding:x-gm-message-state; bh=MGXXSwaWBqlQg4obHfgP7bwwSRWldPOkJSYOvK8HAKY=; b=Cjq6xc8ZshIsJOxsPLgFGIDSizKvOT24iP2M/D3qB8dwiFBtLkxXvU7ZIFVQxJ+GTg Qw/X0cm4ZmgTeFaAn5E5ryTyuXmphbMY8+NWFsuwBQ/9vNHgpktYUobisbHbpGe/Pvga o4DXkzt85bPtNlWBphzIX/RmVGs5Cvz9HXUAVlNDvsYB/En7+cNhs6rOeUB/AK+Cuhb9 kjGHcgQ7scoPwRl451OAvPOQtIDhDB4YXild/hCytNyyZBtyyV3jNDNMf7ABTX7H2+uQ bpLJveUi01njdm1IS/jq9dyjFcXkAswmZFkEqtRBhvpJxGupmfZqEOih6uBpUhQZji2w E/8A== Received: by 10.14.184.131 with SMTP id s3mr21939227eem.38.1355558288474; Fri, 14 Dec 2012 23:58:08 -0800 (PST) Received: from [10.40.86.74] (089144192193.atnat0001.highway.a1.net. [89.144.192.193]) by mx.google.com with ESMTPS id 43sm14243896eed.10.2012.12.14.23.58.06 (version=SSLv3 cipher=OTHER); Fri, 14 Dec 2012 23:58:07 -0800 (PST) From: Bernhard =?ISO-8859-1?Q?Fr=F6hlich?= To: Konstantin Belousov , Baptiste Daroussin Subject: Re: Fix overlinking in base aka import pkgconf X-Mailer: Modest 3.90.7 References: <20121214235418.GF18884@ithaqua.etoilebsd.net> <20121215012233.GP71906@kib.kiev.ua> In-Reply-To: <20121215012233.GP71906@kib.kiev.ua> Content-Type: text/plain; charset=utf-8 Content-ID: <1355558294.7933.1.camel@Nokia-N900-42-11> Date: Sat, 15 Dec 2012 08:58:14 +0100 Message-Id: <1355558294.7933.2.camel@Nokia-N900-42-11> Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Gm-Message-State: ALoCoQkBBnh7ryP07O8uET7+su/ZB/wvF5CBfXbwH9soXIjiBFtWIyvqB8JppX6Gy7YFIFjumQQM X-Mailman-Approved-At: Sat, 15 Dec 2012 12:28:34 +0000 Cc: hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Bernhard =?ISO-8859-1?Q?Fr=F6hlich?= List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Dec 2012 07:58:15 -0000 On Sa., 15. Dez. 2012 02:22:34 CET, Konstantin Belousov wrote: > On Sat, Dec 15, 2012 at 12:54:19AM +0100, Baptiste Daroussin wrote: > > Hi, > > > > Some of our binary are overlinked, the way we handle the linking > > doesn't help for that. > What do you mean there ? Do you mean that some libraries specified for > the linking stage of the final binary are not needed for the execution ? > > > > > On proposition could be to use pkgconf > > https://github.com/pkgconf/pkgconf which is BSD license pkg-config > > implementation 100% compatible with pkg-config. > > > > What I propose is to create a new PCADD variable for the Makefiles. > > > > PCADD will invoke pkgconf to gather the libraries and the cflags for a > > given project. > > > > The second thing would be to create .pc files for all of our libraries. > > > > for example: > > usr.bin/fstat dynamic build is overlinked > And how this is better than just removing the unneeded library from > the Makefile ? > > For the port consumption, I believe that the better solution is to > provide a pack of the .pc files describing base libraries, most likely > as port. We should definitely generate some pc files for our base libraries. We already have quite a few ports that need to hack around because of missing pc files for ssl for example. -- http://www.bluelife.at/ From owner-freebsd-hackers@FreeBSD.ORG Sat Dec 15 13:22:49 2012 Return-Path: Delivered-To: hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 717E5F7A; Sat, 15 Dec 2012 13:22:49 +0000 (UTC) (envelope-from uqs@FreeBSD.org) Received: from acme.spoerlein.net (acme.spoerlein.net [IPv6:2a01:4f8:131:23c2::1]) by mx1.freebsd.org (Postfix) with ESMTP id E9F158FC0A; Sat, 15 Dec 2012 13:22:48 +0000 (UTC) Received: from localhost (acme.spoerlein.net [IPv6:2a01:4f8:131:23c2::1]) by acme.spoerlein.net (8.14.5/8.14.5) with ESMTP id qBFDMlR9077003 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Sat, 15 Dec 2012 14:22:47 +0100 (CET) (envelope-from uqs@FreeBSD.org) Date: Sat, 15 Dec 2012 14:22:46 +0100 From: Ulrich =?utf-8?B?U3DDtnJsZWlu?= To: developers@FreeBSD.org, current@FreeBSD.org, hackers@FreeBSD.org Subject: HEADS UP: FreeBSD git mirrors demoted to beta status, need your help Message-ID: <20121215132246.GH69724@acme.spoerlein.net> Mail-Followup-To: developers@FreeBSD.org, current@FreeBSD.org, hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="M/SuVGWktc5uNpra" Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: current@FreeBSD.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Dec 2012 13:22:49 -0000 --M/SuVGWktc5uNpra Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Bad news everyone, tl;dr The git mirror of the source repository needs to be re-rolled to make the conversion deterministically repeatable, this will change pretty much all git commit hashes. The re-roll will be done January 15, 2013. Not affected are the ports and doc repositories, nor is the svn_head (for use with git-svn) affected. Background The converter (svn2git) was handing commits and objects to git's fast-import in arbitrary order, this makes merge commits have an arbitrary order of their parent commits and thus these merge commits have changing commit hashes for each converter run. This has been fixed, but requires us to move all the branches over to this deterministic scheme, which will change all their commit hashes. None of the contents of these commits change, so rebasing/remerging your work into these branches is possible without running into any merge conflicts. We need your help A goal of these conversions is to have them repeatable by you (yes, you!), so the correctness of the conversion can be verified. There are also no backups of the conversion runs, as they should be repeatable anyway. We need 2-3 volunteers to run these conversions themselves and verify that the produced commit hashes match the published ones. The necessary steps to do this are documented on the Wiki under http://wiki.freebsd.org/GitWorkflow#History Please send me your output of git show-ref in a private mail, thanks. Cheers, Uli PS: This re-roll has nothing to do with the recent security incident. --M/SuVGWktc5uNpra Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iQEcBAEBAgAGBQJQzHmmAAoJEKOmmGRKr4LOvoIH/35PTmdC/afsPmOpcaLNYFuN UsS6fmQwYYrw/qNe1x9mCyjrEAigZz/QzMeyc1lxBtgC74LxehMPOc6FcVywHzah 3nU5uNyTl4LS0CLVVaL26N77AKjbsGiO4eX9h/gC4NnlpTnEpntkgg087FAtnDkc 76kMR6n9Yy5zBmmZhBncqGA/O/LEjkuNadBH2+3sd4vMtHebalyxtNCCoTGUjpYk IFrm/flC1GHptgtXqASgXrrh1ky1Beqr5hKWn7i4N8i9/5tclSwGlkZ6Z4hU5cCN Wy3o57yFSU4WDzeZmsj5nEyfXuT3ZOiEHoWpcp/77QB9UH+L5g8MRQrLOQj73qM= =Fvzq -----END PGP SIGNATURE----- --M/SuVGWktc5uNpra-- From owner-freebsd-hackers@FreeBSD.ORG Sat Dec 15 15:02:36 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0A70EDDB for ; Sat, 15 Dec 2012 15:02:36 +0000 (UTC) (envelope-from david.i.noel@gmail.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id BBB028FC0C for ; Sat, 15 Dec 2012 15:02:35 +0000 (UTC) Received: by mail-ob0-f182.google.com with SMTP id 16so4371982obc.13 for ; Sat, 15 Dec 2012 07:02:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=ZmM+5ier41EZFHmmiSXjrNTFXMafZPjuxMj5+/zRJ30=; b=ESFkn07hOyVHx6JWOmJ/ue2LuKPzrZBJHOggVM4cd1nS2+ebs8xpfqmFinWjYvY372 FrfyZ1eZC4hzT3VCsgLK+PYBjB9898FW8SucwyU6BND/qcsvFAZOmuv6/GYoOC92i/x4 DsMWd0ZtawVtkVqQzbVa9kCSLsRKcbGtj37zl7tUHTaJ8aEggCqVvkLt0sCPHZJEmpyU 9ToaBYMs8vbxjxEzHKKUoC6wcifEzlpi1+Nh1Z1D+re/otFJNNe+2G3r6RF4nhg/Vkk5 Hf2jU2OayhhYULBTdTpBMI2WqkArNDjC3tSqE+7BxYnQQxiVXLz9wKp4j1Z41j9vhQNT dJlg== MIME-Version: 1.0 Received: by 10.60.32.234 with SMTP id m10mr6639995oei.7.1355583755010; Sat, 15 Dec 2012 07:02:35 -0800 (PST) Received: by 10.76.172.98 with HTTP; Sat, 15 Dec 2012 07:02:34 -0800 (PST) In-Reply-To: <20121215095324.GU71906@kib.kiev.ua> References: <20121215095324.GU71906@kib.kiev.ua> Date: Sat, 15 Dec 2012 09:02:34 -0600 Message-ID: Subject: Re: postgres, initdb, FreeBSD bug? From: David Noel To: Konstantin Belousov Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: David.I.Noel@gmail.com List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Dec 2012 15:02:36 -0000 Ahh... the security.bsd. sysctl output had the answer: I had security.bsd.hardlink_check_gid and security.bsd.hardlink_check_uid set to 1 in sysctl.conf. Removing that fixed the problem. Many thanks, -David > Show the ktrace from the same error on UFS. 5179 postgres CALL unlink(0x7fffffffe570) 5179 postgres NAMI "pg_notify/0000" 5179 postgres RET unlink 0 5179 postgres CALL getdirentries(0x3,0x801a4b000,0x1000,0x801a4a068) 5179 postgres RET getdirentries 0 5179 postgres CALL lseek(0x3,0,SEEK_SET) 5179 postgres RET lseek 0 5179 postgres CALL close(0x3) 5179 postgres RET close 0 5179 postgres CALL open(0x7fffffffe580,O_RDWR|O_CREAT,S_IRUSR|S_IWUSR) 5179 postgres NAMI "pg_notify/0000" 5179 postgres RET open 3 5179 postgres CALL lseek(0x3,0,SEEK_SET) 5179 postgres RET lseek 0 5179 postgres CALL write(0x3,0x8041c1b40,0x2000) 5179 postgres GIO fd 3 wrote 4096 bytes ... 5179 postgres RET write 8192/0x2000 5179 postgres CALL close(0x3) 5179 postgres RET close 0 5179 postgres CALL unlink(0x7fffffffbc60) 5179 postgres NAMI "pg_xlog/000000010000000000000001" 5179 postgres RET unlink -1 errno 2 No such file or directory 5179 postgres CALL link(0x7fffffffc130,0x7fffffffbc60) 5179 postgres NAMI "pg_xlog/xlogtemp.5179" 5179 postgres NAMI "pg_xlog/000000010000000000000001" 5179 postgres RET link -1 errno 1 Operation not permitted 5179 postgres CALL unlink(0x7fffffffc130) 5179 postgres NAMI "pg_xlog/xlogtemp.5179" 5179 postgres RET unlink 0 5179 postgres CALL open(0x7fffffffc530,O_RDWR,0x180) 5179 postgres NAMI "pg_xlog/000000010000000000000001" 5179 postgres RET open -1 errno 2 No such file or directory > Show the security.bsd sysctl settings, in particular, > harlink_check_{u,g}id.ygg# sysctl security.bsd. security.bsd.map_at_zero: 0 security.bsd.suser_enabled: 1 security.bsd.unprivileged_proc_debug: 0 security.bsd.conservative_signals: 1 security.bsd.see_other_gids: 0 security.bsd.see_other_uids: 0 security.bsd.unprivileged_idprio: 0 security.bsd.unprivileged_read_msgbuf: 0 security.bsd.hardlink_check_gid: 1 security.bsd.hardlink_check_uid: 1 security.bsd.unprivileged_get_quota: 0 security.bsd.stack_guard_page: 0 > Show the ls -la output for the pg_xlog directory. ygg:~> ls -la /zdb/pgsql/data/pg_xlog/ total 5 drwx------ 3 pgsql wheel 3 Dec 15 08:39 . drwx------ 14 pgsql wheel 18 Dec 15 08:39 .. drwx------ 2 pgsql wheel 2 Dec 15 08:39 archive_status From owner-freebsd-hackers@FreeBSD.ORG Sat Dec 15 15:44:06 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1D8F66DC for ; Sat, 15 Dec 2012 15:44:06 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id 67F308FC14 for ; Sat, 15 Dec 2012 15:44:05 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.5/8.14.5) with ESMTP id qBFFhuuf063241; Sat, 15 Dec 2012 17:43:56 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.7.3 kib.kiev.ua qBFFhuuf063241 Received: (from kostik@localhost) by tom.home (8.14.5/8.14.5/Submit) id qBFFhukM063240; Sat, 15 Dec 2012 17:43:56 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 15 Dec 2012 17:43:56 +0200 From: Konstantin Belousov To: David Noel Subject: Re: postgres, initdb, FreeBSD bug? Message-ID: <20121215154356.GW71906@kib.kiev.ua> References: <20121215095324.GU71906@kib.kiev.ua> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="tuYRN1zEaS85jg/Y" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Dec 2012 15:44:06 -0000 --tuYRN1zEaS85jg/Y Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Dec 15, 2012 at 09:02:34AM -0600, David Noel wrote: > Ahh... the security.bsd. sysctl output had the answer: I had > security.bsd.hardlink_check_gid and security.bsd.hardlink_check_uid > set to 1 in sysctl.conf. Removing that fixed the problem. Then, why did you set it ? The sysctl indeed has an interesting interaction with the fact that the created inode inherits gid from the parent directory. If you use it, you must ensure that the directory group is in the group set of the process. >=20 > Many thanks, >=20 > -David >=20 >=20 >=20 > > Show the ktrace from the same error on UFS. >=20 > 5179 postgres CALL unlink(0x7fffffffe570) > 5179 postgres NAMI "pg_notify/0000" > 5179 postgres RET unlink 0 > 5179 postgres CALL getdirentries(0x3,0x801a4b000,0x1000,0x801a4a068) > 5179 postgres RET getdirentries 0 > 5179 postgres CALL lseek(0x3,0,SEEK_SET) > 5179 postgres RET lseek 0 > 5179 postgres CALL close(0x3) > 5179 postgres RET close 0 > 5179 postgres CALL open(0x7fffffffe580,O_RDWR|O_CREAT,S_IRUSR|S_IWUSR) > 5179 postgres NAMI "pg_notify/0000" > 5179 postgres RET open 3 > 5179 postgres CALL lseek(0x3,0,SEEK_SET) > 5179 postgres RET lseek 0 > 5179 postgres CALL write(0x3,0x8041c1b40,0x2000) > 5179 postgres GIO fd 3 wrote 4096 bytes > ... > 5179 postgres RET write 8192/0x2000 > 5179 postgres CALL close(0x3) > 5179 postgres RET close 0 > 5179 postgres CALL unlink(0x7fffffffbc60) > 5179 postgres NAMI "pg_xlog/000000010000000000000001" > 5179 postgres RET unlink -1 errno 2 No such file or directory > 5179 postgres CALL link(0x7fffffffc130,0x7fffffffbc60) > 5179 postgres NAMI "pg_xlog/xlogtemp.5179" > 5179 postgres NAMI "pg_xlog/000000010000000000000001" > 5179 postgres RET link -1 errno 1 Operation not permitted > 5179 postgres CALL unlink(0x7fffffffc130) > 5179 postgres NAMI "pg_xlog/xlogtemp.5179" > 5179 postgres RET unlink 0 > 5179 postgres CALL open(0x7fffffffc530,O_RDWR,0x180) > 5179 postgres NAMI "pg_xlog/000000010000000000000001" > 5179 postgres RET open -1 errno 2 No such file or directory >=20 >=20 > > Show the security.bsd sysctl settings, in particular, > > harlink_check_{u,g}id.ygg# sysctl security.bsd. >=20 > security.bsd.map_at_zero: 0 > security.bsd.suser_enabled: 1 > security.bsd.unprivileged_proc_debug: 0 > security.bsd.conservative_signals: 1 > security.bsd.see_other_gids: 0 > security.bsd.see_other_uids: 0 > security.bsd.unprivileged_idprio: 0 > security.bsd.unprivileged_read_msgbuf: 0 > security.bsd.hardlink_check_gid: 1 > security.bsd.hardlink_check_uid: 1 > security.bsd.unprivileged_get_quota: 0 > security.bsd.stack_guard_page: 0 >=20 > > Show the ls -la output for the pg_xlog directory. >=20 > ygg:~> ls -la /zdb/pgsql/data/pg_xlog/ > total 5 > drwx------ 3 pgsql wheel 3 Dec 15 08:39 . > drwx------ 14 pgsql wheel 18 Dec 15 08:39 .. > drwx------ 2 pgsql wheel 2 Dec 15 08:39 archive_status --tuYRN1zEaS85jg/Y Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJQzJq8AAoJEJDCuSvBvK1Bv/0P/1WsJbIiT2X2ZAzLoxVW3IDX 9KgB8UJ7hO7o1g+cfjcA2PXBvygc2Mf4/jDQdxlzh5E6W+C89H1ZrFc9bCaehkLV JDmNjgLN4EB+fLhOn8SkL00ZKSJwTEQvj271I6MwVcB4YAtr5hLrSLzAmDVMndEz 0MrdrC3AOzvJiUkaC7oZQxE8u2JEjCR0NY4V9kdnPJBlsuUYMBbNK4BqJvgKnZ3O kzgMqr747hAJc6Y6O0HJWIOMfTSoaB+zqjnl5JWtHj+FF+5OptHdnQX2FmyBFjvR cgcvW1Twsof8ynh+ZUPda+3dIu+0DxLK1KWK/IOGo6hHYzaqA9Nz6WVOaIVZ14Qv GXfFQ4/hS35+mVHlhT8RUoVsiH/ZSBpgaKT28EnxJ27zdLDuCodJJPG2iktFMe+6 iRQfjxfvoSOUxs9VRXa6BElJr5TGaBGJqb+BA4uduYzLqNmQNlttgLMfUVB6nB5b ry4s8/yaQVJLimHHdfPl4a/dfIirdPryPcC5DjdkvCe5tRm0rChBpn8psl5HGJ9t Q4vkmqlcUQY5D4tweJUvnu9xFXvxTFIBu8SyfTsCJ0SeMNxWBCQaomdPm8iTzSXR iZWBc2C7l3lQ4UC0Qfq2mq7ZwvKZ5G3Dic6DbvYvLVR5FJXcHLWM6xhQjJjqxJcI bkmf5uEY/P442uziAOXM =GAOR -----END PGP SIGNATURE----- --tuYRN1zEaS85jg/Y-- From owner-freebsd-hackers@FreeBSD.ORG Sat Dec 15 16:11:21 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5654ED71; Sat, 15 Dec 2012 16:11:21 +0000 (UTC) (envelope-from utisoft@gmail.com) Received: from mail-bk0-f54.google.com (mail-bk0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 8227E8FC19; Sat, 15 Dec 2012 16:11:19 +0000 (UTC) Received: by mail-bk0-f54.google.com with SMTP id je9so2107974bkc.13 for ; Sat, 15 Dec 2012 08:11:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:content-type :content-transfer-encoding; bh=4uSl/hUB7zoqq6Vv5ApRV0QfFU8/WS1FBps7OoIOQaA=; b=bxGGzifDJrh0ds/mNq+/6q2JpL9xaMcU06uiuKOOcXY8uu8JJ7J/zfdZDokw+tlNF1 fykk1/xECsqP1hs0erXSylkxPGWmqVNQwnTSvcJOihphtDZ6W3E8YwNA8fDLCJQ0fY65 WXZZGPERh6ZSHa0zvHIZM7m23CWi6gstGIDiU2IxD8PL1gagZ8xINQJ8364IP8s9zUvn ZNDQ2yEvFJNAWcHs/WbCLvC2FLMK0b/WXMRGnb4pFmqzSzDRfBqsA/EnWwM6R0NyHqbd o8WoZZ/JwO1l4Ke5W4d6WvXUVHi5Lev1kHAqK9f+bri0I91/fk8mK9VV9oMulQj/bWnY kvQQ== Received: by 10.204.149.143 with SMTP id t15mr4154624bkv.45.1355587876351; Sat, 15 Dec 2012 08:11:16 -0800 (PST) MIME-Version: 1.0 Sender: utisoft@gmail.com Received: by 10.204.167.71 with HTTP; Sat, 15 Dec 2012 08:10:46 -0800 (PST) In-Reply-To: <20121215132246.GH69724@acme.spoerlein.net> References: <20121215132246.GH69724@acme.spoerlein.net> From: Chris Rees Date: Sat, 15 Dec 2012 16:10:46 +0000 X-Google-Sender-Auth: BSv-TxjQAhpEz0EszjFWaPPrruw Message-ID: Subject: Re: HEADS UP: FreeBSD git mirrors demoted to beta status, need your help To: "current@freebsd.org" , FreeBSD developers , "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Dec 2012 16:11:21 -0000 On 15 December 2012 13:22, Ulrich Sp=F6rlein wrote: > Bad news everyone, > > tl;dr The git mirror of the source repository needs to be re-rolled to > make the conversion deterministically repeatable, this will change > pretty much all git commit hashes. > > The re-roll will be done January 15, 2013. > > Not affected are the ports and doc repositories, nor is the svn_head > (for use with git-svn) affected. > > > Background > > The converter (svn2git) was handing commits and objects to git's > fast-import in arbitrary order, this makes merge commits have an > arbitrary order of their parent commits and thus these merge commits > have changing commit hashes for each converter run. > > This has been fixed, but requires us to move all the branches over to > this deterministic scheme, which will change all their commit hashes. > None of the contents of these commits change, so rebasing/remerging your > work into these branches is possible without running into any merge > conflicts. > > > We need your help > > A goal of these conversions is to have them repeatable by you (yes, > you!), so the correctness of the conversion can be verified. There are > also no backups of the conversion runs, as they should be repeatable > anyway. > > We need 2-3 volunteers to run these conversions themselves and verify > that the produced commit hashes match the published ones. The necessary > steps to do this are documented on the Wiki under > > http://wiki.freebsd.org/GitWorkflow#History > > Please send me your output of git show-ref in a private mail, thanks. Hey, http://www.bayofrum.net/~crees/scratch/git-show-ref.txt I hope it's what you were hoping for :) My local svn mirror is synchronised at midnight GMT (UTC). Need anything else? Chris From owner-freebsd-hackers@FreeBSD.ORG Sat Dec 15 17:54:54 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BD463EC9 for ; Sat, 15 Dec 2012 17:54:54 +0000 (UTC) (envelope-from dieterbsd@engineer.com) Received: from mailout-us.gmx.com (mailout-us.gmx.com [74.208.5.67]) by mx1.freebsd.org (Postfix) with SMTP id 79A4D8FC13 for ; Sat, 15 Dec 2012 17:54:54 +0000 (UTC) Received: (qmail 23801 invoked by uid 0); 15 Dec 2012 17:54:47 -0000 Received: from 67.206.184.176 by rms-us004 with HTTP Content-Type: text/plain; charset="utf-8" Date: Sat, 15 Dec 2012 12:54:46 -0500 From: "Dieter BSD" Message-ID: <20121215175447.310780@gmx.com> MIME-Version: 1.0 Subject: Re: NetBSD's boot select MBR To: freebsd-hackers@freebsd.org X-Authenticated: #74169980 X-Flags: 0001 X-Mailer: GMX.com Web Mailer x-registered: 0 Content-Transfer-Encoding: 8bit X-GMX-UID: MNy+cAIY3zOlNR3dAHAhSnJ+IGRvbwAI X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Dec 2012 17:54:54 -0000 Domagoj writes: > MBR supports max of 4 slices/partitions. 4 primary partitions, there are the "extended/logical partitions", which allow more. > The '/usr/mdec/mbr_bootsel', which you've mentioned, is equivalent > to FreeBSD's '/boot/boot0', which is tuned via 'boot0cfg' util. > It will also show menu, to choose from, a slice to boot or another > drive  OR => 'F6' to choose PXE boot. > If nothing has been selected, it will simply boot an active slice. > So I see FreeBSD's, as superior (additionally supports PXE) => 8.0 onward ... If I understand this correctly FreeBSD's boot0 allows you to set what to boot from the next time and go away while the system reboots. And this method allow booting from not only any slice but any disk, or PXE? But, it does this by writing to the MBR, yes? Some sysadmins want the MBR to be read-only for various reasons. OSes other than FreeBSD aren't going to have boot0cfg. If you're running NetBSD and want to boot Plan 9, do you have to reboot to FreeBSD, run boot0cfg, then reboot to Plan 9? As far as I know, NetBSD's MBR works with any combination of OSes. If the next OS to be booted is on the 1st disk and you don't mind writing to the MBR, you can do the set-and-walk-away thing by setting the active partition. Or you can use something like grub as the next stage and edit grub's default using any text editor, including sed in a script. If you want to boot a slice that is not on the first disk you have to either sit there and type at the menu, or use something like grub. As far as I know, NetBSD's MBR has no support for PXE, but then PXE is intended for diskless machines. > I thought NetBSD's had a MBR code, which could be told via a tool, > from fully booted and running FreeBSD: > "Active slice is 2, but please, boot slice 4 at a next boot". > After that, I would '# reboot' and without any menu, slice 4 > would be booted, for that one time, instaed of an active slice 2. I don't know of a way to do that with just NetBSD's MBR. You could either change the active flag which would remain that way until you changed it again, or sit there and watch for the menu. Given the limited space in the MBR it would be difficult or impossible to add that without giving up something. That feature could be added to a later stage bootstrap. Next time I build a boot disk I'm going to take another shot at multibooting with GPT. I really want to get away from the MBR kludgefest.