From owner-freebsd-hackers@FreeBSD.ORG Sun Sep 26 12:15:03 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 71F7F106566C for ; Sun, 26 Sep 2010 12:15:03 +0000 (UTC) (envelope-from faust64@gmail.com) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id 256F88FC19 for ; Sun, 26 Sep 2010 12:15:02 +0000 (UTC) Received: by qyk7 with SMTP id 7so3330420qyk.13 for ; Sun, 26 Sep 2010 05:15:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:mime-version:received:from:date :message-id:subject:to:content-type; bh=qg0OGc6BdaNp8xa5K7ZNnMhEZ0iLqCg8op17eUeka44=; b=PP3I423RP9971UrtmMmY4gwPY2qIdXp4BVuxZeiOfsc5yD7qed4tAwqj5g1W2tfHwT /X1YwqOz1xutkhxnjuaffjY7pEOjHR7Ps7kKfZhvbJ0CdaUnjVZaN/qpgVY53cjUYaDO 3h0bMtflW+7yYQhG5J0hz2RXLxAJmalLkxkeI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:from:date:message-id:subject:to:content-type; b=ZlQfEL150IERcdIWpO9rJKgpLnfiM+gqqlB2sCqjLILY4XUxf8uaVrLi9ZdyD6S214 nHU9J3Nll5IlM/gU5tUfMEYv1PopCU68Mk8P5gDz7IWjWAVm9UM+H8tqdtiQvEXcY3FN KuACp7c+zjItx2erfWfo/se6mbpkZFHvVk+CY= Received: by 10.224.72.2 with SMTP id k2mr4277426qaj.242.1285501911092; Sun, 26 Sep 2010 04:51:51 -0700 (PDT) MIME-Version: 1.0 Received: by 10.229.187.212 with HTTP; Sun, 26 Sep 2010 04:51:20 -0700 (PDT) From: =?ISO-8859-1?Q?Samuel_Mart=EDn_Moro?= Date: Sun, 26 Sep 2010 13:51:20 +0200 Message-ID: To: freebsd-questions@freebsd.org, freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: pf X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Sep 2010 12:15:03 -0000 Hello, I'm trying to set up pf on my soon-to-be new gateway (8.1-RELEASE amd64). I used the sample configuration file available on calomel After a few tests, it appears that the gate has fully access to the internet, but I can't open connections from clients to distant servers (web= , ssh, ...). Checking pflog log file, I can't see anything about those timeouts, even if I added the log directive in every block/pass command. Everything else seems to work, I can talk with my DNS from the internet, ss= h redirections to another pc also seems to works. I just can't access the Internet from a client of my network... For debugging, I commented out the options and the 'block all in/out' directives. Here's my config file http://pastebin.com/Nim2zBCx Is there someone understanding what I'm doing wrong? Thanks for your help! Regards, Samuel Mart=EDn Moro {EPITECH.} tek5 From owner-freebsd-hackers@FreeBSD.ORG Sun Sep 26 15:44:05 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BE3FF106564A; Sun, 26 Sep 2010 15:44:05 +0000 (UTC) (envelope-from faust64@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id 569EC8FC12; Sun, 26 Sep 2010 15:44:05 +0000 (UTC) Received: by qwd6 with SMTP id 6so2701670qwd.13 for ; Sun, 26 Sep 2010 08:44:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:mime-version:received:in-reply-to :references:from:date:message-id:subject:to:cc:content-type; bh=s0WqTgUJHdjYsVPk9Ijon1lgE1aMHpsAoWM65YBoULA=; b=t/3tv+qyjd+SAe4RYNjH2I+B+2JAhjtql1PpBDtIGoYg5a4h4utfhyyixYWcNt9+L7 OKyFFmye5YWID9WHJxkFT5dj0P3EE+oJzy1alajhU/gu1LvgZD0KkRw+SmDri9oRXsaa MeoWdrdsBdd9AGeva+AE7eNXR6Rj8CXHmwOwo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; b=dKh1toHX0/QuR3urcuPbK6RKLtPgLoTfgLV6EhF0DGUBWiUZugqiJtF5f3956F+11I zGh3CNCYlZ3sL2ivGPrr/XRVAE/JtFxPeOUzT+Orw7rpNzvd5TRRjaJL1E/jOCsXYtWK FheM4v4+DKS+MR7BLVWk4XoB9EI/yZV7hI2dU= Received: by 10.229.11.27 with SMTP id r27mr4575197qcr.294.1285515844383; Sun, 26 Sep 2010 08:44:04 -0700 (PDT) MIME-Version: 1.0 Received: by 10.229.187.212 with HTTP; Sun, 26 Sep 2010 08:43:33 -0700 (PDT) In-Reply-To: References: From: =?ISO-8859-1?Q?Samuel_Mart=EDn_Moro?= Date: Sun, 26 Sep 2010 17:43:33 +0200 Message-ID: To: Michael Powell Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org, freebsd-questions@freebsd.org Subject: Re: pf X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Sep 2010 15:44:05 -0000 On Sun, Sep 26, 2010 at 3:34 PM, Michael Powell wro= te: > Samuel Mart=EDn Moro wrote: > > > Hello, > > > > > > I'm trying to set up pf on my soon-to-be new gateway (8.1-RELEASE amd64= ). > > I used the sample configuration file available on > > calomel > > After a few tests, it appears that the gate has fully access to the > > internet, but I can't open connections from clients to distant servers > > (web, ssh, ...). > > Checking pflog log file, I can't see anything about those timeouts, eve= n > > if I added the log directive in every block/pass command. > > Everything else seems to work, I can talk with my DNS from the internet= , > > ssh redirections to another pc also seems to works. > > I just can't access the Internet from a client of my network... > > > > For debugging, I commented out the options and the 'block all in/out' > > directives. > > > > Here's my config file http://pastebin.com/Nim2zBCx > > > > Is there someone understanding what I'm doing wrong? > > > The firewall ruleset is a trifle overly complex for a quick glance; study > and analysis would take some doing. However, if you can reach the interne= t > from the firewall box and other client computers behind your NAT can't > (which is what it sounds like you're describing) it may be just that you > are > missing gateway_enable=3D"YES" in your /etc/rc.conf. > > Turning this "ON" makes your firewall box into a router. The status of th= is > can be checked with: sysctl net.inet.ip.forwarding - a "0" means no > gateway > and a "1" means gateway. > > -Mike > > > > _______________________________________________ > freebsd-questions@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to " > freebsd-questions-unsubscribe@freebsd.org" > the gateway is already enabled (and forwarding is correctly set) whatever, I had to do quick, I started again I think the missing thing on my old conf was the 'scrub' (at least) I made a more simple configuration, as following: ext_if=3D"bge0" int_if=3D"bge1" localnet =3D $int_if:network emma=3D"10.242.42.200" alpha=3D"10.42.42.42" delta=3D"10.42.42.44" set skip on lo0 scrub in on $ext_if all fragment reassemble #INTERNETZ nat on $ext_if from $localnet to any -> ($ext_if) #EMMA rdr on $ext_if inet proto tcp from any to ($ext_if) port 1101 -> $emma port 22 rdr on $ext_if inet proto tcp from any to ($ext_if) port 307 -> $emma port 80 #WHAT.CD rdr on $ext_if inet proto tcp from any to ($ext_if) port 1666 -> $alpha port 1666 #REMOTE ADM rdr on $ext_if inet proto tcp from any to ($ext_if) port 1667 -> $delta port 22 rdr on $ext_if inet proto tcp from any to ($ext_if) port 1668 -> $alpha port 22 pass in log on $ext_if inet proto tcp from any to $ext_if port 22 pass in log on $ext_if inet proto tcp from any to $ext_if port 53 pass in log on $ext_if inet proto udp from any to $ext_if port 53 pass in log on $ext_if inet proto tcp from any to $ext_if port 1664 pass in log on $int_if inet proto tcp from any to any pass in log on $int_if inet proto udp from any to any block in log on $ext_if inet proto icmp from any to $ext_if it's basically working i'll stuff it when I'll have time. Samuel Mart=EDn Moro {EPITECH.} tek5 From owner-freebsd-hackers@FreeBSD.ORG Sun Sep 26 20:45:12 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 460B7106566B; Sun, 26 Sep 2010 20:45:12 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id 2FD318FC1B; Sun, 26 Sep 2010 20:45:10 +0000 (UTC) Received: by qyk7 with SMTP id 7so3797432qyk.13 for ; Sun, 26 Sep 2010 13:45:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:cc:subject:references:in-reply-to :x-enigmail-version:content-type:content-transfer-encoding; bh=wrVzsPCQw7gmKIxRlNgxMcJKg7suaJALl7/7Vi6H0R8=; b=ppQzUvtEBbKcso8zBnSonGfHF3O6zVKtLd0xjT227+UlxrUxHZIaOksMZdNJfp2ymK jGIGbF5pvFbAtPTamwkWKYruo9xYXpGJtLqjIZkNqLynXzTTp4lUUk1E+4MGNwKOJifG jeE57Ho+ril6TcNogXhCFd3VP252WZD7Ln2Do= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; b=txjnTtDdzQKkoYGkkR8/CvloViAi7QM+LYdZjBV43D6fCA9Mp7QOVo0GuLoNe/kWdt lA/C22fCkiTQymiDnSttngFs45jbsVWJcuRpnTXyaEn9Y+7lukL6BlS6kHBqDRRwOnKR BKOZlqVADrQtEHz7VQ8Zcfs9pnX5hn9lwMO1o= Received: by 10.220.60.10 with SMTP id n10mr2049215vch.45.1285533909528; Sun, 26 Sep 2010 13:45:09 -0700 (PDT) Received: from centel.dataix.local (adsl-99-19-43-205.dsl.klmzmi.sbcglobal.net [99.19.43.205]) by mx.google.com with ESMTPS id t13sm1017229vcj.20.2010.09.26.13.45.07 (version=SSLv3 cipher=RC4-MD5); Sun, 26 Sep 2010 13:45:08 -0700 (PDT) Sender: "J. Hellenthal" Message-ID: <4C9FB0D2.1010205@DataIX.net> Date: Sun, 26 Sep 2010 16:45:06 -0400 From: jhell User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.2.9) Gecko/20100917 Lightning/1.0b1 Thunderbird MIME-Version: 1.0 To: =?ISO-8859-1?Q?Samuel_Mart=EDn_Moro?= References: In-Reply-To: X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Cc: freebsd-hackers@freebsd.org, Michael Powell , freebsd-questions@freebsd.org Subject: Re: pf X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Sep 2010 20:45:12 -0000 This is more for questions@ or pf@ On 09/26/2010 11:43, Samuel Martín Moro wrote: > On Sun, Sep 26, 2010 at 3:34 PM, Michael Powell wrote: > >> Samuel Martín Moro wrote: >> >>> Hello, >>> >>> >>> I'm trying to set up pf on my soon-to-be new gateway (8.1-RELEASE amd64). >>> I used the sample configuration file available on >>> calomel >>> After a few tests, it appears that the gate has fully access to the >>> internet, but I can't open connections from clients to distant servers >>> (web, ssh, ...). >>> Checking pflog log file, I can't see anything about those timeouts, even >>> if I added the log directive in every block/pass command. >>> Everything else seems to work, I can talk with my DNS from the internet, >>> ssh redirections to another pc also seems to works. >>> I just can't access the Internet from a client of my network... >>> >>> For debugging, I commented out the options and the 'block all in/out' >>> directives. >>> >>> Here's my config file http://pastebin.com/Nim2zBCx >>> >>> Is there someone understanding what I'm doing wrong? >>> >> The firewall ruleset is a trifle overly complex for a quick glance; study >> and analysis would take some doing. However, if you can reach the internet >> from the firewall box and other client computers behind your NAT can't >> (which is what it sounds like you're describing) it may be just that you >> are >> missing gateway_enable="YES" in your /etc/rc.conf. >> >> Turning this "ON" makes your firewall box into a router. The status of this >> can be checked with: sysctl net.inet.ip.forwarding - a "0" means no >> gateway >> and a "1" means gateway. >> >> -Mike >> >> >> >> _______________________________________________ >> freebsd-questions@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-questions >> To unsubscribe, send any mail to " >> freebsd-questions-unsubscribe@freebsd.org" >> > > the gateway is already enabled (and forwarding is correctly set) > whatever, I had to do quick, I started again > I think the missing thing on my old conf was the 'scrub' (at least) > I made a more simple configuration, as following: > > ext_if="bge0" > int_if="bge1" > localnet = $int_if:network > emma="10.242.42.200" > alpha="10.42.42.42" > delta="10.42.42.44" > set skip on lo0 > scrub in on $ext_if all fragment reassemble > #INTERNETZ > nat on $ext_if from $localnet to any -> ($ext_if) > #EMMA > rdr on $ext_if inet proto tcp from any to ($ext_if) port 1101 -> > $emma port 22 > rdr on $ext_if inet proto tcp from any to ($ext_if) port 307 -> > $emma port 80 > #WHAT.CD > rdr on $ext_if inet proto tcp from any to ($ext_if) port 1666 -> > $alpha port 1666 > #REMOTE ADM > rdr on $ext_if inet proto tcp from any to ($ext_if) port 1667 -> > $delta port 22 > rdr on $ext_if inet proto tcp from any to ($ext_if) port 1668 -> > $alpha port 22 > pass in log on $ext_if inet proto tcp from any to $ext_if port 22 > pass in log on $ext_if inet proto tcp from any to $ext_if port 53 > pass in log on $ext_if inet proto udp from any to $ext_if port 53 > pass in log on $ext_if inet proto tcp from any to $ext_if port 1664 > pass in log on $int_if inet proto tcp from any to any > pass in log on $int_if inet proto udp from any to any > block in log on $ext_if inet proto icmp from any to $ext_if > > it's basically working > i'll stuff it when I'll have time. > > Samuel Martín Moro > {EPITECH.} tek5 -- jhell,v From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 27 01:29:36 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id EC6061065673; Mon, 27 Sep 2010 01:29:36 +0000 (UTC) Date: Mon, 27 Sep 2010 01:29:36 +0000 From: Alexander Best To: freebsd-hackers@freebsd.org Message-ID: <20100927012936.GA32352@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Subject: adding a new lib for more advanced argument parsing X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Sep 2010 01:29:37 -0000 hi there, looking at applications such as geom (g_*), camcontrol, etc. makes one realise that getopt(3) is clearly not suitable for handling such complex options. camcontrol.c even contains a whole paragraph about why getopt(3) is considered not appropriate to handle camcontrol's argument parsing requirements (that was 1998!). why not do a vendor import of popt 1.16 e.g.? are there license restrictions? or maybe some other lib... cheers. alex -- a13x From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 27 02:47:34 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 48FAC106564A for ; Mon, 27 Sep 2010 02:47:34 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 154388FC15 for ; Mon, 27 Sep 2010 02:47:33 +0000 (UTC) Received: by iwn34 with SMTP id 34so5531552iwn.13 for ; Sun, 26 Sep 2010 19:47:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=k47PzRxF/sugnGUaWj3luU9WuVq/ou2cjekSTJrj3i0=; b=YOCHzupk7PDp7mx+/B6GTZOihHSx89XXa6v60yy18e8OBJxsMQffF2cmGeBld093qj g4UbO3Gc7rVk4TtLFLKz7uFPzl5VUtgHXizeHjOmhj7QsqwPj2V1OLysDPyWFhyVAfa6 7/Xw2woBgEbzTl2sqUPG5HS/OpdqfXLIe+0F8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=PgJriRJMgGY0tvknLTGvPVmsOtB1SD33HZuNa3U5bIjjSpvGZ19/FaaBKM7riP2UC8 LXONZlih5usTwHPx9I+/f46J++C6nLFNQNiNOfL1ww4WDrCewuVIG4Yw/uAt3X5RFGxo QFc2xkxgh+/EHZP9ckMblgBOnR4XuWuozzTLU= MIME-Version: 1.0 Received: by 10.231.157.11 with SMTP id z11mr8252237ibw.147.1285555653238; Sun, 26 Sep 2010 19:47:33 -0700 (PDT) Received: by 10.231.184.223 with HTTP; Sun, 26 Sep 2010 19:47:33 -0700 (PDT) Date: Sun, 26 Sep 2010 22:47:33 -0400 Message-ID: From: Aryeh Friedman To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: csup or svn X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Sep 2010 02:47:34 -0000 I currently use: csup -h cvsup9.us.freebsd.org /usr/share/examples/cvsup/cvs-supfile and when I just ran it I got: Append to CVSROOT-src/access Edit CVSROOT-src/access,v Segmentation fault (core dumped) (8.1-STABLE #0) Should I be using SVN instead and if so what is the repo URL I need to use? From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 27 03:34:38 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B735D106566C for ; Mon, 27 Sep 2010 03:34:38 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-qy0-f175.google.com (mail-qy0-f175.google.com [209.85.216.175]) by mx1.freebsd.org (Postfix) with ESMTP id 6AB628FC0A for ; Mon, 27 Sep 2010 03:34:38 +0000 (UTC) Received: by qyk30 with SMTP id 30so160707qyk.13 for ; Sun, 26 Sep 2010 20:34:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:cc:subject:references:in-reply-to :x-enigmail-version:content-type:content-transfer-encoding; bh=mqjreochZEoZyBE+lhyODgdWTM1eeAOQNiP+v50hyxc=; b=slsQO7wbTDhOgl8sgr5ykFQCFANsPSx8yDSaYQcM7+8laRrLN968NSm/uFBm9l88j6 +5ldv7sA1KYSIrbzByikYaAIXUVl2wCRqd6lUYM/tF5Waw1GnF4icrFQB05/xySfk9rC 7V5IeiDSRxJTEjoVY2lmGGnfmNdHug9VhQA/k= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; b=VFEVc4UNFtb//EcdFG9UFJzpYa/R/3e1h3pcradv1WToUUT0e+HzHTPIM6oW2I+PPA vL+EYz46egNv7zkQZy6+AqpgGC3VzfD3NF0zV0IJMLN3CbWvhjCnrAnTfZHFALLPbsuk 6k5P2eMyVZi8st5ucRMAs3O88ifXRps/qMt90= Received: by 10.224.53.203 with SMTP id n11mr4917643qag.321.1285558477587; Sun, 26 Sep 2010 20:34:37 -0700 (PDT) Received: from centel.dataix.local (adsl-99-19-43-205.dsl.klmzmi.sbcglobal.net [99.19.43.205]) by mx.google.com with ESMTPS id f15sm5967830qcr.37.2010.09.26.20.34.35 (version=SSLv3 cipher=RC4-MD5); Sun, 26 Sep 2010 20:34:36 -0700 (PDT) Sender: "J. Hellenthal" Message-ID: <4CA010CA.6000709@DataIX.net> Date: Sun, 26 Sep 2010 23:34:34 -0400 From: jhell User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.2.9) Gecko/20100917 Lightning/1.0b1 Thunderbird MIME-Version: 1.0 To: Aryeh Friedman References: In-Reply-To: X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org Subject: Re: csup or svn X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Sep 2010 03:34:38 -0000 On 09/26/2010 22:47, Aryeh Friedman wrote: > I currently use: > > csup -h cvsup9.us.freebsd.org /usr/share/examples/cvsup/cvs-supfile > > and when I just ran it I got: > > Append to CVSROOT-src/access > Edit CVSROOT-src/access,v > Segmentation fault (core dumped) > > > (8.1-STABLE #0) > > Should I be using SVN instead and if so what is the repo URL I need to use? You can safely remove that file and continue to do what you do... or Yes you can use SVN ports/devel/subversion-freebsd Regards, -- jhell,v From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 27 04:13:15 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 91DA2106564A for ; Mon, 27 Sep 2010 04:13:15 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 579268FC17 for ; Mon, 27 Sep 2010 04:13:14 +0000 (UTC) Received: by iwn34 with SMTP id 34so5599330iwn.13 for ; Sun, 26 Sep 2010 21:13:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=o9b+u5T9Fu9fedyLpyGpDa5j/wa9ruz7ic1y8YmR7nI=; b=rJCsaaDLK0U8C/c9m6ROBjPcGEgpHdsq/E3Eh9C6+tZrXKV/h+cEeZzcNptApbzx6o V0oD1TNSnhtsr3ZxQr4PFgmR07Ndoqxytd3jaHKuSebU6AGEotPBmTVm1aZdnwRw8UzG I4eM64JK+H5/Blpw8o6ngqmLLTz0hImIJGzlc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=chh+2NuH4YOlRk1W2x5Z2hI7/yFUATZKbBS1ox6FkBWI+Cw3jH62zjVAcKB8fXK6Va P7cAopxtY00C9HibFZ6tkzIeGsLWOhcipHOMwraVsUG3QUWmHpV/NIAv/DsjxhE2KPtw 2OLzNamxL0BtC9EN35UZ6mf3vniqLKnV91EZg= MIME-Version: 1.0 Received: by 10.231.157.135 with SMTP id b7mr8141912ibx.164.1285560794404; Sun, 26 Sep 2010 21:13:14 -0700 (PDT) Received: by 10.231.184.223 with HTTP; Sun, 26 Sep 2010 21:13:14 -0700 (PDT) In-Reply-To: <20100927035533.GA48062@misty.eyesbeyond.com> References: <4CA010CA.6000709@DataIX.net> <20100927035533.GA48062@misty.eyesbeyond.com> Date: Mon, 27 Sep 2010 00:13:14 -0400 Message-ID: From: Aryeh Friedman To: Greg Lewis Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: csup or svn X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Sep 2010 04:13:15 -0000 Isn't that a step backwards? On Sun, Sep 26, 2010 at 11:55 PM, Greg Lewis wrote: > On Sun, Sep 26, 2010 at 11:34:34PM -0400, jhell wrote: >> On 09/26/2010 22:47, Aryeh Friedman wrote: >> > I currently use: >> > >> > csup -h cvsup9.us.freebsd.org /usr/share/examples/cvsup/cvs-supfile >> > >> > and when I just ran it I got: >> > >> > =A0Append to CVSROOT-src/access >> > =A0Edit CVSROOT-src/access,v >> > Segmentation fault (core dumped) >> > >> > >> > (8.1-STABLE #0) >> > >> > Should I be using SVN instead and if so what is the repo URL I need to= use? >> >> You can safely remove that file and continue to do what you do... or >> >> Yes you can use SVN ports/devel/subversion-freebsd > > Or you can use cvsup (it doesn't segfault on that file). > > -- > Greg Lewis =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Email =A0 := glewis@eyesbeyond.com > Eyes Beyond =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 Web =A0 =A0 := http://www.eyesbeyond.com > Information Technology =A0 =A0 =A0 =A0 =A0 =A0 =A0FreeBSD : glewis@FreeBS= D.org > From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 27 04:23:14 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F246B106564A for ; Mon, 27 Sep 2010 04:23:14 +0000 (UTC) (envelope-from glewis@eyesbeyond.com) Received: from misty.eyesbeyond.com (gerbercreations.com [71.39.140.16]) by mx1.freebsd.org (Postfix) with ESMTP id 4EEA58FC0A for ; Mon, 27 Sep 2010 04:23:13 +0000 (UTC) Received: from misty.eyesbeyond.com (localhost.eyesbeyond.com [127.0.0.1]) by misty.eyesbeyond.com (8.14.4/8.14.4) with ESMTP id o8R3tYQk058286; Sun, 26 Sep 2010 20:55:34 -0700 (PDT) (envelope-from glewis@eyesbeyond.com) Received: (from glewis@localhost) by misty.eyesbeyond.com (8.14.4/8.14.4/Submit) id o8R3tYd0058260; Sun, 26 Sep 2010 20:55:34 -0700 (PDT) (envelope-from glewis@eyesbeyond.com) X-Authentication-Warning: misty.eyesbeyond.com: glewis set sender to glewis@eyesbeyond.com using -f Date: Sun, 26 Sep 2010 20:55:33 -0700 From: Greg Lewis To: jhell Message-ID: <20100927035533.GA48062@misty.eyesbeyond.com> References: <4CA010CA.6000709@DataIX.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4CA010CA.6000709@DataIX.net> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-hackers@freebsd.org, Aryeh Friedman Subject: Re: csup or svn X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Sep 2010 04:23:15 -0000 On Sun, Sep 26, 2010 at 11:34:34PM -0400, jhell wrote: > On 09/26/2010 22:47, Aryeh Friedman wrote: > > I currently use: > > > > csup -h cvsup9.us.freebsd.org /usr/share/examples/cvsup/cvs-supfile > > > > and when I just ran it I got: > > > > Append to CVSROOT-src/access > > Edit CVSROOT-src/access,v > > Segmentation fault (core dumped) > > > > > > (8.1-STABLE #0) > > > > Should I be using SVN instead and if so what is the repo URL I need to use? > > You can safely remove that file and continue to do what you do... or > > Yes you can use SVN ports/devel/subversion-freebsd Or you can use cvsup (it doesn't segfault on that file). -- Greg Lewis Email : glewis@eyesbeyond.com Eyes Beyond Web : http://www.eyesbeyond.com Information Technology FreeBSD : glewis@FreeBSD.org From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 27 04:33:20 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D6B33106564A; Mon, 27 Sep 2010 04:33:20 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 9B2E38FC18; Mon, 27 Sep 2010 04:33:20 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id o8R4SrRw067878; Sun, 26 Sep 2010 22:28:53 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Sun, 26 Sep 2010 22:29:03 -0600 (MDT) Message-Id: <20100926.222903.886429907165706116.imp@bsdimp.com> To: arundel@freebsd.org From: "M. Warner Losh" In-Reply-To: <20100927012936.GA32352@freebsd.org> References: <20100927012936.GA32352@freebsd.org> X-Mailer: Mew version 6.3 on Emacs 22.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org Subject: Re: adding a new lib for more advanced argument parsing X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Sep 2010 04:33:20 -0000 In message: <20100927012936.GA32352@freebsd.org> Alexander Best writes: : hi there, : : looking at applications such as geom (g_*), camcontrol, etc. makes one realise : that getopt(3) is clearly not suitable for handling such complex options. : camcontrol.c even contains a whole paragraph about why getopt(3) is considered : not appropriate to handle camcontrol's argument parsing requirements (that was : 1998!). : : why not do a vendor import of popt 1.16 e.g.? are there license restrictions? : or maybe some other lib... popt has an X11 license, which isn't a big deal. However, it depends on gettext, which is pure GPL. Also, POSIX has a lot to say about command line parsing, and popt doesn't quite match what POSIX has to say... Warner From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 27 04:37:03 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8167410656C0 for ; Mon, 27 Sep 2010 04:37:03 +0000 (UTC) (envelope-from eknath.iyer@gmail.com) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id 3A0FF8FC13 for ; Mon, 27 Sep 2010 04:37:02 +0000 (UTC) Received: by qyk7 with SMTP id 7so4304646qyk.13 for ; Sun, 26 Sep 2010 21:37:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=ULKfCfddFflpC+fZloqoUZZeTyKj/fJf1Gpqx/PyW8A=; b=fY5+edLw5BEiSW4Fw0Yyzko6HJqnscaI6YtTHBxfBDjQRYEYXq45IBepMoWIlrcmV6 YylFvb1IEjqIjsvhl+Wi6FEAKKwrRlgt5difOQ+HC7IxUYV6S/39FlScqcqVbrtDmnJD P2q3/F7D7flAfZw1x0gWSWDXXWVrRHgFoiqVU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=S7U/gX/PdllQbU/D55ffKtFSeEmSQT6QI766PUdWxLrn7IVGhM8DxjHNSECnSf0F4K kpBHJSWwOQ3b3gZTmMpqmgXYjLJccx516IEz/1vYmYo3rZUbXoEFGBRLsK9j7hobJ5L1 hI/VdFVFYpSHN8VAnfvG79clYZCnVK+lWHCbM= MIME-Version: 1.0 Received: by 10.224.28.134 with SMTP id m6mr5045060qac.150.1285560482168; Sun, 26 Sep 2010 21:08:02 -0700 (PDT) Received: by 10.229.85.195 with HTTP; Sun, 26 Sep 2010 21:08:02 -0700 (PDT) Date: Mon, 27 Sep 2010 00:08:02 -0400 Message-ID: From: Eknath Venkataramani To: freebsd-hackers@freebsd.org X-Mailman-Approved-At: Mon, 27 Sep 2010 05:08:15 +0000 Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: scheduler modifications X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Sep 2010 04:37:03 -0000 I want to modify the priorities are assigned for the timesharing class of processes. How do I do that? From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 27 05:14:36 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A603C106566C; Mon, 27 Sep 2010 05:14:36 +0000 (UTC) (envelope-from perryh@pluto.rain.com) Received: from agora.rdrop.com (agora.rdrop.com [IPv6:2607:f678:1010::34]) by mx1.freebsd.org (Postfix) with ESMTP id 8638D8FC16; Mon, 27 Sep 2010 05:14:36 +0000 (UTC) Received: from agora.rdrop.com (66@localhost [127.0.0.1]) by agora.rdrop.com (8.13.1/8.12.7) with ESMTP id o8R5EZZm047930 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Sun, 26 Sep 2010 22:14:35 -0700 (PDT) (envelope-from perryh@pluto.rain.com) Received: (from uucp@localhost) by agora.rdrop.com (8.13.1/8.12.9/Submit) with UUCP id o8R5EZZD047929; Sun, 26 Sep 2010 22:14:35 -0700 (PDT) Received: from fbsd61 by pluto.rain.com (4.1/SMI-4.1-pluto-M2060407) id AA20362; Sun, 26 Sep 10 22:08:46 PDT Date: Sun, 26 Sep 2010 22:08:46 -0700 From: perryh@pluto.rain.com To: arundel@freebsd.org Message-Id: <4ca026de.pOkFIzh1Wymrvuow%perryh@pluto.rain.com> References: <20100927012936.GA32352@freebsd.org> In-Reply-To: <20100927012936.GA32352@freebsd.org> User-Agent: nail 11.25 7/29/05 Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org Subject: Re: adding a new lib for more advanced argument parsing X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Sep 2010 05:14:36 -0000 Alexander Best wrote: > ... getopt(3) is clearly not suitable for handling such complex > options. camcontrol.c even contains a whole paragraph about why > getopt(3) is considered not appropriate to handle camcontrol's > argument parsing requirements ... > why not do a vendor import of popt 1.16 e.g.? are there license > restrictions? AFAIK it is GPL; it was used in Red Hat Linux prior to the split into Fedora and RHEL (and may still be, for all I know). > or maybe some other lib... Dunno what-all may be available. popt has its own set of limitations. Check the archives from the RPM mailing list from around the time when RPM switched to popt, or perhaps it was when rpmbuild was split out into a separate executable, for the laundry list of issues they encountered in attempting to maintain compatibility at the command line level between what they had before and after. From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 27 08:29:15 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8015E1065679 for ; Mon, 27 Sep 2010 08:29:15 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 3EFF08FC19 for ; Mon, 27 Sep 2010 08:29:14 +0000 (UTC) Received: by iwn34 with SMTP id 34so5801041iwn.13 for ; Mon, 27 Sep 2010 01:29:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:cc:subject:references:in-reply-to :x-enigmail-version:content-type:content-transfer-encoding; bh=Fi6XtWQuSUzuIRRDNQPg2l0Q5KgvD3T02nPR3NKM8Nc=; b=LgRLXWkzniejte12ake+PZFCu8bpiaJZ/NEtbcfChPSQCkqlb23QHKINbQVkhsC8TB gjcmE0BYdlioO06o+uFar3UlEgE36gVNCaq/6/qutgbra2tk6DVKEp/n+fPl+vhwbRyN dVfwh7yCbMG8xMUDHmavfhhsuQfaw3Cl6D9pU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; b=wELHXofdRtEJUmKI+GkPvQ0pBKkFMPE8MrlFbHNlgVCzTkN8AN0R3KOYM8dkQgMobG GRuwy+9Cp6THlZK6Pg34yQlAtnwSPxX7xYXavf8Yr/dFTl/xaNVy1hHXcivOeLGCeKGD SsGLNMqrUU0kdED5+LHEXckOneSvx7kBRo7sU= Received: by 10.231.160.17 with SMTP id l17mr8773681ibx.102.1285576113539; Mon, 27 Sep 2010 01:28:33 -0700 (PDT) Received: from centel.dataix.local ([99.181.136.118]) by mx.google.com with ESMTPS id g31sm5952844ibh.10.2010.09.27.01.28.31 (version=SSLv3 cipher=RC4-MD5); Mon, 27 Sep 2010 01:28:32 -0700 (PDT) Sender: "J. Hellenthal" Message-ID: <4CA055AE.9050507@DataIX.net> Date: Mon, 27 Sep 2010 04:28:30 -0400 From: jhell User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.2.9) Gecko/20100917 Lightning/1.0b1 Thunderbird MIME-Version: 1.0 To: Aryeh Friedman References: <4CA010CA.6000709@DataIX.net> <20100927035533.GA48062@misty.eyesbeyond.com> In-Reply-To: X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org Subject: Re: csup or svn X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Sep 2010 08:29:15 -0000 Just as much as top-posting I would say ;) On 09/27/2010 00:13, Aryeh Friedman wrote: > Isn't that a step backwards? > > On Sun, Sep 26, 2010 at 11:55 PM, Greg Lewis wrote: >> On Sun, Sep 26, 2010 at 11:34:34PM -0400, jhell wrote: >>> On 09/26/2010 22:47, Aryeh Friedman wrote: >>>> I currently use: >>>> >>>> csup -h cvsup9.us.freebsd.org /usr/share/examples/cvsup/cvs-supfile >>>> >>>> and when I just ran it I got: >>>> >>>> Append to CVSROOT-src/access >>>> Edit CVSROOT-src/access,v >>>> Segmentation fault (core dumped) >>>> >>>> >>>> (8.1-STABLE #0) >>>> >>>> Should I be using SVN instead and if so what is the repo URL I need to use? >>> >>> You can safely remove that file and continue to do what you do... or >>> >>> Yes you can use SVN ports/devel/subversion-freebsd >> >> Or you can use cvsup (it doesn't segfault on that file). >> >> -- >> Greg Lewis Email : glewis@eyesbeyond.com >> Eyes Beyond Web : http://www.eyesbeyond.com >> Information Technology FreeBSD : glewis@FreeBSD.org >> -- jhell,v From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 27 09:47:01 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4ED72106566B; Mon, 27 Sep 2010 09:47:01 +0000 (UTC) (envelope-from jeremie@le-hen.org) Received: from smtp5-g21.free.fr (smtp5-g21.free.fr [212.27.42.5]) by mx1.freebsd.org (Postfix) with ESMTP id 49ADE8FC20; Mon, 27 Sep 2010 09:46:58 +0000 (UTC) Received: from endor.tataz.chchile.org (unknown [82.233.239.98]) by smtp5-g21.free.fr (Postfix) with ESMTP id 7E72CD480FB; Mon, 27 Sep 2010 11:46:52 +0200 (CEST) Received: from felucia.tataz.chchile.org (felucia.tataz.chchile.org [192.168.1.9]) by endor.tataz.chchile.org (Postfix) with ESMTP id 6C49733DD7; Mon, 27 Sep 2010 09:46:51 +0000 (UTC) Received: by felucia.tataz.chchile.org (Postfix, from userid 1000) id 69F0DA12ED; Mon, 27 Sep 2010 09:46:51 +0000 (UTC) Date: Mon, 27 Sep 2010 11:46:51 +0200 From: Jeremie Le Hen To: Kostik Belousov Message-ID: <20100927094651.GB57265@felucia.tataz.chchile.org> References: <20100803150545.GH14016@felucia.tataz.chchile.org> <20100803114651.651e0ea4@kan.dnsalias.net> <20100805191446.GJ14016@felucia.tataz.chchile.org> <20100919081406.GH6864@felucia.tataz.chchile.org> <20100919184146.GE2389@deviant.kiev.zoral.com.ua> <20100920162925.GL6864@felucia.tataz.chchile.org> <20100920192708.GK2389@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="h31gzZEtNLTqOjlF" Content-Disposition: inline In-Reply-To: <20100920192708.GK2389@deviant.kiev.zoral.com.ua> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: kan@freebsd.org, freebsd-hackers@freebsd.org, Jeremie Le Hen Subject: Re: [PATCH] Add -lssp_nonshared to GCC's LIB_SPEC unconditionally X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Sep 2010 09:47:01 -0000 --h31gzZEtNLTqOjlF Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi Kostik, On Mon, Sep 20, 2010 at 10:27:08PM +0300, Kostik Belousov wrote: > > You make the script only useful for the stack protection. If build process > does not use libc.so script, but installed system does, you > - require to maintain two places where (not much) hypothetical libc > changes should go; > - make it very puzzling to debug the issues with the build of the usermode. > > Please, do this in the consistent manner, so that the script can be adopted > for other uses. I've updated the patch. I think it will fulfill your requirements. Now the ld script is generated on the fly during the install step. The patch probably needs some polishing such as removing debugging leftovers. Can you tell me if it looks of for you now? Regards, -- Jeremie Le Hen Humans are born free and equal. But some are more equal than others. Coluche --h31gzZEtNLTqOjlF Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="ld_ssp_nonshared.diff" diff -urNp src.orig/lib/libc/Makefile src/lib/libc/Makefile --- src.orig/lib/libc/Makefile 2010-08-01 12:35:01.000000000 +0000 +++ src/lib/libc/Makefile 2010-09-21 23:40:51.000000000 +0000 @@ -20,6 +20,7 @@ CFLAGS+=-DNLS CLEANFILES+=tags INSTALL_PIC_ARCHIVE= PRECIOUSLIB= +SHLIB_LDSCRIPT=libc.ldscript # # Only link with static libgcc.a (no libgcc_eh.a). diff -urNp src.orig/lib/libc/libc.ldscript src/lib/libc/libc.ldscript --- src.orig/lib/libc/libc.ldscript 1970-01-01 00:00:00.000000000 +0000 +++ src/lib/libc/libc.ldscript 2010-09-24 21:56:57.000000000 +0000 @@ -0,0 +1,2 @@ +/* $FreeBSD */ +GROUP ( @@SHLIB@@ /usr/lib/libssp_nonshared.a ) diff -urNp src.orig/share/mk/bsd.lib.mk src/share/mk/bsd.lib.mk --- src.orig/share/mk/bsd.lib.mk 2010-07-30 15:25:57.000000000 +0000 +++ src/share/mk/bsd.lib.mk 2010-09-24 22:01:04.000000000 +0000 @@ -293,9 +293,19 @@ _libinstall: ${_INSTALLFLAGS} ${_SHLINSTALLFLAGS} \ ${SHLIB_NAME} ${DESTDIR}${SHLIBDIR} .if defined(SHLIB_LINK) +.if defined(SHLIB_LDSCRIPT) && !empty(SHLIB_LDSCRIPT) && exists(${.CURDIR}/${SHLIB_LDSCRIPT}) + @echo "DEBUG: install lib${LIB}.ld to ${DESTDIR}${LIBDIR}/${SHLIB_LINK}" + sed -e 's,@@SHLIB@@,${SHLIBDIR}/${SHLIB_NAME},g' \ + ${.CURDIR}/${SHLIB_LDSCRIPT} > lib${LIB}.ld + ${INSTALL} -S -C -o ${LIBOWN} -g ${LIBGRP} -m ${LIBMODE} \ + ${_INSTALLFLAGS} lib${LIB}.ld ${DESTDIR}${LIBDIR} + ln -sf lib${LIB}.ld ${DESTDIR}${LIBDIR}/${SHLIB_LINK} +.else .if ${SHLIBDIR} == ${LIBDIR} + @echo "DEBUG: symlink (1) ${DESTDIR}${LIBDIR}/${SHLIB_LINK} to ${SHLIB_NAME}" ln -fs ${SHLIB_NAME} ${DESTDIR}${LIBDIR}/${SHLIB_LINK} .else + @echo "DEBUG: symlink (2) ${DESTDIR}${LIBDIR}/${SHLIB_LINK} to ${_SHLIBDIRPREFIX}${SHLIBDIR}/${SHLIB_NAME}" ln -fs ${_SHLIBDIRPREFIX}${SHLIBDIR}/${SHLIB_NAME} \ ${DESTDIR}${LIBDIR}/${SHLIB_LINK} .if exists(${DESTDIR}${LIBDIR}/${SHLIB_NAME}) @@ -303,8 +313,9 @@ _libinstall: rm -f ${DESTDIR}${LIBDIR}/${SHLIB_NAME} .endif .endif -.endif -.endif +.endif # SHLIB_LDSCRIPT +.endif # SHLIB_LINK +.endif # SHIB_NAME .if defined(INSTALL_PIC_ARCHIVE) && defined(LIB) && !empty(LIB) && ${MK_TOOLCHAIN} != "no" ${INSTALL} -o ${LIBOWN} -g ${LIBGRP} -m ${LIBMODE} \ ${_INSTALLFLAGS} lib${LIB}_pic.a ${DESTDIR}${LIBDIR} @@ -372,6 +383,9 @@ clean: .endif .if defined(SHLIB_NAME) .if defined(SHLIB_LINK) +.if defined(SHLIB_LDSCRIPT) && exists(${.CURDIR}/${SHLIB_LDSCRIPT}) + rm -f lib${LIB}.ld +.endif rm -f ${SHLIB_LINK} .endif .if defined(LIB) && !empty(LIB) --h31gzZEtNLTqOjlF-- From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 27 11:20:39 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 222F3106564A for ; Mon, 27 Sep 2010 11:20:39 +0000 (UTC) (envelope-from ubique@peterhost.ru) Received: from fb0.z8.ru (fb0.z8.ru [80.93.58.95]) by mx1.freebsd.org (Postfix) with ESMTP id D13828FC0C for ; Mon, 27 Sep 2010 11:20:38 +0000 (UTC) Received: from smtp.z8.ru ([80.93.57.71]) by fb0.z8.ru with esmtp (Exim 4.69 (FreeBSD)) (envelope-from ) id 1P0BWX-000OjL-Ot for freebsd-hackers@freebsd.org; Mon, 27 Sep 2010 15:05:33 +0400 Received: from [212.116.101.94] (helo=amnesiac.pht) by smtp.z8.ru with esmtpa (Exim 4.71 (FreeBSD)) (envelope-from ) id 1P0BWQ-000F2j-SC for freebsd-hackers@freebsd.org; Mon, 27 Sep 2010 15:05:27 +0400 Date: Mon, 27 Sep 2010 15:04:21 +0400 From: Dmitry Banshchikov To: freebsd-hackers@freebsd.org Message-ID: <20100927150421.260874f0@amnesiac.pht> X-Mailer: Claws Mail 3.7.6 (GTK+ 2.16.6; amd64-portbld-freebsd8.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-SA-Exim-Connect-IP: 212.116.101.94 X-SA-Exim-Mail-From: ubique@peterhost.ru X-SA-Exim-Scanned: No (on smtp.z8.ru); SAEximRunCond expanded to false Subject: little mistake in rc.subr? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Sep 2010 11:20:39 -0000 Hello, In /etc/rc.subr, at line 231, there is: if [ ! -f $_pidfile ]; then debug "pid file ($_pidfile): not readable." return fi Is check "[ ! -r $_pidfile ]" more correct? -- Dmitry Banshchikov From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 27 12:05:34 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 05ED51065673 for ; Mon, 27 Sep 2010 12:05:34 +0000 (UTC) (envelope-from kraduk@gmail.com) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id AFB738FC0C for ; Mon, 27 Sep 2010 12:05:33 +0000 (UTC) Received: by qyk7 with SMTP id 7so4754157qyk.13 for ; Mon, 27 Sep 2010 05:05:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=uYafwklf5jx0cQgp+T9Qj90pKXq+RB7NWRCGT0uyfKI=; b=qS06/+5SJjnxyGKpsMbnKIHJd/MuTljrLunQFNRHrRyiZOzhXfp3xZmVCqOg6jg5qR krEJsfL3AcT3DkoW5YMPOzjJygsIh6TasMA15MuCnTFIVS7TjqFrMUJlNjIu319pvnkV KdRzKtwz6FNmOK09VYolFljusYAXCVaylCGWI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=GL0aVx5JnzbWnHfTXJPh5oInAehe6hhTM7bKUsImK9HN6M8neCnhSNWxD9CF7M10gU wAB9h0xUt+po8nqxQKK3nT2DG2VxrkUoFW62Sue3LZJ4il+TdmU2P5kBgYZV21yzNT2z skinOgeY1Acus37jfcH4v4KMZ+qgEdd7y1nrA= MIME-Version: 1.0 Received: by 10.229.223.198 with SMTP id il6mr5615323qcb.50.1285587641714; Mon, 27 Sep 2010 04:40:41 -0700 (PDT) Received: by 10.229.215.209 with HTTP; Mon, 27 Sep 2010 04:40:41 -0700 (PDT) In-Reply-To: <4C9FB0D2.1010205@DataIX.net> References: <4C9FB0D2.1010205@DataIX.net> Date: Mon, 27 Sep 2010 12:40:41 +0100 Message-ID: From: krad To: jhell X-Mailman-Approved-At: Mon, 27 Sep 2010 12:10:06 +0000 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org, =?ISO-8859-1?Q?Samuel_Mart=EDn_Moro?= , freebsd-questions@freebsd.org Subject: Re: pf X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Sep 2010 12:05:34 -0000 On 26 September 2010 21:45, jhell wrote: > This is more for questions@ or pf@ > > On 09/26/2010 11:43, Samuel Mart=EDn Moro wrote: > > On Sun, Sep 26, 2010 at 3:34 PM, Michael Powell >wrote: > > > >> Samuel Mart=EDn Moro wrote: > >> > >>> Hello, > >>> > >>> > >>> I'm trying to set up pf on my soon-to-be new gateway (8.1-RELEASE > amd64). > >>> I used the sample configuration file available on > >>> calomel > >>> After a few tests, it appears that the gate has fully access to the > >>> internet, but I can't open connections from clients to distant server= s > >>> (web, ssh, ...). > >>> Checking pflog log file, I can't see anything about those timeouts, > even > >>> if I added the log directive in every block/pass command. > >>> Everything else seems to work, I can talk with my DNS from the > internet, > >>> ssh redirections to another pc also seems to works. > >>> I just can't access the Internet from a client of my network... > >>> > >>> For debugging, I commented out the options and the 'block all in/out' > >>> directives. > >>> > >>> Here's my config file http://pastebin.com/Nim2zBCx > >>> > >>> Is there someone understanding what I'm doing wrong? > >>> > >> The firewall ruleset is a trifle overly complex for a quick glance; > study > >> and analysis would take some doing. However, if you can reach the > internet > >> from the firewall box and other client computers behind your NAT can't > >> (which is what it sounds like you're describing) it may be just that y= ou > >> are > >> missing gateway_enable=3D"YES" in your /etc/rc.conf. > >> > >> Turning this "ON" makes your firewall box into a router. The status of > this > >> can be checked with: sysctl net.inet.ip.forwarding - a "0" means no > >> gateway > >> and a "1" means gateway. > >> > >> -Mike > >> > >> > >> > >> _______________________________________________ > >> freebsd-questions@freebsd.org mailing list > >> http://lists.freebsd.org/mailman/listinfo/freebsd-questions > >> To unsubscribe, send any mail to " > >> freebsd-questions-unsubscribe@freebsd.org" > >> > > > > the gateway is already enabled (and forwarding is correctly set) > > whatever, I had to do quick, I started again > > I think the missing thing on my old conf was the 'scrub' (at least) > > I made a more simple configuration, as following: > > > > ext_if=3D"bge0" > > int_if=3D"bge1" > > localnet =3D $int_if:network > > emma=3D"10.242.42.200" > > alpha=3D"10.42.42.42" > > delta=3D"10.42.42.44" > > set skip on lo0 > > scrub in on $ext_if all fragment reassemble > > #INTERNETZ > > nat on $ext_if from $localnet to any -> ($ext_if) > > #EMMA > > rdr on $ext_if inet proto tcp from any to ($ext_if) port 1101 -= > > > $emma port 22 > > rdr on $ext_if inet proto tcp from any to ($ext_if) port 307 -> > > $emma port 80 > > #WHAT.CD > > rdr on $ext_if inet proto tcp from any to ($ext_if) port 1666 -= > > > $alpha port 1666 > > #REMOTE ADM > > rdr on $ext_if inet proto tcp from any to ($ext_if) port 1667 -= > > > $delta port 22 > > rdr on $ext_if inet proto tcp from any to ($ext_if) port 1668 -= > > > $alpha port 22 > > pass in log on $ext_if inet proto tcp from any to $ext_if port 22 > > pass in log on $ext_if inet proto tcp from any to $ext_if port 53 > > pass in log on $ext_if inet proto udp from any to $ext_if port 53 > > pass in log on $ext_if inet proto tcp from any to $ext_if port 1664 > > pass in log on $int_if inet proto tcp from any to any > > pass in log on $int_if inet proto udp from any to any > > block in log on $ext_if inet proto icmp from any to $ext_if > > > > it's basically working > > i'll stuff it when I'll have time. > > > > Samuel Mart=EDn Moro > > {EPITECH.} tek5 > > > -- > > jhell,v > _______________________________________________ > freebsd-questions@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to " > freebsd-questions-unsubscribe@freebsd.org" > its worth doing as restart on pf rather than a reload. Ive seen nat rules not take affect sometimes on reloads From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 27 12:32:45 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 029BC106564A for ; Mon, 27 Sep 2010 12:32:45 +0000 (UTC) (envelope-from gljennjohn@googlemail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 7E82C8FC1C for ; Mon, 27 Sep 2010 12:32:44 +0000 (UTC) Received: by fxm9 with SMTP id 9so3479398fxm.13 for ; Mon, 27 Sep 2010 05:32:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:received:received:date:from:to:cc:subject :message-id:in-reply-to:references:reply-to:x-mailer:mime-version :content-type:content-transfer-encoding; bh=FYLrGxv1aydXUSwPzasXFn6kYQ/xivBCtJJiV/JZETI=; b=lDVV9vTgGALbKIFxhzeBxPGP46sCsyShuV8BwvYX54o3kWN5g08RNziEDsnbV3BiK1 9NvkiOuNu/0JB6u8yxKG10qNne36SYx4OfeWNwong/x/d6zojR/QUJkWl2/OaNEidPZb Jka8TAEO4PQZJtCz/WURW1jF/FG+c5d1REVlU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=date:from:to:cc:subject:message-id:in-reply-to:references:reply-to :x-mailer:mime-version:content-type:content-transfer-encoding; b=x02ys3gL+a5FqTI8Rfqu4F4RSLbAbhUm3ptpRQsAOKefAgWVR2uIrBSR8jmrsqXL40 GaWkPfUfrdfywOguRVhtt24y7XjCHvB97JEz861uTutLM/tcAljvzL0R/1EeiKyeTfGx 7XvMF9kSwvVyjegRDa+ItOTAukMc8IPkPqGcc= Received: by 10.223.120.148 with SMTP id d20mr7351133far.14.1285590763479; Mon, 27 Sep 2010 05:32:43 -0700 (PDT) Received: from ernst.jennejohn.org (p578E3B24.dip.t-dialin.net [87.142.59.36]) by mx.google.com with ESMTPS id r5sm2353982faq.32.2010.09.27.05.32.40 (version=TLSv1/SSLv3 cipher=RC4-MD5); Mon, 27 Sep 2010 05:32:42 -0700 (PDT) Date: Mon, 27 Sep 2010 14:32:38 +0200 From: Gary Jennejohn To: jhell Message-ID: <20100927143238.780b2000@ernst.jennejohn.org> In-Reply-To: <4CA055AE.9050507@DataIX.net> References: <4CA010CA.6000709@DataIX.net> <20100927035533.GA48062@misty.eyesbeyond.com> <4CA055AE.9050507@DataIX.net> X-Mailer: Claws Mail 3.7.6 (GTK+ 2.18.7; amd64-portbld-freebsd9.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, Aryeh Friedman Subject: Re: csup or svn X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: gljennjohn@googlemail.com List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Sep 2010 12:32:45 -0000 On Mon, 27 Sep 2010 04:28:30 -0400 jhell wrote: > Just as much as top-posting I would say ;) > > On 09/27/2010 00:13, Aryeh Friedman wrote: > > Isn't that a step backwards? > > > > On Sun, Sep 26, 2010 at 11:55 PM, Greg Lewis wrote: > >> On Sun, Sep 26, 2010 at 11:34:34PM -0400, jhell wrote: > >>> On 09/26/2010 22:47, Aryeh Friedman wrote: > >>>> I currently use: > >>>> > >>>> csup -h cvsup9.us.freebsd.org /usr/share/examples/cvsup/cvs-supfile > >>>> > >>>> and when I just ran it I got: > >>>> > >>>> Append to CVSROOT-src/access > >>>> Edit CVSROOT-src/access,v > >>>> Segmentation fault (core dumped) > >>>> > >>>> > >>>> (8.1-STABLE #0) > >>>> > >>>> Should I be using SVN instead and if so what is the repo URL I need to use? > >>> > >>> You can safely remove that file and continue to do what you do... or > >>> > >>> Yes you can use SVN ports/devel/subversion-freebsd > >> > >> Or you can use cvsup (it doesn't segfault on that file). > >> I just updated my CVS tree with csup - no problems. The suggestion to delete src/access,v and try again is probably the way to go. -- Gary Jennejohn From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 27 12:44:16 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8D120106566B for ; Mon, 27 Sep 2010 12:44:16 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from tensor.andric.com (cl-327.ede-01.nl.sixxs.net [IPv6:2001:7b8:2ff:146::2]) by mx1.freebsd.org (Postfix) with ESMTP id 4F5E38FC1B for ; Mon, 27 Sep 2010 12:44:16 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7:0:3940:c45b:c1c1:f335] (unknown [IPv6:2001:7b8:3a7:0:3940:c45b:c1c1:f335]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 629135C43; Mon, 27 Sep 2010 14:44:15 +0200 (CEST) Message-ID: <4CA0919F.10405@FreeBSD.org> Date: Mon, 27 Sep 2010 14:44:15 +0200 From: Dimitry Andric Organization: The FreeBSD Project User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.9.2.10pre) Gecko/20100910 Lanikai/3.1.4pre MIME-Version: 1.0 To: Aryeh Friedman References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org Subject: Re: csup or svn X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Sep 2010 12:44:16 -0000 On 2010-09-27 04:47, Aryeh Friedman wrote: > I currently use: > > csup -h cvsup9.us.freebsd.org /usr/share/examples/cvsup/cvs-supfile > > and when I just ran it I got: > > Append to CVSROOT-src/access > Edit CVSROOT-src/access,v > Segmentation fault (core dumped) Unfortunately, the access,v file in the master repository is still corrupted, so every time it gets an update, csup will die on it. The short term solution is to run "rm -f CVSROOT-src/access,v" just before your csup command line. The long term solution consists of 1. fixing the master repo, and 2. fixing csup so it doesn't die on corrupt ,v files... From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 27 12:26:30 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5AC7A1065694 for ; Mon, 27 Sep 2010 12:26:30 +0000 (UTC) (envelope-from lars@e.0x20.net) Received: from mail.0x20.net (mail.0x20.net [217.69.67.217]) by mx1.freebsd.org (Postfix) with ESMTP id 1EC828FC2D for ; Mon, 27 Sep 2010 12:26:29 +0000 (UTC) Received: by mail.0x20.net (Postfix, from userid 1002) id 4A4EF3A581; Mon, 27 Sep 2010 14:01:29 +0200 (CEST) Date: Mon, 27 Sep 2010 14:01:29 +0200 From: Lars Engels To: Dmitry Banshchikov Message-ID: <20100927120129.GF95673@e.0x20.net> References: <20100927150421.260874f0@amnesiac.pht> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ylS2wUBXLOxYXZFQ" Content-Disposition: inline In-Reply-To: <20100927150421.260874f0@amnesiac.pht> X-Editor: VIM - Vi IMproved 7.2 X-Operation-System: FreeBSD 5.5-RELEASE-p19 User-Agent: Mutt/1.5.20 (2009-06-14) X-Mailman-Approved-At: Mon, 27 Sep 2010 12:58:02 +0000 Cc: freebsd-hackers@freebsd.org Subject: Re: little mistake in rc.subr? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Sep 2010 12:26:30 -0000 --ylS2wUBXLOxYXZFQ Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Sep 27, 2010 at 03:04:21PM +0400, Dmitry Banshchikov wrote: > Hello, >=20 > In /etc/rc.subr, at line 231, there is: >=20 > if [ ! -f $_pidfile ]; then > debug "pid file ($_pidfile): not readable." > return > fi >=20 > Is check "[ ! -r $_pidfile ]" more correct? There's pratically no difference. rc stuff is run as root, so if there's a file, it's readable, if there's no file, then the debug message is printed. --ylS2wUBXLOxYXZFQ Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (FreeBSD) iEYEARECAAYFAkygh5kACgkQKc512sD3afi4sQCgmxlKG3nCWUc696Z9j8vEdJqR 6vcAn3ycCyEtWPX6t2OFNJOvaIeJNsr3 =G5Zo -----END PGP SIGNATURE----- --ylS2wUBXLOxYXZFQ-- From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 27 13:05:50 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 666C4106566C for ; Mon, 27 Sep 2010 13:05:50 +0000 (UTC) (envelope-from fb-hackers@psconsult.nl) Received: from mx1.psconsult.nl (unknown [IPv6:2001:7b8:30f:e0::5059:ee8a]) by mx1.freebsd.org (Postfix) with ESMTP id 1ECD98FC15 for ; Mon, 27 Sep 2010 13:05:49 +0000 (UTC) Received: from mx1.psconsult.nl (psc11.adsl.iaf.nl [80.89.238.138]) by mx1.psconsult.nl (8.14.4/8.14.4) with ESMTP id o8RD5hUE038887 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Mon, 27 Sep 2010 15:05:48 +0200 (CEST) (envelope-from fb-hackers@psconsult.nl) Received: (from paul@localhost) by mx1.psconsult.nl (8.14.4/8.14.4/Submit) id o8RD5hQW038886 for freebsd-hackers@freebsd.org; Mon, 27 Sep 2010 15:05:43 +0200 (CEST) (envelope-from fb-hackers@psconsult.nl) X-Authentication-Warning: mx1.psconsult.nl: paul set sender to fb-hackers@psconsult.nl using -f Date: Mon, 27 Sep 2010 15:05:43 +0200 From: Paul Schenkeveld To: freebsd-hackers@freebsd.org Message-ID: <20100927130542.GA38728@psconsult.nl> References: <20100927150421.260874f0@amnesiac.pht> <20100927120129.GF95673@e.0x20.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100927120129.GF95673@e.0x20.net> User-Agent: Mutt/1.5.19 (2009-01-05) Subject: Re: little mistake in rc.subr? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Sep 2010 13:05:50 -0000 On Mon, Sep 27, 2010 at 02:01:29PM +0200, Lars Engels wrote: > On Mon, Sep 27, 2010 at 03:04:21PM +0400, Dmitry Banshchikov wrote: > > Hello, > > > > In /etc/rc.subr, at line 231, there is: > > > > if [ ! -f $_pidfile ]; then > > debug "pid file ($_pidfile): not readable." > > return > > fi > > > > Is check "[ ! -r $_pidfile ]" more correct? > > There's pratically no difference. rc stuff is run as root, so if there's > a file, it's readable, if there's no file, then the debug message is > printed. And -f also checks that it is a regular file (or symlink to a regular file). From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 27 15:45:23 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 56E42106564A; Mon, 27 Sep 2010 15:45:23 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 7AB6F8FC1B; Mon, 27 Sep 2010 15:45:21 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id o8RFj1Yj095554 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 27 Sep 2010 18:45:02 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id o8RFj1ZK073964; Mon, 27 Sep 2010 18:45:01 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id o8RFiv49073960; Mon, 27 Sep 2010 18:44:57 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 27 Sep 2010 18:44:57 +0300 From: Kostik Belousov To: Jeremie Le Hen Message-ID: <20100927154457.GJ43070@deviant.kiev.zoral.com.ua> References: <20100803150545.GH14016@felucia.tataz.chchile.org> <20100803114651.651e0ea4@kan.dnsalias.net> <20100805191446.GJ14016@felucia.tataz.chchile.org> <20100919081406.GH6864@felucia.tataz.chchile.org> <20100919184146.GE2389@deviant.kiev.zoral.com.ua> <20100920162925.GL6864@felucia.tataz.chchile.org> <20100920192708.GK2389@deviant.kiev.zoral.com.ua> <20100927094651.GB57265@felucia.tataz.chchile.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="kr14OxHsRwZHHqxS" Content-Disposition: inline In-Reply-To: <20100927094651.GB57265@felucia.tataz.chchile.org> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: kan@freebsd.org, freebsd-hackers@freebsd.org Subject: Re: [PATCH] Add -lssp_nonshared to GCC's LIB_SPEC unconditionally X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Sep 2010 15:45:23 -0000 --kr14OxHsRwZHHqxS Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Sep 27, 2010 at 11:46:51AM +0200, Jeremie Le Hen wrote: > Hi Kostik, >=20 > On Mon, Sep 20, 2010 at 10:27:08PM +0300, Kostik Belousov wrote: > >=20 > > You make the script only useful for the stack protection. If build proc= ess > > does not use libc.so script, but installed system does, you > > - require to maintain two places where (not much) hypothetical libc > > changes should go; > > - make it very puzzling to debug the issues with the build of the userm= ode. > >=20 > > Please, do this in the consistent manner, so that the script can be ado= pted > > for other uses. >=20 > I've updated the patch. I think it will fulfill your requirements. Now > the ld script is generated on the fly during the install step. >=20 > The patch probably needs some polishing such as removing debugging > leftovers. Can you tell me if it looks of for you now? Hardcoding /usr/lib as the path to the library in the script looks problema= tic. For the buidlworld, you are linking resulting binaries with the host librar= y, instead of the buildworld-produced one. For lib32, it makes non-working combination of 32/64 bit. --kr14OxHsRwZHHqxS Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (FreeBSD) iEYEARECAAYFAkygu/gACgkQC3+MBN1Mb4i9rACgmjDeN0w5dXuPCX8D8dwbU8Bg HBwAoJ8oId9CD5V+24/NyCzZFdqDZECQ =VOWz -----END PGP SIGNATURE----- --kr14OxHsRwZHHqxS-- From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 27 15:49:41 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 468451065784 for ; Mon, 27 Sep 2010 15:49:41 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 15AE08FC0A for ; Mon, 27 Sep 2010 15:49:41 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 9B20946B94; Mon, 27 Sep 2010 11:49:40 -0400 (EDT) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 19B868A050; Mon, 27 Sep 2010 11:49:39 -0400 (EDT) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Mon, 27 Sep 2010 11:04:17 -0400 User-Agent: KMail/1.13.5 (FreeBSD/7.3-CBSD-20100819; KDE/4.4.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201009271104.17478.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Mon, 27 Sep 2010 11:49:39 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.6 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: Neel Natu Subject: Re: PATCH: fix bogus error message "bus_dmamem_alloc failed to align memory properly" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Sep 2010 15:49:41 -0000 On Friday, September 24, 2010 9:00:44 pm Neel Natu wrote: > Hi, > > This patch fixes the bogus error message from bus_dmamem_alloc() about > the buffer not being aligned properly. > > The problem is that the check is against a virtual address as opposed > to the physical address. contigmalloc() makes guarantees about > the alignment of physical addresses but not the virtual address > mapping it. > > Any objections if I commit this patch? Hmmm, I guess you are doing super-page alignment rather than sub-page alignment? In general I thought the busdma code only handled sub-page alignment and doesn't fully handle requests for super-page alignment. For example, since it insists on walking individual pages at a time, if you had an alignment setting of 4 pages and passed in a single, aligned 4-page buffer, bus_dma would actually bounce the last 3 pages so that each individual page is 4-page aligned. At least, I think that is what would happen. For sub-page alignment, the virtual and physical address alignments should be the same. > best > Neel > > Index: sys/powerpc/powerpc/busdma_machdep.c > =================================================================== > --- sys/powerpc/powerpc/busdma_machdep.c (revision 213113) > +++ sys/powerpc/powerpc/busdma_machdep.c (working copy) > @@ -529,7 +529,7 @@ > CTR4(KTR_BUSDMA, "%s: tag %p tag flags 0x%x error %d", > __func__, dmat, dmat->flags, ENOMEM); > return (ENOMEM); > - } else if ((uintptr_t)*vaddr & (dmat->alignment - 1)) { > + } else if (vtophys(*vaddr) & (dmat->alignment - 1)) { > printf("bus_dmamem_alloc failed to align memory properly.\n"); > } > #ifdef NOTYET > Index: sys/sparc64/sparc64/bus_machdep.c > =================================================================== > --- sys/sparc64/sparc64/bus_machdep.c (revision 213113) > +++ sys/sparc64/sparc64/bus_machdep.c (working copy) > @@ -652,7 +652,7 @@ > } > if (*vaddr == NULL) > return (ENOMEM); > - if ((uintptr_t)*vaddr % dmat->dt_alignment) > + if (vtophys(*vaddr) % dmat->dt_alignment) > printf("%s: failed to align memory properly.\n", __func__); > return (0); > } > Index: sys/ia64/ia64/busdma_machdep.c > =================================================================== > --- sys/ia64/ia64/busdma_machdep.c (revision 213113) > +++ sys/ia64/ia64/busdma_machdep.c (working copy) > @@ -455,7 +455,7 @@ > } > if (*vaddr == NULL) > return (ENOMEM); > - else if ((uintptr_t)*vaddr & (dmat->alignment - 1)) > + else if (vtophys(*vaddr) & (dmat->alignment - 1)) > printf("bus_dmamem_alloc failed to align memory properly.\n"); > return (0); > } > Index: sys/i386/i386/busdma_machdep.c > =================================================================== > --- sys/i386/i386/busdma_machdep.c (revision 213113) > +++ sys/i386/i386/busdma_machdep.c (working copy) > @@ -540,7 +540,7 @@ > CTR4(KTR_BUSDMA, "%s: tag %p tag flags 0x%x error %d", > __func__, dmat, dmat->flags, ENOMEM); > return (ENOMEM); > - } else if ((uintptr_t)*vaddr & (dmat->alignment - 1)) { > + } else if (vtophys(*vaddr) & (dmat->alignment - 1)) { > printf("bus_dmamem_alloc failed to align memory properly.\n"); > } > if (flags & BUS_DMA_NOCACHE) > Index: sys/amd64/amd64/busdma_machdep.c > =================================================================== > --- sys/amd64/amd64/busdma_machdep.c (revision 213113) > +++ sys/amd64/amd64/busdma_machdep.c (working copy) > @@ -526,7 +526,7 @@ > CTR4(KTR_BUSDMA, "%s: tag %p tag flags 0x%x error %d", > __func__, dmat, dmat->flags, ENOMEM); > return (ENOMEM); > - } else if ((uintptr_t)*vaddr & (dmat->alignment - 1)) { > + } else if (vtophys(*vaddr) & (dmat->alignment - 1)) { > printf("bus_dmamem_alloc failed to align memory properly.\n"); > } > if (flags & BUS_DMA_NOCACHE) > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 27 17:23:18 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6603A10656C9 for ; Mon, 27 Sep 2010 17:23:18 +0000 (UTC) (envelope-from glewis@eyesbeyond.com) Received: from misty.eyesbeyond.com (gerbercreations.com [71.39.140.16]) by mx1.freebsd.org (Postfix) with ESMTP id 1AE2A8FC0C for ; Mon, 27 Sep 2010 17:23:17 +0000 (UTC) Received: from misty.eyesbeyond.com (localhost.eyesbeyond.com [127.0.0.1]) by misty.eyesbeyond.com (8.14.4/8.14.4) with ESMTP id o8RHN0xH085822; Mon, 27 Sep 2010 10:23:00 -0700 (PDT) (envelope-from glewis@eyesbeyond.com) Received: (from glewis@localhost) by misty.eyesbeyond.com (8.14.4/8.14.4/Submit) id o8RHMxTO085821; Mon, 27 Sep 2010 10:22:59 -0700 (PDT) (envelope-from glewis@eyesbeyond.com) X-Authentication-Warning: misty.eyesbeyond.com: glewis set sender to glewis@eyesbeyond.com using -f Date: Mon, 27 Sep 2010 10:22:59 -0700 From: Greg Lewis To: Aryeh Friedman Message-ID: <20100927172259.GA85798@misty.eyesbeyond.com> References: <4CA010CA.6000709@DataIX.net> <20100927035533.GA48062@misty.eyesbeyond.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-hackers@freebsd.org Subject: Re: csup or svn X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Sep 2010 17:23:18 -0000 On Mon, Sep 27, 2010 at 12:13:14AM -0400, Aryeh Friedman wrote: > Isn't that a step backwards? Segfaulting seems like a bigger step backwards to me ;) > On Sun, Sep 26, 2010 at 11:55 PM, Greg Lewis wrote: > > On Sun, Sep 26, 2010 at 11:34:34PM -0400, jhell wrote: > >> On 09/26/2010 22:47, Aryeh Friedman wrote: > >> > I currently use: > >> > > >> > csup -h cvsup9.us.freebsd.org /usr/share/examples/cvsup/cvs-supfile > >> > > >> > and when I just ran it I got: > >> > > >> > ?Append to CVSROOT-src/access > >> > ?Edit CVSROOT-src/access,v > >> > Segmentation fault (core dumped) > >> > > >> > > >> > (8.1-STABLE #0) > >> > > >> > Should I be using SVN instead and if so what is the repo URL I need to use? > >> > >> You can safely remove that file and continue to do what you do... or > >> > >> Yes you can use SVN ports/devel/subversion-freebsd > > > > Or you can use cvsup (it doesn't segfault on that file). > > > > -- > > Greg Lewis ? ? ? ? ? ? ? ? ? ? ? ? ?Email ? : glewis@eyesbeyond.com > > Eyes Beyond ? ? ? ? ? ? ? ? ? ? ? ? Web ? ? : http://www.eyesbeyond.com > > Information Technology ? ? ? ? ? ? ?FreeBSD : glewis@FreeBSD.org > > -- Greg Lewis Email : glewis@eyesbeyond.com Eyes Beyond Web : http://www.eyesbeyond.com Information Technology FreeBSD : glewis@FreeBSD.org From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 27 20:56:08 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 370951065672 for ; Mon, 27 Sep 2010 20:56:08 +0000 (UTC) (envelope-from pali.gabor@googlemail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id BB63B8FC0A for ; Mon, 27 Sep 2010 20:56:07 +0000 (UTC) Received: by fxm9 with SMTP id 9so4051399fxm.13 for ; Mon, 27 Sep 2010 13:56:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :organization:user-agent:mime-version:to:subject:content-type :content-transfer-encoding; bh=163Bujw3YKu6s6UHTYblqzf1tMEem9aG+U1gLDJD6go=; b=mZsX+GsKfZmG3Alj93Iv3sjyNTS0wK8yNNGjLgjK+VHDJQ+sscL/jFcvgYAnInQuLa pc/bWTFOHWMFkYxawdtZUn6BRP3+8Wp542r3KULZ6SCfdr3AoPY9gSxZvlJVeXMrP6eW FFWTRds4g2itHw32XQlYzmoNSxU8hhT0PLuog= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=sender:message-id:date:from:organization:user-agent:mime-version:to :subject:content-type:content-transfer-encoding; b=vETJ1T0YgRvKZJkkU7DROJMY1lhjqJwSdSCqJPEcfZmPJgLYR/Rywm/s8knbiCmzQu IQs/X/I6dk+XUE4bgyavt2CerwW1nmMF09BtV3Rb28nVCdJIsYk2dnlb4gNXMAhjuxOI efYC3j9LGSe15Hkg7iiIi/epzp7lpJPt2b+Dk= Received: by 10.223.117.206 with SMTP id s14mr1472500faq.81.1285619587819; Mon, 27 Sep 2010 13:33:07 -0700 (PDT) Received: from [157.181.167.71] (oktnb71.inf.elte.hu [157.181.167.71]) by mx.google.com with ESMTPS id b9sm2697206faq.7.2010.09.27.13.33.06 (version=SSLv3 cipher=RC4-MD5); Mon, 27 Sep 2010 13:33:07 -0700 (PDT) Sender: =?UTF-8?B?UMOBTEkgR8OhYm9yIErDoW5vcw==?= Message-ID: <4CA0FF2C.9010108@elte.hu> Date: Mon, 27 Sep 2010 22:31:40 +0200 From: =?ISO-8859-1?Q?P=C1LI_G=E1bor_J=E1nos?= Organization: =?ISO-8859-1?Q?E=F6tv=F6s_Lor=E1nd_University?= User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100925 Thunderbird/3.1.4 MIME-Version: 1.0 To: hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Mon, 27 Sep 2010 21:09:02 +0000 Cc: Subject: clogf(3) (complex.h) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Sep 2010 20:56:08 -0000 Hello, I would like to use the clogf(3) function from complex.h, but it seems there is no such function implemented on FreeBSD (8.1-STABLE). Am I missing something or is there any way to work this around? Thanks, :g From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 27 21:13:05 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9ABC4106564A for ; Mon, 27 Sep 2010 21:13:05 +0000 (UTC) (envelope-from neelnatu@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 0289F8FC14 for ; Mon, 27 Sep 2010 21:13:04 +0000 (UTC) Received: by wwc33 with SMTP id 33so395548wwc.31 for ; Mon, 27 Sep 2010 14:13:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=S85UfBFw7cHzEv+9Q2zdc1V839tui+ltoZelPnNmL/w=; b=qwf0u6dIAvysBkDHQsyR/TbJv/ZWAyVEObKJzK6BNxfXdMYH0uCcCPkyWVYwdcQyKO iNFwX9XBtqg/zHBDLh9uG6mtU8G0VA8Zr/7btxnSigCs3g8v0f05zs7pijxP7Z2jLGEC xa4cuxayawDFht6tYZL8gYv6MiJjx25m9ujQ8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=e2HK6escS8qbDCKR5Gu5n8ZZbZgVEvNA7SH9nG4RtyZ2dGB4NtSx60ps2lhrJAdJPN c6ZD3WLjRZlVSBbuJRJo2v4Zjn1N13zqsIgE6viD3Ldg9LNtvQ1HJjhJmrWR1oHXsR1N z6vWmBQfzFBh/2SEz7pUHNy/ZIsT5c/QIQe4A= MIME-Version: 1.0 Received: by 10.216.186.207 with SMTP id w57mr159664wem.19.1285621983690; Mon, 27 Sep 2010 14:13:03 -0700 (PDT) Received: by 10.216.133.5 with HTTP; Mon, 27 Sep 2010 14:13:03 -0700 (PDT) In-Reply-To: <201009271104.17478.jhb@freebsd.org> References: <201009271104.17478.jhb@freebsd.org> Date: Mon, 27 Sep 2010 14:13:03 -0700 Message-ID: From: Neel Natu To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: PATCH: fix bogus error message "bus_dmamem_alloc failed to align memory properly" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Sep 2010 21:13:05 -0000 Hi John, Thanks for reviewing this. On Mon, Sep 27, 2010 at 8:04 AM, John Baldwin wrote: > On Friday, September 24, 2010 9:00:44 pm Neel Natu wrote: >> Hi, >> >> This patch fixes the bogus error message from bus_dmamem_alloc() about >> the buffer not being aligned properly. >> >> The problem is that the check is against a virtual address as opposed >> to the physical address. contigmalloc() makes guarantees about >> the alignment of physical addresses but not the virtual address >> mapping it. >> >> Any objections if I commit this patch? > > Hmmm, I guess you are doing super-page alignment rather than sub-page > alignment? =A0In general I thought the busdma code only handled sub-page > alignment and doesn't fully handle requests for super-page alignment. > Yes, this is for allocations with sizes greater than PAGE_SIZE and alignment requirements also greater than a PAGE_SIZE. > For example, since it insists on walking individual pages at a time, if y= ou > had an alignment setting of 4 pages and passed in a single, aligned 4-pag= e > buffer, bus_dma would actually bounce the last 3 pages so that each indiv= idual > page is 4-page aligned. =A0At least, I think that is what would happen. > I think you are referring to bus_dmamap_load() operation that would follow the bus_dmamem_alloc(), right? The memory allocated by bus_dmamem_alloc() does not need to be bounced. In fact, the dmamap pointer returned by bus_dmamem_alloc() is NULL. At least for the amd64 implementation there is code in _bus_dmamap_load_buffer() which will coalesce individual dma segments if they satisfy 'boundary' and 'segsize' constraints. 683 /* 684 * Insert chunk into a segment, coalescing with 685 * previous segment if possible. 686 */ 687 if (first) { 688 segs[seg].ds_addr =3D curaddr; 689 segs[seg].ds_len =3D sgsize; 690 first =3D 0; 691 } else { 692 if (curaddr =3D=3D lastaddr && 693 (segs[seg].ds_len + sgsize) <=3D dmat->maxsegsz && 694 (dmat->boundary =3D=3D 0 || 695 (segs[seg].ds_addr & bmask) =3D=3D (curaddr & bmask))) 696 segs[seg].ds_len +=3D sgsize; 697 else { 698 if (++seg >=3D dmat->nsegments) 699 break; 700 segs[seg].ds_addr =3D curaddr; 701 segs[seg].ds_len =3D sgsize; 702 } 703 } I do get the expected result after I call dma_dmamap_load() i.e. a single dma segment with the correct alignment and boundary settings. > For sub-page alignment, the virtual and physical address alignments shoul= d be > the same. > Yes. best Neel >> best >> Neel >> >> Index: sys/powerpc/powerpc/busdma_machdep.c >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> --- sys/powerpc/powerpc/busdma_machdep.c =A0 =A0 =A0(revision 213113) >> +++ sys/powerpc/powerpc/busdma_machdep.c =A0 =A0 =A0(working copy) >> @@ -529,7 +529,7 @@ >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 CTR4(KTR_BUSDMA, "%s: tag %p tag flags 0x%x = error %d", >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 __func__, dmat, dmat->flags, ENOMEM)= ; >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 return (ENOMEM); >> - =A0 =A0 } else if ((uintptr_t)*vaddr & (dmat->alignment - 1)) { >> + =A0 =A0 } else if (vtophys(*vaddr) & (dmat->alignment - 1)) { >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 printf("bus_dmamem_alloc failed to align mem= ory properly.\n"); >> =A0 =A0 =A0 } >> =A0#ifdef NOTYET >> Index: sys/sparc64/sparc64/bus_machdep.c >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> --- sys/sparc64/sparc64/bus_machdep.c (revision 213113) >> +++ sys/sparc64/sparc64/bus_machdep.c (working copy) >> @@ -652,7 +652,7 @@ >> =A0 =A0 =A0 } >> =A0 =A0 =A0 if (*vaddr =3D=3D NULL) >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 return (ENOMEM); >> - =A0 =A0 if ((uintptr_t)*vaddr % dmat->dt_alignment) >> + =A0 =A0 if (vtophys(*vaddr) % dmat->dt_alignment) >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 printf("%s: failed to align memory properly.= \n", __func__); >> =A0 =A0 =A0 return (0); >> =A0} >> Index: sys/ia64/ia64/busdma_machdep.c >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> --- sys/ia64/ia64/busdma_machdep.c =A0 =A0(revision 213113) >> +++ sys/ia64/ia64/busdma_machdep.c =A0 =A0(working copy) >> @@ -455,7 +455,7 @@ >> =A0 =A0 =A0 } >> =A0 =A0 =A0 if (*vaddr =3D=3D NULL) >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 return (ENOMEM); >> - =A0 =A0 else if ((uintptr_t)*vaddr & (dmat->alignment - 1)) >> + =A0 =A0 else if (vtophys(*vaddr) & (dmat->alignment - 1)) >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 printf("bus_dmamem_alloc failed to align mem= ory properly.\n"); >> =A0 =A0 =A0 return (0); >> =A0} >> Index: sys/i386/i386/busdma_machdep.c >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> --- sys/i386/i386/busdma_machdep.c =A0 =A0(revision 213113) >> +++ sys/i386/i386/busdma_machdep.c =A0 =A0(working copy) >> @@ -540,7 +540,7 @@ >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 CTR4(KTR_BUSDMA, "%s: tag %p tag flags 0x%x = error %d", >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 __func__, dmat, dmat->flags, ENOMEM)= ; >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 return (ENOMEM); >> - =A0 =A0 } else if ((uintptr_t)*vaddr & (dmat->alignment - 1)) { >> + =A0 =A0 } else if (vtophys(*vaddr) & (dmat->alignment - 1)) { >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 printf("bus_dmamem_alloc failed to align mem= ory properly.\n"); >> =A0 =A0 =A0 } >> =A0 =A0 =A0 if (flags & BUS_DMA_NOCACHE) >> Index: sys/amd64/amd64/busdma_machdep.c >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> --- sys/amd64/amd64/busdma_machdep.c =A0(revision 213113) >> +++ sys/amd64/amd64/busdma_machdep.c =A0(working copy) >> @@ -526,7 +526,7 @@ >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 CTR4(KTR_BUSDMA, "%s: tag %p tag flags 0x%x = error %d", >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 __func__, dmat, dmat->flags, ENOMEM)= ; >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 return (ENOMEM); >> - =A0 =A0 } else if ((uintptr_t)*vaddr & (dmat->alignment - 1)) { >> + =A0 =A0 } else if (vtophys(*vaddr) & (dmat->alignment - 1)) { >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 printf("bus_dmamem_alloc failed to align mem= ory properly.\n"); >> =A0 =A0 =A0 } >> =A0 =A0 =A0 if (flags & BUS_DMA_NOCACHE) >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.or= g" >> > > -- > John Baldwin > From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 27 21:53:59 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BCE2E1065673 for ; Mon, 27 Sep 2010 21:53:59 +0000 (UTC) (envelope-from db@db.net) Received: from diana.db.net (diana.db.net [66.113.102.10]) by mx1.freebsd.org (Postfix) with ESMTP id A81FA8FC12 for ; Mon, 27 Sep 2010 21:53:59 +0000 (UTC) Received: from night.db.net (localhost [127.0.0.1]) by diana.db.net (Postfix) with ESMTP id B9AD52282A; Mon, 27 Sep 2010 15:27:22 -0600 (MDT) Received: by night.db.net (Postfix, from userid 100) id AAE715C6E; Mon, 27 Sep 2010 17:28:52 -0400 (EDT) Date: Mon, 27 Sep 2010 17:28:52 -0400 From: Diane Bruce To: P?LI G?bor J?nos Message-ID: <20100927212852.GA61431@night.db.net> References: <4CA0FF2C.9010108@elte.hu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4CA0FF2C.9010108@elte.hu> User-Agent: Mutt/1.4.2.3i Cc: hackers@freebsd.org Subject: Re: clogf(3) (complex.h) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Sep 2010 21:53:59 -0000 On Mon, Sep 27, 2010 at 10:31:40PM +0200, P?LI G?bor J?nos wrote: > Hello, > > I would like to use the clogf(3) function from complex.h, but it seems > there is no such function implemented on FreeBSD (8.1-STABLE). Am I > missing something or is there any way to work this around? > http://www.freebsd.org/cgi/query-pr.cgi?pr=147599&cat= - Diane -- - db@FreeBSD.org db@db.net http://www.db.net/~db From owner-freebsd-hackers@FreeBSD.ORG Tue Sep 28 15:25:58 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B891E1065673 for ; Tue, 28 Sep 2010 15:25:58 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 73BC58FC1E for ; Tue, 28 Sep 2010 15:25:58 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 0C22046B8E; Tue, 28 Sep 2010 11:25:58 -0400 (EDT) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id F31468A04F; Tue, 28 Sep 2010 11:25:56 -0400 (EDT) From: John Baldwin To: Neel Natu Date: Tue, 28 Sep 2010 09:36:39 -0400 User-Agent: KMail/1.13.5 (FreeBSD/7.3-CBSD-20100819; KDE/4.4.5; amd64; ; ) References: <201009271104.17478.jhb@freebsd.org> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201009280936.40203.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Tue, 28 Sep 2010 11:25:57 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.6 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: freebsd-hackers@freebsd.org Subject: Re: PATCH: fix bogus error message "bus_dmamem_alloc failed to align memory properly" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Sep 2010 15:25:58 -0000 On Monday, September 27, 2010 5:13:03 pm Neel Natu wrote: > Hi John, > > Thanks for reviewing this. > > On Mon, Sep 27, 2010 at 8:04 AM, John Baldwin wrote: > > On Friday, September 24, 2010 9:00:44 pm Neel Natu wrote: > >> Hi, > >> > >> This patch fixes the bogus error message from bus_dmamem_alloc() about > >> the buffer not being aligned properly. > >> > >> The problem is that the check is against a virtual address as opposed > >> to the physical address. contigmalloc() makes guarantees about > >> the alignment of physical addresses but not the virtual address > >> mapping it. > >> > >> Any objections if I commit this patch? > > > > Hmmm, I guess you are doing super-page alignment rather than sub-page > > alignment? In general I thought the busdma code only handled sub-page > > alignment and doesn't fully handle requests for super-page alignment. > > > > Yes, this is for allocations with sizes greater than PAGE_SIZE and > alignment requirements also greater than a PAGE_SIZE. > > > For example, since it insists on walking individual pages at a time, if you > > had an alignment setting of 4 pages and passed in a single, aligned 4-page > > buffer, bus_dma would actually bounce the last 3 pages so that each individual > > page is 4-page aligned. At least, I think that is what would happen. > > > > I think you are referring to bus_dmamap_load() operation that would > follow the bus_dmamem_alloc(), right? The memory allocated by > bus_dmamem_alloc() does not need to be bounced. In fact, the dmamap > pointer returned by bus_dmamem_alloc() is NULL. > > At least for the amd64 implementation there is code in > _bus_dmamap_load_buffer() which will coalesce individual dma segments > if they satisfy 'boundary' and 'segsize' constraints. So the problem is earlier in the routine where it does this: /* * Get the physical address for this segment. */ if (pmap) curaddr = pmap_extract(pmap, vaddr); else curaddr = pmap_kextract(vaddr); /* * Compute the segment size, and adjust counts. */ max_sgsize = MIN(buflen, dmat->maxsegsz); sgsize = PAGE_SIZE - ((vm_offset_t)curaddr & PAGE_MASK); if (map->pagesneeded != 0 && run_filter(dmat, curaddr)) { sgsize = roundup2(sgsize, dmat->alignment); sgsize = MIN(sgsize, max_sgsize); curaddr = add_bounce_page(dmat, map, vaddr, sgsize); } else { sgsize = MIN(sgsize, max_sgsize); } If you have a map that does need bouncing, then it will split up the pages. It happens to work for bus_dmamem_alloc() because that returns a NULL map which doesn't bounce. But if you had a PCI device which supported only 32-bit addresses on a 64-bit machine with an aligned, 4 page buffer above 4GB and did a bus_dma_map_load() on that buffer, it would get split up into 4 separate 4 page-aligned pages. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Tue Sep 28 18:22:00 2010 Return-Path: Delivered-To: hackers@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A0609106564A for ; Tue, 28 Sep 2010 18:22:00 +0000 (UTC) (envelope-from das@FreeBSD.ORG) Received: from zim.MIT.EDU (ZIM.MIT.EDU [18.95.3.101]) by mx1.freebsd.org (Postfix) with ESMTP id 4A9898FC12 for ; Tue, 28 Sep 2010 18:21:59 +0000 (UTC) Received: from zim.MIT.EDU (localhost [127.0.0.1]) by zim.MIT.EDU (8.14.3/8.14.2) with ESMTP id o8SI1c1v076674; Tue, 28 Sep 2010 14:01:38 -0400 (EDT) (envelope-from das@FreeBSD.ORG) Received: (from das@localhost) by zim.MIT.EDU (8.14.3/8.14.2/Submit) id o8SI1c00076673; Tue, 28 Sep 2010 14:01:38 -0400 (EDT) (envelope-from das@FreeBSD.ORG) Date: Tue, 28 Sep 2010 14:01:38 -0400 From: David Schultz To: =?iso-8859-1?Q?P=C1LI_G=E1bor_J=E1nos?= Message-ID: <20100928180138.GA76574@zim.MIT.EDU> Mail-Followup-To: =?iso-8859-1?Q?P=C1LI_G=E1bor_J=E1nos?= , hackers@freebsd.org References: <4CA0FF2C.9010108@elte.hu> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <4CA0FF2C.9010108@elte.hu> Cc: hackers@FreeBSD.ORG Subject: Re: clogf(3) (complex.h) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Sep 2010 18:22:00 -0000 On Mon, Sep 27, 2010, PÁLI Gábor János wrote: > Hello, > > I would like to use the clogf(3) function from complex.h, but it seems > there is no such function implemented on FreeBSD (8.1-STABLE). Am I > missing something or is there any way to work this around? A simple workaround is something like: logf(cabs(z)) + i * cargf(z) There are a few special cases, e.g., when creal(z) < 0, you have to add i * copysign(M_PI, cimag(z)), and when creal(z) == 0, it's M_PI/2... From owner-freebsd-hackers@FreeBSD.ORG Tue Sep 28 18:49:08 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id 7B31D1065679; Tue, 28 Sep 2010 18:49:08 +0000 (UTC) Date: Tue, 28 Sep 2010 18:49:08 +0000 From: Alexander Best To: perryh@pluto.rain.com Message-ID: <20100928184908.GA99059@freebsd.org> References: <20100927012936.GA32352@freebsd.org> <4ca026de.pOkFIzh1Wymrvuow%perryh@pluto.rain.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4ca026de.pOkFIzh1Wymrvuow%perryh@pluto.rain.com> Cc: freebsd-hackers@freebsd.org Subject: Re: adding a new lib for more advanced argument parsing X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Sep 2010 18:49:08 -0000 On Sun Sep 26 10, perryh@pluto.rain.com wrote: > Alexander Best wrote: > > > ... getopt(3) is clearly not suitable for handling such complex > > options. camcontrol.c even contains a whole paragraph about why > > getopt(3) is considered not appropriate to handle camcontrol's > > argument parsing requirements ... > > > why not do a vendor import of popt 1.16 e.g.? are there license > > restrictions? > > AFAIK it is GPL; it was used in Red Hat Linux prior to the split > into Fedora and RHEL (and may still be, for all I know). > > > or maybe some other lib... > > Dunno what-all may be available. > > popt has its own set of limitations. Check the archives from > the RPM mailing list from around the time when RPM switched > to popt, or perhaps it was when rpmbuild was split out into > a separate executable, for the laundry list of issues they > encountered in attempting to maintain compatibility at the > command line level between what they had before and after. oh i didn't know that. i thought it was superior to getopt() in every fashion. also i'm not voting to completely get rid of getopt(), but merely to make it easier for developers to handle complex options. i mean posix is fine and all, but if they recommend using horses instead of cars shouldn't it be decided that we need both: for comptaibility and performance? cheers. alex -- a13x From owner-freebsd-hackers@FreeBSD.ORG Tue Sep 28 20:03:45 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2B91C106566B; Tue, 28 Sep 2010 20:03:45 +0000 (UTC) (envelope-from neelnatu@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 4393A8FC13; Tue, 28 Sep 2010 20:03:26 +0000 (UTC) Received: by wya21 with SMTP id 21so10455wya.13 for ; Tue, 28 Sep 2010 13:03:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=jQV7DSJ0011K8um/7nOzkkQsKX3hOuNCMosto38uu9M=; b=MfwsnEC+PP1cHE4R1Dvjb3JtOFf7XcNnPqdcbcjQitY0LQoOFdXMXMl9laSArIVKj5 Vh95ttDvC1Qmpi/Yk+Q5UAmwmbRe384pmh7HNX0b4AYrpF38umkhIAqO/s5sdTmar5wd dTDR+0ja1bd7tJDAo/qMLTL8DJadAgbopvhew= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=TjJsBPE30eeacL+ZLaueLiPI+1UBxlAQtWkLXlyYv/LZ+6uD9IgEFgpBgyDUdxeaJp q3tTtji3QYJhz159gZR3TbW1TC/12PC9UAi4eZbsniujuFrjnQEIC/TP/DPxK+oXG1BR 2I86tvg7X2XGExJ2SWGeTO0gW0gyTOLW9/qdo= MIME-Version: 1.0 Received: by 10.216.235.106 with SMTP id t84mr448515weq.46.1285704129092; Tue, 28 Sep 2010 13:02:09 -0700 (PDT) Received: by 10.216.133.5 with HTTP; Tue, 28 Sep 2010 13:02:08 -0700 (PDT) In-Reply-To: <201009280936.40203.jhb@freebsd.org> References: <201009271104.17478.jhb@freebsd.org> <201009280936.40203.jhb@freebsd.org> Date: Tue, 28 Sep 2010 13:02:08 -0700 Message-ID: From: Neel Natu To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: PATCH: fix bogus error message "bus_dmamem_alloc failed to align memory properly" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Sep 2010 20:03:45 -0000 Hi John, On Tue, Sep 28, 2010 at 6:36 AM, John Baldwin wrote: > On Monday, September 27, 2010 5:13:03 pm Neel Natu wrote: >> Hi John, >> >> Thanks for reviewing this. >> >> On Mon, Sep 27, 2010 at 8:04 AM, John Baldwin wrote: >> > On Friday, September 24, 2010 9:00:44 pm Neel Natu wrote: >> >> Hi, >> >> >> >> This patch fixes the bogus error message from bus_dmamem_alloc() abou= t >> >> the buffer not being aligned properly. >> >> >> >> The problem is that the check is against a virtual address as opposed >> >> to the physical address. contigmalloc() makes guarantees about >> >> the alignment of physical addresses but not the virtual address >> >> mapping it. >> >> >> >> Any objections if I commit this patch? >> > >> > Hmmm, I guess you are doing super-page alignment rather than sub-page >> > alignment? =A0In general I thought the busdma code only handled sub-pa= ge >> > alignment and doesn't fully handle requests for super-page alignment. >> > >> >> Yes, this is for allocations with sizes greater than PAGE_SIZE and >> alignment requirements also greater than a PAGE_SIZE. >> >> > For example, since it insists on walking individual pages at a time, i= f you >> > had an alignment setting of 4 pages and passed in a single, aligned 4-= page >> > buffer, bus_dma would actually bounce the last 3 pages so that each in= dividual >> > page is 4-page aligned. =A0At least, I think that is what would happen= . >> > >> >> I think you are referring to bus_dmamap_load() operation that would >> follow the bus_dmamem_alloc(), right? The memory allocated by >> bus_dmamem_alloc() does not need to be bounced. In fact, the dmamap >> pointer returned by bus_dmamem_alloc() is NULL. >> >> At least for the amd64 implementation there is code in >> _bus_dmamap_load_buffer() which will coalesce individual dma segments >> if they satisfy 'boundary' and 'segsize' constraints. > > So the problem is earlier in the routine where it does this: > > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0/* > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 * Get the physical address for this segme= nt. > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 */ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if (pmap) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0curaddr =3D pmap_extract(p= map, vaddr); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0else > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0curaddr =3D pmap_kextract(= vaddr); > > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0/* > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 * Compute the segment size, and adjust co= unts. > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 */ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0max_sgsize =3D MIN(buflen, dmat->maxsegsz)= ; > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0sgsize =3D PAGE_SIZE - ((vm_offset_t)curad= dr & PAGE_MASK); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if (map->pagesneeded !=3D 0 && run_filter(= dmat, curaddr)) { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0sgsize =3D roundup2(sgsize= , dmat->alignment); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0sgsize =3D MIN(sgsize, max= _sgsize); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0curaddr =3D add_bounce_pag= e(dmat, map, vaddr, sgsize); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0} else { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0sgsize =3D MIN(sgsize, max= _sgsize); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0} > > If you have a map that does need bouncing, then it will split up the page= s. > It happens to work for bus_dmamem_alloc() because that returns a NULL map > which doesn't bounce. =A0But if you had a PCI device which supported only > 32-bit addresses on a 64-bit machine with an aligned, 4 page buffer above > 4GB and did a bus_dma_map_load() on that buffer, it would get split up in= to > 4 separate 4 page-aligned pages. > You are right. I assume that you are ok with the patch and the discussion above was an FYI, right? best Neel > -- > John Baldwin > From owner-freebsd-hackers@FreeBSD.ORG Tue Sep 28 20:10:19 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 978F7106566C; Tue, 28 Sep 2010 20:10:19 +0000 (UTC) (envelope-from neelnatu@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 05A3C8FC15; Tue, 28 Sep 2010 20:10:18 +0000 (UTC) Received: by wwb17 with SMTP id 17so55476wwb.31 for ; Tue, 28 Sep 2010 13:10:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=jQV7DSJ0011K8um/7nOzkkQsKX3hOuNCMosto38uu9M=; b=MfwsnEC+PP1cHE4R1Dvjb3JtOFf7XcNnPqdcbcjQitY0LQoOFdXMXMl9laSArIVKj5 Vh95ttDvC1Qmpi/Yk+Q5UAmwmbRe384pmh7HNX0b4AYrpF38umkhIAqO/s5sdTmar5wd dTDR+0ja1bd7tJDAo/qMLTL8DJadAgbopvhew= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=TjJsBPE30eeacL+ZLaueLiPI+1UBxlAQtWkLXlyYv/LZ+6uD9IgEFgpBgyDUdxeaJp q3tTtji3QYJhz159gZR3TbW1TC/12PC9UAi4eZbsniujuFrjnQEIC/TP/DPxK+oXG1BR 2I86tvg7X2XGExJ2SWGeTO0gW0gyTOLW9/qdo= MIME-Version: 1.0 Received: by 10.216.235.106 with SMTP id t84mr448515weq.46.1285704129092; Tue, 28 Sep 2010 13:02:09 -0700 (PDT) Received: by 10.216.133.5 with HTTP; Tue, 28 Sep 2010 13:02:08 -0700 (PDT) In-Reply-To: <201009280936.40203.jhb@freebsd.org> References: <201009271104.17478.jhb@freebsd.org> <201009280936.40203.jhb@freebsd.org> Date: Tue, 28 Sep 2010 13:02:08 -0700 Message-ID: From: Neel Natu To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: PATCH: fix bogus error message "bus_dmamem_alloc failed to align memory properly" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Sep 2010 20:10:19 -0000 Hi John, On Tue, Sep 28, 2010 at 6:36 AM, John Baldwin wrote: > On Monday, September 27, 2010 5:13:03 pm Neel Natu wrote: >> Hi John, >> >> Thanks for reviewing this. >> >> On Mon, Sep 27, 2010 at 8:04 AM, John Baldwin wrote: >> > On Friday, September 24, 2010 9:00:44 pm Neel Natu wrote: >> >> Hi, >> >> >> >> This patch fixes the bogus error message from bus_dmamem_alloc() abou= t >> >> the buffer not being aligned properly. >> >> >> >> The problem is that the check is against a virtual address as opposed >> >> to the physical address. contigmalloc() makes guarantees about >> >> the alignment of physical addresses but not the virtual address >> >> mapping it. >> >> >> >> Any objections if I commit this patch? >> > >> > Hmmm, I guess you are doing super-page alignment rather than sub-page >> > alignment? =A0In general I thought the busdma code only handled sub-pa= ge >> > alignment and doesn't fully handle requests for super-page alignment. >> > >> >> Yes, this is for allocations with sizes greater than PAGE_SIZE and >> alignment requirements also greater than a PAGE_SIZE. >> >> > For example, since it insists on walking individual pages at a time, i= f you >> > had an alignment setting of 4 pages and passed in a single, aligned 4-= page >> > buffer, bus_dma would actually bounce the last 3 pages so that each in= dividual >> > page is 4-page aligned. =A0At least, I think that is what would happen= . >> > >> >> I think you are referring to bus_dmamap_load() operation that would >> follow the bus_dmamem_alloc(), right? The memory allocated by >> bus_dmamem_alloc() does not need to be bounced. In fact, the dmamap >> pointer returned by bus_dmamem_alloc() is NULL. >> >> At least for the amd64 implementation there is code in >> _bus_dmamap_load_buffer() which will coalesce individual dma segments >> if they satisfy 'boundary' and 'segsize' constraints. > > So the problem is earlier in the routine where it does this: > > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0/* > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 * Get the physical address for this segme= nt. > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 */ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if (pmap) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0curaddr =3D pmap_extract(p= map, vaddr); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0else > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0curaddr =3D pmap_kextract(= vaddr); > > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0/* > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 * Compute the segment size, and adjust co= unts. > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 */ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0max_sgsize =3D MIN(buflen, dmat->maxsegsz)= ; > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0sgsize =3D PAGE_SIZE - ((vm_offset_t)curad= dr & PAGE_MASK); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if (map->pagesneeded !=3D 0 && run_filter(= dmat, curaddr)) { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0sgsize =3D roundup2(sgsize= , dmat->alignment); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0sgsize =3D MIN(sgsize, max= _sgsize); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0curaddr =3D add_bounce_pag= e(dmat, map, vaddr, sgsize); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0} else { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0sgsize =3D MIN(sgsize, max= _sgsize); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0} > > If you have a map that does need bouncing, then it will split up the page= s. > It happens to work for bus_dmamem_alloc() because that returns a NULL map > which doesn't bounce. =A0But if you had a PCI device which supported only > 32-bit addresses on a 64-bit machine with an aligned, 4 page buffer above > 4GB and did a bus_dma_map_load() on that buffer, it would get split up in= to > 4 separate 4 page-aligned pages. > You are right. I assume that you are ok with the patch and the discussion above was an FYI, right? best Neel > -- > John Baldwin > From owner-freebsd-hackers@FreeBSD.ORG Wed Sep 29 08:45:05 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 240A0106566B for ; Wed, 29 Sep 2010 08:45:05 +0000 (UTC) (envelope-from kadupl@wp.pl) Received: from mx4.wp.pl (mx4.wp.pl [212.77.101.8]) by mx1.freebsd.org (Postfix) with ESMTP id A6B808FC1A for ; Wed, 29 Sep 2010 08:45:04 +0000 (UTC) Received: (wp-smtpd smtp.wp.pl 48 invoked from network); 29 Sep 2010 10:18:23 +0200 Received: from out.poczta.wp.pl (HELO localhost) ([212.77.101.240]) (envelope-sender ) by smtp.wp.pl (WP-SMTPD) with SMTP for ; 29 Sep 2010 10:18:23 +0200 Date: Wed, 29 Sep 2010 10:18:22 +0200 From: PL To: freebsd-hackers@freebsd.org Message-ID: <4ca2f64ee66683.99262397@wp.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-2 Content-Transfer-Encoding: 8bit Content-Disposition: inline X-Mailer: Interfejs WWW nowej poczty Wirtualnej Polski X-User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.10) Gecko/20100916 Firefox/3.6.10 Organization: Poznaj Poczte WP http://poczta.wp.pl/info-start.html X-WP-IP: 213.17.239.108 X-WP-AV: skaner antywirusowy poczty Wirtualnej Polski S. A. X-WP-SPAM: NO 0000000 [AQPE] Subject: question regarding link_elf.c X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Sep 2010 08:45:05 -0000 Hi everyone, I'm not quiet sure if it is proper place to ask the question I have. If not, please direct me to the correct place I should post questions like: Im working on some modifications around link_elf.c. According to elf(5) man pages, Elf_Shdr structure contains field called 'sh_addr', containing the address at which first byte of a section shall reside in the memory image. I am particularily interested in '.text' and '.data' sections. After link_elf_load_file() loads the file into a memory, we have linker_file structure filled in, including 'address' field. Now, assuming 'lf' being linker_file_t, already filled in by the loading routine, 'text_sh' being 'Elf_Shdr' for text section, and 'data_sh' being 'Elf_Shdr' for data section: - lf->address + text_sh.sh_addr really points to the beginning of a '.text' section in memory, however.. - lf->address + data_sh.sh_addr does not point to the valid location of '.data' section in memory. Sorry if my question is stupid, but im wondering why it is so ? I guess it has something to do with virtual memory mapping (?). Thanks in advance for your kind help. From owner-freebsd-hackers@FreeBSD.ORG Wed Sep 29 08:58:27 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BE4DF1065672 for ; Wed, 29 Sep 2010 08:58:27 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 0DA5A8FC2D for ; Wed, 29 Sep 2010 08:58:26 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id LAA22107; Wed, 29 Sep 2010 11:58:23 +0300 (EEST) (envelope-from avg@icyb.net.ua) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1P0sUZ-0003FR-6X; Wed, 29 Sep 2010 11:58:23 +0300 Message-ID: <4CA2FFAE.6030906@icyb.net.ua> Date: Wed, 29 Sep 2010 11:58:22 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100918 Lightning/1.0b2 Thunderbird/3.1.4 MIME-Version: 1.0 To: PL References: <4ca2f64ee66683.99262397@wp.pl> In-Reply-To: <4ca2f64ee66683.99262397@wp.pl> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-2 Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org Subject: Re: question regarding link_elf.c X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Sep 2010 08:58:27 -0000 on 29/09/2010 11:18 PL said the following: > Hi everyone, > I'm not quiet sure if it is proper place to ask the question I have. If > not, please > direct me to the correct place I should post questions like: > > Im working on some modifications around link_elf.c. According to elf(5) > man pages, > Elf_Shdr structure contains field called 'sh_addr', containing the > address at > which first byte of a section shall reside in the memory image. I am > particularily > interested in '.text' and '.data' sections. After link_elf_load_file() > loads the > file into a memory, we have linker_file structure filled in, including > 'address' > field. Now, assuming 'lf' being linker_file_t, already filled in by the > loading > routine, 'text_sh' being 'Elf_Shdr' for text section, and 'data_sh' > being 'Elf_Shdr' > for data section: > - lf->address + text_sh.sh_addr really points to the beginning of a > '.text' section > in memory, however.. > > - lf->address + data_sh.sh_addr does not point to the valid location of > '.data' section > in memory. > > Sorry if my question is stupid, but im wondering why it is so ? I guess > it has something > to do with virtual memory mapping (?). Perhaps the reason is simpler, like a bug in your code :-) You can do 'readelf -a' on a module that you load and check attributes of .data section and then compare with data_sh that you get at run-time. -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Wed Sep 29 09:03:56 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B19B8106566B for ; Wed, 29 Sep 2010 09:03:56 +0000 (UTC) (envelope-from kpielorz_lst@tdx.co.uk) Received: from mail.tdx.com (mail.tdx.com [62.13.128.18]) by mx1.freebsd.org (Postfix) with ESMTP id 39A7F8FC1A for ; Wed, 29 Sep 2010 09:03:55 +0000 (UTC) Received: from HexaDeca64.dmpriest.net.uk (HPQuadro64.dmpriest.net.uk [62.13.130.30]) (authenticated bits=0) by mail.tdx.com (8.14.3/8.14.3/Kp) with ESMTP id o8T8qhTK026743 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES256-SHA bits=256 verify=NO) for ; Wed, 29 Sep 2010 09:52:43 +0100 (BST) Date: Wed, 29 Sep 2010 09:52:04 +0100 From: Karl Pielorz To: freebsd-hackers@freebsd.org Message-ID: <46DA79B397A14A614CB60A31@HexaDeca64.dmpriest.net.uk> X-Mailer: Mulberry/4.0.8 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Subject: 8.1-R - Marvell 88SX6081 SATA controller via mvs = lots of errors X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Sep 2010 09:03:56 -0000 Hi, I just switched my 8.1-R/amd64 (dual Opteron) system from ATA over to the new mvs driver, and started seeing a whole bunch of errors (which appear to have hosed one of my zfs volumes during a scrub) - anyone know what the following errors actually mean? The machine has 2 * 88SX6081's in it: " Sep 28 19:58:49 kernel: mvs0: port 0x3000-0x30ff mem 0xd0100000-0xd01fffff,0xd0400000-0xd07fffff irq 24 at device 4.0 on pci17 Sep 28 19:58:49 kernel: mvs0: Gen-II, 8 3Gbps ports, Port Multiplier ... Sep 28 19:58:49 kernel: mvs1: port 0x4000-0x40ff mem 0xd0c00000-0xd0cfffff,0xd0800000-0xd0bfffff irq 28 at device 4.0 on pci18 Sep 28 19:58:49 kernel: mvs1: Gen-II, 8 3Gbps ports, Port Multiplier supported " Under 7.2 they ran fine, with the ATA driver. I use ZFS on this machine - and both pools were scrubbed before the upgrade (and backed up fortunately!). With the mvs driver, during a scrub of the main volume, I see: " Sep 29 08:56:13 kernel: mvsch12: EMPTY CRPB 6 (->14) 1 4000 Sep 29 08:56:13 kernel: mvsch12: EMPTY CRPB 7 (->14) 0 4000 Sep 29 08:56:13 kernel: mvsch12: EMPTY CRPB 8 (->14) 2 4000 " [repeated a lot - interspersed with zfs reporting problems with files, on all the devices in the pool] I then also get a whole bunch of: " Sep 29 08:56:56 kernel: mvsch0: Timeout on slot 1 Sep 29 08:56:56 kernel: mvsch0: iec 02000000 sstat 00000123 serr 00000000 edma_s 00001020 dma_c 00000000 dma_s 00000000 rs 00000006 statu s 40 Sep 29 08:56:56 kernel: mvsch0: ... waiting for slots 00000004 Sep 29 08:56:56 kernel: mvsch12: Timeout on slot 5 Sep 29 08:56:56 kernel: mvsch12: iec 02000000 sstat 00000123 serr 00000000 edma_s 00001121 dma_c 00000000 dma_s 00000000 rs 00000028 stat us 40 " The system has 2 pools (one is 12 disks of mirrored pairs - each side of the mirror is on alternate Marvell's), the other is 1 RAIDZ of 4 disks, 2 are on alternate Marvell's - the other 2 drives are on the motherboards nForce CK804 ports). I scrubbed the second pool yesterday without incident, so this only seemed to happen using drives exclusively on the 88SX6081's (or the I/O system is stressed, running the I/O for all 12 drives through the Marvells, as opposed to just the I/O for 2 drives [1 each] through the Marvells when the other pool is scrubbed). -Karl From owner-freebsd-hackers@FreeBSD.ORG Wed Sep 29 13:26:01 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 79EF1106567A for ; Wed, 29 Sep 2010 13:26:01 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 3337C8FC15 for ; Wed, 29 Sep 2010 13:26:01 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id A2CDF46B06; Wed, 29 Sep 2010 09:26:00 -0400 (EDT) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id A7C908A03C; Wed, 29 Sep 2010 09:25:59 -0400 (EDT) From: John Baldwin To: Neel Natu Date: Wed, 29 Sep 2010 07:47:20 -0400 User-Agent: KMail/1.13.5 (FreeBSD/7.3-CBSD-20100819; KDE/4.4.5; amd64; ; ) References: <201009280936.40203.jhb@freebsd.org> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201009290747.20754.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Wed, 29 Sep 2010 09:25:59 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.6 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: freebsd-hackers@freebsd.org Subject: Re: PATCH: fix bogus error message "bus_dmamem_alloc failed to align memory properly" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Sep 2010 13:26:01 -0000 On Tuesday, September 28, 2010 4:02:08 pm Neel Natu wrote: > Hi John, > > On Tue, Sep 28, 2010 at 6:36 AM, John Baldwin wrote: > > On Monday, September 27, 2010 5:13:03 pm Neel Natu wrote: > >> Hi John, > >> > >> Thanks for reviewing this. > >> > >> On Mon, Sep 27, 2010 at 8:04 AM, John Baldwin wrote: > >> > On Friday, September 24, 2010 9:00:44 pm Neel Natu wrote: > >> >> Hi, > >> >> > >> >> This patch fixes the bogus error message from bus_dmamem_alloc() about > >> >> the buffer not being aligned properly. > >> >> > >> >> The problem is that the check is against a virtual address as opposed > >> >> to the physical address. contigmalloc() makes guarantees about > >> >> the alignment of physical addresses but not the virtual address > >> >> mapping it. > >> >> > >> >> Any objections if I commit this patch? > >> > > >> > Hmmm, I guess you are doing super-page alignment rather than sub-page > >> > alignment? In general I thought the busdma code only handled sub-page > >> > alignment and doesn't fully handle requests for super-page alignment. > >> > > >> > >> Yes, this is for allocations with sizes greater than PAGE_SIZE and > >> alignment requirements also greater than a PAGE_SIZE. > >> > >> > For example, since it insists on walking individual pages at a time, if you > >> > had an alignment setting of 4 pages and passed in a single, aligned 4-page > >> > buffer, bus_dma would actually bounce the last 3 pages so that each individual > >> > page is 4-page aligned. At least, I think that is what would happen. > >> > > >> > >> I think you are referring to bus_dmamap_load() operation that would > >> follow the bus_dmamem_alloc(), right? The memory allocated by > >> bus_dmamem_alloc() does not need to be bounced. In fact, the dmamap > >> pointer returned by bus_dmamem_alloc() is NULL. > >> > >> At least for the amd64 implementation there is code in > >> _bus_dmamap_load_buffer() which will coalesce individual dma segments > >> if they satisfy 'boundary' and 'segsize' constraints. > > > > So the problem is earlier in the routine where it does this: > > > > /* > > * Get the physical address for this segment. > > */ > > if (pmap) > > curaddr = pmap_extract(pmap, vaddr); > > else > > curaddr = pmap_kextract(vaddr); > > > > /* > > * Compute the segment size, and adjust counts. > > */ > > max_sgsize = MIN(buflen, dmat->maxsegsz); > > sgsize = PAGE_SIZE - ((vm_offset_t)curaddr & PAGE_MASK); > > if (map->pagesneeded != 0 && run_filter(dmat, curaddr)) { > > sgsize = roundup2(sgsize, dmat->alignment); > > sgsize = MIN(sgsize, max_sgsize); > > curaddr = add_bounce_page(dmat, map, vaddr, sgsize); > > } else { > > sgsize = MIN(sgsize, max_sgsize); > > } > > > > If you have a map that does need bouncing, then it will split up the pages. > > It happens to work for bus_dmamem_alloc() because that returns a NULL map > > which doesn't bounce. But if you had a PCI device which supported only > > 32-bit addresses on a 64-bit machine with an aligned, 4 page buffer above > > 4GB and did a bus_dma_map_load() on that buffer, it would get split up into > > 4 separate 4 page-aligned pages. > > > > You are right. > > I assume that you are ok with the patch and the discussion above was > an FYI, right? I think the patch is ok, but my point is that super-page alignment isn't really part of the design of the current bus_dma and only works for bus_dmammem_alloc() by accident. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Wed Sep 29 14:13:03 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3BA71106566C for ; Wed, 29 Sep 2010 14:13:03 +0000 (UTC) (envelope-from kadupl@wp.pl) Received: from mx4.wp.pl (mx4.wp.pl [212.77.101.8]) by mx1.freebsd.org (Postfix) with ESMTP id BD3118FC1D for ; Wed, 29 Sep 2010 14:13:02 +0000 (UTC) Received: (wp-smtpd smtp.wp.pl 18729 invoked from network); 29 Sep 2010 16:13:01 +0200 Received: from out.poczta.wp.pl (HELO localhost) ([212.77.101.240]) (envelope-sender ) by smtp.wp.pl (WP-SMTPD) with SMTP for ; 29 Sep 2010 16:13:01 +0200 Date: Wed, 29 Sep 2010 16:13:01 +0200 From: PL To: Andriy Gapon Message-ID: <4ca3496d8ce8b3.74842248@wp.pl> References: <4ca2f64ee66683.99262397@wp.pl> <4CA2FFAE.6030906@icyb.net.ua> In-reply-to: <4ca2f64ee66683.99262397@wp.pl> <4CA2FFAE.6030906@icyb.net.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-2 Content-Transfer-Encoding: 8bit Content-Disposition: inline X-Mailer: Interfejs WWW nowej poczty Wirtualnej Polski X-User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.10) Gecko/20100916 Firefox/3.6.10 Organization: Poznaj Poczte WP http://poczta.wp.pl/info-start.html X-WP-IP: 213.17.239.108 X-WP-AV: skaner antywirusowy poczty Wirtualnej Polski S. A. X-WP-SPAM: NO 0000000 [UTME] Cc: freebsd-hackers Subject: Re: question regarding link_elf.c X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Sep 2010 14:13:03 -0000 Dnia 29-09-2010 o godz. 10:58 Andriy Gapon napisa³(a): > on 29/09/2010 11:18 PL said the following: > > Hi everyone, > > I'm not quiet sure if it is proper place to ask the question I have. If > > not, please > > direct me to the correct place I should post questions like: > > > > Im working on some modifications around link_elf.c. According to elf(5) > > man pages, > > Elf_Shdr structure contains field called 'sh_addr', containing the > > address at > > which first byte of a section shall reside in the memory image. I am > > particularily > > interested in '.text' and '.data' sections. After link_elf_load_file() > > loads the > > file into a memory, we have linker_file structure filled in, including > > 'address' > > field. Now, assuming 'lf' being linker_file_t, already filled in by the > > loading > > routine, 'text_sh' being 'Elf_Shdr' for text section, and 'data_sh' > > being 'Elf_Shdr' > > for data section: > > - lf->address + text_sh.sh_addr really points to the beginning of a > > '.text' section > > in memory, however.. > > > > - lf->address + data_sh.sh_addr does not point to the valid location of > > '.data' section > > in memory. > > > > Sorry if my question is stupid, but im wondering why it is so ? I guess > > it has something > > to do with virtual memory mapping (?). > > Perhaps the reason is simpler, like a bug in your code :-) > You can do 'readelf -a' on a module that you load and check attributes > of .data > section and then compare with data_sh that you get at run-time. > > -- > Andriy Gapon > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" It seems like it is not a problem in my own code, since readelf -S on a elf file gives me the same results as my debug messages. I've created an empty module, to simplify debugging. Both my code, and readelf says, that '.text' section address is 0x3e0, and its size is 7 bytes. Adding 0x3e0 to lf->address points to a valid location. '.data' is supposed to be at 0x1424 (again, both my code and readelf returns the same thing), but the actual data starts at lf->address + 0x3e7. How do I know ? I've added global initialized string variable in empty test module, and Im looking at the memory to determine it's location. I'm not sure what is wrong then. From owner-freebsd-hackers@FreeBSD.ORG Wed Sep 29 14:23:44 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5A7A3106566B for ; Wed, 29 Sep 2010 14:23:44 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id C328D8FC13 for ; Wed, 29 Sep 2010 14:23:41 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id RAA28654; Wed, 29 Sep 2010 17:23:12 +0300 (EEST) (envelope-from avg@icyb.net.ua) Message-ID: <4CA34BCF.6090701@icyb.net.ua> Date: Wed, 29 Sep 2010 17:23:11 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100920 Lightning/1.0b2 Thunderbird/3.1.4 MIME-Version: 1.0 To: PL References: <4ca2f64ee66683.99262397@wp.pl> <4CA2FFAE.6030906@icyb.net.ua> <4ca3496d8ce8b3.74842248@wp.pl> In-Reply-To: <4ca3496d8ce8b3.74842248@wp.pl> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-2 Content-Transfer-Encoding: 7bit Cc: freebsd-hackers Subject: Re: question regarding link_elf.c X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Sep 2010 14:23:44 -0000 on 29/09/2010 17:13 PL said the following: > It seems like it is not a problem in my own code, since readelf -S on a > elf file > gives me the same results as my debug messages. I've created an empty > module, to > simplify debugging. Both my code, and readelf says, that '.text' section > address > is 0x3e0, and its size is 7 bytes. Adding 0x3e0 to lf->address points to > a valid location. > > '.data' is supposed to be at 0x1424 (again, both my code and readelf > returns the same thing), > but the actual data starts at lf->address + 0x3e7. How do I know ? I've > added global > initialized string variable in empty test module, and Im looking at the > memory to determine > it's location. I'm not sure what is wrong then. Can you post a link to the compiled test module? -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Wed Sep 29 14:44:19 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 567F4106566B for ; Wed, 29 Sep 2010 14:44:19 +0000 (UTC) (envelope-from kadupl@wp.pl) Received: from mx4.wp.pl (mx4.wp.pl [212.77.101.8]) by mx1.freebsd.org (Postfix) with ESMTP id D3B198FC14 for ; Wed, 29 Sep 2010 14:44:18 +0000 (UTC) Received: (wp-smtpd smtp.wp.pl 26868 invoked from network); 29 Sep 2010 16:44:17 +0200 Received: from out.poczta.wp.pl (HELO localhost) ([212.77.101.240]) (envelope-sender ) by smtp.wp.pl (WP-SMTPD) with SMTP for ; 29 Sep 2010 16:44:17 +0200 Date: Wed, 29 Sep 2010 16:44:17 +0200 From: PL To: Andriy Gapon Message-ID: <4ca350c1278010.73850128@wp.pl> References: <4ca2f64ee66683.99262397@wp.pl> <4CA2FFAE.6030906@icyb.net.ua> <4ca3496d8ce8b3.74842248@wp.pl> <4CA34BCF.6090701@icyb.net.ua> In-reply-to: <4ca2f64ee66683.99262397@wp.pl> <4CA2FFAE.6030906@icyb.net.ua> <4ca3496d8ce8b3.74842248@wp.pl> <4CA34BCF.6090701@icyb.net.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-2 Content-Transfer-Encoding: 8bit Content-Disposition: inline X-Mailer: Interfejs WWW nowej poczty Wirtualnej Polski X-User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.10) Gecko/20100916 Firefox/3.6.10 Organization: Poznaj Poczte WP http://poczta.wp.pl/info-start.html X-WP-IP: 213.17.239.108 X-WP-AV: skaner antywirusowy poczty Wirtualnej Polski S. A. X-WP-SPAM: NO 0000000 [cXMk] Cc: freebsd-hackers Subject: Re: question regarding link_elf.c X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Sep 2010 14:44:19 -0000 Dnia 29-09-2010 o godz. 16:23 Andriy Gapon napisa³(a): > on 29/09/2010 17:13 PL said the following: > > It seems like it is not a problem in my own code, since readelf -S on a > > elf file > > gives me the same results as my debug messages. I've created an empty > > module, to > > simplify debugging. Both my code, and readelf says, that '.text' section > > address > > is 0x3e0, and its size is 7 bytes. Adding 0x3e0 to lf->address points to > > a valid location. > > > > '.data' is supposed to be at 0x1424 (again, both my code and readelf > > returns the same thing), > > but the actual data starts at lf->address + 0x3e7. How do I know ? I've > > added global > > initialized string variable in empty test module, and Im looking at the > > memory to determine > > it's location. I'm not sure what is wrong then. > > > Can you post a link to the compiled test module? > > -- > Andriy Gapon Well.. i don't have any public 'hosting', but I put it in here: http://www.4shared.com/dir/LHn_I393/sharing.html Also, the code is as simple as: 1 #include 2 #include 3 #include 4 5 6 char *str = "THIS IS A STR MARKING DATA START"; 7 8 static int kms_null_handler(module_t m, int op, void *data) 9 { 10 return (0); 11 } 12 13 static moduledata_t kms_null_data = { 14 "kms_null", 15 kms_null_handler, 16 NULL, 17 }; 18 19 DECLARE_MODULE(kms_null, kms_null_data, SI_SUB_EXEC, SI_ORDER_ANY); It is being compiled on i386/GENERIC kernel. From owner-freebsd-hackers@FreeBSD.ORG Wed Sep 29 14:59:01 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8B7A0106566B for ; Wed, 29 Sep 2010 14:59:01 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id D5B7B8FC08 for ; Wed, 29 Sep 2010 14:59:00 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id RAA29235; Wed, 29 Sep 2010 17:58:40 +0300 (EEST) (envelope-from avg@icyb.net.ua) Message-ID: <4CA35420.1030108@icyb.net.ua> Date: Wed, 29 Sep 2010 17:58:40 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100920 Lightning/1.0b2 Thunderbird/3.1.4 MIME-Version: 1.0 To: PL References: <4ca2f64ee66683.99262397@wp.pl> <4CA2FFAE.6030906@icyb.net.ua> <4ca3496d8ce8b3.74842248@wp.pl> <4CA34BCF.6090701@icyb.net.ua> <4ca350c1278010.73850128@wp.pl> In-Reply-To: <4ca350c1278010.73850128@wp.pl> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-2 Content-Transfer-Encoding: 8bit Cc: freebsd-hackers Subject: Re: question regarding link_elf.c X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Sep 2010 14:59:01 -0000 on 29/09/2010 17:44 PL said the following: > Dnia 29-09-2010 o godz. 16:23 Andriy Gapon napisa³(a): >> on 29/09/2010 17:13 PL said the following: >>> It seems like it is not a problem in my own code, since readelf -S on a >>> elf file >>> gives me the same results as my debug messages. I've created an empty >>> module, to >>> simplify debugging. Both my code, and readelf says, that '.text' section >>> address >>> is 0x3e0, and its size is 7 bytes. Adding 0x3e0 to lf->address points to >>> a valid location. >>> >>> '.data' is supposed to be at 0x1424 (again, both my code and readelf >>> returns the same thing), >>> but the actual data starts at lf->address + 0x3e7. How do I know ? I've >>> added global >>> initialized string variable in empty test module, and Im looking at the >>> memory to determine >>> it's location. I'm not sure what is wrong then. >> >> >> Can you post a link to the compiled test module? >> >> -- >> Andriy Gapon > > Well.. i don't have any public 'hosting', but I put it in here: > > http://www.4shared.com/dir/LHn_I393/sharing.html > > Also, the code is as simple as: > > 1 #include > 2 #include > 3 #include > 4 > 5 > 6 char *str = "THIS IS A STR MARKING DATA START"; You marker was put into .rodata section. Try char str[] instead. > 8 static int kms_null_handler(module_t m, int op, void *data) > 9 { > 10 return (0); > 11 } > 12 > 13 static moduledata_t kms_null_data = { > 14 "kms_null", > 15 kms_null_handler, > 16 NULL, > 17 }; > 18 > 19 DECLARE_MODULE(kms_null, kms_null_data, SI_SUB_EXEC, SI_ORDER_ANY); > > It is being compiled on i386/GENERIC kernel. > > -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Wed Sep 29 15:03:54 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 939CF1065675 for ; Wed, 29 Sep 2010 15:03:54 +0000 (UTC) (envelope-from kadupl@wp.pl) Received: from mx4.wp.pl (mx4.wp.pl [212.77.101.8]) by mx1.freebsd.org (Postfix) with ESMTP id 1D2B38FC08 for ; Wed, 29 Sep 2010 15:03:53 +0000 (UTC) Received: (wp-smtpd smtp.wp.pl 1655 invoked from network); 29 Sep 2010 17:03:53 +0200 Received: from out.poczta.wp.pl (HELO localhost) ([212.77.101.240]) (envelope-sender ) by smtp.wp.pl (WP-SMTPD) with SMTP for ; 29 Sep 2010 17:03:53 +0200 Date: Wed, 29 Sep 2010 17:03:53 +0200 From: PL To: Andriy Gapon Message-ID: <4ca3555906ee81.72544407@wp.pl> References: <4ca2f64ee66683.99262397@wp.pl> <4CA2FFAE.6030906@icyb.net.ua><4ca3496d8ce8b3.74842248@wp.pl> <4CA34BCF.6090701@icyb.net.ua><4ca350c1278010.73850128@wp.pl> <4CA35420.1030108@icyb.net.ua> In-reply-to: <4ca2f64ee66683.99262397@wp.pl> <4CA2FFAE.6030906@icyb.net.ua><4ca3496d8ce8b3.74842248@wp.pl> <4CA34BCF.6090701@icyb.net.ua><4ca350c1278010.73850128@wp.pl> <4CA35420.1030108@icyb.net.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-2 Content-Transfer-Encoding: 8bit Content-Disposition: inline X-Mailer: Interfejs WWW nowej poczty Wirtualnej Polski X-User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.10) Gecko/20100916 Firefox/3.6.10 Organization: Poznaj Poczte WP http://poczta.wp.pl/info-start.html X-WP-IP: 213.17.239.108 X-WP-AV: skaner antywirusowy poczty Wirtualnej Polski S. A. X-WP-SPAM: NO 0000000 [YeMk] Cc: freebsd-hackers Subject: Re: question regarding link_elf.c X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Sep 2010 15:03:54 -0000 Dnia 29-09-2010 o godz. 16:58 Andriy Gapon napisa³(a): > on 29/09/2010 17:44 PL said the following: > > Dnia 29-09-2010 o godz. 16:23 Andriy Gapon napisa³(a): > >> on 29/09/2010 17:13 PL said the following: > >>> It seems like it is not a problem in my own code, since readelf -S on a > >>> elf file > >>> gives me the same results as my debug messages. I've created an empty > >>> module, to > >>> simplify debugging. Both my code, and readelf says, that '.text' section > >>> address > >>> is 0x3e0, and its size is 7 bytes. Adding 0x3e0 to lf->address points to > >>> a valid location. > >>> > >>> '.data' is supposed to be at 0x1424 (again, both my code and readelf > >>> returns the same thing), > >>> but the actual data starts at lf->address + 0x3e7. How do I know ? I've > >>> added global > >>> initialized string variable in empty test module, and Im looking at the > >>> memory to determine > >>> it's location. I'm not sure what is wrong then. > >> > >> > >> Can you post a link to the compiled test module? > >> > >> -- > >> Andriy Gapon > > > > Well.. i don't have any public 'hosting', but I put it in here: > > > > http://www.4shared.com/dir/LHn_I393/sharing.html > > > > Also, the code is as simple as: > > > > 1 #include > > 2 #include > > 3 #include > > 4 > > 5 > > 6 char *str = "THIS IS A STR MARKING DATA START"; > > You marker was put into .rodata section. > Try char str[] instead. > > > 8 static int kms_null_handler(module_t m, int op, void *data) > > 9 { > > 10 return (0); > > 11 } > > 12 > > 13 static moduledata_t kms_null_data = { > > 14 "kms_null", > > 15 kms_null_handler, > > 16 NULL, > > 17 }; > > 18 > > 19 DECLARE_MODULE(kms_null, kms_null_data, SI_SUB_EXEC, SI_ORDER_ANY); > > > > It is being compiled on i386/GENERIC kernel. > > > > > > > -- > Andriy Gapon > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" THANKS ! How could I miss that ? :) From owner-freebsd-hackers@FreeBSD.ORG Wed Sep 29 16:03:03 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8B936106564A for ; Wed, 29 Sep 2010 16:03:03 +0000 (UTC) (envelope-from crossd@cs.rpi.edu) Received: from cliffclavin.cs.rpi.edu (cliffclavin.cs.rpi.edu [128.113.126.25]) by mx1.freebsd.org (Postfix) with ESMTP id 38AF68FC17 for ; Wed, 29 Sep 2010 16:03:02 +0000 (UTC) X-Hash: SCtCte|9a8bd3d2a61412997e2c6491aa4e35c5011afd78|726e88d60504330b352482fcd659de87 X-Countries: United States X-SMTP-From: accepted static-98-113-167-42.nycmny.fios.verizon.net [98.113.167.42] (dcross2.office.okcupid.com) {United States} DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=cs.rpi.edu; h= message-id:date:from:mime-version:to:subject:content-type :content-transfer-encoding; s=default; i=crossd@cs.rpi.edu; t= 1285774885; x=1286379685; l=1587; bh=GiUWohVUMt5GXUUIAqcOMOIkHyU =; b=lTZRt453joYSVhpA2njzkzbHelgBiBFHcSG7+ldWe6ORNwNFTHhJe7SMqUR 4HMR3jyic31mIu3DWPHXzUdqtyw031+gOWkHGpCvhmouahNVejm2l/5bzx9dODCj k6sf609EB1Aztw48Ow1YZ8Uk3wOweSPbyWCz2NHrFoQt3ZP8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=cs.rpi.edu; h=message-id :date:from:mime-version:to:subject:content-type :content-transfer-encoding; q=dns; s=default; b=kyZXH/vH+zbLKYi0 xlMoWbLRqwU4fil0laa0Ltbbb02ilkJ6pZx+j3gYiqdETZBWpPJ/f54t9zAD4iFK vcnRThzdKHDMVZbzZlRqfOk5KSMttHqf4CfQ/S+Suundl4T/8qk/Umzszc9oK8jY JBDWylV0d9uT1GpQMRPlHO+qcgs= X-Spam-Score: -2.6 () ALL_TRUSTED,AWL,BAYES_00 X-Spam-Report: Spam Report from cliffclavin.cs.rpi.edu (SA:3.2.5): -1.8 ALL_TRUSTED Passed through trusted hosts only via SMTP -0.9 BAYES_00 BODY: Bayesian spam probability is 0 to 1% [score: 0.0000] 0.1 AWL AWL: From: address is in the auto white-list X-Spam-Info: -2.6; ALL_TRUSTED,AWL,BAYES_00 X-Spam-Scanned-By: cliffclavin.cs.rpi.edu using SpamAssassin 3.2.5 (hard limit 15) Authentication-Results: cliffclavin.cs.rpi.edu; DKIM=neutral (none) header.from=crossd@cs.rpi.edu; SPF=neutral (mfrom; Mechanism '?all' matched) smtp.mail=crossd@cs.rpi.edu X-Auth-Passed: cliffclavin.cs.rpi.edu:o8TFfLWH034003 Auth:crossd X-Virus-Scanned-By: cliffclavin.cs.rpi.edu Received: from dcross2.office.okcupid.com (static-98-113-167-42.nycmny.fios.verizon.net [98.113.167.42]) (authenticated bits=0) by cliffclavin.cs.rpi.edu (8.14.3/8.14.3) with ESMTP id o8TFfLWH034003 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Wed, 29 Sep 2010 11:41:22 -0400 (EDT) (envelope-from crossd@cs.rpi.edu) Message-ID: <4CA35DD5.7070900@cs.rpi.edu> Date: Wed, 29 Sep 2010 11:40:05 -0400 From: David Cross User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.8) Gecko/20100329 Thunderbird/3.0.3 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 128.113.126.25 Subject: Unnecessary reads on write load X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Sep 2010 16:03:03 -0000 Tracking down a performance issue where the system apparently needlessly reads on a 100% write load... consider the following C test case: (more after the code) #include #include #include #include int main(int argc, char **argv) { unsigned char dir1[4], dir2[4], filename[15], pathname[1024], buffer[130944]; unsigned int filenum, count, filesize; int fd; arc4random_buf(buffer, 131072); count=atoi(argv[1]); for(;count>0;count--) { filenum=arc4random(); filesize=arc4random_uniform(130944) + 128; sprintf(filename, "%08x", filenum); strncpy(dir1,filename,3); strncpy(dir2,filename+3, 3); dir1[3]=dir2[3]=0; sprintf(pathname, "%s/%s/%s", dir1, dir2, filename); fd=open(pathname, O_CREAT | O_WRONLY, 0644); if (fd < 0) { sprintf(pathname, "%s/%s", dir1, dir2); if (mkdir(pathname, 0755) < 0) { mkdir(dir1, 0755); mkdir(pathname, 0755); }; sprintf(pathname, "%s/%s/%s", dir1, dir2, filename); fd=open(pathname, O_CREAT | O_WRONLY, 0644); } if (fd <0) continue; write(fd,buffer, filesize); close(fd); } return 0; } In running that in an empty directory (it takes one argument, the number of files to create), I see that it spends most of its time in BIORD?!. If I have a debugging kernel I can see that its all in NAMI cache misses, and doing block fetches... but "why?" the only directories that exist are ones that it just created, and should therefore be in the cache, right? Any ideas? Give it a try yourself and tell me what you get. -- David E. Cross From owner-freebsd-hackers@FreeBSD.ORG Wed Sep 29 19:40:59 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3248D106566C for ; Wed, 29 Sep 2010 19:40:59 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id F1AD08FC16 for ; Wed, 29 Sep 2010 19:40:58 +0000 (UTC) Received: by iwn34 with SMTP id 34so1751724iwn.13 for ; Wed, 29 Sep 2010 12:40:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=UDaFUmNsTFMGJzc5Ssz39fuXHT5g3w9QwHN9ulTruy0=; b=A3Rf8pLpJHvBlcRWMRs3XI9iU7Q5cUOYGGKpQlfENqH8+XApcMogr6WZBxGulfuJVO wu0FASSWFwhMZMhTVDC6KL8NRpJbyWzFvQKqQMSvBF0p9SzVFipspik6Xog73BuyGo7h 856JFJArqSKpWw+lRJO8V2f5Z2lCVUJ5hWwqU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=VN+lAWgPDy0MLXHhFePN8j02BqUnp6XjILxMUCM58sAyKqNkgGEBxvje1LGUbuLnXb wcX9/MEzMYrJVuLe39wG6iaeRGfXIfxwLa2Fgsm/nTo97JveDGcSQSTxOoOBYAeXKEV8 d/PZA1TN8Cs3pzNh+6m6bYHhMyjV1wqxdNxmk= MIME-Version: 1.0 Received: by 10.231.34.139 with SMTP id l11mr2265643ibd.141.1285789257249; Wed, 29 Sep 2010 12:40:57 -0700 (PDT) Received: by 10.231.167.140 with HTTP; Wed, 29 Sep 2010 12:40:57 -0700 (PDT) Date: Wed, 29 Sep 2010 12:40:57 -0700 Message-ID: From: Matthew Fleming To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: Adding a V=R mapping for amd64? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Sep 2010 19:40:59 -0000 I'm hacking around with making a "fast reboot" that puts a copy of the MBR from disk into address 0x7c00 and, after disabling various translation bits and stopping other CPUs, branches to it, to skip the hardware self test that normally happens on boot. I haven't gotten to the point of attempting to run the code at 0x7c00 because I'm first hitting a different error. Despite my attempts to enter a translation into the hardware page table, I get a panic trying to write to address 0x7000, where I intended to put the trampoline code that turns off translation. Rebooting... Attempt to reset to MBR... XXX attempting pmap_kenter()... XXX copying bootstrap code... panic @ time 1285760103.939, thread 0xffffff000775d960: Fatal trap 12: page fault while in kernel mode cpuid = 0 Panic occurred in module kernel loaded at 0xffffffff80100000: Stack: -------------------------------------------------- kernel:trap_fatal+0xac kernel:trap_pfault+0x24c kernel:trap+0x42e kernel:bcopy+0x16 kernel:shutdown_reset+0x48 kernel:boot+0x317 kernel:reboot+0x60 kernel:ia32_syscall+0x1cd -------------------------------------------------- cpuid = 0; apic id = 00 fault virtual address = 0x7000 fault code = supervisor write data, page not present stack pointer = 0x10:0xffffff8059e07670 frame pointer = 0x10:0xffffff8059e07780 Here's what I think is the relevant snippets of code. Note that I reserved the vm_page_t for physical page 7 as mbr_page early in boot, so I know the memory is free. void pmap_kenter_VR(vm_paddr_t pa) { pmap_t pmap = kernel_pmap; vm_page_t mpte; pd_entry_t *pde; pt_entry_t *pte; vm_page_lock_queues(); PMAP_LOCK(pmap); mpte = pmap_allocpte(pmap, pa, M_WAITOK); pde = pmap_pde(pmap, pa); if (pde == NULL || (*pde & PG_V) == 0) panic("%s: invalid page directory va=%#lx", __func__, pa); if ((*pde & PG_PS) != 0) panic("%s: attempted pmap_enter on 2MB page", __func__); pte = pmap_pde_to_pte(pde, pa); if (pte == NULL) panic("%s: no pte va=%#lx", __func__, pa); if (*pte != 0) { /* Remove extra pte reference. */ mpte->wire_count--; } pte_store(pte, pa | PG_RW | PG_V | PG_G | pg_nx); vm_page_unlock_queues(); PMAP_UNLOCK(pmap); } Then in cpu_reset(): /* * Establish a V=R mapping for the MBR page, and copy a * reasonable guess at the size of the bootstrap code into the * beginning of the page. */ printf("XXX attempting pmap_kenter()...\n"); pmap_kenter_VR(trunc_page(mbaddr)); printf("XXX copying bootstrap code...\n"); to_copy = (uintptr_t)xxx_reset_end - (uintptr_t)xxx_reset_real; if (to_copy > mbaddr - trunc_page(mbaddr)) to_copy = mbaddr - trunc_page(mbaddr); bcopy(xxx_reset_real, (void *)trunc_page(mbaddr), to_copy); /* die here */ printf("XXX attempting to turn off xlation and re-run MBR...\n"); xxx_reset_real(mbaddr); My first attempt was a call to pmap_kenter(trunc_page(0x7c00), trunc_page(0x7c00)); which failed trying to dereference the non-existent PDE. My second attempt called pmap_enter(kernel_pmap, trunc_page(0x7c00), VM_PROT_WRITE, mbr_page, VM_PROT_ALL, 0); That failed with the same crash as the attempt using pmap_kenter_VR(). So... any thoughts as to why, after an apparently successful installation of an xlation, I still get a panic as though there were no xlation? Thanks, matthew From owner-freebsd-hackers@FreeBSD.ORG Wed Sep 29 20:12:26 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E03771065670 for ; Wed, 29 Sep 2010 20:12:26 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 794748FC13 for ; Wed, 29 Sep 2010 20:12:25 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id o8TKCLwx060454 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 29 Sep 2010 23:12:21 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id o8TKCLvo025918; Wed, 29 Sep 2010 23:12:21 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id o8TKCLlc025917; Wed, 29 Sep 2010 23:12:21 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 29 Sep 2010 23:12:21 +0300 From: Kostik Belousov To: Matthew Fleming Message-ID: <20100929201221.GG43070@deviant.kiev.zoral.com.ua> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="rPFbv4B2w2tcwGo5" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-2.5 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_20, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-hackers@freebsd.org Subject: Re: Adding a V=R mapping for amd64? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Sep 2010 20:12:27 -0000 --rPFbv4B2w2tcwGo5 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Sep 29, 2010 at 12:40:57PM -0700, Matthew Fleming wrote: > I'm hacking around with making a "fast reboot" that puts a copy of the > MBR from disk into address 0x7c00 and, after disabling various > translation bits and stopping other CPUs, branches to it, to skip the > hardware self test that normally happens on boot. >=20 > I haven't gotten to the point of attempting to run the code at 0x7c00 > because I'm first hitting a different error. Despite my attempts to > enter a translation into the hardware page table, I get a panic trying > to write to address 0x7000, where I intended to put the trampoline > code that turns off translation. >=20 > Rebooting... > Attempt to reset to MBR... > XXX attempting pmap_kenter()... > XXX copying bootstrap code... > panic @ time 1285760103.939, thread 0xffffff000775d960: Fatal trap 12: > page fault while in kernel mode >=20 > cpuid =3D 0 > Panic occurred in module kernel loaded at 0xffffffff80100000: >=20 > Stack: -------------------------------------------------- > kernel:trap_fatal+0xac > kernel:trap_pfault+0x24c > kernel:trap+0x42e > kernel:bcopy+0x16 > kernel:shutdown_reset+0x48 > kernel:boot+0x317 > kernel:reboot+0x60 > kernel:ia32_syscall+0x1cd > -------------------------------------------------- > cpuid =3D 0; apic id =3D 00 > fault virtual address =3D 0x7000 > fault code =3D supervisor write data, page not present > stack pointer =3D 0x10:0xffffff8059e07670 > frame pointer =3D 0x10:0xffffff8059e07780 >=20 > Here's what I think is the relevant snippets of code. Note that I > reserved the vm_page_t for physical page 7 as mbr_page early in boot, > so I know the memory is free. >=20 > void > pmap_kenter_VR(vm_paddr_t pa) > { > pmap_t pmap =3D kernel_pmap; > vm_page_t mpte; > pd_entry_t *pde; > pt_entry_t *pte; >=20 > vm_page_lock_queues(); > PMAP_LOCK(pmap); > mpte =3D pmap_allocpte(pmap, pa, M_WAITOK); >=20 > pde =3D pmap_pde(pmap, pa); > if (pde =3D=3D NULL || (*pde & PG_V) =3D=3D 0) > panic("%s: invalid page directory va=3D%#lx", __func__, pa); > if ((*pde & PG_PS) !=3D 0) > panic("%s: attempted pmap_enter on 2MB page", __func__); > pte =3D pmap_pde_to_pte(pde, pa); > if (pte =3D=3D NULL) > panic("%s: no pte va=3D%#lx", __func__, pa); >=20 > if (*pte !=3D 0) { > /* Remove extra pte reference. */ > mpte->wire_count--; > } > pte_store(pte, pa | PG_RW | PG_V | PG_G | pg_nx); > =09 > vm_page_unlock_queues(); > PMAP_UNLOCK(pmap); > } >=20 > Then in cpu_reset(): >=20 > /* > * Establish a V=3DR mapping for the MBR page, and copy a > * reasonable guess at the size of the bootstrap code into the > * beginning of the page. > */ > printf("XXX attempting pmap_kenter()...\n"); > pmap_kenter_VR(trunc_page(mbaddr)); > printf("XXX copying bootstrap code...\n"); > to_copy =3D (uintptr_t)xxx_reset_end - (uintptr_t)xxx_reset_real; > if (to_copy > mbaddr - trunc_page(mbaddr)) > to_copy =3D mbaddr - trunc_page(mbaddr); > bcopy(xxx_reset_real, (void *)trunc_page(mbaddr), to_copy); /* die here= */ > printf("XXX attempting to turn off xlation and re-run MBR...\n"); > xxx_reset_real(mbaddr); >=20 >=20 > My first attempt was a call to > pmap_kenter(trunc_page(0x7c00), trunc_page(0x7c00)); > which failed trying to dereference the non-existent PDE. >=20 > My second attempt called > pmap_enter(kernel_pmap, trunc_page(0x7c00), VM_PROT_WRITE, mbr_page, > VM_PROT_ALL, 0); > That failed with the same crash as the attempt using pmap_kenter_VR(). >=20 > So... any thoughts as to why, after an apparently successful > installation of an xlation, I still get a panic as though there were > no xlation? Weird formatting of backtrace. Is this some proprietary code ? Why do you try to create 1-1 mapping at all ? The MBR code should be executing in real mode anyway, and the mapping address at the moment of bcopy does not matter at all. I think that the use of PHYS_TO_DMAP() should give you direct mapping. About the #pf that you see. I think that this is due to the fact that you are modifying kernel pmap, while the active one is the pmap of the user process which context issued reboot(). --rPFbv4B2w2tcwGo5 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (FreeBSD) iEYEARECAAYFAkyjnaUACgkQC3+MBN1Mb4gTmQCcCEQFXgRtClu9NDwO6wA8KpoV 37sAoN5LaB1RquPFkER/F4KGmIza+GfE =YNMd -----END PGP SIGNATURE----- --rPFbv4B2w2tcwGo5-- From owner-freebsd-hackers@FreeBSD.ORG Wed Sep 29 20:22:58 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0AE6D1065674 for ; Wed, 29 Sep 2010 20:22:58 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id B272B8FC20 for ; Wed, 29 Sep 2010 20:22:57 +0000 (UTC) Received: by iwn34 with SMTP id 34so1803391iwn.13 for ; Wed, 29 Sep 2010 13:22:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=JuNJBLl9wEtWR/egGUvacerpAz5GzxbQ+Il6D88LPqI=; b=bdrXEGKzt4ECcCt7VwrHnUgf0OsmtJCOSxjCGAN+UDDYN0bCmwPLM7BLDkpTFtOZiR jiTPfESR1cDWXQnCkBk2IccigdqHGNkOkp6RpDFHm8WAOPnfud/93qlqKfcUUMJbiFYp D2IZ6U/Q6LL2yvXXnVK1lGMVWmKwK0q0xJ3VE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=o70IQ/vc3kO6pL+hLKAuZDH/9SJ6bacrYNsOXe1PKfzyFTIl+OrjsEfS9fLxMMI44D NJ4oQ2RsjOsEnBkm9118TnGotG6FPrfycH+ynUFR90TISREdACneN2y/geKL30cYQtJn MgYx/j2dNQhHC8KVBDobY+a7KQGD09W4dNiKo= MIME-Version: 1.0 Received: by 10.231.167.130 with SMTP id q2mr2330940iby.163.1285791776766; Wed, 29 Sep 2010 13:22:56 -0700 (PDT) Received: by 10.231.167.140 with HTTP; Wed, 29 Sep 2010 13:22:56 -0700 (PDT) In-Reply-To: <20100929201221.GG43070@deviant.kiev.zoral.com.ua> References: <20100929201221.GG43070@deviant.kiev.zoral.com.ua> Date: Wed, 29 Sep 2010 13:22:56 -0700 Message-ID: From: Matthew Fleming To: Kostik Belousov Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: Adding a V=R mapping for amd64? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Sep 2010 20:22:58 -0000 On Wed, Sep 29, 2010 at 1:12 PM, Kostik Belousov wrot= e: > On Wed, Sep 29, 2010 at 12:40:57PM -0700, Matthew Fleming wrote: >> I'm hacking around with making a "fast reboot" that puts a copy of the >> MBR from disk into address 0x7c00 and, after disabling various >> translation bits and stopping other CPUs, branches to it, to skip the >> hardware self test that normally happens on boot. >> >> I haven't gotten to the point of attempting to run the code at 0x7c00 >> because I'm first hitting a different error. =A0Despite my attempts to >> enter a translation into the hardware page table, I get a panic trying >> to write to address 0x7000, where I intended to put the trampoline >> code that turns off translation. >> >> Rebooting... >> Attempt to reset to MBR... >> XXX attempting pmap_kenter()... >> XXX copying bootstrap code... >> panic @ time 1285760103.939, thread 0xffffff000775d960: Fatal trap 12: >> page fault while in kernel mode >> >> cpuid =3D 0 >> Panic occurred in module kernel loaded at 0xffffffff80100000: >> >> Stack: -------------------------------------------------- >> kernel:trap_fatal+0xac >> kernel:trap_pfault+0x24c >> kernel:trap+0x42e >> kernel:bcopy+0x16 >> kernel:shutdown_reset+0x48 >> kernel:boot+0x317 >> kernel:reboot+0x60 >> kernel:ia32_syscall+0x1cd >> -------------------------------------------------- >> cpuid =3D 0; apic id =3D 00 >> fault virtual address =A0 =3D 0x7000 >> fault code =A0 =A0 =A0 =A0 =A0 =A0 =A0=3D supervisor write data, page no= t present >> stack pointer =A0 =A0 =A0 =A0 =A0 =3D 0x10:0xffffff8059e07670 >> frame pointer =A0 =A0 =A0 =A0 =A0 =3D 0x10:0xffffff8059e07780 >> >> Here's what I think is the relevant snippets of code. =A0Note that I >> reserved the vm_page_t for physical page 7 as mbr_page early in boot, >> so I know the memory is free. >> >> void >> pmap_kenter_VR(vm_paddr_t pa) >> { >> =A0 =A0 =A0 pmap_t pmap =3D kernel_pmap; >> =A0 =A0 =A0 vm_page_t mpte; >> =A0 =A0 =A0 pd_entry_t *pde; >> =A0 =A0 =A0 pt_entry_t *pte; >> >> =A0 =A0 =A0 vm_page_lock_queues(); >> =A0 =A0 =A0 PMAP_LOCK(pmap); >> =A0 =A0 =A0 mpte =3D pmap_allocpte(pmap, pa, M_WAITOK); >> >> =A0 =A0 =A0 pde =3D pmap_pde(pmap, pa); >> =A0 =A0 =A0 if (pde =3D=3D NULL || (*pde & PG_V) =3D=3D 0) >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 panic("%s: invalid page directory va=3D%#lx"= , __func__, pa); >> =A0 =A0 =A0 if ((*pde & PG_PS) !=3D 0) >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 panic("%s: attempted pmap_enter on 2MB page"= , __func__); >> =A0 =A0 =A0 pte =3D pmap_pde_to_pte(pde, pa); >> =A0 =A0 =A0 if (pte =3D=3D NULL) >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 panic("%s: no pte va=3D%#lx", __func__, pa); >> >> =A0 =A0 =A0 if (*pte !=3D 0) { >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 /* Remove extra pte reference. */ >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 mpte->wire_count--; >> =A0 =A0 =A0 } >> =A0 =A0 =A0 =A0 pte_store(pte, pa | PG_RW | PG_V | PG_G | pg_nx); >> >> =A0 =A0 =A0 vm_page_unlock_queues(); >> =A0 =A0 =A0 PMAP_UNLOCK(pmap); >> } >> >> Then in cpu_reset(): >> >> =A0 =A0 =A0 /* >> =A0 =A0 =A0 =A0* Establish a V=3DR mapping for the MBR page, and copy a >> =A0 =A0 =A0 =A0* reasonable guess at the size of the bootstrap code into= the >> =A0 =A0 =A0 =A0* beginning of the page. >> =A0 =A0 =A0 =A0*/ >> =A0 =A0 =A0 printf("XXX attempting pmap_kenter()...\n"); >> =A0 =A0 =A0 pmap_kenter_VR(trunc_page(mbaddr)); >> =A0 =A0 =A0 printf("XXX copying bootstrap code...\n"); >> =A0 =A0 =A0 to_copy =3D (uintptr_t)xxx_reset_end - (uintptr_t)xxx_reset_= real; >> =A0 =A0 =A0 if (to_copy > mbaddr - trunc_page(mbaddr)) >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 to_copy =3D mbaddr - trunc_page(mbaddr); >> =A0 =A0 =A0 bcopy(xxx_reset_real, (void *)trunc_page(mbaddr), to_copy); = =A0/* die here */ >> =A0 =A0 =A0 printf("XXX attempting to turn off xlation and re-run MBR...= \n"); >> =A0 =A0 =A0 xxx_reset_real(mbaddr); >> >> >> My first attempt was a call to >> =A0 =A0 =A0 pmap_kenter(trunc_page(0x7c00), trunc_page(0x7c00)); >> which failed trying to dereference the non-existent PDE. >> >> My second attempt called >> =A0 =A0 =A0 pmap_enter(kernel_pmap, trunc_page(0x7c00), VM_PROT_WRITE, m= br_page, >> =A0 =A0 =A0 =A0 =A0 VM_PROT_ALL, 0); >> That failed with the same crash as the attempt using pmap_kenter_VR(). >> >> So... any thoughts as to why, after an apparently successful >> installation of an xlation, I still get a panic as though there were >> no xlation? > > Weird formatting of backtrace. Is this some proprietary code ? Yeah, there's some Isilon local modifications. > Why do you try to create 1-1 mapping at all ? The MBR code should be > executing in real mode anyway, and the mapping address at the moment of > bcopy does not matter at all. I think that the use of PHYS_TO_DMAP() > should give you direct mapping. I assumed I need trampoline code. The moment I turn off the xlation bits, if the instruction pointer I'm running from is the normal kernel addresses, won't I die horribly trying to access 0xffffffff80000000 or so in real mode? > About the #pf that you see. I think that this is due to the fact that > you are modifying kernel pmap, while the active one is the pmap of > the user process which context issued reboot(). Isn't the kernel pmap active in the kernel, since I've entered via the syscall reboot(2) ? Thanks, matthew From owner-freebsd-hackers@FreeBSD.ORG Wed Sep 29 21:41:42 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6D346106566B for ; Wed, 29 Sep 2010 21:41:42 +0000 (UTC) (envelope-from kc5vdj.freebsd@gmail.com) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id 23A848FC16 for ; Wed, 29 Sep 2010 21:41:41 +0000 (UTC) Received: by ywt2 with SMTP id 2so522470ywt.13 for ; Wed, 29 Sep 2010 14:41:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from :user-agent:mime-version:to:cc:subject:references:in-reply-to :content-type:content-transfer-encoding; bh=b1GuFmABF1Mw4cVTQAGRjboCV+uiDUJqOyUymJeEpmk=; b=uDsnxBgUibymzAXDTp4CoV/rrVqVaFm8UX8ecjX8YWeiZDfXfBMOYAm6Co1B8yOJz7 aYPHjB28JCHz0H7oLNqjzl+BrVR7OD/+juJEyfhRjX3cAkTG/PwwReiKMjBpD1qVpkC5 2ddmfxbfzKfXZ6XzprsUg93ou+viSqQLm9Vdo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; b=KOxWzM2cjEBybX8K0mlb3RvumRhiZWds6MXaP/8JQZ+PyiIBwZ1m9WGljANE6rXzvd ovT63bDgidIgERWorV3ujYh906CI0NoHP0i7eLMaz5A0dM6Rf/uaK+XniwUkqNfZoX/Z nEmX4NKzVySjKWqS5yyLeg7/YFks6bDnGW+RM= Received: by 10.231.149.140 with SMTP id t12mr2460476ibv.100.1285796500517; Wed, 29 Sep 2010 14:41:40 -0700 (PDT) Received: from argus.electron-tube.net (desm-47-213.dsl.netins.net [167.142.47.213]) by mx.google.com with ESMTPS id h8sm9359591ibk.9.2010.09.29.14.41.38 (version=TLSv1/SSLv3 cipher=RC4-MD5); Wed, 29 Sep 2010 14:41:39 -0700 (PDT) Message-ID: <4C9AA0F6.6040509@gmail.com> Date: Wed, 22 Sep 2010 19:36:06 -0500 From: Jim Bryant User-Agent: Thunderbird 2.0.0.24 (X11/20100911) MIME-Version: 1.0 To: Atom Smasher References: <1009110004520.2000@smasher> In-Reply-To: <1009110004520.2000@smasher> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, Ivan Voras Subject: Re: How to disallow logout X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Sep 2010 21:41:42 -0000 Atom Smasher wrote: > On Fri, 10 Sep 2010, Ivan Voras wrote: > >> 1) power outage of the server >> 2) power outage on the client >> 3) network problems (ssh or TCP connection drop) >> 4) administrative command (e.g. root executes "killall $shell") >> >> ? >> >> I don't think there is a way to protect from all of those, so any >> effort in protecting from only part of the problem looks useless. > ======================== > > you forgot cosmic rays, nuclear war and zombie apocalypse, among other > failure modes. *NOTHING* is capable of protecting against everything; > a good solution will most always have pitfalls; as a > sysadmin/engineer/manager one has to either accept the pitfalls or > find a more acceptable solution, which usually means different > pitfalls. that doesn't mean a given solution is useless. > > Bah. since you mentioned .logout, i'm assuming you are using tcsh. what i would suggest is that you create an md and check out the files into that. this solves the power fail issue completely, also, it solves the main issue. have the logout script simply umount and mdconfig -d the ramdisk. also, this way, security is enhanced because no fragments, even of deleted files, are left on disk after logout. the only question i have is if a bzero is done before returning the ram to the os, if not, simply dd if=/dev/zero of=/dev/md0 bs=whatever to be sure that the ram formeerly contained in the ramdisk isn't readable by later procs. have you considered trustedbsd? it should perform the bzero by default. TBSD MAC is in fbsd these days to control access to the mountpoint, but that might not help if you are worried about a lifted disk, MAC don't mean shit without physical security, the kind involved in the environments for which it was commissioned. From owner-freebsd-hackers@FreeBSD.ORG Thu Sep 30 06:46:43 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 767A1106566B; Thu, 30 Sep 2010 06:46:43 +0000 (UTC) (envelope-from danger@FreeBSD.org) Received: from services.syscare.sk (services.syscare.sk [188.40.39.36]) by mx1.freebsd.org (Postfix) with ESMTP id 34D3B8FC08; Thu, 30 Sep 2010 06:46:43 +0000 (UTC) Received: from services.syscare.sk (services [188.40.39.36]) by services.syscare.sk (Postfix) with ESMTP id DF6DB4968B; Thu, 30 Sep 2010 08:29:50 +0200 (CEST) X-Virus-Scanned: amavisd-new at rulez.sk Received: from services.syscare.sk ([188.40.39.36]) by services.syscare.sk (services.rulez.sk [188.40.39.36]) (amavisd-new, port 10024) with ESMTP id dF2g5FRSnmbh; Thu, 30 Sep 2010 08:29:48 +0200 (CEST) Received: from danger-mbp.local (59576.ba.3pp.slovanet.sk [84.16.39.226]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: danger@rulez.sk) by services.syscare.sk (Postfix) with ESMTPSA id 9C8DA49680; Thu, 30 Sep 2010 08:29:48 +0200 (CEST) Message-ID: <4CA42E5C.5090609@FreeBSD.org> Date: Thu, 30 Sep 2010 08:29:48 +0200 From: Daniel Gerzo Organization: The FreeBSD Project User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.10pre) Gecko/20100914 Lanikai/3.1.5pre MIME-Version: 1.0 To: current@freebsd.org, hackers@freebsd.org, questions@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: HEADSUP: Call for FreeBSD Status Reports - 3Q/2010 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Sep 2010 06:46:43 -0000 Dear all, I would like to remind you that the next round of status reports covering the third quarter of 2010 is due on October 15th, 2010. This initiative is very welcome in our community. Therefore, I would like to ask you to submit your status reports soon, so that we can compile the report on time. Do not hesitate and write us a few lines - a short description about what you are working on, what are your plans and goals, so we can inform our community about your great work! Check out the reports from the past to get some inspiration of what your submission should look like. If you know about a project that should be included in the status report, please let us know as well, so we can poke the responsible people to provide us with something useful. Updates to submissions from the last report are welcome too. Note that the submissions are accepted from anyone involved with the FreeBSD community, you do not have to be a FreeBSD committer. Submissions about anything related to FreeBSD are very welcome! Please email us the filled-in XML template which can be found at http://www.freebsd.org/news/status/report-sample.xml to monthly@FreeBSD.org, or alternatively use our web based form located at http://www.freebsd.org/cgi/monthly.cgi. For more information, please visit http://www.freebsd.org/news/status/. We are looking forward to see your submissions! -- S pozdravom / Best regards Daniel Gerzo, FreeBSD committer From owner-freebsd-hackers@FreeBSD.ORG Thu Sep 30 08:17:10 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7506C106564A for ; Thu, 30 Sep 2010 08:17:10 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id D24988FC15 for ; Thu, 30 Sep 2010 08:17:09 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id o8U8H5ax011140 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 30 Sep 2010 11:17:05 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id o8U8H5mA037478; Thu, 30 Sep 2010 11:17:05 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id o8U8H4sn037476; Thu, 30 Sep 2010 11:17:04 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 30 Sep 2010 11:17:04 +0300 From: Kostik Belousov To: Matthew Fleming Message-ID: <20100930081704.GH43070@deviant.kiev.zoral.com.ua> References: <20100929201221.GG43070@deviant.kiev.zoral.com.ua> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="CX2kZGZwrCIhBPP7" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-2.1 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_50, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-hackers@freebsd.org Subject: Re: Adding a V=R mapping for amd64? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Sep 2010 08:17:10 -0000 --CX2kZGZwrCIhBPP7 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Sep 29, 2010 at 01:22:56PM -0700, Matthew Fleming wrote: > On Wed, Sep 29, 2010 at 1:12 PM, Kostik Belousov wr= ote: > > On Wed, Sep 29, 2010 at 12:40:57PM -0700, Matthew Fleming wrote: > >> I'm hacking around with making a "fast reboot" that puts a copy of the > >> MBR from disk into address 0x7c00 and, after disabling various > >> translation bits and stopping other CPUs, branches to it, to skip the > >> hardware self test that normally happens on boot. > >> > >> I haven't gotten to the point of attempting to run the code at 0x7c00 > >> because I'm first hitting a different error. =9ADespite my attempts to > >> enter a translation into the hardware page table, I get a panic trying > >> to write to address 0x7000, where I intended to put the trampoline > >> code that turns off translation. > >> > >> Rebooting... > >> Attempt to reset to MBR... > >> XXX attempting pmap_kenter()... > >> XXX copying bootstrap code... > >> panic @ time 1285760103.939, thread 0xffffff000775d960: Fatal trap 12: > >> page fault while in kernel mode > >> > >> cpuid =3D 0 > >> Panic occurred in module kernel loaded at 0xffffffff80100000: > >> > >> Stack: -------------------------------------------------- > >> kernel:trap_fatal+0xac > >> kernel:trap_pfault+0x24c > >> kernel:trap+0x42e > >> kernel:bcopy+0x16 > >> kernel:shutdown_reset+0x48 > >> kernel:boot+0x317 > >> kernel:reboot+0x60 > >> kernel:ia32_syscall+0x1cd > >> -------------------------------------------------- > >> cpuid =3D 0; apic id =3D 00 > >> fault virtual address =9A =3D 0x7000 > >> fault code =9A =9A =9A =9A =9A =9A =9A=3D supervisor write data, page = not present > >> stack pointer =9A =9A =9A =9A =9A =3D 0x10:0xffffff8059e07670 > >> frame pointer =9A =9A =9A =9A =9A =3D 0x10:0xffffff8059e07780 > >> > >> Here's what I think is the relevant snippets of code. =9ANote that I > >> reserved the vm_page_t for physical page 7 as mbr_page early in boot, > >> so I know the memory is free. > >> > >> void > >> pmap_kenter_VR(vm_paddr_t pa) > >> { > >> =9A =9A =9A pmap_t pmap =3D kernel_pmap; > >> =9A =9A =9A vm_page_t mpte; > >> =9A =9A =9A pd_entry_t *pde; > >> =9A =9A =9A pt_entry_t *pte; > >> > >> =9A =9A =9A vm_page_lock_queues(); > >> =9A =9A =9A PMAP_LOCK(pmap); > >> =9A =9A =9A mpte =3D pmap_allocpte(pmap, pa, M_WAITOK); > >> > >> =9A =9A =9A pde =3D pmap_pde(pmap, pa); > >> =9A =9A =9A if (pde =3D=3D NULL || (*pde & PG_V) =3D=3D 0) > >> =9A =9A =9A =9A =9A =9A =9A panic("%s: invalid page directory va=3D%#l= x", __func__, pa); > >> =9A =9A =9A if ((*pde & PG_PS) !=3D 0) > >> =9A =9A =9A =9A =9A =9A =9A panic("%s: attempted pmap_enter on 2MB pag= e", __func__); > >> =9A =9A =9A pte =3D pmap_pde_to_pte(pde, pa); > >> =9A =9A =9A if (pte =3D=3D NULL) > >> =9A =9A =9A =9A =9A =9A =9A panic("%s: no pte va=3D%#lx", __func__, pa= ); > >> > >> =9A =9A =9A if (*pte !=3D 0) { > >> =9A =9A =9A =9A =9A =9A =9A /* Remove extra pte reference. */ > >> =9A =9A =9A =9A =9A =9A =9A mpte->wire_count--; > >> =9A =9A =9A } > >> =9A =9A =9A =9A pte_store(pte, pa | PG_RW | PG_V | PG_G | pg_nx); > >> > >> =9A =9A =9A vm_page_unlock_queues(); > >> =9A =9A =9A PMAP_UNLOCK(pmap); > >> } > >> > >> Then in cpu_reset(): > >> > >> =9A =9A =9A /* > >> =9A =9A =9A =9A* Establish a V=3DR mapping for the MBR page, and copy a > >> =9A =9A =9A =9A* reasonable guess at the size of the bootstrap code in= to the > >> =9A =9A =9A =9A* beginning of the page. > >> =9A =9A =9A =9A*/ > >> =9A =9A =9A printf("XXX attempting pmap_kenter()...\n"); > >> =9A =9A =9A pmap_kenter_VR(trunc_page(mbaddr)); > >> =9A =9A =9A printf("XXX copying bootstrap code...\n"); > >> =9A =9A =9A to_copy =3D (uintptr_t)xxx_reset_end - (uintptr_t)xxx_rese= t_real; > >> =9A =9A =9A if (to_copy > mbaddr - trunc_page(mbaddr)) > >> =9A =9A =9A =9A =9A =9A =9A to_copy =3D mbaddr - trunc_page(mbaddr); > >> =9A =9A =9A bcopy(xxx_reset_real, (void *)trunc_page(mbaddr), to_copy)= ; =9A/* die here */ > >> =9A =9A =9A printf("XXX attempting to turn off xlation and re-run MBR.= ..\n"); > >> =9A =9A =9A xxx_reset_real(mbaddr); > >> > >> > >> My first attempt was a call to > >> =9A =9A =9A pmap_kenter(trunc_page(0x7c00), trunc_page(0x7c00)); > >> which failed trying to dereference the non-existent PDE. > >> > >> My second attempt called > >> =9A =9A =9A pmap_enter(kernel_pmap, trunc_page(0x7c00), VM_PROT_WRITE,= mbr_page, > >> =9A =9A =9A =9A =9A VM_PROT_ALL, 0); > >> That failed with the same crash as the attempt using pmap_kenter_VR(). > >> > >> So... any thoughts as to why, after an apparently successful > >> installation of an xlation, I still get a panic as though there were > >> no xlation? > > > > Weird formatting of backtrace. Is this some proprietary code ? >=20 > Yeah, there's some Isilon local modifications. >=20 > > Why do you try to create 1-1 mapping at all ? The MBR code should be > > executing in real mode anyway, and the mapping address at the moment of > > bcopy does not matter at all. I think that the use of PHYS_TO_DMAP() > > should give you direct mapping. >=20 > I assumed I need trampoline code. The moment I turn off the xlation > bits, if the instruction pointer I'm running from is the normal kernel > addresses, won't I die horribly trying to access 0xffffffff80000000 or > so in real mode? Then, there is more then identity mapping. You need to gradually turn off protected mode, in particular, load segment registers with 64KB limited descriptors. After turning off paging, for far jumps to work, you would need also identity-mapped GDT. The procedure is documented in Intel IA32 arch manual, see vol 3A 9.9.3 "Switching Back to Real-Address Mode". In fact, there is some magic location in the RAM that causes BIOS to skip initialization and memory cleanup after reset, and instead make it to execute jmp to fixed location in real mode. See Ralf Brown interrupt list. In my copy, there is: MEM 0040h:0067h - RESET RESTART ADDRESS Size: DWORD Desc: this address stores the address at which to resume execution after a CPU reset (or jump to F000h:FFF0h) when certain magic values are stored at 0040h:0072h or in CMOS RAM location 0Fh This is typically used to avoid switching to real mode at all, instead cpu is reset and BIOS helps you to get the control after reset. >=20 > > About the #pf that you see. I think that this is due to the fact that > > you are modifying kernel pmap, while the active one is the pmap of > > the user process which context issued reboot(). >=20 > Isn't the kernel pmap active in the kernel, since I've entered via the > syscall reboot(2) ? No, we do not switch pmap and CR3 on the kernel<->user mode transition. Both amd64 and i386 user pmaps have a copy of the top-level kernel directory pointers (pml4 in amd64 case). This way the upper part of the address space of every process has identical kernel mapping. See pmap_init(). --CX2kZGZwrCIhBPP7 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (FreeBSD) iEYEARECAAYFAkykR4AACgkQC3+MBN1Mb4jZMgCcDDwEd0zVUf1ezWffPcvVJE+Z WUAAoIhFJOJboInI7aq5KsNMXYy64oZk =zb2n -----END PGP SIGNATURE----- --CX2kZGZwrCIhBPP7-- From owner-freebsd-hackers@FreeBSD.ORG Thu Sep 30 11:55:31 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 25EB31065674 for ; Thu, 30 Sep 2010 11:55:31 +0000 (UTC) (envelope-from onwahe@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id B0C7B8FC16 for ; Thu, 30 Sep 2010 11:55:30 +0000 (UTC) Received: by wwb17 with SMTP id 17so2417609wwb.31 for ; Thu, 30 Sep 2010 04:55:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=9dC1nghERnqFSaeDnhNL2Pu6mxLuW6JVy5KRgxlr6dI=; b=wCbsrTVxVC1HlN6F6o5XoiAC4XVlBR0ZWzhToJZDbchk3TcrVy1+/WJHhPoxvMn3A/ bY5WBJjYSuAi/MVxI2d36UVzXHiRkCk0HK3na2IhZB6YEC+i9Pvfdv6gr1nMA+wppKFS xDxm7XOUi8dGHitv8h85GZE7x8vGQjZny0IkI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=eAxDQBeEKzyqEfTuBd4QXC6NUfbUIAYLNwkMiH9aTv2ry9Xb191tM9j9c0iP2rXWu3 bZi8cDmFcLqXNRjQEelFvm86O3NKdatImnwsmK5T8HvGy6GACTXyCod5M+J4tChlyHEY ycruJGr0iStENAN4EZaH/PDMXw8vNdQhhgO8Y= MIME-Version: 1.0 Received: by 10.216.11.129 with SMTP id 1mr513754wex.90.1285846090613; Thu, 30 Sep 2010 04:28:10 -0700 (PDT) Received: by 10.216.172.210 with HTTP; Thu, 30 Sep 2010 04:28:10 -0700 (PDT) In-Reply-To: References: <29760054.post@talk.nabble.com> Date: Thu, 30 Sep 2010 13:28:10 +0200 Message-ID: From: Svatopluk Kraus To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: page table fault, which should map kernel virtual address space X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Sep 2010 11:55:31 -0000 On Tue, Sep 21, 2010 at 7:38 PM, Alan Cox wrote: > On Mon, Sep 20, 2010 at 9:32 AM, Svatopluk Kraus wrote= : >> Beyond 'kernel_map', some submaps of 'kernel_map' (buffer_map, >> pager_map,...) exist as result of 'kmem_suballoc' function call. >> When this submaps are used (for example 'kmem_alloc_nofault' >> function) and its virtual address subspace is at the end of >> used kernel virtual address space at the moment (and above 'NKPT' >> preallocation), then missing page tables are not allocated >> and double fault can happen. >> > > No, the page tables are allocated.=A0 If you create a submap X of the ker= nel > map using kmem_suballoc(), then a vm_map_findspace() is performed by > vm_map_find() on the kernel map to find space for the submap X.=A0 As you= note > above, the call to vm_map_findspace() on the kernel map will call > pmap_growkernel() if needed to extend the kernel page table. > > If you create another submap X' of X, then that submap X' can only map > addresses that fall within the range for X.=A0 So, any necessary page tab= le > pages were allocated when X was created. You are right. Mea culpa. I was focused on a solution and made too quick conclusion. The page table fault hitted in 'pager_map', which is submap of 'clean_map' and when I debugged the problem I didn't see a submap stuff as a whole. > That said, there may actually be a problem with the implementation of the > superpage_align parameter to kmem_suballoc().=A0 If a submap is created w= ith > superpage_align equal to TRUE, but the submap's size is not a multiple of > the superpage size, then vm_map_find() may not allocate a page table page > for the last megabyte or so of the submap. > > There are only a few places where kmem_suballoc() is called with > superpage_align set to TRUE.=A0 If you changed them to FALSE, that is an = easy > way to test this hypothesis. Yes, it helps. My story is that the problem started up when I updated a project ('coldfire' port) based on FreeBSD 8.0. to FreeBSD current version. In the current version the 'clean_map' submap is created with superpage_align set to TRUE. I have looked at vm_map_find() and debugged the page table fault once again= . IMO, it looks that a do-while loop does not work in the function as intende= d. A vm_map_findspace() finds a space and calls pmap_growkernel() if needed. A pmap_align_superpage() arrange the space but never call pmap_growkernel()= . A vm_map_insert() inserts the aligned space into a map without error and never call pmap_growkernel() and does not invoke loop iteration. I don't know too much about an idea how a virtual memory model is implement= ed and used in other modules. But it seems that it could be more reasonable to align address space in vm_map_findspace() internally and not to loop extern= ally. I have tried to add a check in vm_map_insert() that checks the 'end' parame= ter against 'kernel_vm_end' variable and returns KERN_NO_SPACE error if needed. In this case the loop in vm_map_find() works and I have no problem with the page table fault. But 'kernel_vm_end' variable must be initializated properly before first use of vm_map_insert(). The 'kernel_vm_end' variable can be self-initializated in pmap_growkernel() in FreeBSD 8.0 (it is too la= te), but it was changed in current version ('i386' port). Thanks for your answer, but I'm still looking for permanent and approved solution. Regards, Svata From owner-freebsd-hackers@FreeBSD.ORG Thu Sep 30 16:37:20 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8D2BE106566B for ; Thu, 30 Sep 2010 16:37:20 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 8ECC98FC0A for ; Thu, 30 Sep 2010 16:37:19 +0000 (UTC) Received: (qmail 20758 invoked from network); 30 Sep 2010 16:29:15 -0000 Received: from unknown (HELO [62.48.0.92]) ([62.48.0.92]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 30 Sep 2010 16:29:15 -0000 Message-ID: <4CA4BCD2.4070303@freebsd.org> Date: Thu, 30 Sep 2010 18:37:38 +0200 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.9) Gecko/20100915 Thunderbird/3.1.4 MIME-Version: 1.0 To: freebsd-current@freebsd.org, freebsd-hackers Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: Examining the VM splay tree effectiveness X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Sep 2010 16:37:20 -0000 Just for the kick of it I decided to take a closer look at the use of splay trees (inherited from Mach if I read the history correctly) in the FreeBSD VM system suspecting an interesting journey. The VM system has two major structures: 1) the VM map which is per process and manages its address space by tracking all regions that are allocated and their permissions. 2) the VM page table which is per VM map and provides the backing memory pages to the regions and interfaces with the pmap layer (virtual->physical). Both of these are very frequently accessed through memory allocations from userspace and page faults from the pmap layer. Their efficiency and SMP scalability is crucial for many operations beginning with fork/ exec, going through malloc/mmap and ending with free/exit of a process. Both the vmmap and page table make use of splay trees to manage the entries and to speed up lookups compared to long to traverse linked lists or more memory expensive hash tables. Some structures though do have an additional linked list to simplify ordered traversals. A splay tree is an interesting binary search tree with insertion, lookup and removal performed in O(log n) *amortized* time. With the *amortized* time being the crucial difference to other binary trees. On every access *including* lookup it rotates the tree to make the just found element the new root node. For all gory details see: http://en.wikipedia.org/wiki/Splay_tree This behavior has a few important implications: 1) the root node and constant rotation works like a cache with least recent looked up node at the top and less recently ones close to the top; 2) every lookup that doesn't match the root node, ie. a cache miss, causes a rotation of the tree to make the newly found node the new root; 3) during the rotate it has to re-arrange and touch possibly a large number of other nodes; 4) in worst case the tree may resemble a linked list. A splay tree makes no guarantee that it is balanced. For the kernel and the splay tree usage in the VM map and page table some more implications come up: a) the amortization effect/cost only balance if the majority of lookups are root node (cache) hits, otherwise the cost of rebalancing destroys any advantage and quickly turns into a large overhead. b) the overhead becomes even worse due to touching many nodes and causing a lot of CPU cache line dirtying. For processes with shared memory or threads across CPU's this overhead becomes almost excessive because the CPU's stomp on each others cached nodes and get their own caches invalidated. The penalty is a high-latency memory refetch not only for the root node lookup but every hop in the following rebalancing operation => thrashing. To quantify the positive and negative effects of the splay tree I instrumented the code and added counters to track the behavior of the splay tree in the vmmap and page table. The test case is an AMD64 kernel build to get a first overview. Other system usages may have different results depending on their fork and memory usage patters. The results are very interesting: The page table shows a *very* poor root node hit rate and an excessive amount of rotational node modifications representing almost the worst case for splay trees. http://chart.apis.google.com/chart?chs=700x300&chbh=a,23&chds=0,127484769&chd=t:16946915,16719791,48872230,131057,74858589,105206121&cht=bvg&chl=insert|remove|lookup|cachehit|rotates|rotatemodifies The vmmap shows a high root node hit rate but still a significant amount of rotational node modifications. http://chart.apis.google.com/chart?chs=700x300&chbh=a,23&chds=0,21881583&chd=t:724293,723701,20881583,19044544,3719582,4553350&cht=bvg&chl=insert|remove|lookup|cachehit|rotates|rotatemodifies From these observations I come to the conclusion that a splay tree is suboptimal for the normal case and quickly horrible even on small deviations from the normal case in the vmmap. For the page table it is always horrible. Not to mention the locking complications due to modifying the tree on lookups. Based on an examination of other binary trees I put forward the hypothesis that a Red-Black tree is a much better, if not the optimal, fit for the vmmap and page table. RB trees are balanced binary trees and O(log n) in all cases. The big advantage in this context is that lookups are pure reads and do not cause CPU cache invalidations on other CPU's and always only require a read lock without the worst case properties of the unbalanced splay tree. The high cache locality of the vmmap lookups can be used with the RB tree as well by simply adding a pointer to the least recently found node. To prevent write locking this can be done lazily. More profiling is required to make a non-speculative statement on this though. In addition a few of the additional linked lists that currently accompany the vmmap and page structures are no longer necessary as they easily can be done with standard RB tree accessors. Our standard userspace jemalloc also uses RB trees for its internal housekeeping. RB tree details: http://en.wikipedia.org/wiki/Red-black_tree I say hypothesis because I haven't measured the difference to an RB tree implementation yet. I've hacked up a crude and somewhat mechanical patch to convert the vmmap and page VM structures to use RB trees, the vmmap part is not stable yet. The page part seems to work fine though. This is what I've hacked together so far: http://people.freebsd.org/~andre/vmmap_vmpage_stats-20100930.diff http://people.freebsd.org/~andre/vmmap_vmpage_rbtree-20100930.diff The diffs are still in their early stages and do not make use of any code simplifications becoming possible by using RB trees instead of the current splay trees. Comments on the VM issue and splay vs. RB tree hypothesis welcome. -- Andre From owner-freebsd-hackers@FreeBSD.ORG Thu Sep 30 17:15:09 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 41D111065670; Thu, 30 Sep 2010 17:15:09 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-gw0-f54.google.com (mail-gw0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id C98F38FC1E; Thu, 30 Sep 2010 17:15:08 +0000 (UTC) Received: by gwb15 with SMTP id 15so1080052gwb.13 for ; Thu, 30 Sep 2010 10:15:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=K+yTQ/3ztvTBJhRhX+Zi4AldNfQ7MaAXdzqmaKbUwyI=; b=xbc/fk4guhrSTnjSEi0g1ScgSpUBVmCV4O4+Rt+hdblD5bexBXBNYmYmXKjbNvjOmt 4fb9RADbs77VVdSMQXp1zRUuFKzzc5h/6naoLTGZOuZVBAJiqutQVhJ/y4lOZT9366IM sMdVfAvsFDLPmvjjVRSHcT7Mq2/seLe5vN1iQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=wwj82wQ3Ja8RctxXIKfITisUFoIHP8hdnBgNfQcT19E/NaJ2442SqqY/pYoXGljV+6 0NhhVpqUZdopiiuhpVvm+2vuf7LWEIYO6ZPUx8yGWtQM6G1wmkYG/TjhfgNvKx6ZYH6a D2QiWlHI3XqHIpTCw+ta1HvuodJyHzNBMUv4s= MIME-Version: 1.0 Received: by 10.231.183.10 with SMTP id ce10mr4097344ibb.96.1285866907574; Thu, 30 Sep 2010 10:15:07 -0700 (PDT) Received: by 10.231.167.140 with HTTP; Thu, 30 Sep 2010 10:15:07 -0700 (PDT) In-Reply-To: <4CA4BCD2.4070303@freebsd.org> References: <4CA4BCD2.4070303@freebsd.org> Date: Thu, 30 Sep 2010 10:15:07 -0700 Message-ID: From: Matthew Fleming To: Andre Oppermann Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers , freebsd-current@freebsd.org Subject: Re: Examining the VM splay tree effectiveness X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Sep 2010 17:15:09 -0000 On Thu, Sep 30, 2010 at 9:37 AM, Andre Oppermann wrote: > Just for the kick of it I decided to take a closer look at the use of > splay trees (inherited from Mach if I read the history correctly) in > the FreeBSD VM system suspecting an interesting journey. > > The VM system has two major structures: > =A01) the VM map which is per process and manages its address space > =A0 =A0by tracking all regions that are allocated and their permissions. > =A02) the VM page table which is per VM map and provides the backing > =A0 =A0memory pages to the regions and interfaces with the pmap layer > =A0 =A0(virtual->physical). > > Both of these are very frequently accessed through memory allocations > from userspace and page faults from the pmap layer. =A0Their efficiency > and SMP scalability is crucial for many operations beginning with fork/ > exec, going through malloc/mmap and ending with free/exit of a process. > > Both the vmmap and page table make use of splay trees to manage the > entries and to speed up lookups compared to long to traverse linked > lists or more memory expensive hash tables. =A0Some structures though > do have an additional linked list to simplify ordered traversals. > > A splay tree is an interesting binary search tree with insertion, > lookup and removal performed in O(log n) *amortized* time. =A0With > the *amortized* time being the crucial difference to other binary trees. > On every access *including* lookup it rotates the tree to make the > just found element the new root node. =A0For all gory details see: > =A0http://en.wikipedia.org/wiki/Splay_tree > > This behavior has a few important implications: > =A01) the root node and constant rotation works like a cache with > =A0 =A0least recent looked up node at the top and less recently ones > =A0 =A0close to the top; > =A02) every lookup that doesn't match the root node, ie. a cache miss, > =A0 =A0causes a rotation of the tree to make the newly found node the new > =A0 =A0root; > =A03) during the rotate it has to re-arrange and touch possibly a large > =A0 =A0number of other nodes; > =A04) in worst case the tree may resemble a linked list. =A0A splay tree > =A0 =A0makes no guarantee that it is balanced. > > For the kernel and the splay tree usage in the VM map and page table > some more implications come up: > =A0a) the amortization effect/cost only balance if the majority of > =A0 =A0lookups are root node (cache) hits, otherwise the cost of > =A0 =A0rebalancing destroys any advantage and quickly turns into a > =A0 =A0large overhead. > =A0b) the overhead becomes even worse due to touching many nodes and > =A0 =A0causing a lot of CPU cache line dirtying. =A0For processes with > =A0 =A0shared memory or threads across CPU's this overhead becomes > =A0 =A0almost excessive because the CPU's stomp on each others cached > =A0 =A0nodes and get their own caches invalidated. =A0The penalty is a > =A0 =A0high-latency memory refetch not only for the root node lookup > =A0 =A0but every hop in the following rebalancing operation =3D> thrashin= g. > > To quantify the positive and negative effects of the splay tree I > instrumented the code and added counters to track the behavior of > the splay tree in the vmmap and page table. > > The test case is an AMD64 kernel build to get a first overview. > Other system usages may have different results depending on their > fork and memory usage patters. > > The results are very interesting: > > =A0The page table shows a *very* poor root node hit rate and an excessive > =A0amount of rotational node modifications representing almost the > =A0worst case for splay trees. > > http://chart.apis.google.com/chart?chs=3D700x300&chbh=3Da,23&chds=3D0,127= 484769&chd=3Dt:16946915,16719791,48872230,131057,74858589,105206121&cht=3Db= vg&chl=3Dinsert|remove|lookup|cachehit|rotates|rotatemodifies > > =A0The vmmap shows a high root node hit rate but still a significant > =A0amount of rotational node modifications. > > http://chart.apis.google.com/chart?chs=3D700x300&chbh=3Da,23&chds=3D0,218= 81583&chd=3Dt:724293,723701,20881583,19044544,3719582,4553350&cht=3Dbvg&chl= =3Dinsert|remove|lookup|cachehit|rotates|rotatemodifies > > From these observations I come to the conclusion that a splay tree > is suboptimal for the normal case and quickly horrible even on small > deviations from the normal case in the vmmap. =A0For the page table it > is always horrible. =A0Not to mention the locking complications due to > modifying the tree on lookups. > > Based on an examination of other binary trees I put forward the > hypothesis that a Red-Black tree is a much better, if not the optimal, > fit for the vmmap and page table. =A0RB trees are balanced binary trees > and O(log n) in all cases. =A0The big advantage in this context is that > lookups are pure reads and do not cause CPU cache invalidations on > other CPU's and always only require a read lock without the worst > case properties of the unbalanced splay tree. =A0The high cache locality > of the vmmap lookups can be used with the RB tree as well by simply > adding a pointer to the least recently found node. =A0To prevent write lo= cking > this can be done lazily. =A0More profiling is required to make > a non-speculative statement on this though. =A0In addition a few of > the additional linked lists that currently accompany the vmmap and > page structures are no longer necessary as they easily can be done > with standard RB tree accessors. =A0Our standard userspace jemalloc > also uses RB trees for its internal housekeeping. =A0RB tree details: > =A0http://en.wikipedia.org/wiki/Red-black_tree > > I say hypothesis because I haven't measured the difference to an > RB tree implementation yet. =A0I've hacked up a crude and somewhat > mechanical patch to convert the vmmap and page VM structures to > use RB trees, the vmmap part is not stable yet. =A0The page part > seems to work fine though. > > This is what I've hacked together so far: > =A0http://people.freebsd.org/~andre/vmmap_vmpage_stats-20100930.diff > =A0http://people.freebsd.org/~andre/vmmap_vmpage_rbtree-20100930.diff > > The diffs are still in their early stages and do not make use of > any code simplifications becoming possible by using RB trees instead > of the current splay trees. > > Comments on the VM issue and splay vs. RB tree hypothesis welcome. Do you have actual performance numbers from a real workload? I implemented an AA tree locally (basically a 2,3 tree that uses coloring to represent the 3 nodes) and the performance was much worse than the splay tree. I assume this is due to the overhead of keeping the tree balanced; I didn't instrument either the splay or the AA code. I suspect that the overhead of an RB tree would similarly wash out the benefits of O(log_2 n) time, but this is theory. The facts would be measuring several different workloads an looking at the system/real times for them between the two solutions. We've talked internally at $work about using a B+-tree with maybe branching factor 5-7; whatever makes sense for the size of a cache line. This seems likely to be better for performance than an RB-tree but does require a lot more changes, since separate memory is needed for the tree's nodes outside the vm_page structure. There just hasn't been time to implement it and try it out. Unfortunately I won't have the time to experiment at $work for a few months on this problem. Thanks, matthew From owner-freebsd-hackers@FreeBSD.ORG Thu Sep 30 17:17:54 2010 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 437B31065694 for ; Thu, 30 Sep 2010 17:17:54 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 7A3248FC14 for ; Thu, 30 Sep 2010 17:17:53 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id UAA25872 for ; Thu, 30 Sep 2010 20:17:51 +0300 (EEST) (envelope-from avg@icyb.net.ua) Message-ID: <4CA4C63F.4070503@icyb.net.ua> Date: Thu, 30 Sep 2010 20:17:51 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100920 Lightning/1.0b2 Thunderbird/3.1.4 MIME-Version: 1.0 To: freebsd-hackers@FreeBSD.org X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Subject: sysctl for querying kmem_map->size X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Sep 2010 17:17:54 -0000 Here's a patch that adds a sysctl for querying kmem_map->size, which may be useful for system state/resources monitoring: http://people.freebsd.org/~avg/sysctl-kmem_map_size.diff I am quite unsure about sizeof(kmem_map->size) == sizeof(int) hack, but I couldn't think of other way to decide whether to use SYSCTL_ADD_UINT or SYSCTL_ADD_ULONG depending on real type behind vm_size_t. Comments, suggestions? -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Thu Sep 30 17:37:21 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A42101065670 for ; Thu, 30 Sep 2010 17:37:21 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 407608FC16 for ; Thu, 30 Sep 2010 17:37:20 +0000 (UTC) Received: (qmail 21178 invoked from network); 30 Sep 2010 17:29:18 -0000 Received: from unknown (HELO [62.48.0.92]) ([62.48.0.92]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 30 Sep 2010 17:29:18 -0000 Message-ID: <4CA4CAE6.2090108@freebsd.org> Date: Thu, 30 Sep 2010 19:37:42 +0200 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.9) Gecko/20100915 Thunderbird/3.1.4 MIME-Version: 1.0 To: freebsd-current@freebsd.org, freebsd-hackers References: <4CA4BCD2.4070303@freebsd.org> In-Reply-To: <4CA4BCD2.4070303@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: Re: Examining the VM splay tree effectiveness X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Sep 2010 17:37:21 -0000 On 30.09.2010 18:37, Andre Oppermann wrote: > Just for the kick of it I decided to take a closer look at the use of > splay trees (inherited from Mach if I read the history correctly) in > the FreeBSD VM system suspecting an interesting journey. Correcting myself regarding the history: The splay tree for vmmap was done about 8 years ago by alc@ to replace a simple linked list and was a huge improvement. The change in vmpage from a hash to the same splay tree as in vmmap was committed by dillon@ about 7.5 years ago with some involvement of alc@. -- Andre From owner-freebsd-hackers@FreeBSD.ORG Thu Sep 30 17:41:55 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D5E32106566B; Thu, 30 Sep 2010 17:41:55 +0000 (UTC) (envelope-from rdivacky@vlk.vlakno.cz) Received: from vlakno.cz (lev.vlakno.cz [77.93.215.190]) by mx1.freebsd.org (Postfix) with ESMTP id 319FD8FC15; Thu, 30 Sep 2010 17:41:55 +0000 (UTC) Received: from lev.vlakno.cz (localhost [127.0.0.1]) by vlakno.cz (Postfix) with ESMTP id 849219CB0F1; Thu, 30 Sep 2010 19:24:40 +0200 (CEST) X-Virus-Scanned: amavisd-new at vlakno.cz Received: from vlakno.cz ([127.0.0.1]) by lev.vlakno.cz (lev.vlakno.cz [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 52YLvdPPXC+X; Thu, 30 Sep 2010 19:24:39 +0200 (CEST) Received: from vlk.vlakno.cz (localhost [127.0.0.1]) by vlakno.cz (Postfix) with ESMTP id 5D47E9CB12E; Thu, 30 Sep 2010 19:24:39 +0200 (CEST) Received: (from rdivacky@localhost) by vlk.vlakno.cz (8.14.4/8.14.4/Submit) id o8UHOdLC035141; Thu, 30 Sep 2010 19:24:39 +0200 (CEST) (envelope-from rdivacky) Date: Thu, 30 Sep 2010 19:24:39 +0200 From: Roman Divacky To: Andre Oppermann Message-ID: <20100930172439.GA34369@freebsd.org> References: <4CA4BCD2.4070303@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4CA4BCD2.4070303@freebsd.org> User-Agent: Mutt/1.4.2.3i Cc: freebsd-hackers , freebsd-current@freebsd.org Subject: Re: Examining the VM splay tree effectiveness X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Sep 2010 17:41:55 -0000 are you aware of Summer of Code 2008 project by Mayur Shardul? quoting: http://www.freebsd.org/projects/summerofcode-2008.html Project: VM Algorithm Improvement Student: Mayur Shardul Mentor: Jeff Roberson Summary: A new data structure, viz. radix tree, was implemented and used for management of the resident pages. The objective is efficient use of memory and faster performance. The biggest challenge was to service insert requests on the data structure without blocking. Because of this constraint the memory allocation failures were not acceptable, to solve the problem the required memory was allocated at the boot time. Both the data structures were used in parallel to check the correctness and we also benchmarked the data structures and found that radix trees gave much better performance over splay trees. Ready to enter CVS/SVN: We will investigate some more approaches to handle allocation failures before the new data structure goes in CVS. On Thu, Sep 30, 2010 at 06:37:38PM +0200, Andre Oppermann wrote: > Just for the kick of it I decided to take a closer look at the use of > splay trees (inherited from Mach if I read the history correctly) in > the FreeBSD VM system suspecting an interesting journey. > > The VM system has two major structures: > 1) the VM map which is per process and manages its address space > by tracking all regions that are allocated and their permissions. > 2) the VM page table which is per VM map and provides the backing > memory pages to the regions and interfaces with the pmap layer > (virtual->physical). > > Both of these are very frequently accessed through memory allocations > from userspace and page faults from the pmap layer. Their efficiency > and SMP scalability is crucial for many operations beginning with fork/ > exec, going through malloc/mmap and ending with free/exit of a process. > > Both the vmmap and page table make use of splay trees to manage the > entries and to speed up lookups compared to long to traverse linked > lists or more memory expensive hash tables. Some structures though > do have an additional linked list to simplify ordered traversals. > > A splay tree is an interesting binary search tree with insertion, > lookup and removal performed in O(log n) *amortized* time. With > the *amortized* time being the crucial difference to other binary trees. > On every access *including* lookup it rotates the tree to make the > just found element the new root node. For all gory details see: > http://en.wikipedia.org/wiki/Splay_tree > > This behavior has a few important implications: > 1) the root node and constant rotation works like a cache with > least recent looked up node at the top and less recently ones > close to the top; > 2) every lookup that doesn't match the root node, ie. a cache miss, > causes a rotation of the tree to make the newly found node the new > root; > 3) during the rotate it has to re-arrange and touch possibly a large > number of other nodes; > 4) in worst case the tree may resemble a linked list. A splay tree > makes no guarantee that it is balanced. > > For the kernel and the splay tree usage in the VM map and page table > some more implications come up: > a) the amortization effect/cost only balance if the majority of > lookups are root node (cache) hits, otherwise the cost of > rebalancing destroys any advantage and quickly turns into a > large overhead. > b) the overhead becomes even worse due to touching many nodes and > causing a lot of CPU cache line dirtying. For processes with > shared memory or threads across CPU's this overhead becomes > almost excessive because the CPU's stomp on each others cached > nodes and get their own caches invalidated. The penalty is a > high-latency memory refetch not only for the root node lookup > but every hop in the following rebalancing operation => thrashing. > > To quantify the positive and negative effects of the splay tree I > instrumented the code and added counters to track the behavior of > the splay tree in the vmmap and page table. > > The test case is an AMD64 kernel build to get a first overview. > Other system usages may have different results depending on their > fork and memory usage patters. > > The results are very interesting: > > The page table shows a *very* poor root node hit rate and an excessive > amount of rotational node modifications representing almost the > worst case for splay trees. > > http://chart.apis.google.com/chart?chs=700x300&chbh=a,23&chds=0,127484769&chd=t:16946915,16719791,48872230,131057,74858589,105206121&cht=bvg&chl=insert|remove|lookup|cachehit|rotates|rotatemodifies > > The vmmap shows a high root node hit rate but still a significant > amount of rotational node modifications. > > http://chart.apis.google.com/chart?chs=700x300&chbh=a,23&chds=0,21881583&chd=t:724293,723701,20881583,19044544,3719582,4553350&cht=bvg&chl=insert|remove|lookup|cachehit|rotates|rotatemodifies > > From these observations I come to the conclusion that a splay tree > is suboptimal for the normal case and quickly horrible even on small > deviations from the normal case in the vmmap. For the page table it > is always horrible. Not to mention the locking complications due to > modifying the tree on lookups. > > Based on an examination of other binary trees I put forward the > hypothesis that a Red-Black tree is a much better, if not the optimal, > fit for the vmmap and page table. RB trees are balanced binary trees > and O(log n) in all cases. The big advantage in this context is that > lookups are pure reads and do not cause CPU cache invalidations on > other CPU's and always only require a read lock without the worst > case properties of the unbalanced splay tree. The high cache locality > of the vmmap lookups can be used with the RB tree as well by simply > adding a pointer to the least recently found node. To prevent write > locking this can be done lazily. More profiling is required to make > a non-speculative statement on this though. In addition a few of > the additional linked lists that currently accompany the vmmap and > page structures are no longer necessary as they easily can be done > with standard RB tree accessors. Our standard userspace jemalloc > also uses RB trees for its internal housekeeping. RB tree details: > http://en.wikipedia.org/wiki/Red-black_tree > > I say hypothesis because I haven't measured the difference to an > RB tree implementation yet. I've hacked up a crude and somewhat > mechanical patch to convert the vmmap and page VM structures to > use RB trees, the vmmap part is not stable yet. The page part > seems to work fine though. > > This is what I've hacked together so far: > http://people.freebsd.org/~andre/vmmap_vmpage_stats-20100930.diff > http://people.freebsd.org/~andre/vmmap_vmpage_rbtree-20100930.diff > > The diffs are still in their early stages and do not make use of > any code simplifications becoming possible by using RB trees instead > of the current splay trees. > > Comments on the VM issue and splay vs. RB tree hypothesis welcome. > > -- > Andre > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Thu Sep 30 17:44:19 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B9D921065679 for ; Thu, 30 Sep 2010 17:44:19 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id EAABD8FC18 for ; Thu, 30 Sep 2010 17:44:18 +0000 (UTC) Received: (qmail 21251 invoked from network); 30 Sep 2010 17:36:16 -0000 Received: from unknown (HELO [62.48.0.92]) ([62.48.0.92]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 30 Sep 2010 17:36:16 -0000 Message-ID: <4CA4CC87.9060605@freebsd.org> Date: Thu, 30 Sep 2010 19:44:39 +0200 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.9) Gecko/20100915 Thunderbird/3.1.4 MIME-Version: 1.0 To: Matthew Fleming References: <4CA4BCD2.4070303@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers , freebsd-current@freebsd.org Subject: Re: Examining the VM splay tree effectiveness X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Sep 2010 17:44:19 -0000 On 30.09.2010 19:15, Matthew Fleming wrote: > On Thu, Sep 30, 2010 at 9:37 AM, Andre Oppermann wrote: >> Just for the kick of it I decided to take a closer look at the use of >> splay trees (inherited from Mach if I read the history correctly) in >> the FreeBSD VM system suspecting an interesting journey. >> >> The VM system has two major structures: >> 1) the VM map which is per process and manages its address space >> by tracking all regions that are allocated and their permissions. >> 2) the VM page table which is per VM map and provides the backing >> memory pages to the regions and interfaces with the pmap layer >> (virtual->physical). >> >> Both of these are very frequently accessed through memory allocations >> from userspace and page faults from the pmap layer. Their efficiency >> and SMP scalability is crucial for many operations beginning with fork/ >> exec, going through malloc/mmap and ending with free/exit of a process. >> >> Both the vmmap and page table make use of splay trees to manage the >> entries and to speed up lookups compared to long to traverse linked >> lists or more memory expensive hash tables. Some structures though >> do have an additional linked list to simplify ordered traversals. >> >> A splay tree is an interesting binary search tree with insertion, >> lookup and removal performed in O(log n) *amortized* time. With >> the *amortized* time being the crucial difference to other binary trees. >> On every access *including* lookup it rotates the tree to make the >> just found element the new root node. For all gory details see: >> http://en.wikipedia.org/wiki/Splay_tree >> >> This behavior has a few important implications: >> 1) the root node and constant rotation works like a cache with >> least recent looked up node at the top and less recently ones >> close to the top; >> 2) every lookup that doesn't match the root node, ie. a cache miss, >> causes a rotation of the tree to make the newly found node the new >> root; >> 3) during the rotate it has to re-arrange and touch possibly a large >> number of other nodes; >> 4) in worst case the tree may resemble a linked list. A splay tree >> makes no guarantee that it is balanced. >> >> For the kernel and the splay tree usage in the VM map and page table >> some more implications come up: >> a) the amortization effect/cost only balance if the majority of >> lookups are root node (cache) hits, otherwise the cost of >> rebalancing destroys any advantage and quickly turns into a >> large overhead. >> b) the overhead becomes even worse due to touching many nodes and >> causing a lot of CPU cache line dirtying. For processes with >> shared memory or threads across CPU's this overhead becomes >> almost excessive because the CPU's stomp on each others cached >> nodes and get their own caches invalidated. The penalty is a >> high-latency memory refetch not only for the root node lookup >> but every hop in the following rebalancing operation => thrashing. >> >> To quantify the positive and negative effects of the splay tree I >> instrumented the code and added counters to track the behavior of >> the splay tree in the vmmap and page table. >> >> The test case is an AMD64 kernel build to get a first overview. >> Other system usages may have different results depending on their >> fork and memory usage patters. >> >> The results are very interesting: >> >> The page table shows a *very* poor root node hit rate and an excessive >> amount of rotational node modifications representing almost the >> worst case for splay trees. >> >> http://chart.apis.google.com/chart?chs=700x300&chbh=a,23&chds=0,127484769&chd=t:16946915,16719791,48872230,131057,74858589,105206121&cht=bvg&chl=insert|remove|lookup|cachehit|rotates|rotatemodifies >> >> The vmmap shows a high root node hit rate but still a significant >> amount of rotational node modifications. >> >> http://chart.apis.google.com/chart?chs=700x300&chbh=a,23&chds=0,21881583&chd=t:724293,723701,20881583,19044544,3719582,4553350&cht=bvg&chl=insert|remove|lookup|cachehit|rotates|rotatemodifies >> >> From these observations I come to the conclusion that a splay tree >> is suboptimal for the normal case and quickly horrible even on small >> deviations from the normal case in the vmmap. For the page table it >> is always horrible. Not to mention the locking complications due to >> modifying the tree on lookups. >> >> Based on an examination of other binary trees I put forward the >> hypothesis that a Red-Black tree is a much better, if not the optimal, >> fit for the vmmap and page table. RB trees are balanced binary trees >> and O(log n) in all cases. The big advantage in this context is that >> lookups are pure reads and do not cause CPU cache invalidations on >> other CPU's and always only require a read lock without the worst >> case properties of the unbalanced splay tree. The high cache locality >> of the vmmap lookups can be used with the RB tree as well by simply >> adding a pointer to the least recently found node. To prevent write locking >> this can be done lazily. More profiling is required to make >> a non-speculative statement on this though. In addition a few of >> the additional linked lists that currently accompany the vmmap and >> page structures are no longer necessary as they easily can be done >> with standard RB tree accessors. Our standard userspace jemalloc >> also uses RB trees for its internal housekeeping. RB tree details: >> http://en.wikipedia.org/wiki/Red-black_tree >> >> I say hypothesis because I haven't measured the difference to an >> RB tree implementation yet. I've hacked up a crude and somewhat >> mechanical patch to convert the vmmap and page VM structures to >> use RB trees, the vmmap part is not stable yet. The page part >> seems to work fine though. >> >> This is what I've hacked together so far: >> http://people.freebsd.org/~andre/vmmap_vmpage_stats-20100930.diff >> http://people.freebsd.org/~andre/vmmap_vmpage_rbtree-20100930.diff >> >> The diffs are still in their early stages and do not make use of >> any code simplifications becoming possible by using RB trees instead >> of the current splay trees. >> >> Comments on the VM issue and splay vs. RB tree hypothesis welcome. > > Do you have actual performance numbers from a real workload? No, not yet. I obviously would have posted those numbers as well. What do you consider a representative real workload? > I implemented an AA tree locally (basically a 2,3 tree that uses > coloring to represent the 3 nodes) and the performance was much worse > than the splay tree. I assume this is due to the overhead of keeping > the tree balanced; I didn't instrument either the splay or the AA > code. I suspect that the overhead of an RB tree would similarly wash > out the benefits of O(log_2 n) time, but this is theory. The facts > would be measuring several different workloads an looking at the > system/real times for them between the two solutions. I think there are two pronounced effects to consider: a) the algorithmic complexity b) the real world implications of said complexity and operations on today's CPU's multi-hierarchy caches and associated SMP side effects. > We've talked internally at $work about using a B+-tree with maybe > branching factor 5-7; whatever makes sense for the size of a cache > line. This seems likely to be better for performance than an RB-tree > but does require a lot more changes, since separate memory is needed > for the tree's nodes outside the vm_page structure. There just hasn't > been time to implement it and try it out. > > Unfortunately I won't have the time to experiment at $work for a few > months on this problem. -- Andre From owner-freebsd-hackers@FreeBSD.ORG Thu Sep 30 18:01:51 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B6D6B1065675; Thu, 30 Sep 2010 18:01:51 +0000 (UTC) (envelope-from alan.l.cox@gmail.com) Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id 841F68FC0C; Thu, 30 Sep 2010 18:01:51 +0000 (UTC) Received: by pzk7 with SMTP id 7so710760pzk.13 for ; Thu, 30 Sep 2010 11:01:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:reply-to :in-reply-to:references:date:message-id:subject:from:to:cc :content-type; bh=HTVWFQJCaDYink14UwYwO6xIP1kkrXGRUiGtf0IW+38=; b=CgQR2qnAc96UOZyvIxxeCC22+9f0GjursgxBQ/+AUSdbq4PQE3Gu+812LMA5mJ26ZI zatqciQ3sCW5yFTJklExlfpxJE0qtX7HDsOCt3whsDejUzEB/p4xhHhyX199vhGF/TYb e5Qv+19iZqzHe6F39eS25/Dl9jRIDefSm9xTk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; b=V/thMLbt2DqFfUQ1853xpb3i8q45kSDAxQPA3xEwnx9oEAcEr0hPzqiTynDyBR0mww fwNtjnJglV7s3oCMchPolQAgbn0eIK6ijE9MkmVPbFqTluP92xlgdy3UU3V4hlWoFAhc VsOaXykt0CALR1OXdXCREKqqK+V2ogOiSQWu0= MIME-Version: 1.0 Received: by 10.114.95.12 with SMTP id s12mr4700695wab.217.1285869710947; Thu, 30 Sep 2010 11:01:50 -0700 (PDT) Received: by 10.42.170.136 with HTTP; Thu, 30 Sep 2010 11:01:50 -0700 (PDT) In-Reply-To: <4CA4CAE6.2090108@freebsd.org> References: <4CA4BCD2.4070303@freebsd.org> <4CA4CAE6.2090108@freebsd.org> Date: Thu, 30 Sep 2010 13:01:50 -0500 Message-ID: From: Alan Cox To: Andre Oppermann Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers , freebsd-current@freebsd.org Subject: Re: Examining the VM splay tree effectiveness X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: alc@freebsd.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Sep 2010 18:01:51 -0000 On Thu, Sep 30, 2010 at 12:37 PM, Andre Oppermann wrote: > On 30.09.2010 18:37, Andre Oppermann wrote: > >> Just for the kick of it I decided to take a closer look at the use of >> splay trees (inherited from Mach if I read the history correctly) in >> the FreeBSD VM system suspecting an interesting journey. >> > > Correcting myself regarding the history: The splay tree for vmmap was > done about 8 years ago by alc@ to replace a simple linked list and was > a huge improvement. The change in vmpage from a hash to the same splay > tree as in vmmap was committed by dillon@ about 7.5 years ago with some > involvement of alc@. > ss > Yes, and there is a substantial difference in the degree of locality of access to these different structures, and thus the effectiveness of a splay tree. When I did the last round of changes to the locking on the vm map, I made some measurements of the splay tree's performance on a JVM running a moderately large bioinformatics application. The upshot was that the average number of map entries visited on an access to the vm map's splay tree was less than the expected depth of a node in a perfectly balanced tree. I teach class shortly. I'll provide more details later. Regards, Alan From owner-freebsd-hackers@FreeBSD.ORG Thu Sep 30 18:24:30 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3E593106566B for ; Thu, 30 Sep 2010 18:24:30 +0000 (UTC) (envelope-from freebsd-hackers@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id B6F7B8FC13 for ; Thu, 30 Sep 2010 18:24:29 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1P1Nnw-00066M-ME for freebsd-hackers@freebsd.org; Thu, 30 Sep 2010 20:24:28 +0200 Received: from 93-138-123-63.adsl.net.t-com.hr ([93.138.123.63]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 30 Sep 2010 20:24:28 +0200 Received: from ivoras by 93-138-123-63.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 30 Sep 2010 20:24:28 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-hackers@freebsd.org From: Ivan Voras Date: Thu, 30 Sep 2010 20:24:19 +0200 Lines: 33 Message-ID: References: <4CA4BCD2.4070303@freebsd.org> <4CA4CAE6.2090108@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 93-138-123-63.adsl.net.t-com.hr User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.9) Gecko/20100620 Thunderbird/3.0.4 In-Reply-To: Cc: freebsd-current@freebsd.org Subject: Re: Examining the VM splay tree effectiveness X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Sep 2010 18:24:30 -0000 On 09/30/10 20:01, Alan Cox wrote: > On Thu, Sep 30, 2010 at 12:37 PM, Andre Oppermann wrote: > >> On 30.09.2010 18:37, Andre Oppermann wrote: >> >>> Just for the kick of it I decided to take a closer look at the use of >>> splay trees (inherited from Mach if I read the history correctly) in >>> the FreeBSD VM system suspecting an interesting journey. >>> >> >> Correcting myself regarding the history: The splay tree for vmmap was >> done about 8 years ago by alc@ to replace a simple linked list and was >> a huge improvement. The change in vmpage from a hash to the same splay >> tree as in vmmap was committed by dillon@ about 7.5 years ago with some >> involvement of alc@. >> ss > Yes, and there is a substantial difference in the degree of locality of > access to these different structures, and thus the effectiveness of a splay > tree. When I did the last round of changes to the locking on the vm map, I > made some measurements of the splay tree's performance on a JVM running a > moderately large bioinformatics application. The upshot was that the > average number of map entries visited on an access to the vm map's splay > tree was less than the expected depth of a node in a perfectly balanced > tree. Sorry, I'm not sure how to parse that - are you saying that the splaying helped, making the number of nodes visited during tree search lesser than the average node depth in a balanced tree? Even if so, wouldn't the excessive bandwidth lost in the splaying ops (and worse - write bandwidth) make it unsuitable today? From owner-freebsd-hackers@FreeBSD.ORG Thu Sep 30 18:26:36 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 79B951065695 for ; Thu, 30 Sep 2010 18:26:36 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 00CD98FC1A for ; Thu, 30 Sep 2010 18:26:35 +0000 (UTC) Received: by bwz15 with SMTP id 15so2131487bwz.13 for ; Thu, 30 Sep 2010 11:26:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:subject:references:in-reply-to :x-enigmail-version:content-type:content-transfer-encoding; bh=qk397ocKtPchNJbDQE9pNnSdMleCsWdeDDQJAUOCJhY=; b=WSQq+tPBPRsftkkFXaGlTloTftaRRqEs/fBzVIy4LD1kTM+1mNw63QbkxXhcjfCFXq gnet9wbkwhPj1n+auqe1UEm5cRMBbjPNsxXdO3YN2qAJnadcuRSXU7Ea9XCI6b3sV8nZ cj4LeXfbc+700xVPPV6zu0iPiiEv28g0J6UCI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:subject :references:in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; b=gyJcOE4B8jQuRvZDPHv56yqJME3WVOJEUkT5gc6o7J4p/8xssbHmw1A3fXCCZC49RL 9grEpEMWPn/6jNCdH81FgBDD1Dtyq16vfZhiCmYrd6OAoIyo+GQ2YZ1mJrBKKs3EWSn4 Znn+7YTQKV00HUzTuwve1vVXVMhZGLGDYBZ8E= Received: by 10.204.82.137 with SMTP id b9mr3027012bkl.127.1285871194698; Thu, 30 Sep 2010 11:26:34 -0700 (PDT) Received: from mavbook2.mavhome.dp.ua (pc.mavhome.dp.ua [212.86.226.226]) by mx.google.com with ESMTPS id x13sm165267bki.12.2010.09.30.11.26.32 (version=SSLv3 cipher=RC4-MD5); Thu, 30 Sep 2010 11:26:33 -0700 (PDT) Sender: Alexander Motin Message-ID: <4CA4D64A.9020807@FreeBSD.org> Date: Thu, 30 Sep 2010 21:26:18 +0300 From: Alexander Motin User-Agent: Thunderbird 2.0.0.23 (X11/20091212) MIME-Version: 1.0 To: Karl Pielorz , freebsd-hackers@freebsd.org References: In-Reply-To: X-Enigmail-Version: 0.96.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Subject: Re: 8.1-R - Marvell 88SX6081 SATA controller via mvs = lots of errors X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Sep 2010 18:26:36 -0000 Hi. Karl Pielorz wrote: > I just switched my 8.1-R/amd64 (dual Opteron) system from ATA over to > the new mvs driver, and started seeing a whole bunch of errors (which > appear to have hosed one of my zfs volumes during a scrub) - anyone know > what the following errors actually mean? > > The machine has 2 * 88SX6081's in it: > > " > Sep 28 19:58:49 kernel: mvs0: port > 0x3000-0x30ff mem 0xd0100000-0xd01fffff,0xd0400000-0xd07fffff irq 24 at > device 4.0 on pci17 > Sep 28 19:58:49 kernel: mvs0: Gen-II, 8 3Gbps ports, Port Multiplier > ... > Sep 28 19:58:49 kernel: mvs1: port > 0x4000-0x40ff mem 0xd0c00000-0xd0cfffff,0xd0800000-0xd0bfffff irq 28 at > device 4.0 on pci18 > Sep 28 19:58:49 kernel: mvs1: Gen-II, 8 3Gbps ports, Port Multiplier > supported > " > > Under 7.2 they ran fine, with the ATA driver. I use ZFS on this machine > - and both pools were scrubbed before the upgrade (and backed up > fortunately!). > > > With the mvs driver, during a scrub of the main volume, I see: > > " > Sep 29 08:56:13 kernel: mvsch12: EMPTY CRPB 6 (->14) 1 4000 > Sep 29 08:56:13 kernel: mvsch12: EMPTY CRPB 7 (->14) 0 4000 > Sep 29 08:56:13 kernel: mvsch12: EMPTY CRPB 8 (->14) 2 4000 > " > > [repeated a lot - interspersed with zfs reporting problems with files, > on all the devices in the pool] > > I then also get a whole bunch of: > > " > Sep 29 08:56:56 kernel: mvsch0: Timeout on slot 1 > Sep 29 08:56:56 kernel: mvsch0: iec 02000000 sstat 00000123 serr > 00000000 edma_s 00001020 dma_c 00000000 dma_s 00000000 rs 00000006 statu > s 40 > Sep 29 08:56:56 kernel: mvsch0: ... waiting for slots 00000004 > Sep 29 08:56:56 kernel: mvsch12: Timeout on slot 5 > Sep 29 08:56:56 kernel: mvsch12: iec 02000000 sstat 00000123 serr > 00000000 edma_s 00001121 dma_c 00000000 dma_s 00000000 rs 00000028 stat > us 40 > " "EMPTY CRPB" error means that controller reported completion for command slot that driver counted as empty at the moment. Can't say if it is hardware or driver issue. Timeouts could be related but I am not sure what is the reason and what is consequence here. It could help if you send me full log of those messages to create full picture. -- Alexander Motin From owner-freebsd-hackers@FreeBSD.ORG Thu Sep 30 18:38:54 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2934F1065672 for ; Thu, 30 Sep 2010 18:38:54 +0000 (UTC) (envelope-from bright@elvis.mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id D49AD8FC13 for ; Thu, 30 Sep 2010 18:38:53 +0000 (UTC) Received: by elvis.mu.org (Postfix, from userid 1192) id 51C901A3D23; Thu, 30 Sep 2010 11:38:53 -0700 (PDT) Date: Thu, 30 Sep 2010 11:38:53 -0700 From: Alfred Perlstein To: Andre Oppermann Message-ID: <20100930183853.GX49807@elvis.mu.org> References: <4CA4BCD2.4070303@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4CA4BCD2.4070303@freebsd.org> User-Agent: Mutt/1.4.2.3i Cc: freebsd-hackers , freebsd-current@freebsd.org Subject: Re: Examining the VM splay tree effectiveness X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Sep 2010 18:38:54 -0000 Andre, Your observations on the effectiveness of the splay tree mirror the concerns I have with it when I read about it. I have always wondered though if the splay-tree algorithm was modified to only perform rotations when a lookup required more than "N" traversals to reach a node. This would self-balance the tree and maintain cache without the expense of writes for nearly all lookups. I'm wondering if you have the time/interest in rerunning your tests, but modifying the algorithm to only rebalance the splay if a lookup requires more than let's say 3, 5, 7 or 10 traversals. what do you think? -Alfred * Andre Oppermann [100930 09:38] wrote: > Just for the kick of it I decided to take a closer look at the use of > splay trees (inherited from Mach if I read the history correctly) in > the FreeBSD VM system suspecting an interesting journey. > > The VM system has two major structures: > 1) the VM map which is per process and manages its address space > by tracking all regions that are allocated and their permissions. > 2) the VM page table which is per VM map and provides the backing > memory pages to the regions and interfaces with the pmap layer > (virtual->physical). > > Both of these are very frequently accessed through memory allocations > from userspace and page faults from the pmap layer. Their efficiency > and SMP scalability is crucial for many operations beginning with fork/ > exec, going through malloc/mmap and ending with free/exit of a process. > > Both the vmmap and page table make use of splay trees to manage the > entries and to speed up lookups compared to long to traverse linked > lists or more memory expensive hash tables. Some structures though > do have an additional linked list to simplify ordered traversals. > > A splay tree is an interesting binary search tree with insertion, > lookup and removal performed in O(log n) *amortized* time. With > the *amortized* time being the crucial difference to other binary trees. > On every access *including* lookup it rotates the tree to make the > just found element the new root node. For all gory details see: > http://en.wikipedia.org/wiki/Splay_tree > > This behavior has a few important implications: > 1) the root node and constant rotation works like a cache with > least recent looked up node at the top and less recently ones > close to the top; > 2) every lookup that doesn't match the root node, ie. a cache miss, > causes a rotation of the tree to make the newly found node the new > root; > 3) during the rotate it has to re-arrange and touch possibly a large > number of other nodes; > 4) in worst case the tree may resemble a linked list. A splay tree > makes no guarantee that it is balanced. > > For the kernel and the splay tree usage in the VM map and page table > some more implications come up: > a) the amortization effect/cost only balance if the majority of > lookups are root node (cache) hits, otherwise the cost of > rebalancing destroys any advantage and quickly turns into a > large overhead. > b) the overhead becomes even worse due to touching many nodes and > causing a lot of CPU cache line dirtying. For processes with > shared memory or threads across CPU's this overhead becomes > almost excessive because the CPU's stomp on each others cached > nodes and get their own caches invalidated. The penalty is a > high-latency memory refetch not only for the root node lookup > but every hop in the following rebalancing operation => thrashing. > > To quantify the positive and negative effects of the splay tree I > instrumented the code and added counters to track the behavior of > the splay tree in the vmmap and page table. > > The test case is an AMD64 kernel build to get a first overview. > Other system usages may have different results depending on their > fork and memory usage patters. > > The results are very interesting: > > The page table shows a *very* poor root node hit rate and an excessive > amount of rotational node modifications representing almost the > worst case for splay trees. > > http://chart.apis.google.com/chart?chs=700x300&chbh=a,23&chds=0,127484769&chd=t:16946915,16719791,48872230,131057,74858589,105206121&cht=bvg&chl=insert|remove|lookup|cachehit|rotates|rotatemodifies > > The vmmap shows a high root node hit rate but still a significant > amount of rotational node modifications. > > http://chart.apis.google.com/chart?chs=700x300&chbh=a,23&chds=0,21881583&chd=t:724293,723701,20881583,19044544,3719582,4553350&cht=bvg&chl=insert|remove|lookup|cachehit|rotates|rotatemodifies > > From these observations I come to the conclusion that a splay tree > is suboptimal for the normal case and quickly horrible even on small > deviations from the normal case in the vmmap. For the page table it > is always horrible. Not to mention the locking complications due to > modifying the tree on lookups. > > Based on an examination of other binary trees I put forward the > hypothesis that a Red-Black tree is a much better, if not the optimal, > fit for the vmmap and page table. RB trees are balanced binary trees > and O(log n) in all cases. The big advantage in this context is that > lookups are pure reads and do not cause CPU cache invalidations on > other CPU's and always only require a read lock without the worst > case properties of the unbalanced splay tree. The high cache locality > of the vmmap lookups can be used with the RB tree as well by simply > adding a pointer to the least recently found node. To prevent write > locking this can be done lazily. More profiling is required to make > a non-speculative statement on this though. In addition a few of > the additional linked lists that currently accompany the vmmap and > page structures are no longer necessary as they easily can be done > with standard RB tree accessors. Our standard userspace jemalloc > also uses RB trees for its internal housekeeping. RB tree details: > http://en.wikipedia.org/wiki/Red-black_tree > > I say hypothesis because I haven't measured the difference to an > RB tree implementation yet. I've hacked up a crude and somewhat > mechanical patch to convert the vmmap and page VM structures to > use RB trees, the vmmap part is not stable yet. The page part > seems to work fine though. > > This is what I've hacked together so far: > http://people.freebsd.org/~andre/vmmap_vmpage_stats-20100930.diff > http://people.freebsd.org/~andre/vmmap_vmpage_rbtree-20100930.diff > > The diffs are still in their early stages and do not make use of > any code simplifications becoming possible by using RB trees instead > of the current splay trees. > > Comments on the VM issue and splay vs. RB tree hypothesis welcome. > > -- > Andre > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" -- - Alfred Perlstein .- VMOA #5191, 03 vmax, 92 gs500, 85 ch250, 07 zx10 .- FreeBSD committer From owner-freebsd-hackers@FreeBSD.ORG Thu Sep 30 18:52:26 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 21CA31065674 for ; Thu, 30 Sep 2010 18:52:26 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-gw0-f54.google.com (mail-gw0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id 6241E8FC1E for ; Thu, 30 Sep 2010 18:52:21 +0000 (UTC) Received: by gwb15 with SMTP id 15so1115749gwb.13 for ; Thu, 30 Sep 2010 11:52:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type; bh=IdIFXcQeZg50CGb06C5Dh9rgMe9OLBBuTq9VvHh+CUU=; b=uOQ5jL2O2mi+Bt0VnsqbAF5b7DMF4XC7uCxwew8z5Zuv702ZxzAZyrep6iGw1/Jk+C C0XSQhJSmy2VOeX2SW3Gi0gN6osHZ7n4/SpVE9RLPRMbXGOo96PSbKTfvdyCQkWopp6v Fux0dDTTTFI6OJXnda4MiXEqRsx++TFDoIGTU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=ayzdkuQLBTIiBYcX9lVhxN7+Efe8WuHtw/0QoBKcXIpbPVZqOwv5ac24wQkknnysaB i4TGa7967MBmcN9zWyXWsRoEoa9+k8RNNqMp01BQzhGv01XVaQlcVWn7owrpB004Bst5 iuoO7kzq+9NrWdhUuzgjj4pTp5WZhMqeE55Qc= MIME-Version: 1.0 Received: by 10.231.38.9 with SMTP id z9mr4255930ibd.24.1285872740277; Thu, 30 Sep 2010 11:52:20 -0700 (PDT) Sender: yanegomi@gmail.com Received: by 10.231.190.68 with HTTP; Thu, 30 Sep 2010 11:52:20 -0700 (PDT) In-Reply-To: <4CA4C63F.4070503@icyb.net.ua> References: <4CA4C63F.4070503@icyb.net.ua> Date: Thu, 30 Sep 2010 11:52:20 -0700 X-Google-Sender-Auth: _0v0aG8OcRR58-DZGBrSqUndDDk Message-ID: From: Garrett Cooper To: Andriy Gapon Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org Subject: Re: sysctl for querying kmem_map->size X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Sep 2010 18:52:26 -0000 On Thu, Sep 30, 2010 at 10:17 AM, Andriy Gapon wrote: > > Here's a patch that adds a sysctl for querying kmem_map->size, which may be useful > for system state/resources monitoring: > http://people.freebsd.org/~avg/sysctl-kmem_map_size.diff > > I am quite unsure about sizeof(kmem_map->size) == sizeof(int) hack, but I couldn't > think of other way to decide whether to use SYSCTL_ADD_UINT or SYSCTL_ADD_ULONG > depending on real type behind vm_size_t. Is the base value of the field size_t? If so, then it's ulong on 64-bit archs and uint on 32-bit archs. Maybe it's a good time then to actually get the sysctl and tunables work that I started on into base. I have a functioning and tested copy of the tunables work, but I'll need to do similar for the sysctls as well (des@ and I kind of got out of sync a few months back). Thanks, -Garrett From owner-freebsd-hackers@FreeBSD.ORG Thu Sep 30 19:06:33 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 818D6106564A for ; Thu, 30 Sep 2010 19:06:33 +0000 (UTC) (envelope-from freebsd-hackers@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 06EEE8FC0C for ; Thu, 30 Sep 2010 19:06:32 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1P1OSc-0007Di-TD for freebsd-hackers@freebsd.org; Thu, 30 Sep 2010 21:06:30 +0200 Received: from 93-138-123-63.adsl.net.t-com.hr ([93.138.123.63]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 30 Sep 2010 21:06:30 +0200 Received: from ivoras by 93-138-123-63.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 30 Sep 2010 21:06:30 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-hackers@freebsd.org From: Ivan Voras Date: Thu, 30 Sep 2010 21:06:20 +0200 Lines: 42 Message-ID: References: <4CA4BCD2.4070303@freebsd.org> <20100930183853.GX49807@elvis.mu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 93-138-123-63.adsl.net.t-com.hr User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.9) Gecko/20100620 Thunderbird/3.0.4 In-Reply-To: <20100930183853.GX49807@elvis.mu.org> Cc: freebsd-current@freebsd.org Subject: Re: Examining the VM splay tree effectiveness X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Sep 2010 19:06:33 -0000 On 09/30/10 20:38, Alfred Perlstein wrote: > Andre, > > Your observations on the effectiveness of the splay tree > mirror the concerns I have with it when I read about it. > > I have always wondered though if the splay-tree algorithm > was modified to only perform rotations when a lookup required > more than "N" traversals to reach a node. > > This would self-balance the tree and maintain cache without > the expense of writes for nearly all lookups. > > I'm wondering if you have the time/interest in rerunning > your tests, but modifying the algorithm to only rebalance > the splay if a lookup requires more than let's say 3, 5, 7 > or 10 traversals. I see two possible problems with this: 1: the validity of this heuristics, since the splay is not meant to help the current lookup but future lookups, and if you "now" require e.g. 5-deep traversal, (barring external information about the structures - meybe some inner relationship of the nodes can be exploitet) it is I think about the same probability that the next lookup will hit that rotated node or the former root node. 2: rotating only on the N'th lookup would have to go like this: 1. take a read-only lock 2. make the lookup, count the depth 3. if depth > N: 1. relock for write (lock upgrade will not always work) 2. recheck if the tree is still the same; bail if it isn't 3. do the splay 4. unlock i.e. suspiciously complicated. That is, if you want to take advantage of read paralelism; if the tree is write-locked all the time it's simpler but only inefficient. Of course, real-world measurements trump theory :) From owner-freebsd-hackers@FreeBSD.ORG Thu Sep 30 20:26:56 2010 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EB6E9106566C for ; Thu, 30 Sep 2010 20:26:56 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 0CF1F8FC16 for ; Thu, 30 Sep 2010 20:26:55 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id XAA28770; Thu, 30 Sep 2010 23:26:54 +0300 (EEST) (envelope-from avg@icyb.net.ua) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1P1PiP-0008GL-Se; Thu, 30 Sep 2010 23:26:53 +0300 Message-ID: <4CA4F28C.3020804@icyb.net.ua> Date: Thu, 30 Sep 2010 23:26:52 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100918 Lightning/1.0b2 Thunderbird/3.1.4 MIME-Version: 1.0 To: Garrett Cooper References: <4CA4C63F.4070503@icyb.net.ua> In-Reply-To: X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@FreeBSD.org Subject: Re: sysctl for querying kmem_map->size X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Sep 2010 20:26:57 -0000 on 30/09/2010 21:52 Garrett Cooper said the following: > On Thu, Sep 30, 2010 at 10:17 AM, Andriy Gapon wrote: >> >> Here's a patch that adds a sysctl for querying kmem_map->size, which may be useful >> for system state/resources monitoring: >> http://people.freebsd.org/~avg/sysctl-kmem_map_size.diff >> >> I am quite unsure about sizeof(kmem_map->size) == sizeof(int) hack, but I couldn't >> think of other way to decide whether to use SYSCTL_ADD_UINT or SYSCTL_ADD_ULONG >> depending on real type behind vm_size_t. > > Is the base value of the field size_t? If so, then it's ulong on > 64-bit archs and uint on 32-bit archs. Maybe it's a good time then to No, it's vm_size_t, but it's defined similarly to size_t I guess: vm_size_t -> __vm_size_t -> {__uint32_t or __uint64_t depending on arch} > actually get the sysctl and tunables work that I started on into base. > I have a functioning and tested copy of the tunables work, but I'll > need to do similar for the sysctls as well (des@ and I kind of got out > of sync a few months back). I believe that this is the first time I hear about this project. -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Thu Sep 30 20:56:53 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DB73D1065672 for ; Thu, 30 Sep 2010 20:56:53 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 3280D8FC17 for ; Thu, 30 Sep 2010 20:56:52 +0000 (UTC) Received: (qmail 22373 invoked from network); 30 Sep 2010 20:48:48 -0000 Received: from localhost (HELO [127.0.0.1]) ([127.0.0.1]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 30 Sep 2010 20:48:48 -0000 Message-ID: <4CA4F998.5000908@freebsd.org> Date: Thu, 30 Sep 2010 22:56:56 +0200 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.9) Gecko/20100825 Thunderbird/3.1.3 MIME-Version: 1.0 To: alc@freebsd.org References: <4CA4BCD2.4070303@freebsd.org> <4CA4CAE6.2090108@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers , Alan Cox , freebsd-current@freebsd.org Subject: Re: Examining the VM splay tree effectiveness X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Sep 2010 20:56:53 -0000 On 30.09.2010 20:01, Alan Cox wrote: > On Thu, Sep 30, 2010 at 12:37 PM, Andre Oppermann wrote: > >> On 30.09.2010 18:37, Andre Oppermann wrote: >> >>> Just for the kick of it I decided to take a closer look at the use of >>> splay trees (inherited from Mach if I read the history correctly) in >>> the FreeBSD VM system suspecting an interesting journey. >>> >> >> Correcting myself regarding the history: The splay tree for vmmap was >> done about 8 years ago by alc@ to replace a simple linked list and was >> a huge improvement. The change in vmpage from a hash to the same splay >> tree as in vmmap was committed by dillon@ about 7.5 years ago with some >> involvement of alc@. >> ss >> > > Yes, and there is a substantial difference in the degree of locality of > access to these different structures, and thus the effectiveness of a splay > tree. When I did the last round of changes to the locking on the vm map, I > made some measurements of the splay tree's performance on a JVM running a > moderately large bioinformatics application. The upshot was that the > average number of map entries visited on an access to the vm map's splay > tree was less than the expected depth of a node in a perfectly balanced > tree. This is indeed an expected property of the splay tree. For the vmmap the root node hit rate, that is the least recently accessed node, is very high. Other nodes that are frequently accessed as well should be high up too. Nonetheless the churn rate on the nodes in tree rotations is considerable. The question now is whether we can get the same positive effect but with less negative side effects. Of particular concern is the CPU cache dirtying and thrashing by the splay rotations. Especially with today's and tomorrows many core, many socket, many threads/processes machines. Another concern is worst case behavior when the root node hit rate is low. Currently this is the case in vmpage for certain. It may also become a concern in vmmap with the use of memguard (fragmenting the address space) or an adversary trying to exploit worst case behavior on a shared host by mapping only every other page. This concern is validated when accepting the measurement by the 2008 SoC student Mayur (thanks to rdivacky@ for digging up his page): http://wiki.freebsd.org/MayurShardul/VM_Algorithm_Improvement In the measurements of his alternative radix trie implementation the lookup time is *230 times less* than for the splay tree (as measured by with the TSC). The description of the performed benchmark is extremely sparse and it is not clear what access pattern he simulated. Probably a random lookup pattern which is of course very bad for the splay tree. Any balanced tree will be better for random lookups. Whether his measured improvement is due to using a radix trie or if it would perform equally to a normal balanced binary tree I can't say (yet). > I teach class shortly. I'll provide more details later. -- Andre From owner-freebsd-hackers@FreeBSD.ORG Thu Sep 30 21:10:38 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6E5FB1065679 for ; Thu, 30 Sep 2010 21:10:38 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id D91088FC1B for ; Thu, 30 Sep 2010 21:10:37 +0000 (UTC) Received: (qmail 22456 invoked from network); 30 Sep 2010 21:02:33 -0000 Received: from localhost (HELO [127.0.0.1]) ([127.0.0.1]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 30 Sep 2010 21:02:33 -0000 Message-ID: <4CA4FCD1.6010709@freebsd.org> Date: Thu, 30 Sep 2010 23:10:41 +0200 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.9) Gecko/20100825 Thunderbird/3.1.3 MIME-Version: 1.0 To: Alfred Perlstein References: <4CA4BCD2.4070303@freebsd.org> <20100930183853.GX49807@elvis.mu.org> In-Reply-To: <20100930183853.GX49807@elvis.mu.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers , freebsd-current@freebsd.org Subject: Re: Examining the VM splay tree effectiveness X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Sep 2010 21:10:38 -0000 On 30.09.2010 20:38, Alfred Perlstein wrote: > Andre, > > Your observations on the effectiveness of the splay tree > mirror the concerns I have with it when I read about it. > > I have always wondered though if the splay-tree algorithm > was modified to only perform rotations when a lookup required > more than "N" traversals to reach a node. > > This would self-balance the tree and maintain cache without > the expense of writes for nearly all lookups. This would break the underlying assumption of the splay tree. A splay tree is not a balanced tree. With this approach you can easily get stuck in a sub-optimal situation with many lookups traversing N-1 nodes and not getting the cache benefit. When N nodes are traversed for an outlier, and the rotate happens, you rotate again on the next lookup or you get stuck in another sub-optimal situation. In effect you get the worst of an splay tree while not being able to gain on the amortization effect when the root node hit rate is high. > I'm wondering if you have the time/interest in rerunning > your tests, but modifying the algorithm to only rebalance > the splay if a lookup requires more than let's say 3, 5, 7 > or 10 traversals. As the assumption of the algorithm is broken I don't think it's useful to spend time on this. I'd rather try and add a last-match pointer to the RB tree to take home the locality effect in vmmap. -- Andre From owner-freebsd-hackers@FreeBSD.ORG Thu Sep 30 22:25:35 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2F7A31065672 for ; Thu, 30 Sep 2010 22:25:35 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id E7D048FC08 for ; Thu, 30 Sep 2010 22:25:34 +0000 (UTC) Received: by iwn34 with SMTP id 34so3688622iwn.13 for ; Thu, 30 Sep 2010 15:25:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=oSZNVKe6g29BXi6jso8kEjFdPxtdwUcnV0wniicAEtM=; b=IruQ5TeTskwu8GgPmIqSKOCSWhtvkencAfV7seHBrJRPG7kj+DhrDnu89x1J502WSO GaVLZuUayQNDYm9SAlep6Lcc8I6sLmx0rlBJ6XtNYF2HMwdezBfjS188I3xqdZfYo3OF cLHtuVK/MMG7WlOugVNbODW50vCKWF8J5JTWc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=jtUz2CsF4AVUtG6OkSC5oEy9SGAysQlPlI8f3tzH3Jcvyz9cY8otSWOjCSv0ALsnP+ Di43WhLlXLwZf5tZLY/nfQ8UJIKgvIcpQ2RX1+fDfRFEDJ2Fumc92evtL3oW8LOzijdC kn193KJAODZi8aVbhJyAgbwXXnaUJrMTKyiWw= MIME-Version: 1.0 Received: by 10.231.146.134 with SMTP id h6mr4514007ibv.170.1285885532971; Thu, 30 Sep 2010 15:25:32 -0700 (PDT) Sender: yanegomi@gmail.com Received: by 10.231.190.68 with HTTP; Thu, 30 Sep 2010 15:25:32 -0700 (PDT) In-Reply-To: <4CA4F28C.3020804@icyb.net.ua> References: <4CA4C63F.4070503@icyb.net.ua> <4CA4F28C.3020804@icyb.net.ua> Date: Thu, 30 Sep 2010 15:25:32 -0700 X-Google-Sender-Auth: yzFIQCExDd8IHISgr5kbBiL_PTw Message-ID: From: Garrett Cooper To: Andriy Gapon Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: sysctl for querying kmem_map->size X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Sep 2010 22:25:35 -0000 On Thu, Sep 30, 2010 at 1:26 PM, Andriy Gapon wrote: > on 30/09/2010 21:52 Garrett Cooper said the following: >> On Thu, Sep 30, 2010 at 10:17 AM, Andriy Gapon wrote: >>> >>> Here's a patch that adds a sysctl for querying kmem_map->size, which ma= y be useful >>> for system state/resources monitoring: >>> http://people.freebsd.org/~avg/sysctl-kmem_map_size.diff >>> >>> I am quite unsure about sizeof(kmem_map->size) =3D=3D sizeof(int) hack,= but I couldn't >>> think of other way to decide whether to use SYSCTL_ADD_UINT or SYSCTL_A= DD_ULONG >>> depending on real type behind vm_size_t. >> >> =A0 =A0 Is the base value of the field size_t? If so, then it's ulong on >> 64-bit archs and uint on 32-bit archs. Maybe it's a good time then to > > No, it's vm_size_t, but it's defined similarly to size_t I guess: > vm_size_t -> __vm_size_t -> {__uint32_t or __uint64_t depending on arch} Well... that's basically size_t right? At first inspection it seems silly to define it as __uint32_t or __uint64_t if it's really the same thing (in theory) as __size_t . Of course the vmem folks (like alc@) might have more context as to why things were done this way. >> actually get the sysctl and tunables work that I started on into base. >> I have a functioning and tested copy of the tunables work, but I'll >> need to do similar for the sysctls as well (des@ and I kind of got out >> of sync a few months back). > > I believe that this is the first time I hear about this project. It wasn't an official project. It was more or less an email thread that sort of turned into a challenge, which turned into a project :) : http://unix.derkeiler.com/Mailing-Lists/FreeBSD/hackers/2010-08/msg00054.ht= ml . As with all things though, I'm not a committer by any means, so unless someone works with me to review and commit my code, it will exist purely in the ether for all eternity. Part of the drawbacks of being in GSoC purgatory. Thanks! -Garrett From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 1 08:53:49 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3B816106566B for ; Fri, 1 Oct 2010 08:53:49 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 9CA7D8FC18 for ; Fri, 1 Oct 2010 08:53:48 +0000 (UTC) Received: (qmail 27992 invoked from network); 1 Oct 2010 08:45:37 -0000 Received: from unknown (HELO [62.48.0.92]) ([62.48.0.92]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 1 Oct 2010 08:45:37 -0000 Message-ID: <4CA5A1B1.7050902@freebsd.org> Date: Fri, 01 Oct 2010 10:54:09 +0200 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.9) Gecko/20100915 Thunderbird/3.1.4 MIME-Version: 1.0 To: freebsd-current@freebsd.org References: <4CA4BCD2.4070303@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers Subject: Re: Examining the VM splay tree effectiveness X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Oct 2010 08:53:49 -0000 On 30.09.2010 19:51, Ivan Voras wrote: > On 09/30/10 18:37, Andre Oppermann wrote: > >> Both the vmmap and page table make use of splay trees to manage the >> entries and to speed up lookups compared to long to traverse linked >> lists or more memory expensive hash tables. Some structures though >> do have an additional linked list to simplify ordered traversals. > > The property of splay tree requiring *writes* for nearly every read > really is a thorn in the eye for SMP. It seems to me that even if the > immediate benefits from converting to something else are not directly > observable, it will still be worth doing it. Fully agreed. > It's a shame that RCU is still a patent minefield :/ > > http://mirror.leaseweb.com/kernel/people/npiggin/patches/lockless/2.6.16-rc5/radix-intro.pdf I'm not convinced that RCU is the only scalable way of sharing a data structure across a possibly large number of CPU's. The term "lockless" is often used and frequently misunderstood. Having a lockess data structure *doesn't* mean that is either performant, scalable or both. It heavily depends on a number of additional factors. Many times "lockless" just replaces a simple lock/unlock cycle with a number of atomic operations on the data structure. This can easily backfire because an atomic operation just hides the computational complexity and also dirties the CPU's cache lines. Generally on cache coherent architectures almost all of the atomic operation is in HW with bus lock cycles, bus snooping and whatnot. While seemingly simple form the programmers point of view, the overhead and latency is still there. Needless to say that on more relaxed architectures (SPARC, Alpha, ...) the overhead is higher. Also important in the overall picture are the memory barrier semantics of locks. Some newer CPU's have started to provide hardware implemented lock managers and combine it with SMT features so that access to an already locked lock causes an immediate HW thread switch and on unlock a switch back. We also have rm_locks (read mostly locks) that do not require synchronization on read but have a more expensive write lock. In UMA we use a mix of global pools of elements with per-CPU caching of free elements. As always the best approach depends on the dominant access pattern of a structure. It all comes down to the amortization cost of the different locking or "lockless" strategies. It's also important to make sure that worst case behavior doesn't bring down the box. As a rule of thumb I use this: a) make sure the lock is held for only a small amount of time to avoid lock contention. b) do everything you can outside of the lock. c) if the lock is found to be heavily contended rethink the whole approach and check if other data structures can be used. d) minimize write accesses to memory in the lock protected shared data structure. e) PROFILE, DON'T SPECULATE! Measure the access pattern and measure the locking/data access strategy's cost in terms of CPU cycles consumed. f) on lookup heavy data structures avoid writing to memory and by it dirtying CPU caches. g) on modify heavy data structures avoid touching too many elements. h) on lookup and modify heavy data structure that are used across many CPU's all bets are off and a different data structure approach should be considered resulting ideally in case f). It all starts with the hypothesis that a data structure is not optimally locked. -- Andre From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 1 18:28:58 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 137A8106566C; Fri, 1 Oct 2010 18:28:58 +0000 (UTC) (envelope-from ed@hoeg.nl) Received: from mx0.hoeg.nl (unknown [IPv6:2a01:4f8:101:5343::aa]) by mx1.freebsd.org (Postfix) with ESMTP id 97E668FC0A; Fri, 1 Oct 2010 18:28:57 +0000 (UTC) Received: by mx0.hoeg.nl (Postfix, from userid 1000) id 553072A28D80; Fri, 1 Oct 2010 20:28:56 +0200 (CEST) Date: Fri, 1 Oct 2010 20:28:56 +0200 From: Ed Schouten To: Andre Oppermann Message-ID: <20101001182856.GF87427@hoeg.nl> References: <4CA4BCD2.4070303@freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="bNOIqPvWVsuhXMpy" Content-Disposition: inline In-Reply-To: <4CA4BCD2.4070303@freebsd.org> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-hackers , freebsd-current@freebsd.org Subject: Re: Examining the VM splay tree effectiveness X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Oct 2010 18:28:58 -0000 --bNOIqPvWVsuhXMpy Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Andre, * Andre Oppermann wrote: > A splay tree is an interesting binary search tree with insertion, > lookup and removal performed in O(log n) *amortized* time. With > the *amortized* time being the crucial difference to other binary trees. > On every access *including* lookup it rotates the tree to make the > just found element the new root node. For all gory details see: > http://en.wikipedia.org/wiki/Splay_tree Even though a red-black tree is quite good since it guarantees a $2 \log n$ upperbound, the problem is that it's quite computationally intensive. Maybe it would be worth looking at other types of balanced trees? For example, another type of tree which has only $O(\log n)$ amortized insertion/removal/lookup time, but could already be a lot better in practice, is a Treap. Greetings, --=20 Ed Schouten WWW: http://80386.nl/ --bNOIqPvWVsuhXMpy Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (FreeBSD) iEYEARECAAYFAkymKGgACgkQ52SDGA2eCwVl5wCdHzG4YpvaU5cPp4uU0BB5fUM8 rd4An2KVKMItdIuTQVBkEopnKVc8/X+B =WSCh -----END PGP SIGNATURE----- --bNOIqPvWVsuhXMpy-- From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 1 19:32:09 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F3F22106564A for ; Fri, 1 Oct 2010 19:32:08 +0000 (UTC) (envelope-from oppermann@networx.ch) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 681B08FC15 for ; Fri, 1 Oct 2010 19:32:07 +0000 (UTC) Received: (qmail 32509 invoked from network); 1 Oct 2010 18:57:12 -0000 Received: from localhost (HELO [127.0.0.1]) ([127.0.0.1]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 1 Oct 2010 18:57:12 -0000 Message-ID: <4CA630F5.9060500@networx.ch> Date: Fri, 01 Oct 2010 21:05:25 +0200 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.9) Gecko/20100825 Thunderbird/3.1.3 MIME-Version: 1.0 To: freebsd-current@freebsd.org, freebsd-hackers@freebsd.org Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-Mailman-Approved-At: Fri, 01 Oct 2010 19:36:11 +0000 Cc: Subject: Very interesting paper: An Analysis of Linux Scalability to many Cores X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Oct 2010 19:32:09 -0000 Just saw the link to a very interesting paper on SMP scalability. A very good read and highly relevant for our efforts as well. In certain areas we may already fare better, in others we still have some work to do. An Analysis of Linux Scalability to many Cores ABSTRACT This paper analyzes the scalability of seven system applications (Exim, memcached, Apache, PostgreSQL, gmake, Psearchy, and MapReduce) running on Linux on a 48-core computer. Except for gmake, all applications trigger scalability bottlenecks inside a recent Linux kernel. Using mostly standard parallel programming techniques— this paper introduces one new technique, sloppy counters— these bottlenecks can be removed from the kernel or avoided by changing the applications slightly. Modifying the kernel required in total 3002 lines of code changes. A speculative conclusion from this analysis is that there is no scalability reason to give up on traditional operating system organizations just yet. http://pdos.csail.mit.edu/papers/linux:osdi10.pdf -- Andre