From owner-freebsd-hackers@FreeBSD.ORG Mon Feb 15 11:17:06 2010 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3ABF0106568F for ; Mon, 15 Feb 2010 11:17:06 +0000 (UTC) (envelope-from sobomax@FreeBSD.org) Received: from sippysoft.com (gk1.360sip.com [72.236.70.240]) by mx1.freebsd.org (Postfix) with ESMTP id E36068FC1B for ; Mon, 15 Feb 2010 11:17:05 +0000 (UTC) Received: from [192.168.1.38] ([70.71.167.197]) (authenticated bits=0) by sippysoft.com (8.14.3/8.14.3) with ESMTP id o1FB11lS007253 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 15 Feb 2010 03:01:02 -0800 (PST) (envelope-from sobomax@FreeBSD.org) Message-ID: <4B79297D.9080403@FreeBSD.org> Date: Mon, 15 Feb 2010 03:01:17 -0800 From: Maxim Sobolev Organization: Sippy Software, Inc. User-Agent: Thunderbird 2.0.0.23 (Windows/20090812) MIME-Version: 1.0 To: freebsd-net@FreeBSD.org, FreeBSD Hackers Content-Type: text/plain; charset=KOI8-U; format=flowed Content-Transfer-Encoding: 7bit Cc: Jack Vogel Subject: Sudden mbuf demand increase and shortage under the load X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Feb 2010 11:17:06 -0000 Hi, Our company have a FreeBSD based product that consists of the numerous interconnected processes and it does some high-PPS UDP processing (30-50K PPS is not uncommon). We are seeing some strange periodic failures under the load in several such systems, which usually evidences itself in IPC (even through unix domain sockets) suddenly either breaking down or pausing and restoring only some time later (like 5-10 minutes). The only sign of failure I managed to find was the increase of the "requests for mbufs denied" in the netstat -m and number of total mbuf clusters (nmbclusters) raising up to the limit. I have tried to raise some network-related limits (most notably maxusers and nmbclusters), but it has not helped with the issue - it's still happening from time to time to us. Below you can find output from the netstat -m few minutes right after that shortage period - you see that somehow the system has allocated huge amount of memory for the network (700MB), with only tiny amount of that being actually in use. This is for the kern.ipc.nmbclusters: 302400. Eventually the system reclaims all that memory and goes back to its normal use of 30-70MB. This problem is killing us, so any suggestions are greatly appreciated. My current hypothesis is that due to some issues either with the network driver or network subsystem itself, the system goes insane and "eats" up all mbufs up to nmbclusters limit. But since mbufs are shared between network and local IPC, IPC goes down as well. We observe this issue with systems using both em(4) driver and igb(4) driver. I believe both drivers share the same design, however I am not sure if this is some kind of design flaw in the driver or part of a larger problem with the network subsystem. This happens on amd64 7.2-RELEASE and 7.3-PRERELEASE alike, with 8GB of memory. I have not tried upgrading to 8.0, this is production system so upgrading will not be easy. I don't believe there are some differences that let us hope that this problem will go away after upgrade, but I can try it as the last resort. As I said, this is very critical issue, so I can provide any additional debug information upon request. We are ready to go as far as paying somebody reasonable amount of money for tracking down and resolving the issue. Regards, -- Maksym Sobolyev Sippy Software, Inc. Internet Telephony (VoIP) Experts T/F: +1-646-651-1110 Web: http://www.sippysoft.com MSN: sales@sippysoft.com Skype: SippySoft [ssp-root@ds-467 /usr/src]$ netstat -m 17061/417669/434730 mbufs in use (current/cache/total) 10420/291980/302400/302400 mbuf clusters in use (current/cache/total/max) 10420/0 mbuf+clusters out of packet secondary zone in use (current/cache) 19/1262/1281/51200 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/25600 9k jumbo clusters in use (current/cache/total/max) 0/0/0/12800 16k jumbo clusters in use (current/cache/total/max) 25181K/693425K/718606K bytes allocated to network (current/cache/total) 1246681/129567494/67681640 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/0/0 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 0 calls to protocol drain routines [FEW MINUTES LATER] [ssp-root@ds-467 /usr/src]$ netstat -m 10001/84574/94575 mbufs in use (current/cache/total) 6899/6931/13830/302400 mbuf clusters in use (current/cache/total/max) 6899/6267 mbuf+clusters out of packet secondary zone in use (current/cache) 2/1151/1153/51200 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/25600 9k jumbo clusters in use (current/cache/total/max) 0/0/0/12800 16k jumbo clusters in use (current/cache/total/max) 16306K/39609K/55915K bytes allocated to network (current/cache/total) 1246681/129567494/67681640 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/0/0 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 0 calls to protocol drain routines From owner-freebsd-hackers@FreeBSD.ORG Mon Feb 15 11:46:15 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DD1ED106566B for ; Mon, 15 Feb 2010 11:46:15 +0000 (UTC) (envelope-from shrikanth07@gmail.com) Received: from qw-out-2122.google.com (qw-out-2122.google.com [74.125.92.26]) by mx1.freebsd.org (Postfix) with ESMTP id 993AB8FC0A for ; Mon, 15 Feb 2010 11:46:15 +0000 (UTC) Received: by qw-out-2122.google.com with SMTP id 8so409042qwh.7 for ; Mon, 15 Feb 2010 03:46:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type; bh=aq4Qr1D148yvMhrt6zw1kPK5E64rRpIKLYSvb6uaeTU=; b=vXgxFMCeEOmT3h+xL1gn0YOMOd9TE4TKPPCMHy10QjTrfhDG+a94Mw/rKL8/MADWgG RSekqzE8RH6WswsZXQvxGyHPtxd7pEWhE7xt/tdBcyHGZf5xW95ifW170Euz4zg+ttX4 j8aHshbWo5M+pvftE3a7Z3jMJ1TBYj37Mh7jE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=uKVElLuZ6rzUzySDlrlBB/934GVxc18bf/Zl3GETycOmFNVQmb1G4C9aSeobIIMe8e xLyfrcwe+quUznQGA8uZ1FmAZ/lEPhprXk++OnX5sDjm2cqwNVrVDgFAoPu4drNhcrh1 y09qcS7X35+0nBFUlwaxCmDNzTHYx4LJME0TY= MIME-Version: 1.0 Received: by 10.229.20.77 with SMTP id e13mr2015226qcb.14.1266232900644; Mon, 15 Feb 2010 03:21:40 -0800 (PST) Date: Mon, 15 Feb 2010 16:51:40 +0530 Message-ID: <291941b81002150321w7b0479beo1c6fec39ef6a7922@mail.gmail.com> From: Shrikanth Kamath To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Ktrace'ing kernel threads X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Feb 2010 11:46:15 -0000 Can ktrace trace another kernel thread which has roughly the semantics as below, right now it does not hit any of the designated interesting points that ktrace is built for, but what if I could define those, will ktrace still allow tracing another kernel thread? thread(client_info) { ... ... build_msg(client_info); /* this will malloc a mbuf and fill the data in it */ ... sosend(client_info); } I want to time the entry/return of build_msg, and the time sosend, dump client_info (some specific fields). From owner-freebsd-hackers@FreeBSD.ORG Mon Feb 15 14:53:21 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DC7DE1065693 for ; Mon, 15 Feb 2010 14:53:21 +0000 (UTC) (envelope-from fergleiser@yahoo.com) Received: from web31703.mail.mud.yahoo.com (web31703.mail.mud.yahoo.com [68.142.201.183]) by mx1.freebsd.org (Postfix) with SMTP id 98C768FC12 for ; Mon, 15 Feb 2010 14:53:21 +0000 (UTC) Received: (qmail 98102 invoked by uid 60001); 15 Feb 2010 14:53:20 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1266245600; bh=6kV6YzfdJMBaTA+I6jBpsquA70in4x3A00cdxTNGN5Y=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type; b=lh6lXfuP9cSRJRunHpCpoBQpHHXviyJizupqRiL6e+MMfS6zXu+S6bD6jfMYCuU9hg99jBxm1nTIjI/1JoXVjwPsBc0tu2QtBDBg8LWy65gMjy5aUY/bNAMZKp7bw+qIGduWxan6kNfP6XO8HnM2vOH5c2YlEWuCOdRQO/QQeYU= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type; b=Sve5baNOCyUM97F3Q0e5ChsyBd9qSsiVcKVjhz0Svr7vvVbAAjgdGOIJm3vukh1x6ST8RW2rfNYRNjIY124SSuvAR0HnvQ8pQDK7mn3uUWJbct6IsWOda4hYt60pQeagZxETEJZczvrFHqJGST84uiQu0OSX6qPwpX6m1R01D+s=; Message-ID: <909320.96307.qm@web31703.mail.mud.yahoo.com> X-YMail-OSG: w5whr7UVM1nNyjwrH71msOKe.0X.928G2fFlh29m2H0xpUzHXt0PEjrquteyveC.V_6D6IFCwkKWPJRCbU3HLQuiBtFlNvy0O4MrpWWBz4tOv8l13ZINRuTmNttuUT.bCbZwMRxA9TaZY4xeGlp553tBxsHo2sTv5b849xDYj.0IwimFwmi0P4wiGsWAvQisTb_IpMGO6Vq8KsWBo_iTRGansFiqsGh4iMmewXi1gPTlmvKxUzKrRAATa8n_0XCK86ydOzgHitlTw2XHUG0.JJlAflB.TjYSQ2eqStQfCqoEnwNomCirKwTzQk3bf.3WVqaZ.VcP7QvYdvRWkPJ8gC7jOrVriM_N7_.UTctSHMubuVtIxvnhlm2HtCDm Received: from [190.136.181.68] by web31703.mail.mud.yahoo.com via HTTP; Mon, 15 Feb 2010 06:53:20 PST X-Mailer: YahooMailRC/300.3 YahooMailWebService/0.8.100.260964 References: <291941b81002150321w7b0479beo1c6fec39ef6a7922@mail.gmail.com> Date: Mon, 15 Feb 2010 06:53:20 -0800 (PST) From: Fernando Gleiser To: Shrikanth Kamath , freebsd-hackers@freebsd.org In-Reply-To: <291941b81002150321w7b0479beo1c6fec39ef6a7922@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Subject: Re: Ktrace'ing kernel threads X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Feb 2010 14:53:21 -0000 ----- Original Message ---- > From: Shrikanth Kamath > To: freebsd-hackers@freebsd.org > Sent: Mon, February 15, 2010 8:21:40 AM > Subject: Ktrace'ing kernel threads > > Can ktrace trace another kernel thread which has roughly the semantics as > below, right now it > does not hit any of the designated interesting points that ktrace is built > for, but what if I could define those, > will ktrace still allow tracing another kernel thread? Yo can do that easily with dtrace #!/usr/sbin/dtrace -s fbt::build_msg:entry { printf("%d\n", timestamp) } fbt::build_msg:return { printf("%d\n", timestamp) } or if you want to get an histogram of the call time distributions: fbt::build_msg:entry { self->ts=timestamp; } fbt::build_msg:return { @[probefunc]=quantize(timestamp - self->ts); } Those are from the top of my head, I don't have a FreeBSD system at hand to test them, but I hope you'll get the idea Fer From owner-freebsd-hackers@FreeBSD.ORG Mon Feb 15 14:50:28 2010 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C406710656A4; Mon, 15 Feb 2010 14:50:28 +0000 (UTC) (envelope-from babkin@verizon.net) Received: from vms173005pub.verizon.net (vms173005pub.verizon.net [206.46.173.5]) by mx1.freebsd.org (Postfix) with ESMTP id A31D18FC22; Mon, 15 Feb 2010 14:50:28 +0000 (UTC) Received: from verizon.net ([unknown] [173.54.27.21]) by vms173005.mailsrvcs.net (Sun Java(tm) System Messaging Server 7u2-7.02 32bit (built Apr 16 2009)) with ESMTPA id <0KXV00L0DYFDI671@vms173005.mailsrvcs.net>; Mon, 15 Feb 2010 07:50:02 -0600 (CST) Sender: root Message-id: <4B79205B.619A0A1A@verizon.net> Date: Mon, 15 Feb 2010 05:22:19 -0500 From: Sergey Babkin X-Mailer: Mozilla 4.7 [en] (X11; U; FreeBSD 4.7-RELEASE i386) X-Accept-Language: en, ru MIME-version: 1.0 To: Maxim Sobolev References: <4B79297D.9080403@FreeBSD.org> Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7bit X-Mailman-Approved-At: Mon, 15 Feb 2010 16:22:20 +0000 Cc: freebsd-net@FreeBSD.org, Jack Vogel , FreeBSD Hackers Subject: Re: Sudden mbuf demand increase and shortage under the load X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Feb 2010 14:50:28 -0000 Maxim Sobolev wrote: > > Hi, > > Our company have a FreeBSD based product that consists of the numerous > interconnected processes and it does some high-PPS UDP processing > (30-50K PPS is not uncommon). We are seeing some strange periodic > failures under the load in several such systems, which usually evidences > itself in IPC (even through unix domain sockets) suddenly either > breaking down or pausing and restoring only some time later (like 5-10 > minutes). The only sign of failure I managed to find was the increase of > the "requests for mbufs denied" in the netstat -m and number of total > mbuf clusters (nmbclusters) raising up to the limit. As a simple idea: UDP is not flow-controlled. So potentially nothing stops an application from sending the packets as fast as it can. If it's faster than the network card can process, they would start collecting. So this might be worth a try as a way to reproduce the problem and see if the system has a safeguard against it or not. Another possibility: what happens if a process is bound to an UDP socket but doesn't actually read the data from it? FreeBSD used to be pretty good at it, just throwing away the data beyond a certain limit, SVR4 was running out of network memory. But it might have changed, so might be worth a look too. -SB From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 16 00:55:09 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 533E01065672 for ; Tue, 16 Feb 2010 00:55:09 +0000 (UTC) (envelope-from yanefbsd@gmail.com) Received: from mail-pz0-f202.google.com (mail-pz0-f202.google.com [209.85.222.202]) by mx1.freebsd.org (Postfix) with ESMTP id 2A7D48FC08 for ; Tue, 16 Feb 2010 00:55:08 +0000 (UTC) Received: by pzk40 with SMTP id 40so2680052pzk.7 for ; Mon, 15 Feb 2010 16:55:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=MVX0tpaFntXZ5G6sjvFfy4lpgPH/iON9ZyPyiklq2Tg=; b=mLo8iTqOkL+bBUGEFf8cTaSGONCrSRWdV3k3mTq+OMAwhBHul8t2yvhDD0g4jHwPtR B9WyJXHko2m4fDgmn8DJJNQkDYsZ7FPaRpCnr8FlwqeFj3peaX0eDiqc43+0p4JH0JtR N0x4txEN6Vokbghc0yI5e5EYv7rzul0V6GpPE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=u6GrSRhY0PxPwzuqStsROOt5vWfFTxUjJU+Cp7d+fpaA5/D8v7Q7bO2osHDQXArif0 c5jJjoMR2/jFBqXfYnsj0tpFlBYXvFLeZcd/tkqWCzlg1xzmLXcs+c9oFYQFGFo2c2Rs Q7Yo9BY49EtLE2Yhmmwdt6oniPaGFxomeFhog= MIME-Version: 1.0 Received: by 10.142.59.21 with SMTP id h21mr3900246wfa.182.1266281708151; Mon, 15 Feb 2010 16:55:08 -0800 (PST) In-Reply-To: <364299f41002151649y2e4d4120p918759afb1fd8f6c@mail.gmail.com> References: <364299f41002151649y2e4d4120p918759afb1fd8f6c@mail.gmail.com> Date: Mon, 15 Feb 2010 16:55:08 -0800 Message-ID: <7d6fde3d1002151655q184c8a21k8a0c6c07b9b0ae79@mail.gmail.com> From: Garrett Cooper To: FreeBSD-Hackers Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: read(1) garbage when input redirected from make incorrectly X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2010 00:55:09 -0000 Hi Hackers, =A0 =A0I accidentally reproduced the following after executing read properly in a pipeline with make: [garrcoop@garrcoop-fbsd /usr/home/garrcoop]$ read DESTDIR SRCCONF < /usr/bin/make -V DESTDIR -V SRCCONF bash: read: `-V': not a valid identifier [garrcoop@garrcoop-fbsd /usr/home/garrcoop]$ echo $DESTDIR =A0ELF [garrcoop@garrcoop-fbsd /usr/home/garrcoop]$ hexdump -C foo 00000000 7f 45 4c 46 01 01 01 0a |.ELF....| 00000008 [garrcoop@garrcoop-fbsd /usr/home/garrcoop]$ =A0 =A0Is this an issue to be concerned about apart from cosmetic noise, i.e. potential buffer access problem? I see the same garbage from bash/coreutils on RHEL 4.6 as well as read(1) and /bin/sh on FreeBSD with RELENG_8, so the issue appears to be consistent on multiple OSes... Thanks, -Garrett From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 16 00:57:01 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1FD97106566B for ; Tue, 16 Feb 2010 00:57:01 +0000 (UTC) (envelope-from yanefbsd@gmail.com) Received: from mail-px0-f176.google.com (mail-px0-f176.google.com [209.85.216.176]) by mx1.freebsd.org (Postfix) with ESMTP id EA3478FC15 for ; Tue, 16 Feb 2010 00:57:00 +0000 (UTC) Received: by pxi6 with SMTP id 6so4696446pxi.14 for ; Mon, 15 Feb 2010 16:57:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=fRDh6jgK0yoS3wVUIVIdY5lETTqr7OLuKcdtfq3HMvc=; b=CrElESefyTh+0fdLxbCyPFf6d5IFPq2d4DmtKgY61zV4KG9Wk8HSp0StC5yTdxo/2R ZLrMv5Hi8VXsg20mqIpOYjX0lepZxGxhs42b1Uu9WapwRchSybNOfgofbTKV8UwL3W73 O72dVDMUl9d0GZXW4rcXqGtiLbUkZiUj9oCHE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=wIA+KVp7ul8ef0U9HZTvSJ6rFgntaA7gy0eyKRryu5zH7sUS7QmINcP9g+77RP/1XU Z4kRfmfsAYkUJ2BxzJkgKeT6zbrNp+PgRGSIr8oF+k2xNDs3mxzdXwiclgZrgm6L8Xrn flzVAkkBOTkkCL9iPZJV46Hbw4mGVWeT5iezI= MIME-Version: 1.0 Received: by 10.142.196.14 with SMTP id t14mr3898279wff.326.1266281820362; Mon, 15 Feb 2010 16:57:00 -0800 (PST) In-Reply-To: <7d6fde3d1002151655q184c8a21k8a0c6c07b9b0ae79@mail.gmail.com> References: <364299f41002151649y2e4d4120p918759afb1fd8f6c@mail.gmail.com> <7d6fde3d1002151655q184c8a21k8a0c6c07b9b0ae79@mail.gmail.com> Date: Mon, 15 Feb 2010 16:57:00 -0800 Message-ID: <7d6fde3d1002151657q560b701bre44419d56e61f7ac@mail.gmail.com> From: Garrett Cooper To: FreeBSD-Hackers Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: read(1) garbage when input redirected from make incorrectly X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2010 00:57:01 -0000 On Mon, Feb 15, 2010 at 4:55 PM, Garrett Cooper wrote: > Hi Hackers, > =A0 =A0I accidentally reproduced the following after executing read > properly in a pipeline with make: s/properly/improperly/ > [garrcoop@garrcoop-fbsd /usr/home/garrcoop]$ read DESTDIR SRCCONF < > /usr/bin/make -V DESTDIR -V SRCCONF > bash: read: `-V': not a valid identifier > [garrcoop@garrcoop-fbsd /usr/home/garrcoop]$ echo $DESTDIR > =A0ELF > [garrcoop@garrcoop-fbsd /usr/home/garrcoop]$ hexdump -C foo > 00000000 =A07f 45 4c 46 01 01 01 0a =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 |.ELF....| > 00000008 > [garrcoop@garrcoop-fbsd /usr/home/garrcoop]$ And just for completeness because I executed the above directly in bash... [root@garrcoop-fbsd /usr/home/garrcoop]# sh -c 'read DESTDIR SRCCONF < /usr/bin/make -V DESTDIR -V SRCCONF; echo $DESTDIR > foo; hexdump -C foo' read: -V: bad variable name 00000000 7f 45 4c 46 01 01 01 0a |.ELF....| 00000008 > =A0 =A0Is this an issue to be concerned about apart from cosmetic noise, > i.e. potential buffer access problem? I see the same garbage from > bash/coreutils on RHEL 4.6 as well as read(1) and /bin/sh on FreeBSD > with RELENG_8, so the issue appears to be consistent on multiple > OSes... Thanks, -Garrett From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 16 02:12:35 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 35D70106566C for ; Tue, 16 Feb 2010 02:12:35 +0000 (UTC) (envelope-from janm@transactionware.com) Received: from mail.transactionware.com (mail.transactionware.com [203.14.245.7]) by mx1.freebsd.org (Postfix) with SMTP id 73A018FC14 for ; Tue, 16 Feb 2010 02:12:33 +0000 (UTC) Received: (qmail 39044 invoked from network); 16 Feb 2010 01:53:37 -0000 Received: from midgard.transactionware.com (192.168.1.55) by dm.transactionware.com with SMTP; 16 Feb 2010 01:53:37 -0000 Received: (qmail 17613 invoked by uid 907); 16 Feb 2010 01:45:51 -0000 Received: from jmmacpro.transactionware.com (HELO jmmacpro.transactionware.com) (192.168.1.33) by midgard.transactionware.com (qpsmtpd/0.82) with ESMTP; Tue, 16 Feb 2010 12:45:51 +1100 Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: Jan Mikkelsen In-Reply-To: <7d6fde3d1002151655q184c8a21k8a0c6c07b9b0ae79@mail.gmail.com> Date: Tue, 16 Feb 2010 12:45:50 +1100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <364299f41002151649y2e4d4120p918759afb1fd8f6c@mail.gmail.com> <7d6fde3d1002151655q184c8a21k8a0c6c07b9b0ae79@mail.gmail.com> To: Garrett Cooper X-Mailer: Apple Mail (2.1077) X-Mailman-Approved-At: Tue, 16 Feb 2010 03:18:40 +0000 Cc: FreeBSD-Hackers Subject: Re: read(1) garbage when input redirected from make incorrectly X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2010 02:12:35 -0000 On 16/02/2010, at 11:55 AM, Garrett Cooper wrote: > Hi Hackers, > I accidentally reproduced the following after executing read > properly in a pipeline with make: >=20 > [garrcoop@garrcoop-fbsd /usr/home/garrcoop]$ read DESTDIR SRCCONF < > /usr/bin/make -V DESTDIR -V SRCCONF > bash: read: `-V': not a valid identifier > [garrcoop@garrcoop-fbsd /usr/home/garrcoop]$ echo $DESTDIR > ELF > [garrcoop@garrcoop-fbsd /usr/home/garrcoop]$ hexdump -C foo > 00000000 7f 45 4c 46 01 01 01 0a |.ELF....| > 00000008 > [garrcoop@garrcoop-fbsd /usr/home/garrcoop]$ >=20 > Is this an issue to be concerned about apart from cosmetic noise, > i.e. potential buffer access problem? I see the same garbage from > bash/coreutils on RHEL 4.6 as well as read(1) and /bin/sh on FreeBSD > with RELENG_8, so the issue appears to be consistent on multiple > OSes... > Thanks, > -Garrett I think you meant to type: make -V DESTDIR -V SRCCONF | read DESTDIR SRCCONF What you are actually doing is feeding the contents of the make binary = into: read DESTDIR SRCCONF -V DESTDIR -V SRCCONF and the shell is correctly complaining about '-V' not being a valid = identifier, and populating DESTDIR with data it got from the binary. Jan From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 16 06:19:22 2010 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D18E21065676; Tue, 16 Feb 2010 06:19:22 +0000 (UTC) (envelope-from sobomax@FreeBSD.org) Received: from sippysoft.com (gk1.360sip.com [72.236.70.240]) by mx1.freebsd.org (Postfix) with ESMTP id 23B1E8FC1C; Tue, 16 Feb 2010 06:19:22 +0000 (UTC) Received: from [192.168.1.38] ([70.71.167.197]) (authenticated bits=0) by sippysoft.com (8.14.3/8.14.3) with ESMTP id o1G6JHTL017344 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 15 Feb 2010 22:19:18 -0800 (PST) (envelope-from sobomax@FreeBSD.org) Message-ID: <4B7A38F5.3090404@FreeBSD.org> Date: Mon, 15 Feb 2010 22:19:33 -0800 From: Maxim Sobolev Organization: Sippy Software, Inc. User-Agent: Thunderbird 2.0.0.23 (Windows/20090812) MIME-Version: 1.0 To: Sergey Babkin References: <4B79297D.9080403@FreeBSD.org> <4B79205B.619A0A1A@verizon.net> In-Reply-To: <4B79205B.619A0A1A@verizon.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Alfred Perlstein , freebsd-net@FreeBSD.org, "David G. Lawrence" , Jack Vogel , FreeBSD Hackers Subject: Re: Sudden mbuf demand increase and shortage under the load X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2010 06:19:23 -0000 Sergey Babkin wrote: > Maxim Sobolev wrote: >> Hi, >> >> Our company have a FreeBSD based product that consists of the numerous >> interconnected processes and it does some high-PPS UDP processing >> (30-50K PPS is not uncommon). We are seeing some strange periodic >> failures under the load in several such systems, which usually evidences >> itself in IPC (even through unix domain sockets) suddenly either >> breaking down or pausing and restoring only some time later (like 5-10 >> minutes). The only sign of failure I managed to find was the increase of >> the "requests for mbufs denied" in the netstat -m and number of total >> mbuf clusters (nmbclusters) raising up to the limit. > > As a simple idea: UDP is not flow-controlled. So potentially > nothing stops an application from sending the packets as fast > as it can. If it's faster than the network card can process, > they would start collecting. So this might be worth a try > as a way to reproduce the problem and see if the system has > a safeguard against it or not. > > Another possibility: what happens if a process is bound to > an UDP socket but doesn't actually read the data from it? > FreeBSD used to be pretty good at it, just throwing away > the data beyond a certain limit, SVR4 was running out of > network memory. But it might have changed, so might be > worth a look too. Thanks. Yes, the latter could be actually the case. The former is less likely since the system doesn't generate so much traffic by itself, but rather relays what it receives from the network pretty much in 1:1 ratio. It could happen though, if somehow the output path has been stalled. However, netstat -I igb0 shows zero Oerrs, which I guess means that we can rule that out too, unless there is some bug in the driver. So we are looking for potential issues that can cause UDP forwarding application to stall and not dequeue packets on time. So far we have identified some culprits in application logic that can cause such stalls in the unlikely event of gettimeofday() time going backwards. I've seen some messages from ntpd around the time of the problem, although it's unclear whether those are result of the that mbuf shortage or could indicate the root issue. We've also added some debug output to catch any abnormalities in the processing times. In any case I am a little bit surprised on how easy the FreeBSD can let mbuf storage to overflow. I'd expect it to be more aggressive in dropping things received from network once one application stalls. Combined with the fact that we apparently use shared storage for different kinds of network activity and perhaps IPC too, this gives an easy opportunity for DOS attacks. To me, separate limits for separate protocols or even classes of traffic (i.e. local/remote) would make much sense. Thanks to everybody for useful tips and suggestions, I will do more research along the lines and let you know once we either resolve the case or when I have more diagnostic output. Regards, -- Maksym Sobolyev Sippy Software, Inc. Internet Telephony (VoIP) Experts T/F: +1-646-651-1110 Web: http://www.sippysoft.com MSN: sales@sippysoft.com Skype: SippySoft From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 16 10:33:46 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 781EF106566B for ; Tue, 16 Feb 2010 10:33:46 +0000 (UTC) (envelope-from osharoiko@gmail.com) Received: from mail-ew0-f225.google.com (mail-ew0-f225.google.com [209.85.219.225]) by mx1.freebsd.org (Postfix) with ESMTP id 0B9958FC0C for ; Tue, 16 Feb 2010 10:33:45 +0000 (UTC) Received: by ewy25 with SMTP id 25so610550ewy.3 for ; Tue, 16 Feb 2010 02:33:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:subject:from:to:content-type :date:message-id:mime-version:x-mailer:content-transfer-encoding; bh=Ainzcc6JLQBH+3gCC5vVsbc5GFblWncUAzgBo4B5IZ0=; b=i5IAOk6N4t5SaZLDrxIVC9eQwNTe1VH+R6cYC6ez5CtFyrp4d4sc5MVJeN6+/G0+Zn 5gMHa4lDsh8NzKKjGk6q//6e7nIvhnNAuf1yEX4cfVSpXdHom1B9EGPDzfq5943FQ9lb tVIcJRQ9kyQZ0KtuK9ykKSZyIIGyAAREI4Ogs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=subject:from:to:content-type:date:message-id:mime-version:x-mailer :content-transfer-encoding; b=i+WgAoGRExWPSBCtAOUnQFNHaqeUCTDZPI46E6tOE7KPcLKUkmwyuCEqtEgBWiFfWt Y2XYphjAFEpu3huE5dAUX61pW9YZBIbJmUFr2rGIo7H9XFOAukyaqcjVdq+7px6bX33o Y9s4ubcQHVBr7YAcheruP3g+TWyCjW+tekJPA= Received: by 10.213.0.212 with SMTP id 20mr3861804ebc.41.1266314576487; Tue, 16 Feb 2010 02:02:56 -0800 (PST) Received: from ?195.208.252.154? (brain.cc.rsu.ru [195.208.252.154]) by mx.google.com with ESMTPS id 5sm10505207eyf.0.2010.02.16.02.02.55 (version=TLSv1/SSLv3 cipher=RC4-MD5); Tue, 16 Feb 2010 02:02:55 -0800 (PST) From: Oleg Sharoyko To: freebsd-scsi@freebsd.org, freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" Date: Tue, 16 Feb 2010 13:02:50 +0300 Message-ID: <1266314570.73716.42.camel@brain.cc.rsu.ru> Mime-Version: 1.0 X-Mailer: Evolution 2.28.2 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: Subject: Qlogic (isp) cannot login into fabric after link loss X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2010 10:33:46 -0000 Hi! I'm sorry for cross-posting, but I'm in trouble with isp driver and I really need your help. Here we have IBM blade with Qlogic (seems to be 2462s card) and EMC CX3-40 both connected to Brocade DS4900. Everything seems to be ok, until FC link between switch and server fails. When it later restores isp cannot anymore log into fabric. I have a complete debug log of isp output. Here is small excerpt from it which I think is relevant to my problem: isp0: Chan 0 got 3 ports back from name server isp0: Chan 0 skip ourselves on @ PortID 0x011500 isp0: Chan 0 Checking Fabric Port 0x010e00 isp0: IN mbox 0 = 0x0064 isp0: IN mbox 1 = 0x0003 isp0: IN mbox 2 = 0x05e8 isp0: IN mbox 3 = 0xd000 isp0: IN mbox 6 = 0x0000 isp0: IN mbox 7 = 0x0000 isp0: IN mbox 9 = 0x0000 isp0: IN mbox 10 = 0x0000 isp0: RISC2HOST ISR 0x40000010 isp0: RISC2HOST ISR 0x40068011 isp0: OUT mbox 0 = 0x4006 isp0: IOCB LOGX: isp0: 0x00000000: 52 01 00 00 ff ff ff ff 00 00 03 00 00 00 00 00 isp0: 0x00000010: 00 0e 01 00 00 00 00 00 00 00 00 00 00 00 00 00 isp0: 0x00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 isp0: 0x00000030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 isp0: IN mbox 0 = 0x0054 isp0: IN mbox 1 = 0x0040 isp0: IN mbox 2 = 0x05e8 isp0: IN mbox 3 = 0xd000 isp0: IN mbox 6 = 0x0000 isp0: IN mbox 7 = 0x0000 isp0: RISC2HOST ISR 0x40060011 isp0: RISC2HOST ISR 0x40060011 isp0: RISC2HOST ISR 0x40060011 isp0: RISC2HOST ISR 0x40008010 isp0: OUT mbox 0 = 0x4000 isp0: IOCB LOGX response: isp0: 0x00000000: 52 01 00 00 ff ff ff ff 31 00 03 00 00 00 00 00 isp0: 0x00000010: 00 0e 01 02 1a 00 00 00 01 00 00 00 00 00 00 00 isp0: 0x00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 isp0: 0x00000030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 isp0: Chan 0 PLOGX PortID 0x010e00 to N-Port handle 0x3: already logged in with N-Port handle 0x1 isp0: IN mbox 0 = 0x0064 isp0: IN mbox 1 = 0x0001 isp0: IN mbox 2 = 0x05e8 isp0: IN mbox 3 = 0xd000 isp0: IN mbox 6 = 0x0000 isp0: IN mbox 7 = 0x0000 isp0: IN mbox 9 = 0x0000 isp0: IN mbox 10 = 0x0000 isp0: RISC2HOST ISR 0x40000010 isp0: RISC2HOST ISR 0x40008010 isp0: OUT mbox 0 = 0x4000 isp0: Chan 0 Port 0x010e00 flags 0x0 curstate 7 isp0: Chan 0 new device 0x010e00@0x1 disappeared As far as I can tell from this log and isp.c things are happening in the following order: 1. Get the list of devices on fabric 2. Try to log into each device (excluding self) 3. The result is failure with code "already logged in" and old login handle 4. Try to login with that handle and expect that to work, but it fails with code PDB2400_STATE_PORT_UNAVAIL. Bear in mind, that I'm not anywhere close to FC expert, so I may be totally wrong here. So who is wrong and would it be possible to do resolve this issue? I have an access to all the components (blade, fc switch and storage system) and can provide additional information if it's needed. I'm really stuck and would greatly appreciate any help! Thanks in advance. -- Oleg Sharoyko From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 16 11:49:55 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 383A9106568D for ; Tue, 16 Feb 2010 11:49:55 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id EF4C08FC0C for ; Tue, 16 Feb 2010 11:49:54 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id 072CF1FFC53; Tue, 16 Feb 2010 11:49:54 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id D5BC6844BD; Tue, 16 Feb 2010 12:49:53 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Garrett Cooper References: <364299f41002151649y2e4d4120p918759afb1fd8f6c@mail.gmail.com> <7d6fde3d1002151655q184c8a21k8a0c6c07b9b0ae79@mail.gmail.com> Date: Tue, 16 Feb 2010 12:49:53 +0100 In-Reply-To: <7d6fde3d1002151655q184c8a21k8a0c6c07b9b0ae79@mail.gmail.com> (Garrett Cooper's message of "Mon, 15 Feb 2010 16:55:08 -0800") Message-ID: <86eikljt7i.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.95 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD-Hackers Subject: Re: read(1) garbage when input redirected from make incorrectly X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2010 11:49:55 -0000 Garrett Cooper writes: > $ read DESTDIR SRCCONF < /usr/bin/make -V DESTDIR -V SRCCONF The LHS of < is a command, the RHS is the name of the file to be read. After that, you can have further redirections, a command separator (semicolon, single or double ampersand, single or double pipe etc.), or, depending on context, various other stuff such as a paren, bracket, backquote etc. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 16 17:01:29 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E76CA106566C for ; Tue, 16 Feb 2010 17:01:29 +0000 (UTC) (envelope-from yanefbsd@gmail.com) Received: from mail-pz0-f202.google.com (mail-pz0-f202.google.com [209.85.222.202]) by mx1.freebsd.org (Postfix) with ESMTP id BA3C98FC0A for ; Tue, 16 Feb 2010 17:01:29 +0000 (UTC) Received: by pzk40 with SMTP id 40so3415914pzk.7 for ; Tue, 16 Feb 2010 09:01:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=SlhBSGTfcaurp68wJeIjSPtbmNuv+MjRWiARh6viueA=; b=nUDTEPneyoY/fWDGQTuNsYTDpq5xe6WzjPZ4D/GIq+0mKp7XX2lNztWdAOENKkdQgx 0xkpYcv0YNDYrdroj34MWNhFISwVkHUPSn7Ycu4xvWR/CMP5KHHXI9yqMgPzSPEuN7kw UsOHLILl7Snh3CS85L1AA3anj/ualk+kMgV8Q= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=Are2ilE8DX8YRGrHjA2u17ADKT4BCvuglrZVImZOPvoLrIw4Bd4ko/6OQ3y3IuBulR lANLcF8R17WcCIRM0cDfhLMw+Sf6IUupr6T2updS+8xG2S6Kq7YCQgoR7yO2t+hCruGq 1cthVAy/HDIo5TQ4ZZHrF+pT+X26wA3giHGtQ= MIME-Version: 1.0 Received: by 10.142.55.13 with SMTP id d13mr4513989wfa.50.1266339689134; Tue, 16 Feb 2010 09:01:29 -0800 (PST) In-Reply-To: References: <364299f41002151649y2e4d4120p918759afb1fd8f6c@mail.gmail.com> <7d6fde3d1002151655q184c8a21k8a0c6c07b9b0ae79@mail.gmail.com> Date: Tue, 16 Feb 2010 09:01:29 -0800 Message-ID: <7d6fde3d1002160901r600bb514u4a3219d2e59b16aa@mail.gmail.com> From: Garrett Cooper To: Jan Mikkelsen Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD-Hackers Subject: Re: read(1) garbage when input redirected from make incorrectly X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2010 17:01:30 -0000 On Mon, Feb 15, 2010 at 5:45 PM, Jan Mikkelsen w= rote: > On 16/02/2010, at 11:55 AM, Garrett Cooper wrote: > >> Hi Hackers, >> =A0 =A0I accidentally reproduced the following after executing read >> properly in a pipeline with make: >> >> [garrcoop@garrcoop-fbsd /usr/home/garrcoop]$ read DESTDIR SRCCONF < >> /usr/bin/make -V DESTDIR -V SRCCONF >> bash: read: `-V': not a valid identifier >> [garrcoop@garrcoop-fbsd /usr/home/garrcoop]$ echo $DESTDIR >> =A0ELF >> [garrcoop@garrcoop-fbsd /usr/home/garrcoop]$ hexdump -C foo >> 00000000 =A07f 45 4c 46 01 01 01 0a =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 |.ELF....| >> 00000008 >> [garrcoop@garrcoop-fbsd /usr/home/garrcoop]$ >> >> =A0 =A0Is this an issue to be concerned about apart from cosmetic noise, >> i.e. potential buffer access problem? I see the same garbage from >> bash/coreutils on RHEL 4.6 as well as read(1) and /bin/sh on FreeBSD >> with RELENG_8, so the issue appears to be consistent on multiple >> OSes... >> Thanks, >> -Garrett > > I think you meant to type: > > =A0 =A0make -V DESTDIR -V SRCCONF | read DESTDIR SRCCONF > > What you are actually doing is feeding the contents of the make binary in= to: > > =A0 =A0read DESTDIR SRCCONF -V DESTDIR -V SRCCONF > > and the shell is correctly complaining about '-V' not being a valid ident= ifier, and populating DESTDIR with data it got from the binary. Yes, now it all makes sense to me. It's just interesting why read (1) is the only thing I've found (so far) that has this behavior, but it's probably a biproduct of how it scans its arguments on stdin: [gcooper@optimus ~]$ python -c 'import sys; sys.stdin.read()' < make -V bash: make: No such file or directory [gcooper@optimus ~]$ perl -e 'while (<>) { print; }' < make -V bash: make: No such file or directory Thanks, -Garrett From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 16 18:03:13 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B36481065676 for ; Tue, 16 Feb 2010 18:03:13 +0000 (UTC) (envelope-from devin@spamcop.net) Received: from mail.distalzou.net (203.141.139.231.static.zoot.jp [203.141.139.231]) by mx1.freebsd.org (Postfix) with ESMTP id 76B908FC15 for ; Tue, 16 Feb 2010 18:03:13 +0000 (UTC) Received: from plexi.pun-pun.prv ([192.168.7.29]) by mail.distalzou.net with esmtps (TLSv1:AES256-SHA:256) (Exim 4.71 (FreeBSD)) (envelope-from ) id 1NhRVA-000H5k-1T; Wed, 17 Feb 2010 02:46:24 +0900 Date: Wed, 17 Feb 2010 02:46:23 +0900 (JST) From: Tod McQuillin X-X-Sender: devin@plexi.pun-pun.prv To: Garrett Cooper In-Reply-To: <7d6fde3d1002160901r600bb514u4a3219d2e59b16aa@mail.gmail.com> Message-ID: <20100217024315.O62495@plexi.pun-pun.prv> References: <364299f41002151649y2e4d4120p918759afb1fd8f6c@mail.gmail.com> <7d6fde3d1002151655q184c8a21k8a0c6c07b9b0ae79@mail.gmail.com> <7d6fde3d1002160901r600bb514u4a3219d2e59b16aa@mail.gmail.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: FreeBSD-Hackers , Jan Mikkelsen Subject: Re: read(1) garbage when input redirected from make incorrectly X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2010 18:03:13 -0000 On Tue, 16 Feb 2010, Garrett Cooper wrote: > [gcooper@optimus ~]$ python -c 'import sys; sys.stdin.read()' < make -V > bash: make: No such file or directory > [gcooper@optimus ~]$ perl -e 'while (<>) { print; }' < make -V > bash: make: No such file or directory No, you have to say: python -c 'import sys; sys.stdin.read()' < /usr/bin/make perl -e 'while (<>) { print; }' < /usr/bin/make -V What was happening in your first example is that the actual file /usr/bin/make on disk is being read directly ... /usr/bin/make is never executed at all. That is why your hexdump showed an ELF header -- it came from /usr/bin/make which is an ELF executable. -- Tod From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 16 18:11:06 2010 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3D53A1065698; Tue, 16 Feb 2010 18:11:06 +0000 (UTC) (envelope-from sobomax@FreeBSD.org) Received: from sippysoft.com (gk1.360sip.com [72.236.70.240]) by mx1.freebsd.org (Postfix) with ESMTP id B1C918FC13; Tue, 16 Feb 2010 18:11:05 +0000 (UTC) Received: from [192.168.1.38] ([70.71.167.197]) (authenticated bits=0) by sippysoft.com (8.14.3/8.14.3) with ESMTP id o1GIB2Zj022403 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 16 Feb 2010 10:11:02 -0800 (PST) (envelope-from sobomax@FreeBSD.org) Message-ID: <4B7ADFC6.7020202@FreeBSD.org> Date: Tue, 16 Feb 2010 10:11:18 -0800 From: Maxim Sobolev Organization: Sippy Software, Inc. User-Agent: Thunderbird 2.0.0.23 (Windows/20090812) MIME-Version: 1.0 To: Sergey Babkin References: <4B79297D.9080403@FreeBSD.org> <4B79205B.619A0A1A@verizon.net> In-Reply-To: <4B79205B.619A0A1A@verizon.net> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: Alfred Perlstein , freebsd-net@FreeBSD.org, "David G. Lawrence" , Jack Vogel , FreeBSD Hackers Subject: Re: Sudden mbuf demand increase and shortage under the load (igb issue?) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2010 18:11:06 -0000 OK, here is some new data that I think rules out any issues with the applications. Following Alfred's suggestion I have made a script to run every second and output some system statistics: date netstat -m vmstat -i ps -axl pstat -T vmstat -z sysctl -a The problem had hit us again today several times and upon investigating the log I found that increase in the mbuf usage happened in one step - going from normal 10% to 100% between two script runs. What is more interesting, is that time from two such subsequent runs were about 2 minutes apart (instead of 1 second as it should be) and when inspecting cron logs I noticed the same time gap in there. I ruled out any VM starvation as a cause of the delay because system has plenty of free memory. The incoming network traffic was not sufficient to starve VM so quickly either - it was about 7MB/sec at that time, so even if all receivers stopped draining their buffers it should have taken at least 1-2 seconds to fill up mbuf cache and create demand for an additional kernel memory. The failure would likely to be more gradual and I should have seen how it builds up in the debug log. So it looks like kernel issue of a sort, which causes all userland activity to cease for 2 minutes when the system reaches certain load. Mbuf build-up is only the by-product of this, not really a cause. igb(4) is being the primary suspect now, since we have other machines with more load not having this problem and we don't have anybody else using this driver. The chip is the following: igb0@pci0:5:0:0: class=0x020000 card=0x323f103c chip=0x10c98086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' class = network subclass = ethernet igb1@pci0:5:0:1: class=0x020000 card=0x323f103c chip=0x10c98086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' class = network subclass = ethernet Hardware in question is a new HP DL160G6. I have also checked IPMI logs and sensors and have not found any issue in there as well. No sensors reported off-range values and chassis temperature is within normal limits. I am not sure how to debug this problem further. We are now investigating opportunity to install external non-igb card to the server and see if it solves the issue. I have the whole log if anyone wants to take a closer peek. Regards, -- Maksym Sobolyev Sippy Software, Inc. Internet Telephony (VoIP) Experts T/F: +1-646-651-1110 Web: http://www.sippysoft.com MSN: sales@sippysoft.com Skype: SippySoft From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 16 18:54:25 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 73C65106566B; Tue, 16 Feb 2010 18:54:25 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-qy0-f173.google.com (mail-qy0-f173.google.com [209.85.221.173]) by mx1.freebsd.org (Postfix) with ESMTP id D46EB8FC17; Tue, 16 Feb 2010 18:54:24 +0000 (UTC) Received: by qyk4 with SMTP id 4so3869244qyk.8 for ; Tue, 16 Feb 2010 10:54:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=eGqe6sls0r5N+1WduLvK0Ddg5D6lF51v1ED33UHP8rw=; b=eKKYoVFlAZvXJjbUxtcMMO1HBthocSEcQSCvHmIV9bX/wgrc6YxxNfK77OdeQFF/6R bgRvHfTDSGYIEtnP5rowhhb7pEBR8RKPcGeTfEbPd9/dChmShBbOQdjqArmuV/Ww2YHd N4K4ygKTjiPf41hO0X6aCPGBK3pDYJlOL356k= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=bfl1XIQjlmjHZ3RzoN5QWBsfgx2G+1+hvm3Ai3jmXviD9pongetVJnWBl3U/wEUIl1 8Bd066MdhDuH/OvaNdtElml9rNO+xwNDBCftZ/heob7oMRjJtxF0TK2BWO9yQVJF7Nh4 /N5FdpbNlTeDaWQY6JVvGgPN+UHM8IkEPe1gs= Received: by 10.224.107.77 with SMTP id a13mr955235qap.312.1266346463767; Tue, 16 Feb 2010 10:54:23 -0800 (PST) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id 4sm19427854qwe.33.2010.02.16.10.54.21 (version=TLSv1/SSLv3 cipher=RC4-MD5); Tue, 16 Feb 2010 10:54:22 -0800 (PST) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Tue, 16 Feb 2010 10:53:41 -0800 From: Pyun YongHyeon Date: Tue, 16 Feb 2010 10:53:41 -0800 To: Maxim Sobolev Message-ID: <20100216185341.GE1394@michelle.cdnetworks.com> References: <4B79297D.9080403@FreeBSD.org> <4B79205B.619A0A1A@verizon.net> <4B7ADFC6.7020202@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B7ADFC6.7020202@FreeBSD.org> User-Agent: Mutt/1.4.2.3i Cc: Sergey Babkin , freebsd-net@freebsd.org, Alfred Perlstein , Jack Vogel , FreeBSD Hackers , "David G. Lawrence" Subject: Re: Sudden mbuf demand increase and shortage under the load (igb issue?) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2010 18:54:25 -0000 On Tue, Feb 16, 2010 at 10:11:18AM -0800, Maxim Sobolev wrote: > OK, here is some new data that I think rules out any issues with the > applications. Following Alfred's suggestion I have made a script to run > every second and output some system statistics: > > date > netstat -m > vmstat -i > ps -axl > pstat -T > vmstat -z > sysctl -a > > The problem had hit us again today several times and upon investigating > the log I found that increase in the mbuf usage happened in one step - > going from normal 10% to 100% between two script runs. What is more > interesting, is that time from two such subsequent runs were about 2 > minutes apart (instead of 1 second as it should be) and when inspecting > cron logs I noticed the same time gap in there. I ruled out any VM > starvation as a cause of the delay because system has plenty of free > memory. The incoming network traffic was not sufficient to starve VM so > quickly either - it was about 7MB/sec at that time, so even if all > receivers stopped draining their buffers it should have taken at least > 1-2 seconds to fill up mbuf cache and create demand for an additional > kernel memory. The failure would likely to be more gradual and I should > have seen how it builds up in the debug log. > > So it looks like kernel issue of a sort, which causes all userland > activity to cease for 2 minutes when the system reaches certain load. > Mbuf build-up is only the by-product of this, not really a cause. igb(4) > is being the primary suspect now, since we have other machines with more > load not having this problem and we don't have anybody else using this > driver. The chip is the following: > > igb0@pci0:5:0:0: class=0x020000 card=0x323f103c chip=0x10c98086 > rev=0x01 hdr=0x00 > vendor = 'Intel Corporation' > class = network > subclass = ethernet > igb1@pci0:5:0:1: class=0x020000 card=0x323f103c chip=0x10c98086 > rev=0x01 hdr=0x00 > vendor = 'Intel Corporation' > class = network > subclass = ethernet > > Hardware in question is a new HP DL160G6. I have also checked IPMI logs > and sensors and have not found any issue in there as well. No sensors > reported off-range values and chassis temperature is within normal limits. > > I am not sure how to debug this problem further. We are now > investigating opportunity to install external non-igb card to the server > and see if it solves the issue. > I know there were a couple of igb(4) issues in 7.x but it seems recent fixes didn't make it into stable/7 and 7.3-RELEASE. But the issues I know does not explain the symptom of your issue. One of issues that could be related with the issue was igb(4) took a lot of CPU cycles as it incorrectly rescheduled to get more frames instead of dropping some frames when heavy UDP traffic hits the controller(e.g. 64bytes UDP torture test). > I have the whole log if anyone wants to take a closer peek. > From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 16 18:55:03 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B98E91065670 for ; Tue, 16 Feb 2010 18:55:03 +0000 (UTC) (envelope-from yanefbsd@gmail.com) Received: from mail-px0-f194.google.com (mail-px0-f194.google.com [209.85.216.194]) by mx1.freebsd.org (Postfix) with ESMTP id 829DB8FC23 for ; Tue, 16 Feb 2010 18:55:01 +0000 (UTC) Received: by pxi32 with SMTP id 32so1805134pxi.3 for ; Tue, 16 Feb 2010 10:55:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=L9AAdai/JdCQK+lBhhjmumjlC5XiFg+gTw0ofXQs5Ls=; b=rPxeYoP0+O81F/n6p/Y5bViGcsZiYL9MNGCl65OliwsBDUH7oCfDZSAUxVJcgDn0jy pdDKa5vsNQygQJbe0kznSXwmV2VbbWMrS33rjI7wttnHlVgm78NBTnUmtl/X13DgOyih fX5Nlp1BAweyeaSxYgY1HTKanY8/IBGhikP0o= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=QRvQHXsSLAH1c0PDnGtGj86Xc2Am7fC/JEpPvVWNrk1SFsuEvgbvFFKQm5cnJZRgsD ux8B3krODxIiPD6jOZGSqDrV4sUrQ3TVXbb/rlONJwaHNZ8MdrrxWSoSMwomv8tVeDzM YOE/78jg3O9Vr9eZQFRjtGw+8Lek1yhpvt6hA= MIME-Version: 1.0 Received: by 10.142.59.21 with SMTP id h21mr4605694wfa.182.1266346501347; Tue, 16 Feb 2010 10:55:01 -0800 (PST) In-Reply-To: <20100217024315.O62495@plexi.pun-pun.prv> References: <364299f41002151649y2e4d4120p918759afb1fd8f6c@mail.gmail.com> <7d6fde3d1002151655q184c8a21k8a0c6c07b9b0ae79@mail.gmail.com> <7d6fde3d1002160901r600bb514u4a3219d2e59b16aa@mail.gmail.com> <20100217024315.O62495@plexi.pun-pun.prv> Date: Tue, 16 Feb 2010 10:55:01 -0800 Message-ID: <7d6fde3d1002161055l306d493di80a6392574a0f135@mail.gmail.com> From: Garrett Cooper To: Tod McQuillin Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD-Hackers , Jan Mikkelsen Subject: Re: read(1) garbage when input redirected from make incorrectly X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2010 18:55:03 -0000 On Tue, Feb 16, 2010 at 9:46 AM, Tod McQuillin wrote: > On Tue, 16 Feb 2010, Garrett Cooper wrote: > >> [gcooper@optimus ~]$ python -c 'import sys; sys.stdin.read()' < make -V >> bash: make: No such file or directory >> [gcooper@optimus ~]$ perl -e 'while (<>) { print; }' < make -V >> bash: make: No such file or directory > > No, you have to say: > > python -c 'import sys; sys.stdin.read()' < /usr/bin/make > > perl -e 'while (<>) { print; }' < /usr/bin/make -V > > What was happening in your first example is that the actual file > /usr/bin/make on disk is being read directly ... /usr/bin/make is never > executed at all. =A0That is why your hexdump showed an ELF header -- it c= ame > from /usr/bin/make which is an ELF executable. Yeah... nevermind duh. Case closed :P... Thanks! -Garrett From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 16 19:59:00 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2B136106568D for ; Tue, 16 Feb 2010 19:59:00 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id A86AE8FC08 for ; Tue, 16 Feb 2010 19:58:59 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id o1GJwt3I003727 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 16 Feb 2010 21:58:55 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.3/8.14.3) with ESMTP id o1GJwtKV027562; Tue, 16 Feb 2010 21:58:55 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.3/8.14.3/Submit) id o1GJwtoa027561; Tue, 16 Feb 2010 21:58:55 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 16 Feb 2010 21:58:55 +0200 From: Kostik Belousov To: Maxim Sobolev Message-ID: <20100216195855.GG50403@deviant.kiev.zoral.com.ua> References: <4B79297D.9080403@FreeBSD.org> <4B79205B.619A0A1A@verizon.net> <4B7ADFC6.7020202@FreeBSD.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Bqc0IY4JZZt50bUr" Content-Disposition: inline In-Reply-To: <4B7ADFC6.7020202@FreeBSD.org> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-net@freebsd.org, FreeBSD Hackers Subject: Re: Sudden mbuf demand increase and shortage under the load (igb issue?) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2010 19:59:00 -0000 --Bqc0IY4JZZt50bUr Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable [Trimmed Cc: list] On Tue, Feb 16, 2010 at 10:11:18AM -0800, Maxim Sobolev wrote: > OK, here is some new data that I think rules out any issues with the=20 > applications. Following Alfred's suggestion I have made a script to run= =20 > every second and output some system statistics: >=20 > date > netstat -m > vmstat -i > ps -axl > pstat -T > vmstat -z > sysctl -a >=20 > The problem had hit us again today several times and upon investigating= =20 > the log I found that increase in the mbuf usage happened in one step -=20 > going from normal 10% to 100% between two script runs. What is more=20 > interesting, is that time from two such subsequent runs were about 2=20 > minutes apart (instead of 1 second as it should be) and when inspecting= =20 > cron logs I noticed the same time gap in there. I ruled out any VM=20 > starvation as a cause of the delay because system has plenty of free=20 > memory. The incoming network traffic was not sufficient to starve VM so= =20 > quickly either - it was about 7MB/sec at that time, so even if all=20 > receivers stopped draining their buffers it should have taken at least=20 > 1-2 seconds to fill up mbuf cache and create demand for an additional=20 > kernel memory. The failure would likely to be more gradual and I should= =20 > have seen how it builds up in the debug log. >=20 > So it looks like kernel issue of a sort, which causes all userland=20 > activity to cease for 2 minutes when the system reaches certain load.=20 > Mbuf build-up is only the by-product of this, not really a cause. igb(4)= =20 > is being the primary suspect now, since we have other machines with more= =20 > load not having this problem and we don't have anybody else using this=20 > driver. The chip is the following: >=20 > igb0@pci0:5:0:0: class=3D0x020000 card=3D0x323f103c chip=3D0x10c98= 086=20 > rev=3D0x01 hdr=3D0x00 > vendor =3D 'Intel Corporation' > class =3D network > subclass =3D ethernet > igb1@pci0:5:0:1: class=3D0x020000 card=3D0x323f103c chip=3D0x10c98= 086=20 > rev=3D0x01 hdr=3D0x00 > vendor =3D 'Intel Corporation' > class =3D network > subclass =3D ethernet >=20 > Hardware in question is a new HP DL160G6. I have also checked IPMI logs= =20 > and sensors and have not found any issue in there as well. No sensors=20 > reported off-range values and chassis temperature is within normal limits. >=20 > I am not sure how to debug this problem further. We are now=20 > investigating opportunity to install external non-igb card to the server= =20 > and see if it solves the issue. >=20 > I have the whole log if anyone wants to take a closer peek. How much physical memory do you have installed in the machine ? If it is > 16Gb, try to remove some. --Bqc0IY4JZZt50bUr Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (FreeBSD) iEYEARECAAYFAkt6+P4ACgkQC3+MBN1Mb4gn+QCgvaSwNrcvigYcLCXLwV81i8j/ mzYAoNghlDps8yyiQieR1r9ejiPpnkKx =9c1c -----END PGP SIGNATURE----- --Bqc0IY4JZZt50bUr-- From owner-freebsd-hackers@FreeBSD.ORG Tue Feb 16 23:41:31 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 87E60106566B for ; Tue, 16 Feb 2010 23:41:31 +0000 (UTC) (envelope-from janm-freebsd-hackers@transactionware.com) Received: from mail.transactionware.com (mail.transactionware.com [203.14.245.7]) by mx1.freebsd.org (Postfix) with SMTP id A72678FC1D for ; Tue, 16 Feb 2010 23:41:30 +0000 (UTC) Received: (qmail 5240 invoked from network); 16 Feb 2010 23:49:19 -0000 Received: from midgard.transactionware.com (192.168.1.55) by dm.transactionware.com with SMTP; 16 Feb 2010 23:49:19 -0000 Received: (qmail 37332 invoked by uid 907); 16 Feb 2010 23:41:27 -0000 Received: from jmmacpro.transactionware.com (HELO jmmacpro.transactionware.com) (192.168.1.33) by midgard.transactionware.com (qpsmtpd/0.82) with ESMTP; Wed, 17 Feb 2010 10:41:27 +1100 Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=iso-8859-1 From: Jan Mikkelsen In-Reply-To: <86eikljt7i.fsf@ds4.des.no> Date: Wed, 17 Feb 2010 10:41:27 +1100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <364299f41002151649y2e4d4120p918759afb1fd8f6c@mail.gmail.com> <7d6fde3d1002151655q184c8a21k8a0c6c07b9b0ae79@mail.gmail.com> <86eikljt7i.fsf@ds4.des.no> To: =?iso-8859-1?Q?Dag-Erling_Sm=F8rgrav?= X-Mailer: Apple Mail (2.1077) Cc: Garrett Cooper , FreeBSD-Hackers Subject: Re: read(1) garbage when input redirected from make incorrectly X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2010 23:41:31 -0000 On 16/02/2010, at 10:49 PM, Dag-Erling Sm=F8rgrav wrote: > The LHS of < is a command, the RHS is the name of the file to be read. > After that, you can have further redirections, a command separator > (semicolon, single or double ampersand, single or double pipe etc.), = or, > depending on context, various other stuff such as a paren, bracket, > backquote etc. A redirection doesn't terminate the argument list. For example: echo a b < /dev/null c d Produces: a b c d And: < /etc/passwd cat Will emit /etc/passwd to stdout. Regards, Jan Mikkelsen From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 17 02:13:52 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 34662106566B for ; Wed, 17 Feb 2010 02:13:52 +0000 (UTC) (envelope-from mj@feral.com) Received: from ns1.feral.com (ns1.feral.com [192.67.166.1]) by mx1.freebsd.org (Postfix) with ESMTP id D6F2A8FC0A for ; Wed, 17 Feb 2010 02:13:51 +0000 (UTC) Received: from [10.8.0.2] (remotevpn [10.8.0.2]) by ns1.feral.com (8.14.3/8.14.3) with ESMTP id o1H1bTOV011007 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 16 Feb 2010 17:37:29 -0800 (PST) (envelope-from mj@feral.com) Message-ID: <4B7B4854.7020206@feral.com> Date: Tue, 16 Feb 2010 17:37:24 -0800 From: Matthew Jacob Organization: Feral Software User-Agent: Thunderbird 2.0.0.23 (Windows/20090812) MIME-Version: 1.0 To: Oleg Sharoyko References: <1266314570.73716.42.camel@brain.cc.rsu.ru> In-Reply-To: <1266314570.73716.42.camel@brain.cc.rsu.ru> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender DNS name whitelisted, not delayed by milter-greylist-4.2.3 (ns1.feral.com [10.8.0.1]); Tue, 16 Feb 2010 17:37:30 -0800 (PST) X-Mailman-Approved-At: Wed, 17 Feb 2010 04:37:18 +0000 Cc: freebsd-scsi@freebsd.org, freebsd-hackers@freebsd.org Subject: Re: Qlogic (isp) cannot login into fabric after link loss X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Feb 2010 02:13:52 -0000 Oleg Sharoyko wrote: > Hi! > > I I've reproduced the problem and I think I know what's going on. I've filed a PR which I'll put you on the CC list for as soon as I find it. From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 17 08:32:29 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 481F11065679 for ; Wed, 17 Feb 2010 08:32:29 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 0817E8FC0C for ; Wed, 17 Feb 2010 08:32:28 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id F3CC31FFC22; Wed, 17 Feb 2010 08:32:27 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id B82BC844CC; Wed, 17 Feb 2010 09:32:27 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Jan Mikkelsen References: <364299f41002151649y2e4d4120p918759afb1fd8f6c@mail.gmail.com> <7d6fde3d1002151655q184c8a21k8a0c6c07b9b0ae79@mail.gmail.com> <86eikljt7i.fsf@ds4.des.no> Date: Wed, 17 Feb 2010 09:32:27 +0100 In-Reply-To: (Jan Mikkelsen's message of "Wed, 17 Feb 2010 10:41:27 +1100") Message-ID: <86eikkntyc.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.95 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Garrett Cooper , FreeBSD-Hackers Subject: Re: read(1) garbage when input redirected from make incorrectly X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Feb 2010 08:32:29 -0000 Jan Mikkelsen writes: > A redirection doesn't terminate the argument list. [...] Huh, you learn something every day... :) DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 17 10:52:38 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 50D121065672 for ; Wed, 17 Feb 2010 10:52:38 +0000 (UTC) (envelope-from freebsd-hackers@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 0ADD98FC15 for ; Wed, 17 Feb 2010 10:52:37 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1NhhWE-00014W-1S for freebsd-hackers@freebsd.org; Wed, 17 Feb 2010 11:52:34 +0100 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 17 Feb 2010 11:52:34 +0100 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 17 Feb 2010 11:52:34 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-hackers@freebsd.org From: Ivan Voras Date: Wed, 17 Feb 2010 11:51:45 +0100 Lines: 6 Message-ID: References: <4B79297D.9080403@FreeBSD.org> <4B79205B.619A0A1A@verizon.net> <4B7ADFC6.7020202@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Thunderbird 2.0.0.23 (X11/20090817) In-Reply-To: <4B7ADFC6.7020202@FreeBSD.org> Sender: news Cc: freebsd-net@freebsd.org Subject: Re: Sudden mbuf demand increase and shortage under the load (igb issue?) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Feb 2010 10:52:38 -0000 Maxim Sobolev wrote: > So it looks like kernel issue of a sort, which causes all userland > activity to cease for 2 minutes when the system reaches certain load. You are not using ZFS, are you? :)) From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 17 18:09:06 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5CCE2106568B for ; Wed, 17 Feb 2010 18:09:06 +0000 (UTC) (envelope-from brampton@gmail.com) Received: from mail-fx0-f226.google.com (mail-fx0-f226.google.com [209.85.220.226]) by mx1.freebsd.org (Postfix) with ESMTP id E607A8FC20 for ; Wed, 17 Feb 2010 18:09:05 +0000 (UTC) Received: by fxm26 with SMTP id 26so7508488fxm.13 for ; Wed, 17 Feb 2010 10:09:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:date :x-google-sender-auth:message-id:subject:from:to:content-type; bh=R2rpb3f0jxq9P6AVpMh/b6mWCiyAOiBI8bOxYFwhto4=; b=uMGQwdxpXAGkJeCeWehiQ8lEkEX++YPMhHhDdO5Zxyb2xSZsKyrFtjPuJAtUG4c97h ovRqFV6zB1kRKUZYCn9v0X38/0NEHBAMk1m1ewEHLrG/wcktROuUOBPKju9R2cq8GOHL +kQHSnltgi1/D9JhVJrou7psFchT/fTkI2gKs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:content-type; b=g57qpdqU868BCEwbEtqheUqXZKJLpaisk1QtM49qkxavQfJD696GTkim4KPa015jJF yBBt/qDwQHY1di3HSK5iAzaR30KxDd0Fncrj3/CxkYHcjdwW8SQFE891VI2HMYqFKUH/ HJwiW0ezfvHIY3zLRRAhN6z7nTOFy6PV/k2aA= MIME-Version: 1.0 Sender: brampton@gmail.com Received: by 10.223.143.21 with SMTP id s21mr3727160fau.51.1266430144783; Wed, 17 Feb 2010 10:09:04 -0800 (PST) Date: Wed, 17 Feb 2010 18:09:04 +0000 X-Google-Sender-Auth: 93b25839c5306fcf Message-ID: From: Andrew Brampton To: freebsd-hackers Content-Type: text/plain; charset=UTF-8 Subject: Per core, per device interrupt counts X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Feb 2010 18:09:06 -0000 After reading though the kernel source I realise what I want isn't implemented at the moment but I wanted to discuss if this feature would be an useful addition. Basically I want to see counts of how many interrupts for a particular interrupt have fired on each core. Linux has provided this kind of information for a while and I've found it quite useful. I would like this information when I am pinning particular interrupts to one (or more cores). This is useful when I'm tweaking a system with, for example, 10Gig network cards which have multiple queues (thus multiple IRQs). Having a look in the kernel I see that the count is kept in the is_count field of the intsrc struct. This field seems to be backed by the global intrcnt array. Could this be modified to perhaps use the new PCPU macros, so there is a different count for each core? If I was given a few pointers I might find time to implement this myself. thanks Andrew From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 17 18:51:14 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 698F51065679 for ; Wed, 17 Feb 2010 18:51:14 +0000 (UTC) (envelope-from fernando.apesteguia@gmail.com) Received: from mail-ew0-f225.google.com (mail-ew0-f225.google.com [209.85.219.225]) by mx1.freebsd.org (Postfix) with ESMTP id 03A638FC12 for ; Wed, 17 Feb 2010 18:51:13 +0000 (UTC) Received: by ewy25 with SMTP id 25so919910ewy.3 for ; Wed, 17 Feb 2010 10:51:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type; bh=BDUWLBbr0lffE2VbFqwT/UTWMySh5BDywgLpmcVN0S0=; b=Oovw5/v+bMXqeN58qsLdZpY3pozczdNgIIyWXuVyao5V9ENvzV0XdHB4QCN3MKVFyb PJ11+I0ZItvwhIQBR6rqWVo5KOH/Zh/nGRdFcbIWPeQcajhhgYEauqvLOKutmXfmMVJO V5PqdFW5g2HHeQsf6dkZfe1mE2F1NtKt1NBCs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=dp1+RYIq+lADcHFPv8uWgX/kHy30FZwE30d3aioUzIlhCd62QZIr+2Tm+zf2XNzmju y9TUMpQNd+GRvIJ+9nvS7/pc44Wx6Pp0J8SZr30saedzHpC5LSutY1F2OIAU2w9EJ2Pw ui/udmIGh+gDXMSJDRCNpyoATF3qJf4yyGwSY= MIME-Version: 1.0 Received: by 10.213.109.74 with SMTP id i10mr3104492ebp.61.1266432666343; Wed, 17 Feb 2010 10:51:06 -0800 (PST) Date: Wed, 17 Feb 2010 19:51:06 +0100 Message-ID: <1bd550a01002171051n7117895avb5cf57fb7fbb9388@mail.gmail.com> From: =?ISO-8859-1?Q?Fernando_Apestegu=EDa?= To: FreeBSD Hackers Content-Type: text/plain; charset=ISO-8859-1 Subject: linprocfs proc/pid/environ patch & list question X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Feb 2010 18:51:14 -0000 Hi, I have a small patch (against 8.0-RELEASE-p2) that _should_ implement the /proc/pid/environ file under linprocfs. However, it seems it does not work properly but I don't know what I'm doing wrong. Is this list the place to ask for help? I tried in the forums[1] but got no answer. Don't we have a 'kernel newbies'-like list? Thanks in advance. [1] http://forums.freebsd.org/showthread.php?t=11329 --- sys/compat/linprocfs/linprocfs.c.orig 2009-10-25 02:10:29.000000000 +0100 +++ sys/compat/linprocfs/linprocfs.c 2010-02-16 19:38:36.000000000 +0100 @@ -939,8 +939,38 @@ static int linprocfs_doprocenviron(PFS_FILL_ARGS) { + int i, error; + struct ps_strings pss; + char **ps_envstr; - sbuf_printf(sb, "doprocenviron\n%c", '\0'); + PROC_LOCK(p); + if (p_cansee(td, p) != 0) + return (0); + PROC_UNLOCK(p); + + error = copyin((void *)p->p_sysent->sv_psstrings, &pss, + sizeof(pss)); + if (error) + return (error); + + ps_envstr = malloc(pss.ps_nenvstr * sizeof(char *), + M_TEMP, M_WAITOK); + + error = copyin((void *)pss.ps_envstr, ps_envstr, + pss.ps_nenvstr * sizeof(char *)); + + if (error) { + free(ps_envstr, M_TEMP); + return (error); + } + + /* NULL separated list of variable=value pairs */ + + for (i = 0; i < pss.ps_nenvstr; i++) { + sbuf_copyin(sb, ps_envstr[i], 0); + } + + free(ps_envstr, M_TEMP); return (0); } From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 17 19:12:04 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8B7D11065670 for ; Wed, 17 Feb 2010 19:12:04 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 7054C8FC3A for ; Wed, 17 Feb 2010 19:12:03 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id o1HJBueV012630 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 17 Feb 2010 21:11:56 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.3/8.14.3) with ESMTP id o1HJBufg022524; Wed, 17 Feb 2010 21:11:56 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.3/8.14.3/Submit) id o1HJBu80022523; Wed, 17 Feb 2010 21:11:56 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 17 Feb 2010 21:11:56 +0200 From: Kostik Belousov To: Fernando Apestegu?a Message-ID: <20100217191156.GP50403@deviant.kiev.zoral.com.ua> References: <1bd550a01002171051n7117895avb5cf57fb7fbb9388@mail.gmail.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="z4D23EFnZpzTzcHd" Content-Disposition: inline In-Reply-To: <1bd550a01002171051n7117895avb5cf57fb7fbb9388@mail.gmail.com> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: FreeBSD Hackers Subject: Re: linprocfs proc/pid/environ patch & list question X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Feb 2010 19:12:04 -0000 --z4D23EFnZpzTzcHd Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Feb 17, 2010 at 07:51:06PM +0100, Fernando Apestegu?a wrote: > Hi, >=20 > I have a small patch (against 8.0-RELEASE-p2) that _should_ implement > the /proc/pid/environ file > under linprocfs. > However, it seems it does not work properly but I don't know what I'm > doing wrong. > Is this list the place to ask for help? I tried in the forums[1] but > got no answer. Putting aside any "does not work" questions, please see comment below. >=20 > Don't we have a 'kernel newbies'-like list? >=20 > Thanks in advance. >=20 > [1] http://forums.freebsd.org/showthread.php?t=3D11329 >=20 > --- sys/compat/linprocfs/linprocfs.c.orig 2009-10-25 02:10:29.000000000 += 0100 > +++ sys/compat/linprocfs/linprocfs.c 2010-02-16 19:38:36.000000000 +0100 > @@ -939,8 +939,38 @@ > static int > linprocfs_doprocenviron(PFS_FILL_ARGS) > { > + int i, error; > + struct ps_strings pss; > + char **ps_envstr; >=20 > - sbuf_printf(sb, "doprocenviron\n%c", '\0'); > + PROC_LOCK(p); > + if (p_cansee(td, p) !=3D 0) > + return (0); > + PROC_UNLOCK(p); > + > + error =3D copyin((void *)p->p_sysent->sv_psstrings, &pss, > + sizeof(pss)); > + if (error) > + return (error); > + > + ps_envstr =3D malloc(pss.ps_nenvstr * sizeof(char *), > + M_TEMP, M_WAITOK); This is essentially "panic me" code. ps_nenvstr is user-controlled, and allows to specify arbitrary integers. Even ignoring exhaustion of the kernel map, it can cause allocation of big amount of physical memory. Note that execve(2) implementation uses swappable memory to store arguments and environment strings passed from vm spaces. > + > + error =3D copyin((void *)pss.ps_envstr, ps_envstr, > + pss.ps_nenvstr * sizeof(char *)); > + > + if (error) { > + free(ps_envstr, M_TEMP); > + return (error); > + } > + > + /* NULL separated list of variable=3Dvalue pairs */ > +=09 > + for (i =3D 0; i < pss.ps_nenvstr; i++) { > + sbuf_copyin(sb, ps_envstr[i], 0); > + } > + > + free(ps_envstr, M_TEMP); > return (0); > } > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" --z4D23EFnZpzTzcHd Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (FreeBSD) iEYEARECAAYFAkt8P3sACgkQC3+MBN1Mb4i8vACg20L6f/ExO+ob4sDZo9T+mkuU ktcAn0hWvo5P1EPTKH3H7DIOICFjo3yZ =F2oo -----END PGP SIGNATURE----- --z4D23EFnZpzTzcHd-- From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 17 20:49:26 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8F9D9106566B for ; Wed, 17 Feb 2010 20:49:26 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 624558FC1D for ; Wed, 17 Feb 2010 20:49:26 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 14E6846B06; Wed, 17 Feb 2010 15:49:26 -0500 (EST) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id 428038A021; Wed, 17 Feb 2010 15:49:25 -0500 (EST) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Wed, 17 Feb 2010 14:13:56 -0500 User-Agent: KMail/1.12.1 (FreeBSD/7.2-CBSD-20100120; KDE/4.3.1; amd64; ; ) References: <291941b81002150321w7b0479beo1c6fec39ef6a7922@mail.gmail.com> In-Reply-To: <291941b81002150321w7b0479beo1c6fec39ef6a7922@mail.gmail.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201002171413.56920.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Wed, 17 Feb 2010 15:49:25 -0500 (EST) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.4 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: Shrikanth Kamath Subject: Re: Ktrace'ing kernel threads X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Feb 2010 20:49:26 -0000 On Monday 15 February 2010 6:21:40 am Shrikanth Kamath wrote: > Can ktrace trace another kernel thread which has roughly the semantics as > below, right now it > does not hit any of the designated interesting points that ktrace is built > for, but what if I could define those, > will ktrace still allow tracing another kernel thread? > > thread(client_info) > { > ... > ... > build_msg(client_info); /* this will malloc a mbuf and fill the data in > it */ > ... > sosend(client_info); > } > > I want to time the entry/return of build_msg, and the time sosend, dump > client_info (some specific fields). It is probably easier to do this with DTrace (albeit possibly with more overhead). You can ktrace a kthread fine, but you would need to write your own ktrace hooks (and record parser for kdump) which would take a bit longer than a D script with DTrace. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 17 20:49:27 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A5EFA106566C for ; Wed, 17 Feb 2010 20:49:27 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 782FF8FC08 for ; Wed, 17 Feb 2010 20:49:27 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 2920746B66; Wed, 17 Feb 2010 15:49:27 -0500 (EST) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id 62E7B8A025; Wed, 17 Feb 2010 15:49:26 -0500 (EST) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Wed, 17 Feb 2010 14:17:24 -0500 User-Agent: KMail/1.12.1 (FreeBSD/7.2-CBSD-20100120; KDE/4.3.1; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <201002171417.24951.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Wed, 17 Feb 2010 15:49:26 -0500 (EST) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.4 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: Andrew Brampton Subject: Re: Per core, per device interrupt counts X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Feb 2010 20:49:27 -0000 On Wednesday 17 February 2010 1:09:04 pm Andrew Brampton wrote: > After reading though the kernel source I realise what I want isn't > implemented at the moment but I wanted to discuss if this feature > would be an useful addition. > > Basically I want to see counts of how many interrupts for a particular > interrupt have fired on each core. Linux has provided this kind of > information for a while and I've found it quite useful. I would like > this information when I am pinning particular interrupts to one (or > more cores). This is useful when I'm tweaking a system with, for > example, 10Gig network cards which have multiple queues (thus multiple > IRQs). > > Having a look in the kernel I see that the count is kept in the > is_count field of the intsrc struct. This field seems to be backed by > the global intrcnt array. Could this be modified to perhaps use the > new PCPU macros, so there is a different count for each core? If I was > given a few pointers I might find time to implement this myself. The simplest method would probably be to make intrcnt grow per-CPU counts, but that would change the ABI of intrcnt and require a good bit of userland hacking to fix vmstat -i, etc. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 17 21:34:13 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 855CA1065670 for ; Wed, 17 Feb 2010 21:34:13 +0000 (UTC) (envelope-from fergleiser@yahoo.com) Received: from web31702.mail.mud.yahoo.com (web31702.mail.mud.yahoo.com [68.142.201.182]) by mx1.freebsd.org (Postfix) with SMTP id 43EC68FC16 for ; Wed, 17 Feb 2010 21:34:12 +0000 (UTC) Received: (qmail 43591 invoked by uid 60001); 17 Feb 2010 21:34:12 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1266442452; bh=kXjjyCpYDqVZ5JStTXEvuKCR4oiqQJh8KSCuCSmvugE=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=lRUE2LrP0uxP2hOy8rlJek/+UsNZgEsmqPQkBCazVRpuldWk7iOAp+5t2nMm4x1/v3HIt+QFlKjzrl8RyJks95l/KqZKx9eKTadu5XT7v1mM+13UgOIHLQusCIpQ9f+4x+TW7x/n6g7ISU13CMyxwQW007V9yVWbmBVzuFnH86g= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=GRlJ7DE90clxQpdnwctf63yP6wMm0uK+s0NCvlCNqoAveXMnnba7oncW1uE0U+rqwcxuToabbwANj/1UULnpdMaeX6MJKwdwXYkQBezPqL0kayCrpJvhh0O9/JTOLmnggDDjVNmQ9DY6O+WEYU00KTSjNyymxlU1f2z9bG0hLeo=; Message-ID: <21791.42462.qm@web31702.mail.mud.yahoo.com> X-YMail-OSG: S35PTJ4VM1mAMEz7jNBC7EyeXzNd7Mxt._y2QeSxJ__glY1HMHKgYDzXpmeg38tHWYl.K9_Q3Gr12rxfdahNYO3_yw4_Bmejkngz4RVR7HcxTcWnl34uo66O70LjJ3COsNjcC5E6nAFnS69sS7IzcCORiQ8ozuUimDOXiHbwEFD7YBl3ojloADlLWDkKCGDRraqyHJG_6D37RgRU.NcJ2L2zWFvDs35DtQ8GNlC9na5RmDkWWdJvPmQEsMLFAOEsrO1fWETEcU2bxHvSUxhHFSJKlSSE8wYM3DodiXUw1mdEJOhjsG3TakBodSLv1cSWYbGKx9Rd0tM54XVM5.vQVYTv5BooySgATr72mpwCUPeEJFqknkTAxPr8ffc- Received: from [190.231.9.29] by web31702.mail.mud.yahoo.com via HTTP; Wed, 17 Feb 2010 13:34:11 PST X-Mailer: YahooMailRC/272.7 YahooMailWebService/0.8.100.260964 References: <201002171417.24951.jhb@freebsd.org> Date: Wed, 17 Feb 2010 13:34:11 -0800 (PST) From: Fernando Gleiser To: freebsd-hackers@freebsd.org In-Reply-To: <201002171417.24951.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Andrew Brampton Subject: Re: Per core, per device interrupt counts X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Feb 2010 21:34:13 -0000 ----- Original Message ---- > From: John Baldwin > To: freebsd-hackers@freebsd.org > Cc: Andrew Brampton > Sent: Wed, February 17, 2010 4:17:24 PM > Subject: Re: Per core, per device interrupt counts > > > The simplest method would probably be to make intrcnt grow per-CPU counts, but > that would change the ABI of intrcnt and require a good bit of userland > hacking to fix vmstat -i, etc. Or he can add some DTrace SDT probes after intrctl gets updated and export the device name, cpu index and count number from there as the probe argument list. Then he can get the stats he wants from a D script Solaris' intrstat is built as a DTrace consumer, IIRC Fer From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 17 22:20:15 2010 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 87DB41065697; Wed, 17 Feb 2010 22:20:15 +0000 (UTC) (envelope-from nox@jelal.kn-bremen.de) Received: from smtp.kn-bremen.de (gelbbaer.kn-bremen.de [78.46.108.116]) by mx1.freebsd.org (Postfix) with ESMTP id 49C3C8FC13; Wed, 17 Feb 2010 22:20:15 +0000 (UTC) Received: by smtp.kn-bremen.de (Postfix, from userid 10) id A2EB11E00771; Wed, 17 Feb 2010 23:01:38 +0100 (CET) Received: from triton8.kn-bremen.de (noident@localhost [127.0.0.1]) by triton8.kn-bremen.de (8.14.3/8.14.3) with ESMTP id o1HLxfZQ019932; Wed, 17 Feb 2010 22:59:41 +0100 (CET) (envelope-from nox@triton8.kn-bremen.de) Received: (from nox@localhost) by triton8.kn-bremen.de (8.14.3/8.14.3/Submit) id o1HLxfZx019931; Wed, 17 Feb 2010 22:59:41 +0100 (CET) (envelope-from nox) From: Juergen Lock Date: Wed, 17 Feb 2010 22:59:40 +0100 To: freebsd-hackers@FreeBSD.org Message-ID: <20100217215940.GA19713@triton8.kn-bremen.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.20 (2009-06-14) X-Mailman-Approved-At: Wed, 17 Feb 2010 22:47:25 +0000 Cc: kientzle@FreeBSD.org Subject: "tar tfv /dev/cd0" speedup patch X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Feb 2010 22:20:15 -0000 Hi! I recently wanted to quickly look at an optical disc without mounting it and since bsdtar/libarchive know iso9660 I just did the command in the Subject. It worked, but it was sloow... :( Apparently it read all of the disc without seeking. The following patch fixes this, is something like this desired? If yes I could look how to do the same for Linux, I _think_ there you could just check for S_ISBLK and try to lseek to the end and back, at least that seems to be how you find out the size of a block device there... Cheers, Juergen Index: lib/libarchive/archive_read_open_filename.c @@ -44,6 +44,10 @@ #ifdef HAVE_UNISTD_H #include #endif +#ifdef __FreeBSD__ +#include +#include +#endif #include "archive.h" @@ -83,6 +87,9 @@ struct read_file_data *mine; void *b; int fd; +#ifdef __FreeBSD__ + off_t mediasize = 0; +#endif archive_clear_error(a); if (filename == NULL || filename[0] == '\0') { @@ -143,6 +150,17 @@ */ mine->can_skip = 1; } +#ifdef __FreeBSD__ + /* + * on FreeBSD if a device supports the DIOCGMEDIASIZE ioctl + * it is a disk-like device and should be seekable. + */ + else if (S_ISCHR(st.st_mode) && + !ioctl(fd, DIOCGMEDIASIZE, &mediasize) && mediasize) { + archive_read_extract_set_skip_file(a, st.st_dev, st.st_ino); + mine->can_skip = 1; + } +#endif return (archive_read_open2(a, mine, NULL, file_read, file_skip, file_close)); } From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 18 04:19:37 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8DBEF1065672 for ; Thu, 18 Feb 2010 04:19:37 +0000 (UTC) (envelope-from anderson_underground@hotmail.com) Received: from snt0-omc2-s32.snt0.hotmail.com (snt0-omc2-s32.snt0.hotmail.com [65.55.90.107]) by mx1.freebsd.org (Postfix) with ESMTP id 610648FC18 for ; Thu, 18 Feb 2010 04:19:37 +0000 (UTC) Received: from SNT137-W4 ([65.55.90.73]) by snt0-omc2-s32.snt0.hotmail.com with Microsoft SMTPSVC(6.0.3790.3959); Wed, 17 Feb 2010 20:07:36 -0800 Message-ID: Content-Type: multipart/mixed; boundary="_0b5bcd7e-f970-44b3-a991-b9dfefddba14_" X-Originating-IP: [189.105.64.253] From: Anderson Eduardo To: Date: Thu, 18 Feb 2010 01:07:36 -0300 Importance: Normal MIME-Version: 1.0 X-OriginalArrivalTime: 18 Feb 2010 04:07:36.0045 (UTC) FILETIME=[E72691D0:01CAB04F] X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Debug registers on FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 04:19:37 -0000 --_0b5bcd7e-f970-44b3-a991-b9dfefddba14_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hello Folks=2C First=2C I'm starting in the world of the debugging FreeBSD Kernel and=2C I= don't know to use kgdb and ddb properly and=2C I need set breakpoints usin= g debug register. Those breakpoints in the all tasks. Second=2C I'm reading some articles about Debug Registers on Intel Architec= ture and=2C I'm using AMD processors. Third=2C It's only the begin of the my questions. Sorry my poor English. TIA Follow in attached a old C file that I coded. =20 _________________________________________________________________ Quer deixar seus v=EDdeos mais divertidos? Com o Movie Maker isso fica f=E1= cil. http://www.windowslive.com.br/public/tip.aspx/view/87?product=3D4&ocid=3DWi= ndows Live:Dicas - Movie Maker:Hotmail:Tagline:1x1:Titulo Legendas Creditos= --_0b5bcd7e-f970-44b3-a991-b9dfefddba14_ Content-Type: text/plain Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="DR_FreeBSD.c" I2luY2x1ZGUgPHN5cy9rZW52Lmg+DQojaW5jbHVkZSA8c3lzL3R5cGVzLmg+DQojaW5jbHVkZSA8 c3lzL3BhcmFtLmg+DQojaW5jbHVkZSA8c3lzL3Byb2MuaD4NCiNpbmNsdWRlIDxzeXMvbW9kdWxl Lmg+DQojaW5jbHVkZSA8c3lzL3N5c2VudC5oPg0KI2luY2x1ZGUgPHN5cy9rZXJuZWwuaD4NCiNp bmNsdWRlIDxzeXMvc3lzdG0uaD4NCiNpbmNsdWRlIDxzeXMvc3lzY2FsbC5oPg0KI2luY2x1ZGUg PHN5cy9zeXNwcm90by5oPg0KDQpzdGF0aWMgaW50IG15X2Z1bmN0aW9uKHZvaWQpDQp7DQogICAg ICAgIHVuc2lnbmVkIGludCByX2RyMCxyX2RyMSxyX2RyMixyX2RyMyxyX2RyNCxyX2RyNSxyX2Ry NixyX2RyNzsNCiAgICAgICAgbG9uZyAqd19kcjAsKndfZHIxLCp3X2RyNCwqd19kcjU7DQoNCiAg ICAgICAgd19kcjAgPSAweDI4MTVkYzdjOw0KICAgICAgICB3X2RyMSA9IDB4MjgwZTcyOTQ7DQog ICAgICAgIHdfZHI0ID0gMHhmZmZmNGZmMg0KICAgICAgICB3X2RyNSA9IDB4NDAwOw0KLyoNCkRS MCA9IDB4MjgxNWRjN2MNCkRSMSA9IDB4MjgwZTcyOTQNCkRSMiA9IDB4MA0KRFIzID0gMHgwDQpE UjQgPSAweGZmZmY0ZmYyDQpEUjUgPSAweDQwMA0KRFI2ID0gMHhmZmZmNGZmMg0KRFI3ID0gMHg0 MDANCiovDQoNCg0KDQogICAgICAgIF9fYXNtX18gX192b2xhdGlsZSgibW92ICUwLCUlZHIwIjog OiJhIiggd19kcjAgKSk7DQogICAgICAgIF9fYXNtX18gX192b2xhdGlsZSgibW92ICUwLCUlZHIx IjogOiJhIiggd19kcjEgKSk7DQogICAgICAgIF9fYXNtX18gX192b2xhdGlsZSgibW92ICUwLCUl ZHI0IjogOiJhIiggd19kcjQgKSk7DQogICAgICAgIF9fYXNtX18gX192b2xhdGlsZSgibW92ICUw LCUlZHI1IjogOiJhIiggd19kcjUgKSk7DQoNCiAgICAgICAgX19hc21fXyBfX3ZvbGF0aWxlKCJt b3YgJSVkcjAsJTAiOiAiPWEiKCByX2RyMCApKTsNCiAgICAgICAgX19hc21fXyBfX3ZvbGF0aWxl KCJtb3YgJSVkcjEsJTAiOiAiPWEiKCByX2RyMSApKTsNCiAgICAgICAgX19hc21fXyBfX3ZvbGF0 aWxlKCJtb3YgJSVkcjIsJTAiOiAiPWEiKCByX2RyMiApKTsNCiAgICAgICAgX19hc21fXyBfX3Zv bGF0aWxlKCJtb3YgJSVkcjMsJTAiOiAiPWEiKCByX2RyMyApKTsNCiAgICAgICAgX19hc21fXyBf X3ZvbGF0aWxlKCJtb3YgJSVkcjQsJTAiOiAiPWEiKCByX2RyNCApKTsNCiAgICAgICAgX19hc21f XyBfX3ZvbGF0aWxlKCJtb3YgJSVkcjUsJTAiOiAiPWEiKCByX2RyNSApKTsNCiAgICAgICAgX19h c21fXyBfX3ZvbGF0aWxlKCJtb3YgJSVkcjYsJTAiOiAiPWEiKCByX2RyNiApKTsNCiAgICAgICAg X19hc21fXyBfX3ZvbGF0aWxlKCJtb3YgJSVkcjcsJTAiOiAiPWEiKCByX2RyNyApKTsNCg0KICAg ICAgICB1cHJpbnRmKCJEUjAgPSAweCV4XG4iLHJfZHIwKTsNCiAgICAgICAgdXByaW50ZigiRFIx ID0gMHgleFxuIixyX2RyMSk7DQogICAgICAgIHVwcmludGYoIkRSMiA9IDB4JXhcbiIscl9kcjIp Ow0KICAgICAgICB1cHJpbnRmKCJEUjMgPSAweCV4XG4iLHJfZHIzKTsNCiAgICAgICAgdXByaW50 ZigiRFI0ID0gMHgleFxuIixyX2RyNCk7DQogICAgICAgIHVwcmludGYoIkRSNSA9IDB4JXhcbiIs cl9kcjUpOw0KICAgICAgICB1cHJpbnRmKCJEUjYgPSAweCV4XG4iLHJfZHI2KTsNCiAgICAgICAg dXByaW50ZigiRFI3ID0gMHgleFxuIixyX2RyNyk7DQoNCnJldHVybiAwOw0KfQ0KDQpzdGF0aWMg aW50IGxvYWQoc3RydWN0IG1vZHVsZSAqbW9kdWxlLCBpbnQgY21kLCBjaGFyICphcmcpDQp7DQog ICAgICAgIGludCBlcnJvciA9IDA7DQoNCiAgICAgICAgc3dpdGNoIChjbWQpIHsNCiAgICAgICAg Y2FzZSBNT0RfTE9BRDoNCiAgICAgICAgICAgICAgICAgICAgICAgIG15X2Z1bmN0aW9uKCk7DQog ICAgICAgICAgICAgICAgYnJlYWs7DQoNCiAgICAgICAgICAgICAgICBjYXNlIE1PRF9VTkxPQUQ6 DQogICAgICAgICAgICAgICAgYnJlYWs7DQoNCiAgICAgICAgZGVmYXVsdDoNCiAgICAgICAgICAg ICAgICBlcnJvciA9IEVPUE5PVFNVUFA7DQogICAgICAgICAgICAgICAgYnJlYWs7DQogICAgICAg IH0NCg0KICAgICAgICByZXR1cm4oZXJyb3IpOw0KfQ0KDQpzdGF0aWMgbW9kdWxlZGF0YV90IERS X0ZyZWVCU0RfbW9kID0gew0KICAgICAgICAiRFJfRnJlZUJTRCIsICAgICAgICAgICAvKiBtb2R1 bGUgbmFtZSAqLw0KICAgICAgICAobW9kZXZlbnRoYW5kX3QpbG9hZCwgICAgICAgICAgICAgICAg ICAgLyogZXZlbnQgaGFuZGxlciAqLw0KICAgICAgICBOVUxMICAgICAgICAgICAgICAgICAgICAv KiBleHRyYSBkYXRhICovDQp9Ow0KDQpERUNMQVJFX01PRFVMRShEUl9GcmVlQlNELCBEUl9GcmVl QlNEX21vZCwgU0lfU1VCX0RSSVZFUlMsIFNJX09SREVSX01JRERMRSk7DQo= --_0b5bcd7e-f970-44b3-a991-b9dfefddba14_-- From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 18 06:37:34 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1B7CD1065670 for ; Thu, 18 Feb 2010 06:37:34 +0000 (UTC) (envelope-from kientzle@freebsd.org) Received: from monday.kientzle.com (kientzle.com [66.166.149.50]) by mx1.freebsd.org (Postfix) with ESMTP id D05EC8FC12 for ; Thu, 18 Feb 2010 06:37:33 +0000 (UTC) Received: (from root@localhost) by monday.kientzle.com (8.14.3/8.14.3) id o1I6bcw1070970; Thu, 18 Feb 2010 06:37:38 GMT (envelope-from kientzle@freebsd.org) Received: from dark.x.kientzle.com (fw2.kientzle.com [10.123.1.2]) by kientzle.com with SMTP id ia59izfsjt8rfbxyf46m363fha; Thu, 18 Feb 2010 06:37:38 +0000 (UTC) (envelope-from kientzle@freebsd.org) Message-ID: <4B7CE066.4030403@freebsd.org> Date: Wed, 17 Feb 2010 22:38:30 -0800 From: Tim Kientzle User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.8.1.21) Gecko/20090601 SeaMonkey/1.1.16 MIME-Version: 1.0 To: Juergen Lock References: <20100217215940.GA19713@triton8.kn-bremen.de> In-Reply-To: <20100217215940.GA19713@triton8.kn-bremen.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org Subject: Re: "tar tfv /dev/cd0" speedup patch X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 06:37:34 -0000 Juergen Lock wrote: > > ... since bsdtar/libarchive know iso9660 I just did the command in the > Subject. It worked, but it was sloow... :( Apparently it read all of > the disc without seeking. The following patch fixes this, is something > like this desired? If yes I could look how to do the same for Linux, Juergen, This is great! If you can figure out how to get this right, I would really appreciate it. If you have a tape drive handy, definitely test with that. My first attempts here actually broke reading from tape drives, which is why the current code is so conservative. Minor style comments: > else if (S_ISCHR(st.st_mode) && > !ioctl(fd, DIOCGMEDIASIZE, &mediasize) && mediasize) { Please be explicit: S_ISCHR() && ioctl() == 0 && mediasize > 0 > archive_read_extract_set_skip_file(a, st.st_dev, st.st_ino); extract_skip_file isn't needed here; we don't read the contents of device nodes. Let me know as soon as you have something you're confident of. Cheers, Tim From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 18 07:50:38 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DAB951065676 for ; Thu, 18 Feb 2010 07:50:38 +0000 (UTC) (envelope-from to.my.trociny@gmail.com) Received: from mail-fx0-f219.google.com (mail-fx0-f219.google.com [209.85.220.219]) by mx1.freebsd.org (Postfix) with ESMTP id 702C38FC08 for ; Thu, 18 Feb 2010 07:50:38 +0000 (UTC) Received: by fxm19 with SMTP id 19so1121103fxm.3 for ; Wed, 17 Feb 2010 23:50:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:to:subject:organization:from :date:message-id:user-agent:mime-version:content-type; bh=oAnRk3RiGbVmyO5aWB6Q8XC4TLOQyX0P4jbffyI2N4U=; b=OdF65dh7KEmvEPiFYs5Hb8FPN6ovC9DMuW9yTg8uYZKtD2a+8CVHKkwPQIf0w1a1N/ ZVRnP3lc3LaOyJCJd0+zn02/tTLsAWZp5Xbg1PiMsN81y5HoKu79vQYN5rN9y8JUP3Tr kId+TZ4WQOJIFjMLfaeBVtZ8NjGwebk1RMYMc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=to:subject:organization:from:date:message-id:user-agent :mime-version:content-type; b=uBY3cFHlqxOQVIXof7HHmETM88de4L2CznSlS7IMGGJyIU8yZgMyF1GJ+tPPlL81Vc p2oYN2i3Y+sEQEHotEjKPMmKkMmrYbUw2Ce5AVnKWKQna2KRNpqKGAeUAhZuI4JVcjfz Aps9R+HbNqToee0GDOu1EiXlVCMOga2mMspVA= Received: by 10.87.69.17 with SMTP id w17mr16482441fgk.41.1266479437052; Wed, 17 Feb 2010 23:50:37 -0800 (PST) Received: from localhost (ms.singlescrowd.net [80.85.90.67]) by mx.google.com with ESMTPS id e3sm13594344fga.6.2010.02.17.23.50.35 (version=TLSv1/SSLv3 cipher=RC4-MD5); Wed, 17 Feb 2010 23:50:35 -0800 (PST) To: freebsd-hackers@FreeBSD.ORG Organization: TOA Ukraine From: Mikolaj Golub Date: Thu, 18 Feb 2010 09:50:33 +0200 Message-ID: <86ocjn3rue.fsf@zhuzha.ua1> User-Agent: Gnus/5.11 (Gnus v5.11) Emacs/22.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Subject: unix socket: race on close? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 07:50:39 -0000 Hi, Below is a simple test code with unix sockets: the client does connect()/close() in loop and the server -- accept()/close(). Sometimes close() fails with 'Socket is not connected' error: a.out: parent: close error: 57 or a.out: child: close error: 57 It looks for me like some race in close(). Looking at uipc_socket.c:soclose(): int soclose(struct socket *so) { int error = 0; KASSERT(!(so->so_state & SS_NOFDREF), ("soclose: SS_NOFDREF on enter")); CURVNET_SET(so->so_vnet); funsetown(&so->so_sigio); if (so->so_state & SS_ISCONNECTED) { if ((so->so_state & SS_ISDISCONNECTING) == 0) { error = sodisconnect(so); if (error) goto drop; } Isn't the problem here? so_state is checked for SS_ISCONNECTED and SS_ISDISCONNECTING without locking and then sodisconnect() is called, which closes both sockets of the connection. So it looks for me that if the close() is called for both ends simultaneously it is possible that sodisconnect() will be called for both ends and for one ENOTCONN will be returned. Or may I have missed something? We have been observing periodically ENOTCONN errors on unix socket close in our applications, so it is not just curiosity :-) (I posted about our problem to freebsd-net@ some time ago but then did not attract any attention http://lists.freebsd.org/pipermail/freebsd-net/2009-December/024047.html). #include #include #include #include #include #include #include #include #include #include #include #include #include #define UNIXSTR_PATH "/tmp/mytest.socket" #define USLEEP 100 int main(int argc, char **argv) { int listenfd, connfd, pid; struct sockaddr_un servaddr; pid = fork(); if (-1 == pid) errx(1, "fork(): %d", errno); if (0 != pid) { /* parent */ if ((listenfd = socket(AF_LOCAL, SOCK_STREAM, 0)) < 0) errx(1, "parent: socket error: %d", errno); unlink(UNIXSTR_PATH); bzero(&servaddr, sizeof(servaddr)); servaddr.sun_family = AF_LOCAL; strcpy(servaddr.sun_path, UNIXSTR_PATH); if (bind(listenfd, (struct sockaddr *) &servaddr, sizeof(servaddr)) < 0) errx(1, "parent: bind error: %d", errno); if (listen(listenfd, 1024) < 0) errx(1, "parent: listen error: %d", errno); for ( ; ; ) { if ((connfd = accept(listenfd, (struct sockaddr *) NULL, NULL)) < 0) errx(1, "parent: accept error: %d", errno); //usleep(USLEEP / 2); // (I) uncomment this or (II) below to avoid the race if (close(connfd) < 0) errx(1, "parent: close error: %d", errno); } } else { /* child */ sleep(1); /* give the parent some time to create the socket */ for ( ; ; ) { if ((connfd = socket(AF_LOCAL, SOCK_STREAM, 0)) < 0) errx(1, "child: socket error: %d", errno); bzero(&servaddr, sizeof(servaddr)); servaddr.sun_family = AF_LOCAL; strcpy(servaddr.sun_path, UNIXSTR_PATH); if (connect(connfd, (struct sockaddr *) &servaddr, sizeof(servaddr)) < 0) errx(1, "child: connect error %d", errno); // usleep(USLEEP); // (II) uncomment this or (I) above to avoid the race if (close(connfd) != 0) errx(1, "child: close error: %d", errno); usleep(USLEEP); } } return 0; } -- Mikolaj Golub From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 18 10:50:00 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ECC9C106566C for ; Thu, 18 Feb 2010 10:50:00 +0000 (UTC) (envelope-from kpielorz_lst@tdx.co.uk) Received: from mail.tdx.com (mail.tdx.com [62.13.128.18]) by mx1.freebsd.org (Postfix) with ESMTP id 761018FC1F for ; Thu, 18 Feb 2010 10:50:00 +0000 (UTC) Received: from HPQuadro64.dmpriest.net.uk (HPQuadro64.dmpriest.net.uk [62.13.130.30]) (authenticated bits=0) by mail.tdx.com (8.14.3/8.14.3/Kp) with ESMTP id o1IAc6Xo034305 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES256-SHA bits=256 verify=NO) for ; Thu, 18 Feb 2010 10:38:06 GMT Date: Thu, 18 Feb 2010 10:37:50 +0000 From: Karl Pielorz To: freebsd-hackers@freebsd.org Message-ID: <0AC6C93D50BE19A97047BAE3@HPQuadro64.dmpriest.net.uk> X-Mailer: Mulberry/4.0.8 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Subject: ZFS'inodes' (as reported by 'df -i') running out? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 10:50:01 -0000 Hi All, I originally posted this in freebsd-fs - but didn't get a reply... I have a number of systems (mostly 7.2-S/amd64) running ZFS. Some of these handle millions of files. I've noticed recently, according to "df -i" I'm starting to run out of inodes on some of them (96% used). e.g. " Filesystem iused ifree %iused Mounted on vol/imap 1726396 69976 96% /vol/imap " I know ZFS doesn't have inodes (think they're znodes), and is capable of handling more files than you can probably sensibly think about on a filesystem - but is "df -i" just getting confused, or do I need to be concerned? Thanks, -Karl From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 18 11:41:53 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 31E5B1065670 for ; Thu, 18 Feb 2010 11:41:53 +0000 (UTC) (envelope-from freebsd-hackers@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id E00948FC08 for ; Thu, 18 Feb 2010 11:41:52 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1Ni4lT-0000jl-Lz for freebsd-hackers@freebsd.org; Thu, 18 Feb 2010 12:41:51 +0100 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 18 Feb 2010 12:41:51 +0100 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 18 Feb 2010 12:41:51 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-hackers@freebsd.org From: Ivan Voras Date: Thu, 18 Feb 2010 12:41:37 +0100 Lines: 27 Message-ID: References: <0AC6C93D50BE19A97047BAE3@HPQuadro64.dmpriest.net.uk> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Thunderbird 2.0.0.23 (X11/20090817) In-Reply-To: <0AC6C93D50BE19A97047BAE3@HPQuadro64.dmpriest.net.uk> Sender: news Subject: Re: ZFS'inodes' (as reported by 'df -i') running out? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 11:41:53 -0000 Karl Pielorz wrote: > > Hi All, > > I originally posted this in freebsd-fs - but didn't get a reply... I > have a number of systems (mostly 7.2-S/amd64) running ZFS. Some of these > handle millions of files. > > I've noticed recently, according to "df -i" I'm starting to run out of > inodes on some of them (96% used). > > e.g. > > " > Filesystem iused ifree %iused Mounted on > vol/imap 1726396 69976 96% /vol/imap > " > > > I know ZFS doesn't have inodes (think they're znodes), and is capable of > handling more files than you can probably sensibly think about on a > filesystem - but is "df -i" just getting confused, or do I need to be > concerned? AFAIK ZFS allocates inodes when needed so df -i reports the previously allocated value. The number of available inodes should automatically grow as you add more files. From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 18 11:59:40 2010 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C0926106566C for ; Thu, 18 Feb 2010 11:59:40 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 7C18C8FC15 for ; Thu, 18 Feb 2010 11:59:40 +0000 (UTC) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id 19C4346B0C; Thu, 18 Feb 2010 06:59:40 -0500 (EST) Date: Thu, 18 Feb 2010 11:59:40 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Mikolaj Golub In-Reply-To: <86ocjn3rue.fsf@zhuzha.ua1> Message-ID: References: <86ocjn3rue.fsf@zhuzha.ua1> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: unix socket: race on close? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 11:59:40 -0000 On Thu, 18 Feb 2010, Mikolaj Golub wrote: > Below is a simple test code with unix sockets: the client does > connect()/close() in loop and the server -- accept()/close(). > > Sometimes close() fails with 'Socket is not connected' error: Hi Mikolaj: Thanks for this report, and sorry about not spotting your earlier post to freebsd-net. I've been fairly preoccupied the last month and not keeping up with the mailing lists. Could I ask you to file a PR on this, and forward me the PR number so I can claim ownership? This should prevent it from getting lost while I catch up. In short, your evaluation seems reasonable to me -- have you tried tweaking soclose() to ignore ENOTCONN from sodisconnect() to confirm this diagnosis fixes all the instances you've been seeing? Robert N M Watson Computer Laboratory University of Cambridge > > a.out: parent: close error: 57 > > or > > a.out: child: close error: 57 > > It looks for me like some race in close(). Looking at uipc_socket.c:soclose(): > > int > soclose(struct socket *so) > { > int error = 0; > > KASSERT(!(so->so_state & SS_NOFDREF), ("soclose: SS_NOFDREF on enter")); > > CURVNET_SET(so->so_vnet); > funsetown(&so->so_sigio); > if (so->so_state & SS_ISCONNECTED) { > if ((so->so_state & SS_ISDISCONNECTING) == 0) { > error = sodisconnect(so); > if (error) > goto drop; > } > > Isn't the problem here? so_state is checked for SS_ISCONNECTED and > SS_ISDISCONNECTING without locking and then sodisconnect() is called, which > closes both sockets of the connection. So it looks for me that if the close() > is called for both ends simultaneously it is possible that sodisconnect() will > be called for both ends and for one ENOTCONN will be returned. Or may I have > missed something? > > We have been observing periodically ENOTCONN errors on unix socket close in > our applications, so it is not just curiosity :-) (I posted about our problem > to freebsd-net@ some time ago but then did not attract any attention > http://lists.freebsd.org/pipermail/freebsd-net/2009-December/024047.html). > > #include > #include > #include > #include > #include > #include > #include > #include > #include > #include > #include > #include > #include > > #define UNIXSTR_PATH "/tmp/mytest.socket" > #define USLEEP 100 > > int main(int argc, char **argv) > { > int listenfd, connfd, pid; > struct sockaddr_un servaddr; > > pid = fork(); > if (-1 == pid) > errx(1, "fork(): %d", errno); > > if (0 != pid) { /* parent */ > > if ((listenfd = socket(AF_LOCAL, SOCK_STREAM, 0)) < 0) > errx(1, "parent: socket error: %d", errno); > > unlink(UNIXSTR_PATH); > bzero(&servaddr, sizeof(servaddr)); > servaddr.sun_family = AF_LOCAL; > strcpy(servaddr.sun_path, UNIXSTR_PATH); > > if (bind(listenfd, (struct sockaddr *) &servaddr, sizeof(servaddr)) < 0) > errx(1, "parent: bind error: %d", errno); > > if (listen(listenfd, 1024) < 0) > errx(1, "parent: listen error: %d", errno); > > for ( ; ; ) { > if ((connfd = accept(listenfd, (struct sockaddr *) NULL, NULL)) < 0) > errx(1, "parent: accept error: %d", errno); > > //usleep(USLEEP / 2); // (I) uncomment this or (II) below to avoid the race > > if (close(connfd) < 0) > errx(1, "parent: close error: %d", errno); > } > > } else { /* child */ > > sleep(1); /* give the parent some time to create the socket */ > > for ( ; ; ) { > > if ((connfd = socket(AF_LOCAL, SOCK_STREAM, 0)) < 0) > errx(1, "child: socket error: %d", errno); > > bzero(&servaddr, sizeof(servaddr)); > servaddr.sun_family = AF_LOCAL; > strcpy(servaddr.sun_path, UNIXSTR_PATH); > > if (connect(connfd, (struct sockaddr *) &servaddr, sizeof(servaddr)) < 0) > errx(1, "child: connect error %d", errno); > > // usleep(USLEEP); // (II) uncomment this or (I) above to avoid the race > > if (close(connfd) != 0) > errx(1, "child: close error: %d", errno); > > usleep(USLEEP); > } > } > > return 0; > } > > -- > Mikolaj Golub > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 18 12:04:48 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B2338106566B; Thu, 18 Feb 2010 12:04:48 +0000 (UTC) (envelope-from pluknet@gmail.com) Received: from mail-bw0-f224.google.com (mail-bw0-f224.google.com [209.85.218.224]) by mx1.freebsd.org (Postfix) with ESMTP id 1873E8FC1D; Thu, 18 Feb 2010 12:04:47 +0000 (UTC) Received: by bwz24 with SMTP id 24so51424bwz.13 for ; Thu, 18 Feb 2010 04:04:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=EaDd/eEiVh6nP8CNn+c91O+mwpKTJY9Aj4WZJ8ejtvo=; b=dS66fjwyIyOPrLqwtv+HO6ma9hcPssEsmymsioyYre9QuUXLyJ9YydWtg7LSUPtYNU 7wuUuemb1jl/yAgNMAOddqyaDuiZfKNHhykvNAJ/j7kM/CpbkQhlz/LhmOXiEyLUBnss cCgo2r7G/rY2ZLo70w5rqdg4nubMX/qZLFc3o= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=RzQyHh/Absq8eWeatQkW1JMtqS9MNKd1rRcecwWvMe9oA3a9VrEdYTI0AC6jzLTkCi srCPOUQudVwmClJagqE5exKFDYwMqS6SGCz/Ssew5ItPMjdbKT8oCJaNQHDBgL8fulPp 1SQMYPpQ0u336hLrmrAIGnQpXqT/pF7r6h9OE= MIME-Version: 1.0 Received: by 10.204.39.211 with SMTP id h19mr3528298bke.164.1266494686896; Thu, 18 Feb 2010 04:04:46 -0800 (PST) In-Reply-To: References: <0AC6C93D50BE19A97047BAE3@HPQuadro64.dmpriest.net.uk> Date: Thu, 18 Feb 2010 15:04:46 +0300 Message-ID: From: pluknet To: Ivan Voras Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: ZFS'inodes' (as reported by 'df -i') running out? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 12:04:48 -0000 On 18 February 2010 14:41, Ivan Voras wrote: > Karl Pielorz wrote: >> >> Hi All, >> >> I originally posted this in freebsd-fs - but didn't get a reply... I >> have a number of systems (mostly 7.2-S/amd64) running ZFS. Some of these >> handle millions of files. >> >> I've noticed recently, according to "df -i" I'm starting to run out of >> inodes on some of them (96% used). >> >> e.g. >> >> " >> Filesystem =A0 =A0 iused =A0 =A0ifree %iused Mounted on >> vol/imap =A0 =A0 =A01726396 =A0 69976 =A0 96% =A0/vol/imap >> " >> >> >> I know ZFS doesn't have inodes (think they're znodes), and is capable of >> handling more files than you can probably sensibly think about on a >> filesystem - but is "df -i" just getting confused, or do I need to be >> concerned? > > AFAIK ZFS allocates inodes when needed so df -i reports the previously > allocated value. The number of available inodes should automatically > grow as you add more files. Sorta jfyi. That's what I see on Solaris: df: operation not applicable for FSType zfs --=20 wbr, pluknet From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 18 12:26:37 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 904A01065672; Thu, 18 Feb 2010 12:26:37 +0000 (UTC) (envelope-from kraduk@googlemail.com) Received: from mail-fx0-f219.google.com (mail-fx0-f219.google.com [209.85.220.219]) by mx1.freebsd.org (Postfix) with ESMTP id 4B87B8FC08; Thu, 18 Feb 2010 12:26:34 +0000 (UTC) Received: by fxm19 with SMTP id 19so1405924fxm.3 for ; Thu, 18 Feb 2010 04:26:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=Uy7QNb0VlmIk0WNP+IKrQrTVXN5DDFCr5yF9u7LVtD8=; b=eEGfmDSFuqpxijXGgfbF3RLxc3CiA+jzXebRisBtg9e4SJCavs/bvYkDuHmRz/LapC nRdGaj0VYGRvG4wAX6H16I04PIP0COgnLbV4bsD6iuKJLTieiGOCueGem/FzA7hTKokD wFPX+xukyWHPVn0IYAR1P1xGB0Z5IDhPBxMtk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=A2re0YZ5UDn28XibAi8B0lyem150YdCgiKBD2/11dpKZvfgnJBcXN38Mb8rKlIpu3j BvMT8G/4+/SIN7R7IQLElwaJWrOAOu1zEJAPOQAf+jVGatN9DBWIdGg1xiHz1jVm7VoY W8eBwSW3kpSToNxdF2OnUZDmrjvguqqb0hJ4Y= MIME-Version: 1.0 Received: by 10.239.189.76 with SMTP id s12mr1006114hbh.111.1266495993844; Thu, 18 Feb 2010 04:26:33 -0800 (PST) In-Reply-To: References: <0AC6C93D50BE19A97047BAE3@HPQuadro64.dmpriest.net.uk> Date: Thu, 18 Feb 2010 12:26:33 +0000 Message-ID: From: krad To: pluknet X-Mailman-Approved-At: Thu, 18 Feb 2010 12:53:35 +0000 Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org, Ivan Voras Subject: Re: ZFS'inodes' (as reported by 'df -i') running out? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 12:26:37 -0000 On 18 February 2010 12:04, pluknet wrote: > On 18 February 2010 14:41, Ivan Voras wrote: > > Karl Pielorz wrote: > >> > >> Hi All, > >> > >> I originally posted this in freebsd-fs - but didn't get a reply... I > >> have a number of systems (mostly 7.2-S/amd64) running ZFS. Some of these > >> handle millions of files. > >> > >> I've noticed recently, according to "df -i" I'm starting to run out of > >> inodes on some of them (96% used). > >> > >> e.g. > >> > >> " > >> Filesystem iused ifree %iused Mounted on > >> vol/imap 1726396 69976 96% /vol/imap > >> " > >> > >> > >> I know ZFS doesn't have inodes (think they're znodes), and is capable of > >> handling more files than you can probably sensibly think about on a > >> filesystem - but is "df -i" just getting confused, or do I need to be > >> concerned? > > > > AFAIK ZFS allocates inodes when needed so df -i reports the previously > > allocated value. The number of available inodes should automatically > > grow as you add more files. > > Sorta jfyi. That's what I see on Solaris: > df: operation not applicable for FSType zfs > > > -- > wbr, > pluknet > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > Just wait until you start using dedup and get magically growing disks with df 8)) $ dd if=/dev/urandom of=/tmp/test bs=128k count=1 1+0 records in 1+0 records out 131072 bytes (131 kB) copied, 0.00317671 s, 41.3 MB/s $ zpool list NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT dedup 1016M 95.5K 1016M 0% 1.00x ONLINE - rpool 148G 103G 45.0G 69% 1.00x ONLINE - $ zfs set dedup=on dedup $ df -h /dedup Filesystem Size Used Avail Use% Mounted on dedup 984M 21K 984M 1% /dedup $ seq 1 1000| while read a ; do cp /tmp/test /dedup/test.$RANDOM; done $ df -h /dedup/ Filesystem Size Used Avail Use% Mounted on dedup 1.1G 116M 984M 11% /dedup $ zpool list dedup NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT dedup 1016M 360K 1016M 0% 921.00x ONLINE - Its only available on opensolaris dev at the moment so dont get to excited, but in a year or so i mat hit freebsd. You will need a beefy machine though with a ssd backed l2arc From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 18 13:21:00 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2D36B1065672 for ; Thu, 18 Feb 2010 13:21:00 +0000 (UTC) (envelope-from kpielorz_lst@tdx.co.uk) Received: from mail.tdx.com (mail.tdx.com [62.13.128.18]) by mx1.freebsd.org (Postfix) with ESMTP id C06358FC12 for ; Thu, 18 Feb 2010 13:20:58 +0000 (UTC) Received: from HPQuadro64.dmpriest.net.uk (HPQuadro64.dmpriest.net.uk [62.13.130.30]) (authenticated bits=0) by mail.tdx.com (8.14.3/8.14.3/Kp) with ESMTP id o1IDKvMW048161 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES256-SHA bits=256 verify=NO); Thu, 18 Feb 2010 13:20:58 GMT Date: Thu, 18 Feb 2010 13:20:41 +0000 From: Karl Pielorz To: Ivan Voras , freebsd-hackers@freebsd.org Message-ID: In-Reply-To: References: <0AC6C93D50BE19A97047BAE3@HPQuadro64.dmpriest.net.uk> X-Mailer: Mulberry/4.0.8 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Cc: Subject: Re: ZFS'inodes' (as reported by 'df -i') running out? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 13:21:00 -0000 --On 18 February 2010 12:41 +0100 Ivan Voras wrote: >> I know ZFS doesn't have inodes (think they're znodes), and is capable of >> handling more files than you can probably sensibly think about on a >> filesystem - but is "df -i" just getting confused, or do I need to be >> concerned? > > AFAIK ZFS allocates inodes when needed so df -i reports the previously > allocated value. The number of available inodes should automatically > grow as you add more files. Fantastic! Thanks for the clarification - I though it'd be something like that, but thought I'd better check before running out, Regards, -Karl From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 18 13:28:18 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3BEE0106566B for ; Thu, 18 Feb 2010 13:28:18 +0000 (UTC) (envelope-from anderson_underground@hotmail.com) Received: from snt0-omc2-s24.snt0.hotmail.com (snt0-omc2-s24.snt0.hotmail.com [65.55.90.99]) by mx1.freebsd.org (Postfix) with ESMTP id 1175B8FC1B for ; Thu, 18 Feb 2010 13:28:17 +0000 (UTC) Received: from SNT137-W22 ([65.55.90.73]) by snt0-omc2-s24.snt0.hotmail.com with Microsoft SMTPSVC(6.0.3790.3959); Thu, 18 Feb 2010 05:28:17 -0800 Message-ID: X-Originating-IP: [189.105.64.253] From: Anderson Eduardo To: Date: Thu, 18 Feb 2010 10:28:17 -0300 Importance: Normal MIME-Version: 1.0 X-OriginalArrivalTime: 18 Feb 2010 13:28:17.0318 (UTC) FILETIME=[3AE97460:01CAB09E] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: =?utf-8?q?Debug_registers_on_FreeBSD=E2=80=8F?= X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 13:28:18 -0000 DQpJIHdhbnQgc29tZSBwYXBlcnMsIGRvY3VtZW50YXRpb25zIG9yIGFydGljbGVzIGFib3V0IERl YnVnIFJlZ2lzdGVycyBvbiBGcmVlQlNELCBJIGdvb2dsZWQgYW5kIGRpZG4ndCBmaW5kIGFueSBy ZWZlcmVuY2VzIGFib3V0IGl0LiBJJ20gdXNpbmcgaTM4NiBBTUQuDQoNCkNhbiBzb21lb25lIGhl bHAgbWU/DQogCQkgCSAgIAkJICANCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fDQpRdWVyIGRlaXhhciBzZXVzIHbDrWRlb3Mg bWFpcyBkaXZlcnRpZG9zPyBDb20gbyBNb3ZpZSBNYWtlciBpc3NvIGZpY2EgZsOhY2lsLg0KaHR0 cDovL3d3dy53aW5kb3dzbGl2ZS5jb20uYnIvcHVibGljL3RpcC5hc3B4L3ZpZXcvODc/cHJvZHVj dD00Jm9jaWQ9V2luZG93cyBMaXZlOkRpY2FzIC0gTW92aWUgTWFrZXI6SG90bWFpbDpUYWdsaW5l OjF4MTpUaXR1bG8gTGVnZW5kYXMgQ3JlZGl0b3M= From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 18 14:00:58 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1A1381065670 for ; Thu, 18 Feb 2010 14:00:58 +0000 (UTC) (envelope-from dunceor@gmail.com) Received: from mail-bw0-f224.google.com (mail-bw0-f224.google.com [209.85.218.224]) by mx1.freebsd.org (Postfix) with ESMTP id A0DFB8FC19 for ; Thu, 18 Feb 2010 14:00:57 +0000 (UTC) Received: by bwz24 with SMTP id 24so143790bwz.13 for ; Thu, 18 Feb 2010 06:00:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=5ZDKERZ2XBYhMTV3QREIAzXqTEVDI/QfxLeFB/DIlZQ=; b=WoyYkk5spdCdjSwB9v8jDAmhn+v3mqruevkOp6o+hugP+rKEUWYKJxlOK25EciWLkf zST72Qrv5os1fAKAsIiOr1pXduswzLZUyITbsePxUMZakn4HGHBPkpKu4GJvX4MX8AcO gMsy3wn9TUjvk+7Cyb2Spqqhe2O/b4+bInV/o= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=DSjbe89roYUfTJIHoU56wfFch6jbB1unDjstZcPFKkwnrJk7VFAhF2q9zxkjXgFUtD PMABKrX8NS3H6c4V5S1dR3gTjiZSL12UXqyPBm3Yx1EB4nQa/eBFgxoBodlSzILCuQTg UFL2nUbwK6CpC5PZvrAY8MuSrLHEDe9lIcvxo= MIME-Version: 1.0 Received: by 10.204.10.151 with SMTP id p23mr259228bkp.80.1266499839296; Thu, 18 Feb 2010 05:30:39 -0800 (PST) In-Reply-To: References: Date: Thu, 18 Feb 2010 14:30:39 +0100 Message-ID: <5d84cb31002180530j35a77a3ayfdaffd9660e5129e@mail.gmail.com> From: Dunceor To: Anderson Eduardo Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: =?utf-8?q?Re=3A_Debug_registers_on_FreeBSD=E2=80=8F?= X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 14:00:58 -0000 You probobly need to specify it some more. Debug registers for what? The x86 processor, some other hardware or what? What do you need it for since you need debug registers? BR Dunceor On Thu, Feb 18, 2010 at 2:28 PM, Anderson Eduardo wrote: > > I want some papers, documentations or articles about Debug Registers on F= reeBSD, I googled and didn't find any references about it. I'm using i386 A= MD. > > Can someone help me? > > _________________________________________________________________ > Quer deixar seus v=C3=ADdeos mais divertidos? Com o Movie Maker isso fica= f=C3=A1cil. > http://www.windowslive.com.br/public/tip.aspx/view/87?product=3D4&ocid=3D= Windows Live:Dicas - Movie Maker:Hotmail:Tagline:1x1:Titulo Legendas Credit= os > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " > From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 18 14:46:30 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 101591065692; Thu, 18 Feb 2010 14:46:30 +0000 (UTC) (envelope-from to.my.trociny@gmail.com) Received: from mail-bw0-f216.google.com (mail-bw0-f216.google.com [209.85.218.216]) by mx1.freebsd.org (Postfix) with ESMTP id 68DB28FC1B; Thu, 18 Feb 2010 14:46:28 +0000 (UTC) Received: by bwz8 with SMTP id 8so198174bwz.3 for ; Thu, 18 Feb 2010 06:46:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:to:cc:subject:references :organization:from:date:in-reply-to:message-id:user-agent :mime-version:content-type; bh=KeTsnlfQkhFI8iLMZH9bSfdnlRp583tOFVCFIVcJFEg=; b=rDilOHAnf0Hs1LYh8VtVe7HOdUzQuDEGZhNf0VlaQVsnXIuMq2X+aTkKuMqkCb83Nh 51lTDlpqzABqOAE+07QD1xJseswRzDtcNscRld8gP1yS8/UlTmePjBj3v25USZLVo7xI fND6eKqgsdr4xnIgnPJY7xc/r2MRjDZwB35HM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=to:cc:subject:references:organization:from:date:in-reply-to :message-id:user-agent:mime-version:content-type; b=cp76PqHwxxG48hoJMVJzItqvizEWM0Af1GGXkIv5sYCNcu099kC/pXdIrGPsmk3Unr AyGJJCipC1JVemBPbcillVVYg+IZYGA+X0crq3+StXMDLq3jdmdNQJOr3wTyecQ/kx3j lL/RY1tRbh5Bcxa3hdgPJH46aaQZUwsoz37SA= Received: by 10.204.24.139 with SMTP id v11mr5880892bkb.121.1266504388051; Thu, 18 Feb 2010 06:46:28 -0800 (PST) Received: from localhost (ms.singlescrowd.net [80.85.90.67]) by mx.google.com with ESMTPS id 13sm102583bwz.15.2010.02.18.06.46.26 (version=TLSv1/SSLv3 cipher=RC4-MD5); Thu, 18 Feb 2010 06:46:27 -0800 (PST) To: Robert Watson References: <86ocjn3rue.fsf@zhuzha.ua1> Organization: TOA Ukraine From: Mikolaj Golub Date: Thu, 18 Feb 2010 16:46:25 +0200 In-Reply-To: (Robert Watson's message of "Thu\, 18 Feb 2010 11\:59\:40 +0000 \(GMT\)") Message-ID: <86635ufvpa.fsf@zhuzha.ua1> User-Agent: Gnus/5.11 (Gnus v5.11) Emacs/22.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: unix socket: race on close? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 14:46:30 -0000 On Thu, 18 Feb 2010 11:59:40 +0000 (GMT) Robert Watson wrote: > On Thu, 18 Feb 2010, Mikolaj Golub wrote: > >> Below is a simple test code with unix sockets: the client does >> connect()/close() in loop and the server -- accept()/close(). >> >> Sometimes close() fails with 'Socket is not connected' error: > > Hi Mikolaj: > > Thanks for this report, and sorry about not spotting your earlier post > to freebsd-net. I've been fairly preoccupied the last month and not > keeping up with the mailing lists. Could I ask you to file a PR on > this, and forward me the PR number so I can claim ownership? This > should prevent it from getting lost while I catch up. kern/144061 > In short, your evaluation seems reasonable to me -- have you tried > tweaking soclose() to ignore ENOTCONN from sodisconnect() to confirm > this diagnosis fixes all the instances you've been seeing? I just have done this: 1) add logging the error when sodisconnect() returns error: --- uipc_socket.c.orig 2010-02-18 14:25:25.000000000 +0200 +++ uipc_socket.c 2010-02-18 14:55:26.000000000 +0200 @@ -120,6 +120,7 @@ __FBSDID("$FreeBSD: src/sys/kern/uipc_so #include #include #include +#include #include #include #include @@ -136,6 +137,7 @@ __FBSDID("$FreeBSD: src/sys/kern/uipc_so #include + #ifdef COMPAT_IA32 #include #include @@ -657,7 +659,7 @@ soclose(struct socket *so) if ((so->so_state & SS_ISDISCONNECTING) == 0) { error = sodisconnect(so); if (error) - goto drop; + log(LOG_INFO, "soclose: sodisconnect error: %d\n", error); } if (so->so_options & SO_LINGER) { if ((so->so_state & SS_ISDISCONNECTING) && Then on every error exit of the test application, like this a.out: parent: close error: 57 I have in the message log: Feb 18 15:35:32 zhuzha kernel: soclose: sodisconnect error: 57 2) add logging the error when sodisconnect() returns error and ignore the error: --- uipc_socket.c.orig 2010-02-18 14:25:25.000000000 +0200 +++ uipc_socket.c 2010-02-18 15:41:07.000000000 +0200 @@ -120,6 +120,7 @@ __FBSDID("$FreeBSD: src/sys/kern/uipc_so #include #include #include +#include #include #include #include @@ -136,6 +137,7 @@ __FBSDID("$FreeBSD: src/sys/kern/uipc_so #include + #ifdef COMPAT_IA32 #include #include @@ -656,8 +658,11 @@ soclose(struct socket *so) if (so->so_state & SS_ISCONNECTED) { if ((so->so_state & SS_ISDISCONNECTING) == 0) { error = sodisconnect(so); - if (error) - goto drop; + if (error) { + log(LOG_INFO, "soclose: sodisconnect error: %d\n", error); + if (error == ENOTCONN) + error = 0; + } } if (so->so_options & SO_LINGER) { if ((so->so_state & SS_ISDISCONNECTING) && After this the test application does not exits and I see in the message log: Feb 18 16:02:37 zhuzha kernel: soclose: sodisconnect error: 57 Feb 18 16:03:31 zhuzha kernel: soclose: sodisconnect error: 57 Feb 18 16:05:49 zhuzha last message repeated 4 times Feb 18 16:15:50 zhuzha last message repeated 13 times -- Mikolaj Golub From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 18 14:48:41 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 236C01065672 for ; Thu, 18 Feb 2010 14:48:41 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id EB5118FC08 for ; Thu, 18 Feb 2010 14:48:40 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 9F82146B4C; Thu, 18 Feb 2010 09:48:40 -0500 (EST) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id EF47F8A025; Thu, 18 Feb 2010 09:48:39 -0500 (EST) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Thu, 18 Feb 2010 09:44:24 -0500 User-Agent: KMail/1.12.1 (FreeBSD/7.2-CBSD-20100120; KDE/4.3.1; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201002180944.24085.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Thu, 18 Feb 2010 09:48:40 -0500 (EST) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.5 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: Anderson Eduardo Subject: Re: Debug registers on FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 14:48:41 -0000 On Wednesday 17 February 2010 11:07:36 pm Anderson Eduardo wrote: > > Hello Folks, > > First, I'm starting in the world of the debugging FreeBSD Kernel and, I don't know to use kgdb and ddb properly and, I need set breakpoints using debug register. Those breakpoints in the all tasks. This last might be tricky to accomplish because cpu_switch() will swap out the debug registers if a thread is using them. However, you can easily manage hardware breakpoints from ddb using the 'hwatch' command. It is documented in ddb(4). kgdb doesn't support setting hardware watch points it seems. Note that you can try to use 'db_md_set_watchpoint()' from your kernel module (with DDB compiled into your kernel) if you want to set the address progammatically rather than via the 'hwatch' command in ddb. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 18 15:12:44 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D30E3106566B; Thu, 18 Feb 2010 15:12:44 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 13A228FC17; Thu, 18 Feb 2010 15:12:43 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id o1IFCdPX009430 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 18 Feb 2010 17:12:39 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.3/8.14.3) with ESMTP id o1IFCdoW041318; Thu, 18 Feb 2010 17:12:39 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.3/8.14.3/Submit) id o1IFCdBV041317; Thu, 18 Feb 2010 17:12:39 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 18 Feb 2010 17:12:39 +0200 From: Kostik Belousov To: John Baldwin Message-ID: <20100218151238.GR50403@deviant.kiev.zoral.com.ua> References: <201002180944.24085.jhb@freebsd.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="NOl3kBTBh+CNLogO" Content-Disposition: inline In-Reply-To: <201002180944.24085.jhb@freebsd.org> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-hackers@freebsd.org, Anderson Eduardo Subject: Re: Debug registers on FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 15:12:44 -0000 --NOl3kBTBh+CNLogO Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Feb 18, 2010 at 09:44:24AM -0500, John Baldwin wrote: > On Wednesday 17 February 2010 11:07:36 pm Anderson Eduardo wrote: > >=20 > > Hello Folks, > >=20 > > First, I'm starting in the world of the debugging FreeBSD Kernel and, I= =20 > don't know to use kgdb and ddb properly and, I need set breakpoints using= =20 > debug register. Those breakpoints in the all tasks. >=20 > This last might be tricky to accomplish because cpu_switch() will swap out > the debug registers if a thread is using them. However, you can easily > manage hardware breakpoints from ddb using the 'hwatch' command. It is= =20 > documented in ddb(4). kgdb doesn't support setting hardware watch points > it seems. Note that you can try to use 'db_md_set_watchpoint()' from your > kernel module (with DDB compiled into your kernel) if you want to set the > address progammatically rather than via the 'hwatch' command in ddb. I remember that ddb only sets %dr on the CPU it happens to execute. --NOl3kBTBh+CNLogO Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (FreeBSD) iEYEARECAAYFAkt9WOYACgkQC3+MBN1Mb4i15QCgr5s1ltpufoPK906E5sP8NX7V mjIAn0QNVW2Pm4v+GIVzMIeCq1exUmcW =+rbd -----END PGP SIGNATURE----- --NOl3kBTBh+CNLogO-- From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 18 15:13:49 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EBD37106566C for ; Thu, 18 Feb 2010 15:13:49 +0000 (UTC) (envelope-from freebsd-hackers@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id A6CB78FC0C for ; Thu, 18 Feb 2010 15:13:49 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1Ni84Z-0003uv-4u for freebsd-hackers@freebsd.org; Thu, 18 Feb 2010 16:13:47 +0100 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 18 Feb 2010 16:13:47 +0100 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 18 Feb 2010 16:13:47 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-hackers@freebsd.org From: Ivan Voras Date: Thu, 18 Feb 2010 16:13:25 +0100 Lines: 11 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.5) Gecko/20100118 Thunderbird/3.0 In-Reply-To: Sender: news Subject: Re: Debug registers on =?utf-8?q?FreeBSD=E2=80=8F?= X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 15:13:50 -0000 On 02/18/10 14:28, Anderson Eduardo wrote: > > I want some papers, documentations or articles about Debug Registers on FreeBSD, I googled and didn't find any references about it. I'm using i386 AMD. > > Can someone help me? Maybe you are thinking about this: http://www.freebsd.org/cgi/man.cgi?pmc ? From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 18 15:24:05 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 03F08106566C for ; Thu, 18 Feb 2010 15:24:05 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id CB7558FC1A for ; Thu, 18 Feb 2010 15:24:04 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 8602746B39; Thu, 18 Feb 2010 10:24:04 -0500 (EST) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id D554A8A021; Thu, 18 Feb 2010 10:24:03 -0500 (EST) From: John Baldwin To: Kostik Belousov Date: Thu, 18 Feb 2010 10:23:34 -0500 User-Agent: KMail/1.12.1 (FreeBSD/7.2-CBSD-20100120; KDE/4.3.1; amd64; ; ) References: <201002180944.24085.jhb@freebsd.org> <20100218151238.GR50403@deviant.kiev.zoral.com.ua> In-Reply-To: <20100218151238.GR50403@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201002181023.34503.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Thu, 18 Feb 2010 10:24:03 -0500 (EST) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.5 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: freebsd-hackers@freebsd.org, Anderson Eduardo Subject: Re: Debug registers on FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 15:24:05 -0000 On Thursday 18 February 2010 10:12:39 am Kostik Belousov wrote: > On Thu, Feb 18, 2010 at 09:44:24AM -0500, John Baldwin wrote: > > On Wednesday 17 February 2010 11:07:36 pm Anderson Eduardo wrote: > > > > > > Hello Folks, > > > > > > First, I'm starting in the world of the debugging FreeBSD Kernel and, I > > don't know to use kgdb and ddb properly and, I need set breakpoints using > > debug register. Those breakpoints in the all tasks. > > > > This last might be tricky to accomplish because cpu_switch() will swap out > > the debug registers if a thread is using them. However, you can easily > > manage hardware breakpoints from ddb using the 'hwatch' command. It is > > documented in ddb(4). kgdb doesn't support setting hardware watch points > > it seems. Note that you can try to use 'db_md_set_watchpoint()' from your > > kernel module (with DDB compiled into your kernel) if you want to set the > > address progammatically rather than via the 'hwatch' command in ddb. > > I remember that ddb only sets %dr on the CPU it happens to execute. Yes, same with the db_md_* callbacks. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 18 17:06:57 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E0EFA1065695 for ; Thu, 18 Feb 2010 17:06:57 +0000 (UTC) (envelope-from anderson_underground@hotmail.com) Received: from snt0-omc3-s33.snt0.hotmail.com (snt0-omc3-s33.snt0.hotmail.com [65.55.90.172]) by mx1.freebsd.org (Postfix) with ESMTP id B1C318FC17 for ; Thu, 18 Feb 2010 17:06:57 +0000 (UTC) Received: from SNT137-W57 ([65.55.90.137]) by snt0-omc3-s33.snt0.hotmail.com with Microsoft SMTPSVC(6.0.3790.3959); Thu, 18 Feb 2010 09:06:56 -0800 Message-ID: X-Originating-IP: [200.223.134.34] From: Anderson Eduardo To: , Date: Thu, 18 Feb 2010 14:06:57 -0300 Importance: Normal In-Reply-To: <201002181023.34503.jhb@freebsd.org> References: <201002180944.24085.jhb@freebsd.org> <20100218151238.GR50403@deviant.kiev.zoral.com.ua>, <201002181023.34503.jhb@freebsd.org> MIME-Version: 1.0 X-OriginalArrivalTime: 18 Feb 2010 17:06:56.0816 (UTC) FILETIME=[C6BDFB00:01CAB0BC] Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org Subject: RE: Debug registers on FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 17:06:58 -0000 Thanks all=2C Now I understood too more. I found some manuals of the AMD and I saw more s= ome details about debug registers. I'm going to try doing some hacks today.= If I have questions I send it again. > From: jhb@freebsd.org > To: kostikbel@gmail.com > Subject: Re: Debug registers on FreeBSD > Date: Thu=2C 18 Feb 2010 10:23:34 -0500 > CC: freebsd-hackers@freebsd.org=3B anderson_underground@hotmail.com >=20 > On Thursday 18 February 2010 10:12:39 am Kostik Belousov wrote: > > On Thu=2C Feb 18=2C 2010 at 09:44:24AM -0500=2C John Baldwin wrote: > > > On Wednesday 17 February 2010 11:07:36 pm Anderson Eduardo wrote: > > > >=20 > > > > Hello Folks=2C > > > >=20 > > > > First=2C I'm starting in the world of the debugging FreeBSD Kernel = and=2C I=20 > > > don't know to use kgdb and ddb properly and=2C I need set breakpoints= using=20 > > > debug register. Those breakpoints in the all tasks. > > >=20 > > > This last might be tricky to accomplish because cpu_switch() will swa= p out > > > the debug registers if a thread is using them. However=2C you can ea= sily > > > manage hardware breakpoints from ddb using the 'hwatch' command. It = is=20 > > > documented in ddb(4). kgdb doesn't support setting hardware watch po= ints > > > it seems. Note that you can try to use 'db_md_set_watchpoint()' from= your > > > kernel module (with DDB compiled into your kernel) if you want to set= the > > > address progammatically rather than via the 'hwatch' command in ddb. > >=20 > > I remember that ddb only sets %dr on the CPU it happens to execute. >=20 > Yes=2C same with the db_md_* callbacks. >=20 > --=20 > John Baldwin =20 _________________________________________________________________ Quer deixar seus v=EDdeos mais divertidos? Com o Movie Maker isso fica f=E1= cil. http://www.windowslive.com.br/public/tip.aspx/view/87?product=3D4&ocid=3DWi= ndows Live:Dicas - Movie Maker:Hotmail:Tagline:1x1:Titulo Legendas Creditos= From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 18 17:48:36 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 82FE5106566B for ; Thu, 18 Feb 2010 17:48:36 +0000 (UTC) (envelope-from fernando.apesteguia@gmail.com) Received: from mail-ew0-f211.google.com (mail-ew0-f211.google.com [209.85.219.211]) by mx1.freebsd.org (Postfix) with ESMTP id 17AE18FC14 for ; Thu, 18 Feb 2010 17:48:35 +0000 (UTC) Received: by mail-ew0-f211.google.com with SMTP id 3so9237795ewy.13 for ; Thu, 18 Feb 2010 09:48:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=UK0ja8T1yzv1BLk0xKlNp1wwq2Cm71t2VgH6XWuQ9kc=; b=SUn6GUyfs66NYJVSeN8Pg8Yr9GxnE469Q6HGUyg+hnmgBMfEZrZWnoISFotZ7Ef3dC mc+5kvMgUCN75zlFDmmYdMGUjA9r8BSB4Fr5vZPHaZCMQnVg08tOAuE9GFWs/bW9lfmr NH7k4pgySPE/3rfsaE8b4g/m6pBT2MnuKjkog= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=Gc4JdqW0xZ82utFurqNY/bd8i70f5Gj3cCi0OfJLcZHWnNM/964f+955OWZrYv535Q YrUU83IqfL7AkAfK4o5qeaoMGTCLpRUf393xWNeKdRyE4oLHUce+DV39F/a5jH44pYLd TKP96q3OaylzI4PFovNoSL1GqVZvvj/iNtwOQ= MIME-Version: 1.0 Received: by 10.213.109.143 with SMTP id j15mr6773279ebp.84.1266515315664; Thu, 18 Feb 2010 09:48:35 -0800 (PST) In-Reply-To: <20100217191156.GP50403@deviant.kiev.zoral.com.ua> References: <1bd550a01002171051n7117895avb5cf57fb7fbb9388@mail.gmail.com> <20100217191156.GP50403@deviant.kiev.zoral.com.ua> Date: Thu, 18 Feb 2010 18:48:35 +0100 Message-ID: <1bd550a01002180948r48300f8em15efca184d99ed98@mail.gmail.com> From: =?ISO-8859-1?Q?Fernando_Apestegu=EDa?= To: Kostik Belousov Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Hackers Subject: Re: linprocfs proc/pid/environ patch & list question X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 17:48:36 -0000 On Wed, Feb 17, 2010 at 8:11 PM, Kostik Belousov wrot= e: > On Wed, Feb 17, 2010 at 07:51:06PM +0100, Fernando Apestegu?a wrote: >> Hi, >> >> I have a small patch (against 8.0-RELEASE-p2) that _should_ implement >> the /proc/pid/environ file >> under linprocfs. >> However, it seems it does not work properly but I don't know what I'm >> doing wrong. >> Is this list the place to ask for help? I tried in the forums[1] but >> got no answer. > Putting aside any "does not work" questions, please see comment below. Sorry I didn't explain this. If I have a process forked from bash shell in which I have exported VAR=3DXXXX the /compat/linux/proc/pid/environ for the child proces= s does not show the VAR variable. >> >> Don't we have a 'kernel newbies'-like list? >> >> Thanks in advance. >> >> [1] http://forums.freebsd.org/showthread.php?t=3D11329 >> >> --- sys/compat/linprocfs/linprocfs.c.orig =A0 =A0 2009-10-25 02:10:29.00= 0000000 +0100 >> +++ sys/compat/linprocfs/linprocfs.c =A02010-02-16 19:38:36.000000000 +0= 100 >> @@ -939,8 +939,38 @@ >> =A0static int >> =A0linprocfs_doprocenviron(PFS_FILL_ARGS) >> =A0{ >> + =A0 =A0 int i, error; >> + =A0 =A0 struct ps_strings pss; >> + =A0 =A0 char **ps_envstr; >> >> - =A0 =A0 sbuf_printf(sb, "doprocenviron\n%c", '\0'); >> + =A0 =A0 PROC_LOCK(p); >> + =A0 =A0 if (p_cansee(td, p) !=3D 0) >> + =A0 =A0 =A0 =A0 =A0 =A0 return (0); >> + =A0 =A0 PROC_UNLOCK(p); >> + >> + =A0 =A0 error =3D copyin((void *)p->p_sysent->sv_psstrings, &pss, >> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 sizeof(pss)); >> + =A0 =A0 if (error) >> + =A0 =A0 =A0 =A0 =A0 =A0 return (error); >> + >> + =A0 =A0 ps_envstr =3D malloc(pss.ps_nenvstr * sizeof(char *), >> + =A0 =A0 =A0 =A0 M_TEMP, M_WAITOK); > This is essentially "panic me" code. =A0ps_nenvstr is user-controlled, > and allows to specify arbitrary integers. > > Even ignoring exhaustion of the kernel map, it can cause allocation of > big amount of physical memory. Note that execve(2) implementation uses > swappable memory to store arguments and environment strings passed from > vm spaces. Thanks for the comment. If I want to check ps_envstr, which threshold would= be reasonable? PAGE_SIZE maybe? Thanks again. > >> + >> + =A0 =A0 error =3D copyin((void *)pss.ps_envstr, ps_envstr, >> + =A0 =A0 =A0 =A0 pss.ps_nenvstr * sizeof(char *)); >> + >> + =A0 =A0 if (error) { >> + =A0 =A0 =A0 =A0 =A0 =A0 free(ps_envstr, M_TEMP); >> + =A0 =A0 =A0 =A0 =A0 =A0 return (error); >> + =A0 =A0 } >> + >> + =A0 =A0 /* NULL separated list of variable=3Dvalue pairs */ >> + >> + =A0 =A0 for (i =3D 0; i < pss.ps_nenvstr; i++) { >> + =A0 =A0 =A0 =A0 =A0 =A0 sbuf_copyin(sb, ps_envstr[i], 0); >> + =A0 =A0 } >> + >> + =A0 =A0 free(ps_envstr, M_TEMP); >> =A0 =A0 =A0 return (0); >> =A0} >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.or= g" > From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 18 18:38:05 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E8B551065676; Thu, 18 Feb 2010 18:38:04 +0000 (UTC) (envelope-from nox@jelal.kn-bremen.de) Received: from smtp.kn-bremen.de (gelbbaer.kn-bremen.de [78.46.108.116]) by mx1.freebsd.org (Postfix) with ESMTP id 75C6C8FC0C; Thu, 18 Feb 2010 18:38:04 +0000 (UTC) Received: by smtp.kn-bremen.de (Postfix, from userid 10) id 5F0121E00775; Thu, 18 Feb 2010 19:38:03 +0100 (CET) Received: from triton8.kn-bremen.de (noident@localhost [127.0.0.1]) by triton8.kn-bremen.de (8.14.3/8.14.3) with ESMTP id o1IIYxkZ065849; Thu, 18 Feb 2010 19:34:59 +0100 (CET) (envelope-from nox@triton8.kn-bremen.de) Received: (from nox@localhost) by triton8.kn-bremen.de (8.14.3/8.14.3/Submit) id o1IIYxjU065848; Thu, 18 Feb 2010 19:34:59 +0100 (CET) (envelope-from nox) From: Juergen Lock Date: Thu, 18 Feb 2010 19:34:59 +0100 To: Tim Kientzle Message-ID: <20100218183459.GA65508@triton8.kn-bremen.de> References: <20100217215940.GA19713@triton8.kn-bremen.de> <4B7CE066.4030403@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B7CE066.4030403@freebsd.org> User-Agent: Mutt/1.5.20 (2009-06-14) X-Mailman-Approved-At: Thu, 18 Feb 2010 19:01:55 +0000 Cc: freebsd-hackers@freebsd.org, Juergen Lock Subject: Re: "tar tfv /dev/cd0" speedup patch X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 18:38:05 -0000 On Wed, Feb 17, 2010 at 10:38:30PM -0800, Tim Kientzle wrote: > Juergen Lock wrote: > > > > ... since bsdtar/libarchive know iso9660 I just did the command in the > > Subject. It worked, but it was sloow... :( Apparently it read all of > > the disc without seeking. The following patch fixes this, is something > > like this desired? If yes I could look how to do the same for Linux, > > Juergen, > > This is great! If you can figure out how to get this > right, I would really appreciate it. If you have a > tape drive handy, definitely test with that. My first > attempts here actually broke reading from tape drives, > which is why the current code is so conservative. > Hmm I can't test on a tape atm but if I look at the kernel, http://fxr.watson.org/fxr/ident?i=DIOCGMEDIASIZE DIOCGMEDIASIZE is only handled for geom, xen block devices, old CD drives and pc98 floppies, and I'm pretty sure tapes don't use geom. :) > Minor style comments: > > else if (S_ISCHR(st.st_mode) && > > !ioctl(fd, DIOCGMEDIASIZE, &mediasize) && mediasize) { > > Please be explicit: S_ISCHR() && ioctl() == 0 && mediasize > 0 > > > archive_read_extract_set_skip_file(a, st.st_dev, st.st_ino); > > extract_skip_file isn't needed here; we don't read the > contents of device nodes. > > Let me know as soon as you have something you're confident of. Ok here is a new version of the patch with these things fixed and the Linux case added: (Linux case not tested yet, and yes I did this on stable/8.) Index: src/lib/libarchive/archive_read_open_filename.c =================================================================== RCS file: /home/scvs/src/lib/libarchive/archive_read_open_filename.c,v retrieving revision 1.25.2.1 diff -u -p -r1.25.2.1 archive_read_open_filename.c --- src/lib/libarchive/archive_read_open_filename.c 3 Aug 2009 08:13:06 -0000 1.25.2.1 +++ src/lib/libarchive/archive_read_open_filename.c 18 Feb 2010 18:14:16 -0000 @@ -44,6 +44,10 @@ __FBSDID("$FreeBSD: src/lib/libarchive/a #ifdef HAVE_UNISTD_H #include #endif +#ifdef __FreeBSD__ +#include +#include +#endif #include "archive.h" @@ -83,6 +87,9 @@ archive_read_open_filename(struct archiv struct read_file_data *mine; void *b; int fd; +#ifdef __FreeBSD__ + off_t mediasize = 0; +#endif archive_clear_error(a); if (filename == NULL || filename[0] == '\0') { @@ -143,6 +150,27 @@ archive_read_open_filename(struct archiv */ mine->can_skip = 1; } +#ifdef __FreeBSD__ + /* + * on FreeBSD if a device supports the DIOCGMEDIASIZE ioctl + * it is a disk-like device and should be seekable. + */ + else if (S_ISCHR(st.st_mode) && + ioctl(fd, DIOCGMEDIASIZE, &mediasize) == 0 && mediasize > 0) { + mine->can_skip = 1; + } +#endif +#ifdef __linux__ + /* + * on Linux just check whether its a block device and that + * lseek works. (Tapes are character devices there.) + */ + else if (S_ISBLK(st.st_mode) && + lseek(fd, 0, SEEK_CUR) == 0 && lseek(fd, 0, SEEK_SET) == 0 && + lseek(fd, 0, SEEK_END) > 0 && lseek(fd, 0, SEEK_SET) == 0) { + mine->can_skip = 1; + } +#endif return (archive_read_open2(a, mine, NULL, file_read, file_skip, file_close)); } From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 18 19:14:24 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7DDA21065672 for ; Thu, 18 Feb 2010 19:14:24 +0000 (UTC) (envelope-from joerg@britannica.bec.de) Received: from www.sonnenberger.org (www.sonnenberger.org [92.79.50.50]) by mx1.freebsd.org (Postfix) with ESMTP id 3EFF68FC23 for ; Thu, 18 Feb 2010 19:14:24 +0000 (UTC) Received: from britannica.bec.de (www.sonnenberger.org [192.168.1.10]) by www.sonnenberger.org (Postfix) with ESMTP id CD2E3667BE for ; Thu, 18 Feb 2010 20:14:20 +0100 (CET) Received: by britannica.bec.de (Postfix, from userid 1000) id 9A4DD15C64; Thu, 18 Feb 2010 20:13:47 +0100 (CET) Date: Thu, 18 Feb 2010 20:13:47 +0100 From: Joerg Sonnenberger To: freebsd-hackers@freebsd.org Message-ID: <20100218191347.GA4040@britannica.bec.de> Mail-Followup-To: freebsd-hackers@freebsd.org References: <20100217215940.GA19713@triton8.kn-bremen.de> <4B7CE066.4030403@freebsd.org> <20100218183459.GA65508@triton8.kn-bremen.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100218183459.GA65508@triton8.kn-bremen.de> User-Agent: Mutt/1.5.20 (2009-06-14) Subject: Re: "tar tfv /dev/cd0" speedup patch X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 19:14:24 -0000 On Thu, Feb 18, 2010 at 07:34:59PM +0100, Juergen Lock wrote: > Ok here is a new version of the patch with these things fixed and the > Linux case added: (Linux case not tested yet, and yes I did this on > stable/8.) Why the check at all? Shouldn't devices that don't allow seek fail that? E.g. for devices, just try to seek ahead and fallback to normal reading? Joerg From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 18 19:23:26 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 42D271065672 for ; Thu, 18 Feb 2010 19:23:26 +0000 (UTC) (envelope-from eitanadlerlist@gmail.com) Received: from qw-out-2122.google.com (qw-out-2122.google.com [74.125.92.24]) by mx1.freebsd.org (Postfix) with ESMTP id 043F78FC19 for ; Thu, 18 Feb 2010 19:23:25 +0000 (UTC) Received: by qw-out-2122.google.com with SMTP id 8so1409659qwh.7 for ; Thu, 18 Feb 2010 11:23:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:from:date:message-id :subject:to:content-type; bh=rPoVRPq9aB0CNZXBEy9qzQrLNnDSWLgzfFzeaqqixdA=; b=vpHp8AbYxGegV+VbuBcTrymPtZWAu7oZGqojWP3nZ2VcToXES8+F3MTmzAXop/02uw Cywx4i2gVjpxJVXmfl20vZ51X92j0fgQRVIfcr3H3g/B5BrELAJHodCGwba//9CA/G5+ xF4gFn6vBpJoDwlJ+yFjQHbUvkiWa2CnXys3U= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:from:date:message-id:subject:to:content-type; b=Afo9Fmm3zUXoqPGPY5ycgKY5zouFg8hElY8vMofRyDBgAPT9qb303ZN45p+n8mO8il EYZ9xD+g6m2eKauS9eiijzvJ3+MpdMa2AgtuiFX8YlU38JIG7cvWtxspc6KpBA8g4a4b NJIflEHsKKtjBK7/XTok7lTqW34mv6iT0Ba0U= MIME-Version: 1.0 Received: by 10.239.187.139 with SMTP id l11mr1136436hbh.96.1266521004477; Thu, 18 Feb 2010 11:23:24 -0800 (PST) From: Eitan Adler Date: Thu, 18 Feb 2010 21:22:51 +0200 Message-ID: To: hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: Subject: junior tasks wiki page - please add stuff X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 19:23:26 -0000 I made a wiki page[1] to list small odd-jobs that beginners to work on. It is intended to answer the question "I'm just starting out with coding, what can I help with?" The tasks put on the wiki page should be things that require little programming skill. It would become a lot more useful if people added things to do. [1] http://wiki.freebsd.org/JuniorJobs From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 18 19:24:23 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D02C81065695 for ; Thu, 18 Feb 2010 19:24:23 +0000 (UTC) (envelope-from julian@elischer.org) Received: from out-0.mx.aerioconnect.net (out-0-35.mx.aerioconnect.net [216.240.47.95]) by mx1.freebsd.org (Postfix) with ESMTP id 31D738FC24 for ; Thu, 18 Feb 2010 19:24:23 +0000 (UTC) Received: from idiom.com (postfix@mx0.idiom.com [216.240.32.160]) by out-0.mx.aerioconnect.net (8.13.8/8.13.8) with ESMTP id o1IJ4Oqr015966; Thu, 18 Feb 2010 11:04:24 -0800 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (h-67-100-89-137.snfccasy.static.covad.net [67.100.89.137]) by idiom.com (Postfix) with ESMTP id C84A62D6013; Thu, 18 Feb 2010 11:04:23 -0800 (PST) Message-ID: <4B7D8F6B.3040504@elischer.org> Date: Thu, 18 Feb 2010 11:05:15 -0800 From: Julian Elischer User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: Anderson Eduardo References: In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 216.240.47.51 Cc: freebsd-hackers@freebsd.org Subject: Re: Debug registers on =?utf-8?q?FreeBSD=E2=80=8F?= X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 19:24:23 -0000 Anderson Eduardo wrote: > I want some papers, documentations or articles about Debug Registers on FreeBSD, I googled and didn't find any references about it. I'm using i386 AMD. > > Can someone help me? Usually one does not use the debug registers directlym but instead uses one of the drivers that controls them. for example the pmcstat program gives you access to the process accounting registers and control 'virtual' counters per process. gdb (via the kernel) and other debug tools use various registers, so you want to make sure you are not interfering with them either. From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 18 20:21:51 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D189D1065694 for ; Thu, 18 Feb 2010 20:21:51 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 462828FC1C for ; Thu, 18 Feb 2010 20:21:50 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id o1IKLjoR041283 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 18 Feb 2010 22:21:45 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.3/8.14.3) with ESMTP id o1IKLjeF042322; Thu, 18 Feb 2010 22:21:45 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.3/8.14.3/Submit) id o1IKLhT3042321; Thu, 18 Feb 2010 22:21:43 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 18 Feb 2010 22:21:43 +0200 From: Kostik Belousov To: Fernando Apestegu?a Message-ID: <20100218202143.GT50403@deviant.kiev.zoral.com.ua> References: <1bd550a01002171051n7117895avb5cf57fb7fbb9388@mail.gmail.com> <20100217191156.GP50403@deviant.kiev.zoral.com.ua> <1bd550a01002180948r48300f8em15efca184d99ed98@mail.gmail.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="LbN0412894TjpI52" Content-Disposition: inline In-Reply-To: <1bd550a01002180948r48300f8em15efca184d99ed98@mail.gmail.com> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: FreeBSD Hackers Subject: Re: linprocfs proc/pid/environ patch & list question X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 20:21:51 -0000 --LbN0412894TjpI52 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Feb 18, 2010 at 06:48:35PM +0100, Fernando Apestegu?a wrote: > On Wed, Feb 17, 2010 at 8:11 PM, Kostik Belousov wr= ote: > > On Wed, Feb 17, 2010 at 07:51:06PM +0100, Fernando Apestegu?a wrote: > >> Hi, > >> > >> I have a small patch (against 8.0-RELEASE-p2) that _should_ implement > >> the /proc/pid/environ file > >> under linprocfs. > >> However, it seems it does not work properly but I don't know what I'm > >> doing wrong. > >> Is this list the place to ask for help? I tried in the forums[1] but > >> got no answer. > > Putting aside any "does not work" questions, please see comment below. >=20 > Sorry I didn't explain this. If I have a process forked from bash > shell in which I have > exported VAR=3DXXXX the /compat/linux/proc/pid/environ for the child proc= ess > does not show the VAR variable. Copyin copies the data from the address space of the current process. You are interested in the content of address space of different process. Look at the proc_rwmem(). >=20 > >> > >> Don't we have a 'kernel newbies'-like list? > >> > >> Thanks in advance. > >> > >> [1] http://forums.freebsd.org/showthread.php?t=3D11329 > >> > >> --- sys/compat/linprocfs/linprocfs.c.orig =9A =9A 2009-10-25 02:10:29.= 000000000 +0100 > >> +++ sys/compat/linprocfs/linprocfs.c =9A2010-02-16 19:38:36.000000000 = +0100 > >> @@ -939,8 +939,38 @@ > >> =9Astatic int > >> =9Alinprocfs_doprocenviron(PFS_FILL_ARGS) > >> =9A{ > >> + =9A =9A int i, error; > >> + =9A =9A struct ps_strings pss; > >> + =9A =9A char **ps_envstr; > >> > >> - =9A =9A sbuf_printf(sb, "doprocenviron\n%c", '\0'); > >> + =9A =9A PROC_LOCK(p); > >> + =9A =9A if (p_cansee(td, p) !=3D 0) > >> + =9A =9A =9A =9A =9A =9A return (0); > >> + =9A =9A PROC_UNLOCK(p); > >> + > >> + =9A =9A error =3D copyin((void *)p->p_sysent->sv_psstrings, &pss, > >> + =9A =9A =9A =9A =9A =9A =9A =9A =9A =9A =9A =9A =9A =9A =9A =9A =9A = =9A =9A =9A sizeof(pss)); > >> + =9A =9A if (error) > >> + =9A =9A =9A =9A =9A =9A return (error); > >> + > >> + =9A =9A ps_envstr =3D malloc(pss.ps_nenvstr * sizeof(char *), > >> + =9A =9A =9A =9A M_TEMP, M_WAITOK); > > This is essentially "panic me" code. =9Aps_nenvstr is user-controlled, > > and allows to specify arbitrary integers. > > > > Even ignoring exhaustion of the kernel map, it can cause allocation of > > big amount of physical memory. Note that execve(2) implementation uses > > swappable memory to store arguments and environment strings passed from > > vm spaces. >=20 > Thanks for the comment. If I want to check ps_envstr, which threshold wou= ld be > reasonable? PAGE_SIZE maybe? >=20 > Thanks again. >=20 > > > >> + > >> + =9A =9A error =3D copyin((void *)pss.ps_envstr, ps_envstr, > >> + =9A =9A =9A =9A pss.ps_nenvstr * sizeof(char *)); > >> + > >> + =9A =9A if (error) { > >> + =9A =9A =9A =9A =9A =9A free(ps_envstr, M_TEMP); > >> + =9A =9A =9A =9A =9A =9A return (error); > >> + =9A =9A } > >> + > >> + =9A =9A /* NULL separated list of variable=3Dvalue pairs */ > >> + > >> + =9A =9A for (i =3D 0; i < pss.ps_nenvstr; i++) { > >> + =9A =9A =9A =9A =9A =9A sbuf_copyin(sb, ps_envstr[i], 0); > >> + =9A =9A } > >> + > >> + =9A =9A free(ps_envstr, M_TEMP); > >> =9A =9A =9A return (0); > >> =9A} > >> _______________________________________________ > >> freebsd-hackers@freebsd.org mailing list > >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.= org" > > --LbN0412894TjpI52 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (FreeBSD) iEYEARECAAYFAkt9oVcACgkQC3+MBN1Mb4jl4ACeNQtZwvxExzWHtZFca3rKekf5 sy4AoK6b7vJU0WoX1ns60AmRgvLsM4P0 =HlgO -----END PGP SIGNATURE----- --LbN0412894TjpI52-- From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 18 20:43:18 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A70201065676; Thu, 18 Feb 2010 20:43:18 +0000 (UTC) (envelope-from nox@jelal.kn-bremen.de) Received: from smtp.kn-bremen.de (gelbbaer.kn-bremen.de [78.46.108.116]) by mx1.freebsd.org (Postfix) with ESMTP id 648188FC15; Thu, 18 Feb 2010 20:43:18 +0000 (UTC) Received: by smtp.kn-bremen.de (Postfix, from userid 10) id 8F63F1E00772; Thu, 18 Feb 2010 21:43:17 +0100 (CET) Received: from triton8.kn-bremen.de (noident@localhost [127.0.0.1]) by triton8.kn-bremen.de (8.14.3/8.14.3) with ESMTP id o1IKe0DH004616; Thu, 18 Feb 2010 21:40:00 +0100 (CET) (envelope-from nox@triton8.kn-bremen.de) Received: (from nox@localhost) by triton8.kn-bremen.de (8.14.3/8.14.3/Submit) id o1IKe0NZ004615; Thu, 18 Feb 2010 21:40:00 +0100 (CET) (envelope-from nox) From: Juergen Lock Date: Thu, 18 Feb 2010 21:39:59 +0100 To: Juergen Lock Message-ID: <20100218203959.GA4457@triton8.kn-bremen.de> References: <20100217215940.GA19713@triton8.kn-bremen.de> <4B7CE066.4030403@freebsd.org> <20100218183459.GA65508@triton8.kn-bremen.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100218183459.GA65508@triton8.kn-bremen.de> User-Agent: Mutt/1.5.20 (2009-06-14) X-Mailman-Approved-At: Thu, 18 Feb 2010 20:49:32 +0000 Cc: freebsd-hackers@freebsd.org, Tim Kientzle Subject: Re: "tar tfv /dev/cd0" speedup patch X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 20:43:18 -0000 On Thu, Feb 18, 2010 at 07:34:59PM +0100, Juergen Lock wrote: > On Wed, Feb 17, 2010 at 10:38:30PM -0800, Tim Kientzle wrote: > > Juergen Lock wrote: > > > > > > ... since bsdtar/libarchive know iso9660 I just did the command in the > > > Subject. It worked, but it was sloow... :( Apparently it read all of > > > the disc without seeking. The following patch fixes this, is something > > > like this desired? If yes I could look how to do the same for Linux, > > > > Juergen, > > > > This is great! If you can figure out how to get this > > right, I would really appreciate it. If you have a > > tape drive handy, definitely test with that. My first > > attempts here actually broke reading from tape drives, > > which is why the current code is so conservative. > > > Hmm I can't test on a tape atm but if I look at the kernel, > http://fxr.watson.org/fxr/ident?i=DIOCGMEDIASIZE > DIOCGMEDIASIZE is only handled for geom, xen block devices, old CD drives > and pc98 floppies, and I'm pretty sure tapes don't use geom. :) > > > Minor style comments: > > > else if (S_ISCHR(st.st_mode) && > > > !ioctl(fd, DIOCGMEDIASIZE, &mediasize) && mediasize) { > > > > Please be explicit: S_ISCHR() && ioctl() == 0 && mediasize > 0 > > > > > archive_read_extract_set_skip_file(a, st.st_dev, st.st_ino); > > > > extract_skip_file isn't needed here; we don't read the > > contents of device nodes. > > > > Let me know as soon as you have something you're confident of. > > Ok here is a new version of the patch with these things fixed and the > Linux case added: (Linux case not tested yet, and yes I did this on > stable/8.) Ok just tested the patch on libarchive-2.8.0.tar.gz on debian sid on /dev/sr0 (optical drive) and it worked. (Had to use the tarball from code.google.com since sid's own libarchive version still was at 2.6.something where the patch didn't apply...) Cheers, Juergen From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 18 22:13:05 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2519D10656AC; Thu, 18 Feb 2010 22:13:05 +0000 (UTC) (envelope-from yanefbsd@gmail.com) Received: from mail-pz0-f202.google.com (mail-pz0-f202.google.com [209.85.222.202]) by mx1.freebsd.org (Postfix) with ESMTP id EA3288FC32; Thu, 18 Feb 2010 22:13:04 +0000 (UTC) Received: by pzk40 with SMTP id 40so6361870pzk.7 for ; Thu, 18 Feb 2010 14:13:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=PBDiEXx2/TrZBp+MS8J9lSWFt2D9SDYPGQxD0U5QqWE=; b=DZ1Q+IjxsD9mlbuUzA2V/Cbmg4tNrLuVscRWYngzgL3LHIyk+f18fPQSghMP44PQAS NR2VeNfafgo7IV/xcuWTRGMmCjnVmdsmCqlqxONAN0Qbe1U7+MEjUmrJEOAwNbwSshk7 4BierMUUwYaIedyXXAyMWsU8AsH9CFHOgF1tU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=WBW5HmMofJZEsBJUwloMgopti54SztXhFxNA+lzxnH73ubOiiHtniBWxeZzNqd7ky9 0khgChObYuDIETSMW/uWbUBluFXzPUNEWNJUMow04KiXGl4BdnXVS8xQBqX4fSxQwbv6 I22pSuJwB6yIPvpDZqSq3mF3Gds7xhFMawBnQ= MIME-Version: 1.0 Received: by 10.142.249.16 with SMTP id w16mr2182638wfh.346.1266531184381; Thu, 18 Feb 2010 14:13:04 -0800 (PST) In-Reply-To: References: Date: Thu, 18 Feb 2010 14:13:04 -0800 Message-ID: <7d6fde3d1002181413p214389d3nd73798e597b9e26f@mail.gmail.com> From: Garrett Cooper To: Eitan Adler Content-Type: text/plain; charset=ISO-8859-1 Cc: hackers@freebsd.org, linimon@freebsd.org Subject: Re: junior tasks wiki page - please add stuff X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 22:13:05 -0000 On Thu, Feb 18, 2010 at 11:22 AM, Eitan Adler wrote: > I made a wiki page[1] to list small odd-jobs that beginners to work > on. It is intended to answer the question "I'm just starting out with > coding, what can I help with?" The tasks put on the wiki page should > be things that require little programming skill. > > It would become a lot more useful if people added things to do. > > [1] http://wiki.freebsd.org/JuniorJobs 1. Who should the wiki content `regulators' be (i.e. project-only folks, outside folks, etc)? Anyone can sign up for an account and because the information reflects on the project, it would be best (IMO) if `proposals' // `tasks' were vetted by someone in the project with sufficient say to prioritize the item. 2. Would it be a good idea to also provide a list of tasks that are non-coding related (or at least have a smaller scope) so that others could assist with in the project to help free up resources? For instance, some folks (like linimon@) really need a junior person (I've debated whether or not I should do it, but I don't want to assume too much responsibility) to do semi-menial tasks like bug triage, feedback, basic administration items, etc to free up Mark's time. That way it would allow someone with less background and exposure to the project to become more integrated with the project, outside of doing an SoC project for instance. Just curious... Thanks, -Garrett From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 19 01:05:06 2010 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 028331065676; Fri, 19 Feb 2010 01:05:06 +0000 (UTC) (envelope-from sobomax@FreeBSD.org) Received: from sippysoft.com (gk1.360sip.com [72.236.70.240]) by mx1.freebsd.org (Postfix) with ESMTP id 5ABCA8FC0C; Fri, 19 Feb 2010 01:05:05 +0000 (UTC) Received: from [192.168.1.38] ([70.71.167.197]) (authenticated bits=0) by sippysoft.com (8.14.3/8.14.3) with ESMTP id o1J151pb046553 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 18 Feb 2010 17:05:02 -0800 (PST) (envelope-from sobomax@FreeBSD.org) Message-ID: <4B7DE3CC.7040704@FreeBSD.org> Date: Thu, 18 Feb 2010 17:05:16 -0800 From: Maxim Sobolev Organization: Sippy Software, Inc. User-Agent: Thunderbird 2.0.0.23 (Windows/20090812) MIME-Version: 1.0 To: Sergey Babkin References: <4B79297D.9080403@FreeBSD.org> <4B79205B.619A0A1A@verizon.net> <4B7ADFC6.7020202@FreeBSD.org> In-Reply-To: <4B7ADFC6.7020202@FreeBSD.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: Alfred Perlstein , freebsd-net@FreeBSD.org, "David G. Lawrence" , Jack Vogel , FreeBSD Hackers Subject: Re: Sudden mbuf demand increase and shortage under the load (igb issue?) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2010 01:05:06 -0000 Folks, Indeed, it looks like igb(4) issue. Replacing the card with the desktop-grade em(4)-supported card has fixed the problem for us. The system has been happily pushing 110mbps worth of RTP traffic and 2000 concurrent calls without any problems for two days now. em0@pci0:7:0:0: class=0x020000 card=0xa01f8086 chip=0x10d38086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' class = network subclass = ethernet em0: port 0xec00-0xec1f mem 0xfbee0000-0xfbefffff,0xfbe00000-0xfbe7ffff,0xfbedc000-0xfbedffff irq 24 at device 0.0 on pci7 em0: Using MSIX interrupts em0: [ITHREAD] em0: [ITHREAD] em0: [ITHREAD] em0: Ethernet address: 00:1b:21:50:02:49 I really think that this has to be addressed before 7.3 release is out. FreeBSD used to be famous for its excellent network performance and it's shame to see that deteriorating due to sub-standard quality drivers. Especially when there is a multi-billion vendor supporting the driver in question. No finger pointing, but it really looks like either somebody is not doing his job or the said vendor doesn't care so much about supporting FreeBSD. I am pretty sure the vendor in question has access to numerous load-testing tools, that should have catched this issue. This is the second time during the past 6 months I have issue with the quality of the Intel-based drivers - the first one is filed as kern/140326, which has stalled apparently despite me providing all necessary debug information. Regards, -- Maksym Sobolyev Sippy Software, Inc. Internet Telephony (VoIP) Experts T/F: +1-646-651-1110 Web: http://www.sippysoft.com MSN: sales@sippysoft.com Skype: SippySoft From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 19 03:43:37 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 886FA106566C; Fri, 19 Feb 2010 03:43:37 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id E0E8F8FC0A; Fri, 19 Feb 2010 03:43:36 +0000 (UTC) Received: by vws14 with SMTP id 14so1034266vws.13 for ; Thu, 18 Feb 2010 19:43:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=nhp52rbB1zv6YcnoYDDpyul2eQw7N/Erc47mTnTIPTc=; b=uo9cJIR6hwUMRnC5KpCas3m0Cuvl2JKLuREF87gN2Rw+iuXkM7efx1jtvczTOAIzTX 0oZH0VSoS9BHvLhdq5jlFBBIR8fjskxAsvksQ8LT+NXxYOMlVsbBujYauAu56Mc3QC+g qvSgctcT/v7FvkRpgr/zjaie420KL4Lc3klTE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=geHJnWcQGOk3hPXobx7Sj0l4l6C/k1+ZO7HbsgCgkCR2Hy0vVAYfqNy6Hk9vdd6tSm yB+7axgQJ8eVnEGYD6u6bIj1d5hM/uRu4xT62ykLD8obR2zBQL9LEoc2CHAcfNlfpqaV BdlqseZ878HS7XTc5hIi1IssathshIsbsfbqo= Received: by 10.220.127.4 with SMTP id e4mr3740751vcs.79.1266551016030; Thu, 18 Feb 2010 19:43:36 -0800 (PST) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id 30sm33174334vws.1.2010.02.18.19.43.33 (version=TLSv1/SSLv3 cipher=RC4-MD5); Thu, 18 Feb 2010 19:43:34 -0800 (PST) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Thu, 18 Feb 2010 19:42:55 -0800 From: Pyun YongHyeon Date: Thu, 18 Feb 2010 19:42:55 -0800 To: Maxim Sobolev Message-ID: <20100219034255.GG11675@michelle.cdnetworks.com> References: <4B79297D.9080403@FreeBSD.org> <4B79205B.619A0A1A@verizon.net> <4B7ADFC6.7020202@FreeBSD.org> <4B7DE3CC.7040704@FreeBSD.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="Md/poaVZ8hnGTzuv" Content-Disposition: inline In-Reply-To: <4B7DE3CC.7040704@FreeBSD.org> User-Agent: Mutt/1.4.2.3i Cc: Sergey Babkin , freebsd-net@freebsd.org, Alfred Perlstein , Jack Vogel , FreeBSD Hackers , "David G. Lawrence" Subject: Re: Sudden mbuf demand increase and shortage under the load (igb issue?) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2010 03:43:37 -0000 --Md/poaVZ8hnGTzuv Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Thu, Feb 18, 2010 at 05:05:16PM -0800, Maxim Sobolev wrote: > Folks, > > Indeed, it looks like igb(4) issue. Replacing the card with the > desktop-grade em(4)-supported card has fixed the problem for us. The > system has been happily pushing 110mbps worth of RTP traffic and 2000 > concurrent calls without any problems for two days now. > > em0@pci0:7:0:0: class=0x020000 card=0xa01f8086 chip=0x10d38086 rev=0x00 > hdr=0x00 > vendor = 'Intel Corporation' > class = network > subclass = ethernet > > em0: port 0xec00-0xec1f mem > 0xfbee0000-0xfbefffff,0xfbe00000-0xfbe7ffff,0xfbedc000-0xfbedffff irq 24 > at device 0.0 on pci7 > em0: Using MSIX interrupts > em0: [ITHREAD] > em0: [ITHREAD] > em0: [ITHREAD] > em0: Ethernet address: 00:1b:21:50:02:49 > > I really think that this has to be addressed before 7.3 release is out. > FreeBSD used to be famous for its excellent network performance and it's > shame to see that deteriorating due to sub-standard quality drivers. > Especially when there is a multi-billion vendor supporting the driver in > question. No finger pointing, but it really looks like either somebody > is not doing his job or the said vendor doesn't care so much about > supporting FreeBSD. I am pretty sure the vendor in question has access > to numerous load-testing tools, that should have catched this issue. > > This is the second time during the past 6 months I have issue with the > quality of the Intel-based drivers - the first one is filed as > kern/140326, which has stalled apparently despite me providing all > necessary debug information. > I can reproduce this bug on my box and I guess the root cause comes from PBA(Packet Buffer Allocation) configuration. Some controllers seems to require more TX buffer size to use 9000 MTU. The datasheet is not clear which controller has how much amount of Packet Buffer storage. This parameter seems to affect performance a lot because increasing TX buffer size results in decreasing RX buffer size. The attached patch seems to fix the issue for me but Jack may know better the hardware details as publicly available datasheet seems to be useless here. --Md/poaVZ8hnGTzuv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="em.ich10.jumbo.diff" Index: sys/dev/e1000/if_em.c =================================================================== --- sys/dev/e1000/if_em.c (revision 204011) +++ sys/dev/e1000/if_em.c (working copy) @@ -1384,7 +1384,12 @@ case e1000_ich9lan: case e1000_ich10lan: case e1000_pchlan: +#if 1 + /* Make 9000 MTU work. */ + pba = E1000_PBA_18K; +#else pba = E1000_PBA_10K; +#endif break; case e1000_ich8lan: pba = E1000_PBA_8K; --Md/poaVZ8hnGTzuv-- From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 19 05:02:33 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 29387106566C for ; Fri, 19 Feb 2010 05:02:33 +0000 (UTC) (envelope-from kientzle@freebsd.org) Received: from monday.kientzle.com (kientzle.com [66.166.149.50]) by mx1.freebsd.org (Postfix) with ESMTP id F29118FC1B for ; Fri, 19 Feb 2010 05:02:32 +0000 (UTC) Received: (from root@localhost) by monday.kientzle.com (8.14.3/8.14.3) id o1J52fgA083184 for freebsd-hackers@freebsd.org; Fri, 19 Feb 2010 05:02:41 GMT (envelope-from kientzle@freebsd.org) Received: from dark.x.kientzle.com (fw2.kientzle.com [10.123.1.2]) by kientzle.com with SMTP id q4mquszeibuiqwinuxqgszkt8e; for freebsd-hackers@freebsd.org; Fri, 19 Feb 2010 05:02:41 +0000 (UTC) (envelope-from kientzle@freebsd.org) Message-ID: <4B7E1BA5.7050209@freebsd.org> Date: Thu, 18 Feb 2010 21:03:33 -0800 From: Tim Kientzle User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.8.1.21) Gecko/20090601 SeaMonkey/1.1.16 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org References: <20100217215940.GA19713@triton8.kn-bremen.de> <4B7CE066.4030403@freebsd.org> <20100218183459.GA65508@triton8.kn-bremen.de> <20100218191347.GA4040@britannica.bec.de> In-Reply-To: <20100218191347.GA4040@britannica.bec.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: "tar tfv /dev/cd0" speedup patch X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2010 05:02:33 -0000 Joerg Sonnenberger wrote: > On Thu, Feb 18, 2010 at 07:34:59PM +0100, Juergen Lock wrote: >> Ok here is a new version of the patch with these things fixed and the >> Linux case added: (Linux case not tested yet, and yes I did this on >> stable/8.) > > Why the check at all? Shouldn't devices that don't allow seek fail that? > E.g. for devices, just try to seek ahead and fallback to normal reading? That was the initial implementation in libarchive, but I had a number of reports of that not working for tape drives. I recently dug out and connected an old DDS drive I had in the closet, so I should probably try again and see if I misunderstood something along the way. Tim From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 19 05:22:32 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CD7D3106566C for ; Fri, 19 Feb 2010 05:22:32 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-qy0-f189.google.com (mail-qy0-f189.google.com [209.85.221.189]) by mx1.freebsd.org (Postfix) with ESMTP id 647528FC0C for ; Fri, 19 Feb 2010 05:22:32 +0000 (UTC) Received: by qyk27 with SMTP id 27so6279717qyk.3 for ; Thu, 18 Feb 2010 21:22:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:date:from:to:cc :subject:in-reply-to:message-id:references:user-agent :x-openpgp-key-id:x-openpgp-key-fingerprint:mime-version :content-type; bh=CFB0fNHET8bWyO4XkzyBAgKBJ673OcGKkVI/71KxmtI=; b=By91IVcXUgEABcUhLXbSjya9Pjw0yi0qtaTmjhcmvEB6jicPvnO7x1Rsq/aBfCOElf ZFgYAhWVH35uCI6yzwa/xKpGQo3I8kbznRyaWTnwkEMN4WqjVYDhnnBws6wxIdR5oG4m DWg8CRiaLJAf8Nl9IBerAE5QVaoKTeThNot/I= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:date:from:to:cc:subject:in-reply-to:message-id:references :user-agent:x-openpgp-key-id:x-openpgp-key-fingerprint:mime-version :content-type; b=f56dG9Ts4VvRtfLD7IXChdtI/J6Vte7gST27I99cDpOqANtbfLRuT0CtXfKYYrB7df YnLbI0tjiGcxgKsqKU8+3FWhzwS40LXrEHUgErpKE4z8LhG1nGz3QuKc1nLdx9xR17hA nAEXzcy+XGdPUYOrrfkw+cD9B3pLrYBpvensM= Received: by 10.224.66.163 with SMTP id n35mr5848444qai.30.1266556951563; Thu, 18 Feb 2010 21:22:31 -0800 (PST) Received: from ppp-22.62.dialinfree.com (ppp-22.62.dialinfree.com [209.172.22.62]) by mx.google.com with ESMTPS id 21sm7227065qyk.12.2010.02.18.21.22.25 (version=TLSv1/SSLv3 cipher=RC4-MD5); Thu, 18 Feb 2010 21:22:28 -0800 (PST) Sender: "J. Hellenthal" Date: Fri, 19 Feb 2010 00:22:07 -0500 From: jhell To: Garrett Cooper In-Reply-To: <7d6fde3d1002181413p214389d3nd73798e597b9e26f@mail.gmail.com> Message-ID: References: <7d6fde3d1002181413p214389d3nd73798e597b9e26f@mail.gmail.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) X-OpenPGP-Key-Id: 0x89D8547E X-OpenPGP-Key-Fingerprint: 85EF E26B 07BB 3777 76BE B12A 9057 8789 89D8 547E MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Eitan Adler , hackers@freebsd.org, linimon@freebsd.org Subject: Re: junior tasks wiki page - please add stuff X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2010 05:22:33 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 18 Feb 2010 17:13, yanefbsd@ wrote: > On Thu, Feb 18, 2010 at 11:22 AM, Eitan Adler wrote: >> I made a wiki page[1] to list small odd-jobs that beginners to work on. >> It is intended to answer the question "I'm just starting out with >> coding, what can I help with?" The tasks put on the wiki page should be >> things that require little programming skill. >> >> It would become a lot more useful if people added things to do. >> >> [1] http://wiki.freebsd.org/JuniorJobs > > 1. Who should the wiki content `regulators' be (i.e. project-only > folks, outside folks, etc)? Anyone can sign up for an account and > because the information reflects on the project, it would be best (IMO) > if `proposals' // `tasks' were vetted by someone in the project with > sufficient say to prioritize the item. > 2. Would it be a good idea to also provide a list of tasks that are > non-coding related (or at least have a smaller scope) so that others > could assist with in the project to help free up resources? For > instance, some folks (like linimon@) really need a junior person (I've > debated whether or not I should do it, but I don't want to assume too > much responsibility) to do semi-menial tasks like bug triage, feedback, > basic administration items, etc to free up Mark's time. That way it > would allow someone with less background and exposure to the project to > become more integrated with the project, outside of doing an SoC project > for instance. > Just curious... > Thanks, > -Garrett > It would be awesome if for some way the PR database could be tied into this so items in the database could be promoted as junior tasks. Adding to that, an option in the PR web for someone to be able to vote on certain PR's that should be promoted and ultimately giving a little more free time to the PR admin by not having to visually inspect every PR that has not been categorized for its worth as a junior project, and putting the community as a whole to work in the voting. Ultimately I would see something like the above reducing the PR database greatly and freeing up some man hours for linimon@ so he can get some more coffee ;). Think-Tank open for business ;) Regards, - -- jhell -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iQEcBAEBAgAGBQJLfiAFAAoJEJBXh4mJ2FR+8GcH/j4/44yr/iKUSDHl8GIP6Bhq M6d9rq+rYVuuTO8vJy899XVpYinD0ztFRIpAZb1iQBS6+Tc+PYARllc1BQkKTv0m xAjro78KdrGsoYcIrJ8d/vSlrtV8ojJql7EMZfB9bJyAVX6wAtvnMULn8TvIG6AS QLjasNpi2Zsw+g1dBy416Gy5YnNyZbYEFnh3e67bjF1rxlKSuEWPixOQnEWpEZLh 2G4u4mtZTHnC19zWW/43vW+WbXZBdbcQe8MZr6QnBoSPyjtburHyChKy3ca0szTX TFh/a9Cs84GmW8aDrtYxIGl2GsfZTr8EBuummoSLu8UoLurMA3sW37/xONVni9o= =7fYP -----END PGP SIGNATURE----- From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 19 06:12:25 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 67964106568F for ; Fri, 19 Feb 2010 06:12:25 +0000 (UTC) (envelope-from yanefbsd@gmail.com) Received: from mail-px0-f180.google.com (mail-px0-f180.google.com [209.85.216.180]) by mx1.freebsd.org (Postfix) with ESMTP id 336588FC19 for ; Fri, 19 Feb 2010 06:12:24 +0000 (UTC) Received: by pxi10 with SMTP id 10so2424432pxi.13 for ; Thu, 18 Feb 2010 22:12:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=40MEnjACSLW6p933sbyvyR5qXy64OgSeRwHzgaq7Mj4=; b=JXLjI/YoMdbvjN8cp/wivNwyd1A3x70i4ET4K407lNfARY7fHIkkaLCktNN5kWz9L2 ecJ3ocPztbG8IxM8bKo024PXBm0o2YB2zK6YgdaLD9BDb+VsB4yEBGFeK1sASWN/1/U0 naFr8m9hgKt0o0R6LRwWgfS+x8+f1F5HusCYE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=vrEpu2Z5D2ZRqPkn011yN0CEbfqPmyoJEjNX4SOPrT0hvJsNUO79OlXob7vH8wMzWP jVRMQOZz3KxWQW6ix3GuiVj6gA9Tz0iDeNj/5C3Z2eJgsX0whSsEsJcY2RNKJG5LzIWY fyp9x3Qi4H5HZKPTO9bY5xOWuHm1Or/92tUXw= MIME-Version: 1.0 Received: by 10.143.21.18 with SMTP id y18mr3557668wfi.58.1266559944549; Thu, 18 Feb 2010 22:12:24 -0800 (PST) In-Reply-To: <20100218183459.GA65508@triton8.kn-bremen.de> References: <20100217215940.GA19713@triton8.kn-bremen.de> <4B7CE066.4030403@freebsd.org> <20100218183459.GA65508@triton8.kn-bremen.de> Date: Thu, 18 Feb 2010 22:12:24 -0800 Message-ID: <7d6fde3d1002182212k3bdc866ckdc7c5f380b40f7d8@mail.gmail.com> From: Garrett Cooper To: Juergen Lock Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org, Tim Kientzle Subject: Re: "tar tfv /dev/cd0" speedup patch X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2010 06:12:25 -0000 On Thu, Feb 18, 2010 at 10:34 AM, Juergen Lock wro= te: > On Wed, Feb 17, 2010 at 10:38:30PM -0800, Tim Kientzle wrote: >> Juergen Lock wrote: >> > >> > =A0... =A0since bsdtar/libarchive know iso9660 I just did the command = in the >> > Subject. =A0It worked, but it was sloow... :( =A0Apparently it read al= l of >> > the disc without seeking. =A0The following patch fixes this, is someth= ing >> > like this desired? =A0If yes I could look how to do the same for Linux= , >> >> Juergen, >> >> This is great! =A0If you can figure out how to get this >> right, I would really appreciate it. =A0If you have a >> tape drive handy, definitely test with that. =A0My first >> attempts here actually broke reading from tape drives, >> which is why the current code is so conservative. >> > Hmm I can't test on a tape atm but if I look at the kernel, > =A0 =A0 =A0 =A0http://fxr.watson.org/fxr/ident?i=3DDIOCGMEDIASIZE > DIOCGMEDIASIZE is only handled for geom, xen block devices, old CD drives > and pc98 floppies, and I'm pretty sure tapes don't use geom. :) > >> Minor style comments: >> > =A0else if (S_ISCHR(st.st_mode) && >> > =A0 =A0 !ioctl(fd, DIOCGMEDIASIZE, &mediasize) && mediasize) { >> >> Please be explicit: =A0S_ISCHR() && ioctl() =3D=3D 0 =A0&& mediasize > 0 >> >> > =A0 =A0 archive_read_extract_set_skip_file(a, st.st_dev, st.st_ino); >> >> extract_skip_file isn't needed here; we don't read the >> contents of device nodes. >> >> Let me know as soon as you have something you're confident of. > > =A0Ok here is a new version of the patch with these things fixed and the > Linux case added: =A0(Linux case not tested yet, and yes I did this on > stable/8.) > > Index: src/lib/libarchive/archive_read_open_filename.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > RCS file: /home/scvs/src/lib/libarchive/archive_read_open_filename.c,v > retrieving revision 1.25.2.1 > diff -u -p -r1.25.2.1 archive_read_open_filename.c > --- src/lib/libarchive/archive_read_open_filename.c =A0 =A0 3 Aug 2009 08= :13:06 -0000 =A0 =A0 =A0 1.25.2.1 > +++ src/lib/libarchive/archive_read_open_filename.c =A0 =A0 18 Feb 2010 1= 8:14:16 -0000 > @@ -44,6 +44,10 @@ __FBSDID("$FreeBSD: src/lib/libarchive/a > =A0#ifdef HAVE_UNISTD_H > =A0#include > =A0#endif > +#ifdef __FreeBSD__ > +#include > +#include > +#endif > > =A0#include "archive.h" > > @@ -83,6 +87,9 @@ archive_read_open_filename(struct archiv > =A0 =A0 =A0 =A0struct read_file_data *mine; > =A0 =A0 =A0 =A0void *b; > =A0 =A0 =A0 =A0int fd; > +#ifdef __FreeBSD__ > + =A0 =A0 =A0 off_t mediasize =3D 0; > +#endif > > =A0 =A0 =A0 =A0archive_clear_error(a); > =A0 =A0 =A0 =A0if (filename =3D=3D NULL || filename[0] =3D=3D '\0') { > @@ -143,6 +150,27 @@ archive_read_open_filename(struct archiv > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 */ > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0mine->can_skip =3D 1; > =A0 =A0 =A0 =A0} > +#ifdef __FreeBSD__ > + =A0 =A0 =A0 /* > + =A0 =A0 =A0 =A0* on FreeBSD if a device supports the DIOCGMEDIASIZE ioc= tl > + =A0 =A0 =A0 =A0* it is a disk-like device and should be seekable. > + =A0 =A0 =A0 =A0*/ > + =A0 =A0 =A0 else if (S_ISCHR(st.st_mode) && > + =A0 =A0 =A0 =A0 =A0 ioctl(fd, DIOCGMEDIASIZE, &mediasize) =3D=3D 0 && m= ediasize > 0) { > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 mine->can_skip =3D 1; > + =A0 =A0 =A0 } > +#endif > +#ifdef __linux__ Is there any reason why a) you wouldn't check for other BSDs? b) you aren't doing something of the flavor... #if defined(__FreeBSD__) #elif defined(__linux__) #endif > + =A0 =A0 =A0 /* > + =A0 =A0 =A0 =A0* on Linux just check whether its a block device and tha= t > + =A0 =A0 =A0 =A0* lseek works. =A0(Tapes are character devices there.) > + =A0 =A0 =A0 =A0*/ > + =A0 =A0 =A0 else if (S_ISBLK(st.st_mode) && > + =A0 =A0 =A0 =A0 =A0 lseek(fd, 0, SEEK_CUR) =3D=3D 0 && lseek(fd, 0, SEE= K_SET) =3D=3D 0 && > + =A0 =A0 =A0 =A0 =A0 lseek(fd, 0, SEEK_END) > 0 && lseek(fd, 0, SEEK_SET= ) =3D=3D 0) { > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 mine->can_skip =3D 1; > + =A0 =A0 =A0 =A0} > +#endif > =A0 =A0 =A0 =A0return (archive_read_open2(a, mine, > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0NULL, file_read, file_skip, file_close)); > =A0} Cheers! -Garrett From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 19 07:59:20 2010 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1D937106566C; Fri, 19 Feb 2010 07:59:20 +0000 (UTC) (envelope-from sobomax@FreeBSD.org) Received: from sippysoft.com (gk1.360sip.com [72.236.70.240]) by mx1.freebsd.org (Postfix) with ESMTP id D1AD98FC19; Fri, 19 Feb 2010 07:59:19 +0000 (UTC) Received: from [192.168.1.38] ([70.71.167.197]) (authenticated bits=0) by sippysoft.com (8.14.3/8.14.3) with ESMTP id o1J7xGZJ048745 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 18 Feb 2010 23:59:17 -0800 (PST) (envelope-from sobomax@FreeBSD.org) Message-ID: <4B7E44E4.4090608@FreeBSD.org> Date: Thu, 18 Feb 2010 23:59:32 -0800 From: Maxim Sobolev Organization: Sippy Software, Inc. User-Agent: Thunderbird 2.0.0.23 (Windows/20090812) MIME-Version: 1.0 To: Jack Vogel References: <4B79297D.9080403@FreeBSD.org> <4B79205B.619A0A1A@verizon.net> <4B7ADFC6.7020202@FreeBSD.org> <4B7DE3CC.7040704@FreeBSD.org> <20100219034255.GG11675@michelle.cdnetworks.com> <2a41acea1002182343i1ef0124fm8e2f6cc56846454c@mail.gmail.com> In-Reply-To: <2a41acea1002182343i1ef0124fm8e2f6cc56846454c@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: pyunyh@gmail.com, Sergey Babkin , freebsd-net@FreeBSD.org, Alfred Perlstein , FreeBSD Hackers , "David G. Lawrence" Subject: Re: Sudden mbuf demand increase and shortage under the load (igb issue?) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2010 07:59:20 -0000 Jack Vogel wrote: > This thread is confusing, first he says its an igb problem, then you > offer an em patch :) I suspect it could be patch for the kern/140326. -Maxim From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 19 08:34:31 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AFD361065692 for ; Fri, 19 Feb 2010 08:34:31 +0000 (UTC) (envelope-from matthias.andree@gmx.de) Received: from mail.gmx.net (mail.gmx.net [213.165.64.20]) by mx1.freebsd.org (Postfix) with SMTP id 1578A8FC0C for ; Fri, 19 Feb 2010 08:34:30 +0000 (UTC) Received: (qmail invoked by alias); 19 Feb 2010 08:34:29 -0000 Received: from g227156056.adsl.alicedsl.de (EHLO mandree.no-ip.org) [92.227.156.56] by mail.gmx.net (mp055) with SMTP; 19 Feb 2010 09:34:29 +0100 X-Authenticated: #428038 X-Provags-ID: V01U2FsdGVkX18tSxk4uBMt66y633KTodxhOyG8qAT6gvKyEcJVS4 5v1Gue6pHwcc6Z Received: from merlin.emma.line.org (localhost [127.0.0.1]) by merlin.emma.line.org (Postfix) with ESMTP id 14BFB94657; Fri, 19 Feb 2010 09:34:28 +0100 (CET) Content-Type: text/plain; charset=utf-8; format=flowed; delsp=yes To: "Tim Kientzle" , freebsd-hackers@freebsd.org References: <20100217215940.GA19713@triton8.kn-bremen.de> <4B7CE066.4030403@freebsd.org> <20100218183459.GA65508@triton8.kn-bremen.de> <20100218191347.GA4040@britannica.bec.de> <4B7E1BA5.7050209@freebsd.org> Date: Fri, 19 Feb 2010 09:34:27 +0100 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: "Matthias Andree" Organization: Message-ID: In-Reply-To: <4B7E1BA5.7050209@freebsd.org> User-Agent: Opera Mail/10.10 (Linux) X-Y-GMX-Trusted: 0 X-FuHaFi: 0.52000000000000002 Cc: Subject: seeking and seekability (was: "tar tfv /dev/cd0" speedup patch) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2010 08:34:31 -0000 Am 19.02.2010, 06:03 Uhr, schrieb Tim Kientzle : > Joerg Sonnenberger wrote: >> On Thu, Feb 18, 2010 at 07:34:59PM +0100, Juergen Lock wrote: >>> Ok here is a new version of the patch with these things fixed and the >>> Linux case added: (Linux case not tested yet, and yes I did this on >>> stable/8.) >> Why the check at all? Shouldn't devices that don't allow seek fail >> that? >> E.g. for devices, just try to seek ahead and fallback to normal reading? > > That was the initial implementation in libarchive, but > I had a number of reports of that not working for > tape drives. I recently dug out and connected an > old DDS drive I had in the closet, so I should > probably try again and see if I misunderstood > something along the way. I'd already written this in a private email to Tim before I came across this continued discussion on -hackers@, I'll paste a modified version of my own part: I strongly believe that someone should really fix lseek() in FreeBSD outside GEOM. There is precedence, namely and it was never properly fixed. The PR was closed because the actual reported problem on da(4) no longer occurred since FreeBSD 5 after the GEOM migration, and my issue was with da(4) on FreeBSD 4. Now I'm learning that the very same bug persists through -CURRENT nearly a decade later: after lseek devices such as sa(4) will return "garbage" (i. e. from the unchanged offset) rather than failing. It should be sorted out before 8.1. Am I naive if I expect lseek to return (off_t)-1 with errno == ESPIPE on non-seekable devices? I'll concede that sa(4) is neither socket nor pipe nor fifo in the strict sense, but all share the non-seekability. If lseek() can't know the device isn't seekable, subsequent I/O operations should return EIO along with a proper kernel log message for the first occasion per process, but that would not help applications or libarchive for that matter - they will need a canonical way to find out if a device is seekable. Unfortunately FreeBSD maps many physical block devices that are unbuffered to character special device nodes, so the obvious way of calling [f]stat() and then checking S_ISBLK(st.st_mode) will return FALSE even for devices that can seek. -- Matthias Andree From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 19 07:43:22 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BAFFB1065670; Fri, 19 Feb 2010 07:43:22 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from mail-ew0-f225.google.com (mail-ew0-f225.google.com [209.85.219.225]) by mx1.freebsd.org (Postfix) with ESMTP id C64038FC08; Fri, 19 Feb 2010 07:43:21 +0000 (UTC) Received: by ewy25 with SMTP id 25so799632ewy.3 for ; Thu, 18 Feb 2010 23:43:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=gS2pZogCVsn7ZQAc+VM2BGGhSh+VhuZyej80Um4K6uc=; b=X2tOIdCfIMHw4bLGCa+sYOlxw/sTi7dQH0dQCxDVWBHOeDiaX83jxvublnkZ7x33uK QUD0x5DUBnugM0c9hUh3j9Wy576yJK1uxbi3TJ+BIjsuVffvWsovI6W1F2srQYCRoixv Kiig0AQDoZtNePO/N5Y3SxcZxX4afomRFGvQA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=SC6f/hSW8+2nJohdIu5Jz4EBRtL04TUmBo9+wMsLN5Kho7q3zx5ZaY56JIwzp4MQwC yLeN61gScXCKLmh+dwnS2bsjYWkO50BA3kqP0IcDe5336+KTB9/D82sduw8vE/IDBhhM eA9I/9Uv2ZbgaBkcPu/vS/n5R84WFdt856cR0= MIME-Version: 1.0 Received: by 10.216.90.137 with SMTP id e9mr1044955wef.141.1266565400684; Thu, 18 Feb 2010 23:43:20 -0800 (PST) In-Reply-To: <20100219034255.GG11675@michelle.cdnetworks.com> References: <4B79297D.9080403@FreeBSD.org> <4B79205B.619A0A1A@verizon.net> <4B7ADFC6.7020202@FreeBSD.org> <4B7DE3CC.7040704@FreeBSD.org> <20100219034255.GG11675@michelle.cdnetworks.com> Date: Thu, 18 Feb 2010 23:43:20 -0800 Message-ID: <2a41acea1002182343i1ef0124fm8e2f6cc56846454c@mail.gmail.com> From: Jack Vogel To: pyunyh@gmail.com X-Mailman-Approved-At: Fri, 19 Feb 2010 12:26:08 +0000 Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Maxim Sobolev , Sergey Babkin , FreeBSD Hackers , Alfred Perlstein , freebsd-net@freebsd.org, "David G. Lawrence" Subject: Re: Sudden mbuf demand increase and shortage under the load (igb issue?) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2010 07:43:22 -0000 This thread is confusing, first he says its an igb problem, then you offer an em patch :) I have an important rev of igb that I am about ready to release, anyone that wishes to test against a problem they have would be welcome to have early access, just let me know. I am not sure about this ich10 change, there are client NICs that specifically do NOT support jumbo frames, I'll need to look into it tomorrow at work. Jack On Thu, Feb 18, 2010 at 7:42 PM, Pyun YongHyeon wrote: > On Thu, Feb 18, 2010 at 05:05:16PM -0800, Maxim Sobolev wrote: > > Folks, > > > > Indeed, it looks like igb(4) issue. Replacing the card with the > > desktop-grade em(4)-supported card has fixed the problem for us. The > > system has been happily pushing 110mbps worth of RTP traffic and 2000 > > concurrent calls without any problems for two days now. > > > > em0@pci0:7:0:0: class=0x020000 card=0xa01f8086 chip=0x10d38086 rev=0x00 > > hdr=0x00 > > vendor = 'Intel Corporation' > > class = network > > subclass = ethernet > > > > em0: port 0xec00-0xec1f mem > > 0xfbee0000-0xfbefffff,0xfbe00000-0xfbe7ffff,0xfbedc000-0xfbedffff irq 24 > > at device 0.0 on pci7 > > em0: Using MSIX interrupts > > em0: [ITHREAD] > > em0: [ITHREAD] > > em0: [ITHREAD] > > em0: Ethernet address: 00:1b:21:50:02:49 > > > > I really think that this has to be addressed before 7.3 release is out. > > FreeBSD used to be famous for its excellent network performance and it's > > shame to see that deteriorating due to sub-standard quality drivers. > > Especially when there is a multi-billion vendor supporting the driver in > > question. No finger pointing, but it really looks like either somebody > > is not doing his job or the said vendor doesn't care so much about > > supporting FreeBSD. I am pretty sure the vendor in question has access > > to numerous load-testing tools, that should have catched this issue. > > > > This is the second time during the past 6 months I have issue with the > > quality of the Intel-based drivers - the first one is filed as > > kern/140326, which has stalled apparently despite me providing all > > necessary debug information. > > > > I can reproduce this bug on my box and I guess the root cause comes > from PBA(Packet Buffer Allocation) configuration. Some controllers > seems to require more TX buffer size to use 9000 MTU. The datasheet > is not clear which controller has how much amount of Packet Buffer > storage. This parameter seems to affect performance a lot because > increasing TX buffer size results in decreasing RX buffer size. The > attached patch seems to fix the issue for me but Jack may know > better the hardware details as publicly available datasheet seems > to be useless here. > From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 19 11:05:49 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 31F1D106566C for ; Fri, 19 Feb 2010 11:05:49 +0000 (UTC) (envelope-from bounces@nabble.com) Received: from kuber.nabble.com (kuber.nabble.com [216.139.236.158]) by mx1.freebsd.org (Postfix) with ESMTP id 1451B8FC12 for ; Fri, 19 Feb 2010 11:05:48 +0000 (UTC) Received: from isper.nabble.com ([192.168.236.156]) by kuber.nabble.com with esmtp (Exim 4.63) (envelope-from ) id 1NiQNm-0006jE-Vm for freebsd-hackers@freebsd.org; Fri, 19 Feb 2010 02:46:50 -0800 Message-ID: <27652360.post@talk.nabble.com> Date: Fri, 19 Feb 2010 02:46:50 -0800 (PST) From: Jakub Lach To: freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Nabble-From: jakub_lach@mailplus.pl X-Mailman-Approved-At: Fri, 19 Feb 2010 12:26:27 +0000 Subject: mptable(1) typo X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2010 11:05:49 -0000 Hello. There's small typo in mptable output - Bus "Heirarchy" from mptable.c tableEntry extendedtableEntryTypes[] = { { 128, 20, "System Address Space" }, { 129, 8, "Bus Heirarchy" }, { 130, 8, "Compatibility Bus Address" } }; -best regards, Jakub Lach -- View this message in context: http://old.nabble.com/mptable%281%29-typo-tp27652360p27652360.html Sent from the freebsd-hackers mailing list archive at Nabble.com. From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 19 14:46:29 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A55F71065676 for ; Fri, 19 Feb 2010 14:46:29 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 78CD08FC15 for ; Fri, 19 Feb 2010 14:46:29 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 24B2B46B06; Fri, 19 Feb 2010 09:46:29 -0500 (EST) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id B29708A024; Fri, 19 Feb 2010 09:46:27 -0500 (EST) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Fri, 19 Feb 2010 09:27:43 -0500 User-Agent: KMail/1.12.1 (FreeBSD/7.2-CBSD-20100120; KDE/4.3.1; amd64; ; ) References: <27652360.post@talk.nabble.com> In-Reply-To: <27652360.post@talk.nabble.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201002190927.43406.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Fri, 19 Feb 2010 09:46:27 -0500 (EST) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.5 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: Jakub Lach Subject: Re: mptable(1) typo X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2010 14:46:29 -0000 On Friday 19 February 2010 5:46:50 am Jakub Lach wrote: > > Hello. > > There's small typo in mptable > output - > > Bus "Heirarchy" > > from mptable.c > > tableEntry extendedtableEntryTypes[] = > { > { 128, 20, "System Address Space" }, > { 129, 8, "Bus Heirarchy" }, > { 130, 8, "Compatibility Bus Address" } > }; Fixed in HEAD, thanks. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 19 14:56:19 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2C73F1065679; Fri, 19 Feb 2010 14:56:19 +0000 (UTC) (envelope-from ray@dlink.ua) Received: from dlink.ua (smtp.dlink.ua [193.138.187.146]) by mx1.freebsd.org (Postfix) with ESMTP id E03638FC15; Fri, 19 Feb 2010 14:56:18 +0000 (UTC) Received: from gw ([192.168.10.10] helo=terran) by dlink.ua with esmtpsa (TLS-1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.63) (envelope-from ) id 1NiTtK-0006NL-Cl; Fri, 19 Feb 2010 16:31:38 +0200 Date: Fri, 19 Feb 2010 16:36:44 +0200 From: Alexandr Rybalko To: geom@freebsd.org, hackers@freebsd.org, embedded@freebsd.org Message-Id: <20100219163644.da89e882.ray@dlink.ua> Organization: D-Link X-Mailer: Sylpheed 2.7.1 (GTK+ 2.16.6; i386-portbld-freebsd8.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: Subject: GEOM_ULZMA X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2010 14:56:19 -0000 Hi, I wrote a module GEOM_ULZMA (such as GEOM_UZIP, but compression with lzma), in connection with this is an issue best left lzma code in the file "geom_ulzma.c" or store lzma library separately. If separately, then where better? Maybe in future make lzma and gzip library kernel interface for embedded? Then in one instance of code, userland can use compression via kernel. -- Alexander Rybalko From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 19 15:38:24 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 17055106566C for ; Fri, 19 Feb 2010 15:38:24 +0000 (UTC) (envelope-from freebsd-hackers@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id C40B88FC08 for ; Fri, 19 Feb 2010 15:38:23 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1NiUv5-0007QD-Eb for freebsd-hackers@freebsd.org; Fri, 19 Feb 2010 16:37:31 +0100 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 19 Feb 2010 16:37:31 +0100 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 19 Feb 2010 16:37:31 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-hackers@freebsd.org From: Ivan Voras Date: Fri, 19 Feb 2010 16:36:40 +0100 Lines: 28 Message-ID: References: <20100219163644.da89e882.ray__27111.9062825621$1266591431$gmane$org@dlink.ua> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.5) Gecko/20100118 Thunderbird/3.0 In-Reply-To: <20100219163644.da89e882.ray__27111.9062825621$1266591431$gmane$org@dlink.ua> Sender: news Subject: Re: GEOM_ULZMA X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2010 15:38:24 -0000 On 02/19/10 15:36, Alexandr Rybalko wrote: > Hi, > I wrote a module GEOM_ULZMA (such as GEOM_UZIP, but compression with lzma), in connection with this is an issue best left lzma One of the (relatively) unexpected problems with such block-level compression is its efficienty - since you need to compress individual blocks to be seekable (if you are not designing a special algorithm), the compression ratios tend to not increase in a big way as the algorithm improves. It would be interesting to see a comparison of sizes with geom_uzip if you have them. > code in the file "geom_ulzma.c" or store lzma library separately. If separately, then where better? The code organization depends on what you want to do with it and how you want to update the code in the future, if your lzma library is third party. If you never intend to update the lzma code then I guess it's fine to embed it in a big .c file. For a port, it doesn't matter much since it is your own thing. There are stricter rules on maintainability and style if you want it in the base system. > Maybe in future make lzma and gzip library kernel interface for embedded? > Then in one instance of code, userland can use compression via kernel. This never happens now (I mean userland calling into the kernel for compression) so you will need to explain the benefits first (kernel code is not magically faster than userland - it runs on the same CPU). From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 19 15:49:03 2010 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 61D19106566B for ; Fri, 19 Feb 2010 15:49:03 +0000 (UTC) (envelope-from giffunip@tutopia.com) Received: from web113519.mail.gq1.yahoo.com (web113519.mail.gq1.yahoo.com [98.136.167.59]) by mx1.freebsd.org (Postfix) with SMTP id 22B938FC1E for ; Fri, 19 Feb 2010 15:49:02 +0000 (UTC) Received: (qmail 87937 invoked by uid 60001); 19 Feb 2010 15:49:02 -0000 Message-ID: <346566.86917.qm@web113519.mail.gq1.yahoo.com> X-YMail-OSG: QUm8FgcVM1kiL_RX9uP1w3mue92M_NVikZ8ImHzUYB8_e0saWzETlgARTxLc8VuqZacGLy_ye26WoKKE1wo0PD1Zq.31UncsD_amsK3AXLU0i.mkw_xOS2CjFyYomV7J34VCirFYWS_81r2PxE_EKNfoflFn9aVT8IUfqqC072qrd4A1NBDpO5br2c2CGH.y2dNXFdfWu2QOi9_tcNqrMLAPv9cXWgCbKVxaQ4Ym4mwQxUwNCzQRPOQDub_K_SIlpFcYxk7pSd.ZU_fJYJ9ENukOGzN9Ep02tBu1CkF6Rwu9qQl_04BO_wKjgg4W4wTBhlX6uX_VNfdBtm3SSM0yQF36RVl7FDzj278pJ3YfMPZpnal8MFTmmAdT9Q-- Received: from [190.157.123.47] by web113519.mail.gq1.yahoo.com via HTTP; Fri, 19 Feb 2010 07:49:02 PST X-RocketYMMF: giffunip X-Mailer: YahooMailClassic/9.2.12 YahooMailWebService/0.8.100.260964 Date: Fri, 19 Feb 2010 07:49:02 -0800 (PST) From: "Pedro F. Giffuni" To: freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailman-Approved-At: Fri, 19 Feb 2010 17:21:07 +0000 Cc: Subject: Anyone updating the Project Ideas page? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: giffunip@tutopia.com List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2010 15:49:03 -0000 Hello; I've sent some private messages before trying to get some updates to the Project ideas page http://www.freebsd.org/projects/ideas/ideas.html For example, thinking of the next SoC: Analyze NetBSD's ext2fs regarding valuable improvements - This was done as GSoC project. Nothing there to look at there anymore :-). - There is a lot of documentation to look at though http://docs.freebsd.org/cgi/mid.cgi?454084.84334.qm Implement co-location for UFS2 -This would require a VM expert too. UDF could get an update from the NetBSD code: http://docs.freebsd.org/cgi/mid.cgi?4B7B04F6.3010302 Porting HFS+ would still be desirable. Fixing/updating the SVR4 emulation code to work with Opensolaris? cheers, Pedro. From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 19 17:30:24 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2A8FC106566B for ; Fri, 19 Feb 2010 17:30:24 +0000 (UTC) (envelope-from psteele@maxiscale.com) Received: from server505.appriver.com (server505c.appriver.com [98.129.35.7]) by mx1.freebsd.org (Postfix) with ESMTP id E54418FC24 for ; Fri, 19 Feb 2010 17:30:23 +0000 (UTC) X-Policy: GLOBAL - maxiscale.com X-Primary: psteele@maxiscale.com X-Note: This Email was scanned by AppRiver SecureTide X-ALLOW: psteele@maxiscale.com ALLOWED X-Virus-Scan: V- X-Note: Spam Tests Failed: X-Country-Path: UNITED STATES->UNITED STATES->UNITED STATES X-Note-Sending-IP: 98.129.23.14 X-Note-Reverse-DNS: ht01.exg5.exghost.com X-Note-WHTLIST: psteele@maxiscale.com X-Note: User Rule Hits: X-Note: Global Rule Hits: G173 G174 G175 G176 G180 G181 G192 G279 X-Note: Encrypt Rule Hits: X-Note: Mail Class: ALLOWEDSENDER X-Note: Headers Injected Received: from [98.129.23.14] (HELO ht01.exg5.exghost.com) by server505.appriver.com (CommuniGate Pro SMTP 5.3.2) with ESMTPS id 28638168 for freebsd-hackers@freebsd.org; Fri, 19 Feb 2010 11:30:23 -0600 Received: from mbx03.exg5.exghost.com ([169.254.1.200]) by ht01.exg5.exghost.com ([98.129.23.14]) with mapi; Fri, 19 Feb 2010 11:30:22 -0600 From: Peter Steele To: "freebsd-hackers@freebsd.org" Date: Fri, 19 Feb 2010 11:30:21 -0600 Thread-Topic: ntpd hangs under FBSD 8 Thread-Index: AcqxiTaYmJQ81U31QBSUPTiyqVxE7Q== Message-ID: <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D5C73@MBX03.exg5.exghost.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: ntpd hangs under FBSD 8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2010 17:30:24 -0000 I posted this originally on the -questions list but did not make any headwa= y. We have an application where the user can change the date/time via a GUI= . One of the options the user has is to specify that the time is to be sync= ed using ntp. Our coding worked fine under BSD 7 but since we've moved to B= SD 8 we've encountered a problem where the command that we initiate from th= e GUI: ntpd -g -q to perform the initial time sync is hanging indefinitely. Logs we've captur= ed do not give any clues. This is the log from a BSD 7 system produced when= this ntpd command is run: 17 Feb 06:35:36 ntpd[3578]: logging to file /var/log/ntpd.log 17 Feb 06:35:36 ntpd[3578]: ntpd 4.2.0-a Sun Feb 24 09:12:07 UTC 2008 (1) 17 Feb 06:35:36 ntpd[3578]: precision =3D 1.676 usec 17 Feb 06:35:36 ntpd[3578]: kernel time sync status 2040 17 Feb 06:35:36 ntpd[3578]: frequency initialized -10.706 PPM from /var/db/= ntpd.drift 17 Feb 06:35:45 ntpd[3578]: synchronized to 198.186.191.229, stratum=3D2 17 Feb 06:35:45 ntpd[3578]: time slew +0.003648 s and this is the log from a BSD 8 system: 17 Feb 06:35:36 ntpd[2293]: logging to file /var/log/ntpd.log 17 Feb 06:35:36 ntpd[2293]: precision =3D 1.676 usec 17 Feb 06:35:36 ntpd[2293]: Listening on interface #0 wildcard, 0.0.0.0#123= Disabled 17 Feb 06:35:36 ntpd[2293]: Listening on interface #1 wildcard, ::#123 Disa= bled 17 Feb 06:35:36 ntpd[2293]: Listening on interface #2 nic0, fe80::2a0:d1ff:= fee3:53cc#123 Enabled 17 Feb 06:35:36 ntpd[2293]: Listening on interface #3 nic1, fe80::2a0:d1ff:= fee3:53cd#123 Enabled 17 Feb 06:35:36 ntpd[2293]: Listening on interface #4 lo0, fe80::1#123 Enab= led 17 Feb 06:35:36 ntpd[2293]: Listening on interface #5 lo0, ::1#123 Enabled 17 Feb 06:35:36 ntpd[2293]: Listening on interface #6 lo0, 127.0.0.1#123 En= abled 17 Feb 06:35:36 ntpd[2293]: Listening on interface #7 lagg0, 192.168.17.46#= 123 Enabled 17 Feb 06:35:36 ntpd[2293]: Listening on routing socket on fd #29 for inter= face updates 17 Feb 06:35:36 ntpd[2293]: kernel time sync status 2040 17 Feb 06:35:36 ntpd[2293]: frequency initialized -10.706 PPM from /var/db/= ntpd.drift It never gets past this last log line and we have to do a kill -9 on the nt= pd process. The ntp.conf file we're using is # General Configuration server 0.us.pool.ntp.org server 1.us.pool.ntp.org server 2.us.pool.ntp.org server 3.us.pool.ntp.org # Drift file driftfile /var/db/ntpd.drift The versions of the two ntpd binaries are different--4.2.0-a for FBSD 7 and= 4.2.4p5 for FBSD 8. Someone suggested that I try the command: ntpq -pc rv localhost But I'm not sure how to interpret the output. On a FBSD 7 system I get this= : remote refid st t when poll reach delay offset jit= ter =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D +169.229.70.183 169.229.128.214 3 u 40 512 37 7.921 9.170 8.= 836 *208.75.88.4 192.12.19.20 2 u 43 512 37 12.049 8.224 8.= 168 +217.160.254.116 209.51.161.238 2 u 38 512 37 55.111 -7.128 10.= 347 +198.247.173.220 128.206.12.130 3 u 39 512 37 47.401 -1.149 3.= 659 status=3Dc624 sync_alarm, sync_ntp, 2 events, event_peer/strat_chg, version= =3D"ntpd 4.2.0-a Sun Feb 24 09:12:07 UTC 2008 (1)", processor=3D"amd64", sy= stem=3D"FreeBSD/7.0-RELEASE-p9", leap=3D11, stratum=3D16, precision=3D-20, = rootdelay=3D0.000, rootdispersion=3D8.340, peer=3D25349, refid=3DINIT, reft= ime=3D00000000.00000000 Wed, Feb 6 2036 22:28:16.000, poll=3D4, clock=3Dc= f26c2d5.ea2b4541 Wed, Feb 17 2010 11:32:37.914, state=3D1, offset=3D0.000,= frequency=3D-13.269, jitter=3D0.001, stability=3D0.000 and on a FBSD 8 system I get this: remote refid st t when poll reach delay offset jit= ter =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D assID=3D0 status=3Dc011 sync_alarm, sync_unspec, 1 event, event_restart, ve= rsion=3D"ntpd 4.2.4p5-a (1)", processor=3D"amd64", system=3D"FreeBSD/8.0-CU= RRENT", leap=3D11, stratum=3D16, precision=3D-19, rootdelay=3D0.000, rootdi= spersion=3D0.000, peer=3D0, refid=3DINIT, reftime=3D00000000.00000000 Wed, Feb 6 2036 22:28:16.000, poll=3D6, clock=3Dcf26c4d1.d21b33f1 Wed, Feb 17 2010 11:41:05.820, state=3D1, offset= =3D0.000, frequency=3D-14.299, jitter=3D0.002, noise=3D0.002, stability=3D0= .000, tai=3D0 169.229.70.183 .INIT. 16 u - 64 0 0.000 0.000 0.0= 02 208.75.88.4 .INIT. 16 u - 64 0 0.000 0.000 0.0= 02 217.160.254.116 .INIT. 16 u - 64 0 0.000 0.000 0.0= 02 198.137.202.16 .INIT. 16 u - 64 0 0.000 0.000 0.0= 02 In the case of the FBSD8 output, I collected this while one of these hangs = was happening. The most obvious difference is the .INIT. entries, but there= also appear to be several "0.0" type of entries that look like the ntpd pr= ocess is stuck in some kind of initialization state. Anyone have any ideas what's going on here? From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 19 17:52:38 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A3BEF106566B; Fri, 19 Feb 2010 17:52:38 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-qy0-f183.google.com (mail-qy0-f183.google.com [209.85.221.183]) by mx1.freebsd.org (Postfix) with ESMTP id 0DCF58FC1D; Fri, 19 Feb 2010 17:52:37 +0000 (UTC) Received: by qyk13 with SMTP id 13so195235qyk.13 for ; Fri, 19 Feb 2010 09:52:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=W8Zj/EEqIbdkgFJwYHKKxZFwq/gDt86PoOFCFUuFgEQ=; b=HKcGOGrvUor6k49wbis9rM2NF+z1vyZJigFxJ5cPPbhyvpa+/MSaqXN0jPST2gFSjr sXY16uM5wpii2r9HzWi6vNC+WbtXiBRgAkTtZi+EsJsO5Mmw27O39k3ut4lKq6WWYD2X 4sfTAYpLWeNF8XuY77Ss1GvDsMvZURT9HEzfs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=G+bAMMPYMd2QyLOQK5lbb1mzdi0ln1V8vwCgw4XIkP5XmiuG3ulfs7onpBfcyDxtP7 4tvffD23is3VsBreuxFjdYvPaHMhblJUH3OHzsIhg1QPr8LWPFvlPu6hLkp6Bhh/JVFt 4zmzZYEvU9+9KjGa2wcI0Fp5cBH8jP/xP9Otc= Received: by 10.224.73.76 with SMTP id p12mr3612694qaj.187.1266601957068; Fri, 19 Feb 2010 09:52:37 -0800 (PST) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id 6sm1221684qwk.10.2010.02.19.09.52.33 (version=TLSv1/SSLv3 cipher=RC4-MD5); Fri, 19 Feb 2010 09:52:35 -0800 (PST) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Fri, 19 Feb 2010 09:51:55 -0800 From: Pyun YongHyeon Date: Fri, 19 Feb 2010 09:51:55 -0800 To: Jack Vogel Message-ID: <20100219175155.GH11675@michelle.cdnetworks.com> References: <4B79297D.9080403@FreeBSD.org> <4B79205B.619A0A1A@verizon.net> <4B7ADFC6.7020202@FreeBSD.org> <4B7DE3CC.7040704@FreeBSD.org> <20100219034255.GG11675@michelle.cdnetworks.com> <2a41acea1002182343i1ef0124fm8e2f6cc56846454c@mail.gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2a41acea1002182343i1ef0124fm8e2f6cc56846454c@mail.gmail.com> User-Agent: Mutt/1.4.2.3i Cc: Maxim Sobolev , Sergey Babkin , FreeBSD Hackers , Alfred Perlstein , freebsd-net@freebsd.org, "David G. Lawrence" Subject: Re: Sudden mbuf demand increase and shortage under the load (igb issue?) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2010 17:52:38 -0000 On Thu, Feb 18, 2010 at 11:43:20PM -0800, Jack Vogel wrote: > This thread is confusing, first he says its an igb problem, then you offer > an em patch :) > > I have an important rev of igb that I am about ready to release, anyone that > wishes to > test against a problem they have would be welcome to have early access, just > let me > know. > > I am not sure about this ich10 change, there are client NICs that > specifically do NOT > support jumbo frames, I'll need to look into it tomorrow at work. Hmm, then how could I see advertised MSS 8960 on this controller? I guess em(4) already checks MAC type to limit jumbo frame support on these controllers. 09:44:18.701271 IP (tos 0x0, ttl 64, id 32888, offset 0, flags [DF], proto TCP (6), length 60) 192.168.30.2.55754 > 192.168.30.1.22: S, cksum 0xbd82 (incorrect (-> 0x3f62), 529388419:529388419(0) win 65535 09:44:18.701538 IP (tos 0x0, ttl 64, id 40268, offset 0, flags [DF], proto TCP (6), length 60) 192.168.30.1.22 > 192.168.30.2.55754: S, cksum 0xa6ce (correct), 3828990973:3828990973(0) ack 529388420 win 65535 09:44:18.701549 IP (tos 0x0, ttl 64, id 32889, offset 0, flags [DF], proto TCP (6), length 52) 192.168.30.2.55754 > 192.168.30.1.22: ., cksum 0xbd7a (incorrect (-> 0xd2e1), 1:1(0) ack 1 win 8192 em0@pci0:0:25:0: class=0x020000 card=0x027f1028 chip=0x10de8086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Intel Gigabit network connection (83567LM-3 )' class = network subclass = ethernet bar [10] = type Memory, range 32, base 0xfdae0000, size 131072, enabled bar [14] = type Memory, range 32, base 0xfdad9000, size 4096, enabled bar [18] = type I/O Port, range 32, base 0xecc0, size 32, enabled cap 01[c8] = powerspec 2 supports D0 D3 current D0 cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message cap 13[e0] = PCI Advanced Features: FLR TP > > Jack > > > On Thu, Feb 18, 2010 at 7:42 PM, Pyun YongHyeon wrote: > > > On Thu, Feb 18, 2010 at 05:05:16PM -0800, Maxim Sobolev wrote: > > > Folks, > > > > > > Indeed, it looks like igb(4) issue. Replacing the card with the > > > desktop-grade em(4)-supported card has fixed the problem for us. The > > > system has been happily pushing 110mbps worth of RTP traffic and 2000 > > > concurrent calls without any problems for two days now. > > > > > > em0@pci0:7:0:0: class=0x020000 card=0xa01f8086 chip=0x10d38086 rev=0x00 > > > hdr=0x00 > > > vendor = 'Intel Corporation' > > > class = network > > > subclass = ethernet > > > > > > em0: port 0xec00-0xec1f mem > > > 0xfbee0000-0xfbefffff,0xfbe00000-0xfbe7ffff,0xfbedc000-0xfbedffff irq 24 > > > at device 0.0 on pci7 > > > em0: Using MSIX interrupts > > > em0: [ITHREAD] > > > em0: [ITHREAD] > > > em0: [ITHREAD] > > > em0: Ethernet address: 00:1b:21:50:02:49 > > > > > > I really think that this has to be addressed before 7.3 release is out. > > > FreeBSD used to be famous for its excellent network performance and it's > > > shame to see that deteriorating due to sub-standard quality drivers. > > > Especially when there is a multi-billion vendor supporting the driver in > > > question. No finger pointing, but it really looks like either somebody > > > is not doing his job or the said vendor doesn't care so much about > > > supporting FreeBSD. I am pretty sure the vendor in question has access > > > to numerous load-testing tools, that should have catched this issue. > > > > > > This is the second time during the past 6 months I have issue with the > > > quality of the Intel-based drivers - the first one is filed as > > > kern/140326, which has stalled apparently despite me providing all > > > necessary debug information. > > > > > > > I can reproduce this bug on my box and I guess the root cause comes > > from PBA(Packet Buffer Allocation) configuration. Some controllers > > seems to require more TX buffer size to use 9000 MTU. The datasheet > > is not clear which controller has how much amount of Packet Buffer > > storage. This parameter seems to affect performance a lot because > > increasing TX buffer size results in decreasing RX buffer size. The > > attached patch seems to fix the issue for me but Jack may know > > better the hardware details as publicly available datasheet seems > > to be useless here. > > From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 19 18:27:35 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0D67B106566B for ; Fri, 19 Feb 2010 18:27:35 +0000 (UTC) (envelope-from fernando.apesteguia@gmail.com) Received: from mail-ew0-f211.google.com (mail-ew0-f211.google.com [209.85.219.211]) by mx1.freebsd.org (Postfix) with ESMTP id 944BB8FC17 for ; Fri, 19 Feb 2010 18:27:34 +0000 (UTC) Received: by ewy3 with SMTP id 3so392580ewy.13 for ; Fri, 19 Feb 2010 10:27:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=2WceeSA+RS9FEO8SwpQMZixr88g8RqgpIT/tzqxmgKg=; b=WuykgMAYxyOcuVITqm5wFIzPg4VBPF71Pbd+zMtE4Y6jEFQSv1aBDO3rzX48KzAkHL w+cRrAu/D0vs+VwCH0Ur5a294ewq3CoJNuAM7eHxLTXmoxkIECA5z127z8+8YCFjx05s c5bZMW/GK+R4cRKeUG4JpozAhGAJl+R4nank0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=PUYVF06w1chvu3yqSR/fadCSffKPYWA+wR3zVfcy//zlqDjEXr0rymqvy5jJGxajYi 2DfJHAYCax+7tsq0R7lkyE4k7cxO8HZCAhuVvrPVKnLfUiCr8ZAx7sYryw2L6l4906BC CfKiwNPMNpLbNq7YKtgYJ/oZJSogYWBy6btMI= MIME-Version: 1.0 Received: by 10.213.1.24 with SMTP id 24mr1039118ebd.57.1266604052721; Fri, 19 Feb 2010 10:27:32 -0800 (PST) In-Reply-To: <20100218202143.GT50403@deviant.kiev.zoral.com.ua> References: <1bd550a01002171051n7117895avb5cf57fb7fbb9388@mail.gmail.com> <20100217191156.GP50403@deviant.kiev.zoral.com.ua> <1bd550a01002180948r48300f8em15efca184d99ed98@mail.gmail.com> <20100218202143.GT50403@deviant.kiev.zoral.com.ua> Date: Fri, 19 Feb 2010 19:27:32 +0100 Message-ID: <1bd550a01002191027m75657308n8227b738dc64486e@mail.gmail.com> From: =?ISO-8859-1?Q?Fernando_Apestegu=EDa?= To: Kostik Belousov Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Hackers Subject: Re: linprocfs proc/pid/environ patch & list question X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2010 18:27:35 -0000 2010/2/18 Kostik Belousov : > On Thu, Feb 18, 2010 at 06:48:35PM +0100, Fernando Apestegu?a wrote: >> On Wed, Feb 17, 2010 at 8:11 PM, Kostik Belousov w= rote: >> > On Wed, Feb 17, 2010 at 07:51:06PM +0100, Fernando Apestegu?a wrote: >> >> Hi, >> >> >> >> I have a small patch (against 8.0-RELEASE-p2) that _should_ implement >> >> the /proc/pid/environ file >> >> under linprocfs. >> >> However, it seems it does not work properly but I don't know what I'm >> >> doing wrong. >> >> Is this list the place to ask for help? I tried in the forums[1] but >> >> got no answer. >> > Putting aside any "does not work" questions, please see comment below. >> >> Sorry I didn't explain this. If I have a process forked from bash >> shell in which I have >> exported VAR=3DXXXX the /compat/linux/proc/pid/environ for the child pro= cess >> does not show the VAR variable. > Copyin copies the data from the address space of the current process. > You are interested in the content of address space of different process. > Look at the proc_rwmem(). Thanks > >> >> >> >> >> Don't we have a 'kernel newbies'-like list? >> >> >> >> Thanks in advance. >> >> >> >> [1] http://forums.freebsd.org/showthread.php?t=3D11329 >> >> >> >> --- sys/compat/linprocfs/linprocfs.c.orig =A0 =A0 2009-10-25 02:10:29= .000000000 +0100 >> >> +++ sys/compat/linprocfs/linprocfs.c =A02010-02-16 19:38:36.000000000= +0100 >> >> @@ -939,8 +939,38 @@ >> >> =A0static int >> >> =A0linprocfs_doprocenviron(PFS_FILL_ARGS) >> >> =A0{ >> >> + =A0 =A0 int i, error; >> >> + =A0 =A0 struct ps_strings pss; >> >> + =A0 =A0 char **ps_envstr; >> >> >> >> - =A0 =A0 sbuf_printf(sb, "doprocenviron\n%c", '\0'); >> >> + =A0 =A0 PROC_LOCK(p); >> >> + =A0 =A0 if (p_cansee(td, p) !=3D 0) >> >> + =A0 =A0 =A0 =A0 =A0 =A0 return (0); >> >> + =A0 =A0 PROC_UNLOCK(p); >> >> + >> >> + =A0 =A0 error =3D copyin((void *)p->p_sysent->sv_psstrings, &pss, >> >> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= =A0 =A0 =A0 sizeof(pss)); >> >> + =A0 =A0 if (error) >> >> + =A0 =A0 =A0 =A0 =A0 =A0 return (error); >> >> + >> >> + =A0 =A0 ps_envstr =3D malloc(pss.ps_nenvstr * sizeof(char *), >> >> + =A0 =A0 =A0 =A0 M_TEMP, M_WAITOK); >> > This is essentially "panic me" code. =A0ps_nenvstr is user-controlled, >> > and allows to specify arbitrary integers. >> > >> > Even ignoring exhaustion of the kernel map, it can cause allocation of >> > big amount of physical memory. Note that execve(2) implementation uses >> > swappable memory to store arguments and environment strings passed fro= m >> > vm spaces. >> >> Thanks for the comment. If I want to check ps_envstr, which threshold wo= uld be >> reasonable? PAGE_SIZE maybe? >> >> Thanks again. >> >> > >> >> + >> >> + =A0 =A0 error =3D copyin((void *)pss.ps_envstr, ps_envstr, >> >> + =A0 =A0 =A0 =A0 pss.ps_nenvstr * sizeof(char *)); >> >> + >> >> + =A0 =A0 if (error) { >> >> + =A0 =A0 =A0 =A0 =A0 =A0 free(ps_envstr, M_TEMP); >> >> + =A0 =A0 =A0 =A0 =A0 =A0 return (error); >> >> + =A0 =A0 } >> >> + >> >> + =A0 =A0 /* NULL separated list of variable=3Dvalue pairs */ >> >> + >> >> + =A0 =A0 for (i =3D 0; i < pss.ps_nenvstr; i++) { >> >> + =A0 =A0 =A0 =A0 =A0 =A0 sbuf_copyin(sb, ps_envstr[i], 0); >> >> + =A0 =A0 } >> >> + >> >> + =A0 =A0 free(ps_envstr, M_TEMP); >> >> =A0 =A0 =A0 return (0); >> >> =A0} >> >> _______________________________________________ >> >> freebsd-hackers@freebsd.org mailing list >> >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd= .org" >> > > From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 19 18:24:11 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5B5B71065676; Fri, 19 Feb 2010 18:24:11 +0000 (UTC) (envelope-from nox@jelal.kn-bremen.de) Received: from smtp.kn-bremen.de (gelbbaer.kn-bremen.de [78.46.108.116]) by mx1.freebsd.org (Postfix) with ESMTP id E841A8FC3F; Fri, 19 Feb 2010 18:24:10 +0000 (UTC) Received: by smtp.kn-bremen.de (Postfix, from userid 10) id 7418C1E00772; Fri, 19 Feb 2010 19:24:09 +0100 (CET) Received: from triton8.kn-bremen.de (noident@localhost [127.0.0.1]) by triton8.kn-bremen.de (8.14.3/8.14.3) with ESMTP id o1JIClsf035888; Fri, 19 Feb 2010 19:12:47 +0100 (CET) (envelope-from nox@triton8.kn-bremen.de) Received: (from nox@localhost) by triton8.kn-bremen.de (8.14.3/8.14.3/Submit) id o1JIClYS035887; Fri, 19 Feb 2010 19:12:47 +0100 (CET) (envelope-from nox) From: Juergen Lock Date: Fri, 19 Feb 2010 19:12:47 +0100 To: Garrett Cooper Message-ID: <20100219181247.GA35702@triton8.kn-bremen.de> References: <20100217215940.GA19713@triton8.kn-bremen.de> <4B7CE066.4030403@freebsd.org> <20100218183459.GA65508@triton8.kn-bremen.de> <7d6fde3d1002182212k3bdc866ckdc7c5f380b40f7d8@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <7d6fde3d1002182212k3bdc866ckdc7c5f380b40f7d8@mail.gmail.com> User-Agent: Mutt/1.5.20 (2009-06-14) X-Mailman-Approved-At: Fri, 19 Feb 2010 18:36:11 +0000 Cc: freebsd-hackers@freebsd.org, Tim Kientzle , Juergen Lock Subject: Re: "tar tfv /dev/cd0" speedup patch X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2010 18:24:11 -0000 On Thu, Feb 18, 2010 at 10:12:24PM -0800, Garrett Cooper wrote: > On Thu, Feb 18, 2010 at 10:34 AM, Juergen Lock wrote: > > On Wed, Feb 17, 2010 at 10:38:30PM -0800, Tim Kientzle wrote: > >> Juergen Lock wrote: > >> > > >> >  ...  since bsdtar/libarchive know iso9660 I just did the command in the > >> > Subject.  It worked, but it was sloow... :(  Apparently it read all of > >> > the disc without seeking.  The following patch fixes this, is something > >> > like this desired?  If yes I could look how to do the same for Linux, > >> > >> Juergen, > >> > >> This is great!  If you can figure out how to get this > >> right, I would really appreciate it.  If you have a > >> tape drive handy, definitely test with that.  My first > >> attempts here actually broke reading from tape drives, > >> which is why the current code is so conservative. > >> > > Hmm I can't test on a tape atm but if I look at the kernel, > >        http://fxr.watson.org/fxr/ident?i=DIOCGMEDIASIZE > > DIOCGMEDIASIZE is only handled for geom, xen block devices, old CD drives > > and pc98 floppies, and I'm pretty sure tapes don't use geom. :) > > > >> Minor style comments: > >> >  else if (S_ISCHR(st.st_mode) && > >> >     !ioctl(fd, DIOCGMEDIASIZE, &mediasize) && mediasize) { > >> > >> Please be explicit:  S_ISCHR() && ioctl() == 0  && mediasize > 0 > >> > >> >     archive_read_extract_set_skip_file(a, st.st_dev, st.st_ino); > >> > >> extract_skip_file isn't needed here; we don't read the > >> contents of device nodes. > >> > >> Let me know as soon as you have something you're confident of. > > > >  Ok here is a new version of the patch with these things fixed and the > > Linux case added:  (Linux case not tested yet, and yes I did this on > > stable/8.) > > > > Index: src/lib/libarchive/archive_read_open_filename.c > > =================================================================== > > RCS file: /home/scvs/src/lib/libarchive/archive_read_open_filename.c,v > > retrieving revision 1.25.2.1 > > diff -u -p -r1.25.2.1 archive_read_open_filename.c > > --- src/lib/libarchive/archive_read_open_filename.c     3 Aug 2009 08:13:06 -0000       1.25.2.1 > > +++ src/lib/libarchive/archive_read_open_filename.c     18 Feb 2010 18:14:16 -0000 > > @@ -44,6 +44,10 @@ __FBSDID("$FreeBSD: src/lib/libarchive/a > >  #ifdef HAVE_UNISTD_H > >  #include > >  #endif > > +#ifdef __FreeBSD__ > > +#include > > +#include > > +#endif > > > >  #include "archive.h" > > > > @@ -83,6 +87,9 @@ archive_read_open_filename(struct archiv > >        struct read_file_data *mine; > >        void *b; > >        int fd; > > +#ifdef __FreeBSD__ > > +       off_t mediasize = 0; > > +#endif > > > >        archive_clear_error(a); > >        if (filename == NULL || filename[0] == '\0') { > > @@ -143,6 +150,27 @@ archive_read_open_filename(struct archiv > >                 */ > >                mine->can_skip = 1; > >        } > > +#ifdef __FreeBSD__ > > +       /* > > +        * on FreeBSD if a device supports the DIOCGMEDIASIZE ioctl > > +        * it is a disk-like device and should be seekable. > > +        */ > > +       else if (S_ISCHR(st.st_mode) && > > +           ioctl(fd, DIOCGMEDIASIZE, &mediasize) == 0 && mediasize > 0) { > > +               mine->can_skip = 1; > > +       } > > +#endif > > +#ifdef __linux__ > > Is there any reason why a) you wouldn't check for other BSDs? b) you > aren't doing something of the flavor... > > #if defined(__FreeBSD__) > > #elif defined(__linux__) > > #endif Okok, here's a new version of the patch that now also knows about other BSDs and switched to #elif: (only NetBSD actually tested tho since I had a VM lying around, more testing welcome!) (Also if anyone wants to switch this to autoconf-style HAVE_FOO handling they are welcome too... :) Enjoy, Juergen Index: src/lib/libarchive/archive_read_open_filename.c =================================================================== RCS file: /home/scvs/src/lib/libarchive/archive_read_open_filename.c,v retrieving revision 1.25.2.1 diff -u -p -u -p -r1.25.2.1 archive_read_open_filename.c --- src/lib/libarchive/archive_read_open_filename.c 3 Aug 2009 08:13:06 -0000 1.25.2.1 +++ src/lib/libarchive/archive_read_open_filename.c 19 Feb 2010 18:18:24 -0000 @@ -44,6 +44,17 @@ __FBSDID("$FreeBSD: src/lib/libarchive/a #ifdef HAVE_UNISTD_H #include #endif +#if defined(__FreeBSD__) || defined(__FreeBSD_kernel__) +#include +#include +#elif defined(__NetBSD__) || defined(__OpenBSD__) +#include +#include +#include +#elif defined(__DragonFly__) +#include +#include +#endif #include "archive.h" @@ -83,6 +94,13 @@ archive_read_open_filename(struct archiv struct read_file_data *mine; void *b; int fd; +#if defined(__FreeBSD__) || defined(__FreeBSD_kernel__) + off_t mediasize = 0; +#elif defined(__NetBSD__) || defined(__OpenBSD__) + struct disklabel dl; +#elif defined(__DragonFly__) + struct partinfo pi; +#endif archive_clear_error(a); if (filename == NULL || filename[0] == '\0') { @@ -143,6 +161,45 @@ archive_read_open_filename(struct archiv */ mine->can_skip = 1; } +#if defined(__FreeBSD__) || defined(__FreeBSD_kernel__) + /* + * on FreeBSD if a device supports the DIOCGMEDIASIZE ioctl + * it is a disk-like device and should be seekable. + */ + else if (S_ISCHR(st.st_mode) && + ioctl(fd, DIOCGMEDIASIZE, &mediasize) == 0 && mediasize > 0) { + mine->can_skip = 1; + } +#elif defined(__NetBSD__) || defined(__OpenBSD__) + /* + * on Net/OpenBSD if a device supports the DIOCGDINFO ioctl + * it is a disk-like device and should be seekable. + */ + else if ((S_ISCHR(st.st_mode) || S_ISBLK(st.st_mode)) && + ioctl(fd, DIOCGDINFO, &dl) == 0 && + dl.d_partitions[DISKPART(st.st_rdev)].p_size > 0) { + mine->can_skip = 1; + } +#elif defined(__DragonFly__) + /* + * on DragonFly BSD if a device supports the DIOCGPART ioctl + * it is a disk-like device and should be seekable. + */ + else if (S_ISCHR(st.st_mode) && + ioctl(fd, DIOCGPART, &pi) == 0 && pi.media_size > 0) { + mine->can_skip = 1; + } +#elif defined(__linux__) + /* + * on Linux just check whether its a block device and that + * lseek works. (Tapes are character devices there.) + */ + else if (S_ISBLK(st.st_mode) && + lseek(fd, 0, SEEK_CUR) == 0 && lseek(fd, 0, SEEK_SET) == 0 && + lseek(fd, 0, SEEK_END) > 0 && lseek(fd, 0, SEEK_SET) == 0) { + mine->can_skip = 1; + } +#endif return (archive_read_open2(a, mine, NULL, file_read, file_skip, file_close)); } From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 19 18:39:36 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0A10A1065670 for ; Fri, 19 Feb 2010 18:39:36 +0000 (UTC) (envelope-from joerg@britannica.bec.de) Received: from www.sonnenberger.org (www.sonnenberger.org [92.79.50.50]) by mx1.freebsd.org (Postfix) with ESMTP id BE6878FC0C for ; Fri, 19 Feb 2010 18:39:35 +0000 (UTC) Received: from britannica.bec.de (www.sonnenberger.org [192.168.1.10]) by www.sonnenberger.org (Postfix) with ESMTP id 1F9E066790 for ; Fri, 19 Feb 2010 19:39:33 +0100 (CET) Received: by britannica.bec.de (Postfix, from userid 1000) id F1E1315C6C; Fri, 19 Feb 2010 19:39:01 +0100 (CET) Date: Fri, 19 Feb 2010 19:39:01 +0100 From: Joerg Sonnenberger To: freebsd-hackers@freebsd.org Message-ID: <20100219183901.GA11388@britannica.bec.de> Mail-Followup-To: freebsd-hackers@freebsd.org References: <20100217215940.GA19713@triton8.kn-bremen.de> <4B7CE066.4030403@freebsd.org> <20100218183459.GA65508@triton8.kn-bremen.de> <20100218191347.GA4040@britannica.bec.de> <4B7E1BA5.7050209@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B7E1BA5.7050209@freebsd.org> User-Agent: Mutt/1.5.20 (2009-06-14) Subject: Re: "tar tfv /dev/cd0" speedup patch X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2010 18:39:36 -0000 On Thu, Feb 18, 2010 at 09:03:33PM -0800, Tim Kientzle wrote: > Joerg Sonnenberger wrote: > >On Thu, Feb 18, 2010 at 07:34:59PM +0100, Juergen Lock wrote: > >> Ok here is a new version of the patch with these things fixed and the > >>Linux case added: (Linux case not tested yet, and yes I did this on > >>stable/8.) > > > >Why the check at all? Shouldn't devices that don't allow seek fail that? > >E.g. for devices, just try to seek ahead and fallback to normal reading? > > That was the initial implementation in libarchive, but > I had a number of reports of that not working for > tape drives. I recently dug out and connected an > old DDS drive I had in the closet, so I should > probably try again and see if I misunderstood > something along the way. I can think of some complications that for block devices the seek is to be a multiple of the block size (just like reads), but that's all I can think of in terms of portable problems. Joerg From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 19 20:48:13 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 142931065692; Fri, 19 Feb 2010 20:48:13 +0000 (UTC) (envelope-from ray@ddteam.net) Received: from mail-fx0-f223.google.com (mail-fx0-f223.google.com [209.85.220.223]) by mx1.freebsd.org (Postfix) with ESMTP id 78E3C8FC08; Fri, 19 Feb 2010 20:48:12 +0000 (UTC) Received: by fxm23 with SMTP id 23so538012fxm.3 for ; Fri, 19 Feb 2010 12:48:11 -0800 (PST) Received: by 10.87.51.13 with SMTP id d13mr7435050fgk.11.1266611173606; Fri, 19 Feb 2010 12:26:13 -0800 (PST) Received: from localhost (251-190-94-178.pool.ukrtel.net [178.94.190.251]) by mx.google.com with ESMTPS id 16sm273895fxm.3.2010.02.19.12.26.11 (version=SSLv3 cipher=RC4-MD5); Fri, 19 Feb 2010 12:26:12 -0800 (PST) Date: Fri, 19 Feb 2010 22:26:04 +0200 From: Alex RAY To: Ivan Voras Message-Id: <20100219222604.44e50248.ray@ddteam.net> In-Reply-To: References: <20100219163644.da89e882.ray__27111.9062825621$1266591431$gmane$org@dlink.ua> Organization: DDTeam.net X-Mailer: Sylpheed 2.7.1 (GTK+ 2.16.6; i386-portbld-freebsd8.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Fri, 19 Feb 2010 21:10:23 +0000 Cc: freebsd-hackers@freebsd.org Subject: Re: GEOM_ULZMA X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2010 20:48:13 -0000 On Fri, 19 Feb 2010 16:36:40 +0100 Ivan Voras wrote: > On 02/19/10 15:36, Alexandr Rybalko wrote: > > Hi, > > I wrote a module GEOM_ULZMA (such as GEOM_UZIP, but compression with lzma), in connection with this is an issue best left lzma > > One of the (relatively) unexpected problems with such block-level > compression is its efficienty - since you need to compress individual > blocks to be seekable (if you are not designing a special algorithm), > the compression ratios tend to not increase in a big way as the > algorithm improves. It would be interesting to see a comparison of sizes > with geom_uzip if you have them. Simple answer: -rwxr-xr-x 1 ray wheel 16299520 Feb 19 20:59 fsimg.iso.64k.lzma -rwxr-xr-x 1 ray wheel 20882432 Feb 19 20:57 fsimg.iso.64k.uzip -rw-r--r-- 1 ray wheel 65472512 Feb 19 20:56 fsimg.iso How I have, not done, but worked rootfs for embedded router (D-Link DIR-320) 2.4MB of size. > > > code in the file "geom_ulzma.c" or store lzma library separately. If > separately, then where better? > > The code organization depends on what you want to do with it and how you > want to update the code in the future, if your lzma library is third party. LZMA made by Igor Pavlov, and since 4.62 it licensed under Public Domain. So we can use it, if we need. > > If you never intend to update the lzma code then I guess it's fine to > embed it in a big .c file. For a port, it doesn't matter much since it > is your own thing. There are stricter rules on maintainability and style > if you want it in the base system. So, I think right place for lzma library under sys/contrib directory, if I "promise" maintain it? I also have plane, to use this code for making self-extracting lzma compressed kernel. > > > Maybe in future make lzma and gzip library kernel interface for embedded? > > Then in one instance of code, userland can use compression via kernel. > > This never happens now (I mean userland calling into the kernel for > compression) so you will need to explain the benefits first (kernel code > is not magically faster than userland - it runs on the same CPU). :) No, I don`t think about "magically faster", now I near to release FreeBSD firmware for D-Link DIR-320 router which have only 4MB of flash memory. Maybe in next time I try to make it for some router with only 2MB of flash. In that way, I need not copy of any code. In ideal embedded systems, if we have code, we must use it everywhere we need it. > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" -- Alex RAY From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 19 23:07:32 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5C3E2106566B; Fri, 19 Feb 2010 23:07:32 +0000 (UTC) (envelope-from yanefbsd@gmail.com) Received: from mail-pw0-f54.google.com (mail-pw0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 283E28FC12; Fri, 19 Feb 2010 23:07:31 +0000 (UTC) Received: by pwj7 with SMTP id 7so707741pwj.13 for ; Fri, 19 Feb 2010 15:07:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=fYmk9eTwPyrazxCaBT2LE3Eft/vw0FtUXgE8XQC26Go=; b=iF7pDgipqFsFA9zFDOuMhZZTsw4eJw0H50xPItIKDdFG8TfudhniOlN4Am4XMx2Ex8 zxmvimcBXCU8uyehukD9QAavqqWdjH6XWPjj2+kpfnyeKQRdv2GV7g3S0Cynoru6kEyM WGm7YSaGae/JKMCzypiUUupkg1vefbAYcFWys= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=mOmKWkHzZsmVLR7Ut85QQKD0vm2R2EZf082QEkD4YgmhnHG1b1PqMJC4YkSNTigEsE FAY8Tx6HO9ES9VJuAKFzT65xNBTFw5+MjjnKZ4xTzOCNsbYUfyTIlESNs07MJ6rrHgOs Q7fZk2p2LmqkSOqfT/g3jkBKlp2s+nqdR/7KA= MIME-Version: 1.0 Received: by 10.142.56.10 with SMTP id e10mr693554wfa.309.1266620851577; Fri, 19 Feb 2010 15:07:31 -0800 (PST) In-Reply-To: <20100219225120.GA32735@kiwi.sharlinx.com> References: <7d6fde3d1002181413p214389d3nd73798e597b9e26f@mail.gmail.com> <20100219225120.GA32735@kiwi.sharlinx.com> Date: Fri, 19 Feb 2010 15:07:31 -0800 Message-ID: <7d6fde3d1002191507g5ff5260fk3a7f1310561f8b8d@mail.gmail.com> From: Garrett Cooper To: jhell , Garrett Cooper , Eitan Adler , hackers@freebsd.org, linimon@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Subject: Re: junior tasks wiki page - please add stuff X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2010 23:07:32 -0000 On Fri, Feb 19, 2010 at 2:51 PM, R. Tyler Ballance wr= ote: > > On Fri, 19 Feb 2010, jhell wrote: > >> It would be awesome if for some way the PR database could be tied into >> this so items in the database could be promoted as junior tasks. =A0Addi= ng >> to that, an option in the PR web for someone to be able to vote on certa= in >> PR's that should be promoted and ultimately giving a little more free ti= me >> to the PR admin by not having to visually inspect every PR that has not >> been categorized for its worth as a junior project, and putting the >> community as a whole to work in the voting. > > Agreed, other projects I've seen that do something along this lines use t= ags in > bugzilla or $OTHER_ISSUE_TRACKER to denote that a task has is suitable fo= r > tenderfoots. > > I thought the PR database had tags, so perhaps it just needs some folks > spending some time tagging PRs? Yes, which gets back to the [mostly] interdependent relationship dealing with Mark's $TIME and PR database tagging. Cheers, -Garrett From owner-freebsd-hackers@FreeBSD.ORG Fri Feb 19 23:23:47 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A31101065696 for ; Fri, 19 Feb 2010 23:23:47 +0000 (UTC) (envelope-from tyler@monkeypox.org) Received: from starfish.geekisp.com (mail.geekisp.com [216.168.135.169]) by mx1.freebsd.org (Postfix) with ESMTP id 479F68FC21 for ; Fri, 19 Feb 2010 23:23:47 +0000 (UTC) Received: (qmail 786 invoked by uid 1003); 19 Feb 2010 22:51:28 -0000 Received: from localhost (HELO kiwi.sharlinx.com) (tyler@monkeypox.org@127.0.0.1) by mail.geekisp.com with SMTP; 19 Feb 2010 22:51:27 -0000 Date: Fri, 19 Feb 2010 14:51:20 -0800 From: "R. Tyler Ballance" To: jhell Message-ID: <20100219225120.GA32735@kiwi.sharlinx.com> Mail-Followup-To: jhell , Garrett Cooper , Eitan Adler , hackers@freebsd.org, linimon@freebsd.org References: <7d6fde3d1002181413p214389d3nd73798e597b9e26f@mail.gmail.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ZGiS0Q5IWpPtfppv" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) Cc: Garrett Cooper , Eitan Adler , hackers@freebsd.org, linimon@freebsd.org Subject: Re: junior tasks wiki page - please add stuff X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2010 23:23:47 -0000 --ZGiS0Q5IWpPtfppv Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, 19 Feb 2010, jhell wrote: > It would be awesome if for some way the PR database could be tied into=20 > this so items in the database could be promoted as junior tasks. Adding= =20 > to that, an option in the PR web for someone to be able to vote on certai= n=20 > PR's that should be promoted and ultimately giving a little more free tim= e=20 > to the PR admin by not having to visually inspect every PR that has not= =20 > been categorized for its worth as a junior project, and putting the=20 > community as a whole to work in the voting. Agreed, other projects I've seen that do something along this lines use tag= s in bugzilla or $OTHER_ISSUE_TRACKER to denote that a task has is suitable for tenderfoots. I thought the PR database had tags, so perhaps it just needs some folks spending some time tagging PRs? Cheers, -R. Tyler Ballance -------------------------------------- Jabber: rtyler@jabber.org GitHub: http://github.com/rtyler Twitter: http://twitter.com/agentdero Blog: http://unethicalblogger.com --ZGiS0Q5IWpPtfppv Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.12 (GNU/Linux) iEYEARECAAYFAkt/FecACgkQFCbH3D9R4W94fACfY23HeKoLjOODxG3uU/ib2GVC zqcAn2dL1iIRPH7yrylqwcrlIFaawhSV =GDmG -----END PGP SIGNATURE----- --ZGiS0Q5IWpPtfppv-- From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 20 05:19:31 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D2EF7106566B for ; Sat, 20 Feb 2010 05:19:31 +0000 (UTC) (envelope-from kientzle@freebsd.org) Received: from monday.kientzle.com (kientzle.com [66.166.149.50]) by mx1.freebsd.org (Postfix) with ESMTP id 8DD5A8FC08 for ; Sat, 20 Feb 2010 05:19:31 +0000 (UTC) Received: (from root@localhost) by monday.kientzle.com (8.14.3/8.14.3) id o1K5JcXZ098490; Sat, 20 Feb 2010 05:19:38 GMT (envelope-from kientzle@freebsd.org) Received: from dark.x.kientzle.com (fw2.kientzle.com [10.123.1.2]) by kientzle.com with SMTP id 57cz5sipheprmhd9dxfqqwv7in; Sat, 20 Feb 2010 05:19:38 +0000 (UTC) (envelope-from kientzle@freebsd.org) Message-ID: <4B7F711E.6040402@freebsd.org> Date: Fri, 19 Feb 2010 21:20:30 -0800 From: Tim Kientzle User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.8.1.21) Gecko/20090601 SeaMonkey/1.1.16 MIME-Version: 1.0 To: Juergen Lock References: <20100217215940.GA19713@triton8.kn-bremen.de> <4B7CE066.4030403@freebsd.org> <20100218183459.GA65508@triton8.kn-bremen.de> <7d6fde3d1002182212k3bdc866ckdc7c5f380b40f7d8@mail.gmail.com> <20100219181247.GA35702@triton8.kn-bremen.de> In-Reply-To: <20100219181247.GA35702@triton8.kn-bremen.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Garrett Cooper , freebsd-hackers@freebsd.org Subject: Re: "tar tfv /dev/cd0" speedup patch X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Feb 2010 05:19:32 -0000 Juergen, I was looking at your Linux code here and thought the technique of trying lseek(SEEK_END) might work. Unfortunately, it doesn't: lseek(fd, 0, SEEK_END) gives zero for both /dev/sa0 (a tape drive) and /dev/cd0 (an optical drive). Are you sure it works on Linux? Tim P.S. Here's the program I used to test. Running it against regular files and various device nodes is quite illuminating. #include #include #include #include int main(int argc, char **argv) { int fd; if (argv[1] == NULL) { fprintf(stderr, "Need to specify a pathname.\n"); exit(1); } fd = open(argv[1], O_RDWR); printf("fd=%d\n", fd); printf("lseek(fd, 0, SEEK_SET)=%d\n", (int)lseek(fd, 0, SEEK_SET)); printf("lseek(fd, 10240, SEEK_SET)=%d\n", (int)lseek(fd, 10240, SEEK_SET)); printf("lseek(fd, 10240, SEEK_CUR)=%d\n", (int)lseek(fd, 10240, SEEK_CUR)); printf("lseek(fd, 0, SEEK_END)=%d\n", (int)lseek(fd, 0, SEEK_END)); printf("lseek(fd, 0, SEEK_SET)=%d\n", (int)lseek(fd, 0, SEEK_SET)); return (0); } Juergen Lock wrote: > On Thu, Feb 18, 2010 at 10:12:24PM -0800, Garrett Cooper wrote: >> On Thu, Feb 18, 2010 at 10:34 AM, Juergen Lock wrote: >>> On Wed, Feb 17, 2010 at 10:38:30PM -0800, Tim Kientzle wrote: >>>> Juergen Lock wrote: >>>>> ... since bsdtar/libarchive know iso9660 I just did the command in the >>>>> Subject. It worked, but it was sloow... :( Apparently it read all of >>>>> the disc without seeking. The following patch fixes this, is something >>>>> like this desired? If yes I could look how to do the same for Linux, >>>> Juergen, >>>> >>>> This is great! If you can figure out how to get this >>>> right, I would really appreciate it. If you have a >>>> tape drive handy, definitely test with that. My first >>>> attempts here actually broke reading from tape drives, >>>> which is why the current code is so conservative. >>>> >>> Hmm I can't test on a tape atm but if I look at the kernel, >>> http://fxr.watson.org/fxr/ident?i=DIOCGMEDIASIZE >>> DIOCGMEDIASIZE is only handled for geom, xen block devices, old CD drives >>> and pc98 floppies, and I'm pretty sure tapes don't use geom. :) >>> >>>> Minor style comments: >>>>> else if (S_ISCHR(st.st_mode) && >>>>> !ioctl(fd, DIOCGMEDIASIZE, &mediasize) && mediasize) { >>>> Please be explicit: S_ISCHR() && ioctl() == 0 && mediasize > 0 >>>> >>>>> archive_read_extract_set_skip_file(a, st.st_dev, st.st_ino); >>>> extract_skip_file isn't needed here; we don't read the >>>> contents of device nodes. >>>> >>>> Let me know as soon as you have something you're confident of. >>> Ok here is a new version of the patch with these things fixed and the >>> Linux case added: (Linux case not tested yet, and yes I did this on >>> stable/8.) >>> >>> Index: src/lib/libarchive/archive_read_open_filename.c >>> =================================================================== >>> RCS file: /home/scvs/src/lib/libarchive/archive_read_open_filename.c,v >>> retrieving revision 1.25.2.1 >>> diff -u -p -r1.25.2.1 archive_read_open_filename.c >>> --- src/lib/libarchive/archive_read_open_filename.c 3 Aug 2009 08:13:06 -0000 1.25.2.1 >>> +++ src/lib/libarchive/archive_read_open_filename.c 18 Feb 2010 18:14:16 -0000 >>> @@ -44,6 +44,10 @@ __FBSDID("$FreeBSD: src/lib/libarchive/a >>> #ifdef HAVE_UNISTD_H >>> #include >>> #endif >>> +#ifdef __FreeBSD__ >>> +#include >>> +#include >>> +#endif >>> >>> #include "archive.h" >>> >>> @@ -83,6 +87,9 @@ archive_read_open_filename(struct archiv >>> struct read_file_data *mine; >>> void *b; >>> int fd; >>> +#ifdef __FreeBSD__ >>> + off_t mediasize = 0; >>> +#endif >>> >>> archive_clear_error(a); >>> if (filename == NULL || filename[0] == '\0') { >>> @@ -143,6 +150,27 @@ archive_read_open_filename(struct archiv >>> */ >>> mine->can_skip = 1; >>> } >>> +#ifdef __FreeBSD__ >>> + /* >>> + * on FreeBSD if a device supports the DIOCGMEDIASIZE ioctl >>> + * it is a disk-like device and should be seekable. >>> + */ >>> + else if (S_ISCHR(st.st_mode) && >>> + ioctl(fd, DIOCGMEDIASIZE, &mediasize) == 0 && mediasize > 0) { >>> + mine->can_skip = 1; >>> + } >>> +#endif >>> +#ifdef __linux__ >> Is there any reason why a) you wouldn't check for other BSDs? b) you >> aren't doing something of the flavor... >> >> #if defined(__FreeBSD__) >> >> #elif defined(__linux__) >> >> #endif > > Okok, here's a new version of the patch that now also knows about > other BSDs and switched to #elif: (only NetBSD actually tested tho > since I had a VM lying around, more testing welcome!) > > (Also if anyone wants to switch this to autoconf-style HAVE_FOO > handling they are welcome too... :) > > Enjoy, > Juergen > > Index: src/lib/libarchive/archive_read_open_filename.c > =================================================================== > RCS file: /home/scvs/src/lib/libarchive/archive_read_open_filename.c,v > retrieving revision 1.25.2.1 > diff -u -p -u -p -r1.25.2.1 archive_read_open_filename.c > --- src/lib/libarchive/archive_read_open_filename.c 3 Aug 2009 08:13:06 -0000 1.25.2.1 > +++ src/lib/libarchive/archive_read_open_filename.c 19 Feb 2010 18:18:24 -0000 > @@ -44,6 +44,17 @@ __FBSDID("$FreeBSD: src/lib/libarchive/a > #ifdef HAVE_UNISTD_H > #include > #endif > +#if defined(__FreeBSD__) || defined(__FreeBSD_kernel__) > +#include > +#include > +#elif defined(__NetBSD__) || defined(__OpenBSD__) > +#include > +#include > +#include > +#elif defined(__DragonFly__) > +#include > +#include > +#endif > > #include "archive.h" > > @@ -83,6 +94,13 @@ archive_read_open_filename(struct archiv > struct read_file_data *mine; > void *b; > int fd; > +#if defined(__FreeBSD__) || defined(__FreeBSD_kernel__) > + off_t mediasize = 0; > +#elif defined(__NetBSD__) || defined(__OpenBSD__) > + struct disklabel dl; > +#elif defined(__DragonFly__) > + struct partinfo pi; > +#endif > > archive_clear_error(a); > if (filename == NULL || filename[0] == '\0') { > @@ -143,6 +161,45 @@ archive_read_open_filename(struct archiv > */ > mine->can_skip = 1; > } > +#if defined(__FreeBSD__) || defined(__FreeBSD_kernel__) > + /* > + * on FreeBSD if a device supports the DIOCGMEDIASIZE ioctl > + * it is a disk-like device and should be seekable. > + */ > + else if (S_ISCHR(st.st_mode) && > + ioctl(fd, DIOCGMEDIASIZE, &mediasize) == 0 && mediasize > 0) { > + mine->can_skip = 1; > + } > +#elif defined(__NetBSD__) || defined(__OpenBSD__) > + /* > + * on Net/OpenBSD if a device supports the DIOCGDINFO ioctl > + * it is a disk-like device and should be seekable. > + */ > + else if ((S_ISCHR(st.st_mode) || S_ISBLK(st.st_mode)) && > + ioctl(fd, DIOCGDINFO, &dl) == 0 && > + dl.d_partitions[DISKPART(st.st_rdev)].p_size > 0) { > + mine->can_skip = 1; > + } > +#elif defined(__DragonFly__) > + /* > + * on DragonFly BSD if a device supports the DIOCGPART ioctl > + * it is a disk-like device and should be seekable. > + */ > + else if (S_ISCHR(st.st_mode) && > + ioctl(fd, DIOCGPART, &pi) == 0 && pi.media_size > 0) { > + mine->can_skip = 1; > + } > +#elif defined(__linux__) > + /* > + * on Linux just check whether its a block device and that > + * lseek works. (Tapes are character devices there.) > + */ > + else if (S_ISBLK(st.st_mode) && > + lseek(fd, 0, SEEK_CUR) == 0 && lseek(fd, 0, SEEK_SET) == 0 && > + lseek(fd, 0, SEEK_END) > 0 && lseek(fd, 0, SEEK_SET) == 0) { > + mine->can_skip = 1; > + } > +#endif > return (archive_read_open2(a, mine, > NULL, file_read, file_skip, file_close)); > } > > From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 20 06:01:05 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from [127.0.0.1] (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by hub.freebsd.org (Postfix) with ESMTP id 260E9106566C; Sat, 20 Feb 2010 06:01:05 +0000 (UTC) (envelope-from jkim@FreeBSD.org) From: Jung-uk Kim To: freebsd-hackers@freebsd.org Date: Sat, 20 Feb 2010 01:00:40 -0500 User-Agent: KMail/1.6.2 References: <20100217215940.GA19713@triton8.kn-bremen.de> <20100219181247.GA35702@triton8.kn-bremen.de> <4B7F711E.6040402@freebsd.org> In-Reply-To: <4B7F711E.6040402@freebsd.org> MIME-Version: 1.0 Content-Disposition: inline Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201002200100.48161.jkim@FreeBSD.org> Cc: Garrett Cooper , Tim Kientzle , Juergen Lock Subject: Re: "tar tfv /dev/cd0" speedup patch X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Feb 2010 06:01:05 -0000 On Saturday 20 February 2010 12:20 am, Tim Kientzle wrote: > Juergen, > > I was looking at your Linux code here and thought > the technique of trying lseek(SEEK_END) might work. > Unfortunately, it doesn't: lseek(fd, 0, SEEK_END) gives > zero for both /dev/sa0 (a tape drive) and /dev/cd0 > (an optical drive). Are you sure it works on Linux? Can you please try ioctl(fd, BLKGETSIZE64, &some_uint64_var) or ioctl(fd, BLKGETSIZE, &some_u_long_var)? Jung-uk Kim From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 20 06:18:46 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8297410656A6; Sat, 20 Feb 2010 06:18:46 +0000 (UTC) (envelope-from kientzle@freebsd.org) Received: from monday.kientzle.com (kientzle.com [66.166.149.50]) by mx1.freebsd.org (Postfix) with ESMTP id 53C1C8FC33; Sat, 20 Feb 2010 06:18:45 +0000 (UTC) Received: (from root@localhost) by monday.kientzle.com (8.14.3/8.14.3) id o1K6IsZt098951; Sat, 20 Feb 2010 06:18:54 GMT (envelope-from kientzle@freebsd.org) Received: from dark.x.kientzle.com (fw2.kientzle.com [10.123.1.2]) by kientzle.com with SMTP id vshm7w9fge9ir2vsmez6t7t5g2; Sat, 20 Feb 2010 06:18:54 +0000 (UTC) (envelope-from kientzle@freebsd.org) Message-ID: <4B7F7F02.1000004@freebsd.org> Date: Fri, 19 Feb 2010 22:19:46 -0800 From: Tim Kientzle User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.8.1.21) Gecko/20090601 SeaMonkey/1.1.16 MIME-Version: 1.0 To: Jung-uk Kim References: <20100217215940.GA19713@triton8.kn-bremen.de> <20100219181247.GA35702@triton8.kn-bremen.de> <4B7F711E.6040402@freebsd.org> <201002200100.48161.jkim@FreeBSD.org> In-Reply-To: <201002200100.48161.jkim@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, Juergen Lock , Garrett Cooper Subject: Re: "tar tfv /dev/cd0" speedup patch X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Feb 2010 06:18:46 -0000 I should have been clearer; I tried this techniqe on FreeBSD; I've not tried it on Linux yet. (I don't have a Linux machine with a tape drive at the moment.) It doesn't work on FreeBSD; I was questioning whether anyone else had tested it on Linux. If Juergen's technique doesn't work, I'll try the BLKGETSIZE ioctl. Tim Jung-uk Kim wrote: > On Saturday 20 February 2010 12:20 am, Tim Kientzle wrote: >> Juergen, >> >> I was looking at your Linux code here and thought >> the technique of trying lseek(SEEK_END) might work. >> Unfortunately, it doesn't: lseek(fd, 0, SEEK_END) gives >> zero for both /dev/sa0 (a tape drive) and /dev/cd0 >> (an optical drive). Are you sure it works on Linux? > > Can you please try ioctl(fd, BLKGETSIZE64, &some_uint64_var) or > ioctl(fd, BLKGETSIZE, &some_u_long_var)? > > Jung-uk Kim > > From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 20 08:08:08 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CBCC2106566C for ; Sat, 20 Feb 2010 08:08:08 +0000 (UTC) (envelope-from scdbackup@gmx.net) Received: from mail.gmx.net (mail.gmx.net [213.165.64.20]) by mx1.freebsd.org (Postfix) with SMTP id 258F18FC08 for ; Sat, 20 Feb 2010 08:08:07 +0000 (UTC) Received: (qmail invoked by alias); 20 Feb 2010 07:41:25 -0000 Received: from 165.126.46.212.adsl.ncore.de (HELO 192.168.2.69) [212.46.126.165] by mail.gmx.net (mp066) with SMTP; 20 Feb 2010 08:41:25 +0100 X-Authenticated: #2145628 X-Provags-ID: V01U2FsdGVkX1+aD2oXCKiak/3p+5zx6lJQCBxq7uWi2PYwdDEFRd kN6I/VgCKGFyqN Date: Sat, 20 Feb 2010 08:43:27 +0100 From: "Thomas Schmitt" To: freebsd-hackers@freebsd.org Message-Id: <107129897910301@192.168.2.69> X-Y-GMX-Trusted: 0 X-FuHaFi: 0.53000000000000003 Subject: -reply:<4B7F7F02.1000004@freebsd.org> X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Feb 2010 08:08:08 -0000 Hi > I was looking at your Linux code here and thought > the technique of trying lseek(SEEK_END) might work. Linux 2.6.18, /dev/hda and /dev/sr0, with a CD-RW written by write type TAO lseek(fd, 0, SEEK_END)= 636499968 lseek(fd, -1, SEEK_END)= 636499967 lseek(fd, -300k, SEEK_END)= 636192768 and with a CD-ROM media: lseek(fd, 0, SEEK_END)= 618311680 lseek(fd, -1, SEEK_END)= 618311679 lseek(fd, -300k, SEEK_END)= 618004480 (No tape drive at hand. Sorry.) Nevertheless: SEEK_END brings you to the first unwritten address of the file object. With a read-only object, this address is not necessarily defined. Further: If you test a CD that was written by write type TAO, then the last two blocks of the track are no readable data blocks. It depends on the implementation of lseek() whether it will allow you to address bytes in those two blocks. (Read will fail, of course. On Linux it even fails shortly before the end because of the old TAO read-ahead bug.) So on CD i would test rather lseek() to an address that is several hundred kB before the end. 300 kB is the traditional fear-padding of CD burn programs on Linux. (On Linux i decide by stat(2). Considered as seekable are S_IFREG and S_IFBLK. Actually tested only with optical drives and USB sticks. To my knowledge ioctl(BLKGETSIZE) works only on S_IFBLK.) Have a nice day :) Thomas From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 20 11:33:57 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E526F1065670 for ; Sat, 20 Feb 2010 11:33:57 +0000 (UTC) (envelope-from tyler@monkeypox.org) Received: from starfish.geekisp.com (mail.geekisp.com [216.168.135.169]) by mx1.freebsd.org (Postfix) with ESMTP id 753248FC16 for ; Sat, 20 Feb 2010 11:33:57 +0000 (UTC) Received: (qmail 12992 invoked by uid 1003); 20 Feb 2010 11:33:56 -0000 Received: from localhost (HELO kiwi.sharlinx.com) (tyler@monkeypox.org@127.0.0.1) by mail.geekisp.com with SMTP; 20 Feb 2010 11:33:55 -0000 Date: Sat, 20 Feb 2010 03:33:49 -0800 From: "R. Tyler Ballance" To: Peter Steele Message-ID: <20100220113349.GA22800@kiwi.sharlinx.com> Mail-Followup-To: Peter Steele , "freebsd-hackers@freebsd.org" References: <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D5C73@MBX03.exg5.exghost.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="LZvS9be/3tNcYl/X" Content-Disposition: inline In-Reply-To: <7B9397B189EB6E46A5EE7B4C8A4BB7CB385D5C73@MBX03.exg5.exghost.com> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: "freebsd-hackers@freebsd.org" Subject: Re: ntpd hangs under FBSD 8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Feb 2010 11:33:58 -0000 --LZvS9be/3tNcYl/X Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Fri, 19 Feb 2010, Peter Steele wrote: > I posted this originally on the -questions list but did not make any headway. We have an application where the user can change the date/time via a GUI. One of the options the user has is to specify that the time is to be synced using ntp. Our coding worked fine under BSD 7 but since we've moved to BSD 8 we've encountered a problem where the command that we initiate from the GUI: Just out of curiosity, can you attach to the process via gdb and get a backtrace? This smells like a locked pthread_join I hit in my own code a few weeks ago Cheers, -R. Tyler Ballance -------------------------------------- Jabber: rtyler@jabber.org GitHub: http://github.com/rtyler Twitter: http://twitter.com/agentdero Blog: http://unethicalblogger.com --LZvS9be/3tNcYl/X Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.12 (GNU/Linux) iEYEARECAAYFAkt/yJ0ACgkQFCbH3D9R4W+9QgCePhV+lBCU/R8zYZuPMRKsYlEB HOkAn2oQV2tMwYIrnZFSqtFEQQSN9QqY =1k4W -----END PGP SIGNATURE----- --LZvS9be/3tNcYl/X-- From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 20 14:36:05 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 80A701065670; Sat, 20 Feb 2010 14:36:05 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-iw0-f185.google.com (mail-iw0-f185.google.com [209.85.223.185]) by mx1.freebsd.org (Postfix) with ESMTP id 38D528FC08; Sat, 20 Feb 2010 14:36:03 +0000 (UTC) Received: by iwn15 with SMTP id 15so1193454iwn.7 for ; Sat, 20 Feb 2010 06:36:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type; bh=nbcnz53aBB+wO542EWv3MCoZBSgnmGpiSGAWnNe639A=; b=AHJtuG2laIKrgRf48twmuDQWN//zUtBk6ZNQWC07WTC3ArAr6VzSdlEZ/sQNmB8hjn 5wyQNS6lJnImUwP7eX+02RpghINoFsTrDlQSQt7Dvn9hg1W+VOJIrBaHPxtTciR7Vbvo GUz+lHvxnYcEkUbnEzEata7mGOnGbvZCifE9g= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=fsM4kcmYPmrmARCWf6xoagCG6NMy4IB1y+YJ4QAqrjmg6GhFOJJvbsFCoWrsQYXqwU OI5ECDLkyy043Xe5CTkJNzNoBvngRR52LaFrbjU1x/3yK9CEf3WM4F/JCL1m/JSweJ+a hO8cxVRu+oksJjU6G0Ad0wTpl+DlL3nXZQ1P0= MIME-Version: 1.0 Sender: adrian.chadd@gmail.com Received: by 10.231.174.147 with SMTP id t19mr968992ibz.74.1266676562732; Sat, 20 Feb 2010 06:36:02 -0800 (PST) In-Reply-To: <20100219222604.44e50248.ray@ddteam.net> References: <20100219163644.da89e882.ray__27111.9062825621$1266591431$gmane$org@dlink.ua> <20100219222604.44e50248.ray@ddteam.net> Date: Sat, 20 Feb 2010 22:36:02 +0800 X-Google-Sender-Auth: da9ba6cd05441f36 Message-ID: From: Adrian Chadd To: Alex RAY Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org, Ivan Voras Subject: Re: GEOM_ULZMA X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Feb 2010 14:36:05 -0000 On 20 February 2010 04:26, Alex RAY wrote: > :) No, I don`t think about "magically faster", now I near to release FreeBSD firmware for D-Link DIR-320 router which have only 4MB of flash memory. Maybe in next time I try to make it for some router with only 2MB of flash. In that way, > I need not copy of any code. > In ideal embedded systems, if we have code, we must use it everywhere we need it. Interesting! The Redboot loader that I'm toying with on the ubiquiti hardware supports LZMA as well as GZIP for compressed kernel images; I may have to experiment with that and this module. Thanks! Adrian From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 20 10:22:09 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3E236106566B; Sat, 20 Feb 2010 10:22:08 +0000 (UTC) (envelope-from nox@jelal.kn-bremen.de) Received: from smtp.kn-bremen.de (gelbbaer.kn-bremen.de [78.46.108.116]) by mx1.freebsd.org (Postfix) with ESMTP id BDDC58FC08; Sat, 20 Feb 2010 10:22:07 +0000 (UTC) Received: by smtp.kn-bremen.de (Postfix, from userid 10) id 8D7FF1E00773; Sat, 20 Feb 2010 11:22:06 +0100 (CET) Received: from triton8.kn-bremen.de (noident@localhost [127.0.0.1]) by triton8.kn-bremen.de (8.14.3/8.14.3) with ESMTP id o1KAHOI4030977; Sat, 20 Feb 2010 11:17:24 +0100 (CET) (envelope-from nox@triton8.kn-bremen.de) Received: (from nox@localhost) by triton8.kn-bremen.de (8.14.3/8.14.3/Submit) id o1KAHOiY030976; Sat, 20 Feb 2010 11:17:24 +0100 (CET) (envelope-from nox) From: Juergen Lock Date: Sat, 20 Feb 2010 11:17:24 +0100 To: Tim Kientzle Message-ID: <20100220101724.GA26604@triton8.kn-bremen.de> References: <20100217215940.GA19713@triton8.kn-bremen.de> <4B7CE066.4030403@freebsd.org> <20100218183459.GA65508@triton8.kn-bremen.de> <7d6fde3d1002182212k3bdc866ckdc7c5f380b40f7d8@mail.gmail.com> <20100219181247.GA35702@triton8.kn-bremen.de> <4B7F711E.6040402@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B7F711E.6040402@freebsd.org> User-Agent: Mutt/1.5.20 (2009-06-14) X-Mailman-Approved-At: Sat, 20 Feb 2010 15:35:04 +0000 Cc: Garrett Cooper , Juergen Lock , freebsd-hackers@freebsd.org Subject: Re: "tar tfv /dev/cd0" speedup patch X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Feb 2010 10:22:09 -0000 On Fri, Feb 19, 2010 at 09:20:30PM -0800, Tim Kientzle wrote: > Juergen, Hi! > > I was looking at your Linux code here and thought > the technique of trying lseek(SEEK_END) might work. > Unfortunately, it doesn't: lseek(fd, 0, SEEK_END) gives > zero for both /dev/sa0 (a tape drive) and /dev/cd0 > (an optical drive). Are you sure it works on Linux? > Yeah that code is Linux-specific, I know it doesn't work on BSDs. :) Here's some output on Linux after changing O_RDWR to O_RDONLY: $ ./a.out /dev/sr0 fd=3 lseek(fd, 0, SEEK_SET)=0 lseek(fd, 10240, SEEK_SET)=10240 lseek(fd, 10240, SEEK_CUR)=20480 lseek(fd, 0, SEEK_END)=-2057306112 lseek(fd, 0, SEEK_SET)=0 $ ./a.out /dev/fd0 fd=3 lseek(fd, 0, SEEK_SET)=0 lseek(fd, 10240, SEEK_SET)=10240 lseek(fd, 10240, SEEK_CUR)=20480 lseek(fd, 0, SEEK_END)=1474560 lseek(fd, 0, SEEK_SET)=0 $ Ok /dev/sr0 was a dvd iso and you casted the lseek return value to int... (And this was on amd64, on i386 your version gets an overflow error for SEEK_END there too i.e. -1 because on Linux off_t defaults to be a long.) If I fix that I get: $ ./a.out /dev/sr0 fd=3 lseek(fd, 0, SEEK_SET)=0 lseek(fd, 10240, SEEK_SET)=10240 lseek(fd, 10240, SEEK_CUR)=20480 lseek(fd, 0, SEEK_END)=2237661184 lseek(fd, 0, SEEK_SET)=0 $ ...which matches the size of the iso. (and bsdtar with the patch also is much faster on /dev/sr0 there than without it, which was how I originally confirmed the patch is working. There are two 850 MB files on that iso...) I'll append my version of the test program below. Cheers, Juergen PS: Seeking on tape is a whole other can of worms, I don't even think lseek on Linux works as you might expect there... If you really want to implement this you'd have to try the MTFSR ioctl (at least that's what its usually called, look for `fsr' in the mt(1) manpage and source), but since that counts in blocks not bytes you'd have to know the tape's blocksize too (which can also be variable i.e. depend on how that particular tape was written, tho I think in case of a tar archive you can get away with just using the blocksize arg passed to bsdtar there.) And also I'm not sure how some drives may behave if you use lots of MTFSR ioctls with small block counts so maybe you should only use them when the amount of data to skip is big enough so that switching to `fast forward' is actually worth it. (and continue to skip over small amounts by doing regular reads.) -------snip----------- /* make sure we don't get 32 bit off_t */ #define _FILE_OFFSET_BITS 64 #include #include #include #include #include int main(int argc, char **argv) { int fd; if (argv[1] == NULL) { fprintf(stderr, "Need to specify a pathname.\n"); exit(1); } fd = open(argv[1], O_RDONLY); printf("fd=%d\n", fd); printf("lseek(fd, 0, SEEK_SET)=%jd\n", (intmax_t)lseek(fd, 0, SEEK_SET)); printf("lseek(fd, 10240, SEEK_SET)=%jd\n", (intmax_t)lseek(fd, 10240, SEEK_SET)); printf("lseek(fd, 10240, SEEK_CUR)=%jd\n", (intmax_t)lseek(fd, 10240, SEEK_CUR)); printf("lseek(fd, 0, SEEK_END)=%jd\n", (intmax_t)lseek(fd, 0, SEEK_END)); printf("lseek(fd, 0, SEEK_SET)=%jd\n", (intmax_t)lseek(fd, 0, SEEK_SET)); return (0); } From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 20 10:36:48 2010 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 59B2D1065692; Sat, 20 Feb 2010 10:36:48 +0000 (UTC) (envelope-from nox@jelal.kn-bremen.de) Received: from smtp.kn-bremen.de (gelbbaer.kn-bremen.de [78.46.108.116]) by mx1.freebsd.org (Postfix) with ESMTP id AEA4E8FC0C; Sat, 20 Feb 2010 10:36:47 +0000 (UTC) Received: by smtp.kn-bremen.de (Postfix, from userid 10) id CB2BC1E00773; Sat, 20 Feb 2010 11:36:46 +0100 (CET) Received: from triton8.kn-bremen.de (noident@localhost [127.0.0.1]) by triton8.kn-bremen.de (8.14.3/8.14.3) with ESMTP id o1KAXUwB035533; Sat, 20 Feb 2010 11:33:30 +0100 (CET) (envelope-from nox@triton8.kn-bremen.de) Received: (from nox@localhost) by triton8.kn-bremen.de (8.14.3/8.14.3/Submit) id o1KAXUdR035532; Sat, 20 Feb 2010 11:33:30 +0100 (CET) (envelope-from nox) From: Juergen Lock Date: Sat, 20 Feb 2010 11:33:29 +0100 To: Jung-uk Kim Message-ID: <20100220103329.GA35467@triton8.kn-bremen.de> References: <20100217215940.GA19713@triton8.kn-bremen.de> <20100219181247.GA35702@triton8.kn-bremen.de> <4B7F711E.6040402@freebsd.org> <201002200100.48161.jkim@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201002200100.48161.jkim@FreeBSD.org> User-Agent: Mutt/1.5.20 (2009-06-14) X-Mailman-Approved-At: Sat, 20 Feb 2010 15:35:20 +0000 Cc: freebsd-hackers@FreeBSD.org, Tim Kientzle , Juergen Lock , Garrett Cooper Subject: Re: "tar tfv /dev/cd0" speedup patch X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Feb 2010 10:36:48 -0000 On Sat, Feb 20, 2010 at 01:00:40AM -0500, Jung-uk Kim wrote: > On Saturday 20 February 2010 12:20 am, Tim Kientzle wrote: > > Juergen, > > > > I was looking at your Linux code here and thought > > the technique of trying lseek(SEEK_END) might work. > > Unfortunately, it doesn't: lseek(fd, 0, SEEK_END) gives > > zero for both /dev/sa0 (a tape drive) and /dev/cd0 > > (an optical drive). Are you sure it works on Linux? > > Can you please try ioctl(fd, BLKGETSIZE64, &some_uint64_var) or > ioctl(fd, BLKGETSIZE, &some_u_long_var)? Yeah I've stumbled across these ioctls in the meantime too, I was just not sure if all Linux versions currently in use already have the 64 bit version i.e. BLKGETSIZE64... (since we don't want the 32 bit version for the same reason we don't want 32 bit off_t. :) Cheers, Juergen From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 20 17:00:55 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7E21C1065679 for ; Sat, 20 Feb 2010 17:00:55 +0000 (UTC) (envelope-from mahan@mahan.org) Received: from ns.mahan.org (ns.mahan.org [67.116.10.138]) by mx1.freebsd.org (Postfix) with ESMTP id 47D3E8FC13 for ; Sat, 20 Feb 2010 17:00:54 +0000 (UTC) Received: from Gypsy.mahan.org (crowTrobot [67.116.10.140]) by ns.mahan.org (8.13.6/8.13.6) with ESMTP id o1KGdO71089805 for ; Sat, 20 Feb 2010 08:39:25 -0800 (PST) (envelope-from mahan@mahan.org) Message-ID: <4B800F60.60700@mahan.org> Date: Sat, 20 Feb 2010 08:35:44 -0800 From: Patrick Mahan User-Agent: Thunderbird 2.0.0.22 (X11/20090605) MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Building FreeBSD on a linux FC11 box. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Feb 2010 17:00:55 -0000 Hopefully, this is not too ignorant a question. But has anyone every built the FreeBSD sources, both kernel and apps, on a linux platform? I did a google on 'cross-compile freebsd (linux)' and found (mostly) discussions regarding building the linux apps for FreeBSD, but not for building a FreeBSD kernel and world. There was a brief discussion on the mailing list back in 2005, but the poster never reported if he had success or failure. Also, there were discussions about building freebsd amd64 on a freebsd i386. I have done lots of development in linux using cross-compilers (mips embedded systems, powerpc, etc). But not with FreeBSD. I know I'll need to use 'bsdmake' no gnu 'make'. The other worry is kernel-toolchain target. I guess I'll just dive in and swim around and see what happens. Thanks for any pointers, Patrick From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 20 18:15:24 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 732461065672; Sat, 20 Feb 2010 18:15:24 +0000 (UTC) (envelope-from ray@ddteam.net) Received: from mail-fx0-f223.google.com (mail-fx0-f223.google.com [209.85.220.223]) by mx1.freebsd.org (Postfix) with ESMTP id D688A8FC1A; Sat, 20 Feb 2010 18:15:23 +0000 (UTC) Received: by fxm23 with SMTP id 23so1133481fxm.3 for ; Sat, 20 Feb 2010 10:15:22 -0800 (PST) Received: by 10.87.63.20 with SMTP id q20mr654112fgk.27.1266689722428; Sat, 20 Feb 2010 10:15:22 -0800 (PST) Received: from localhost (119-96-94-178.pool.ukrtel.net [178.94.96.119]) by mx.google.com with ESMTPS id 16sm648732fxm.15.2010.02.20.10.15.20 (version=SSLv3 cipher=RC4-MD5); Sat, 20 Feb 2010 10:15:21 -0800 (PST) Date: Sat, 20 Feb 2010 20:15:10 +0200 From: Alex RAY To: Adrian Chadd Message-Id: <20100220201510.720c62fd.ray@ddteam.net> In-Reply-To: References: <20100219163644.da89e882.ray__27111.9062825621$1266591431$gmane$org@dlink.ua> <20100219222604.44e50248.ray@ddteam.net> Organization: DDTeam.net X-Mailer: Sylpheed 2.7.1 (GTK+ 2.16.6; i386-portbld-freebsd8.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, Ivan Voras Subject: Re: GEOM_ULZMA X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Feb 2010 18:15:24 -0000 Hi, On Sat, 20 Feb 2010 22:36:02 +0800 Adrian Chadd wrote: > On 20 February 2010 04:26, Alex RAY wrote: > > > :) No, I don`t think about "magically faster", now I near to release FreeBSD firmware for D-Link DIR-320 router which have only 4MB of flash memory. Maybe in next time I try to make it for some router with only 2MB of flash. In that way, > > I need not copy of any code. > > In ideal embedded systems, if we have code, we must use it everywhere we need it. > > Interesting! The Redboot loader that I'm toying with on the ubiquiti > hardware supports LZMA as well as GZIP for compressed kernel images; I > may have to experiment with that and this module. No, this module not fore compress/decompress kernel. This module used like geom_uzip (man geom_uzip, man mkuzip), for compressing blocks of filesystem to reducing size of FS image. To use with bootloader You may use lzma utility. > > Thanks! > > > Adrian -- Alex RAY From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 20 18:20:29 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2CF91106566C; Sat, 20 Feb 2010 18:20:29 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-iw0-f185.google.com (mail-iw0-f185.google.com [209.85.223.185]) by mx1.freebsd.org (Postfix) with ESMTP id D69318FC16; Sat, 20 Feb 2010 18:20:28 +0000 (UTC) Received: by iwn15 with SMTP id 15so1276297iwn.7 for ; Sat, 20 Feb 2010 10:20:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type; bh=oj2djcxPV2y0eWp3NQfNiLJVve8tPkGFfUXIyHd9XBA=; b=SutUIBtOXyAswXHKfC4JOF5SDU4gi76uGo9Y8kxGTTef+Fiqq/CeEhmWzcHtmRxP9f 4Y4kb/P2FRtbBY1D6WbvEf4/TgaoQKQsT/sQIdKZ00R+bXQHFTNGlXFNL3xF7aNeuKCi VI520GtDBnxpDggA89YNafBrwmrVXXEiFbtJc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=q/RLXjuB8TqBMT7N8xtF9lGNnVReDtRmlX2ljaeP9+ZuAr+Gkag4A5g3S6NbaaalIF ZMdQR+hDirhdq+zg02VnZjX4eeuRwpi+Q6jhwS8OcMA3GHqgCKQMoBn+D9XfyMXQ7LC0 sJE9qd7XIedenueIIaafYYg0XA41BLlLCfu8M= MIME-Version: 1.0 Sender: adrian.chadd@gmail.com Received: by 10.231.170.14 with SMTP id b14mr1032703ibz.26.1266690028049; Sat, 20 Feb 2010 10:20:28 -0800 (PST) In-Reply-To: <20100220201510.720c62fd.ray@ddteam.net> References: <20100219163644.da89e882.ray__27111.9062825621$1266591431$gmane$org@dlink.ua> <20100219222604.44e50248.ray@ddteam.net> <20100220201510.720c62fd.ray@ddteam.net> Date: Sun, 21 Feb 2010 02:20:28 +0800 X-Google-Sender-Auth: 58861a74fa57f45d Message-ID: From: Adrian Chadd To: Alex RAY Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org, Ivan Voras Subject: Re: GEOM_ULZMA X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Feb 2010 18:20:29 -0000 Oh I know that! I'm just saying that I may try lzma'ing the kernel and rootfs's to see what kind of savings I get over gzip. :) Adrian On 21 February 2010 02:15, Alex RAY wrote: > Hi, > > On Sat, 20 Feb 2010 22:36:02 +0800 > Adrian Chadd wrote: > >> On 20 February 2010 04:26, Alex RAY wrote: >> >> > :) No, I don`t think about "magically faster", now I near to release FreeBSD firmware for D-Link DIR-320 router which have only 4MB of flash memory. Maybe in next time I try to make it for some router with only 2MB of flash. In that way, >> > I need not copy of any code. >> > In ideal embedded systems, if we have code, we must use it everywhere we need it. >> >> Interesting! The Redboot loader that I'm toying with on the ubiquiti >> hardware supports LZMA as well as GZIP for compressed kernel images; I >> may have to experiment with that and this module. > > No, this module not fore compress/decompress kernel. This module used like geom_uzip (man geom_uzip, man mkuzip), for compressing blocks of filesystem to reducing size of FS image. > > To use with bootloader You may use lzma utility. > > >> >> Thanks! >> >> >> Adrian > > > -- > Alex RAY > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 20 21:03:20 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4FE97106566B for ; Sat, 20 Feb 2010 21:03:20 +0000 (UTC) (envelope-from tyler@monkeypox.org) Received: from starfish.geekisp.com (mail.geekisp.com [216.168.135.169]) by mx1.freebsd.org (Postfix) with ESMTP id E2A2C8FC12 for ; Sat, 20 Feb 2010 21:03:19 +0000 (UTC) Received: (qmail 18582 invoked by uid 1003); 20 Feb 2010 21:03:19 -0000 Received: from localhost (HELO kiwi.sharlinx.com) (tyler@monkeypox.org@127.0.0.1) by mail.geekisp.com with SMTP; 20 Feb 2010 21:03:18 -0000 Date: Sat, 20 Feb 2010 13:03:14 -0800 From: "R. Tyler Ballance" To: Patrick Mahan Message-ID: <20100220210314.GB22800@kiwi.sharlinx.com> Mail-Followup-To: Patrick Mahan , freebsd-hackers@freebsd.org References: <4B800F60.60700@mahan.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="gatW/ieO32f1wygP" Content-Disposition: inline In-Reply-To: <4B800F60.60700@mahan.org> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-hackers@freebsd.org Subject: Re: Building FreeBSD on a linux FC11 box. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Feb 2010 21:03:20 -0000 --gatW/ieO32f1wygP Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, 20 Feb 2010, Patrick Mahan wrote: >=20 > Hopefully, this is not too ignorant a question. But has anyone every > built the FreeBSD sources, both kernel and apps, on a linux platform? >=20 > I did a google on 'cross-compile freebsd (linux)' and found (mostly) > discussions regarding building the linux apps for FreeBSD, but not > for building a FreeBSD kernel and world. There was a brief discussion > on the mailing list back in 2005, but the poster never reported if > he had success or failure. Also, there were discussions about building > freebsd amd64 on a freebsd i386. >=20 > I have done lots of development in linux using cross-compilers (mips > embedded systems, powerpc, etc). But not with FreeBSD. >=20 > I know I'll need to use 'bsdmake' no gnu 'make'. The other worry is > kernel-toolchain target. I guess I'll just dive in and swim around > and see what happens. You might want to ask the Debian GNU/kFreeBSD guys: http://www.debian.org/ports/kfreebsd-gnu/ I bet they've got a good idea :) Cheers, -R. Tyler Ballance -------------------------------------- Jabber: rtyler@jabber.org GitHub: http://github.com/rtyler Twitter: http://twitter.com/agentdero Blog: http://unethicalblogger.com --gatW/ieO32f1wygP Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.12 (GNU/Linux) iEYEARECAAYFAkuAThIACgkQFCbH3D9R4W8zzwCfWss3oHuYIIipcGE4QLwmgTqD 7D0AmwV/ur1ttBSzvk8djULZYszlpylX =eox0 -----END PGP SIGNATURE----- --gatW/ieO32f1wygP--