From owner-freebsd-hackers@FreeBSD.ORG Sun Mar 18 07:19:02 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DCE8C106566B; Sun, 18 Mar 2012 07:19:02 +0000 (UTC) (envelope-from to.my.trociny@gmail.com) Received: from mail-bk0-f54.google.com (mail-bk0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 102EE8FC0C; Sun, 18 Mar 2012 07:19:01 +0000 (UTC) Received: by bkcjc3 with SMTP id jc3so5108024bkc.13 for ; Sun, 18 Mar 2012 00:19:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:references:x-comment-to:sender:date:in-reply-to :message-id:user-agent:mime-version:content-type; bh=lAFMcUrqi76gEZ3wnfpa3J4DkYTsq8QeIFAathh8EJk=; b=JGMVcpHzw4gENjmx8M7DpldnGpEmisUs/h6BYebnO+zfI9LQ+BPegU7mdwdHX/MxSt E4j/iRsz4R0pOhFV6wshxqaw/xAcUbPU5RLhhA9JmiCV6MvvJBOgxfsrTsPEMW2V3V1D N9N3nNEasf3mInFd7DHPEYvJahMIoeeboOJvOlJh/0loSpCTxp6dlilZ09keHP+MWW+V q6nkyS/BL+kSjG9ztHrp3nZeAuIusMTCtazvdv8ujyqVSvf2U6ObHyBU+kmq5izpNTH/ MLYZO9cugHEZDSY5Tj0jbpXF82EldrW7j55AK5c2wQxsvyPMxgm8HnT3TsliRWXji8FW XM/A== Received: by 10.204.13.72 with SMTP id b8mr3059552bka.105.1332055140842; Sun, 18 Mar 2012 00:19:00 -0700 (PDT) Received: from localhost ([95.69.173.122]) by mx.google.com with ESMTPS id bw9sm18648408bkb.8.2012.03.18.00.18.58 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 18 Mar 2012 00:18:59 -0700 (PDT) From: Mikolaj Golub To: Jilles Tjoelker References: <8662e3m3eq.fsf@kopusha.home.net> <20120317212901.GA44534@stack.nl> X-Comment-To: Jilles Tjoelker Sender: Mikolaj Golub Date: Sun, 18 Mar 2012 09:18:57 +0200 In-Reply-To: <20120317212901.GA44534@stack.nl> (Jilles Tjoelker's message of "Sat, 17 Mar 2012 22:29:01 +0100") Message-ID: <86d38apeam.fsf@kopusha.home.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Kostik Belousov , freebsd-hackers@freebsd.org, Robert Watson Subject: Re: a sysctl for process binary osreldate X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Mar 2012 07:19:03 -0000 On Sat, 17 Mar 2012 22:29:01 +0100 Jilles Tjoelker wrote: JT> On Sat, Mar 17, 2012 at 09:30:05PM +0200, Mikolaj Golub wrote: >> I added osrel output to procstat -b option: >> kopusha:~% procstat -b 2975 >> PID COMM OSREL PATH >> 2975 emacs 1000001 /usr/local/bin/emacs-23.3 >> Would this be ok or someone see a better way? JT> Hmm, this means that procstat is not supposed to be used from scripts as JT> it is apparently OK to change its output format like this? Yes, breaking output compatibility worries me too. Although I already broke it recently for '-s' option, adding umask output. Let me cite Robert (taken from our then discussion about procstat umask output): > if we add too many arguments we'll start looking like ps(1), whereas the > point of procstat(1) is that it's *not* ps(1) :-). That is why I decided to not introduce yet another option here too at the cost of breaking compatibility. But I am open for any suggestions. JT> In some ways, querying via ps would be better for scripts since it JT> allows things like JT> ps -p PID -o KEYWORD= JT> which do not need additional parsing except that many of the newer JT> things in procstat do not have ps keywords. JT> -- JT> Jilles Tjoelker -- Mikolaj Golub From owner-freebsd-hackers@FreeBSD.ORG Sun Mar 18 13:01:41 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2665D106566B for ; Sun, 18 Mar 2012 13:01:41 +0000 (UTC) (envelope-from rank1seeker@gmail.com) Received: from mail-we0-f182.google.com (mail-we0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id AE2538FC0C for ; Sun, 18 Mar 2012 13:01:40 +0000 (UTC) Received: by wern13 with SMTP id n13so6796946wer.13 for ; Sun, 18 Mar 2012 06:01:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:from:to:subject:date:content-type :content-transfer-encoding:x-mailer; bh=dpwfXBQcDUBaveK112XJV6X4F4tm4Qokk+lXw9V1+YY=; b=fE/tegQqcrUP44k7CTC9Y2oI8clds+F1U+tfCpvQFVBNLM1UHM6QVdv7vNvUxcfaur P8Xm5VlQgnGvKKbvzOewcLiZZSPmJAWv7lZqX+ys6dWaqmp9FrqCs7zQmqJ3YxypVVSW t6GEbKuIeAAQppHm9/Y1c9U6Qbmy+apqlp7PYb/WXE+2OxM1Y0L+gk8CuZFUAJfeKRFX IWRZPe8WUSfX8Uca0F9YkppcjxSCL6+Jegz+Fxx6J1hVcOx+D6ZbOREVjfH+b4Ed0oSq fQQ9kyVQIBnL+Ag4YMe27v5wBWIz7GSLzwjhiCp8qjkOtATFE1CHJUtBHWpEK6katTCH xXCA== Received: by 10.180.107.101 with SMTP id hb5mr9673965wib.3.1332075699696; Sun, 18 Mar 2012 06:01:39 -0700 (PDT) Received: from DOMYPC (78-1-142-155.adsl.net.t-com.hr. [78.1.142.155]) by mx.google.com with ESMTPS id 9sm26774361wid.2.2012.03.18.06.01.36 (version=SSLv3 cipher=OTHER); Sun, 18 Mar 2012 06:01:38 -0700 (PDT) Message-ID: <20120318.130139.003.1@DOMY-PC> From: rank1seeker@gmail.com To: hackers@freebsd.org Date: Sun, 18 Mar 2012 14:01:39 +0100 Content-Type: text/plain; charset="Windows-1250" Content-Transfer-Encoding: quoted-printable X-Mailer: POP Peeper (3.8.1.0) Cc: Subject: BUG: REL 9.0 - MD malloc of custom sector size X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Mar 2012 13:01:41 -0000 man mdconfig=0D=0A----=0D=0A-S sectorsize to use for malloc backed = device=0D=0A----=0D=0A=0D=0AI want to create MD device, with sector size = of 4 Kb.=0D=0A=0D=0AIt is CRITICAL to NOT append ANY suffixes, when = specifing size, via '-s' flag in order to use sectors, to set it's = size.=0D=0A# mdconfig -a -t malloc -S 4096 -s 32768=0D=0A=0D=0AThis = should created dev of 128 Mb in size.=0D=0A32768 sectors * 4 Kb each =3D = 131072 Kb =3D 128 Mb=0D=0ANot! It created dev of 16 Mb in size, because = sector size remained at 512 bytes.=0D=0A=0D=0AAnd look what 'diskinfo' = has to say/lie ...=0D=0A# diskinfo -v md0=0D=0Amd0=0D=0A 4096 = # sectorsize=0D=0A 16777216 # mediasize in bytes = (16M)=0D=0A 4096 # mediasize in sectors=0D=0A 0 = # stripesize=0D=0A 0 # = stripeoffset=0D=0A=0D=0AFalse lines:=0D=0A # sectorsize=0D=0A # = mediasize in sectors (doesn't match '-s 32768')=0D=0A=0D=0A=0D=0ADoes = anyone has any advice of creating image / file(vnode) of custom sector = sizes (2k, 4k, 8k)=0D=0AThanks in advance.=0D=0A=0D=0A=0D=0ADomagoj = Smol=E8i=E6 From owner-freebsd-hackers@FreeBSD.ORG Sun Mar 18 13:14:33 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A2E6B106566C for ; Sun, 18 Mar 2012 13:14:33 +0000 (UTC) (envelope-from bluezhudong@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 6E90F8FC15 for ; Sun, 18 Mar 2012 13:14:33 +0000 (UTC) Received: by iahk25 with SMTP id k25so11083742iah.13 for ; Sun, 18 Mar 2012 06:14:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:subject:message-id:mime-version:content-type :content-disposition:user-agent; bh=9sl+FvPvQw2/v+2iG77SdUYp5s7HiPsyDpIbO6xxOFY=; b=hAG1s4fVIJXCpARUAwZ6NBLgDglTlFWAp9pw8mUgmuSgbkjo9ElFesqArKcZEOK67T aVgOa7EeTwIVn3Y1fc6s0kiQEtBjUybaMaX5WuyI+oca5Tf9dSVzZQHT1ClqY7NORehs bTwmeBGVjpu2Ncvuk8fr7zGhMz5oisxogyAk9UM6xa0ya7+aTwfCawbOO9cByDSE/Na0 2OEiTs1FTf+FWfnIJXdj2XfxxLdvR7fFFZu0m1k31A4zGr4vMyXFEbPFHoTjSE5y69lh SApC1IXSpJnPiFn4v7Fww8tgkBDh7Jb4l5gHljE+5XSvpenaREbBXyTAqmKKyhqBs1jQ D6Sw== Received: by 10.50.45.167 with SMTP id o7mr4482837igm.22.1332076472885; Sun, 18 Mar 2012 06:14:32 -0700 (PDT) Received: from localhost ([114.249.128.223]) by mx.google.com with ESMTPS id vr4sm11830797igb.1.2012.03.18.06.14.29 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 18 Mar 2012 06:14:32 -0700 (PDT) Date: Sun, 18 Mar 2012 21:11:27 +0800 From: Dong Zhu To: freebsd-hackers@freebsd.org Message-ID: <20120318131127.GA28356@localhost.localdomain> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.20 (2009-06-14) Subject: issues about FreeBSD 2012 Gsoc ideas X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Mar 2012 13:14:33 -0000 Hi, I am student from China.I am looking forward to joining the Gsoc 2012 and I am very very interested in this idea [1].As you know,the FreeBSD organization has been accepted into Google Summer of Code 2012 [2],so I really want to join it. I am a intern working as a Linux kernel-qe,and I am very interested in Timer/Power-Management,so I really want to join this project.How can I join this project ? Could you please give me some suggestions ? Many Many thanks! [1] http://wiki.freebsd.org/IdeasPage#CPU_online.2BAC8-offline_project_.28GSoC.29 [2] http://www.google-melange.com/gsoc/org/google/gsoc2012/freebsd -- Best Regards, Dong Zhu ------------------ http://bluezd.info ------------------ From owner-freebsd-hackers@FreeBSD.ORG Sun Mar 18 13:42:06 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4EEC8106566B for ; Sun, 18 Mar 2012 13:42:06 +0000 (UTC) (envelope-from rwmaillists@googlemail.com) Received: from mail-wi0-f178.google.com (mail-wi0-f178.google.com [209.85.212.178]) by mx1.freebsd.org (Postfix) with ESMTP id D2FFE8FC14 for ; Sun, 18 Mar 2012 13:42:05 +0000 (UTC) Received: by wibhq7 with SMTP id hq7so2186398wib.13 for ; Sun, 18 Mar 2012 06:41:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20120113; h=date:from:to:subject:message-id:in-reply-to:references:x-mailer :mime-version:content-type:content-transfer-encoding; bh=CvAqu9xirNIASZ8dKTEid878aF4XiqtMlKlxNrDl/pk=; b=e7WBhCOeNCmL8NEkFNIlnrliy10OOtgk4pb8y4jS77RuDsSRrGW0JTzONIMQepB5Pc /V0N6b6QAHmxJGwsoCEuKuZ9RveWS0jrqrYbpEVBCedhfgM5UxPhwezMKJ9WM1TR0Syw o3HVl3EWsceUTtFygXOtEYLuAy/b11cAr4WOS9UNmvlSfsNt011sNPYRicMnLHv+bWBP GwMhIKAxnJb2qQlR96XlWY0eyO1BtIBvinPHAL2m7RVHqS/P41jcZBgdT+aJsb0WwQQS mzkbb2UaKoGEDm6YYTNxW+ziXoAg2zlKgDKi+sT1iyUfzzy7kRl0pmuKI4eJ65ZU3jek tB/w== Received: by 10.180.103.97 with SMTP id fv1mr12567658wib.17.1332078119456; Sun, 18 Mar 2012 06:41:59 -0700 (PDT) Received: from gumby.homeunix.com (87-194-105-247.bethere.co.uk. [87.194.105.247]) by mx.google.com with ESMTPS id gg2sm27137987wib.7.2012.03.18.06.41.57 (version=SSLv3 cipher=OTHER); Sun, 18 Mar 2012 06:41:58 -0700 (PDT) Date: Sun, 18 Mar 2012 13:41:56 +0000 From: RW To: freebsd-hackers@freebsd.org Message-ID: <20120318134156.61d6db61@gumby.homeunix.com> In-Reply-To: <20120318.130139.003.1@DOMY-PC> References: <20120318.130139.003.1@DOMY-PC> X-Mailer: Claws Mail 3.8.0 (GTK+ 2.24.6; amd64-portbld-freebsd8.2) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Subject: Re: BUG: REL 9.0 - MD malloc of custom sector size X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Mar 2012 13:42:06 -0000 On Sun, 18 Mar 2012 14:01:39 +0100 rank1seeker@gmail.com wrote: > man mdconfig > ---- > -S sectorsize to use for malloc backed device > ---- >=20 > I want to create MD device, with sector size of 4 Kb. >=20 > It is CRITICAL to NOT append ANY suffixes, when specifing size, via > '-s' flag in order to use sectors, to set it's size. # mdconfig -a -t > malloc -S 4096 -s 32768 >=20 > This should created dev of 128 Mb in size. > 32768 sectors * 4 Kb each =3D 131072 Kb =3D 128 Mb > Not! It created dev of 16 Mb in size, because sector size remained at > 512 bytes. =46rom mdconfig 8 "Size is the number of 512 byte sectors unless ..." Looks to me like it's doing what it said it would. BTW are you sure you want to use "-t malloc". This keeps the files (even the deleted ones) in memory unconditionally while ordinary process memory is paged-out.=20 From owner-freebsd-hackers@FreeBSD.ORG Sun Mar 18 22:56:34 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 31FDA1065670 for ; Sun, 18 Mar 2012 22:56:34 +0000 (UTC) (envelope-from dieterbsd@engineer.com) Received: from mailout-us.gmx.com (mailout-us.gmx.com [74.208.5.67]) by mx1.freebsd.org (Postfix) with SMTP id D03CC8FC0A for ; Sun, 18 Mar 2012 22:56:33 +0000 (UTC) Received: (qmail 18759 invoked by uid 0); 18 Mar 2012 22:56:33 -0000 Received: from 67.206.186.200 by rms-us004.v300.gmx.net with HTTP Content-Type: text/plain; charset="utf-8" Date: Sun, 18 Mar 2012 18:56:29 -0400 From: "Dieter BSD" Message-ID: <20120318225630.155070@gmx.com> MIME-Version: 1.0 To: freebsd-hackers@freebsd.org,freebsd-drivers@freebsd.org X-Authenticated: #74169980 X-Flags: 0001 X-Mailer: GMX.com Web Mailer x-registered: 0 Content-Transfer-Encoding: 8bit X-GMX-UID: SQ0Ub/dd3zOlNR3dAHAhK4h+IGRvb4AB Cc: Subject: Reigning in disk cache buffer hogs X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Mar 2012 22:56:34 -0000 amd64 4 GiB nVidia nForce CK804 (aka nforce4-ultra), SiI3132, JMB363 sata disks FreeBSD 8.2 FFS/SU Problem: I/O to one disk can fill up the disk buffer cache, starving other processes of disk I/O.  If the other processes are logging real time data (from a closed source "black box"), then data is lost.  Each process has its own private disk to minimise latency due to seeking and such, but that isn't sufficient to prevent the problem. It is always the onboard nforce4-ultra controller that causes the problem. The expansion card controllers never cause the problem. The onboard nforce4-ultra controller is supported by ata(4). The expansion card controllers are supported by ahci(4) and siis(4). The onboard nforce4-ultra controller is much faster than the expansion card controllers, except that ata doesn't support NCQ, so the nforce has a serious bottleneck writing. Is there something about NCQ other than the obvious performance improvement that would explain why the NCQ controllers never have the problem? What sort of miracle would it take to get NCQ support for the nforce4-ultra controller?  Linix has had NCQ support since 2006. Is there some knob to control reserving a few buffers per disk to prevent one disk from hogging all the buffer cache? From owner-freebsd-hackers@FreeBSD.ORG Mon Mar 19 02:46:05 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1CB76106564A; Mon, 19 Mar 2012 02:46:05 +0000 (UTC) (envelope-from bluezhudong@gmail.com) Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id DF9958FC0A; Mon, 19 Mar 2012 02:46:04 +0000 (UTC) Received: by dald2 with SMTP id d2so9859342dal.13 for ; Sun, 18 Mar 2012 19:46:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:mime-version:content-type :content-disposition:user-agent; bh=5Wvl38+c4WETa9Eg8Ou1kPsaaD45pEkJdbCByfKSTbw=; b=zE0MpWnlBYijButxGYvEG15vn3895gHBEfi/w+LWrIve34E3rZpRGIkbEAra6ckZta 9JVP6k8v5iCrnYxK6qmOsEIQnbZ9ZjmRwa1FKx55zqn2tXalORPjmG/bM5Q8W1bb0+8T g7TqfdtoLCXV5M8iljyfByCgbFeUM8U2jLnUaaiP6WY48soIjvpCvlOne3VHl2kyENaA b14jW23m8WFxnQhxVOZIiwbAiSmADf3A7VOynr/hNXS8kgK5tgj69H4ZLAw7MjWB7pNo o5U7HXDuwF9K7L23QK+GZDthVDw9SPzlUYfKbOH4eMlbIFQpHB9yDUl2lfhsn6uKAhy4 OTTg== Received: by 10.68.213.8 with SMTP id no8mr35312800pbc.144.1332125164486; Sun, 18 Mar 2012 19:46:04 -0700 (PDT) Received: from localhost ([203.114.244.88]) by mx.google.com with ESMTPS id s10sm10187915pbp.14.2012.03.18.19.46.02 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 18 Mar 2012 19:46:03 -0700 (PDT) Date: Mon, 19 Mar 2012 10:46:29 +0800 From: Dong Zhu To: jhb@FreeBSD.org Message-ID: <20120319024629.GA3110@localhost.localdomain> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-hackers@freebsd.org Subject: Participate FreeBSD 2012 Gsoc ideas CPU online/offline project X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Mar 2012 02:46:05 -0000 Hi, I am student from China.I am looking forward to joining the Gsoc 2012 and I am very very interested in this idea [1]. I am a intern working as a Linux kernel-qe,and I am very interested in Timer/Power-Management,so I really want to join this project.But I do not know how to start,How can I join this project ? Could you please give me some suggestions ? Many Many thanks! [1] http://wiki.freebsd.org/IdeasPage#CPU_online.2BAC8-offline_project_.28GSoC.29 -- Best Regards, Dong Zhu ---------------------------- WebSite: http://bluezd.info ---------------------------- From owner-freebsd-hackers@FreeBSD.ORG Mon Mar 19 14:15:10 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 68EA61065673 for ; Mon, 19 Mar 2012 14:15:10 +0000 (UTC) (envelope-from kotasaikrishna28@gmail.com) Received: from mail-wi0-f178.google.com (mail-wi0-f178.google.com [209.85.212.178]) by mx1.freebsd.org (Postfix) with ESMTP id EDE1E8FC1F for ; Mon, 19 Mar 2012 14:15:09 +0000 (UTC) Received: by wibhq7 with SMTP id hq7so3049015wib.13 for ; Mon, 19 Mar 2012 07:15:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=t50lcXNq3ArL6QRtWQyMsnhuPXuOPHgQRtFhj3zpedw=; b=lirkE7i7bGdHpOmL50ljP30aT8c/5JLuZSwZ0CU5CdoIgjtL0SSn/sathbAiYPiCqp JMfNM0Em8K9PbQjfaq3PUvHyVhnsqVpxv4umNt/Y2YouKO/2qJB3N1aFCmw4tO03BIjw MsVJ87xlPGf+Qp7YRqh+YviMk4+lMU/t6OMLp8l6NwlB/Hsz73jk+/16HKIU7dNNiWHK cwKYHURcidZChSp4+fkVMSLqajIjz9b9jf4uF6zd0uBp4CoOHbS7z97TgNe3EdMhJXOr x78n8eg+ddwF3yXFv4UNvXbp3L6VjtrezH90Dn/j0mDPh8TMpl/m6qc3WApeHearYJXv 8EYw== MIME-Version: 1.0 Received: by 10.180.14.230 with SMTP id s6mr20392899wic.2.1332166503610; Mon, 19 Mar 2012 07:15:03 -0700 (PDT) Received: by 10.223.114.73 with HTTP; Mon, 19 Mar 2012 07:15:03 -0700 (PDT) Date: Mon, 19 Mar 2012 19:45:03 +0530 Message-ID: From: kota saikrishna To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Checkpointing of simple programs in FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Mar 2012 14:15:10 -0000 I found that checkpointing and restart of a process is available for DragonFlyBSD. I need this feature for FreeBSD. How can I checkpoint simple programs in FreeBSD such that they can be restored and run from the checkpointed state? For example, can someone tell me the steps so that I can checkpoint a small program (that prints a few lines say) half way through its run, and then restore it and print the rest of the lines? From owner-freebsd-hackers@FreeBSD.ORG Mon Mar 19 14:33:22 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A080F1065670 for ; Mon, 19 Mar 2012 14:33:22 +0000 (UTC) (envelope-from maninya@gmail.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id 53FB88FC1F for ; Mon, 19 Mar 2012 14:33:22 +0000 (UTC) Received: by ghrr20 with SMTP id r20so6596126ghr.13 for ; Mon, 19 Mar 2012 07:33:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=zEdrikb7AlQEoa2NgyYawgple8tzAWnqzp5QRgPDfNg=; b=wO47EwqqY822ix0jo/FeH5rGhroAJF9Ia5hVFGs9mGoFOk4/5kw8TszrbaIeT6EIED MoQpvqS57gt2zqBYlh0aVLoFDXs55k4dMaHNh8ZeAbRS6d31y5KObeKd7tfS/1/zsPGW VUaC/V5oaCqWzxTw7AHJGRMpVJ1mo7lJhc3+68+ToiaE3klxPlLD2mb5I4pzp/ydz0Md hf4JXE0yJuuIVg+wKA+yVx2+2E37HlrT6wvAZu9WsTqGzQqns3OeLpLLtTRn3arm98hS WIiGcCQa7bjbl+MekFzAT2Punit8OEPHZPv0ASy6nv+v2OUyskXnA+/ha0J5o7+KirVi 9CFA== MIME-Version: 1.0 Received: by 10.236.116.195 with SMTP id g43mr12575060yhh.26.1332167601921; Mon, 19 Mar 2012 07:33:21 -0700 (PDT) Received: by 10.147.9.4 with HTTP; Mon, 19 Mar 2012 07:33:21 -0700 (PDT) In-Reply-To: References: <4F60FF28.8010104@freebsd.org> Date: Mon, 19 Mar 2012 20:03:21 +0530 Message-ID: From: Maninya M To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Fwd: Capture states of all processes at the same time X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Mar 2012 14:33:22 -0000 Thank you, Artem Belevich. It's working, I was able to get coredump files core.txt.0 and vmcore.0. What I want now is to restore the processes as they were when the crash occurred. Is there a way to do this? On 16 March 2012 11:38, Artem Belevich wrote: > On Thu, Mar 15, 2012 at 10:49 PM, Maninya M wrote: > > # sysctl debug.kdb.enter=1 > > > > ------ > > > > But when I type that the computer hangs! > > Did you by any chance do that from a terminal window in X11 > environment? If that's the case, then kernel debugger is running, you > just don't see anything because it can print stuff out only on console > or serial port. If that indeed what happened, typing c and then ENTER > should unhang your system. After that you can switch to the console > with CTRL-ALT-F1 and enter debugger from there. > > --Artem > -- Maninya -- Maninya From owner-freebsd-hackers@FreeBSD.ORG Mon Mar 19 15:25:47 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 668CB106566C for ; Mon, 19 Mar 2012 15:25:47 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 3A57B8FC17 for ; Mon, 19 Mar 2012 15:25:47 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) by cyrus.watson.org (Postfix) with ESMTPSA id DE1DD46B2D; Mon, 19 Mar 2012 11:25:46 -0400 (EDT) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 6AD95B924; Mon, 19 Mar 2012 11:25:46 -0400 (EDT) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Mon, 19 Mar 2012 09:30:09 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p10; KDE/4.5.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201203190930.09782.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 19 Mar 2012 11:25:46 -0400 (EDT) Cc: Matthew Story Subject: Re: FTSENT: name and path on `/' versus name and path on `*/' X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Mar 2012 15:25:47 -0000 On Thursday, March 15, 2012 4:46:03 pm Matthew Story wrote: > Found a curious incongruent behavior in fts(3), wondering if there is some > reason for this, or if it's just a bug. If you include the path > > `/' > > the FTSENT at depth 0 that is returned for the path has both fts_path = "/" > and fts_name = "/", compared to other entries, like /var which has fts_path > = "/" and fts_path = "/" and fts_name = "var", or /var/, which has fts_path > = "/var/" and fts_name = "". > > Given the behavior of other paths used in fts(3), my expectation here is > that FTSENT for path "/" on depth 0 would have fts_path = "/" and fts_name > = "". Haven't delved down into the code enough to figure out where this is > happening, but from a cursory read through libc/gen/fts.c there doesn't > seem to be any explicit special casing of the path "/". > > Can anyone shed light on why this behavior is desirable, or if it's just a > bug I'm happy to file a PR and delve further into fts.c ... My guess is that it is a bug. / is a bit special as it serves two purposes (i.e., both the idea of "/var" and "/var/" map to "/" for the root). -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Mon Mar 19 15:31:05 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DE6C31065675 for ; Mon, 19 Mar 2012 15:31:04 +0000 (UTC) (envelope-from kmacybsd@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id A16FD8FC15 for ; Mon, 19 Mar 2012 15:31:04 +0000 (UTC) Received: by iahk25 with SMTP id k25so13489354iah.13 for ; Mon, 19 Mar 2012 08:31:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=8cjZTZLMrP5nWv0qIDoizMBJR11LJAV9GlBdMbv/WIU=; b=ZbvJJjEAMugPhG3XL6cpIEfBOoCw8tx4WkNOEQ+SVb/3bgUKw7R9Ycrt1uH0pgHYu8 bu529AdSjEuGxte6clmMFk1RO+Bbhn1qcPoEore6dGT57D47H46GvSdASWcMP+IK+QGa ismA7J69N8cCvsJsoPMQtUzGR6Cvc7pI/GgYDKk7WaFaAk7dL1bHW34QkggeDtkEQixX aalLzm9ZRh/ZmjRalQfTq63a/X3O61wOItt0y74dn1fT2L8ryG4ANzStAtzegjkTXQl6 wWywr3D2ZLP4oP86716OQW5xjM5DdcBuj9Tr3/BvrQzyRnHZxUoUS7sxWPxqwHdX4GoN /C1A== MIME-Version: 1.0 Received: by 10.50.42.136 with SMTP id o8mr6347896igl.38.1332171064266; Mon, 19 Mar 2012 08:31:04 -0700 (PDT) Sender: kmacybsd@gmail.com Received: by 10.50.134.106 with HTTP; Mon, 19 Mar 2012 08:31:04 -0700 (PDT) In-Reply-To: References: Date: Mon, 19 Mar 2012 16:31:04 +0100 X-Google-Sender-Auth: 5ue702A6VJxl_GgJuPAez-EuOHg Message-ID: From: "K. Macy" To: kota saikrishna Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: Checkpointing of simple programs in FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Mar 2012 15:31:05 -0000 Take a look at gcore. That will let you checkpoint memory mappings and registers. The dragonfly implementation also keeps track of the sbrk value and open file handles which you will need to handle separately. Restoring the mappings and register state should be straightforward. I'm not sure at how you get the open file handles - look at how 'lsof' does it. Restoring pipes obviously makes no sense unless you checkpoint all processes in the graph and restoring sockets is something you'd want to defer to a restore handler. I can't speak for the current implementation, but the original was my first foray in to kernel hacking and only took a few days to implement so I would wager that porting it can't be all that difficult even with the divergence in kernel internals between FreeBSD and DragonFly. Cheers, Kip On Mon, Mar 19, 2012 at 3:15 PM, kota saikrishna wrote: > I found that checkpointing and restart of a process is available for > DragonFlyBSD. > I need this feature for FreeBSD. How can I checkpoint simple programs in > FreeBSD such that they can be restored and run from the checkpointed stat= e? > For example, can someone tell me the steps so that I can checkpoint a sma= ll > program (that prints a few lines say) half way through its run, and then > restore it and print the rest of the lines? > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " --=20 =A0 =A0=93The real damage is done by those millions who want to 'get by.' The ordinary men who just want to be left in peace. Those who don=92t want their little lives disturbed by anything bigger than themselves. Those with no sides and no causes. Those who won=92t take measure of their own strength, for fear of antagonizing their own weakness. Those who don=92t like to make waves=97or enemies. =A0 =A0Those for whom freedom, honour, truth, and principles are only literature. Those who live small, love small, die small. It=92s the reductionist approach to life: if you keep it small, you=92ll keep it under control. If you don=92t make any noise, the bogeyman won=92t find you. =A0 =A0But it=92s all an illusion, because they die too, those people who roll up their spirits into tiny little balls so as to be safe. Safe?! >From what? Life is always on the edge of death; narrow streets lead to the same place as wide avenues, and a little candle burns itself out just like a flaming torch does. =A0 =A0I choose my own way to burn.=94 =A0 =A0Sophie Scholl From owner-freebsd-hackers@FreeBSD.ORG Tue Mar 20 01:52:27 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id ADDA6106566B; Tue, 20 Mar 2012 01:52:27 +0000 (UTC) (envelope-from bluezhudong@gmail.com) Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id 7C08D8FC0A; Tue, 20 Mar 2012 01:52:27 +0000 (UTC) Received: by dald2 with SMTP id d2so11625127dal.13 for ; Mon, 19 Mar 2012 18:52:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:mime-version:content-type :content-disposition:user-agent; bh=w7Tb22+1rMo5u4iforghMSdDWh8Nqlvrwhx2ZieoXzM=; b=dh9juL+xYrNbqanqikYYzAU1yOri+wq3PRHxy6D3V13W+Ex9pFh1FAiUfDZSnfnJQ/ TT5H4dq9r84Eb+XjgElW/L1nBk1sH5wz0Shq1MmFoAfB+iKYy0ssX4K6zxXkU26XMPgl IBZ+1GPe8UVdIunQQPEHD954DYkk7EffTVieiTYNGCejQHhunU+DGm97HeUd6NKc955F 9642INiBBh7H/Jx74g9cJOfgRejTYD6NsrfZd894Isgn/g/z9Dd1IaAKY+RfbWZBAqB2 zZeupVsuVkLq7Oy1T0Yrj6HAETZajw2z9ruoT8t372gGFdzaDQEckkquk5bhHOOHyUOX /5FQ== Received: by 10.68.201.6 with SMTP id jw6mr38809940pbc.92.1332208345930; Mon, 19 Mar 2012 18:52:25 -0700 (PDT) Received: from localhost ([203.114.244.88]) by mx.google.com with ESMTPS id v6sm75977pbu.41.2012.03.19.18.52.23 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 19 Mar 2012 18:52:24 -0700 (PDT) Date: Tue, 20 Mar 2012 09:52:49 +0800 From: Dong Zhu To: jhb@freebsd.org Message-ID: <20120320015249.GA20328@localhost.localdomain> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-hackers@freebsd.org Subject: [NEED HELP]Participate FreeBSD 2012 Gsoc ideas CPU online/offline project X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Mar 2012 01:52:27 -0000 Hi, I am student from China.I am looking forward to joining the Gsoc 2012 and I am very very interested in this idea [1]. I am a intern working as a Linux kernel-qe,and I am very interested in Timer/Power-Management,so I really want to join this project.But I do not know how to start,How can I join this project ? Could you please give me some suggestions ? Many Many thanks! [1] http://wiki.freebsd.org/IdeasPage#CPU_online.2BAC8-offline_project_.28GSoC.29 -- Best Regards, Dong Zhu ---------------------------- WebSite: http://bluezd.info ---------------------------- From owner-freebsd-hackers@FreeBSD.ORG Tue Mar 20 06:33:08 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5C6311065670; Tue, 20 Mar 2012 06:33:08 +0000 (UTC) (envelope-from gnehzuil@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 50C6A8FC18; Tue, 20 Mar 2012 06:33:02 +0000 (UTC) Received: by iahk25 with SMTP id k25so14798475iah.13 for ; Mon, 19 Mar 2012 23:33:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=pqvNcQduKzfW5ipNHYtx4Hny5hkgJawrT16z3C5X2nU=; b=OABy+alFcErLZfZb/b630D0uAi8+6nOA6TTkUencVyp29KfVdyoxgkHjERk6utXDWX uljk6e8F+XBbvD2sL/w387oOvZe/eNuBvRY+mO62PcDS0fNKUeqhRbkNimyaZYK7Awyk mgntW3hk5NFymrk93uU6+dldyxGKHElufxzdEbwjlMpw2PqN87gjrELJN8AQlo9CRWVD LAjrsbXVwBbIceVaTRBjGeu27fYChIQUwF+vWejaAkMXHY7saN5q1sVBiNWOVpl5pRKC DvnHb9gXSKF+6rQEit326k/sjbK+qrFhuv1brFD66ORM+/q01lfYFECc4elQPhp1g3ef kYwQ== Received: by 10.42.137.67 with SMTP id x3mr8378215ict.52.1332225181560; Mon, 19 Mar 2012 23:33:01 -0700 (PDT) Received: from [10.13.209.191] ([121.0.29.114]) by mx.google.com with ESMTPS id xt2sm7600343igb.6.2012.03.19.23.32.58 (version=SSLv3 cipher=OTHER); Mon, 19 Mar 2012 23:33:00 -0700 (PDT) Message-ID: <4F682498.9000707@gmail.com> Date: Tue, 20 Mar 2012 14:32:56 +0800 From: gnehzuil User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:8.0) Gecko/20111124 Thunderbird/8.0 MIME-Version: 1.0 To: bluezhudong@gmail.com References: <20120320015249.GA20328@localhost.localdomain> In-Reply-To: <20120320015249.GA20328@localhost.localdomain> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, John Baldwin Subject: Re: [NEED HELP]Participate FreeBSD 2012 Gsoc ideas CPU online/offline project X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Mar 2012 06:33:08 -0000 On 03/20/2012 09:52 AM, Dong Zhu wrote: > Hi, > I am student from China.I am looking forward to joining the Gsoc 2012 and I am very very interested in this idea [1]. > > I am a intern working as a Linux kernel-qe,and I am very interested in Timer/Power-Management,so I really want to join this project.But I do not know how to start,How can I join this project ? Could you > please give me some suggestions ? > > > Many Many thanks! > > [1] http://wiki.freebsd.org/IdeasPage#CPU_online.2BAC8-offline_project_.28GSoC.29 > > Hi Dong, If you want to join GSoc 2012, please see the documentation in GSoC (http://www.google-melange.com/gsoc/homepage/google/gsoc2012). I remember that you should register firstly and sign up. Then you will see the docs about how to submit your proposal that describes your schedules and goals. Sorry, I am not familiar with Timer/Power-management. So I don't have any suggestions about this project for you. But, IMO, at least you should have some basic knowledge about Timer/Power-management and FreeBSD. Good luck. Regards, Zheng From owner-freebsd-hackers@FreeBSD.ORG Tue Mar 20 15:37:59 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 64CEF106566B for ; Tue, 20 Mar 2012 15:37:59 +0000 (UTC) (envelope-from pluknet@gmail.com) Received: from mail-lpp01m010-f54.google.com (mail-lpp01m010-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id 960688FC16 for ; Tue, 20 Mar 2012 15:37:58 +0000 (UTC) Received: by lagv3 with SMTP id v3so182343lag.13 for ; Tue, 20 Mar 2012 08:37:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=bWmAXSrcIs+LSm90+1QdbVKxC8HHZhP0VEUZQx/a+ZU=; b=1JE9tiMghvk8VbO77ibzsO2nbzmCQPYU1Jguh5aaCjJmvqseWMRown5QK/zJJ+RhVj nY9h12V+QmkIyQjvctVRfN27zudpnHrcIWNPLHacx9sRk72WPMmwDlluKbKx86HsT4tT ngVSrvUwwclEXyvFXnp0TFlNsQHpCI1OKeuyocvGype/g3F2+2pj1Kz7779KQkkDM8Z3 A+iRnR6ZMRKu/w0xR1XEFNpwT+fssm+ysur8gvvM2yDKjwcPbWhnhWrHahYsHMYTJXAV z07PuG38krMx7wB691H1K6RGOnNPtRzs8D3S3A/PGCkeHX4H2kgGSWJg2GobxaHFnBH8 ma/w== MIME-Version: 1.0 Received: by 10.112.39.36 with SMTP id m4mr76045lbk.100.1332257877213; Tue, 20 Mar 2012 08:37:57 -0700 (PDT) Received: by 10.152.21.73 with HTTP; Tue, 20 Mar 2012 08:37:57 -0700 (PDT) In-Reply-To: References: Date: Tue, 20 Mar 2012 18:37:57 +0300 Message-ID: From: Sergey Kandaurov To: Mark Saad Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: Approaching the limit on PV entries X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Mar 2012 15:37:59 -0000 On 22 November 2011 19:29, Mark Saad wrote: > Hello All [found this mail in my drafts, not sure if my answer is still useful] > =A0I want to get to the bottom of a warning in dmesg. On 7.2-RELEASE and > 7.3-RELEASE I have seen the following warning in dmesg. > > Approaching the limit on PV entries, consider increasing either the > vm.pmap.shpgperproc or the vm.pmap.pv_entry_max sysctl. > > So looking around I see a few posts here and there about how to tune > the sysctls to address the warning however I am not 100% sure what > each value does. > It appears changing vm.pmap.shpgperproc affects the value of > vm.pmap.pv_entry_max . Can someone explain the relationship of the two > sysctls. Also This is how they are calculated. pv_entry_max =3D shpgperproc * maxproc + cnt.v_page_count; and, respectively, shpgperproc =3D (pv_entry_max - cnt.v_page_count) / maxproc; So, changing one sysctl will change another and vice versa. > what pitfalls of changing them are. Not known to me (on amd64 platform). I have had vm.pmap.shpgperproc=3D15000 on 8.1 amd64 with 4G RAM to make some badly written commercial software to work until it was decommissioned to the scrap. > Also why would setting > kern.ipc.shm_use_phys=3D1 =A0effect the pv entries. Is this supposed to > lower the pv entries ? Changing this sysctl with restarting a quite busy PgSQL server helped me to reduce pv entries from 14M to tens of thousands (though that could just coincide with decrease in workload). --=20 wbr, pluknet From owner-freebsd-hackers@FreeBSD.ORG Tue Mar 20 20:02:38 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3003C106566B for ; Tue, 20 Mar 2012 20:02:37 +0000 (UTC) (envelope-from rank1seeker@gmail.com) Received: from mail-bk0-f54.google.com (mail-bk0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id AC4488FC08 for ; Tue, 20 Mar 2012 20:02:36 +0000 (UTC) Received: by bkcjc3 with SMTP id jc3so490602bkc.13 for ; Tue, 20 Mar 2012 13:02:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:from:to:subject:date:content-type :content-transfer-encoding:in-reply-to:references:x-mailer; bh=wx3C3MeSsWtNzW4wYh6JSvInnqncu4QOS4vLIYHF8+c=; b=t1TQLT0pdiSZ2QxqVOHnp7lKuz+GczdMb1OtJUsLqsn6eIsQEAJa/TMcgDiYYC10l6 Cp5Ooj2Xwwd+3mtaqR/2X2x47n6afLBQ86lk6Ecp99G+CVJ7GRR0q1Hmlsn09D5CfRWl +Y84vN29lBp8xQoDtaI5bUcKdVzE/NVKKfQLJ8p/eYKmlU9nC8d0e558aBTNuP1dqQBY zF+tQsX2LHJnleMwd6TYptJpOIPQ15AQ7gGREO3c7xfHiGFI43cv7tBO+0Xz2m2uQ926 qcsY/VOYKsWoeUxNDYVqJ7a/kgGMWUIHrx3SiLIF3xmNan9jWV8UqxYz1gGTlgoW0lEQ c0bQ== Received: by 10.204.173.11 with SMTP id n11mr414921bkz.120.1332273755565; Tue, 20 Mar 2012 13:02:35 -0700 (PDT) Received: from DOMYPC ([82.193.208.173]) by mx.google.com with ESMTPS id u14sm5341576bkp.2.2012.03.20.13.02.31 (version=SSLv3 cipher=OTHER); Tue, 20 Mar 2012 13:02:33 -0700 (PDT) Message-ID: <20120320.200234.909.1@DOMY-PC> From: rank1seeker@gmail.com To: "RW" , hackers@freebsd.org Date: Tue, 20 Mar 2012 21:02:34 +0100 Content-Type: text/plain; charset="Windows-1250" Content-Transfer-Encoding: quoted-printable In-Reply-To: <20120318134156.61d6db61@gumby.homeunix.com> References: <20120318.130139.003.1@DOMY-PC> <20120318134156.61d6db61@gumby.homeunix.com> X-Mailer: POP Peeper (3.8.1.0) Cc: Subject: Re: BUG: REL 9.0 - MD malloc of custom sector size X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Mar 2012 20:02:38 -0000 ----- Original Message -----=0D=0AFrom: RW = =0D=0ATo: = freebsd-hackers@freebsd.org=0D=0ADate: Sun, 18 Mar 2012 13:41:56 = +0000=0D=0ASubject: Re: BUG: REL 9.0 - MD malloc of custom sector = size=0D=0A=0D=0A> On Sun, 18 Mar 2012 14:01:39 +0100=0D=0A> = rank1seeker@gmail.com wrote:=0D=0A> =0D=0A> > man mdconfig=0D=0A> > = ----=0D=0A> > -S sectorsize to use for malloc backed device=0D=0A> > = ----=0D=0A> > =0D=0A> > I want to create MD device, with sector size of 4 = Kb.=0D=0A> > =0D=0A> > It is CRITICAL to NOT append ANY suffixes, when = specifing size, via=0D=0A> > '-s' flag in order to use sectors, to set = it's size. # mdconfig -a -t=0D=0A> > malloc -S 4096 -s 32768=0D=0A> > = =0D=0A> > This should created dev of 128 Mb in size.=0D=0A> > 32768 = sectors * 4 Kb each =3D 131072 Kb =3D 128 Mb=0D=0A> > Not! It created dev = of 16 Mb in size, because sector size remained at=0D=0A> > 512 = bytes.=0D=0A> =0D=0A> =0D=0A> From mdconfig 8=0D=0A> =0D=0A> "Size is the = number of 512 byte sectors unless ..."=0D=0A> =0D=0A> Looks to me like = it's doing what it said it would.=0D=0A> =0D=0A> BTW are you sure you = want to use "-t malloc". This keeps the files=0D=0A> (even the deleted = ones) in memory unconditionally while ordinary=0D=0A> process memory is = paged-out. =0D=0A> =0D=0A=0D=0AMy MAIN reason to hassle with MD here, is = to test a custom sector size.=0D=0AThis can be done with '-S' flag only, = in order to set sectorsize of /dev/md*=0D=0ABut as it is malloc ONLY = option/flag, I must combine it with '-t malloc'=0D=0A=0D=0AThen I've = defined it's size by amount of sectors and as I've redefined size of 1 = sector, it simply isn't doing it's task.=0D=0ABecause it enforces = hardcoded size of 512 bytes, so documentation should not misleadingly = refer to sector in any way, but a hardcode value of 0.5 Kb, no matter of = real/actual sector size is.=0D=0A=0D=0A=0D=0A=0D=0ADomagoj Smol=E8i=E6 From owner-freebsd-hackers@FreeBSD.ORG Wed Mar 21 03:42:53 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 66EA21065675 for ; Wed, 21 Mar 2012 03:42:53 +0000 (UTC) (envelope-from kaduk@mit.edu) Received: from dmz-mailsec-scanner-4.mit.edu (DMZ-MAILSEC-SCANNER-4.MIT.EDU [18.9.25.15]) by mx1.freebsd.org (Postfix) with ESMTP id 027288FC0C for ; Wed, 21 Mar 2012 03:42:52 +0000 (UTC) X-AuditID: 1209190f-b7f8a6d000000914-0c-4f694e36d09b Received: from mailhub-auth-3.mit.edu ( [18.9.21.43]) by dmz-mailsec-scanner-4.mit.edu (Symantec Messaging Gateway) with SMTP id 2A.0B.02324.63E496F4; Tue, 20 Mar 2012 23:42:46 -0400 (EDT) Received: from outgoing.mit.edu (OUTGOING-AUTH.MIT.EDU [18.7.22.103]) by mailhub-auth-3.mit.edu (8.13.8/8.9.2) with ESMTP id q2L3gkM2002081; Tue, 20 Mar 2012 23:42:46 -0400 Received: from multics.mit.edu (MULTICS.MIT.EDU [18.187.1.73]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.6/8.12.4) with ESMTP id q2L3giM7007756 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Tue, 20 Mar 2012 23:42:45 -0400 (EDT) Received: (from kaduk@localhost) by multics.mit.edu (8.12.9.20060308) id q2L3giUY026308; Tue, 20 Mar 2012 23:42:44 -0400 (EDT) Date: Tue, 20 Mar 2012 23:42:44 -0400 (EDT) From: Benjamin Kaduk To: rank1seeker@gmail.com In-Reply-To: <20120320.200234.909.1@DOMY-PC> Message-ID: References: <20120318.130139.003.1@DOMY-PC> <20120318134156.61d6db61@gumby.homeunix.com> <20120320.200234.909.1@DOMY-PC> User-Agent: Alpine 1.10 (GSO 962 2008-03-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrDIsWRmVeSWpSXmKPExsUixCmqrWvml+lv8HO2qMWGBYUWLT+usTsw ecz4NJ/FY+esu+wBTFFcNimpOZllqUX6dglcGW19C9kKLspV9JxaytjAeEi8i5GTQ0LAROLc /FPMELaYxIV769m6GLk4hAT2MUr8+f2DFcLZwCjxZt5VZgjnAJPE5EWbWCCcBkaJC5tPsIH0 swhoS8zpW8gEYrMJqEjMfLMRLC4iICkx5f5KsB3MAuISC+/1AjVzcAgLWEtMumwDEuYU0JE4 23yNHcTmFXCUeN93EKxcSKBC4sLzF2BxUaCa1funsEDUCEqcnPmEBWKkpcS/tb9YJzAKzkKS moUktYCRaRWjbEpulW5uYmZOcWqybnFyYl5eapGuiV5uZoleakrpJkZwoEry72D8dlDpEKMA B6MSD6/28gx/IdbEsuLK3EOMkhxMSqK85j6Z/kJ8SfkplRmJxRnxRaU5qcWHGCU4mJVEeLNW AZXzpiRWVqUW5cOkpDlYlMR51bTe+QkJpCeWpGanphakFsFkZTg4lCR41X2BhgoWpaanVqRl 5pQgpJk4OEGG8wANnwpSw1tckJhbnJkOkT/FqCglztsIkhAASWSU5sH1whLJK0ZxoFeEeSNB qniASQiu+xXQYCagwXE300AGlyQipKQaGOczxejteq806bzIY65/svueL31nvmv2Klf+LX/Z 0+WV/0zq+KswOZqB71f5jaNa5sff5Agwuk/6lu3Rvta96W1caOMR802FDYsF3xxY7W/c//0W T/mL6k+GHpGzpb5Ee+4y7tXVOdPw6kn1/0PPnPyaDhgwb77RHCTz//vpzt433XeDWFU505RY ijMSDbWYi4oTAUi6ZPP/AgAA Cc: hackers@freebsd.org Subject: Re: BUG: REL 9.0 - MD malloc of custom sector size X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Mar 2012 03:42:53 -0000 On Tue, 20 Mar 2012, rank1seeker@gmail.com wrote: > ----- Original Message ----- > From: RW > To: freebsd-hackers@freebsd.org > Date: Sun, 18 Mar 2012 13:41:56 +0000 > Subject: Re: BUG: REL 9.0 - MD malloc of custom sector size > >> On Sun, 18 Mar 2012 14:01:39 +0100 >> rank1seeker@gmail.com wrote: >> >>> man mdconfig >>> ---- >>> -S sectorsize to use for malloc backed device >>> ---- >>> >>> I want to create MD device, with sector size of 4 Kb. >>> >>> It is CRITICAL to NOT append ANY suffixes, when specifing size, via >>> '-s' flag in order to use sectors, to set it's size. # mdconfig -a -t >>> malloc -S 4096 -s 32768 >>> >>> This should created dev of 128 Mb in size. >>> 32768 sectors * 4 Kb each = 131072 Kb = 128 Mb >>> Not! It created dev of 16 Mb in size, because sector size remained at >>> 512 bytes. >> >> >> From mdconfig 8 >> >> "Size is the number of 512 byte sectors unless ..." >> >> Looks to me like it's doing what it said it would. >> >> BTW are you sure you want to use "-t malloc". This keeps the files >> (even the deleted ones) in memory unconditionally while ordinary >> process memory is paged-out. >> > > My MAIN reason to hassle with MD here, is to test a custom sector size. > This can be done with '-S' flag only, in order to set sectorsize of /dev/md* > But as it is malloc ONLY option/flag, I must combine it with '-t malloc' > > Then I've defined it's size by amount of sectors and as I've redefined > size of 1 sector, it simply isn't doing it's task. Because it enforces > hardcoded size of 512 bytes, so documentation should not misleadingly > refer to sector in any way, but a hardcode value of 0.5 Kb, no matter of > real/actual sector size is. It should not be technically challenging to cause mdconfig to have the -s size argument in terms of -S sectorsize sized sectors; the following would probably suffice (untested): %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Index: mdconfig.c =================================================================== --- mdconfig.c (revision 233159) +++ mdconfig.c (working copy) @@ -94,16 +94,19 @@ int main(int argc, char **argv) { - int ch, fd, i, vflag; + int ch, fd, i, vflag, sflag; char *p; char *fflag = NULL, *tflag = NULL, *uflag = NULL; + unsigned bsize; bzero(&mdio, sizeof(mdio)); mdio.md_file = malloc(PATH_MAX); if (mdio.md_file == NULL) err(1, "could not allocate memory"); vflag = 0; + sflag = 0; bzero(mdio.md_file, PATH_MAX); + bsize = DEV_BSIZE; if (argc == 1) usage(); @@ -186,11 +189,12 @@ break; case 'S': mdio.md_sectorsize = strtoul(optarg, &p, 0); + bsize = mdio.md_sectorsize; break; case 's': mdio.md_mediasize = (off_t)strtoumax(optarg, &p, 0); if (p == NULL || *p == '\0') - mdio.md_mediasize *= DEV_BSIZE; + sflag = 1; else if (*p == 'b' || *p == 'B') ; /* do nothing */ else if (*p == 'k' || *p == 'K') @@ -232,6 +236,9 @@ if (action == UNSET) action = ATTACH; + if (sflag == 1) + mdio.md_mediasize *= bsize; + if (action == ATTACH) { if (tflag == NULL) { /* %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Or would you prefer a man page change? %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Index: mdconfig.8 =================================================================== --- mdconfig.8 (revision 233159) +++ mdconfig.8 (working copy) @@ -132,7 +132,7 @@ .It Fl s Ar size Size of the memory disk. .Ar Size -is the number of 512 byte sectors unless suffixed with a +is measured in increments of 512 byes unless suffixed with a .Cm b , k , m , g , or .Cm t %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% BTW I do not see where -S is a malloc-only option; please show the command line and error message using a vnode- or swap-backed device. -Ben Kaduk From owner-freebsd-hackers@FreeBSD.ORG Wed Mar 21 04:55:25 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D999A1065676 for ; Wed, 21 Mar 2012 04:55:25 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id A987D8FC16 for ; Wed, 21 Mar 2012 04:55:25 +0000 (UTC) Received: by dald2 with SMTP id d2so1177276dal.13 for ; Tue, 20 Mar 2012 21:55:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=BQ7LTLBVFUqQsXFYQ8QEdTaVKULD8a28k1x93gxFJtE=; b=RGoyYPvEfQVm6oxljDY+W+TJxM0OTTkak8CRBqVdbKJ8l57ktIXPpnkMBQxQOkITj9 M9V2OFncybx2zcJo2vhV8wHIm0XzduLAoJt2zbGBf2DMzQQDFM6df+7fz8vqioMaImWb zH6q4pY5Hs9xyeDiwuEW+2/yGFensTka42tldsB4SRqZKCKex74urxhMLU16Gcbra95R esUMxarN0ADWtm9TGlCKbb0cna6R365k92hY0hYeOPox9f41mkIF84OugPDG0jpHLyKy LObXTCiQWutWheseApXLQ6gNpOfgV1LGq+3GV0YiKV951Ij23tA8GX/Uq36lQ2r5v9fF ELAA== MIME-Version: 1.0 Received: by 10.68.240.135 with SMTP id wa7mr7587530pbc.7.1332305725240; Tue, 20 Mar 2012 21:55:25 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.143.33.5 with HTTP; Tue, 20 Mar 2012 21:55:25 -0700 (PDT) In-Reply-To: References: <20120312181921.GF75778@deviant.kiev.zoral.com.ua> <20120315112959.GP75778@deviant.kiev.zoral.com.ua> Date: Tue, 20 Mar 2012 21:55:25 -0700 X-Google-Sender-Auth: pMIFPh03rezHvH86CmXFKIRTfMc Message-ID: From: Adrian Chadd To: Svatopluk Kraus Content-Type: text/plain; charset=ISO-8859-1 Cc: Konstantin Belousov , hackers@freebsd.org Subject: Re: [vfs] buf_daemon() slows down write() severely on low-speed CPU X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Mar 2012 04:55:25 -0000 Hi, I'm interested in this, primarily because I'm tinkering with file storage stuff on my little (most wifi targetted) embedded MIPS platforms. So what's the story here? How can I reproduce your issue and do some of my own profiling/investigation? Adrian From owner-freebsd-hackers@FreeBSD.ORG Wed Mar 21 08:36:06 2012 Return-Path: Delivered-To: hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 184AE106566C for ; Wed, 21 Mar 2012 08:36:06 +0000 (UTC) (envelope-from bapt@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id F2B8A8FC0A for ; Wed, 21 Mar 2012 08:36:05 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q2L8a5Ei036344 for ; Wed, 21 Mar 2012 08:36:05 GMT (envelope-from bapt@FreeBSD.org) Received: (from bapt@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q2L8a5cv036341 for hackers@freebsd.org; Wed, 21 Mar 2012 08:36:05 GMT (envelope-from bapt@FreeBSD.org) X-Authentication-Warning: freefall.freebsd.org: bapt set sender to bapt@FreeBSD.org using -f Date: Wed, 21 Mar 2012 09:36:01 +0100 From: Baptiste Daroussin To: hackers@FreeBSD.org Message-ID: <20120321083601.GF9629@azathoth.lan> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Wb5NtZlyOqqy58h0" Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Subject: Playing with include-what-you-use shows interesting stuff X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Mar 2012 08:36:06 -0000 --Wb5NtZlyOqqy58h0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi, I've been playing with the include-what-you-use[1] llvm tool for some on my personnal projects, as it works very well, I have also played with it on our source tree starting with the bin directory. It shows some interesting results, while the default output is quite aggressive, I just chose to remove the "useless" headers in each sources. It show some interesting results which seems to come from maybe bad includes in some of our headers. Apparently some of the #include are false positive because others headers shouldn't include it for example (according to des) here is a diff showing what I find that can be removed: http://people.freebsd.org/~bapt/include-what-you-use.diff I think it shouldn't be applied as it but be more analyzed. regards, Bapt --Wb5NtZlyOqqy58h0 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk9pkvEACgkQ8kTtMUmk6EzTFgCgkHoubxhPED0QLLQFbF7BpjpG za4An1//HfUTChxamjFIgQ//kbXdJ/kt =8tki -----END PGP SIGNATURE----- --Wb5NtZlyOqqy58h0-- From owner-freebsd-hackers@FreeBSD.ORG Wed Mar 21 08:44:16 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D77A91065675 for ; Wed, 21 Mar 2012 08:44:16 +0000 (UTC) (envelope-from freebsd-hackers@herveybayaustralia.com.au) Received: from mail.unitedinsong.com.au (mail.unitedinsong.com.au [150.101.178.33]) by mx1.freebsd.org (Postfix) with ESMTP id 504B38FC2C for ; Wed, 21 Mar 2012 08:44:15 +0000 (UTC) Received: from mail.unitedinsong.com.au (bell.herveybayaustralia.com.au [192.168.0.40]) by mail.unitedinsong.com.au (Postfix) with ESMTP id 1B6E35C2E for ; Wed, 21 Mar 2012 18:57:42 +1000 (EST) Received: from laptop1.herveybayaustralia.com.au (laptop1.herveybayaustralia.com.au [192.168.0.182]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.unitedinsong.com.au (Postfix) with ESMTPSA id A4E185C22 for ; Wed, 21 Mar 2012 18:57:41 +1000 (EST) Message-ID: <4F699391.9070804@herveybayaustralia.com.au> Date: Wed, 21 Mar 2012 18:38:41 +1000 From: Da Rock <9Phackers@herveybayaustralia.com.au> User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:7.0.1) Gecko/20111109 Thunderbird/7.0.1 MIME-Version: 1.0 To: "hackers@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Wed, 21 Mar 2012 11:22:53 +0000 Cc: Subject: iso2flash img X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: "hackers@freebsd.org" List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Mar 2012 08:44:17 -0000 I'm doing some work for a client and I need to get a copy of ubuntu to demo for them. I can only use a usb disk (no cdrom), and unfortunately the guys at ubuntu are too lazy to create a flash img themselves and expect you to do it yourself. Of course the tools to do this are only available on Winblows and Mac (hdiutils? I think). They have instructions on the site - which of course are useless to me being a FreeBSD freak ;) I googled a bit and found an old post here from Luigi (http://osdir.com/ml/freebsd-hackers/2008-11/msg00245.html) which had a script to do this, but I'm having trouble with it- is anyone familiar with this? I'm on a bit of a deadline... and yes, I did install syslinux port. Output: ../iso2flash ubuntu-11.10-desktop-i386.iso usage: realpath [-q] path [...] type <> tree /Downloads/ubuntu-11.10-desktop-i386.iso> image <> Extract files from /usr/home//Downloads/ubuntu-11.10-desktop-i386.iso into /usr/home//Downloads/ubuntu-11.10-desktop-i386.iso.tree chmod: /usr/home//Downloads/ubuntu-11.10-desktop-i386.iso.tree: No such file or directory total 6 drwxr-xr-x 2 512 Mar 21 18:22 . drwxr-xr-x 6 2560 Mar 21 18:22 .. total 2500 drwxr-xr-x 11 512 Mar 21 18:23 . drwxr-xr-x 6 2560 Mar 21 18:22 .. drwx------ 2 512 Oct 13 01:15 .disk -r-------- 1 225 Oct 13 01:15 README.diskdefines -r-------- 1 143 Oct 13 01:15 autorun.inf drwx------ 3 512 Oct 13 01:15 boot drwx------ 2 512 Oct 13 01:15 casper drwx------ 3 512 Oct 13 01:15 dists drwx------ 2 512 Oct 13 01:15 install drwx------ 2 2560 Oct 13 01:15 isolinux -r-------- 1 4418 Oct 13 01:15 md5sum.txt drwx------ 2 512 Oct 13 01:15 pics drwx------ 4 512 Oct 13 01:15 pool drwx------ 2 512 Oct 13 01:15 preseed -r-------- 1 2491552 Oct 13 01:15 wubi.exe image size is 711676 kb guess type 8+0 records in 8+0 records out 1048576 bytes transferred in 0.005349 secs (196035057 bytes/sec) newfs_msdos: /dev/711676: No such file or directory syslinux: invalid media signature (not a FAT filesystem?) moving boot code mv: rename /usr/home//Downloads/ubuntu-11.10-desktop-i386.iso.tree/isolinux.cfg to /usr/home//Downloads/ubuntu-11.10-desktop-i386.iso.tree/syslinux.cfg: No such file or directory moving files... init :: non DOS media Cannot initialize '::' Bad target ::/ init :: non DOS media Cannot initialize '::' I'm going to keep on investigating, I tried this on 8.2 and 9.0 with no luck. I intend to attempt a usb install of FreeBSD at some point (on the todo list for some time now), but at this point its still a bit of a mystery; so I think I may have some trouble understanding this atm. A little instruction would go a long way if someone could provide some pointers. Cheers From owner-freebsd-hackers@FreeBSD.ORG Wed Mar 21 12:55:45 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 791FE1065674 for ; Wed, 21 Mar 2012 12:55:45 +0000 (UTC) (envelope-from vmagerya@gmail.com) Received: from mail-bk0-f54.google.com (mail-bk0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 00BFD8FC0C for ; Wed, 21 Mar 2012 12:55:44 +0000 (UTC) Received: by bkcjc3 with SMTP id jc3so1193351bkc.13 for ; Wed, 21 Mar 2012 05:55:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=3qNOLrA+GOoZmiFDc4C38p2DK8+C3y9yEoR0ggBQYGI=; b=Vm5fHOYiGk8uxU1TOzCZL1bLAFQ7jpe05/CoW3bCbnnq6UZtnu0XoLOKn3wDAz33iq YGKyo11icDWqqFPVo6MsXKV2T2DqPnYNlPC5i634JJJK6QkWyNoXcIniSCORJBZvsdF1 TkF5kWa0+YZmRDt5SDD4V7IZlZdrxYEMYg5NCm6HWGlU94gxagIBT0u47Zso7qPz5NXU TjZK/Wq+aEre8a6raPssFKA2s7fEtr/QnXA9ByM8zLZaKWpL+EQ7Uj+ju2dd1FbnDe7x ZNvMsJhif0j4vh48DZ81i5reKDaMp+bGW4aj/RAjJ4gHuPIHatuRy+7R/1qHDbdBliU5 8j2g== Received: by 10.204.133.195 with SMTP id g3mr1425368bkt.73.1332334543836; Wed, 21 Mar 2012 05:55:43 -0700 (PDT) Received: from [172.29.1.142] (altimet-gw.cs2.dp.wnet.ua. [217.20.178.249]) by mx.google.com with ESMTPS id u5sm3321841bka.5.2012.03.21.05.55.41 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 21 Mar 2012 05:55:42 -0700 (PDT) Message-ID: <4F69CFAC.7030501@gmail.com> Date: Wed, 21 Mar 2012 14:55:08 +0200 From: Vitaly Magerya User-Agent: Thunderbird MIME-Version: 1.0 To: Da Rock <9Phackers@herveybayaustralia.com.au> References: <4F699391.9070804@herveybayaustralia.com.au> In-Reply-To: <4F699391.9070804@herveybayaustralia.com.au> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: "hackers@freebsd.org" Subject: Re: iso2flash img X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Mar 2012 12:55:45 -0000 Da Rock wrote: > I googled a bit and found an old post here from Luigi > (http://osdir.com/ml/freebsd-hackers/2008-11/msg00245.html) which had a > script to do this, but I'm having trouble with it- is anyone familiar > with this? I'm on a bit of a deadline... Can't help you with that script (I failed to make it work too), but you might want to try to dd the iso image directly onto USB instead; there where talks that Ubuntu would support this starting at 11.10. From owner-freebsd-hackers@FreeBSD.ORG Wed Mar 21 13:07:04 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 258ED106564A for ; Wed, 21 Mar 2012 13:07:04 +0000 (UTC) (envelope-from guru@unixarea.de) Received: from ms16-1.1blu.de (ms16-1.1blu.de [89.202.0.34]) by mx1.freebsd.org (Postfix) with ESMTP id D720B8FC17 for ; Wed, 21 Mar 2012 13:07:03 +0000 (UTC) Received: from [82.113.98.61] (helo=tiny.Sisis.de) by ms16-1.1blu.de with esmtpsa (TLS-1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.69) (envelope-from ) id 1SALFd-0002iZ-Vs; Wed, 21 Mar 2012 14:06:54 +0100 Received: from tiny.Sisis.de (localhost [127.0.0.1]) by tiny.Sisis.de (8.14.5/8.14.3) with ESMTP id q2LD6owt001734; Wed, 21 Mar 2012 14:06:51 +0100 (CET) (envelope-from guru@unixarea.de) Received: (from guru@localhost) by tiny.Sisis.de (8.14.5/8.14.3/Submit) id q2LD6mLt001732; Wed, 21 Mar 2012 14:06:48 +0100 (CET) (envelope-from guru@unixarea.de) X-Authentication-Warning: tiny.Sisis.de: guru set sender to guru@unixarea.de using -f Date: Wed, 21 Mar 2012 14:06:48 +0100 From: Matthias Apitz To: 9Phackers@herveybayaustralia.com.au Message-ID: <20120321130648.GA1714@tiny> References: <20120321120551.GA1547@tiny> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20120321120551.GA1547@tiny> X-Operating-System: FreeBSD 10.0-CURRENT r226986 (i386) User-Agent: Mutt/1.5.21 (2010-09-15) X-Con-Id: 51246 X-Con-U: 0-guru X-Originating-IP: 82.113.98.61 Cc: hackers@freebsd.org Subject: Re: iso2flash img X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Matthias Apitz List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Mar 2012 13:07:04 -0000 Hello, I've forwarded your question to a colleague who is an Ubuntu fan and he replied, that the creation of a booting Ubuntu USB key is possile with Ubuntu tools/methods only, i.e. booting the Ubuntu live CD and following the ways to install it onto the key (and not on disk). He wrote some years ago a small Wiki page about, it is in German but maybe you can clue something from the commands there: http://mikiwiki.org/wiki/Ubuntu_8.10_Intrepid_Ibex/Installation_2009.04.07_usbstick HIH matthias -- Matthias Apitz e - w http://www.unixarea.de/ UNIX since V7 on PDP-11, UNIX on mainframe since ESER 1055 (IBM /370) UNIX on x86 since SVR4.2 UnixWare 2.1.2, FreeBSD since 2.2.5 From owner-freebsd-hackers@FreeBSD.ORG Wed Mar 21 13:17:10 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 77E27106567F for ; Wed, 21 Mar 2012 13:17:10 +0000 (UTC) (envelope-from freebsd-hackers@herveybayaustralia.com.au) Received: from mail.unitedinsong.com.au (mail.unitedinsong.com.au [150.101.178.33]) by mx1.freebsd.org (Postfix) with ESMTP id 2D24F8FC27 for ; Wed, 21 Mar 2012 13:17:10 +0000 (UTC) Received: from laptop1.herveybayaustralia.com.au (laptop1.herveybayaustralia.com.au [192.168.0.182]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.unitedinsong.com.au (Postfix) with ESMTPSA id A6B6C5C22 for ; Wed, 21 Mar 2012 23:30:42 +1000 (EST) Message-ID: <4F69D38E.9070805@herveybayaustralia.com.au> Date: Wed, 21 Mar 2012 23:11:42 +1000 From: Da Rock User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:7.0.1) Gecko/20111109 Thunderbird/7.0.1 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org References: <4F699391.9070804@herveybayaustralia.com.au> <4F69CFAC.7030501@gmail.com> In-Reply-To: <4F69CFAC.7030501@gmail.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: iso2flash img X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Mar 2012 13:17:10 -0000 On 03/21/12 22:55, Vitaly Magerya wrote: > Da Rock wrote: >> I googled a bit and found an old post here from Luigi >> (http://osdir.com/ml/freebsd-hackers/2008-11/msg00245.html) which had a >> script to do this, but I'm having trouble with it- is anyone familiar >> with this? I'm on a bit of a deadline... > Can't help you with that script (I failed to make it work too), but you > might want to try to dd the iso image directly onto USB instead; there > where talks that Ubuntu would support this starting at 11.10. Interesting. I'll have to look further into how that would work. Also, why wouldn't they just tell Mac users to do that? In the meantime I think I may have stumbled on the solution to the script: In the midst of all the output it mentions "usage realpath [-q] path". I wasn't 100% sure exactly what that meant, but I put the full path to the iso and a full path to an img file and I *think* that worked. I've yet to test the result; and I have no idea of the '-q' option.... From owner-freebsd-hackers@FreeBSD.ORG Wed Mar 21 13:22:27 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 010E9106564A for ; Wed, 21 Mar 2012 13:22:27 +0000 (UTC) (envelope-from freebsd-hackers@herveybayaustralia.com.au) Received: from mail.unitedinsong.com.au (mail.unitedinsong.com.au [150.101.178.33]) by mx1.freebsd.org (Postfix) with ESMTP id AADB08FC08 for ; Wed, 21 Mar 2012 13:22:26 +0000 (UTC) Received: from laptop1.herveybayaustralia.com.au (laptop1.herveybayaustralia.com.au [192.168.0.182]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.unitedinsong.com.au (Postfix) with ESMTPSA id 045675C22 for ; Wed, 21 Mar 2012 23:35:58 +1000 (EST) Message-ID: <4F69D4CB.1010401@herveybayaustralia.com.au> Date: Wed, 21 Mar 2012 23:16:59 +1000 From: Da Rock User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:7.0.1) Gecko/20111109 Thunderbird/7.0.1 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org References: <20120321120551.GA1547@tiny> <20120321130648.GA1714@tiny> In-Reply-To: <20120321130648.GA1714@tiny> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: iso2flash img X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Mar 2012 13:22:27 -0000 On 03/21/12 23:06, Matthias Apitz wrote: > Hello, > > I've forwarded your question to a colleague who is an Ubuntu fan and he > replied, that the creation of a booting Ubuntu USB key is possile with > Ubuntu tools/methods only, i.e. booting the Ubuntu live CD and following the > ways to install it onto the key (and not on disk). > > He wrote some years ago a small Wiki page about, it is in German but > maybe you can clue something from the commands there: > > http://mikiwiki.org/wiki/Ubuntu_8.10_Intrepid_Ibex/Installation_2009.04.07_usbstick I'll have to translate that, but I am trying to get a 'live' usb disk to demo on the clients cdrom less unit. I know the cd is live, I assumed I could get a live usb disk from that based on their instructions. For a supposedly user friendly system, obtaining install media is not.. :/ Maybe a little too much debian in the system ;) From owner-freebsd-hackers@FreeBSD.ORG Wed Mar 21 14:37:13 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 60E9D106566B for ; Wed, 21 Mar 2012 14:37:13 +0000 (UTC) (envelope-from freebsd-hackers@herveybayaustralia.com.au) Received: from mail.unitedinsong.com.au (mail.unitedinsong.com.au [150.101.178.33]) by mx1.freebsd.org (Postfix) with ESMTP id 1348A8FC18 for ; Wed, 21 Mar 2012 14:37:12 +0000 (UTC) Received: from laptop1.herveybayaustralia.com.au (laptop1.herveybayaustralia.com.au [192.168.0.182]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.unitedinsong.com.au (Postfix) with ESMTPSA id 69F235C22 for ; Thu, 22 Mar 2012 00:50:45 +1000 (EST) Message-ID: <4F69E651.3070105@herveybayaustralia.com.au> Date: Thu, 22 Mar 2012 00:31:45 +1000 From: Da Rock User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:7.0.1) Gecko/20111109 Thunderbird/7.0.1 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org References: <4F699391.9070804@herveybayaustralia.com.au> <4F69CFAC.7030501@gmail.com> <4F69D38E.9070805@herveybayaustralia.com.au> In-Reply-To: <4F69D38E.9070805@herveybayaustralia.com.au> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: iso2flash img X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Mar 2012 14:37:13 -0000 On 03/21/12 23:11, Da Rock wrote: > On 03/21/12 22:55, Vitaly Magerya wrote: >> Da Rock wrote: >>> I googled a bit and found an old post here from Luigi >>> (http://osdir.com/ml/freebsd-hackers/2008-11/msg00245.html) which had a >>> script to do this, but I'm having trouble with it- is anyone familiar >>> with this? I'm on a bit of a deadline... >> Can't help you with that script (I failed to make it work too), but you >> might want to try to dd the iso image directly onto USB instead; there >> where talks that Ubuntu would support this starting at 11.10. > Interesting. I'll have to look further into how that would work. Also, > why wouldn't they just tell Mac users to do that? > > In the meantime I think I may have stumbled on the solution to the > script: In the midst of all the output it mentions "usage realpath > [-q] path". I wasn't 100% sure exactly what that meant, but I put the > full path to the iso and a full path to an img file and I *think* that > worked. I've yet to test the result; and I have no idea of the '-q' > option.... Nada. Booted it, and all I got was a blinking cursor, and I couldn't even type "hello world!" :( Back to square one... I guess I'll try a direct copy. From owner-freebsd-hackers@FreeBSD.ORG Wed Mar 21 15:53:07 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B8D591065674 for ; Wed, 21 Mar 2012 15:53:07 +0000 (UTC) (envelope-from feld@feld.me) Received: from feld.me (unknown [IPv6:2607:f4e0:100:300::2]) by mx1.freebsd.org (Postfix) with ESMTP id 80ACB8FC0C for ; Wed, 21 Mar 2012 15:53:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=feld.me; s=blargle; h=In-Reply-To:Message-Id:From:Mime-Version:Date:References:Subject:To:Content-Type; bh=c1MAX+3AiD8CL8M3aLd9JAjBJrORrzFYRWHrI5SmgXs=; b=oEUmMlae0btD7psyUAvvy3ITAh7GCVHz5LdtzoU3F3K3jlmhh6gIf7K6NINY/Ri4MlzjMPsJN7j8HAGyK0Sq2RdZLjlS5bAeXgwf2IqqSZbilR0gGw3w4DzteN8mg0JT; Received: from localhost ([127.0.0.1] helo=mwi1.coffeenet.org) by feld.me with esmtp (Exim 4.77 (FreeBSD)) (envelope-from ) id 1SANqO-00042S-LQ for freebsd-hackers@freebsd.org; Wed, 21 Mar 2012 10:53:07 -0500 Received: from feld@feld.me by mwi1.coffeenet.org (Archiveopteryx 3.1.4) with esmtpsa id 1332345174-34990-34989/5/42; Wed, 21 Mar 2012 15:52:54 +0000 Content-Type: text/plain; charset=utf-8; format=flowed; delsp=yes To: freebsd-hackers@freebsd.org References: <4F699391.9070804@herveybayaustralia.com.au> Date: Wed, 21 Mar 2012 10:52:53 -0500 Mime-Version: 1.0 From: Mark Felder Message-Id: In-Reply-To: <4F699391.9070804@herveybayaustralia.com.au> User-Agent: Opera Mail/11.62 (FreeBSD) X-SA-Score: -1.5 Subject: Re: iso2flash img X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Mar 2012 15:53:07 -0000 As an alternative I recently purchased a Zalman ZM-VE200 device (there's also a USB3.0 flavor) that lets you copy ISOs to it and it will emulate a CDROM/DVDROM/BDROM for you so you never have to deal with this mess again. It works amazingly well. I was tired of fighting this problem and this is an amazing solution -- I can keep every ISO I ever need on a single drive. http://www.zalman.com/eng/product/Product_Read.asp?idx=431 http://www.zalman.com/eng/product/Product_Read.asp?idx=459 http://www.rmprepusb.com/tutorials/ve200 Hope that helps someone.... From owner-freebsd-hackers@FreeBSD.ORG Wed Mar 21 16:17:37 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B62C2106564A for ; Wed, 21 Mar 2012 16:17:37 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 6DBB58FC29 for ; Wed, 21 Mar 2012 16:17:37 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) by cyrus.watson.org (Postfix) with ESMTPSA id 247C346B0D; Wed, 21 Mar 2012 12:17:37 -0400 (EDT) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 87B21B960; Wed, 21 Mar 2012 12:17:36 -0400 (EDT) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Wed, 21 Mar 2012 11:19:42 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p10; KDE/4.5.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201203211119.43022.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 21 Mar 2012 12:17:36 -0400 (EDT) Cc: Mark Saad , Sergey Kandaurov Subject: Re: Approaching the limit on PV entries X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Mar 2012 16:17:37 -0000 On Tuesday, March 20, 2012 11:37:57 am Sergey Kandaurov wrote: > On 22 November 2011 19:29, Mark Saad wrote: > > Hello All > > [found this mail in my drafts, not sure if my answer is still useful] > > > I want to get to the bottom of a warning in dmesg. On 7.2-RELEASE and > > 7.3-RELEASE I have seen the following warning in dmesg. > > > > Approaching the limit on PV entries, consider increasing either the > > vm.pmap.shpgperproc or the vm.pmap.pv_entry_max sysctl. > > > > So looking around I see a few posts here and there about how to tune > > the sysctls to address the warning however I am not 100% sure what > > each value does. > > It appears changing vm.pmap.shpgperproc affects the value of > > vm.pmap.pv_entry_max . Can someone explain the relationship of the two > > sysctls. Also > > This is how they are calculated. > > pv_entry_max = shpgperproc * maxproc + cnt.v_page_count; > > and, respectively, > > shpgperproc = (pv_entry_max - cnt.v_page_count) / maxproc; > > So, changing one sysctl will change another and vice versa. > > > what pitfalls of changing them are. > > Not known to me (on amd64 platform). > I have had vm.pmap.shpgperproc=15000 on 8.1 amd64 with 4G RAM > to make some badly written commercial software to work until it > was decommissioned to the scrap. FYI, Alan just removed this warning and the associated sysctls from HEAD yesterday because they were made obsolete several years ago. I think they are obsolete even on 7. Certainly on 8. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Wed Mar 21 16:39:16 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3FD15106564A; Wed, 21 Mar 2012 16:39:16 +0000 (UTC) (envelope-from pluknet@gmail.com) Received: from mail-lpp01m010-f54.google.com (mail-lpp01m010-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id 7FC158FC0C; Wed, 21 Mar 2012 16:39:15 +0000 (UTC) Received: by lagv3 with SMTP id v3so1276283lag.13 for ; Wed, 21 Mar 2012 09:39:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=BrOawhPno8Jpwp6zLgHfBz8reGWumz2U+GdHLiB3Pw0=; b=doHkaa6X9Bkfp6A/sG5QE1DrBNkzqi0WTRZDTvkDUrKuQnCbN8u00JtbMCRcJ5tqVB KJQmrjKPU7zfnemAB1WvdNhIv0v36hAT/YvJnBer8SsbGQZEpZDsTdH6tXUsSO7nvpIv JsLCPTtSjGnIuHU9EXLQQk7PvycqwIXtuQ1ahqiTtn3k5Qj3v09gKWMF4JoCXg7ml+vK +uQXVUZ2aGvFTRyYvRtZmijlNUvRnxVCgLRlAw2btKPKbE5xrDC8EeUSk0Dpwc0maQ2C R9DKMzosQzyAqx8RIRT+0iOhy/ZX0mYoFjqSnM7BK0ITAR4sJ8mShPKlnlLQmn1WLeIO JHvw== MIME-Version: 1.0 Received: by 10.112.100.232 with SMTP id fb8mr1632503lbb.86.1332347954216; Wed, 21 Mar 2012 09:39:14 -0700 (PDT) Received: by 10.152.21.73 with HTTP; Wed, 21 Mar 2012 09:39:14 -0700 (PDT) In-Reply-To: <201203211119.43022.jhb@freebsd.org> References: <201203211119.43022.jhb@freebsd.org> Date: Wed, 21 Mar 2012 19:39:14 +0300 Message-ID: From: Sergey Kandaurov To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org, Mark Saad Subject: Re: Approaching the limit on PV entries X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Mar 2012 16:39:16 -0000 On 21 March 2012 19:19, John Baldwin wrote: > On Tuesday, March 20, 2012 11:37:57 am Sergey Kandaurov wrote: >> On 22 November 2011 19:29, Mark Saad wrote: >> > Hello All >> >> [found this mail in my drafts, not sure if my answer is still useful] >> >> > =A0I want to get to the bottom of a warning in dmesg. On 7.2-RELEASE a= nd >> > 7.3-RELEASE I have seen the following warning in dmesg. >> > >> > Approaching the limit on PV entries, consider increasing either the >> > vm.pmap.shpgperproc or the vm.pmap.pv_entry_max sysctl. >> > >> > So looking around I see a few posts here and there about how to tune >> > the sysctls to address the warning however I am not 100% sure what >> > each value does. >> > It appears changing vm.pmap.shpgperproc affects the value of >> > vm.pmap.pv_entry_max . Can someone explain the relationship of the two >> > sysctls. Also >> >> This is how they are calculated. >> >> pv_entry_max =3D shpgperproc * maxproc + cnt.v_page_count; >> >> and, respectively, >> >> shpgperproc =3D (pv_entry_max - cnt.v_page_count) / maxproc; >> >> So, changing one sysctl will change another and vice versa. >> >> > what pitfalls of changing them are. >> >> Not known to me (on amd64 platform). >> I have had vm.pmap.shpgperproc=3D15000 on 8.1 amd64 with 4G RAM >> to make some badly written commercial software to work until it >> was decommissioned to the scrap. > > FYI, Alan just removed this warning and the associated sysctls from HEAD > yesterday because they were made obsolete several years ago. =A0I think t= hey are > obsolete even on 7. =A0Certainly on 8. Yep, and since switching to direct map (somewhere around 7.x on amd64?) made PV entry limit factually obsolete, this is really cool. --=20 wbr, pluknet From owner-freebsd-hackers@FreeBSD.ORG Wed Mar 21 16:42:44 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C83F21065676 for ; Wed, 21 Mar 2012 16:42:44 +0000 (UTC) (envelope-from simond@irrelevant.org) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 774E38FC17 for ; Wed, 21 Mar 2012 16:42:44 +0000 (UTC) Received: by vcmm1 with SMTP id m1so1704954vcm.13 for ; Wed, 21 Mar 2012 09:42:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=irrelevant.org; s=irrelevant; h=mime-version:x-originating-ip:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=CjISDOc5NXHCLwfKYZKXWOnH66JcFfP1tLoCXyXITQw=; b=Ag/iSq6wSin4bzk2PyiCEsX5ImAOivl3GDMjoeA+EFrSoHEws8tnQPNoYT/mji2iKS 5iko1gcYW/PwWzSc/LgTiHHtGsnBLIRoXpyqst/4UCQfHmd4fk/NjTuyKcZgeW4BuMSH C9PfKAmY9yjkUTiSntP71i3CeLdr4EHtNZgJ8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-originating-ip:in-reply-to:references:date :message-id:subject:from:to:cc:content-type:x-gm-message-state; bh=CjISDOc5NXHCLwfKYZKXWOnH66JcFfP1tLoCXyXITQw=; b=c1rpZrVMjj6S8F7n7xRNmn9pwXhsdLMtFxRhEgPB1ZYwMn7kBmRDrXjJ7ZQYdWo+E8 Rb+k2Ama9nw9Ahyfm3abRigTSJtJH88zGACSoA1GkLJguFOm7GGZ6v/iIMBrEAF79Z6i L8C1VHBuR9c3Dv4912FFtkNAkG6MqJfQbOgSSAja1Z5m73kiBsmN4lQc30zTX4RBxE1J fGGKpkQjVPdr1/zWH7mWa9XEFUR/cNIUsiBJO10vr/Gu7zYIadhdDnXIPOCTxvJdk3Ch HK/sJ/AdDMm3I6ylcdjVhznFzA+mRD3SQAF7geKnnrIsYgn/BeDn8CCEbpJXqKlq2MLV cV8w== MIME-Version: 1.0 Received: by 10.220.151.67 with SMTP id b3mr2149474vcw.51.1332348158585; Wed, 21 Mar 2012 09:42:38 -0700 (PDT) Received: by 10.52.173.68 with HTTP; Wed, 21 Mar 2012 09:42:38 -0700 (PDT) X-Originating-IP: [87.82.211.34] In-Reply-To: References: <4F699391.9070804@herveybayaustralia.com.au> Date: Wed, 21 Mar 2012 16:42:38 +0000 Message-ID: From: Simon Dick To: Mark Felder Content-Type: text/plain; charset=ISO-8859-1 X-Gm-Message-State: ALoCoQn/ZKK01FFHqwb2K/L6arMRh5Sp1Y8xU+pc7eoG5IExOVA/KyKvYxwC4VFrymsvYtDBfdDW Cc: freebsd-hackers@freebsd.org Subject: Re: iso2flash img X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Mar 2012 16:42:44 -0000 On 21 March 2012 15:52, Mark Felder wrote: > As an alternative I recently purchased a Zalman ZM-VE200 device (there's > also a USB3.0 flavor) that lets you copy ISOs to it and it will emulate a > CDROM/DVDROM/BDROM for you so you never have to deal with this mess again. > It works amazingly well. I was tired of fighting this problem and this is an > amazing solution -- I can keep every ISO I ever need on a single drive. > > http://www.zalman.com/eng/product/Product_Read.asp?idx=431 > http://www.zalman.com/eng/product/Product_Read.asp?idx=459 > http://www.rmprepusb.com/tutorials/ve200 > > > Hope that helps someone.... Have to say that I have one too and have had no problems with it yet :) From owner-freebsd-hackers@FreeBSD.ORG Wed Mar 21 20:20:19 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4CF21106564A for ; Wed, 21 Mar 2012 20:20:19 +0000 (UTC) (envelope-from nonesuch@longcount.org) Received: from mail-lpp01m010-f54.google.com (mail-lpp01m010-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id BDFF48FC14 for ; Wed, 21 Mar 2012 20:20:18 +0000 (UTC) Received: by lagv3 with SMTP id v3so1491190lag.13 for ; Wed, 21 Mar 2012 13:20:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-originating-ip:in-reply-to:references:date :message-id:subject:from:to:content-type:content-transfer-encoding :x-gm-message-state; bh=hzqE6oj1ClX9IJH/aAZW/ym4g9edHkbiGECnB5j3HpM=; b=UOSXoEL3cZaLJsIq0NZua3UjCbDK9qGdlwAYfXA1T+zJ+6zX1zao+nJNC9pd+3ondC UMDB9GQy+d8CSGQmXjKkeV9ZYwKVjV0YptLyPgaRX/v1cBUUzo9VoGtw3UQ0BIEUbxAz xDdCqxSoq7YxmEGx2QLIy49tmQ0+IKiUl7iVBbWmiFhG1fb7P3z+LXJ/DRPnHdze7598 sjWaJrogMe6PWs6F44G7P0OpiIsk7HquTRE7Ay7Eo5yOrzha0hsJhIQ0opbJkiluEFj9 E7iwXHhkf9jzIg70tAOYkYrsj20eFxZmW4ECJCLiBLPpbjeVzKjlA+pC5uLpOnhD3KT6 MEgg== MIME-Version: 1.0 Received: by 10.112.83.105 with SMTP id p9mr1924123lby.43.1332361217575; Wed, 21 Mar 2012 13:20:17 -0700 (PDT) Received: by 10.112.36.135 with HTTP; Wed, 21 Mar 2012 13:20:17 -0700 (PDT) X-Originating-IP: [216.223.13.128] In-Reply-To: References: <201203211119.43022.jhb@freebsd.org> Date: Wed, 21 Mar 2012 16:20:17 -0400 Message-ID: From: Mark Saad To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Gm-Message-State: ALoCoQnw/jcvPSnRSCr+eyko4s5a8jRSjG3ZUtjlLZ+OpKUNOUUY/p1oa+Y6zEv5bprNIciezwiX Subject: Re: Approaching the limit on PV entries X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Mar 2012 20:20:19 -0000 On Wed, Mar 21, 2012 at 12:39 PM, Sergey Kandaurov wrot= e: > On 21 March 2012 19:19, John Baldwin wrote: >> On Tuesday, March 20, 2012 11:37:57 am Sergey Kandaurov wrote: >>> On 22 November 2011 19:29, Mark Saad wrote: >>> > Hello All >>> >>> [found this mail in my drafts, not sure if my answer is still useful] >>> >>> > =C2=A0I want to get to the bottom of a warning in dmesg. On 7.2-RELEA= SE and >>> > 7.3-RELEASE I have seen the following warning in dmesg. >>> > >>> > Approaching the limit on PV entries, consider increasing either the >>> > vm.pmap.shpgperproc or the vm.pmap.pv_entry_max sysctl. >>> > >>> > So looking around I see a few posts here and there about how to tune >>> > the sysctls to address the warning however I am not 100% sure what >>> > each value does. >>> > It appears changing vm.pmap.shpgperproc affects the value of >>> > vm.pmap.pv_entry_max . Can someone explain the relationship of the tw= o >>> > sysctls. Also >>> >>> This is how they are calculated. >>> >>> pv_entry_max =3D shpgperproc * maxproc + cnt.v_page_count; >>> >>> and, respectively, >>> >>> shpgperproc =3D (pv_entry_max - cnt.v_page_count) / maxproc; >>> >>> So, changing one sysctl will change another and vice versa. >>> >>> > what pitfalls of changing them are. >>> >>> Not known to me (on amd64 platform). >>> I have had vm.pmap.shpgperproc=3D15000 on 8.1 amd64 with 4G RAM >>> to make some badly written commercial software to work until it >>> was decommissioned to the scrap. >> >> FYI, Alan just removed this warning and the associated sysctls from HEAD >> yesterday because they were made obsolete several years ago. =C2=A0I thi= nk they are >> obsolete even on 7. =C2=A0Certainly on 8. > > Yep, and since switching to direct map (somewhere around 7.x on amd64?) > made PV entry limit factually obsolete, this is really cool. > > -- > wbr, > pluknet Interesting so this warning is relevant in 7.x ? --=20 mark saad | nonesuch@longcount.org From owner-freebsd-hackers@FreeBSD.ORG Wed Mar 21 20:38:37 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 394991065670 for ; Wed, 21 Mar 2012 20:38:37 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 6DD928FC16 for ; Wed, 21 Mar 2012 20:38:36 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q2LKcT90068141; Wed, 21 Mar 2012 22:38:29 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q2LKcTq0055106; Wed, 21 Mar 2012 22:38:29 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q2LKcSsk055105; Wed, 21 Mar 2012 22:38:28 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 21 Mar 2012 22:38:28 +0200 From: Konstantin Belousov To: Svatopluk Kraus Message-ID: <20120321203828.GW2358@deviant.kiev.zoral.com.ua> References: <20120312181921.GF75778@deviant.kiev.zoral.com.ua> <20120315112959.GP75778@deviant.kiev.zoral.com.ua> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="j2Klb18PAKd8hQ5U" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: hackers@freebsd.org Subject: Re: [vfs] buf_daemon() slows down write() severely on low-speed CPU X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Mar 2012 20:38:37 -0000 --j2Klb18PAKd8hQ5U Content-Type: text/plain; charset=koi8-r Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Mar 15, 2012 at 08:00:41PM +0100, Svatopluk Kraus wrote: > 2012/3/15 Konstantin Belousov : > > On Tue, Mar 13, 2012 at 01:54:38PM +0100, Svatopluk Kraus wrote: > >> On Mon, Mar 12, 2012 at 7:19 PM, Konstantin Belousov > >> wrote: > >> > On Mon, Mar 12, 2012 at 04:00:58PM +0100, Svatopluk Kraus wrote: > >> >> Hi, > >> >> > >> >> =9A =9AI have solved a following problem. If a big file (according = to > >> >> 'hidirtybuffers') is being written, the write speed is very poor. > >> >> > >> >> =9A =9AIt's observed on system with elan 486 and 32MB RAM (i.e., lo= w speed > >> >> CPU and not too much memory) running FreeBSD-9. > >> >> > >> >> =9A =9AAnalysis: A file is being written. All or almost all dirty b= uffers > >> >> belong to the file. The file vnode is almost all time locked by > >> >> writing process. The buf_daemon() can not flush any dirty buffer as= a > >> >> chance to acquire the file vnode lock is very low. A number of dirty > >> >> buffers grows up very slow and with each new dirty buffer slower, > >> >> because buf_daemon() eats more and more CPU time by looping on dirty > >> >> buffers queue (with very low or no effect). > >> >> > >> >> =9A =9AThis slowing down effect is started by buf_daemon() itself, = when > >> >> 'numdirtybuffers' reaches 'lodirtybuffers' threshold and buf_daemon= () > >> >> is waked up by own timeout. The timeout fires at 'hz' period, but > >> >> starts to fire at 'hz/10' immediately as buf_daemon() fails to reach > >> >> 'lodirtybuffers' threshold. When 'numdirtybuffers' (now slowly) > >> >> reaches ((lodirtybuffers + hidirtybuffers) / 2) threshold, the > >> >> buf_daemon() can be waked up within bdwrite() too and it's much wor= se. > >> >> Finally and with very slow speed, the 'hidirtybuffers' or > >> >> 'dirtybufthresh' is reached, the dirty buffers are flushed, and > >> >> everything starts from beginning... > >> > Note that for some time, bufdaemon work is distributed among bufdaem= on > >> > thread itself and any thread that fails to allocate a buffer, esp. > >> > a thread that owns vnode lock and covers long queue of dirty buffers. > >> > >> However, the problem starts when numdirtybuffers reaches > >> lodirtybuffers count and ends around hidirtybuffers count. There are > >> still plenty of free buffers in system. > >> > >> >> > >> >> =9A =9AOn the system, a buffer size is 512 bytes and the default > >> >> thresholds are following: > >> >> > >> >> =9A =9Avfs.hidirtybuffers =3D 134 > >> >> =9A =9Avfs.lodirtybuffers =3D 67 > >> >> =9A =9Avfs.dirtybufthresh =3D 120 > >> >> > >> >> =9A =9AFor example, a 2MB file is copied into flash disk in about 3 > >> >> minutes and 15 second. If dirtybufthresh is set to 40, the copy time > >> >> is about 20 seconds. > >> >> > >> >> =9A =9AMy solution is a mix of three things: > >> >> =9A =9A1. Suppresion of buf_daemon() wakeup by setting bd_request t= o 1 in > >> >> the main buf_daemon() loop. > >> > I cannot understand this. Please provide a patch that shows what do > >> > you mean there. > >> > > >> =9A =9A =9A curthread->td_pflags |=3D TDP_NORUNNINGBUF | TDP_BUFNEED; > >> =9A =9A =9A mtx_lock(&bdlock); > >> =9A =9A =9A for (;;) { > >> - =9A =9A =9A =9A =9A =9A bd_request =3D 0; > >> + =9A =9A =9A =9A =9A =9A bd_request =3D 1; > >> =9A =9A =9A =9A =9A =9A =9A mtx_unlock(&bdlock); > > Is this a complete patch ? The change just causes lost wakeups for bufd= aemon, > > nothing more. > Yes, it's a complete patch. And exactly, it causes lost wakeups which are: > 1. !! UNREASONABLE !!, because bufdaemon is not sleeping, > 2. not wanted, because it looks that it's correct behaviour for the > sleep with hz/10 period. However, if the sleep with hz/10 period is > expected to be waked up by bd_wakeup(), then bd_request should be set > to 0 just before sleep() call, and then bufdaemon behaviour will be > clear. No, your description is wrong. If bufdaemon is unable to flush enough buffers and numdirtybuffers still greater then lodirtybuffers, then bufdaemon enters qsleep state without resetting bd_request, with timeouts of one tens of second. Your patch will cause all wakeups for this case to be lost. This is exactly the situation when we want bufdaemon to run harder to avoid possible deadlocks, not to slow down. >=20 > All stuff around bd_request and bufdaemon sleep is under bd_lock, so > if bd_request is 0 and bufdaemon is not sleeping, then all wakeups are > unreasonable! The patch is about that mainly. Wakeups itself are very cheap for the running process. Mostly, it comes down to locking sleepq and waking all threads that are present in the sleepq blocked queue. If there is no threads in queue, nothing is done. >=20 > > > >> > >> I read description of bd_request variable. However, bd_request should > >> serve as an indicator that buf_daemon() is in sleep. I.e., the > >> following paradigma should be used: > >> > >> mtx_lock(&bdlock); > >> bd_request =3D 0; =9A =9A/* now, it's only time when wakeup() will be = meaningful */ > >> sleep(&bd_request, ..., hz/10); > >> bd_request =3D 1; =9A /* in case of timeout, we must set it (bd_wakeup= () > >> already set it) */ > >> mtx_unlock(&bdlock); > >> > >> My patch follows the paradigma. What happens without the patch in > >> described problem: buf_daemon() fails in its job and goes to sleep > >> with hz/10 period. It supposes that next early wakeup will do nothing > >> too. bd_request is untouched but buf_daemon() doesn't know if its last > >> wakeup was made by bd_wakeup() or by timeout. So, bd_request could be > >> 0 and buf_daemon() can be waked up before hz/10 just by bd_wakeup(). > >> Moreover, setting bd_request to 0 when buf_daemon() is not in sleep > >> can cause time consuming and useless wakeup() calls without effect. --j2Klb18PAKd8hQ5U Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAk9qPEQACgkQC3+MBN1Mb4gF3gCglmQi1NLIKxyk7VPkLC3Ug6q4 VbkAoJm/kWup/q+dhb2JI7I5JJud2HTd =gBvo -----END PGP SIGNATURE----- --j2Klb18PAKd8hQ5U-- From owner-freebsd-hackers@FreeBSD.ORG Wed Mar 21 23:28:29 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DBBF4106564A for ; Wed, 21 Mar 2012 23:28:29 +0000 (UTC) (envelope-from eric.saintetienne@gmail.com) Received: from mail-vb0-f54.google.com (mail-vb0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 9A05E8FC12 for ; Wed, 21 Mar 2012 23:28:29 +0000 (UTC) Received: by vbmv11 with SMTP id v11so1021310vbm.13 for ; Wed, 21 Mar 2012 16:28:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to:content-type :content-transfer-encoding; bh=W4jJ2X7w+dpSBojDkTRUYvaYAWk5CH8W93UWiYXZ+ic=; b=ND/0W84cPzUFZsZeVjSKDkdgCHs2Lc9MwzKiVgb9wosJmGpYzuOvPKY/a9CV38nrcO 27v6nDkpMQArceHMzm+PhygdnYOdwiji5ytuFS9Xi4HU/YmQ6P14HUYvJdOndh3b+VJf qzx1jo7dwr/TjixJZeT3xMqs49Le11Merw/i+aTIpaTF7Uw/eQvWCSOounFIWTmhCogK VtEeugH7853iP375pGKL8fnJ2m7Rrl3pq/Y22X4oe9lAa60gPwDMRj2t+ieJ51pDA6y5 1Mkr/1tMi1J17rh1EkVIO4dStKKrTt3okt2hOOcQVgCgBW5w/SfcZKecPuOUlChOy+ek AJJA== Received: by 10.52.29.244 with SMTP id n20mr2344381vdh.22.1332372508975; Wed, 21 Mar 2012 16:28:28 -0700 (PDT) MIME-Version: 1.0 Received: by 10.220.149.9 with HTTP; Wed, 21 Mar 2012 16:27:58 -0700 (PDT) From: Eric Saint-Etienne Date: Wed, 21 Mar 2012 23:27:58 +0000 Message-ID: To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: malloc pages map to user space X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Mar 2012 23:28:29 -0000 Hi, >From within the freeBSD kernel, not all malloc are made equal: * malloc() smaller than=C2=A0KMEM_ZMAX (set to=C2=A0one page size) end up = in UMA SLABs, themselves laid out in powers of 2 (from 16 bytes, 32... to 4096 bytes) * bigger malloc() are done through=C2=A0uma_large_malloc() which uses the kmem wired space In my driver, I need to map some malloc-ed memory, obtained from another module, into userspace. The problem: on the smaller mallocs, as well as on some bigeer ones (8k seems fine, 64k fails): vm_map_lookup() fails finding the underlying vm object. Do somebody know how (or better, have a piece of code!) to retrieve the vm_object associated with malloc-ed memory? (small and big ones) As far as I can see in the vm code, there isn't any object associated with the slabs (the smaller mallocs), it seems that a huge chunk of virtual space is used "as is", so I presume the virtual addresses where the SLABs are have some remarkable property, with respect to physical addresses, that could allow creating an object from scratch? The usual answer is: use mmap(). It seems mmap() is the solution to everything. But what I dislike with mmap() is the following cost *for each page*: 1/ a page fault 2/ a call to a pager function that will do the "on demand" mapping. Thank you. Eric From owner-freebsd-hackers@FreeBSD.ORG Wed Mar 21 23:35:14 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2ECF51065673 for ; Wed, 21 Mar 2012 23:35:14 +0000 (UTC) (envelope-from sushanth_rai@yahoo.com) Received: from nm3.bullet.mail.sp2.yahoo.com (nm3.bullet.mail.sp2.yahoo.com [98.139.91.73]) by mx1.freebsd.org (Postfix) with SMTP id 064C78FC15 for ; Wed, 21 Mar 2012 23:35:13 +0000 (UTC) Received: from [98.139.91.65] by nm3.bullet.mail.sp2.yahoo.com with NNFMP; 21 Mar 2012 23:35:13 -0000 Received: from [98.139.44.68] by tm5.bullet.mail.sp2.yahoo.com with NNFMP; 21 Mar 2012 23:35:13 -0000 Received: from [127.0.0.1] by omp1005.access.mail.sp2.yahoo.com with NNFMP; 21 Mar 2012 23:35:13 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 546240.61140.bm@omp1005.access.mail.sp2.yahoo.com Received: (qmail 39969 invoked by uid 60001); 21 Mar 2012 23:35:13 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1332372913; bh=9NpSRjHEJlUjR8pnDPQfOgHolbaPfp3ejbk98miTrls=; h=X-YMail-OSG:Received:X-Mailer:Message-ID:Date:From:Subject:To:MIME-Version:Content-Type; b=U0/FHIY83Z5HKbNuAXJp3MaYmKm12ju8k5gcz+3dWMM6HHpYGEB4DfU1crZm6mpmvSXFEWd7ECu+RoUQ8SI5WQTXKGC2nP1a9RbWyjufle0zvfpmUkfoX+rsOa4WVEcdWTtuuuYSND2EDQsIezpEyMHkIF2FOtHoR1zrYaa9kF8= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Mailer:Message-ID:Date:From:Subject:To:MIME-Version:Content-Type; b=E9fGrx9CxAcdlhYt5iqMkHZ3fjQw8NLak5z/45OcN4pRVddtqXRCGzHiclcfjdgVsK4UrwDCZLXnBguJmets+ibjDf2EtNJ7D80ojgkYyDM9xeTCjK1GVQ8qTr0oPeG/9Ud8uv4C2sYq2WWdPdLP5WDBaVWZo6xZEF4bmPIyT/U=; X-YMail-OSG: M9z4j10VM1kOh73fHBL.k5XWkr6YUA01ERrxBX4z6nwR93H iL1qhLPOz_GJjspZFxllfVUDx.k8dSbqOj6VG5cR2oOHHhVMQN01XmBIfUda nvH7zj8okXpJzXXvn_ifnmjOFQqWw4.Ni3apuGYEy8f0VjqSuF9oVvUWAnc1 fVmRGFRqiUnYHvDbRomg_yNkY.tUlwMKLC.35LGiH3tkH1aAXGC88MwcmsSM B5RYTWr0S0udPoosvHvpaTRvMCUakUWF40rGnJpVm5r9JSJAqQQfHeJEKB2A _0Mer8Rppe04VTkYrjX4vIv4ZZ5w8JLL.tBwVs3PXI9m8WK3JiVNjP_.5Npu m19IpLHg3J89UH_vxh114X87BSy9q9.l9gbgpyb34zQJGFWySdFEjwNu7.3L o_QVbjw_cdI7NIIGOiFSJ6aHXXYDNQ8SGobAtEuYhCkpZMGb0HkyXw6W2z17 vuUcEh6E- Received: from [209.119.38.67] by web180013.mail.gq1.yahoo.com via HTTP; Wed, 21 Mar 2012 16:35:13 PDT X-Mailer: YahooMailClassic/15.0.5 YahooMailWebService/0.8.116.338427 Message-ID: <1332372913.7195.YahooMailClassic@web180013.mail.gq1.yahoo.com> Date: Wed, 21 Mar 2012 16:35:13 -0700 (PDT) From: Sushanth Rai To: freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: Improving gcore X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Mar 2012 23:35:14 -0000 Sometimes I have trouble capturing the "correct" state of a multithreaded process using gcore. That is, it looks like target process might have done some work since the time command was issued and the core file was generated. Looking at the code, gcore calls ptrace(PT_ATTACH...), which internally issues SIGSTOP, and calls waitpid() to wait until the process stops. So, it's quite possible that some threads that are not sleeping interruptibly will continue to run until the process notices the signal. Signals are only checked when a thread that is tagged to handle the signal crosses the user boundary (return from syscall, trap). When the thread finally handles SIGSTOP, it needs to stop all threads, which is done by lighting a flag-bit it each thread. This bit is checked as each thread crosses the user boundary. So, there will always be some state change in the target process from the time SIGSTOP is posted to the time all threads are actually stopped. I was wondering if I could improve this a bit by calling PT_SUSPEND on all threads, instead of posting SIGSTOP and waiting for all threads to stop. Once the core is generated, unsuspend all threads. As with SIGSTOP, individual thread will only notice suspension as they cross user boundary. But there is no overhead of tagging a thread to handle the signal and that thread doing the suspension. The idea is to try and generate the core file which reflects the running state of the process as closely as possible. Does this sound reasonable ? Thanks, Sushanth From owner-freebsd-hackers@FreeBSD.ORG Thu Mar 22 02:25:23 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AFD16106564A for ; Thu, 22 Mar 2012 02:25:23 +0000 (UTC) (envelope-from freebsd-hackers@herveybayaustralia.com.au) Received: from mail.unitedinsong.com.au (mail.unitedinsong.com.au [150.101.178.33]) by mx1.freebsd.org (Postfix) with ESMTP id 658138FC08 for ; Thu, 22 Mar 2012 02:25:23 +0000 (UTC) Received: from laptop1.herveybayaustralia.com.au (laptop1.herveybayaustralia.com.au [192.168.0.182]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.unitedinsong.com.au (Postfix) with ESMTPSA id A307F5C22 for ; Thu, 22 Mar 2012 12:38:55 +1000 (EST) Message-ID: <4F6A8C4B.2040605@herveybayaustralia.com.au> Date: Thu, 22 Mar 2012 12:19:55 +1000 From: Da Rock User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:7.0.1) Gecko/20111109 Thunderbird/7.0.1 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org References: <4F699391.9070804@herveybayaustralia.com.au> <4F69CFAC.7030501@gmail.com> In-Reply-To: <4F69CFAC.7030501@gmail.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: iso2flash img X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Mar 2012 02:25:23 -0000 On 03/21/12 22:55, Vitaly Magerya wrote: > Da Rock wrote: >> I googled a bit and found an old post here from Luigi >> (http://osdir.com/ml/freebsd-hackers/2008-11/msg00245.html) which had a >> script to do this, but I'm having trouble with it- is anyone familiar >> with this? I'm on a bit of a deadline... > Can't help you with that script (I failed to make it work too), but you > might want to try to dd the iso image directly onto USB instead; there > where talks that Ubuntu would support this starting at 11.10. Nada. Tried that and it didn't work. I'm not sure how that would work given that it uses isolinux to boot- ergo needs a cd to load the kernel. Maybe some way to determine the install media? Bit of an unnecessary complication though if you ask me... just make an iso _and_ an img. I'm sure canonical are not that strapped for resources... From owner-freebsd-hackers@FreeBSD.ORG Thu Mar 22 08:03:40 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E8A9A1065680 for ; Thu, 22 Mar 2012 08:03:40 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id A92418FC20 for ; Thu, 22 Mar 2012 08:03:40 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id 3AB7D7300A; Thu, 22 Mar 2012 09:22:28 +0100 (CET) Date: Thu, 22 Mar 2012 09:22:28 +0100 From: Luigi Rizzo To: Da Rock Message-ID: <20120322082228.GA48911@onelab2.iet.unipi.it> References: <4F699391.9070804@herveybayaustralia.com.au> <4F69CFAC.7030501@gmail.com> <4F69D38E.9070805@herveybayaustralia.com.au> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4F69D38E.9070805@herveybayaustralia.com.au> User-Agent: Mutt/1.4.2.3i Cc: freebsd-hackers@freebsd.org Subject: Re: iso2flash img X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Mar 2012 08:03:41 -0000 On Wed, Mar 21, 2012 at 11:11:42PM +1000, Da Rock wrote: ... > In the meantime I think I may have stumbled on the solution to the > script: In the midst of all the output it mentions "usage realpath [-q] > path". I wasn't 100% sure exactly what that meant, but I put the full > path to the iso and a full path to an img file and I *think* that > worked. I've yet to test the result; and I have no idea of the '-q' > option.... REALPATH(1) FreeBSD General Commands Manual REALPATH(1) NAME realpath -- return resolved physical path SYNOPSIS realpath [-q] path [...] DESCRIPTION The realpath utility uses the realpath(3) function to resolve all sym- bolic links, extra `/' characters and references to /./ and /../ in path. If -q is specified, warnings will not be printed when realpath(3) fails. EXIT STATUS The realpath utility exits 0 on success, and >0 if an error occurs. SEE ALSO realpath(3) HISTORY The realpath utility first appeared in FreeBSD 4.3. FreeBSD 8.1 November 24, 2000 FreeBSD 8.1 From owner-freebsd-hackers@FreeBSD.ORG Thu Mar 22 10:10:41 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id 892151065670; Thu, 22 Mar 2012 10:10:41 +0000 (UTC) Date: Thu, 22 Mar 2012 10:10:41 +0000 From: Alexander Best To: freebsd-hackers@freebsd.org Message-ID: <20120322101041.GA99431@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Subject: still issues with unreadable dmesg output on SMP systems X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Mar 2012 10:10:41 -0000 hi there, a few years ago there were huge issues with SMP and dmesg output, where messages from various drivers were output to /dev/ttyv0 without any timing, which caused a lot of unreadable lines. this was fixed and almost all of the dmesg lines i see now look similar to dmesg on a non-SMP aware kernel. however there still seems to be an issue within the cd(4) driver. maybe somebody could fix the driver output to match the drivers, where the dmesg output looks correct (da(4), ada(4), ...). here's a bit of dmesg output. i'm running HEAD on amd64 btw.: ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 ada0: ATA-7 SATA 2.x device ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 238474MB (488395055 512 byte sectors: 16H 63S/T 16383C) ada1 at ahcich1 bus 0 scbus1 target 0 lun 0 ada1: ATA-8 SATA 2.x device ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada1: Command Queueing enabled ada1: 953869MB (1953525168 512 byte sectors: 16H 63S/T 16383C) SMP: AP CPU #1 Launched! cd0 at ahcich2 bus 0 scbus2 target 0 lun 0 cd0: w " Rfermeoqvuaebnlcey CD1-4R0O6M2 5S0C0S IH-z0 qdueavliityc e1 000 cd0: 150.000MB/s transfers (SATA 1.x, UDMA5, ATAPI 12bytes, PIO 8192bytes) cd0: Attempt to query device size failed: NOT READY, Medium not present - tray closed Root mount waiting for: usbus7 usbus3 Root mount waiting for: usbus7 usbus3 uhub3: 6 ports with 6 removable, self powered uhub7: 6 ports with 6 removable, self powered Root mount waiting for: usbus3 ugen3.2: at usbus3 umass0: on usbus3 da0 at umass-sim0 bus 0 scbus6 target 0 lun 0 da0: Fixed Direct Access SCSI-6 device da0: 40.000MB/s transfers da0: 953837MB (1953458176 512 byte sectors: 255H 63S/T 121597C) again: apart from cd(4) all the drivers i have enabled in my kernel config produce proper dmesg lines. cheers. alex From owner-freebsd-hackers@FreeBSD.ORG Thu Mar 22 10:57:01 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 254B81065670 for ; Thu, 22 Mar 2012 10:57:01 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id D6CCF8FC19 for ; Thu, 22 Mar 2012 10:57:00 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id A641E73027; Thu, 22 Mar 2012 12:15:47 +0100 (CET) Date: Thu, 22 Mar 2012 12:15:47 +0100 From: Luigi Rizzo To: Mark Felder Message-ID: <20120322111547.GD50510@onelab2.iet.unipi.it> References: <4F699391.9070804@herveybayaustralia.com.au> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: freebsd-hackers@freebsd.org Subject: Re: iso2flash img X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Mar 2012 10:57:01 -0000 On Wed, Mar 21, 2012 at 10:52:53AM -0500, Mark Felder wrote: > As an alternative I recently purchased a Zalman ZM-VE200 device (there's > also a USB3.0 flavor) that lets you copy ISOs to it and it will emulate a > CDROM/DVDROM/BDROM for you so you never have to deal with this mess again. > It works amazingly well. I was tired of fighting this problem and this is > an amazing solution -- I can keep every ISO I ever need on a single drive. > > http://www.zalman.com/eng/product/Product_Read.asp?idx=431 > http://www.zalman.com/eng/product/Product_Read.asp?idx=459 > http://www.rmprepusb.com/tutorials/ve200 really nice, thanks for the link. Now if they had something that supported a USB key it would be even nicer... cheers luigi From owner-freebsd-hackers@FreeBSD.ORG Thu Mar 22 11:13:25 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 33FFB106566C for ; Thu, 22 Mar 2012 11:13:25 +0000 (UTC) (envelope-from rodrigo@bebik.net) Received: from smtp3-g21.free.fr (smtp3-g21.free.fr [IPv6:2a01:e0c:1:1599::12]) by mx1.freebsd.org (Postfix) with ESMTP id 8F7818FC0A for ; Thu, 22 Mar 2012 11:13:22 +0000 (UTC) Received: from oldfaithful.bebik.local (unknown [82.227.164.69]) by smtp3-g21.free.fr (Postfix) with ESMTP id 43A4EA61E2; Thu, 22 Mar 2012 12:13:16 +0100 (CET) Received: by oldfaithful.bebik.local (Postfix, from userid 1001) id CAFAD2E6C8; Thu, 22 Mar 2012 12:18:37 +0100 (CET) Date: Thu, 22 Mar 2012 12:18:37 +0100 From: Rodrigo OSORIO To: Luigi Rizzo Message-ID: <20120322111837.GA15952@oldfaithful.bebik.local> References: <4F699391.9070804@herveybayaustralia.com.au> <20120322111547.GD50510@onelab2.iet.unipi.it> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20120322111547.GD50510@onelab2.iet.unipi.it> User-Agent: Mutt/1.4.2.3i Cc: freebsd-hackers@freebsd.org, Mark Felder Subject: Re: iso2flash img X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Mar 2012 11:13:25 -0000 On 22/03/12 12:15 +0100, Luigi Rizzo wrote: > On Wed, Mar 21, 2012 at 10:52:53AM -0500, Mark Felder wrote: > > As an alternative I recently purchased a Zalman ZM-VE200 device (there's > > also a USB3.0 flavor) that lets you copy ISOs to it and it will emulate a > > CDROM/DVDROM/BDROM for you so you never have to deal with this mess again. > > It works amazingly well. I was tired of fighting this problem and this is > > an amazing solution -- I can keep every ISO I ever need on a single drive. > > > > http://www.zalman.com/eng/product/Product_Read.asp?idx=431 > > http://www.zalman.com/eng/product/Product_Read.asp?idx=459 > > http://www.rmprepusb.com/tutorials/ve200 > > really nice, thanks for the link. Now if they had something > that supported a USB key it would be even nicer... > > cheers > luigi > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" This remembers me a recent kickstarter project called isostick :) - rodrigo http://www.kickstarter.com/projects/elegantinvention/isostick-the-optical-drive-in-a-usb-stick From owner-freebsd-hackers@FreeBSD.ORG Thu Mar 22 12:23:56 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 04146106564A for ; Thu, 22 Mar 2012 12:23:56 +0000 (UTC) (envelope-from eric.saintetienne@gmail.com) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id B40C28FC12 for ; Thu, 22 Mar 2012 12:23:55 +0000 (UTC) Received: by yhgm50 with SMTP id m50so1998096yhg.13 for ; Thu, 22 Mar 2012 05:23:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=9ky3pD+y3/8/qOLlf0ofCCDEV5Xc+p5zYoH9R0LlaY4=; b=mUOyemIyJpHjqq+UPhk+/i1RQwNiAXRZlS2GI8H/4U0bH7fdz5FPV4Qmvxsl4rqSYn wdqn9LXlJKvdCHRV/vUgdnqvksHUpcjjWvJY43ukQiF6ga//wOFxhcpfspH++n32xhse astkPMFDJ9HSDUOw0LmfbFf+EWjVLcLLbUtTJ55DV424LKJBoV6v7kAnexxt7tLyASTG vGArzHVi1Fb+2xsx3B1hyWz+xOv6BvjT5XCGw9xpGpMVOmWxdGcraDE8XlYvDNsAgRC3 KYEXy2T+BIJC8N6m7ubtxybUZJ1RuCWTG8Q/5BjeOcGLckapZ7loUkxe/AlDlMum5jBc pFwg== Received: by 10.68.136.162 with SMTP id qb2mr20672817pbb.67.1332419034885; Thu, 22 Mar 2012 05:23:54 -0700 (PDT) MIME-Version: 1.0 Received: by 10.68.47.129 with HTTP; Thu, 22 Mar 2012 05:23:24 -0700 (PDT) In-Reply-To: References: From: Eric Saint-Etienne Date: Thu, 22 Mar 2012 12:23:24 +0000 Message-ID: To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=UTF-8 Subject: Re: malloc pages map to user space X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Mar 2012 12:23:56 -0000 I've refined the behaviour I observe, which isn't consistent depending on the size one mallocates. (see interleaved comments) > In my driver, I need to map some malloc-ed memory, obtained from > another module, into userspace. > > The problem: on the smaller mallocs, as well as on some bigeer ones > (8k seems fine, 64k fails): vm_map_lookup() fails finding the > underlying vm object. In the current implementation I'm calling vm_map_lookup() against the kmem_map. As a result it either return the kmem_object, or fails at all (for smaller or larger mallocs.) > Do somebody know how (or better, have a piece of code!) to retrieve > the vm_object associated with malloc-ed memory? (small and big ones) > > As far as I can see in the vm code, there isn't any object associated > with the slabs (the smaller mallocs), it seems that a huge chunk of > virtual space is used "as is", so I presume the virtual addresses > where the SLABs are have some remarkable property, with respect to > physical addresses, that could allow creating an object from scratch? By using kernel_map instead of kmem_map, vm_map_lookup() now always return a vm_object. That's a big progress. As expected, when this object is kmem_object, the user mapping works fine (for smaller or larger mallocs.) Otherwise that object doesn't match kernel_object. It's an anonymous object to me. Using that "anonymous" vm_object for mapping into user map (using vm_map_find()) doesn't directly fail, it does provide a virtual address in the user map. However I read zeros at that address, from within the user process. Any help would be highly appreciated. Thanks! Eric From owner-freebsd-hackers@FreeBSD.ORG Thu Mar 22 14:01:59 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B3DFC1065670 for ; Thu, 22 Mar 2012 14:01:59 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 425A08FC12 for ; Thu, 22 Mar 2012 14:01:58 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q2ME1pAd007214; Thu, 22 Mar 2012 16:01:51 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q2ME1p2x063767; Thu, 22 Mar 2012 16:01:51 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q2ME1pjE063766; Thu, 22 Mar 2012 16:01:51 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 22 Mar 2012 16:01:51 +0200 From: Konstantin Belousov To: Sushanth Rai Message-ID: <20120322140151.GB2358@deviant.kiev.zoral.com.ua> References: <1332372913.7195.YahooMailClassic@web180013.mail.gq1.yahoo.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="RacQGezy2Y99S6cT" Content-Disposition: inline In-Reply-To: <1332372913.7195.YahooMailClassic@web180013.mail.gq1.yahoo.com> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-hackers@freebsd.org Subject: Re: Improving gcore X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Mar 2012 14:01:59 -0000 --RacQGezy2Y99S6cT Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Wed, Mar 21, 2012 at 04:35:13PM -0700, Sushanth Rai wrote: > Sometimes I have trouble capturing the "correct" state of a > multithreaded process using gcore. That is, it looks like target > process might have done some work since the time command was issued > and the core file was generated. > > Looking at the code, gcore calls ptrace(PT_ATTACH...), which > internally issues SIGSTOP, and calls waitpid() to wait until the > process stops. So, it's quite possible that some threads that are not > sleeping interruptibly will continue to run until the process notices > the signal. Signals are only checked when a thread that is tagged to > handle the signal crosses the user boundary (return from syscall, > trap). When the thread finally handles SIGSTOP, it needs to stop all > threads, which is done by lighting a flag-bit it each thread. This > bit is checked as each thread crosses the user boundary. So, there > will always be some state change in the target process from the time > SIGSTOP is posted to the time all threads are actually stopped. Yes, this is how things work. There are two factors causing the asynchronous stopping: first, other CPUs may execute several threads of the process, so the suspension of that other threads require an IPI to be generated. IPI_AST handler just returns, which causes kernel->usermode transition and possible signal delivery and suspend check. second, kernel never allows to suspend thread executing and blocked in kernel. Doing otherwise would cause deadlocks, because executing threads own resources that are shared with other threads. So, the only safe points to suspend the threads is at kernel->user boundary or at some sleep points that are not marked as unsafe with PBDRY flag. On the other hand, since kernel waits for all threads to suspend before reporting the wait(2) event, the usermode state shall be consistent with itself, or rather, it shall be not worse then if the threads reach the stop point executing asynchronously on different CPUs. See the check for p->p_suspcount == p->p_numthreads in the kern_wait() function before it decides that the found process is satisfactory for wait request. > > I was wondering if I could improve this a bit by calling PT_SUSPEND on > all threads, instead of posting SIGSTOP and waiting for all threads > to stop. Once the core is generated, unsuspend all threads. As with > SIGSTOP, individual thread will only notice suspension as they cross > user boundary. But there is no overhead of tagging a thread to handle > the signal and that thread doing the suspension. The idea is to try > and generate the core file which reflects the running state of the > process as closely as possible. PT_SUSPEND can only be called on the process which you alread attached to. So the call to suspend all threads of the just attached threads is mostly nop for your purposes. > > Does this sound reasonable ? I think you need to describe in more details what do you mean by inconsistent state of the threads in gcore-generated core file, before some conclusion could be made. --RacQGezy2Y99S6cT Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAk9rMM8ACgkQC3+MBN1Mb4jh5wCcCYnkElINR5CMmmLDtol2d4b2 xKYAoNKHyI+iOGS6sCRWqNkREtYJ+2hp =4/WV -----END PGP SIGNATURE----- --RacQGezy2Y99S6cT-- From owner-freebsd-hackers@FreeBSD.ORG Thu Mar 22 14:09:22 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D161D1065678 for ; Thu, 22 Mar 2012 14:09:22 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id A25938FC12 for ; Thu, 22 Mar 2012 14:09:22 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) by cyrus.watson.org (Postfix) with ESMTPSA id 5889646B2C; Thu, 22 Mar 2012 10:09:22 -0400 (EDT) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 9D652B940; Thu, 22 Mar 2012 10:09:21 -0400 (EDT) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Thu, 22 Mar 2012 08:03:56 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p10; KDE/4.5.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <201203220803.57000.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 22 Mar 2012 10:09:21 -0400 (EDT) Cc: Mark Saad Subject: Re: Approaching the limit on PV entries X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Mar 2012 14:09:22 -0000 On Wednesday, March 21, 2012 4:20:17 pm Mark Saad wrote: > On Wed, Mar 21, 2012 at 12:39 PM, Sergey Kandaurov wrote: > > On 21 March 2012 19:19, John Baldwin wrote: > >> On Tuesday, March 20, 2012 11:37:57 am Sergey Kandaurov wrote: > >>> On 22 November 2011 19:29, Mark Saad wrote: > >>> > Hello All > >>> > >>> [found this mail in my drafts, not sure if my answer is still useful] > >>> > >>> > I want to get to the bottom of a warning in dmesg. On 7.2-RELEASE and > >>> > 7.3-RELEASE I have seen the following warning in dmesg. > >>> > > >>> > Approaching the limit on PV entries, consider increasing either the > >>> > vm.pmap.shpgperproc or the vm.pmap.pv_entry_max sysctl. > >>> > > >>> > So looking around I see a few posts here and there about how to tune > >>> > the sysctls to address the warning however I am not 100% sure what > >>> > each value does. > >>> > It appears changing vm.pmap.shpgperproc affects the value of > >>> > vm.pmap.pv_entry_max . Can someone explain the relationship of the two > >>> > sysctls. Also > >>> > >>> This is how they are calculated. > >>> > >>> pv_entry_max = shpgperproc * maxproc + cnt.v_page_count; > >>> > >>> and, respectively, > >>> > >>> shpgperproc = (pv_entry_max - cnt.v_page_count) / maxproc; > >>> > >>> So, changing one sysctl will change another and vice versa. > >>> > >>> > what pitfalls of changing them are. > >>> > >>> Not known to me (on amd64 platform). > >>> I have had vm.pmap.shpgperproc=15000 on 8.1 amd64 with 4G RAM > >>> to make some badly written commercial software to work until it > >>> was decommissioned to the scrap. > >> > >> FYI, Alan just removed this warning and the associated sysctls from HEAD > >> yesterday because they were made obsolete several years ago. I think they are > >> obsolete even on 7. Certainly on 8. > > > > Yep, and since switching to direct map (somewhere around 7.x on amd64?) > > made PV entry limit factually obsolete, this is really cool. > > > > -- > > wbr, > > pluknet > > Interesting so this warning is relevant in 7.x ? No, looks like it was obsolete starting with 7.0. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Thu Mar 22 14:09:24 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1A628106566C for ; Thu, 22 Mar 2012 14:09:24 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id E11FB8FC14 for ; Thu, 22 Mar 2012 14:09:23 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) by cyrus.watson.org (Postfix) with ESMTPSA id 97DE846B0C; Thu, 22 Mar 2012 10:09:23 -0400 (EDT) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 2E266B943; Thu, 22 Mar 2012 10:09:23 -0400 (EDT) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Thu, 22 Mar 2012 08:11:12 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p10; KDE/4.5.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <201203220811.12954.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 22 Mar 2012 10:09:23 -0400 (EDT) Cc: Eric Saint-Etienne Subject: Re: malloc pages map to user space X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Mar 2012 14:09:24 -0000 On Wednesday, March 21, 2012 7:27:58 pm Eric Saint-Etienne wrote: > Hi, > > >From within the freeBSD kernel, not all malloc are made equal: > * malloc() smaller than KMEM_ZMAX (set to one page size) end up in > UMA SLABs, themselves laid out in powers of 2 (from 16 bytes, 32... to > 4096 bytes) > * bigger malloc() are done through uma_large_malloc() which uses the > kmem wired space > > In my driver, I need to map some malloc-ed memory, obtained from > another module, into userspace. > > The problem: on the smaller mallocs, as well as on some bigeer ones > (8k seems fine, 64k fails): vm_map_lookup() fails finding the > underlying vm object. > > Do somebody know how (or better, have a piece of code!) to retrieve > the vm_object associated with malloc-ed memory? (small and big ones) > > As far as I can see in the vm code, there isn't any object associated > with the slabs (the smaller mallocs), it seems that a huge chunk of > virtual space is used "as is", so I presume the virtual addresses > where the SLABs are have some remarkable property, with respect to > physical addresses, that could allow creating an object from scratch? > > The usual answer is: use mmap(). It seems mmap() is the solution to > everything. But what I dislike with mmap() is the following cost *for > each page*: > 1/ a page fault > 2/ a call to a pager function that will do the "on demand" mapping. You can prefault each page in userland after you call mmap() if you want to pay the cost up front vs. later during runtime. However, there is another option you can use (though it might require you to rework your interfaces a bit, esp. depending on how you get the malloc'd memory in the first place). Specifically, I recently added an extension to the 'shm_open()' API in the kernel that lets you map a shared memory object into the kernel (shm_map() and shm_unmap(), the change should be easy to MFC to 8 and 9 (I use it in 8 and will MFC it soonish to those stable branches)). The way the workflow works in this case is that userland creates a shared memory object (usually an anonymous one, so using SHM_ANON), and then passes the fd in as part of an ioctl request to something in the kernel. The kernel code uses fget() to convert the fd to a reference to a 'struct file *'. You can then pass that file to shm_map(). Once the kernel code is done with the mapping it should call shm_unmap() to release it as well as using fdrop() to release the reference to the 'fp' obtained from fget(). shm_map() will wire down pages for the mapped region of the shm (and alloc them if needed). As a bonus then, if you mmap() the shm in userland after the ioctl, you can pass MAP_PREFAULT_READ to mmap() and it should remove all the page faults in userland. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Thu Mar 22 14:09:25 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4947B106566B; Thu, 22 Mar 2012 14:09:25 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 1D3518FC15; Thu, 22 Mar 2012 14:09:25 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) by cyrus.watson.org (Postfix) with ESMTPSA id C701C46B32; Thu, 22 Mar 2012 10:09:24 -0400 (EDT) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 2CF44B959; Thu, 22 Mar 2012 10:09:24 -0400 (EDT) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Thu, 22 Mar 2012 08:13:12 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p10; KDE/4.5.5; amd64; ; ) References: <20120322101041.GA99431@freebsd.org> In-Reply-To: <20120322101041.GA99431@freebsd.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201203220813.13113.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 22 Mar 2012 10:09:24 -0400 (EDT) Cc: Alexander Best Subject: Re: still issues with unreadable dmesg output on SMP systems X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Mar 2012 14:09:25 -0000 On Thursday, March 22, 2012 6:10:41 am Alexander Best wrote: > hi there, > > a few years ago there were huge issues with SMP and dmesg output, where > messages from various drivers were output to /dev/ttyv0 without any timing, > which caused a lot of unreadable lines. > > this was fixed and almost all of the dmesg lines i see now look similar to > dmesg on a non-SMP aware kernel. however there still seems to be an issue > within the cd(4) driver. maybe somebody could fix the driver output to match > the drivers, where the dmesg output looks correct (da(4), ada(4), ...). s/fixed/mostly worked around/ The problem is that the work around isn't perfect. The root cause is still not fixed. avg@ has some ideas on better ways to handle this, but it is a bit tricky to get this right since we also dont' want printfs to be delayed too much (and to fix this perfectly requires buffering printf output). -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Thu Mar 22 14:19:09 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 000D71065670 for ; Thu, 22 Mar 2012 14:19:08 +0000 (UTC) (envelope-from freebsd-hackers@herveybayaustralia.com.au) Received: from mail.unitedinsong.com.au (mail.unitedinsong.com.au [150.101.178.33]) by mx1.freebsd.org (Postfix) with ESMTP id A36668FC12 for ; Thu, 22 Mar 2012 14:19:08 +0000 (UTC) Received: from laptop1.herveybayaustralia.com.au (laptop1.herveybayaustralia.com.au [192.168.0.182]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.unitedinsong.com.au (Postfix) with ESMTPSA id 42E6D5C22; Fri, 23 Mar 2012 00:32:41 +1000 (EST) Message-ID: <4F6B34D9.3020806@herveybayaustralia.com.au> Date: Fri, 23 Mar 2012 00:19:05 +1000 From: Da Rock User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:7.0.1) Gecko/20111109 Thunderbird/7.0.1 MIME-Version: 1.0 To: Andrzej Tobola , freebsd-hackers@freebsd.org References: <20120321120551.GA1547@tiny> <20120321130648.GA1714@tiny> <4F69D4CB.1010401@herveybayaustralia.com.au> <20120321133452.GJ20963@amp2.iem.pw.edu.pl> In-Reply-To: <20120321133452.GJ20963@amp2.iem.pw.edu.pl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: Re: iso2flash img X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Mar 2012 14:19:09 -0000 On 03/21/12 23:34, Andrzej Tobola wrote: > On Wed, Mar 21, 2012 at 11:16:59PM +1000, Da Rock wrote: >> On 03/21/12 23:06, Matthias Apitz wrote: >>> Hello, >>> >>> I've forwarded your question to a colleague who is an Ubuntu fan and he >>> replied, that the creation of a booting Ubuntu USB key is possile with >>> Ubuntu tools/methods only, i.e. booting the Ubuntu live CD and following the >>> ways to install it onto the key (and not on disk). >>> >>> He wrote some years ago a small Wiki page about, it is in German but >>> maybe you can clue something from the commands there: >>> >>> http://mikiwiki.org/wiki/Ubuntu_8.10_Intrepid_Ibex/Installation_2009.04.07_usbstick >> I'll have to translate that, but I am trying to get a 'live' usb disk to >> demo on the clients cdrom less unit. I know the cd is live, I assumed I >> could get a live usb disk from that based on their instructions. >> >> For a supposedly user friendly system, obtaining install media is not.. >> :/ Maybe a little too much debian in the system ;) > You can use VirtualBox - boot live iso, connect usb and use native tool there. The saga continues... That doesn't work either- the live disk doesn't have the tools required! They obviously have no concept of simplicity... this really is insane :/ Back to the drawing board... From owner-freebsd-hackers@FreeBSD.ORG Thu Mar 22 14:34:49 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 584B5106567E; Thu, 22 Mar 2012 14:34:49 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-pb0-f54.google.com (mail-pb0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 1DB628FC1A; Thu, 22 Mar 2012 14:34:48 +0000 (UTC) Received: by pbcwz17 with SMTP id wz17so1929853pbc.13 for ; Thu, 22 Mar 2012 07:34:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=VE+31/NnG96PHizrCyhCAkWqGtj7RZ0Vd8Vm5n0E5kA=; b=zym9lhPDGMVdluPuF98CE5LzeJ8Qx5N/uRKFvzrdIuDT8q3hEWASnb6wC4Swh0ZV2u 4VZBDblGdIgi2N512K2etcDbo7KPj4HMJzTsPNjkivnnYtt6bUpTSHylm0Uv5jwKWqad N/3+2w7ZWGwYLFnUpkniow53vUGPE7bpm0+n59hxb/ysCQZudjuM4HT5dS1HKb4n9Js4 EKEnjEsgAfUXjlH/s0gleb6uCnFcf+4HGIWEqJFM80eFrZBXoOqqaegGgdOzh6Vc0JNk 2RJODOBjr9BPF/N9NvAI9euHnmC9+aIgqUEBZ2DIEzk4Mcv0W+CzwMbapNJv0VCBDB2a KI/Q== MIME-Version: 1.0 Received: by 10.68.195.38 with SMTP id ib6mr9571085pbc.28.1332426888625; Thu, 22 Mar 2012 07:34:48 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.143.33.5 with HTTP; Thu, 22 Mar 2012 07:34:48 -0700 (PDT) In-Reply-To: <201203220813.13113.jhb@freebsd.org> References: <20120322101041.GA99431@freebsd.org> <201203220813.13113.jhb@freebsd.org> Date: Thu, 22 Mar 2012 07:34:48 -0700 X-Google-Sender-Auth: xykJVteMYbaVvjQ-ZZOhPUczsjE Message-ID: From: Adrian Chadd To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org, Alexander Best Subject: Re: still issues with unreadable dmesg output on SMP systems X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Mar 2012 14:34:49 -0000 Hi, On 22 March 2012 05:13, John Baldwin wrote: > s/fixed/mostly worked around/ > > The problem is that the work around isn't perfect. =A0The root cause is s= till > not fixed. =A0avg@ has some ideas on better ways to handle this, but it i= s a bit > tricky to get this right since we also dont' want printfs to be delayed t= oo > much (and to fix this perfectly requires buffering printf output). I'm rapidly liking the brokenness of the IO. Why? Because it becomes _very freaking obvious_ when things are running concurrently. I've found many ath/net80211 bugs because of this. Having it as an option would be nice. :-) Adrian From owner-freebsd-hackers@FreeBSD.ORG Thu Mar 22 14:42:54 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B2746106566B for ; Thu, 22 Mar 2012 14:42:54 +0000 (UTC) (envelope-from eric.saintetienne@gmail.com) Received: from mail-vb0-f54.google.com (mail-vb0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 2EBBD8FC0A for ; Thu, 22 Mar 2012 14:42:53 +0000 (UTC) Received: by vbmv11 with SMTP id v11so1431097vbm.13 for ; Thu, 22 Mar 2012 07:42:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=G8rBCFhFISeRMoVFuXwD3NN+Bf7WqpoT5UFSBTvjUYY=; b=Kmw1jjPs8gAlo5QJ8STFGUbZ+5TSSltO/4TC5uNPLErMdZxPpk9d56GrPj2MCkP1Pb wIG8dd7Xi7KfOOLD6X3htq4AYSsHOlALJFLL42GGJ2Sm0HgPJfqoSSjAq4sUFHqp4mi1 t8scXuZeJ5BIs4NpoJkKo5b15sRtNkkJ5xAzhCI0Mmyv/DS49L4DFTwavdwIlls6KtQr Xh7QTdOM4/RUGfNmhvHbNFM7y1TOzoLW9UywtcHWy/x30aVAZE5zx0HkYr+CHb9a4IE9 qMcuHxzioZhGr7oHWkmNU3BVSoA5mrtfEhVgtIYPHD0+VCnq6pTLk3q7+H7olwynJDVJ O9gw== Received: by 10.220.116.68 with SMTP id l4mr3954781vcq.4.1332427373468; Thu, 22 Mar 2012 07:42:53 -0700 (PDT) MIME-Version: 1.0 Received: by 10.220.149.9 with HTTP; Thu, 22 Mar 2012 07:42:22 -0700 (PDT) In-Reply-To: References: From: Eric Saint-Etienne Date: Thu, 22 Mar 2012 14:42:22 +0000 Message-ID: To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=UTF-8 Subject: Re: malloc pages map to user space X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Mar 2012 14:42:54 -0000 > By using kernel_map instead of kmem_map, vm_map_lookup() now always > return a vm_object. That's a big progress. > As expected, when this object is kmem_object, the user mapping works > fine (for smaller or larger mallocs.) > > Otherwise that object doesn't match kernel_object. It's an anonymous > object to me. > Using that "anonymous" vm_object for mapping into user map (using > vm_map_find()) doesn't directly fail, > it does provide a virtual address in the user map. However I read > zeros at that address, from within the user process. Actually when using kernel_map, the object returned is NULL! However the the vm_entry_t it returns seems a valid address, its 'object' field is NULL too (that's consistent) That's the reason why I didn't find it in any existing 'puclic' map (such as kernel_map, buffers_map, kmem_map, exec_map or pipe_map) But a NULL object isn't good at anything and I'm not sure what to do with a vm_entry_t only... Any idea how to insert it in the process map? From owner-freebsd-hackers@FreeBSD.ORG Thu Mar 22 15:29:21 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 874B2106566C for ; Thu, 22 Mar 2012 15:29:21 +0000 (UTC) (envelope-from royce.williams@gmail.com) Received: from mail-we0-f182.google.com (mail-we0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 164F38FC12 for ; Thu, 22 Mar 2012 15:29:20 +0000 (UTC) Received: by wern13 with SMTP id n13so2577889wer.13 for ; Thu, 22 Mar 2012 08:29:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=2gL+PLLoZRF7kQCX25doyaaozw4Ax1kRSb25kf3ypUw=; b=aUVRa2ajZe8RkHygek3fzhLi8mjtbvUa/7diCJAcM96JVMPIW0qfMAtjkk8xdRa1/4 2mok23gGnHkcvOa+UI8EgiOENRC5kckyERA7UV/42JcvK/NuSWSWzOszk1vItmbgd1UQ 3L2dSTvC2EQVqfIooEJbWAhDB1Q8fdF98Fehblk7lIICdKp9z/67+1Sm57C+rVg+lRzj 43GHhx91Knoh5rn8/4aUmZwR16HBrIcUaS/nxY9MKjiPP/dNSTrbUUhWs/PhKG0YEb4l xNO2+iDhUWANuuoeY99d9dEJh5/X8QSEFPBShz97qizqVDQSO5/AfXTDkzNvD1dCk4Ou bXvA== Received: by 10.50.88.168 with SMTP id bh8mr1873432igb.29.1332430159385; Thu, 22 Mar 2012 08:29:19 -0700 (PDT) MIME-Version: 1.0 Received: by 10.231.39.20 with HTTP; Thu, 22 Mar 2012 08:28:59 -0700 (PDT) In-Reply-To: <20120322111547.GD50510@onelab2.iet.unipi.it> References: <4F699391.9070804@herveybayaustralia.com.au> <20120322111547.GD50510@onelab2.iet.unipi.it> From: Royce Williams Date: Thu, 22 Mar 2012 07:28:59 -0800 Message-ID: To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: Re: iso2flash img X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Mar 2012 15:29:21 -0000 On Thu, Mar 22, 2012 at 3:15 AM, Luigi Rizzo wrote: > On Wed, Mar 21, 2012 at 10:52:53AM -0500, Mark Felder wrote: >> As an alternative I recently purchased a Zalman ZM-VE200 device (there's >> also a USB3.0 flavor) that lets you copy ISOs to it and it will emulate a >> CDROM/DVDROM/BDROM for you so you never have to deal with this mess again. >> It works amazingly well. I was tired of fighting this problem and this is >> an amazing solution -- I can keep every ISO I ever need on a single drive. >> >> http://www.zalman.com/eng/product/Product_Read.asp?idx=431 >> http://www.zalman.com/eng/product/Product_Read.asp?idx=459 >> http://www.rmprepusb.com/tutorials/ve200 > > really nice, thanks for the link. Now if they had something > that supported a USB key it would be even nicer... I *love* mine - it almost always Just Works. Since all you do is buy the enclosure (around $30), you can put in whatever size 2.5" drive you'd like. I threw in a 750G, so I have ~98G of CD and DVD images. There is a physical rocker switch for navigating the list of ISOs and mounting/unmounting them. You can also toggle ISO-only, drive-only, or hybrid/both mode, so I've got lots of other stuff on there that's handy once you've booted. It also has a physical read-only switch - great feature. The only hangup I've seen so far is if the BIOS doesn't support USB-attached optical drives. There are probably some workarounds for that out there (boot from a USB and then chainload-ish the optical drive), but I have not yet pursued them. There is undocumented support for floppy images as well. I haven't tested it, but there are success stories. *Highly* recommended. From owner-freebsd-hackers@FreeBSD.ORG Thu Mar 22 16:41:52 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id EDD1E106564A for ; Thu, 22 Mar 2012 16:41:52 +0000 (UTC) (envelope-from scdbackup@gmx.net) Received: from mailout-de.gmx.net (mailout-de.gmx.net [213.165.64.22]) by mx1.freebsd.org (Postfix) with SMTP id 62E2C8FC15 for ; Thu, 22 Mar 2012 16:41:51 +0000 (UTC) Received: (qmail invoked by alias); 22 Mar 2012 16:41:50 -0000 Received: from 165.126.46.212.adsl.ncore.de (HELO 192.168.2.69) [212.46.126.165] by mail.gmx.net (mp034) with SMTP; 22 Mar 2012 17:41:50 +0100 X-Authenticated: #2145628 X-Provags-ID: V01U2FsdGVkX19fpTg1BXVfxc9hW8FdiAphV/VmN2K4LnrRvMnM1S wXQIOWMC17MuCN Date: Thu, 22 Mar 2012 17:42:27 +0100 From: "Thomas Schmitt" To: freebsd-hackers@freebsd.org References: <4F6A8C4B.2040605@herveybayaustralia.com.au> In-Reply-To: <4F6A8C4B.2040605@herveybayaustralia.com.au> Message-Id: <10033882394537@192.168.2.69> X-Y-GMX-Trusted: 0 Subject: Re: iso2flash img X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Mar 2012 16:41:53 -0000 Hi, Vitaly Magerya : > > you might want to try to dd the iso image directly onto USB instead; there > > where talks that Ubuntu would support this starting at 11.10. Da Rock : > Nada. Tried that and it didn't work. I'm not sure how that would work given > that it uses isolinux to boot- ergo needs a cd to load the kernel. Maybe > some way to determine the install media? The trick is called "isohybrid". It works by a DOS MBR which starts the same executable boot image that is pointed to by the El Torito boot catalog. If the ISO is on a hard disk (or alike), then the BIOS boots via MBR. If it is on an optical medium, then the BIOS boots via El Torito. The question is rather why it does not work for you. I downloaded ubuntu-11.10-desktop-i386.iso from http://www.ubuntu.com/download/ubuntu/download and put it onto an USB stick (by a Linux machine, but that should not matter) dd of=/dev/sdc if=ubuntu-11.10-desktop-i386.iso bs=2048 Note that /dev/sdc is not the first partition but the whole USB stick. This stick boots on amd64 hardware. After some waiting with sparse iconography i get to the question whether i want to try or to install. I choose to try and get a graphical desktop. From the icon list i start Firefox and google a bit via my internet router. All seems well. On FreeBSD, GEOM complains about the DOS partition alignment. Partition 1 starts at block 64. fdisk -p /dev/da0 # /dev/da0 g c243 h255 s63 p 1 0x17 64 1423896 a 1 Nevertheless these two commands work and open access to the image content: mount -t cd9660 /dev/da0 /mnt mount -t cd9660 /dev/da0s1 /mnt (The ISO has two superblocks and two directory trees.) Does your hardware boot from USB stick at all ? Is its firmware (U)EFI rather than BIOS ? Have a nice day :) Thomas From owner-freebsd-hackers@FreeBSD.ORG Thu Mar 22 15:29:33 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DC0A1106567A; Thu, 22 Mar 2012 15:29:33 +0000 (UTC) (envelope-from jake@xz.cx) Received: from floor13.viper.enta.net (floor13.viper.enta.net [78.33.24.2]) by mx1.freebsd.org (Postfix) with ESMTP id 941EC8FC24; Thu, 22 Mar 2012 15:29:33 +0000 (UTC) Received: from xz.cx (floor13.viper.enta.net [78.33.24.2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: jake) by floor13.viper.enta.net (Postfix) with ESMTPSA id 77E18BDCB7; Thu, 22 Mar 2012 15:24:17 +0000 (GMT) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 22 Mar 2012 16:24:16 +0100 From: Jake Smith To: , , , Message-ID: X-Sender: jake@xz.cx User-Agent: Roundcube Webmail/0.6 X-Mailman-Approved-At: Thu, 22 Mar 2012 17:16:49 +0000 Cc: Subject: LSI mps(4) driver issues with PIKE 2008/IMR (LSI SAS2008) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Mar 2012 15:29:34 -0000 Hello, I am trying to get the latest mps(4) driver in FreeBSD 9-STABLE working with am LSI SAS2008 variant from ASUS, they call it PIKE 2008/IMR. Link http://www.asus.com/Server_Workstation/Accessories/PIKE_2008IMR/ From what I can see this card should be compatible with the mps(4) driver MFC'd to 9-STABLE about 6 weeks ago. # uname -a FreeBSD xxx 9.0-STABLE FreeBSD 9.0-STABLE #3 r233304M: Thu Mar 22 12:53:17 GMT 2012 root@xxx:/usr/obj/usr/src/sys/GENERIC amd64 Revision: 233304 Initially the card is not seen at all by the driver, however pciconf shows us why that is. mps0@pci0:2:0:0: class=0x010700 card=0x843b1043 chip=0x00731000 rev=0x03 hdr=0x00 vendor = 'LSI Logic / Symbios Logic' device = 'MegaRAID SAS 9240' class = mass storage subclass = SAS It seems on other models of LSI SAS2008 the chip device ID is 0x0072, however for some reason this card has 0x0073. So I patched the mps(4) driver and recompiled. diff -ruN mps.orig/mpi/mpi2_cnfg.h mps/mpi/mpi2_cnfg.h --- mps.orig/mpi/mpi2_cnfg.h 2012-03-22 14:50:53.000000000 +0000 +++ mps/mpi/mpi2_cnfg.h 2012-03-22 14:52:23.000000000 +0000 @@ -416,7 +416,8 @@ /* SAS */ #define MPI2_MFGPAGE_DEVID_SAS2004 (0x0070) -#define MPI2_MFGPAGE_DEVID_SAS2008 (0x0072) +#define MPI2_MFGPAGE_DEVID_SAS2008_1 (0x0072) +#define MPI2_MFGPAGE_DEVID_SAS2008_2 (0x0073) #define MPI2_MFGPAGE_DEVID_SAS2108_1 (0x0074) #define MPI2_MFGPAGE_DEVID_SAS2108_2 (0x0076) #define MPI2_MFGPAGE_DEVID_SAS2108_3 (0x0077) diff -ruN mps.orig/mps_pci.c mps/mps_pci.c --- mps.orig/mps_pci.c 2012-03-22 14:48:41.000000000 +0000 +++ mps/mps_pci.c 2012-03-22 14:51:59.000000000 +0000 @@ -99,7 +99,9 @@ } mps_identifiers[] = { { MPI2_MFGPAGE_VENDORID_LSI, MPI2_MFGPAGE_DEVID_SAS2004, 0xffff, 0xffff, 0, "LSI SAS2004" }, - { MPI2_MFGPAGE_VENDORID_LSI, MPI2_MFGPAGE_DEVID_SAS2008, + { MPI2_MFGPAGE_VENDORID_LSI, MPI2_MFGPAGE_DEVID_SAS2008_1, + 0xffff, 0xffff, 0, "LSI SAS2008" }, + { MPI2_MFGPAGE_VENDORID_LSI, MPI2_MFGPAGE_DEVID_SAS2008_2, 0xffff, 0xffff, 0, "LSI SAS2008" }, { MPI2_MFGPAGE_VENDORID_LSI, MPI2_MFGPAGE_DEVID_SAS2108_1, 0xffff, 0xffff, 0, "LSI SAS2108" }, After reboot it now loads the mps(4) module and attempts to init the card but fails. # dmesg | grep mps mps0: port 0xd800-0xd8ff mem 0xfbd7c000-0xfbd7ffff,0xfbdc0000-0xfbdfffff irq 16 at device 0.0 on pci2 mps0: Doorbell failed to activate device_attach: mps0 attach returned 6 From this point I'm stuck on what to try next, google does not provide any answers for this situation. Does any one have any advice or ideas as to why this is not working? I am able to provide ssh access to the server if any one wants to log on and have a look at it. Really appreciate any help you can give. Kind Regards Jake From owner-freebsd-hackers@FreeBSD.ORG Thu Mar 22 17:18:35 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 551781065674 for ; Thu, 22 Mar 2012 17:18:35 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id 120798FC16 for ; Thu, 22 Mar 2012 17:18:34 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id 9D2FF7300B; Thu, 22 Mar 2012 18:37:17 +0100 (CET) Date: Thu, 22 Mar 2012 18:37:17 +0100 From: Luigi Rizzo To: Thomas Schmitt Message-ID: <20120322173717.GA54005@onelab2.iet.unipi.it> References: <4F6A8C4B.2040605@herveybayaustralia.com.au> <10033882394537@192.168.2.69> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <10033882394537@192.168.2.69> User-Agent: Mutt/1.4.2.3i Cc: freebsd-hackers@freebsd.org Subject: Re: iso2flash img X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Mar 2012 17:18:35 -0000 On Thu, Mar 22, 2012 at 05:42:27PM +0100, Thomas Schmitt wrote: > Hi, > > Vitaly Magerya : > > > you might want to try to dd the iso image directly onto USB instead; there > > > where talks that Ubuntu would support this starting at 11.10. > > Da Rock : > > Nada. Tried that and it didn't work. I'm not sure how that would work given > > that it uses isolinux to boot- ergo needs a cd to load the kernel. Maybe > > some way to determine the install media? > > The trick is called "isohybrid". > It works by a DOS MBR which starts the same executable boot image > that is pointed to by the El Torito boot catalog. > If the ISO is on a hard disk (or alike), then the BIOS boots via MBR. > If it is on an optical medium, then the BIOS boots via El Torito. interesting. It does work for me indeed. And it might be a nice trick for our images too, so we don't have to build a memstick and an ISO image... cheers luigi > The question is rather why it does not work for you. > > I downloaded > ubuntu-11.10-desktop-i386.iso > from > http://www.ubuntu.com/download/ubuntu/download > and put it onto an USB stick (by a Linux machine, but that should not matter) > dd of=/dev/sdc if=ubuntu-11.10-desktop-i386.iso bs=2048 > Note that /dev/sdc is not the first partition but the whole USB stick. > > This stick boots on amd64 hardware. > After some waiting with sparse iconography i get to the question > whether i want to try or to install. I choose to try and get a > graphical desktop. From the icon list i start Firefox and google > a bit via my internet router. All seems well. > > > On FreeBSD, GEOM complains about the DOS partition alignment. > Partition 1 starts at block 64. > fdisk -p /dev/da0 > # /dev/da0 > g c243 h255 s63 > p 1 0x17 64 1423896 > a 1 > Nevertheless these two commands work and open access to the image content: > mount -t cd9660 /dev/da0 /mnt > mount -t cd9660 /dev/da0s1 /mnt > (The ISO has two superblocks and two directory trees.) > > > Does your hardware boot from USB stick at all ? > Is its firmware (U)EFI rather than BIOS ? > > > Have a nice day :) > > Thomas > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Thu Mar 22 17:48:31 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C7773106566C for ; Thu, 22 Mar 2012 17:48:31 +0000 (UTC) (envelope-from nonesuch@longcount.org) Received: from mail-lpp01m010-f54.google.com (mail-lpp01m010-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id 3D6C78FC14 for ; Thu, 22 Mar 2012 17:48:30 +0000 (UTC) Received: by lagv3 with SMTP id v3so2462714lag.13 for ; Thu, 22 Mar 2012 10:48:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-originating-ip:in-reply-to:references:date :message-id:subject:from:to:content-type:content-transfer-encoding :x-gm-message-state; bh=JLfV0o5exSLjN67kpUAPlzd2l7thCKBXK4bHuxGTUjI=; b=iXPq8DGvR/6guuXQHwRrWuVea50zfgDOCbCCsUSUHbewt2Hg8sGM3ubn2puL/phBjI wdilTuxdQHKGAqh0aTf6UX5rAmZz54a13zVjw2RQBWL62ahlzbYod4JHACVFXvI0FeuJ awEneE3KJdIGkenOtDdgbVYeSWmds4pDQV9ke0Pp4Pgf8Vty0DkjflXHmQejjD155nAO +g4OJWhOYoX/TWG5rEFlXNHuRz1JJvZgh/39+givhQKtp9jTqZOmlzLdulTSZybztXgt GtuwQ3fmyobiO/AkctAr7bycXz99J5QWCR4Fef9qtg3d3pvhE5BoOqRlDL+BE5yTZF/u zwzw== MIME-Version: 1.0 Received: by 10.112.23.38 with SMTP id j6mr3372506lbf.15.1332438509616; Thu, 22 Mar 2012 10:48:29 -0700 (PDT) Received: by 10.112.36.135 with HTTP; Thu, 22 Mar 2012 10:48:29 -0700 (PDT) X-Originating-IP: [209.66.78.50] In-Reply-To: <201203220803.57000.jhb@freebsd.org> References: <201203220803.57000.jhb@freebsd.org> Date: Thu, 22 Mar 2012 13:48:29 -0400 Message-ID: From: Mark Saad To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Gm-Message-State: ALoCoQlEMEHA8INcUj714+q66bBU19Y+en8m42PCTUlAwdSBpDa8ie9bYsD7CCp6KL8GzUO+VSFr Subject: Re: Approaching the limit on PV entries X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Mar 2012 17:48:32 -0000 On Thu, Mar 22, 2012 at 8:03 AM, John Baldwin wrote: > On Wednesday, March 21, 2012 4:20:17 pm Mark Saad wrote: >> On Wed, Mar 21, 2012 at 12:39 PM, Sergey Kandaurov w= rote: >> > On 21 March 2012 19:19, John Baldwin wrote: >> >> On Tuesday, March 20, 2012 11:37:57 am Sergey Kandaurov wrote: >> >>> On 22 November 2011 19:29, Mark Saad wrote: >> >>> > Hello All >> >>> >> >>> [found this mail in my drafts, not sure if my answer is still useful= ] >> >>> >> >>> > =C2=A0I want to get to the bottom of a warning in dmesg. On 7.2-RE= LEASE and >> >>> > 7.3-RELEASE I have seen the following warning in dmesg. >> >>> > >> >>> > Approaching the limit on PV entries, consider increasing either th= e >> >>> > vm.pmap.shpgperproc or the vm.pmap.pv_entry_max sysctl. >> >>> > >> >>> > So looking around I see a few posts here and there about how to tu= ne >> >>> > the sysctls to address the warning however I am not 100% sure what >> >>> > each value does. >> >>> > It appears changing vm.pmap.shpgperproc affects the value of >> >>> > vm.pmap.pv_entry_max . Can someone explain the relationship of the= two >> >>> > sysctls. Also >> >>> >> >>> This is how they are calculated. >> >>> >> >>> pv_entry_max =3D shpgperproc * maxproc + cnt.v_page_count; >> >>> >> >>> and, respectively, >> >>> >> >>> shpgperproc =3D (pv_entry_max - cnt.v_page_count) / maxproc; >> >>> >> >>> So, changing one sysctl will change another and vice versa. >> >>> >> >>> > what pitfalls of changing them are. >> >>> >> >>> Not known to me (on amd64 platform). >> >>> I have had vm.pmap.shpgperproc=3D15000 on 8.1 amd64 with 4G RAM >> >>> to make some badly written commercial software to work until it >> >>> was decommissioned to the scrap. >> >> >> >> FYI, Alan just removed this warning and the associated sysctls from H= EAD >> >> yesterday because they were made obsolete several years ago. =C2=A0I = think they are >> >> obsolete even on 7. =C2=A0Certainly on 8. >> > >> > Yep, and since switching to direct map (somewhere around 7.x on amd64?= ) >> > made PV entry limit factually obsolete, this is really cool. >> > >> > -- >> > wbr, >> > pluknet >> >> Interesting so this warning is relevant in 7.x ? > > No, looks like it was obsolete starting with 7.0. > > -- > John Baldwin Any chance it could be mfc'ed to 7-STABLE ? --=20 mark saad | nonesuch@longcount.org From owner-freebsd-hackers@FreeBSD.ORG Thu Mar 22 18:46:03 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4FB61106564A for ; Thu, 22 Mar 2012 18:46:03 +0000 (UTC) (envelope-from scdbackup@gmx.net) Received: from mailout-de.gmx.net (mailout-de.gmx.net [213.165.64.22]) by mx1.freebsd.org (Postfix) with SMTP id 9F2E08FC08 for ; Thu, 22 Mar 2012 18:46:02 +0000 (UTC) Received: (qmail invoked by alias); 22 Mar 2012 18:46:01 -0000 Received: from 165.126.46.212.adsl.ncore.de (HELO 192.168.2.69) [212.46.126.165] by mail.gmx.net (mp012) with SMTP; 22 Mar 2012 19:46:01 +0100 X-Authenticated: #2145628 X-Provags-ID: V01U2FsdGVkX18CNQMiHN0CkZHY+5n/WjTbjZvpx50M7kGcc/YhT2 4NzgOw7jY9pn0u Date: Thu, 22 Mar 2012 19:46:37 +0100 From: "Thomas Schmitt" To: freebsd-hackers@freebsd.org References: <20120322173717.GA54005@onelab2.iet.unipi.it> In-Reply-To: <20120322173717.GA54005@onelab2.iet.unipi.it> Message-Id: <10033788658207@192.168.2.69> X-Y-GMX-Trusted: 0 Subject: Re: iso2flash img X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Mar 2012 18:46:03 -0000 Hi, > > The trick is called "isohybrid". Luigi Rizzo wrote: > interesting. It does work for me indeed. So why not for Da Rock ? > And it might be a nice trick for our images too, so we don't > have to build a memstick and an ISO image... I would be happy to help with that. I am the developer of program xorriso which in the role of mkisofs has composed that Ubuntu image. My knowlege is only about pointing BIOS to the boot loader start programs, not about those boot systems themselves. A while ago i exercised the most simple case of http://wiki.freebsd.org/AvgLiveCD with the mkisofs emulation of xorriso. It booted. An MBR can be inserted easily by mkisofs option -G. isohybrid demands to patch that MBR with the LBA of the boot image and to set up the DOS partition table. GRUB2 demands only to set up the partition table. (Special xorrisofs options get employed.) What would a FreeBSD bootloader MBR need to know about the data in the ISO image to start up and handle it like a read-only hard disk ? Do programs of the first boot stages need to know their own LBA in the image resp. partition ? The El Torito and MBR equipment of GRUB2 can provide the same functionality as ISOLINUX with isohybrid. GRUB2 script grub-mkrescue demonstrates this. I understand Debian GNU/kFreeBSD boots via El Torito and GRUB2. But it makes no use of the opportunity to have an MBR too. I boot my own FreeBSD 8-STABLE from hard disk via MBR, GRUB2 and a chainloaded FreeBSD boot loader. Have a nice day :) Thomas From owner-freebsd-hackers@FreeBSD.ORG Thu Mar 22 19:33:55 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 56DF31065673 for ; Thu, 22 Mar 2012 19:33:55 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mail-we0-f182.google.com (mail-we0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id D8AFF8FC0C for ; Thu, 22 Mar 2012 19:33:54 +0000 (UTC) Received: by wern13 with SMTP id n13so2850686wer.13 for ; Thu, 22 Mar 2012 12:33:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=ckX2asfQk9YPHZ7cIQ2oCj7fvk2/6G7cWeLO6W6Y1Jw=; b=yDMz0cDtAujwiFxS3XyyaWM6FXBe5IIvWJHLlGswPSTUP3tGYouw36ITV7SEVYnWya Rq7ccvjJmdAM6BbJF002GnhV2AKhtwQ2nsK6u+2jGvSmzRV7x/hY/C8OfYP4BD0vTYtE k6YUgLZJcVpLALBEBU0xT/j9rQh+87Q4pW+vofWvAybLFE+Ls7miS6wzboX7oVa8iD/1 +IjFIl4swAwVy+CM7zwxN2trPBdp9YjjLSGvZjMqaujRsODbBk7oJy/eZP5+2wBdwT9w qgBUIcBWO9/2WWjVn831iN+P8jykqCIDsKoTj6CekZ1pYe/VstWh6Hs2gKkK+RafQQ0t hk1A== MIME-Version: 1.0 Received: by 10.180.14.73 with SMTP id n9mr196950wic.16.1332444833959; Thu, 22 Mar 2012 12:33:53 -0700 (PDT) Received: by 10.180.79.137 with HTTP; Thu, 22 Mar 2012 12:33:53 -0700 (PDT) In-Reply-To: References: Date: Thu, 22 Mar 2012 15:33:53 -0400 Message-ID: From: Ryan Stone To: Eric Saint-Etienne Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org Subject: Re: malloc pages map to user space X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Mar 2012 19:33:55 -0000 On Thu, Mar 22, 2012 at 10:42 AM, Eric Saint-Etienne wrote: > Actually when using kernel_map, the object returned is NULL! However the > the vm_entry_t it returns seems a valid address, its 'object' field is NULL > too (that's consistent) > That's the reason why I didn't find it in any existing 'puclic' map (such as > kernel_map, buffers_map, kmem_map, exec_map or pipe_map) > > But a NULL object isn't good at anything and I'm not sure what to do with > a vm_entry_t only... Any idea how to insert it in the process map? If your kernel module creates a device in /dev that implements the mmap method, then you don't need to worry about mucking around with vm_maps and objects and whatnot. Your mmap method just needs to be able to convert offsets into the device into physical memory addresses, and the vm infrastructure will do the rest for you. From owner-freebsd-hackers@FreeBSD.ORG Thu Mar 22 19:57:49 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F0E46106564A for ; Thu, 22 Mar 2012 19:57:49 +0000 (UTC) (envelope-from eric.saintetienne@gmail.com) Received: from mail-gx0-f182.google.com (mail-gx0-f182.google.com [209.85.161.182]) by mx1.freebsd.org (Postfix) with ESMTP id A77C28FC12 for ; Thu, 22 Mar 2012 19:57:49 +0000 (UTC) Received: by ggnk4 with SMTP id k4so2621923ggn.13 for ; Thu, 22 Mar 2012 12:57:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding; bh=2dE+vMPUWGXP5cDte6poQDgVHJrRMwI1xmZCc34nItk=; b=Je2hveELowlHTptazNIOjJqaGcdSH/+euaqDAePFgaS6WguOV7OPJcjCiRWLJ7uE5H jgptl112DCiraQBV2MH7PhtunlphOwO5a/2rYWnanq8uuHJwFZhUyVXfNSWE1Y5fqO4I qGALwmFCPFnSaLxu9bD0jHwRDvMkqAQA56h0OVNmdhxl8I/SnE2CVpHCO6i3tR8KtfYJ QKlYo6v7bXVkO6t9jk1Q5X842AV3jKyEgmMmC4lRCa6Zr87GYxaV+faVQ1lCKZC6XOnO MfbfZFraS/Fx9v+Ln/t7oma156ms3P0mSF477RZpZlbLFF/ixsYkL6yfRIdJ3I5V44AX 8W6w== Received: by 10.68.237.1 with SMTP id uy1mr8110507pbc.151.1332446262575; Thu, 22 Mar 2012 12:57:42 -0700 (PDT) MIME-Version: 1.0 Received: by 10.68.47.129 with HTTP; Thu, 22 Mar 2012 12:57:11 -0700 (PDT) In-Reply-To: References: From: Eric Saint-Etienne Date: Thu, 22 Mar 2012 19:57:11 +0000 Message-ID: To: Ryan Stone Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: malloc pages map to user space X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Mar 2012 19:57:50 -0000 > If your kernel module creates a device in /dev that implements the > mmap method, then you don't need to worry about mucking around with > vm_maps and objects and whatnot. =C2=A0Your mmap method just needs to be > able to convert offsets into the device into physical memory > addresses, Yes I'm aware of this facility, thank you. > and the vm infrastructure will do the rest for you. Since this mapping is on the main path of the driver, I'm worried that the overhead on each access of a page fault and a function call (the pager associated with a cdev mmap) is too much to bear. So I'd like to do it the "hard" way which I feel is the most optimized. Thanks! From owner-freebsd-hackers@FreeBSD.ORG Thu Mar 22 20:11:10 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6B65C1065673 for ; Thu, 22 Mar 2012 20:11:10 +0000 (UTC) (envelope-from eric.saintetienne@gmail.com) Received: from mail-yx0-f182.google.com (mail-yx0-f182.google.com [209.85.213.182]) by mx1.freebsd.org (Postfix) with ESMTP id 2372F8FC15 for ; Thu, 22 Mar 2012 20:11:09 +0000 (UTC) Received: by yenl9 with SMTP id l9so2617024yen.13 for ; Thu, 22 Mar 2012 13:11:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=Kum5JQox+Xze9zc+NNfKu7IK9AAHYmqhmKe01qhqsW4=; b=HvLmX//7elSo8VfuJgIs4osKFmxK9gczYTPIu/xmTisBHEMFqy7lI0nJJOVAdEaKLP j51R6DLoDja6dkKpunXMt3CXERHWOjpLJcnDHccICTUpyr15TSS73NQnXQQJXoZJ1drq DlJ1Oavxnc6nmUO36b2EMei9DIR38yi3hh1JhtmNCr7Ft5EZRP2hgfdnCIoFIQ3kx/Vi FILROIhF85BYSIEznGYzUkbg320Fv8e/Ll+rGq7pbSfJf28ASed42l8AWuPb8/K8VN7l fDWDp8QVxeY5ehi/NVeI90UNKmeKCwplTLguJsYMCoj6F5ORw5v5BFAGfLui6U3rYiU7 lrqQ== Received: by 10.68.237.1 with SMTP id uy1mr8203533pbc.151.1332447069224; Thu, 22 Mar 2012 13:11:09 -0700 (PDT) MIME-Version: 1.0 Received: by 10.68.47.129 with HTTP; Thu, 22 Mar 2012 13:10:39 -0700 (PDT) In-Reply-To: References: From: Eric Saint-Etienne Date: Thu, 22 Mar 2012 20:10:39 +0000 Message-ID: To: Ryan Stone Content-Type: text/plain; charset=UTF-8 Cc: freebsd-hackers@freebsd.org Subject: Re: malloc pages map to user space X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Mar 2012 20:11:10 -0000 Here is some code which fails with malloc < 1 page and sometimes succeeds with large mallocs (> 16 pages) What's wrong? #include #include #include #include #include #include #include #include #include // Copyright: skeleton took from an older post on the freebsd list #include #include #include #include #include #include #include #include MALLOC_DEFINE(M_FIVEG_SYSC, "fiveg_sysc", "fiveg_sysc test"); struct args { unsigned char **p; }; /* String to be located in maped buffer */ #define SIZE PAGE_SIZE // 1 page always fail static void initialize_array(char *p) { int i; for (i = 0; i < 26; i++) p[i] = i+'a'; p[26] = '!'; p[27] = '\0'; } static vm_offset_t addr; // allocated/freed at module load/unload /* Syscall func */ static int syscf(struct thread *td, void *sa) { vm_offset_t user_addr; /* User space address */ struct args *uap = (struct args*) sa; struct proc *procp = (struct proc *)td->td_proc; struct vmspace *vms = procp->p_vmspace; vm_map_t map; int result; vm_object_t object; vm_ooffset_t objoffset; vm_map_entry_t entry; vm_pindex_t pindex; vm_prot_t prot; boolean_t wired; map = kernel_map; // it always return data within kmeme anyway uprintf("KERNEL string is '%s' (%p)\n", (char*) addr, (void*) addr); result = vm_map_lookup(&map, addr, VM_PROT_ALL, &entry, &object, &pindex, &prot, &wired); if (result != KERN_SUCCESS) { uprintf("KERNEL vm_map_lookup failed (%d)\n", result); return ENOMEM; } vm_map_lookup_done(map, entry); if (object == kernel_object) uprintf("object is kernel_object\n"); else if (object == kmem_object) uprintf("object is kmem_object\n"); else uprintf("object=%p (not kmem, not kernel)\n", object); uprintf("entry=%p\n", entry); /* Offset in vm_object */ objoffset = addr - entry->start + entry->offset; user_addr = 0; result = vm_map_find(&vms->vm_map, object, objoffset, (vm_offset_t *) &user_addr, SIZE, VMFS_ANY_SPACE, VM_PROT_RW, VM_PROT_RW, 0); if (result != KERN_SUCCESS) uprintf("vm_map_find failed: %d\n", result); else { *uap->p = (char*) user_addr; uprintf("KERNEL ---> Syscall: user_addr for allocating space = 0x%lx\n", user_addr); } return result; } /* Sysent entity for syscall */ static struct sysent sc_sysent = { 1, /* Number of arguments */ syscf /* Syscall function */ }; //static struct sysent *old_sysent; /* Offset in sysent[] */ static int offset = NO_SYSCALL; /* Loader */ static int load (struct module *m, int cmd, void *something) { int error = 0; switch(cmd){ case MOD_LOAD: //MALLOC(addr, vm_offset_t, SIZE, M_FIVEG_SYSC, M_WAITOK | M_ZERO); addr = (vm_offset_t) malloc(SIZE, M_FIVEG_SYSC, M_WAITOK); initialize_array((char*) addr); uprintf("KERNEL Module with sysc loaded. Offset = %d \n", offset); break; case MOD_UNLOAD: free((void*) addr, M_FIVEG_SYSC); uprintf("KERNEL Module with sysc unloaded. Offset = %d \n", offset); break; default: error = EOPNOTSUPP; break; } return (error); /* Syscall macro*/ SYSCALL_MODULE(fiveg_sysc, &offset, &sc_sysent, load, NULL); /* eof */ ------------------------------------------------ -- USERLAND PROGRAM ------------------------------------------------ #include #include #include #include #include #include int main(int argc, char *argv[]) { int sysc_num, error; struct module_stat mstat; /* This Variable will save the addres of remapped buffer */ unsigned char *some_var = NULL; /* Pointer to pointer to remapped buffer */ unsigned char **p = &some_var; /* Search module with system call */ mstat.version = sizeof(mstat); if (!(modstat(modfind("sys/fiveg_sysc"), &mstat))){ /* Our module */ sysc_num = mstat.data.intval; printf("USER: Module found, Syscall number = %d \n", sysc_num); /* make system call */ error = syscall(sysc_num, p); if (error != 0) { printf("USER: an error occured: %d\n", error); return -1; } /* Read the string from remapped buffer */ printf("USER: p = %p\n", p); printf("USER: *p = %p\n", *p); printf("USER: String = %s\n", *p); } else printf("USER: Module seems to be not loaded! \n"); return 0; } /* eof */ From owner-freebsd-hackers@FreeBSD.ORG Thu Mar 22 20:38:17 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2DD6F1065670; Thu, 22 Mar 2012 20:38:17 +0000 (UTC) (envelope-from to.my.trociny@gmail.com) Received: from mail-yx0-f182.google.com (mail-yx0-f182.google.com [209.85.213.182]) by mx1.freebsd.org (Postfix) with ESMTP id BB02F8FC08; Thu, 22 Mar 2012 20:38:16 +0000 (UTC) Received: by yenl9 with SMTP id l9so2650728yen.13 for ; Thu, 22 Mar 2012 13:38:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=MJOL8WRvGeE2nS6uhGQQ/yaBR2gXtbBylIs0u9b6jK4=; b=Juk/bAKDum8OKpFHciGK7H/1GQ4G8gf6YeNpiQ+H3kQgXJ1zDHtsJ6HEmy1oMAdwf0 ou2jvj+Km8jENmZW7hcs/QHWUEQubrWPbc+wewm1NCEBZHSj/GojQkSbnZ1s1WC7omtu I71iEueILbs+AsigS8XOF2yy3E/Dku4butvU45fmMUGSzLcfMDUI6wh1C4qdRa+DZzNt udO+sxoRSw2q8jcFTs4vc6uMEgI4jm0Crsa14cG65eqgrWzwNhggLihKTJMCRP/iActw ofNI2COyCy1nq94XjFCpdm+Ax7/jij4iknoezAQ4qKWbbeAQVVhyC6Bqp95R/u15QGmy owBQ== MIME-Version: 1.0 Received: by 10.50.11.137 with SMTP id q9mr242021igb.11.1332448695920; Thu, 22 Mar 2012 13:38:15 -0700 (PDT) Received: by 10.231.4.24 with HTTP; Thu, 22 Mar 2012 13:38:15 -0700 (PDT) In-Reply-To: <8662e3m3eq.fsf@kopusha.home.net> References: <8662e3m3eq.fsf@kopusha.home.net> Date: Thu, 22 Mar 2012 22:38:15 +0200 Message-ID: From: Mikolaj Golub To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: Robert Watson , Kostik Belousov Subject: Re: a sysctl for process binary osreldate X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Mar 2012 20:38:17 -0000 On Sat, Mar 17, 2012 at 9:30 PM, Mikolaj Golub wrote: > Hi, > > Currently we can check and change binary osreldate of another process via > procfs(5). > > Kostik suggested to add a new sysctl for the same purpose and also extend > procstat to show osrel. > > Here are patches I am going to commit if there are no objections or > suggestions. > > http://people.freebsd.org/~trociny/kern_proc_osrel.1.patch > http://people.freebsd.org/~trociny/procstat.osrel.1.patch > > I set the same permissions as for procfs(5) osrel -- so only user can read it, > but may be this is too restrictive and p_cansee on read would be ok? Actually I don't see reasons why this may not be p_cansee, so I updated the patch and going to commit it if there is no objections. -- Mikolaj Golub From owner-freebsd-hackers@FreeBSD.ORG Thu Mar 22 21:31:50 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 913ED106566C; Thu, 22 Mar 2012 21:31:50 +0000 (UTC) (envelope-from to.my.trociny@gmail.com) Received: from mail-bk0-f54.google.com (mail-bk0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id AEC008FC08; Thu, 22 Mar 2012 21:31:49 +0000 (UTC) Received: by bkcjc3 with SMTP id jc3so2890062bkc.13 for ; Thu, 22 Mar 2012 14:31:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:references:x-comment-to:sender:date:in-reply-to :message-id:user-agent:mime-version:content-type; bh=iE/GKXpZOjY4ZSAoZaDt70eoI+Q8RMQQJe3JuTxQ2IY=; b=bcTgy0Z8TH8W5DpYa94mErU8TpTibJn2tfe5KB0vEI0fYS6PohSFQXKzX3R3HdRBa5 a6Sf3tc9VVClnontY1atbamgQhuFjYvslhcsHm6Soli8W0Rc9iJQ7lrkctuqpZNnh8mk kPnuH1Pwo/qwmX3BkQ1v5o+4hGnHur3sIlvTOn69Rr3MEZzNFaFrRRqYylU37IhC+0cv dztVx9ytb2QjAd/EEbqhAXsun1xEQ+5WzyWrvJz2ZBt+h+7Id797UJoBOd489ju69tun LbtWdJDB+i0doWlPycVnuhRcsifsLxV8OAH5nKMus6Mv/gAaTZ6IUtLrR+yS7oMSLNOh 6PKg== Received: by 10.204.154.202 with SMTP id p10mr3522769bkw.79.1332451908626; Thu, 22 Mar 2012 14:31:48 -0700 (PDT) Received: from localhost ([95.69.173.122]) by mx.google.com with ESMTPS id jr13sm12255502bkb.14.2012.03.22.14.31.46 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 22 Mar 2012 14:31:47 -0700 (PDT) From: Mikolaj Golub To: freebsd-hackers@freebsd.org References: <8662e3m3eq.fsf@kopusha.home.net> X-Comment-To: Mikolaj Golub Sender: Mikolaj Golub Date: Thu, 22 Mar 2012 23:31:44 +0200 In-Reply-To: (Mikolaj Golub's message of "Thu, 22 Mar 2012 22:38:15 +0200") Message-ID: <86iphw2ugv.fsf@kopusha.home.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Robert Watson , Kostik Belousov Subject: Re: a sysctl for process binary osreldate X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Mar 2012 21:31:50 -0000 On Thu, 22 Mar 2012 22:38:15 +0200 Mikolaj Golub wrote: MG> Actually I don't see reasons why this may not be p_cansee, so I MG> updated the patch and going to commit it if there is no objections. The updated patch: http://people.freebsd.org/~trociny/kern_proc_osrel.2.patch -- Mikolaj Golub From owner-freebsd-hackers@FreeBSD.ORG Thu Mar 22 22:19:06 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C8E6C106566C; Thu, 22 Mar 2012 22:19:06 +0000 (UTC) (envelope-from ken@kdm.org) Received: from nargothrond.kdm.org (nargothrond.kdm.org [70.56.43.81]) by mx1.freebsd.org (Postfix) with ESMTP id 7EAF58FC18; Thu, 22 Mar 2012 22:19:06 +0000 (UTC) Received: from nargothrond.kdm.org (localhost [127.0.0.1]) by nargothrond.kdm.org (8.14.2/8.14.2) with ESMTP id q2MMJ0I1003501; Thu, 22 Mar 2012 16:19:00 -0600 (MDT) (envelope-from ken@nargothrond.kdm.org) Received: (from ken@localhost) by nargothrond.kdm.org (8.14.2/8.14.2/Submit) id q2MMJ0Ik003500; Thu, 22 Mar 2012 16:19:00 -0600 (MDT) (envelope-from ken) Date: Thu, 22 Mar 2012 16:19:00 -0600 From: "Kenneth D. Merry" To: Jake Smith Message-ID: <20120322221900.GA3458@nargothrond.kdm.org> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2i Cc: freebsd-scsi@freebsd.org, freebsd-hackers@freebsd.org, freebsd-hardware@freebsd.org Subject: Re: LSI mps(4) driver issues with PIKE 2008/IMR (LSI SAS2008) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Mar 2012 22:19:06 -0000 On Thu, Mar 22, 2012 at 16:24:16 +0100, Jake Smith wrote: > Hello, > > I am trying to get the latest mps(4) driver in FreeBSD 9-STABLE working > with am LSI SAS2008 variant from ASUS, they call it PIKE 2008/IMR. Link > http://www.asus.com/Server_Workstation/Accessories/PIKE_2008IMR/ > > From what I can see this card should be compatible with the mps(4) > driver MFC'd to 9-STABLE about 6 weeks ago. > > # uname -a > FreeBSD xxx 9.0-STABLE FreeBSD 9.0-STABLE #3 r233304M: Thu Mar 22 > 12:53:17 GMT 2012 root@xxx:/usr/obj/usr/src/sys/GENERIC amd64 > > Revision: 233304 > > Initially the card is not seen at all by the driver, however pciconf > shows us why that is. > > mps0@pci0:2:0:0: class=0x010700 card=0x843b1043 chip=0x00731000 > rev=0x03 hdr=0x00 > vendor = 'LSI Logic / Symbios Logic' > device = 'MegaRAID SAS 9240' > class = mass storage > subclass = SAS > > It seems on other models of LSI SAS2008 the chip device ID is 0x0072, > however for some reason this card has 0x0073. So I patched the mps(4) > driver and recompiled. > > diff -ruN mps.orig/mpi/mpi2_cnfg.h mps/mpi/mpi2_cnfg.h > --- mps.orig/mpi/mpi2_cnfg.h 2012-03-22 14:50:53.000000000 +0000 > +++ mps/mpi/mpi2_cnfg.h 2012-03-22 14:52:23.000000000 +0000 > @@ -416,7 +416,8 @@ > > /* SAS */ > #define MPI2_MFGPAGE_DEVID_SAS2004 (0x0070) > -#define MPI2_MFGPAGE_DEVID_SAS2008 (0x0072) > +#define MPI2_MFGPAGE_DEVID_SAS2008_1 (0x0072) > +#define MPI2_MFGPAGE_DEVID_SAS2008_2 (0x0073) > #define MPI2_MFGPAGE_DEVID_SAS2108_1 (0x0074) > #define MPI2_MFGPAGE_DEVID_SAS2108_2 (0x0076) > #define MPI2_MFGPAGE_DEVID_SAS2108_3 (0x0077) > diff -ruN mps.orig/mps_pci.c mps/mps_pci.c > --- mps.orig/mps_pci.c 2012-03-22 14:48:41.000000000 +0000 > +++ mps/mps_pci.c 2012-03-22 14:51:59.000000000 +0000 > @@ -99,7 +99,9 @@ > } mps_identifiers[] = { > { MPI2_MFGPAGE_VENDORID_LSI, MPI2_MFGPAGE_DEVID_SAS2004, > 0xffff, 0xffff, 0, "LSI SAS2004" }, > - { MPI2_MFGPAGE_VENDORID_LSI, MPI2_MFGPAGE_DEVID_SAS2008, > + { MPI2_MFGPAGE_VENDORID_LSI, MPI2_MFGPAGE_DEVID_SAS2008_1, > + 0xffff, 0xffff, 0, "LSI SAS2008" }, > + { MPI2_MFGPAGE_VENDORID_LSI, MPI2_MFGPAGE_DEVID_SAS2008_2, > 0xffff, 0xffff, 0, "LSI SAS2008" }, > { MPI2_MFGPAGE_VENDORID_LSI, MPI2_MFGPAGE_DEVID_SAS2108_1, > 0xffff, 0xffff, 0, "LSI SAS2108" }, > > After reboot it now loads the mps(4) module and attempts to init the > card but fails. > > # dmesg | grep mps > mps0: port 0xd800-0xd8ff mem > 0xfbd7c000-0xfbd7ffff,0xfbdc0000-0xfbdfffff irq 16 at device 0.0 on pci2 > mps0: Doorbell failed to activate > device_attach: mps0 attach returned 6 > > From this point I'm stuck on what to try next, google does not provide > any answers for this situation. Does any one have any advice or ideas as > to why this is not working? > I am able to provide ssh access to the server if any one wants to log > on and have a look at it. In looking at the specs, that card supports RAID-5 and RAID-50. That means it isn't a SAS card supported by mps(4), but rather a MegaRAID card. It should be supported by mfi(4). Try adding the PCI ID to that driver and see if that works. Or you can grab the driver from the head_mfi branch, it looks like it already supports that card. Here's the mfi_pci.c file, you can see the PCI ID in there: http://svnweb.freebsd.org/base/projects/head_mfi/sys/dev/mfi/mfi_pci.c?revision=232888&view=markup Ken -- Kenneth Merry ken@FreeBSD.ORG From owner-freebsd-hackers@FreeBSD.ORG Fri Mar 23 00:41:17 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 82C041065675 for ; Fri, 23 Mar 2012 00:41:17 +0000 (UTC) (envelope-from freebsd-hackers@herveybayaustralia.com.au) Received: from mail.unitedinsong.com.au (mail.unitedinsong.com.au [150.101.178.33]) by mx1.freebsd.org (Postfix) with ESMTP id 329998FC12 for ; Fri, 23 Mar 2012 00:41:16 +0000 (UTC) Received: from laptop1.herveybayaustralia.com.au (laptop1.herveybayaustralia.com.au [192.168.0.182]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.unitedinsong.com.au (Postfix) with ESMTPSA id 9E6955C22 for ; Fri, 23 Mar 2012 10:54:49 +1000 (EST) Message-ID: <4F6BC6AB.30709@herveybayaustralia.com.au> Date: Fri, 23 Mar 2012 10:41:15 +1000 From: Da Rock User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:7.0.1) Gecko/20111109 Thunderbird/7.0.1 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org References: <20120322173717.GA54005@onelab2.iet.unipi.it> <10033788658207@192.168.2.69> In-Reply-To: <10033788658207@192.168.2.69> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: iso2flash img X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Mar 2012 00:41:17 -0000 On 03/23/12 04:46, Thomas Schmitt wrote: > Hi, > >>> The trick is called "isohybrid". > Luigi Rizzo wrote: >> interesting. It does work for me indeed. > So why not for Da Rock ? Starting to feel left out here :) I tried with your flags to dd (as opposed to those on Ubuntu - bs=1m - not that I thought it would make much diff), and it got as far as the last time. It shows isolinux 4.04, blah blah, and a blinking cursor. It goes no further than that, which I why I commented that it seemed an unlikely solution. The system is an Acer AspireOne Netbook D255. I'm using an i386 image because its only an Atom. I did test a amd64 system and it worked though... hmmm. I wonder if they mixed up their images? That'd be a funny cock-up :D > > >> And it might be a nice trick for our images too, so we don't >> have to build a memstick and an ISO image... > I would be happy to help with that. > I am the developer of program xorriso which in the role of mkisofs > has composed that Ubuntu image. My knowlege is only about pointing BIOS > to the boot loader start programs, not about those boot systems themselves. > > A while ago i exercised the most simple case of > http://wiki.freebsd.org/AvgLiveCD > with the mkisofs emulation of xorriso. It booted. > > An MBR can be inserted easily by mkisofs option -G. > isohybrid demands to patch that MBR with the LBA of the boot image > and to set up the DOS partition table. GRUB2 demands only to set up > the partition table. (Special xorrisofs options get employed.) > > What would a FreeBSD bootloader MBR need to know about the data in > the ISO image to start up and handle it like a read-only hard disk ? > Do programs of the first boot stages need to know their own LBA in > the image resp. partition ? > > The El Torito and MBR equipment of GRUB2 can provide the same functionality > as ISOLINUX with isohybrid. GRUB2 script grub-mkrescue demonstrates this. > I understand Debian GNU/kFreeBSD boots via El Torito and GRUB2. But it > makes no use of the opportunity to have an MBR too. > I boot my own FreeBSD 8-STABLE from hard disk via MBR, GRUB2 and a > chainloaded FreeBSD boot loader. > > > Have a nice day :) > > Thomas > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Fri Mar 23 07:05:36 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B4041106564A for ; Fri, 23 Mar 2012 07:05:36 +0000 (UTC) (envelope-from scdbackup@gmx.net) Received: from mailout-de.gmx.net (mailout-de.gmx.net [213.165.64.23]) by mx1.freebsd.org (Postfix) with SMTP id 281B08FC14 for ; Fri, 23 Mar 2012 07:05:35 +0000 (UTC) Received: (qmail invoked by alias); 23 Mar 2012 07:05:29 -0000 Received: from 165.126.46.212.adsl.ncore.de (HELO 192.168.2.69) [212.46.126.165] by mail.gmx.net (mp017) with SMTP; 23 Mar 2012 08:05:29 +0100 X-Authenticated: #2145628 X-Provags-ID: V01U2FsdGVkX1/U2IcxrCbUsFn3FevJNDsK4Bi7fCI/+cevvLFFe0 TPP9Yn0gNvHoVP Date: Fri, 23 Mar 2012 08:06:05 +0100 From: "Thomas Schmitt" To: freebsd-hackers@freebsd.org References: <4F6BC6AB.30709@herveybayaustralia.com.au> In-Reply-To: <4F6BC6AB.30709@herveybayaustralia.com.au> Message-Id: <100303152110179@192.168.2.69> X-Y-GMX-Trusted: 0 Subject: Re: iso2flash img X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Mar 2012 07:05:36 -0000 Hi, Da Rock wrote: > It shows isolinux 4.04, blah blah, and a blinking cursor. It goes no further > than that, which I why I commented that it seemed an unlikely solution. If it can say "isolinux" then the boot process has succeeded as far as the boot sectors of the ISO image are responsible. > The system is an Acer AspireOne Netbook D255. I'm using an i386 image > because its only an Atom. Can you try whether the Ubuntu image boots from CD or DVD ? > I did test a amd64 system and it worked though... hmmm. I wonder if they > mixed up their images? That'd be a funny cock-up :D At that stage you are still in the SYSLINUX/ISOLINUX system. Afaik, there is no 64 bit version of it. So that one can hardly be totally unsuitable for 32 bit systems. I am not familiar with the entrails of the boot loaders. Maybe you can get help at the SYSLINUX mailing list syslinux@zytor.com. http://www.zytor.com/mailman/listinfo/syslinux Google "ubuntu atom isolinux" finds an older issue: https://bugs.launchpad.net/ubuntu/+bug/774552 points to https://bugs.launchpad.net/ubuntu/+source/syslinux/+bug/617779 "Just type "help" on the BOOT prompt, and when you get the help menu, just hit enter. The system will now boot!" Does ISOLINUX allow you to enter commands ? ------------------------------------ One thing comes to my mind which you could try. It is quite unlikely to be the culprit though: By a bug in xorriso-1.0.8 the image size is not aligned to a full megabyte, as is prescribed for isohybrid. So you could try to set the end of the USB stick DOS partition 1 to the next higher multiple of 2048 disk blocks minus 1. (Make sure that no block content gets changed after byte 64 * 512.) If this happens to work, then we should inform Ubuntu to upgrade their xorriso to 1.1.0 or later. (Up to now i only know that the correct size silences warnings of Linux fdisk about "different physical/logical beginnings".) Have a nice day :) Thomas From owner-freebsd-hackers@FreeBSD.ORG Fri Mar 23 07:35:24 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6BF93106564A for ; Fri, 23 Mar 2012 07:35:24 +0000 (UTC) (envelope-from sushanth_rai@yahoo.com) Received: from nm9.bullet.mail.sp2.yahoo.com (nm9.bullet.mail.sp2.yahoo.com [98.139.91.79]) by mx1.freebsd.org (Postfix) with SMTP id 3F8C38FC08 for ; Fri, 23 Mar 2012 07:35:24 +0000 (UTC) Received: from [98.139.91.68] by nm9.bullet.mail.sp2.yahoo.com with NNFMP; 23 Mar 2012 07:35:18 -0000 Received: from [98.139.44.77] by tm8.bullet.mail.sp2.yahoo.com with NNFMP; 23 Mar 2012 07:35:18 -0000 Received: from [127.0.0.1] by omp1014.access.mail.sp2.yahoo.com with NNFMP; 23 Mar 2012 07:35:18 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 569448.29131.bm@omp1014.access.mail.sp2.yahoo.com Received: (qmail 33558 invoked by uid 60001); 23 Mar 2012 07:35:18 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1332488118; bh=QiY/WRcZzEWJ6tdJ++YjAsaIdYiRlutrnu8Ad1XGwAo=; h=X-YMail-OSG:Received:X-Mailer:Message-ID:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=KL3a+4Enkt4BXhY2V9p3vfQdm/6kxNI+Tqx+SdNscuOrFapZcOyHKoR9oNBMlXlJDTrMg3lFrBObmx//YGbXGYfglW+P1aoYBM7pfyNi6ojpbpOCwLDR8NwdMWMPKBDP0pyh+rSyVn6E5pXcEUEOs3g4PlQMaQTFFu2WMC9fnRM= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Mailer:Message-ID:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=2CGaY/HL5KkqaKcamEBdt2Xx+QaaMAXs64rbTi/hXROegAWa07wegfd3fdD/VMlC9tHYuCM+4rP133JG+Xg+z4tIiDW/gABM9H0NEEoaNhuQOoAGt3/aU/hjaRMKhejhlll7t+11cohzt3NqQ+OQT5wXej5oc8OzxnRMK3S49Jo=; X-YMail-OSG: TYHgBQkVM1mnmzacGXCD4LFI4ahzNgFeROZprPI5eYn.vkX tLjUBh9OzxacCMGSPQsladSxP4W7w7aNGBOmZndjmI5OF6skLxTLcR7JSgdP Vtih9lJ3TzfNr8i6EOEVNU8pq0Qs7rjpkDyHpbj9k1i83VHTCxP3TEWdhYW. 3HXVA9JjOD5ELpaA1IJBaWRyvyxtTcQ1soBaSz1HFmc.IEMRTIex6AU32kVR C.bV6An7c92hCskf0BSAEg4ghBbW.n6yBpihdCDjbMofP2XGF.VnDQI7yq.1 Ac9_dhwZZ3x0pAoy2ToBzRbHGk9ROQ5wt1ssQcdYzv_Qx8wEDlGxq79Q0_m. 72pQ._8.5dhMAJgrZpT24qsTnPyuxfX5uPfc_I0qix3Kz5vnO2H9Jf5fWMkU 98qm0WS8TBY9PT4msycw2nzZa2LVX8wQfEM8iFxnOkU_59wYYtb8_5vhrSFB JBNUElvcl5JtcQAVRF7hNF7ex9jT9gb0j7YCkvtH5N9QJnW54D7tOHLJiM_x IRX4dP07Hu_Th6qJB Received: from [75.62.236.160] by web180005.mail.gq1.yahoo.com via HTTP; Fri, 23 Mar 2012 00:35:17 PDT X-Mailer: YahooMailClassic/15.0.5 YahooMailWebService/0.8.117.340979 Message-ID: <1332488117.24609.YahooMailClassic@web180005.mail.gq1.yahoo.com> Date: Fri, 23 Mar 2012 00:35:17 -0700 (PDT) From: Sushanth Rai To: Konstantin Belousov In-Reply-To: <20120322140151.GB2358@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: freebsd-hackers@freebsd.org Subject: Re: Improving gcore X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Mar 2012 07:35:25 -0000 What I mean by inconsistent is that, a process with lots of threads takes a while before all threads are suspended. When I look at the resulting core file, the state of some of the shared data is not exactly what I was expecting when I issued the gcore command. It is quite possible that state might have changed even before ptrace() had a chance to issue SIGSTOP. But I am looking at any improvement that can be reasonably done in kernel. As you described suspension are checked at safe points and only when threads reach those, they get suspended. I understand and agree with the reasons behind asynchronous stopping. But the net effect is that threads can potentially run for a short duration before they suspend themselves. So, I am trying to figure out ways to reduce this duration as much as possible. One thing I noticed is that in sig_suspend_threads(), we check if the threads are sleeping interruptibly. If so, they get suspended immediately. Otherwise, set TDF_ASTPENDING and if the thread is running on CPU we send IPI_AST to that CPU. What about the target process's threads that are on the runq ? It looks like the thread will only notice the flag when it is at user->kernel boundary. Can we safely remove them out of the runq ? With respect to PT_SUSPEND, as part of PT_ATTACH request I was thinking of explicitly suspending all the threads by setting TDF_DBSUSPEND instead of posting SIGSTOP. As each thread in the target process calls thread_suspend_check(), it would notice this flag and suspend itself. PT_ATTACH command would then wait until all threads are suspended before returning to the caller. This is the general approach and ofcourse it is missing details at this point. The idea again is to suspend all threads as quickly as possible. I'm running on 7.2. Cursory look at trunk version didn't show major changes in this area. Thanks, Sushanth --- On Thu, 3/22/12, Konstantin Belousov wrote: > From: Konstantin Belousov > Subject: Re: Improving gcore > To: "Sushanth Rai" > Cc: freebsd-hackers@freebsd.org > Date: Thursday, March 22, 2012, 7:01 AM > On Wed, Mar 21, 2012 at 04:35:13PM > -0700, Sushanth Rai wrote: > > Sometimes I have trouble capturing the "correct" state > of a > > multithreaded process using gcore. That is, it looks > like target > > process might have done some work since the time > command was issued > > and the core file was generated. > > > > Looking at the code, gcore calls ptrace(PT_ATTACH...), > which > > internally issues SIGSTOP, and calls waitpid() to wait > until the > > process stops. So, it's quite possible that some > threads that are not > > sleeping interruptibly will continue to run until the > process notices > > the signal. Signals are only checked when a thread that > is tagged to > > handle the signal crosses the user boundary (return > from syscall, > > trap). When the thread finally handles SIGSTOP, it > needs to stop all > > threads, which is done by lighting a flag-bit it each > thread. This > > bit is checked as each thread crosses the user > boundary. So, there > > will always be some state change in the target process > from the time > > SIGSTOP is posted to the time all threads are actually > stopped. > Yes, this is how things work. There are two factors causing > the asynchronous > stopping: > first, other CPUs may execute several threads of the > process, so the > suspension of that other threads require an IPI to be > generated. IPI_AST > handler just returns, which causes kernel->usermode > transition and > possible signal delivery and suspend check. > > second, kernel never allows to suspend thread executing and > blocked in > kernel. Doing otherwise would cause deadlocks, because > executing threads > own resources that are shared with other threads. > > So, the only safe points to suspend the threads is at > kernel->user boundary > or at some sleep points that are not marked as unsafe with > PBDRY flag. > On the other hand, since kernel waits for all threads to > suspend before > reporting the wait(2) event, the usermode state shall be > consistent with > itself, or rather, it shall be not worse then if the threads > reach the > stop point executing asynchronously on different CPUs. > > See the check for p->p_suspcount == p->p_numthreads in > the kern_wait() > function before it decides that the found process is > satisfactory > for wait request. > > > > > I was wondering if I could improve this a bit by > calling PT_SUSPEND on > > all threads, instead of posting SIGSTOP and waiting for > all threads > > to stop. Once the core is generated, unsuspend all > threads. As with > > SIGSTOP, individual thread will only notice suspension > as they cross > > user boundary. But there is no overhead of tagging a > thread to handle > > the signal and that thread doing the suspension. The > idea is to try > > and generate the core file which reflects the running > state of the > > process as closely as possible. > PT_SUSPEND can only be called on the process which you > alread attached to. > So the call to suspend all threads of the just attached > threads is mostly > nop for your purposes. > > > > > Does this sound reasonable ? > I think you need to describe in more details what do you > mean by > inconsistent state of the threads in gcore-generated core > file, before > some conclusion could be made. > From owner-freebsd-hackers@FreeBSD.ORG Fri Mar 23 11:27:55 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1762B106566B for ; Fri, 23 Mar 2012 11:27:55 +0000 (UTC) (envelope-from freebsd-hackers@herveybayaustralia.com.au) Received: from mail.unitedinsong.com.au (mail.unitedinsong.com.au [150.101.178.33]) by mx1.freebsd.org (Postfix) with ESMTP id BBB2B8FC1D for ; Fri, 23 Mar 2012 11:27:54 +0000 (UTC) Received: from laptop1.herveybayaustralia.com.au (laptop1.herveybayaustralia.com.au [192.168.0.182]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.unitedinsong.com.au (Postfix) with ESMTPSA id C2CE65C22 for ; Fri, 23 Mar 2012 21:41:21 +1000 (EST) Message-ID: <4F6C5E33.7000506@herveybayaustralia.com.au> Date: Fri, 23 Mar 2012 21:27:47 +1000 From: Da Rock User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:7.0.1) Gecko/20111109 Thunderbird/7.0.1 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org References: <4F6BC6AB.30709@herveybayaustralia.com.au> <100303152110179@192.168.2.69> In-Reply-To: <100303152110179@192.168.2.69> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: iso2flash img X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Mar 2012 11:27:55 -0000 On 03/23/12 17:06, Thomas Schmitt wrote: > Hi, > > Da Rock wrote: >> It shows isolinux 4.04, blah blah, and a blinking cursor. It goes no further >> than that, which I why I commented that it seemed an unlikely solution. > If it can say "isolinux" then the boot process has succeeded as far > as the boot sectors of the ISO image are responsible. > > >> The system is an Acer AspireOne Netbook D255. I'm using an i386 image >> because its only an Atom. > Can you try whether the Ubuntu image boots from CD or DVD ? Thats the whole point of this exercise - I can't, no cdrom: its a netbook. >> I did test a amd64 system and it worked though... hmmm. I wonder if they >> mixed up their images? That'd be a funny cock-up :D > At that stage you are still in the SYSLINUX/ISOLINUX system. Afaik, there > is no 64 bit version of it. So that one can hardly be totally unsuitable > for 32 bit systems. > > I am not familiar with the entrails of the boot loaders. Maybe you can get > help at the SYSLINUX mailing list syslinux@zytor.com. > http://www.zytor.com/mailman/listinfo/syslinux > > Google "ubuntu atom isolinux" finds an older issue: > https://bugs.launchpad.net/ubuntu/+bug/774552 > points to > https://bugs.launchpad.net/ubuntu/+source/syslinux/+bug/617779 > "Just type "help" on the BOOT prompt, and when you get the help menu, > just hit enter. The system will now boot!" > > Does ISOLINUX allow you to enter commands ? Nope. Can't even type 'hello world!'. > > ------------------------------------ > > One thing comes to my mind which you could try. It is quite unlikely > to be the culprit though: > > By a bug in xorriso-1.0.8 the image size is not aligned to a full > megabyte, as is prescribed for isohybrid. > So you could try to set the end of the USB stick DOS partition 1 to > the next higher multiple of 2048 disk blocks minus 1. > (Make sure that no block content gets changed after byte 64 * 512.) > > If this happens to work, then we should inform Ubuntu to upgrade > their xorriso to 1.1.0 or later. > (Up to now i only know that the correct size silences warnings of > Linux fdisk about "different physical/logical beginnings".) Dunno. Tried all kinds of tricks, but no go. The client chose FreeBSD anyway, so yay! :) Ubuntu issue not my problem; I'm sure they'll work it out if it comes up again. My disk worked in VBox, so I'm sure it is just a netbook thing. I also use that disk as my "install" disk, so I'm not sure exactly what partitions been on it now, it has been used for FreeBSD, PC-BSD, Linux distros, etc. > Have a nice day :) > > Thomas > Thanks Thomas. From owner-freebsd-hackers@FreeBSD.ORG Fri Mar 23 11:38:48 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D263F1065670 for ; Fri, 23 Mar 2012 11:38:48 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 36E438FC16 for ; Fri, 23 Mar 2012 11:38:47 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q2NBcgXr017471; Fri, 23 Mar 2012 13:38:42 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q2NBcfRZ070838; Fri, 23 Mar 2012 13:38:41 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q2NBcfmx070837; Fri, 23 Mar 2012 13:38:41 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 23 Mar 2012 13:38:41 +0200 From: Konstantin Belousov To: Sushanth Rai Message-ID: <20120323113841.GJ2358@deviant.kiev.zoral.com.ua> References: <20120322140151.GB2358@deviant.kiev.zoral.com.ua> <1332488117.24609.YahooMailClassic@web180005.mail.gq1.yahoo.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="szn/SQ7qrxPzVmtp" Content-Disposition: inline In-Reply-To: <1332488117.24609.YahooMailClassic@web180005.mail.gq1.yahoo.com> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-hackers@freebsd.org Subject: Re: Improving gcore X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Mar 2012 11:38:48 -0000 --szn/SQ7qrxPzVmtp Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Fri, Mar 23, 2012 at 12:35:17AM -0700, Sushanth Rai wrote: > What I mean by inconsistent is that, a process with lots of threads > takes a while before all threads are suspended. When I look at the > resulting core file, the state of some of the shared data is not > exactly what I was expecting when I issued the gcore command. It is > quite possible that state might have changed even before ptrace() had > a chance to issue SIGSTOP. But I am looking at any improvement that > can be reasonably done in kernel. > > As you described suspension are checked at safe points and only when > threads reach those, they get suspended. I understand and agree with > the reasons behind asynchronous stopping. But the net effect is that > threads can potentially run for a short duration before they suspend > themselves. So, I am trying to figure out ways to reduce this duration > as much as possible. > > One thing I noticed is that in sig_suspend_threads(), we check if > the threads are sleeping interruptibly. If so, they get suspended > immediately. therwise, set TDF_ASTPENDING and if the thread is > running on CPU we send IPI_AST to that CPU. What about the target > process's threads that are on the runq ? It looks like the thread > will only notice the flag when it is at user->kernel boundary. Can we > safely remove them out of the runq ? No, since thread on runq shall be considered the same as the thread actually executing on CPU. It is unsafe to suspend the thread in this state, due to it potentially owning a kernel resource. It the thread on runq but not on CPU is set up to return to usermode 'immediately' after putting back on CPU, then normal AST check would cause its suspend. > > With respect to PT_SUSPEND, as part of PT_ATTACH request I was > thinking of explicitly suspending all the threads by setting > TDF_DBSUSPEND instead of posting SIGSTOP. As each thread in the target > process calls thread_suspend_check(), it would notice this flag and > suspend itself. PT_ATTACH command would then wait until all threads > are suspended before returning to the caller. This is the general > approach and ofcourse it is missing details at this point. The idea > again is to suspend all threads as quickly as possible. I do not see how this would provide any significant difference comparing with SIGSTOP delivery. The points were signals are checked and the points were suspension can be applied are essentially the same. > > I'm running on 7.2. Cursory look at trunk version didn't show major > changes in this area. Except bug fixes, there were no big changes I could remember. --szn/SQ7qrxPzVmtp Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAk9sYMEACgkQC3+MBN1Mb4hjVgCeNT51zx5YfQ5RcrwxA7ubdqMR wVAAni7MOwzMpr9S6yWDMMknAFenjLfm =kKFD -----END PGP SIGNATURE----- --szn/SQ7qrxPzVmtp-- From owner-freebsd-hackers@FreeBSD.ORG Fri Mar 23 12:03:33 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 92216106567A for ; Fri, 23 Mar 2012 12:03:33 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 62C528FC25 for ; Fri, 23 Mar 2012 12:03:33 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) by cyrus.watson.org (Postfix) with ESMTPSA id 161CD46B23; Fri, 23 Mar 2012 08:03:33 -0400 (EDT) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 78436B911; Fri, 23 Mar 2012 08:03:32 -0400 (EDT) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Thu, 22 Mar 2012 17:13:35 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p10; KDE/4.5.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <201203221713.35202.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 23 Mar 2012 08:03:32 -0400 (EDT) Cc: Eric Saint-Etienne , Ryan Stone Subject: Re: malloc pages map to user space X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Mar 2012 12:03:33 -0000 On Thursday, March 22, 2012 3:57:11 pm Eric Saint-Etienne wrote: > > If your kernel module creates a device in /dev that implements the > > mmap method, then you don't need to worry about mucking around with > > vm_maps and objects and whatnot. Your mmap method just needs to be > > able to convert offsets into the device into physical memory > > addresses, > > Yes I'm aware of this facility, thank you. > > > and the vm infrastructure will do the rest for you. > > Since this mapping is on the main path of the driver, I'm worried that > the overhead on each access of a page fault and a function call (the > pager associated with a cdev mmap) is too much to bear. It only does this on the first page fault though, not every access to the page. This can be a bit of a downfall as well as you can't easily invalidate a mapping once you've established it. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Fri Mar 23 12:08:04 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 941FC106566B for ; Fri, 23 Mar 2012 12:08:04 +0000 (UTC) (envelope-from scdbackup@gmx.net) Received: from mailout-de.gmx.net (mailout-de.gmx.net [213.165.64.22]) by mx1.freebsd.org (Postfix) with SMTP id E36738FC0A for ; Fri, 23 Mar 2012 12:08:03 +0000 (UTC) Received: (qmail invoked by alias); 23 Mar 2012 12:07:57 -0000 Received: from 165.126.46.212.adsl.ncore.de (HELO 192.168.2.69) [212.46.126.165] by mail.gmx.net (mp016) with SMTP; 23 Mar 2012 13:07:57 +0100 X-Authenticated: #2145628 X-Provags-ID: V01U2FsdGVkX19pBoQS4LVpgqOakoHfzcKckTy7sTip8UMgqqdeVB 6MT1ExKl8pctN9 Date: Fri, 23 Mar 2012 13:08:33 +0100 From: "Thomas Schmitt" To: freebsd-hackers@freebsd.org References: <4F6C5E33.7000506@herveybayaustralia.com.au> In-Reply-To: <4F6C5E33.7000506@herveybayaustralia.com.au> Message-Id: <10030481899038@192.168.2.69> X-Y-GMX-Trusted: 0 Subject: Re: iso2flash img X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Mar 2012 12:08:04 -0000 Hi, > Thats the whole point of this exercise - I can't, no cdrom: its a netbook. I hoped that you had a USB attachable optical drive in reach for development. > My disk worked in VBox, so I'm sure it is just a netbook thing. I > also use that disk as my "install" disk, so I'm not sure exactly what > partitions been on it now, it has been used for FreeBSD, PC-BSD, Linux > distros, etc. After copying the ISO image to the base device (/dev/da0 rather than /dev/da0s1), it now carries the isohybrid MBR which marks a single DOS partition and leaves the rest of the disk unclaimed. A partition editor should be able to push the end of the partition to the next 1 MiB boundary, without altering the partition content. But as said, it is unlikely that this misalignement of the partition end is the cause. The observable state of ISOLINUX rather points to a problem between hardware, firmware, and the SYSLINUX programs. Have a nice day :) Thomas From owner-freebsd-hackers@FreeBSD.ORG Fri Mar 23 12:20:54 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AD74D106566C for ; Fri, 23 Mar 2012 12:20:54 +0000 (UTC) (envelope-from freebsd-hackers@herveybayaustralia.com.au) Received: from mail.unitedinsong.com.au (mail.unitedinsong.com.au [150.101.178.33]) by mx1.freebsd.org (Postfix) with ESMTP id 614AD8FC0A for ; Fri, 23 Mar 2012 12:20:54 +0000 (UTC) Received: from laptop1.herveybayaustralia.com.au (laptop1.herveybayaustralia.com.au [192.168.0.182]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.unitedinsong.com.au (Postfix) with ESMTPSA id 246355C22 for ; Fri, 23 Mar 2012 22:34:27 +1000 (EST) Message-ID: <4F6C6AA4.90006@herveybayaustralia.com.au> Date: Fri, 23 Mar 2012 22:20:52 +1000 From: Da Rock User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:7.0.1) Gecko/20111109 Thunderbird/7.0.1 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org References: <4F6C5E33.7000506@herveybayaustralia.com.au> <10030481899038@192.168.2.69> In-Reply-To: <10030481899038@192.168.2.69> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: iso2flash img X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Mar 2012 12:20:54 -0000 On 03/23/12 22:08, Thomas Schmitt wrote: > Hi, > >> Thats the whole point of this exercise - I can't, no cdrom: its a netbook. > I hoped that you had a USB attachable optical drive in reach for development. Haven't you heard? CD's are so yesterday... ;) The reality is I don't have a working CDROM unless its builtin. Haven't needed one for years now, so no point. The servers and desktops have one as well, but most are not working very well anymore due to non use - too much bump and not enough grind. The laptops give me what I need atm. > >> My disk worked in VBox, so I'm sure it is just a netbook thing. I >> also use that disk as my "install" disk, so I'm not sure exactly what >> partitions been on it now, it has been used for FreeBSD, PC-BSD, Linux >> distros, etc. > After copying the ISO image to the base device (/dev/da0 rather than > /dev/da0s1), it now carries the isohybrid MBR which marks a single DOS > partition and leaves the rest of the disk unclaimed. A partition editor > should be able to push the end of the partition to the next 1 MiB boundary, > without altering the partition content. > > But as said, it is unlikely that this misalignement of the partition end > is the cause. The observable state of ISOLINUX rather points to a problem > between hardware, firmware, and the SYSLINUX programs. My thoughts exactly. Could be Acer, netbook firmware, or the Atom I'd say (or a combination of these). From owner-freebsd-hackers@FreeBSD.ORG Fri Mar 23 12:37:38 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 600B3106566C for ; Fri, 23 Mar 2012 12:37:38 +0000 (UTC) (envelope-from scdbackup@gmx.net) Received: from mailout-de.gmx.net (mailout-de.gmx.net [213.165.64.23]) by mx1.freebsd.org (Postfix) with SMTP id EE54E8FC18 for ; Fri, 23 Mar 2012 12:37:37 +0000 (UTC) Received: (qmail invoked by alias); 23 Mar 2012 12:37:36 -0000 Received: from 165.126.46.212.adsl.ncore.de (HELO 192.168.2.69) [212.46.126.165] by mail.gmx.net (mp041) with SMTP; 23 Mar 2012 13:37:36 +0100 X-Authenticated: #2145628 X-Provags-ID: V01U2FsdGVkX18w4LNjrxMwbADxr8wD+Q9QHztQcPFGQ++mTJGWrD QV1dlJTCGdOj6N Date: Fri, 23 Mar 2012 13:38:13 +0100 From: "Thomas Schmitt" To: freebsd-hackers@freebsd.org References: <4F6C6AA4.90006@herveybayaustralia.com.au> In-Reply-To: <4F6C6AA4.90006@herveybayaustralia.com.au> Message-Id: <100304628118291@192.168.2.69> X-Y-GMX-Trusted: 0 Subject: Re: iso2flash img X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Mar 2012 12:37:38 -0000 Hi, > Haven't you heard? CD's are so yesterday... ;) Just wait until the holodiscs come out. :)) http://en.wikipedia.org/wiki/Holographic_Versatile_Disc Have a nice day :) Thomas From owner-freebsd-hackers@FreeBSD.ORG Fri Mar 23 12:55:07 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E6188106566C for ; Fri, 23 Mar 2012 12:55:07 +0000 (UTC) (envelope-from freebsd-hackers@herveybayaustralia.com.au) Received: from mail.unitedinsong.com.au (mail.unitedinsong.com.au [150.101.178.33]) by mx1.freebsd.org (Postfix) with ESMTP id 8DFA28FC14 for ; Fri, 23 Mar 2012 12:55:07 +0000 (UTC) Received: from laptop1.herveybayaustralia.com.au (laptop1.herveybayaustralia.com.au [192.168.0.182]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.unitedinsong.com.au (Postfix) with ESMTPSA id F0E065C22 for ; Fri, 23 Mar 2012 23:08:39 +1000 (EST) Message-ID: <4F6C72A9.5010805@herveybayaustralia.com.au> Date: Fri, 23 Mar 2012 22:55:05 +1000 From: Da Rock User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:7.0.1) Gecko/20111109 Thunderbird/7.0.1 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org References: <4F6C6AA4.90006@herveybayaustralia.com.au> <100304628118291@192.168.2.69> In-Reply-To: <100304628118291@192.168.2.69> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: iso2flash img X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Mar 2012 12:55:08 -0000 On 03/23/12 22:38, Thomas Schmitt wrote: > Hi, > >> Haven't you heard? CD's are so yesterday... ;) > Just wait until the holodiscs come out. :)) > http://en.wikipedia.org/wiki/Holographic_Versatile_Disc Now thats what I'm talkin' 'bout! Although I could still probably fill one in a matter of hours... ;) From owner-freebsd-hackers@FreeBSD.ORG Fri Mar 23 19:14:21 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C5AFF1065674 for ; Fri, 23 Mar 2012 19:14:21 +0000 (UTC) (envelope-from rank1seeker@gmail.com) Received: from mail-we0-f182.google.com (mail-we0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 544878FC0C for ; Fri, 23 Mar 2012 19:14:20 +0000 (UTC) Received: by wern13 with SMTP id n13so3857097wer.13 for ; Fri, 23 Mar 2012 12:14:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:from:to:subject:date:content-type :content-transfer-encoding:in-reply-to:references:x-mailer; bh=RRiwg48cqjZLjcM5Vke+d8xL36ECiSozo1tDD/A53dE=; b=FupAMrg3coEMCGZ6NP1fiBXQ+eXyatNRLnyapUmVZie4r8/BQmyXOvuQjiFIRXlKd+ XEWM8kZuBiU5xe9EZJl0mn7xz1V87jwO+ctfqztQx7C9XOQwFx6uvdeaApYW843BMs1d idP6GQsppoZFQ1QI57lxnzQFpQzQKcN+dRmvJRvJU+xTcx7hPfgxH0AzCWlhDTxN22bD xi1ikDfynA2IsfQC0mAmqYNQZm4H1ohnsSEXd1ek9AibvX9U6MaNeVK0YFF2Lyg+SjPB dMjPabiv3klJYfisIvuJY83JV2RXhLkZ+MAMWH6HFRsEdatgSxS6f4xrXS7xINTwvcMt 4C2A== Received: by 10.216.134.136 with SMTP id s8mr7798065wei.6.1332530060133; Fri, 23 Mar 2012 12:14:20 -0700 (PDT) Received: from DOMYPC ([82.193.208.173]) by mx.google.com with ESMTPS id k6sm16112105wie.9.2012.03.23.12.13.58 (version=SSLv3 cipher=OTHER); Fri, 23 Mar 2012 12:14:17 -0700 (PDT) Message-ID: <20120323.191420.168.1@DOMY-PC> From: rank1seeker@gmail.com To: "Benjamin Kaduk" , hackers@freebsd.org Date: Fri, 23 Mar 2012 20:14:20 +0100 Content-Type: text/plain; charset="Windows-1250" Content-Transfer-Encoding: quoted-printable In-Reply-To: References: <20120318.130139.003.1@DOMY-PC> <20120318134156.61d6db61@gumby.homeunix.com> <20120320.200234.909.1@DOMY-PC> X-Mailer: POP Peeper (3.8.1.0) Cc: Subject: Re: BUG: REL 9.0 - MD malloc of custom sector size X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Mar 2012 19:14:21 -0000 ----- Original Message -----=0D=0AFrom: Benjamin Kaduk = =0D=0ATo: rank1seeker@gmail.com=0D=0ACc: = hackers@freebsd.org=0D=0ADate: Tue, 20 Mar 2012 23:42:44 -0400 = (EDT)=0D=0ASubject: Re: BUG: REL 9.0 - MD malloc of custom sector = size=0D=0A=0D=0A> On Tue, 20 Mar 2012, rank1seeker@gmail.com = wrote:=0D=0A> =0D=0A> > ----- Original Message -----=0D=0A> > From: RW = =0D=0A> > To: = freebsd-hackers@freebsd.org=0D=0A> > Date: Sun, 18 Mar 2012 13:41:56 = +0000=0D=0A> > Subject: Re: BUG: REL 9.0 - MD malloc of custom sector = size=0D=0A> >=0D=0A> >> On Sun, 18 Mar 2012 14:01:39 +0100=0D=0A> >> = rank1seeker@gmail.com wrote:=0D=0A> >>=0D=0A> >>> man mdconfig=0D=0A> >>> = ----=0D=0A> >>> -S sectorsize to use for malloc backed device=0D=0A> >>> = ----=0D=0A> >>>=0D=0A> >>> I want to create MD device, with sector size = of 4 Kb.=0D=0A> >>>=0D=0A> >>> It is CRITICAL to NOT append ANY suffixes, = when specifing size, via=0D=0A> >>> '-s' flag in order to use sectors, to = set it's size. # mdconfig -a -t=0D=0A> >>> malloc -S 4096 -s 32768=0D=0A> = >>>=0D=0A> >>> This should created dev of 128 Mb in size.=0D=0A> >>> = 32768 sectors * 4 Kb each =3D 131072 Kb =3D 128 Mb=0D=0A> >>> Not! It = created dev of 16 Mb in size, because sector size remained at=0D=0A> >>> = 512 bytes.=0D=0A> >>=0D=0A> >>=0D=0A> >> From mdconfig 8=0D=0A> >>=0D=0A> = >> "Size is the number of 512 byte sectors unless ..."=0D=0A> >>=0D=0A> = >> Looks to me like it's doing what it said it would.=0D=0A> >>=0D=0A> >> = BTW are you sure you want to use "-t malloc". This keeps the = files=0D=0A> >> (even the deleted ones) in memory unconditionally while = ordinary=0D=0A> >> process memory is paged-out.=0D=0A> >>=0D=0A> >=0D=0A> = > My MAIN reason to hassle with MD here, is to test a custom sector = size.=0D=0A> > This can be done with '-S' flag only, in order to set = sectorsize of /dev/md*=0D=0A> > But as it is malloc ONLY option/flag, I = must combine it with '-t malloc'=0D=0A> >=0D=0A> > Then I've defined it's = size by amount of sectors and as I've redefined =0D=0A> > size of 1 = sector, it simply isn't doing it's task. Because it enforces =0D=0A> > = hardcoded size of 512 bytes, so documentation should not misleadingly = =0D=0A> > refer to sector in any way, but a hardcode value of 0.5 Kb, no = matter of =0D=0A> > real/actual sector size is.=0D=0A> =0D=0A> It should = not be technically challenging to cause mdconfig to have the -s =0D=0A> = size argument in terms of -S sectorsize sized sectors; the following = would =0D=0A> probably suffice (untested):=0D=0A> = %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%=0D=0A> Index: mdconfig.c=0D=0A> = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0D=0A> = --- mdconfig.c (revision 233159)=0D=0A> +++ mdconfig.c (working = copy)=0D=0A> @@ -94,16 +94,19 @@=0D=0A> int=0D=0A> main(int argc, = char **argv)=0D=0A> {=0D=0A> - int ch, fd, i, vflag;=0D=0A> + int ch, = fd, i, vflag, sflag;=0D=0A> char *p;=0D=0A> char *fflag =3D NULL, = *tflag =3D NULL, *uflag =3D NULL;=0D=0A> + unsigned bsize;=0D=0A> =0D=0A> = bzero(&mdio, sizeof(mdio));=0D=0A> mdio.md_file =3D = malloc(PATH_MAX);=0D=0A> if (mdio.md_file =3D=3D NULL)=0D=0A> = err(1, "could not allocate memory");=0D=0A> vflag =3D 0;=0D=0A> + = sflag =3D 0;=0D=0A> bzero(mdio.md_file, PATH_MAX);=0D=0A> + bsize =3D = DEV_BSIZE;=0D=0A> =0D=0A> if (argc =3D=3D 1)=0D=0A> = usage();=0D=0A> @@ -186,11 +189,12 @@=0D=0A> break;=0D=0A> case = 'S':=0D=0A> mdio.md_sectorsize =3D strtoul(optarg, &p, 0);=0D=0A> + = bsize =3D mdio.md_sectorsize;=0D=0A> break;=0D=0A> case = 's':=0D=0A> mdio.md_mediasize =3D (off_t)strtoumax(optarg, &p, = 0);=0D=0A> if (p =3D=3D NULL || *p =3D=3D '\0')=0D=0A> - = mdio.md_mediasize *=3D DEV_BSIZE;=0D=0A> + sflag =3D 1;=0D=0A> = else if (*p =3D=3D 'b' || *p =3D=3D 'B')=0D=0A> ; /* do nothing = */=0D=0A> else if (*p =3D=3D 'k' || *p =3D=3D 'K')=0D=0A> @@ -232,6 = +236,9 @@=0D=0A> if (action =3D=3D UNSET)=0D=0A> action =3D = ATTACH;=0D=0A> =0D=0A> + if (sflag =3D=3D 1)=0D=0A> + mdio.md_mediasize = *=3D bsize;=0D=0A> +=0D=0A> if (action =3D=3D ATTACH) {=0D=0A> if = (tflag =3D=3D NULL) {=0D=0A> /*=0D=0A> = %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%=0D=0A> =0D=0A> Or would you prefer a = man page change?=0D=0A=0D=0AI would prefer both.=0D=0AAbove patch to make = it default to sector size of 512 bytes, IF not overridden with -S flag = (then sector size is set to it)=0D=0AInstead to a hardcoded value of 512 = bytes.=0D=0A=0D=0AAnd manpages to document that ... (just mention = sectorsize - it's size may vary)=0D=0A=0D=0A> =0D=0A> = %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%=0D=0A> Index: mdconfig.8=0D=0A> = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0D=0A> = --- mdconfig.8 (revision 233159)=0D=0A> +++ mdconfig.8 (working = copy)=0D=0A> @@ -132,7 +132,7 @@=0D=0A> .It Fl s Ar size=0D=0A> Size = of the memory disk.=0D=0A> .Ar Size=0D=0A> -is the number of 512 byte = sectors unless suffixed with a=0D=0A> +is measured in increments of 512 = byes unless suffixed with a=0D=0A> .Cm b , k , m , g ,=0D=0A> = or=0D=0A> .Cm t=0D=0A> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%=0D=0A> =0D=0A> = BTW I do not see where -S is a malloc-only option; please show the = command =0D=0A> line and error message using a vnode- or swap-backed = device.=0D=0A> =0D=0A> -Ben Kaduk=0D=0A> =0D=0A=0D=0AFrom man = pages:=0D=0A----=0D=0A-S sectorsize Sectorsize to use for MALLOC backed = device.=0D=0A----=0D=0AWhen I tried:=0D=0A# mdconfig -S 4096 -nf = /usr/IT=0D=0Ausage: mdconfig -a -t type [-n] [-o [no]option] ... [-f = file]=0D=0A [-s size] [-S sectorsize] [-u unit]=0D=0A = [-x sectors/track] [-y = heads/cylinder]=0D=0A...=0D=0A----=0D=0ASo it made me into thinking that = manpages were correct.=0D=0A=0D=0AHowever, when I've tried this, it = worked:=0D=0A------------=0D=0A# mdconfig -nf /usr/IT -S 4096=0D=0A# = diskinfo -v md0=0D=0Amd0=0D=0A 4096 # sectorsize=0D=0A = 2147483648 # mediasize in bytes (2.0G)=0D=0A 524288 = # mediasize in sectors=0D=0A 0 # = stripesize=0D=0A 0 # = stripeoffset=0D=0A------------=0D=0A=0D=0A=0D=0AMan page for -S flag, = should be updated, to note that only INT value (without SI suffixes) is = accepted, which represents bytes.=0D=0AThis didn't work:=0D=0A-S = 4k=0D=0AActually it did! I've ended up with device of 4 byte sector size! = :P=0D=0A=0D=0A=0D=0ADomagoj Smol=E8i=E6=0D=0A From owner-freebsd-hackers@FreeBSD.ORG Sat Mar 24 07:53:25 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8902E106566C for ; Sat, 24 Mar 2012 07:53:25 +0000 (UTC) (envelope-from sushanth_rai@yahoo.com) Received: from nm23-vm0.bullet.mail.sp2.yahoo.com (nm23-vm0.bullet.mail.sp2.yahoo.com [98.139.91.224]) by mx1.freebsd.org (Postfix) with SMTP id 452D68FC17 for ; Sat, 24 Mar 2012 07:53:25 +0000 (UTC) Received: from [98.139.91.69] by nm23.bullet.mail.sp2.yahoo.com with NNFMP; 24 Mar 2012 07:53:19 -0000 Received: from [98.139.44.75] by tm9.bullet.mail.sp2.yahoo.com with NNFMP; 24 Mar 2012 07:52:29 -0000 Received: from [127.0.0.1] by omp1012.access.mail.sp2.yahoo.com with NNFMP; 24 Mar 2012 07:52:29 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 724668.95578.bm@omp1012.access.mail.sp2.yahoo.com Received: (qmail 50313 invoked by uid 60001); 24 Mar 2012 07:52:29 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1332575549; bh=t/a98aNrDsL/gnb3k6r9RTJk2xFZjA3U2w+W+VsX+gs=; h=X-YMail-OSG:Received:X-Mailer:Message-ID:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=Y1a7K+Pm7kYr5yFN7xQtewBcYhOaco61hud/N1P0E1XHSTzYJ8f0AjzNzYaDXxj+EUQc1cYWu1T7q+ehQGBiPffZersW+/cEtfHPyUpiRWTkwnABuJoFeHUerz46ZhBCrsC+OxrnHZMRlLPSIdJN/YMrLseK8U9eEDHTcLv3Fn8= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Mailer:Message-ID:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=f5ADZgHV9cmkQAFANvWHqQ8HrjpRNvHgcjg26QGAbC6mHKz0cFu6rxdH74cwFZaHN4QV3V7v8osU7KtbqkmYZP/KrXuCZiLkMyaqelRPge+h+UhD6gF7TTr6vkGjYpvHtlmA8b3hW5T6SQxjelGGCs5IeyF4/9R+R98xzidHaXQ=; X-YMail-OSG: yAa2lXcVM1n0Hc0EYspk9fc03zkwsPNI7OFiT09YZXrOxx8 Ief2tUv7XYiJ85y3kKTqotLiCs9DqxE3vtGQLD0D7E5TBEykyj5dnUmm5g4Q emzWZBYwXUSg0C_4KZE1LwfqZv6lzk9FE_DWEx2xAJyNucmarNiMzFpB0FFT vsDZbbl0N8Cmok6s3pSs61jL.osK2ZoLH1Uk2aayMMZBeVr_fo0Sa3RbHenz 1rDUeS4kqw1fitZWrZElum9W28J1GnIvCByI1FwXrtli25Xaselo1J7MMVcl mWSJvd2YfLh.ypJYjdLLbSCvQRPVOj5CAgZrJG9j8ilicWEEY8_Jwe22ItSr cW.yNPXSd0YhhDGxCwyiEVqn27hjGcIOi0F6vBuLkOCNLbTz1Q.gzV3wAkJo JLg1zI9_YeDqeU8_c3VimHNulL5csyeu64O4q7uH8sKlf8jgP3DtGIXe6xEC fH5gn9HE- Received: from [75.62.236.160] by web180015.mail.gq1.yahoo.com via HTTP; Sat, 24 Mar 2012 00:52:29 PDT X-Mailer: YahooMailClassic/15.0.5 YahooMailWebService/0.8.117.340979 Message-ID: <1332575549.45446.YahooMailClassic@web180015.mail.gq1.yahoo.com> Date: Sat, 24 Mar 2012 00:52:29 -0700 (PDT) From: Sushanth Rai To: Konstantin Belousov In-Reply-To: <20120323113841.GJ2358@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: freebsd-hackers@freebsd.org Subject: Re: Improving gcore X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Mar 2012 07:53:25 -0000 --- On Fri, 3/23/12, Konstantin Belousov wrote: Can we > > safely remove them out of the runq ? > No, since thread on runq shall be considered the same as the > thread > actually executing on CPU. It is unsafe to suspend the > thread in this > state, due to it potentially owning a kernel resource. > > It the thread on runq but not on CPU is set up to return to > usermode > 'immediately' after putting back on CPU, then normal AST > check would > cause its suspend. Threads could have been running in user space and they are on the runq because their time slice expired or they yielded the CPU or got preempted. These threads will only notice the suspension when they enter the kernel via syscall or trap. If we can identify that a thread got switched-out for any of these reasons then it's reasonable to remove it from the runq when dealing with suspensions. >> approach and ofcourse it is missing details at this point. The idea >> again is to suspend all threads as quickly as possible. > I do not see how this would provide any significant difference comparing > with SIGSTOP delivery. The points were signals are checked and the points > were suspension can be applied are essentially the same. I tend to agree with this. Thanks, Sushanth From owner-freebsd-hackers@FreeBSD.ORG Sat Mar 24 08:01:35 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A7DFB106566B for ; Sat, 24 Mar 2012 08:01:35 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 365968FC1C for ; Sat, 24 Mar 2012 08:01:33 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q2O81UaM017817; Sat, 24 Mar 2012 10:01:30 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q2O81TbA013000; Sat, 24 Mar 2012 10:01:29 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q2O81Twn012999; Sat, 24 Mar 2012 10:01:29 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 24 Mar 2012 10:01:29 +0200 From: Konstantin Belousov To: Sushanth Rai Message-ID: <20120324080129.GT2358@deviant.kiev.zoral.com.ua> References: <20120323113841.GJ2358@deviant.kiev.zoral.com.ua> <1332575549.45446.YahooMailClassic@web180015.mail.gq1.yahoo.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="NYBcJxZzAu4ovuvd" Content-Disposition: inline In-Reply-To: <1332575549.45446.YahooMailClassic@web180015.mail.gq1.yahoo.com> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-hackers@freebsd.org Subject: Re: Improving gcore X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Mar 2012 08:01:35 -0000 --NYBcJxZzAu4ovuvd Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Mar 24, 2012 at 12:52:29AM -0700, Sushanth Rai wrote: >=20 >=20 > --- On Fri, 3/23/12, Konstantin Belousov wrote: >=20 >=20 > Can we > > > safely remove them out of the runq ? > > No, since thread on runq shall be considered the same as the > > thread > > actually executing on CPU. It is unsafe to suspend the > > thread in this > > state, due to it potentially owning a kernel resource. > >=20 > > It the thread on runq but not on CPU is set up to return to > > usermode > > 'immediately' after putting back on CPU, then normal AST > > check would > > cause its suspend. >=20 > Threads could have been running in user space and they are on the runq be= cause their time slice expired or they yielded the CPU or got preempted. Th= ese threads will only notice the suspension when they enter the kernel via = syscall or trap. If we can identify that a thread got switched-out for any = of these reasons then it's reasonable to remove it from the runq when deali= ng with suspensions. >=20 No, I mentioned exactly this in paragraph you replied to. To actually start executing from runq, thread needs to transition from kernel to userspace (in other words, thread appears on runq due to interrupt, thus entering kernel space). On the kernel->user transition, the thread will be suspended in AST handler. So, if pending AST catched usermode thread on runq, no single usermode instruction is executed by the thread before suspension. > >> approach and ofcourse it is missing details at this point. The idea > >> again is to suspend all threads as quickly as possible. > > I do not see how this would provide any significant difference comparing > > with SIGSTOP delivery. The points were signals are checked and the poin= ts > > were suspension can be applied are essentially the same. >=20 > I tend to agree with this. >=20 > Thanks, > Sushanth >=20 --NYBcJxZzAu4ovuvd Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAk9tf1kACgkQC3+MBN1Mb4jUagCggAiHhGjluqs25dH1DuJM1Occ aTkAnAwkD5VqXM67QGjUcugkMr0oKyKL =h8ru -----END PGP SIGNATURE----- --NYBcJxZzAu4ovuvd--