From owner-freebsd-current Thu Apr 27 1:26: 3 2000 Delivered-To: freebsd-current@freebsd.org Received: from morpheus.skynet.be (morpheus.skynet.be [195.238.2.39]) by hub.freebsd.org (Postfix) with ESMTP id B4B5C37B57C for ; Thu, 27 Apr 2000 01:25:59 -0700 (PDT) (envelope-from blk@skynet.be) Received: from [195.238.1.121] (brad.techos.skynet.be [195.238.1.121]) by morpheus.skynet.be (Postfix) with ESMTP id C498CDCC1; Thu, 27 Apr 2000 10:24:28 +0200 (MET DST) Mime-Version: 1.0 X-Sender: blk@pop.skynet.be (Unverified) Message-Id: In-Reply-To: <200004270605.XAA00807@apollo.backplane.com> References: <200004270554.BAA34693@bb01f39.unx.sas.com> <200004270605.XAA00807@apollo.backplane.com> Date: Thu, 27 Apr 2000 10:18:34 +0200 To: Matthew Dillon , "John W. DeBoskey" From: Brad Knowles Subject: Re: Support for large mfs Cc: freebsd-current@FreeBSD.ORG Content-Type: text/plain; charset="us-ascii" ; format="flowed" Sender: owner-freebsd-current@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG At 11:05 PM -0700 2000/4/26, Matthew Dillon wrote: > You should be able to create a large virtual VN device. man > vnconfig for more information - you have the choice of making it > file-backed, swap-backed, or swap-backed with the swap pre-reserved. > file-backed VN devices have a sector size of 512 bytes (which is > standard). Swap-backed VN devices have a sector size of a page (4K on > PC's). Hmm. This has got me thinking.... I use a mfs for storing the Diablo history file on our news peering server. Yes, I know the front part of the file is mmap()'ed and effectively kept completely in memory anyway, but I've seen periods of time when we received over 160,000 articles in a single hour (an average of about 45 per second), and if you compare this to our normal ratio of offered versus accepted articles (something in the range of 32,238,303 vs. 612,429; for a 52.64:1 ratio), this would imply we probably did something like 2,368.8 history lookups per second during that period of time -- and this is just for inbound articles. In my experience, it is a non-trivial exercise to build a drive array system that can keep up with the number of disk accesses necessary to handle this many history lookups per second. I think I've recently done it (and reported my testing results on news.software.nntp, along with summarizing the previous test results from Joe Greco and Terry Kennedy), but it's still non-trivial and the solutions I've seen so far are all still rather expensive. Thus the reason why I currently continue to use an mfs for the history database. However, now I'm wondering if it might not be better to use a file-backed or maybe a swap-backed VN device instead of an mfs. Do you have any thoughts? -- These are my opinions -- not to be taken as official Skynet policy ====================================================================== Brad Knowles, || Belgacom Skynet SA/NV Systems Architect, Mail/News/FTP/Proxy Admin || Rue Colonel Bourg, 124 Phone/Fax: +32-2-706.13.11/12.49 || B-1140 Brussels http://www.skynet.be || Belgium To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-current" in the body of the message