From owner-freebsd-questions@FreeBSD.ORG Sun Feb 6 22:11:48 2005 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D2A7516A4CE for ; Sun, 6 Feb 2005 22:11:48 +0000 (GMT) Received: from rwcrmhc11.comcast.net (rwcrmhc11.comcast.net [204.127.198.35]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9A05243D48 for ; Sun, 6 Feb 2005 22:11:48 +0000 (GMT) (envelope-from mhines@cs.fsu.edu) Received: from beethoven.cs.fsu.edu (pcp09078029pcs.mkethn01.fl.comcast.net[69.240.101.248]) by comcast.net (rwcrmhc11) with SMTP id <2005020622114601300k568ve>; Sun, 6 Feb 2005 22:11:48 +0000 Message-Id: <6.2.0.14.2.20050206165241.03375a30@mail.cs.fsu.edu> X-Mailer: QUALCOMM Windows Eudora Version 6.2.0.14 Date: Sun, 06 Feb 2005 17:11:48 -0500 To: Nagilum From: "Michael R. Hines" In-Reply-To: <420620DE.90007@nagilum.org> References: <420620DE.90007@nagilum.org> Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1"; format=flowed Content-Transfer-Encoding: quoted-printable cc: freebsd-questions@freebsd.org Subject: Re: increasing swap activity X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Feb 2005 22:11:49 -0000 Ok....Here's the system's design: What I have created is a 3-part distributed memory system: 1. a client machine setup as a diskless client has a page fault or wants to= =20 swap out a page. The request goes through NFS to a machine. 2. The machine that receives the request is a linux machine running a=20 kernel module. The module does the following: - acts as a purely in-memory NFS server - parses and interprets the RPC/NFS packet - re-schedules the request (and it's transmitted page data) to=20 ANOTHER machine which either holds the requested page or stores the swapped= =20 out page 3. This third machine is also a linux machine running a second kernel=20 module that acts as pure memory storage. It answers the 2nd machine's=20 request by sending back the data or storing the data and alerts the second= =20 machine. 4. finally, the second machine creates and NFS/RPC reply to send back to=20 the client machine, and it can continue processing. So you're prolly asking yourself: Why in the world would someone do=20 something this crazy? Well: - The system only operates within a gigabit-switched cluster of machines. - The system is supposed to scale: i.e. LOTS of client machines, one single= =20 second machine (which I call an engine) and LOTS of 3rd machines, called=20 backend-servers. - All of this is multiplexed and demultiplexed by one central engine. - It creates a pool of memory that is virtualized by the engine. i.e. any=20 client page could be stored on any back-end server machine at any time. - There are no disks whatsoever involved in the whole processes. Disk-swap page-fault latencies range between: 7 to 11 milliseconds My system?: 650 MICROseconds =3D) So, the system isn't the bottle-neck. It's the FreeBSD machine's swapper,=20 which is being used a client. So, I'd like to drastically boost the activity of the FreeBSD swapper. Make sense? At 08:51 AM 2/6/2005, Nagilum wrote: >Michael Hines wrote: > >> >>I've written a remote-memory system for a thesis of mine. As a result,=20 >>I'm able to setup a diskless client that swaps to remote memory instead=20 >>of a remote disk. The specifics, reasons, and design of the system are a= =20 >>long story.... >> >>However - this system allows for much faster page-fault latencies (on the= =20 >>order of 10-20 times faster than using a disk for swap space). The=20 >>problem is: I cannot get the swapper to page-out or page-in data any= faster. >> >>My end question is: how would one DRASTICALLY increase the rate at which= =20 >>the system does its paging in freebsd? >> >>Everthing I find on the net says "don't mess with freebsd's VM system or= =20 >>you'll die and go to hell." >> >>However, I do in fact need to drastically increase the paging bandwidth. >> >>Anybody know how? Preferably during runtime? >> >>Thanks a lot. >> >>/*********************************/ >>Michael R. Hines >>Grad Student, Florida State >>Dept. Computer Science >>http://www.cs.fsu.edu/~mhines/ >>Jusqu'a ce que le futur vienne... >>/*********************************/ > > >I think the first thing to do is determine the bottleneck you're fighting. >Maybe using several machines as swap-slaves would help? (the swapspace=20 >will be used interleaved) >Also it would help to explain a bit more what you have done so far. Have=20 >you written your own swapfs? >Kind regards, >Alex. > >_______________________________________________ >freebsd-questions@freebsd.org mailing list >http://lists.freebsd.org/mailman/listinfo/freebsd-questions >To unsubscribe, send any mail to= "freebsd-questions-unsubscribe@freebsd.org" /*********************************/ Michael R. Hines Grad Student, Florida State Dept. Computer Science http://www.cs.fsu.edu/~mhines/ Jusqu'=E0 ce que le futur vienne... /*********************************/