Date: Sun, 06 Feb 2005 17:11:48 -0500 From: "Michael R. Hines" <mhines@cs.fsu.edu> To: Nagilum <freebsd@nagilum.org> Cc: freebsd-questions@freebsd.org Subject: Re: increasing swap activity Message-ID: <6.2.0.14.2.20050206165241.03375a30@mail.cs.fsu.edu> In-Reply-To: <420620DE.90007@nagilum.org> References: <Pine.GSO.4.61.0502060431480.25671@diablo.cs.fsu.edu> <420620DE.90007@nagilum.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Ok....Here's the system's design:
What I have created is a 3-part distributed memory system:
1. a client machine setup as a diskless client has a page fault or wants to=
=20
swap out a page. The request goes through NFS to a machine.
2. The machine that receives the request is a linux machine running a=20
kernel module. The module does the following:
- acts as a purely in-memory NFS server
- parses and interprets the RPC/NFS packet
- re-schedules the request (and it's transmitted page data) to=20
ANOTHER machine which either holds the requested page or stores the swapped=
=20
out page
3. This third machine is also a linux machine running a second kernel=20
module that acts as pure memory storage. It answers the 2nd machine's=20
request by sending back the data or storing the data and alerts the second=
=20
machine.
4. finally, the second machine creates and NFS/RPC reply to send back to=20
the client machine, and it can continue processing.
So you're prolly asking yourself: Why in the world would someone do=20
something this crazy?
Well:
- The system only operates within a gigabit-switched cluster of machines.
- The system is supposed to scale: i.e. LOTS of client machines, one single=
=20
second machine (which I call an engine) and LOTS of 3rd machines, called=20
backend-servers.
- All of this is multiplexed and demultiplexed by one central engine.
- It creates a pool of memory that is virtualized by the engine. i.e. any=20
client page could be stored on any back-end server machine at any time.
- There are no disks whatsoever involved in the whole processes.
Disk-swap page-fault latencies range between: 7 to 11 milliseconds
My system?: 650 MICROseconds =3D)
So, the system isn't the bottle-neck. It's the FreeBSD machine's swapper,=20
which is being used a client.
So, I'd like to drastically boost the activity of the FreeBSD swapper.
Make sense?
At 08:51 AM 2/6/2005, Nagilum wrote:
>Michael Hines wrote:
>
>>
>>I've written a remote-memory system for a thesis of mine. As a result,=20
>>I'm able to setup a diskless client that swaps to remote memory instead=20
>>of a remote disk. The specifics, reasons, and design of the system are a=
=20
>>long story....
>>
>>However - this system allows for much faster page-fault latencies (on the=
=20
>>order of 10-20 times faster than using a disk for swap space). The=20
>>problem is: I cannot get the swapper to page-out or page-in data any=
faster.
>>
>>My end question is: how would one DRASTICALLY increase the rate at which=
=20
>>the system does its paging in freebsd?
>>
>>Everthing I find on the net says "don't mess with freebsd's VM system or=
=20
>>you'll die and go to hell."
>>
>>However, I do in fact need to drastically increase the paging bandwidth.
>>
>>Anybody know how? Preferably during runtime?
>>
>>Thanks a lot.
>>
>>/*********************************/
>>Michael R. Hines
>>Grad Student, Florida State
>>Dept. Computer Science
>>http://www.cs.fsu.edu/~mhines/
>>Jusqu'a ce que le futur vienne...
>>/*********************************/
>
>
>I think the first thing to do is determine the bottleneck you're fighting.
>Maybe using several machines as swap-slaves would help? (the swapspace=20
>will be used interleaved)
>Also it would help to explain a bit more what you have done so far. Have=20
>you written your own swapfs?
>Kind regards,
>Alex.
>
>_______________________________________________
>freebsd-questions@freebsd.org mailing list
>http://lists.freebsd.org/mailman/listinfo/freebsd-questions
>To unsubscribe, send any mail to=
"freebsd-questions-unsubscribe@freebsd.org"
/*********************************/
Michael R. Hines
Grad Student, Florida State
Dept. Computer Science
http://www.cs.fsu.edu/~mhines/
Jusqu'=E0 ce que le futur vienne...
/*********************************/
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?6.2.0.14.2.20050206165241.03375a30>