Date: Sat, 13 Nov 1999 21:18:46 -0500 From: Greg Lehey <grog@mojave.sitaranetworks.com> To: "Kenneth D. Merry" <ken@kdm.org>, Simon Shapiro <shimon@simon-shapiro.org> Cc: Randell Jesup <rjesup@wgate.com>, freebsd-arch@freebsd.org Subject: Re: I/O Evaluation Questions (Long but interesting!) Message-ID: <19991113211846.31442@mojave.sitaranetworks.com> In-Reply-To: <199911120116.SAA30871@panzer.kdm.org>; from Kenneth D. Merry on Thu, Nov 11, 1999 at 06:16:16PM -0700 References: <382B52F9.2C6D1E00@simon-shapiro.org> <199911120116.SAA30871@panzer.kdm.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Friday, 12 November 1999 at 0:17:56 -0500, Simon Shapiro wrote: > "Kenneth D. Merry" wrote: >> Simon Shapiro wrote... >>> "Kenneth D. Merry" wrote: >>>> Simon Shapiro wrote... >>>>> "Kenneth D. Merry" wrote: >>>>>> How can you get speeds like that with just a 32-bit PCI bus? The specs for >>>>>> the PowerEdge 1300 say it has 5 32-bit PCI slots: >>>>> >>>>> These numbers are for block devices. The kernel obviously >>>>> caches some of this. I should look next time at emory usage; >>>>> The machine has 1GB of memory. The dataset is about 15GB per >>>>> array. >>>> >>>> Is that for random or sequential I/O? With sequential I/O, you would >>>> probably blow away any caching effects. With random I/O, though, you might >>>> get significant help from the cache, especially with that much RAM. >>> >>> Random, of course. >> >> Okay, that fits the results. I missed the beginning of this discussion, but I'd be interested in seeing how you measured it. I'd be interested to see what's going on here. I've seen some very weird corruption in Vinum (though not on the stack; instead, I get specific corruption of buffer headers, which is asynchronous to anything that Vinum is doing: in other words, it it not obviously related to the execution of any particular part of Vinum). Having said that, you shouldn't be running against the block device. We know that there are some problems there, and they're not going to be fixed. For example, it's relatively trivial to panic the machine doing a newfs on a block device. It's easier with Vinum, but it sometimes works with normal disk block devices as well. Since user access to block devices is going away, this will not be fixed. On Thursday, 11 November 1999 at 22:30:22 -0700, Kenneth D. Merry wrote: > Simon Shapiro wrote... >> "Kenneth D. Merry" wrote: >>> It could be that the combination of the DPT controller's 256MB cache and >>> fancy queueing, and your 1GB of RAM is causing the amazingly fast disk speeds. >> >> These DPTs seem to be optimal for RAID-5, very good at RAID-0 >> and nothing exciting for single disks. I have some FC-AL >> gear on order. >> >> What worries me is not the perfromance, but the corruption >> of the stack that I see. I'd be interested to see what's going on here. I've seen some very weird corruption in Vinum (though not on the stack; instead, I get specific corruption of buffer headers, which is asynchronous to anything that Vinum is doing: in other words, it it not obviously related to the execution of any particular part of Vinum). Having said that, you shouldn't be running against the block device. We know that there are some problems there, and they're not going to be fixed. For example, it's relatively trivial to panic the machine doing a newfs on a block device. It's easier with Vinum, but it sometimes works with normal disk block devices as well. Since user access to block devices is going away, this will not be fixed. >> For example, I can run the same 400 processes against the >> raw device all day and all night without a hitch. >> Run them against a block device and something bizzare >> happens; A filesystem get corrupted, the Adaptec driver >> times out, tsleep segfaults, something. At times I can >> get the error in the driver, but then it makes no sense >> either. There are tons of self-checks and state >> verifications in the code. None trip, or when they do >> they are as illogical as the null pointer inside tsleep. > > Well, since you've done a lot of work to try to isolate the problem in your > code, but haven't tracked it down, I'd suggest taking your code out of the > picture as a variable. > > Create a CCD or Vinum array, using the same disks on Adaptec controllers. > Run the same tests, against the raw and block devices, and see if you get > the same sort of weird behavior. > > If you do, you have solid proof that it's not your code, since your code > wasn't in the kernel. If you don't, unfortunately, you don't have solid > proof either way. (Since in that case, it could be some set of > circumstances that your driver tickles that CCD or Vinum don't.) Recall also my problems. You might end up falling over in a different way, in which case you don't know whther > One other thing to make sure of is that you're running a -stable with > Justin's Adaptec driver bug fix from September 20th. It fixed some cases > where corruption could happen with Ultra 2 Adaptec controllers. FWIW, the problems I have seen have been on other host adapters. Greg -- Finger grog@lemis.com for PGP public key See complete headers for address and phone numbers To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?19991113211846.31442>