Date: Tue, 05 Mar 2002 15:26:35 -0800 From: Lars Eggert <larse@ISI.EDU> To: Zhihui Zhang <zzhang@cs.binghamton.edu> Cc: "Rogier R. Mulhuijzen" <drwilco@drwilco.net>, Julian Elischer <julian@elischer.org>, freebsd-hackers@FreeBSD.ORG Subject: Re: A weird disk behaviour Message-ID: <3C85542B.5060100@isi.edu> References: <Pine.SOL.4.21.0203051818510.13181-100000@onyx>
next in thread | previous in thread | raw e-mail | index | archive | help
This is a cryptographically signed message in MIME format.
--------------ms060700090007060508010205
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Zhihui Zhang wrote:
> Several times slower! The point is that writing less data performs
> worse. So I call it weird.
Huh? You originally said:
> (1) Write each block fully and sequentially, ie. 8192 bytes.
>
> (2) I still write these blocks sequentially, but for each block I only
> write part of it.
...
> I find out the the performance of (2) is several times better than the
> performance of (1). Can anyone explain to me why this is the case?
If (2) is better than (1), then writing *less* data is faster. Which is
it, now?
Lars
> -Zhihui
>
> On Tue, 5 Mar 2002, Lars Eggert wrote:
>
>
>>Zhihui Zhang wrote:
>>
>>>Well, the core of my program is as follows (RANDOM(x) return a value
>>>between 0 and x):
>>>
>>> blocksize = 8192;
>>> write_size_low = 512;
>>>
>>> time(&time1);
>>> for (i = 0; i < write_count; i++) {
>>> write_size = write_size_low +
>>> RANDOM(write_size_high-write_size_low);
>>> write_size = roundup(write_size, DEV_BSIZE);
>>> if (testcase == 1)
>>> write_size = blocksize;
>>> write_block(rawfd, sectorno, buf, write_size);
>>> sectorno += blocksize / DEV_BSIZE;
>>> }
>>> time(&time2);
>>>
>>>If testcase is one, then the time elapsed (time2 - time1) is much less.
>>>
>>How "much less" in milliseconds?
>>
>>Also, in your original mail, you said you had 15,000 of these 8K blocks,
>>which is only 120MB or so. Use 150,000 or 1,500,000 and check your
>>results then.
>>
>>Lars
>>
>>
>>
>>
>>>-Zhihui
>>>
>>>On Tue, 5 Mar 2002, Lars Eggert wrote:
>>>
>>>
>>>
>>>>I agree that it's probably caching at some level. You're only writing
>>>>about 120MB of data (and half that in your second case). Bump these to a
>>>>couple of GB and see what happens.
>>>>
>>>>Also, could you post your actual measurements?
>>>>
>>>>Lars
>>>>
>>>>
>>>>Zhihui Zhang wrote:
>>>>
>>>>
>>>>>The machine has 128M memory. I am doing physical I/O one block at a time,
>>>>>so there should be no memory copy.
>>>>>
>>>>>-Zhihui
>>>>>
>>>>>On Tue, 5 Mar 2002, Rogier R. Mulhuijzen wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>At 16:03 5-3-2002 -0500, Zhihui Zhang wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>On Tue, 5 Mar 2002, Julian Elischer wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>more writes fit in the disk's write cache?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>For (1), it writes 15000 * 8192 bytes in all. For (2), it writes 15000 *
>>>>>>>4096 bytes in all (assuming the random number distributes evenly between 0
>>>>>>>and 8192). So your suggestion does not make sense to me.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>How large is your buffercache? it might be that the 15000 * ~4096 roughly
>>>>>>matches with your cache, and 15000 * 8912 doesn't.
>>>>>>
>>>>>>Case (1) would require a lot more physical IO in that case than case (2)
>>>>>>would require.
>>>>>>
>>>>>> Doc
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>-Zhihui
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>On Tue, 5 Mar 2002, Zhihui Zhang wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>I am doing some raw I/O test on a seagate SCSI disk running FreeBSD 4.5.
>>>>>>>>>This situation is like this:
>>>>>>>>>
>>>>>>>>>+-----+----+----+----+----+----+----+----+----+----+---+------
>>>>>>>>>| | | | | | | | | | | | ....
>>>>>>>>>+-----+----+----+----+----+----+----+----+----+----+---+------
>>>>>>>>>
>>>>>>>>>Each block is of fixed size, say 8192 bytes. Now I have a user program
>>>>>>>>>writing each contiguously laid out block sequentially using /dev/daxxx
>>>>>>>>>interface. There are a lot of them, say 15000. I write the blocks in two
>>>>>>>>>ways (the data used in writing are garbage):
>>>>>>>>>
>>>>>>>>>(1) Write each block fully and sequentially, ie. 8192 bytes.
>>>>>>>>>
>>>>>>>>>(2) I still write these blocks sequentially, but for each block I only
>>>>>>>>>write part of it. Exactly how many bytes are written inside each
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>block is
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>>determinted by a random number between 512 .. 8192 bytes (rounded up a
>>>>>>>>>to multiple of 512 bytes).
>>>>>>>>>
>>>>>>>>>I find out the the performance of (2) is several times better than the
>>>>>>>>>performance of (1). Can anyone explain to me why this is the case?
>>>>>>>>>
>>>>>>>>>Thanks for any suggestions or hints.
>>>>>>>>>
>>>>>>>>>-Zhihui
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>To Unsubscribe: send mail to majordomo@FreeBSD.org
>>>>>>>>>with "unsubscribe freebsd-hackers" in the body of the message
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>To Unsubscribe: send mail to majordomo@FreeBSD.org
>>>>>>>with "unsubscribe freebsd-hackers" in the body of the message
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>To Unsubscribe: send mail to majordomo@FreeBSD.org
>>>>>with "unsubscribe freebsd-hackers" in the body of the message
>>>>>
>>>>>
>>>>>
>>>>
>>>>--
>>>>Lars Eggert <larse@isi.edu> Information Sciences Institute
>>>>http://www.isi.edu/larse/ University of Southern California
>>>>
>>>>
>>>>
>>
>>
>>--
>>Lars Eggert <larse@isi.edu> Information Sciences Institute
>>http://www.isi.edu/larse/ University of Southern California
>>
>>
--
Lars Eggert <larse@isi.edu> Information Sciences Institute
http://www.isi.edu/larse/ University of Southern California
--------------ms060700090007060508010205
Content-Type: application/x-pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"
Content-Description: S/MIME Cryptographic Signature
MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIInzCC
ArUwggIeoAMCAQICAwWBRzANBgkqhkiG9w0BAQIFADCBkjELMAkGA1UEBhMCWkExFTATBgNV
BAgTDFdlc3Rlcm4gQ2FwZTESMBAGA1UEBxMJQ2FwZSBUb3duMQ8wDQYDVQQKEwZUaGF3dGUx
HTAbBgNVBAsTFENlcnRpZmljYXRlIFNlcnZpY2VzMSgwJgYDVQQDEx9QZXJzb25hbCBGcmVl
bWFpbCBSU0EgMjAwMC44LjMwMB4XDTAxMDgyNDE2NDAwMFoXDTAyMDgyNDE2NDAwMFowVDEP
MA0GA1UEBBMGRWdnZXJ0MQ0wCwYDVQQqEwRMYXJzMRQwEgYDVQQDEwtMYXJzIEVnZ2VydDEc
MBoGCSqGSIb3DQEJARYNbGFyc2VAaXNpLmVkdTCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkC
gYEA0AvLBsD78nxcUHeHkaMgl3b4qYPnfgbf8Lh+HQP8RgGMRG/Yb+vTpkGezlwt9pkJxiD1
1uZDy4CNNJUu3gKxKSb+zRV70O+lkwwftuHoLHoH4xwo3LcQ2LGDpd+I95tUN4dfJ3TmeEcU
SF50dC/SuUI4w8AlhXQ8IxrhgdayTpECAwEAAaNWMFQwKgYFK2UBBAEEITAfAgEAMBowGAIB
BAQTTDJ1TXlmZkJOVWJOSkpjZFoyczAYBgNVHREEETAPgQ1sYXJzZUBpc2kuZWR1MAwGA1Ud
EwEB/wQCMAAwDQYJKoZIhvcNAQECBQADgYEAheZhn0pQA8zI7U2K1ZIAl11j0a1DKxnp3GtT
vOUrGRB3WvYxidvdZ1kizhEsWeXU81TkNDH0DaRqtOEeu6Q2OhB+jeKEqY7IDAJE4/fI0e+d
6PnG1hd+vEvYmsKHkmzBhPc94XUOKNWO+qVNP2NGyNI3QIDy5wX4fdcOo1S34r4wggK1MIIC
HqADAgECAgMFgUcwDQYJKoZIhvcNAQECBQAwgZIxCzAJBgNVBAYTAlpBMRUwEwYDVQQIEwxX
ZXN0ZXJuIENhcGUxEjAQBgNVBAcTCUNhcGUgVG93bjEPMA0GA1UEChMGVGhhd3RlMR0wGwYD
VQQLExRDZXJ0aWZpY2F0ZSBTZXJ2aWNlczEoMCYGA1UEAxMfUGVyc29uYWwgRnJlZW1haWwg
UlNBIDIwMDAuOC4zMDAeFw0wMTA4MjQxNjQwMDBaFw0wMjA4MjQxNjQwMDBaMFQxDzANBgNV
BAQTBkVnZ2VydDENMAsGA1UEKhMETGFyczEUMBIGA1UEAxMLTGFycyBFZ2dlcnQxHDAaBgkq
hkiG9w0BCQEWDWxhcnNlQGlzaS5lZHUwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBANAL
ywbA+/J8XFB3h5GjIJd2+KmD534G3/C4fh0D/EYBjERv2G/r06ZBns5cLfaZCcYg9dbmQ8uA
jTSVLt4CsSkm/s0Ve9DvpZMMH7bh6Cx6B+McKNy3ENixg6XfiPebVDeHXyd05nhHFEhedHQv
0rlCOMPAJYV0PCMa4YHWsk6RAgMBAAGjVjBUMCoGBStlAQQBBCEwHwIBADAaMBgCAQQEE0wy
dU15ZmZCTlViTkpKY2RaMnMwGAYDVR0RBBEwD4ENbGFyc2VAaXNpLmVkdTAMBgNVHRMBAf8E
AjAAMA0GCSqGSIb3DQEBAgUAA4GBAIXmYZ9KUAPMyO1NitWSAJddY9GtQysZ6dxrU7zlKxkQ
d1r2MYnb3WdZIs4RLFnl1PNU5DQx9A2karThHrukNjoQfo3ihKmOyAwCROP3yNHvnej5xtYX
frxL2JrCh5JswYT3PeF1DijVjvqlTT9jRsjSN0CA8ucF+H3XDqNUt+K+MIIDKTCCApKgAwIB
AgIBDDANBgkqhkiG9w0BAQQFADCB0TELMAkGA1UEBhMCWkExFTATBgNVBAgTDFdlc3Rlcm4g
Q2FwZTESMBAGA1UEBxMJQ2FwZSBUb3duMRowGAYDVQQKExFUaGF3dGUgQ29uc3VsdGluZzEo
MCYGA1UECxMfQ2VydGlmaWNhdGlvbiBTZXJ2aWNlcyBEaXZpc2lvbjEkMCIGA1UEAxMbVGhh
d3RlIFBlcnNvbmFsIEZyZWVtYWlsIENBMSswKQYJKoZIhvcNAQkBFhxwZXJzb25hbC1mcmVl
bWFpbEB0aGF3dGUuY29tMB4XDTAwMDgzMDAwMDAwMFoXDTAyMDgyOTIzNTk1OVowgZIxCzAJ
BgNVBAYTAlpBMRUwEwYDVQQIEwxXZXN0ZXJuIENhcGUxEjAQBgNVBAcTCUNhcGUgVG93bjEP
MA0GA1UEChMGVGhhd3RlMR0wGwYDVQQLExRDZXJ0aWZpY2F0ZSBTZXJ2aWNlczEoMCYGA1UE
AxMfUGVyc29uYWwgRnJlZW1haWwgUlNBIDIwMDAuOC4zMDCBnzANBgkqhkiG9w0BAQEFAAOB
jQAwgYkCgYEA3jMypmPHCSVFPtJueCdngcXaiBmClw7jRCmKYzUqbXA8+tyu9+50bzC8M5B/
+TRxoKNtmPHDT6Jl2w36S/HW3WGl+YXNVZo1Gp2Sdagnrthy+boC9tewkd4c6avgGAOofENC
UFGHgzzwObSbVIoTh/+zm51JZgAtCYnslGvpoWkCAwEAAaNOMEwwKQYDVR0RBCIwIKQeMBwx
GjAYBgNVBAMTEVByaXZhdGVMYWJlbDEtMjk3MBIGA1UdEwEB/wQIMAYBAf8CAQAwCwYDVR0P
BAQDAgEGMA0GCSqGSIb3DQEBBAUAA4GBAHMbbyZli/8VNEtZYortRL5Jx+gNu4+5DWomKmKE
H7iHY3QcbbfPGlORS+HN5jjZ7VD0Omw0kqzmkpxuwSMBwgmn70uuct0GZ/VQby5YuLYLwVBX
tewc1+8XttWIm7eiiBrtOVs5fTT8tpYYJU1q9J3Fw5EvqZa4BTxS/N3pYgNIMYICpjCCAqIC
AQEwgZowgZIxCzAJBgNVBAYTAlpBMRUwEwYDVQQIEwxXZXN0ZXJuIENhcGUxEjAQBgNVBAcT
CUNhcGUgVG93bjEPMA0GA1UEChMGVGhhd3RlMR0wGwYDVQQLExRDZXJ0aWZpY2F0ZSBTZXJ2
aWNlczEoMCYGA1UEAxMfUGVyc29uYWwgRnJlZW1haWwgUlNBIDIwMDAuOC4zMAIDBYFHMAkG
BSsOAwIaBQCgggFhMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8X
DTAyMDMwNTIzMjYzNVowIwYJKoZIhvcNAQkEMRYEFMWZOpDuO7PQ+bNpkucu/8Qm7b1MMFIG
CSqGSIb3DQEJDzFFMEMwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3DQMC
AgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGtBgsqhkiG9w0BCRACCzGBnaCBmjCBkjEL
MAkGA1UEBhMCWkExFTATBgNVBAgTDFdlc3Rlcm4gQ2FwZTESMBAGA1UEBxMJQ2FwZSBUb3du
MQ8wDQYDVQQKEwZUaGF3dGUxHTAbBgNVBAsTFENlcnRpZmljYXRlIFNlcnZpY2VzMSgwJgYD
VQQDEx9QZXJzb25hbCBGcmVlbWFpbCBSU0EgMjAwMC44LjMwAgMFgUcwDQYJKoZIhvcNAQEB
BQAEgYDDwsUBbaGtoiPzexH/wwEOFloeHsS0YSU9VudcX1gBRzkiN4OFSlNyWsbAPjnnKbzI
U85nznR2Cf2jVGXmMAy5dZCusDoA80iGuYFMQMkV6/8e/2F5UiKdkdRcRiNydf/0nY+wJhGd
pb2l5TqKIZTZ+A5nAEWcIMS0T0vnkdsxiAAAAAAAAA==
--------------ms060700090007060508010205--
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3C85542B.5060100>