From owner-freebsd-hackers Tue Mar 5 15:26:51 2002 Delivered-To: freebsd-hackers@freebsd.org Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) by hub.freebsd.org (Postfix) with ESMTP id 4325737B402 for ; Tue, 5 Mar 2002 15:26:39 -0800 (PST) Received: from isi.edu (m7qbo0tn7en1czh1@hbo.isi.edu [128.9.160.75]) by boreas.isi.edu (8.11.6/8.11.2) with ESMTP id g25NQa026901; Tue, 5 Mar 2002 15:26:36 -0800 (PST) Message-ID: <3C85542B.5060100@isi.edu> Date: Tue, 05 Mar 2002 15:26:35 -0800 From: Lars Eggert User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:0.9.8) Gecko/20020224 X-Accept-Language: en-us, de-de MIME-Version: 1.0 To: Zhihui Zhang Cc: "Rogier R. Mulhuijzen" , Julian Elischer , freebsd-hackers@FreeBSD.ORG Subject: Re: A weird disk behaviour References: Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg=sha1; boundary="------------ms060700090007060508010205" Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG This is a cryptographically signed message in MIME format. --------------ms060700090007060508010205 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Zhihui Zhang wrote: > Several times slower! The point is that writing less data performs > worse. So I call it weird. Huh? You originally said: > (1) Write each block fully and sequentially, ie. 8192 bytes. > > (2) I still write these blocks sequentially, but for each block I only > write part of it. ... > I find out the the performance of (2) is several times better than the > performance of (1). Can anyone explain to me why this is the case? If (2) is better than (1), then writing *less* data is faster. Which is it, now? Lars > -Zhihui > > On Tue, 5 Mar 2002, Lars Eggert wrote: > > >>Zhihui Zhang wrote: >> >>>Well, the core of my program is as follows (RANDOM(x) return a value >>>between 0 and x): >>> >>> blocksize = 8192; >>> write_size_low = 512; >>> >>> time(&time1); >>> for (i = 0; i < write_count; i++) { >>> write_size = write_size_low + >>> RANDOM(write_size_high-write_size_low); >>> write_size = roundup(write_size, DEV_BSIZE); >>> if (testcase == 1) >>> write_size = blocksize; >>> write_block(rawfd, sectorno, buf, write_size); >>> sectorno += blocksize / DEV_BSIZE; >>> } >>> time(&time2); >>> >>>If testcase is one, then the time elapsed (time2 - time1) is much less. >>> >>How "much less" in milliseconds? >> >>Also, in your original mail, you said you had 15,000 of these 8K blocks, >>which is only 120MB or so. Use 150,000 or 1,500,000 and check your >>results then. >> >>Lars >> >> >> >> >>>-Zhihui >>> >>>On Tue, 5 Mar 2002, Lars Eggert wrote: >>> >>> >>> >>>>I agree that it's probably caching at some level. You're only writing >>>>about 120MB of data (and half that in your second case). Bump these to a >>>>couple of GB and see what happens. >>>> >>>>Also, could you post your actual measurements? >>>> >>>>Lars >>>> >>>> >>>>Zhihui Zhang wrote: >>>> >>>> >>>>>The machine has 128M memory. I am doing physical I/O one block at a time, >>>>>so there should be no memory copy. >>>>> >>>>>-Zhihui >>>>> >>>>>On Tue, 5 Mar 2002, Rogier R. Mulhuijzen wrote: >>>>> >>>>> >>>>> >>>>> >>>>>>At 16:03 5-3-2002 -0500, Zhihui Zhang wrote: >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>>On Tue, 5 Mar 2002, Julian Elischer wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>more writes fit in the disk's write cache? >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>For (1), it writes 15000 * 8192 bytes in all. For (2), it writes 15000 * >>>>>>>4096 bytes in all (assuming the random number distributes evenly between 0 >>>>>>>and 8192). So your suggestion does not make sense to me. >>>>>>> >>>>>>> >>>>>>> >>>>>>How large is your buffercache? it might be that the 15000 * ~4096 roughly >>>>>>matches with your cache, and 15000 * 8912 doesn't. >>>>>> >>>>>>Case (1) would require a lot more physical IO in that case than case (2) >>>>>>would require. >>>>>> >>>>>> Doc >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>>-Zhihui >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>On Tue, 5 Mar 2002, Zhihui Zhang wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>>I am doing some raw I/O test on a seagate SCSI disk running FreeBSD 4.5. >>>>>>>>>This situation is like this: >>>>>>>>> >>>>>>>>>+-----+----+----+----+----+----+----+----+----+----+---+------ >>>>>>>>>| | | | | | | | | | | | .... >>>>>>>>>+-----+----+----+----+----+----+----+----+----+----+---+------ >>>>>>>>> >>>>>>>>>Each block is of fixed size, say 8192 bytes. Now I have a user program >>>>>>>>>writing each contiguously laid out block sequentially using /dev/daxxx >>>>>>>>>interface. There are a lot of them, say 15000. I write the blocks in two >>>>>>>>>ways (the data used in writing are garbage): >>>>>>>>> >>>>>>>>>(1) Write each block fully and sequentially, ie. 8192 bytes. >>>>>>>>> >>>>>>>>>(2) I still write these blocks sequentially, but for each block I only >>>>>>>>>write part of it. Exactly how many bytes are written inside each >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>block is >>>>>>> >>>>>>> >>>>>>> >>>>>>>>>determinted by a random number between 512 .. 8192 bytes (rounded up a >>>>>>>>>to multiple of 512 bytes). >>>>>>>>> >>>>>>>>>I find out the the performance of (2) is several times better than the >>>>>>>>>performance of (1). Can anyone explain to me why this is the case? >>>>>>>>> >>>>>>>>>Thanks for any suggestions or hints. >>>>>>>>> >>>>>>>>>-Zhihui >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>To Unsubscribe: send mail to majordomo@FreeBSD.org >>>>>>>>>with "unsubscribe freebsd-hackers" in the body of the message >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>To Unsubscribe: send mail to majordomo@FreeBSD.org >>>>>>>with "unsubscribe freebsd-hackers" in the body of the message >>>>>>> >>>>>>> >>>>>>> >>>>>To Unsubscribe: send mail to majordomo@FreeBSD.org >>>>>with "unsubscribe freebsd-hackers" in the body of the message >>>>> >>>>> >>>>> >>>> >>>>-- >>>>Lars Eggert Information Sciences Institute >>>>http://www.isi.edu/larse/ University of Southern California >>>> >>>> >>>> >> >> >>-- >>Lars Eggert Information Sciences Institute >>http://www.isi.edu/larse/ University of Southern California >> >> -- Lars Eggert Information Sciences Institute http://www.isi.edu/larse/ University of Southern California --------------ms060700090007060508010205 Content-Type: application/x-pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIInzCC ArUwggIeoAMCAQICAwWBRzANBgkqhkiG9w0BAQIFADCBkjELMAkGA1UEBhMCWkExFTATBgNV BAgTDFdlc3Rlcm4gQ2FwZTESMBAGA1UEBxMJQ2FwZSBUb3duMQ8wDQYDVQQKEwZUaGF3dGUx HTAbBgNVBAsTFENlcnRpZmljYXRlIFNlcnZpY2VzMSgwJgYDVQQDEx9QZXJzb25hbCBGcmVl bWFpbCBSU0EgMjAwMC44LjMwMB4XDTAxMDgyNDE2NDAwMFoXDTAyMDgyNDE2NDAwMFowVDEP MA0GA1UEBBMGRWdnZXJ0MQ0wCwYDVQQqEwRMYXJzMRQwEgYDVQQDEwtMYXJzIEVnZ2VydDEc MBoGCSqGSIb3DQEJARYNbGFyc2VAaXNpLmVkdTCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkC gYEA0AvLBsD78nxcUHeHkaMgl3b4qYPnfgbf8Lh+HQP8RgGMRG/Yb+vTpkGezlwt9pkJxiD1 1uZDy4CNNJUu3gKxKSb+zRV70O+lkwwftuHoLHoH4xwo3LcQ2LGDpd+I95tUN4dfJ3TmeEcU SF50dC/SuUI4w8AlhXQ8IxrhgdayTpECAwEAAaNWMFQwKgYFK2UBBAEEITAfAgEAMBowGAIB BAQTTDJ1TXlmZkJOVWJOSkpjZFoyczAYBgNVHREEETAPgQ1sYXJzZUBpc2kuZWR1MAwGA1Ud EwEB/wQCMAAwDQYJKoZIhvcNAQECBQADgYEAheZhn0pQA8zI7U2K1ZIAl11j0a1DKxnp3GtT vOUrGRB3WvYxidvdZ1kizhEsWeXU81TkNDH0DaRqtOEeu6Q2OhB+jeKEqY7IDAJE4/fI0e+d 6PnG1hd+vEvYmsKHkmzBhPc94XUOKNWO+qVNP2NGyNI3QIDy5wX4fdcOo1S34r4wggK1MIIC HqADAgECAgMFgUcwDQYJKoZIhvcNAQECBQAwgZIxCzAJBgNVBAYTAlpBMRUwEwYDVQQIEwxX ZXN0ZXJuIENhcGUxEjAQBgNVBAcTCUNhcGUgVG93bjEPMA0GA1UEChMGVGhhd3RlMR0wGwYD VQQLExRDZXJ0aWZpY2F0ZSBTZXJ2aWNlczEoMCYGA1UEAxMfUGVyc29uYWwgRnJlZW1haWwg UlNBIDIwMDAuOC4zMDAeFw0wMTA4MjQxNjQwMDBaFw0wMjA4MjQxNjQwMDBaMFQxDzANBgNV BAQTBkVnZ2VydDENMAsGA1UEKhMETGFyczEUMBIGA1UEAxMLTGFycyBFZ2dlcnQxHDAaBgkq hkiG9w0BCQEWDWxhcnNlQGlzaS5lZHUwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBANAL ywbA+/J8XFB3h5GjIJd2+KmD534G3/C4fh0D/EYBjERv2G/r06ZBns5cLfaZCcYg9dbmQ8uA jTSVLt4CsSkm/s0Ve9DvpZMMH7bh6Cx6B+McKNy3ENixg6XfiPebVDeHXyd05nhHFEhedHQv 0rlCOMPAJYV0PCMa4YHWsk6RAgMBAAGjVjBUMCoGBStlAQQBBCEwHwIBADAaMBgCAQQEE0wy dU15ZmZCTlViTkpKY2RaMnMwGAYDVR0RBBEwD4ENbGFyc2VAaXNpLmVkdTAMBgNVHRMBAf8E AjAAMA0GCSqGSIb3DQEBAgUAA4GBAIXmYZ9KUAPMyO1NitWSAJddY9GtQysZ6dxrU7zlKxkQ d1r2MYnb3WdZIs4RLFnl1PNU5DQx9A2karThHrukNjoQfo3ihKmOyAwCROP3yNHvnej5xtYX frxL2JrCh5JswYT3PeF1DijVjvqlTT9jRsjSN0CA8ucF+H3XDqNUt+K+MIIDKTCCApKgAwIB AgIBDDANBgkqhkiG9w0BAQQFADCB0TELMAkGA1UEBhMCWkExFTATBgNVBAgTDFdlc3Rlcm4g Q2FwZTESMBAGA1UEBxMJQ2FwZSBUb3duMRowGAYDVQQKExFUaGF3dGUgQ29uc3VsdGluZzEo MCYGA1UECxMfQ2VydGlmaWNhdGlvbiBTZXJ2aWNlcyBEaXZpc2lvbjEkMCIGA1UEAxMbVGhh d3RlIFBlcnNvbmFsIEZyZWVtYWlsIENBMSswKQYJKoZIhvcNAQkBFhxwZXJzb25hbC1mcmVl bWFpbEB0aGF3dGUuY29tMB4XDTAwMDgzMDAwMDAwMFoXDTAyMDgyOTIzNTk1OVowgZIxCzAJ BgNVBAYTAlpBMRUwEwYDVQQIEwxXZXN0ZXJuIENhcGUxEjAQBgNVBAcTCUNhcGUgVG93bjEP MA0GA1UEChMGVGhhd3RlMR0wGwYDVQQLExRDZXJ0aWZpY2F0ZSBTZXJ2aWNlczEoMCYGA1UE AxMfUGVyc29uYWwgRnJlZW1haWwgUlNBIDIwMDAuOC4zMDCBnzANBgkqhkiG9w0BAQEFAAOB jQAwgYkCgYEA3jMypmPHCSVFPtJueCdngcXaiBmClw7jRCmKYzUqbXA8+tyu9+50bzC8M5B/ +TRxoKNtmPHDT6Jl2w36S/HW3WGl+YXNVZo1Gp2Sdagnrthy+boC9tewkd4c6avgGAOofENC UFGHgzzwObSbVIoTh/+zm51JZgAtCYnslGvpoWkCAwEAAaNOMEwwKQYDVR0RBCIwIKQeMBwx GjAYBgNVBAMTEVByaXZhdGVMYWJlbDEtMjk3MBIGA1UdEwEB/wQIMAYBAf8CAQAwCwYDVR0P BAQDAgEGMA0GCSqGSIb3DQEBBAUAA4GBAHMbbyZli/8VNEtZYortRL5Jx+gNu4+5DWomKmKE H7iHY3QcbbfPGlORS+HN5jjZ7VD0Omw0kqzmkpxuwSMBwgmn70uuct0GZ/VQby5YuLYLwVBX tewc1+8XttWIm7eiiBrtOVs5fTT8tpYYJU1q9J3Fw5EvqZa4BTxS/N3pYgNIMYICpjCCAqIC AQEwgZowgZIxCzAJBgNVBAYTAlpBMRUwEwYDVQQIEwxXZXN0ZXJuIENhcGUxEjAQBgNVBAcT CUNhcGUgVG93bjEPMA0GA1UEChMGVGhhd3RlMR0wGwYDVQQLExRDZXJ0aWZpY2F0ZSBTZXJ2 aWNlczEoMCYGA1UEAxMfUGVyc29uYWwgRnJlZW1haWwgUlNBIDIwMDAuOC4zMAIDBYFHMAkG BSsOAwIaBQCgggFhMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8X DTAyMDMwNTIzMjYzNVowIwYJKoZIhvcNAQkEMRYEFMWZOpDuO7PQ+bNpkucu/8Qm7b1MMFIG CSqGSIb3DQEJDzFFMEMwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3DQMC AgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGtBgsqhkiG9w0BCRACCzGBnaCBmjCBkjEL MAkGA1UEBhMCWkExFTATBgNVBAgTDFdlc3Rlcm4gQ2FwZTESMBAGA1UEBxMJQ2FwZSBUb3du MQ8wDQYDVQQKEwZUaGF3dGUxHTAbBgNVBAsTFENlcnRpZmljYXRlIFNlcnZpY2VzMSgwJgYD VQQDEx9QZXJzb25hbCBGcmVlbWFpbCBSU0EgMjAwMC44LjMwAgMFgUcwDQYJKoZIhvcNAQEB BQAEgYDDwsUBbaGtoiPzexH/wwEOFloeHsS0YSU9VudcX1gBRzkiN4OFSlNyWsbAPjnnKbzI U85nznR2Cf2jVGXmMAy5dZCusDoA80iGuYFMQMkV6/8e/2F5UiKdkdRcRiNydf/0nY+wJhGd pb2l5TqKIZTZ+A5nAEWcIMS0T0vnkdsxiAAAAAAAAA== --------------ms060700090007060508010205-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message