Date: Mon, 5 Oct 2009 14:04:18 +0200 From: Luigi Rizzo <rizzo@iet.unipi.it> To: rihad <rihad@mail.ru> Cc: freebsd-net@freebsd.org Subject: Re: dummynet dropping too many packets Message-ID: <20091005120418.GA63131@onelab2.iet.unipi.it> In-Reply-To: <4AC9D87E.7000005@mail.ru> References: <4AC8A76B.3050502@mail.ru> <20091005025521.GA52702@svzserv.kemerovo.su> <20091005061025.GB55845@onelab2.iet.unipi.it> <4AC9B400.9020400@mail.ru> <20091005090102.GA70430@svzserv.kemerovo.su> <4AC9BC5A.50902@mail.ru> <20091005095600.GA73335@svzserv.kemerovo.su> <4AC9CFF7.3090208@mail.ru> <20091005110726.GA62598@onelab2.iet.unipi.it> <4AC9D87E.7000005@mail.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Oct 05, 2009 at 04:29:02PM +0500, rihad wrote: > Luigi Rizzo wrote: ... > >you keep omitting the important info i.e. whether individual > >pipes have drops, significant queue lenghts and so on. > > > Sorry. Almost everyone has 0 in the last Drp column, but some have above > zero. I'm not just sure how this can be helpful to anyone. because you were complaining about 'dummynet causing drops and waste of bandwidth'. Now, drops could be due to either 1) some saturation in the dummynet machine (memory shortage, cpu shortage, etc.) which cause unwanted drops; 2) intentional drops introduced by dummynet because a flow exceeds its queue size. These drops are those shown in the 'Drop' column in 'ipfw pipe show' (they are cumulative, so you should do an 'ipfw pipe delete; ipfw pipe 5120 config ...' whenever you want to re-run the stats, or compute the differences between subsequent reads, to figure out what happens. If all drops you are seeing are of type 2, then there is nothing you can do to remove them: you set a bandwidth limit, the client is sending faster than it should, perhaps with UDP so even RED/GRED won't help you, and you see the drops once the queue starts to fill up. Examples below: the entries in bucket 4 and 44 If you are seeing drops that are not listed in 'pipe show' then yun need to investigate where the packets are lost, again it could be on the output queue of the interface (due to the burstiness introduced by dummynet), or shortage of mbufs (but this did not seem to be the case from your previous stats) or something else. It's all up to you to run measurements, possibly without omitting potentially significant data (e.g. sysctl -a net.inet.ip) or making assumptions (e.g. you have configured 5000 slots per queue, but with only 50k mbufs in total there is no chance to guarantee 5000 slots to each queue -- all you will achieve is give a lot of slots to the greedy nodes, and very little to the other ones) cheers luigi > 05120: 5.120 Mbit/s 0 ms 5000 sl. 66 queues (64 buckets) > GRED w_q 0.001999 min_th 4500 max_th 5000 max_p 0.099991 > mask: 0x00 0x00000000/0x0000 -> 0xffffffff/0x0000 > BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes > Pkt/Byte Drp > 0 ip 0.0.0.0/0 <client_ip> 1 131 0 0 0 > 1 ip 0.0.0.0/0 <client_ip> 39 53360 0 0 0 > 2 ip 0.0.0.0/0 <client_ip> 382206 418022848 0 > 0 0 > 3 ip 0.0.0.0/0 <client_ip> 34 2008 0 0 0 > 4 ip 0.0.0.0/0 <client_ip> 4868510 6277077787 15 > 20452 9 > 5 ip 0.0.0.0/0 <client_ip> 14 16675 0 0 0 > 5 ip 0.0.0.0/0 <client_ip> 3 4158 0 0 0 > 6 ip 0.0.0.0/0 <client_ip> 38 43576 0 0 0 > 7 ip 0.0.0.0/0 <client_ip> 1265954 1475400663 0 > 0 0 > 8 ip 0.0.0.0/0 <client_ip> 1081461 1247681879 0 > 0 749 > 9 ip 0.0.0.0/0 <client_ip> 6186589 8737048919 0 > 0 19243 > 10 ip 0.0.0.0/0 <client_ip> 21607 5636447 0 0 5 > 11 ip 0.0.0.0/0 <client_ip> 437 94576 0 0 0 > 12 ip 0.0.0.0/0 <client_ip> 22915 18634779 0 0 0 > 13 ip 0.0.0.0/0 <client_ip> 557988 688051579 0 > 0 0 > 14 ip 0.0.0.0/0 <client_ip> 50339 65685647 0 0 0 > 15 ip 0.0.0.0/0 <client_ip> 554835 546223485 0 > 0 140 > 16 ip 0.0.0.0/0 <client_ip> 32 13104 0 0 0 > 17 ip 0.0.0.0/0 <client_ip> 2034099 2719966792 0 > 0 0 > 18 ip 0.0.0.0/0 <client_ip> 282 36551 0 0 0 > 19 ip 0.0.0.0/0 <client_ip> 8351766 8947643162 0 > 0 0 > 20 ip 0.0.0.0/0 <client_ip> 4 624 0 0 0 > 21 ip 0.0.0.0/0 <client_ip> 22391 29922375 0 0 0 > 22 ip 0.0.0.0/0 <client_ip> 9 424 0 0 0 > 23 ip 0.0.0.0/0 <client_ip> 750322 935365326 0 > 0 0 > 24 ip 0.0.0.0/0 <client_ip> 1 40 0 0 0 > 25 ip 0.0.0.0/0 <client_ip> 3617690 3501375619 0 > 0 602 > 26 ip 0.0.0.0/0 <client_ip> 12116 12039435 0 0 0 > 27 ip 0.0.0.0/0 <client_ip> 524311 653399507 0 > 0 8 > 28 ip 0.0.0.0/0 <client_ip> 3 417 0 0 0 > 29 ip 0.0.0.0/0 <client_ip> 16 2034 0 0 0 > 30 ip 0.0.0.0/0 <client_ip> 64 82661 3 4432 0 > 31 ip 0.0.0.0/0 <client_ip> 946389 1175221367 0 > 0 66 > 32 ip 0.0.0.0/0 <client_ip> 1 168 0 0 0 > 32 ip 0.0.0.0/0 <client_ip> 28 41776 0 0 0 > 33 ip 0.0.0.0/0 <client_ip> 6 6433 0 0 0 > 34 ip 0.0.0.0/0 <client_ip> 1 536 0 0 0 > 35 ip 0.0.0.0/0 <client_ip> 2021 2641048 0 0 0 > 36 ip 0.0.0.0/0 <client_ip> 350 264039 0 0 0 > 37 ip 0.0.0.0/0 <client_ip> 167578 137763107 0 > 0 0 > 38 ip 0.0.0.0/0 <client_ip> 250404 128905757 0 > 0 0 > 39 ip 0.0.0.0/0 <client_ip> 385139 287006012 0 > 0 0 > 40 ip 0.0.0.0/0 <client_ip> 49 68696 0 0 0 > 41 ip 0.0.0.0/0 <client_ip> 23 1813 0 0 0 > 42 ip 0.0.0.0/0 <client_ip> 129 135256 0 0 0 > 43 ip 0.0.0.0/0 <client_ip> 3232 2191027 0 0 0 > 44 ip 0.0.0.0/0 <client_ip> 27935157 24307287646 0 > 0 18802 > 45 ip 0.0.0.0/0 <client_ip> 2166 212635 0 0 0 > 46 ip 0.0.0.0/0 <client_ip> 1127307 1392467620 0 > 0 3 > 47 ip 0.0.0.0/0 <client_ip> 1216900 1258200836 0 > 0 0 > 48 ip 0.0.0.0/0 <client_ip> 2 2984 1 1492 0 > 49 ip 0.0.0.0/0 <client_ip> 1 112 0 0 0 > 50 ip 0.0.0.0/0 <client_ip> 1409 326389 0 0 0 > 51 ip 0.0.0.0/0 <client_ip> 46674 47291021 10 14920 0 > 52 ip 0.0.0.0/0 <client_ip> 86667 66834983 0 0 0 > 53 ip 0.0.0.0/0 <client_ip> 434998 302827189 0 > 0 0 > 54 ip 0.0.0.0/0 <client_ip> 542 277669 0 0 0 > 55 ip 0.0.0.0/0 <client_ip> 1088072 919495021 0 > 0 0 > 56 ip 0.0.0.0/0 <client_ip> 64 81240 0 0 0 > 57 ip 0.0.0.0/0 <client_ip> 41028 59193278 0 0 0 > 58 ip 0.0.0.0/0 <client_ip> 1 210 0 0 0 > 59 ip 0.0.0.0/0 <client_ip> 4 310 0 0 0 > 60 ip 0.0.0.0/0 <client_ip> 2 2984 0 0 0 > 61 ip 0.0.0.0/0 <client_ip> 42874 36616688 0 0 0 > 62 ip 0.0.0.0/0 <client_ip> 4 498 0 0 0 > 63 ip 0.0.0.0/0 <client_ip> 530137 717027403 0 > 0 0
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20091005120418.GA63131>