Date: Sun, 23 Mar 2014 19:57:09 -0400 (EDT) From: Rick Macklem <rmacklem@uoguelph.ca> To: Christopher Forgeron <csforgeron@gmail.com> Cc: FreeBSD Net <freebsd-net@freebsd.org>, Garrett Wollman <wollman@freebsd.org>, Jack Vogel <jfvogel@gmail.com>, Markus Gebert <markus.gebert@hostpoint.ch> Subject: Re: 9.2 ixgbe tx queue hang Message-ID: <1936201708.1670348.1395619029045.JavaMail.root@uoguelph.ca> In-Reply-To: <CAB2_NwB%2BL5T1iTaxZtt00SL22_1H0S9X%2BM_LOGfkoJmy_TJokw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
------=_Part_1670344_208375117.1395619029041 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Christopher Forgeron wrote: > > > > > > > On Sat, Mar 22, 2014 at 6:41 PM, Rick Macklem < rmacklem@uoguelph.ca > > wrote: > > > > Christopher Forgeron wrote: > > #if defined(INET) || defined(INET6) > > /* Initialize to max value. */ > > if (ifp->if_hw_tsomax == 0) > > ifp->if_hw_tsomax = IP_MAXPACKET; > > KASSERT(ifp->if_hw_tsomax <= IP_MAXPACKET && > > ifp->if_hw_tsomax >= IP_MAXPACKET / 8, > > ("%s: tsomax outside of range", __func__)); > > #endif > > > > > > Should this be the location where it's being set rather than in > > ixgbe? I would assume that other drivers could fall prey to this > > issue. > > > All of this should be prepended with "I'm an NFS guy, not a > networking > guy, so I might be wrong". > > Other drivers (and ixgbe for the 82598 chip) can handle a packet that > is in more than 32 mbufs. (I think the 82598 handles 100, grep for > SCATTER > in *.h in sys/dev/ixgbe.) > > > [...] > > > Yes, I agree we have to be careful about the limitations of other > drivers, but I'm thinking setting tso to IP_MAXPACKET is a bad idea, > unless all of the header subtractions are happening elsewhere. Then > again, perhaps every other driver (and possibly ixgbe.. i need to > look more) does a maxtso - various_headers to set a limit for data > packets. > > > I'm not familiar with the Freebsd network conventions/styles - I'm > just asking questions, something I have a bad habit for, but I'm in > charge of code stability issues at my work so it's hard to stop. > Well, IP_MAXPACKET is simply the largest # that fits in the 16bit length field of an IP header (65535). This limit is on the TSO segment (which is really just a TCP/IP packet greater than the MTU) and does not include a MAC level (ethernet) header. Beyond that, it is the specific hardware that limits things, such as this case, which is limited to 32 mbufs (which happens to imply 64K total, including ethernet header using 2K mbuf clusters). (The 64K limit is just a quirk caused by the 32mbuf limit and the fact that mbuf clusters hold 2K of data each.) > > > Now, since several drivers do have this 32 mbufs limit, I can see an > argument > for making the default a little smaller to make these work, since the > driver can override the default. (About now someone usually jumps in > and says > something along the lines of "You can't do that until all the drivers > that > can handle IP_MAXPACKET are fixed to set if_hw_tsomax" and since I > can't fix > drivers I can't test, that pretty much puts a stop on it.) > > > > > Testing is a problem isn't it? I once again offer my stack of network > cards and systems for some sort of testing.. I still have coax and > token ring around. :-) > > > > You see the problem isn't that IP_MAXPACKET is too big, but that the > hardware > has a limit of 32 non-contiguous chunks (mbufs)/packet and 32 * > MCLBYTES = 64K. > (Hardware/network drivers that can handle 35 or more chunks (they > like to call > them transmit segments, although ixgbe uses the term scatter) > shouldn't have > any problems.) > > I have an untested patch that adds a tsomaxseg count to use along > with tsomax > bytes so that a driver could inform tcp_output() it can only handle > 32 mbufs > and then tcp_output() would limit a TSO segment using both, but I > can't test > it, so who knows when/if that might happen. > > > > > I think you give that to me in the next email - if not, please send. > > > > I also have a patch that modifies NFS to use pagesize clusters > (reducing the > mbuf count in the list), but that one causes grief when testing on an > i386 > (seems to run out of kernel memory to the point where it can't > allocate something > called "boundary tags" and pretty well wedges the machine at that > point.) > Since I don't know how to fix this (I thought of making the patch > "amd64 only"), > I can't really commit this to head, either. > > > > > Send me that one too. I love NFS patches. > > > > As such, I think it's going to be "fix the drivers one at a time" and > tell > folks to "disable TSO or limit rsize,wsize to 32K" when they run into > trouble. > (As you might have guessed, I'd rather just be "the NFS guy", but > since NFS > "triggers the problem" I\m kinda stuck with it;-) > > > > I know in some circumstances disabling TSO can be a benefit, but in > general you'd want it on a modern system with heavy data load. > > > > > > Also should we not also subtract ETHER_VLAN_ENCAP_LEN from tsomax > > to > > make sure VLANs fit? > > > No idea. (I wouldn't know a VLAN if it jumped up and tried to > bite me on the nose.;-) So, I have no idea what does this, but > if it means the total ethernet header size can be > 14bytes, then I'd > agree. > > > > Yeah, you need another 4 bytes for VLAN header if you're not using > hardware that strips it before the TCP stack gets it. I have a mix > of hardware and software VLANs running on our backbone, mostly due > to a mixed FreeBSD/OpenBSD/Windows environment. > > > > > > Perhaps there is something in the newer network code that is > > filling > > up the frames to the point where they are full - thus a TSO = > > IP_MAXPACKET is just now causing problems. > > > Yea, I have no idea why this didn't bite running 9.1. (Did 9.1 have > TSO enabled by default?) > > > > I believe 9.0 has TSO on by default.. I seem to recall it always > being there, but I can't easily confirm it now. My last 9.0-STABLE > doesn't have an ixgbe card in it. > > > > Ok, I've attached 3 patches: ixgbe.patch - A slightly updated version of the one that sets if_hw_tsomax, which subtracts out the additional 4bytes for the VLAN header. *** If you can test this, it would be nice to know if this gets rid of all the EFBIG replies, since I think Jack might feel it is ok to commit if it does do so. 4kmcl.patch - This one modifies NFS to use pagesize mbuf clusters for the large RPC messages. It is NOT safe to use on a small i386, but might be ok on a large amd64 box. On a small i386, using a mix of 2K and 4K mbuf clusters seems to fragment kernel memory enough that allocation of "boundary tags" (whatever those are?) fail and this trainwrecks the system. Using pagesize (4K) clusters reduces the mbuf count for an IP_MAXPACKET sized TSO segment to 19, avoiding the 32 limit and any need to call m_defrag() for NFS. *** Only use on a test system, at your own risk. tsomaxseg.patch - This one adds support for if_hw_tsomaxseg, which is a limit on the # of mbufs in an output TSO segment (and defaults to 32). *** This one HAS NOT BEEN TESTED and probably doesn't even work at this point. rick ------=_Part_1670344_208375117.1395619029041 Content-Type: text/x-patch; name=ixgbe.patch Content-Disposition: attachment; filename=ixgbe.patch Content-Transfer-Encoding: base64 LS0tIGRldi9peGdiZS9peGdiZS5jLnNhdgkyMDE0LTAzLTE5IDE3OjQ0OjM0LjAwMDAwMDAwMCAt MDQwMAorKysgZGV2L2l4Z2JlL2l4Z2JlLmMJMjAxNC0wMy0yMiAyMjo0NDo1My4wMDAwMDAwMDAg LTA0MDAKQEAgLTI2MTQsNiArMjYxNCwxMCBAQCBpeGdiZV9zZXR1cF9pbnRlcmZhY2UoZGV2aWNl X3QgZGV2LCBzdHJ1CiAJaWZwLT5pZl9zbmQuaWZxX2Rydl9tYXhsZW4gPSBhZGFwdGVyLT5udW1f dHhfZGVzYyAtIDI7CiAJSUZRX1NFVF9SRUFEWSgmaWZwLT5pZl9zbmQpOwogI2VuZGlmCisJaWYg KChhZGFwdGVyLT5udW1fc2VncyAqIE1DTEJZVEVTIC0gKEVUSEVSX0hEUl9MRU4gKworCSAgICBF VEhFUl9WTEFOX0VOQ0FQX0xFTikpIDwgSVBfTUFYUEFDS0VUKQorCQlpZnAtPmlmX2h3X3Rzb21h eCA9IGFkYXB0ZXItPm51bV9zZWdzICogTUNMQllURVMgLQorCQkgICAgKEVUSEVSX0hEUl9MRU4g KyBFVEhFUl9WTEFOX0VOQ0FQX0xFTik7CiAKIAlldGhlcl9pZmF0dGFjaChpZnAsIGFkYXB0ZXIt Pmh3Lm1hYy5hZGRyKTsKIAo= ------=_Part_1670344_208375117.1395619029041 Content-Type: text/x-patch; name=4kmcl.patch Content-Disposition: attachment; filename=4kmcl.patch Content-Transfer-Encoding: base64 LS0tIGZzL25mc3NlcnZlci9uZnNfbmZzZHBvcnQuYy5zYXYyCTIwMTQtMDEtMjYgMTg6NTQ6Mjku MDAwMDAwMDAwIC0wNTAwCisrKyBmcy9uZnNzZXJ2ZXIvbmZzX25mc2Rwb3J0LmMJMjAxNC0wMy0x NiAyMzoyMjo1Ni4wMDAwMDAwMDAgLTA0MDAKQEAgLTU2Niw4ICs1NjYsNyBAQCBuZnN2bm9fcmVh ZGxpbmsoc3RydWN0IHZub2RlICp2cCwgc3RydWN0CiAJbGVuID0gMDsKIAlpID0gMDsKIAl3aGls ZSAobGVuIDwgTkZTX01BWFBBVEhMRU4pIHsKLQkJTkZTTUdFVChtcCk7Ci0JCU1DTEdFVChtcCwg TV9XQUlUT0spOworCQlORlNNQ0xHRVQobXAsIE1fTk9XQUlUKTsKIAkJbXAtPm1fbGVuID0gTkZT TVNJWihtcCk7CiAJCWlmIChsZW4gPT0gMCkgewogCQkJbXAzID0gbXAyID0gbXA7CkBAIC02MjEs NyArNjIwLDcgQEAgbmZzdm5vX3JlYWQoc3RydWN0IHZub2RlICp2cCwgb2ZmX3Qgb2ZmLAogICAg IHN0cnVjdCB0aHJlYWQgKnAsIHN0cnVjdCBtYnVmICoqbXBwLCBzdHJ1Y3QgbWJ1ZiAqKm1wZW5k cCkKIHsKIAlzdHJ1Y3QgbWJ1ZiAqbTsKLQlpbnQgaTsKKwlpbnQgZG9fcGFnZXNpemUsIGk7CiAJ c3RydWN0IGlvdmVjICppdjsKIAlzdHJ1Y3QgaW92ZWMgKml2MjsKIAlpbnQgZXJyb3IgPSAwLCBs ZW4sIGxlZnQsIHNpeiwgdGxlbiwgaW9mbGFnID0gMDsKQEAgLTYzMCwxNCArNjI5LDMzIEBAIG5m c3Zub19yZWFkKHN0cnVjdCB2bm9kZSAqdnAsIG9mZl90IG9mZiwKIAlzdHJ1Y3QgbmZzaGV1ciAq bmg7CiAKIAlsZW4gPSBsZWZ0ID0gTkZTTV9STkRVUChjbnQpOworCWRvX3BhZ2VzaXplID0gMDsK KyNpZiBNSlVNUEFHRVNJWkUgIT0gTUNMQllURVMKKwlpZiAobGVmdCA+IE1DTEJZVEVTKQorCQlk b19wYWdlc2l6ZSA9IDE7CisjZW5kaWYKIAltMyA9IE5VTEw7CiAJLyoKIAkgKiBHZW5lcmF0ZSB0 aGUgbWJ1ZiBsaXN0IHdpdGggdGhlIHVpb19pb3YgcmVmLiB0byBpdC4KIAkgKi8KIAlpID0gMDsK IAl3aGlsZSAobGVmdCA+IDApIHsKLQkJTkZTTUdFVChtKTsKLQkJTUNMR0VUKG0sIE1fV0FJVE9L KTsKKwkJLyoKKwkJICogRm9yIGxhcmdlIHJlYWRzLCB0cnkgYW5kIGFjcXVpcmUgTUpVTVBBR0VT SVpFIGNsdXN0ZXJzLgorCQkgKiBIb3dldmVyLCBkbyBzbyB3aXRoIE1fTk9XQUlUIHNvIHRoZSB0 aHJlYWQgY2FuJ3QgZ2V0CisJCSAqIHN0dWNrIHNsZWVwaW5nIG9uICJidGFsbG9jIi4KKwkJICog SWYgdGhpcyBmYWlscywgdXNlIE5GU01DTEdFVCguLk1fTk9XQUlUKSwgd2hpY2ggZG9lcyBhbgor CQkgKiBNR0VUKC4uTV9XQUlUT0spIGZvbGxvd2VkIGJ5IGEgTUNMR0VUKC4uTV9OT1dBSVQpLiAg VGhlCisJCSAqIE1DTEdFVCguLk1fTk9XQUlUKSBtYXkgbm90IGdldCBhIGNsdXN0ZXIsIGJ1dCB3 aWxsIGRyYWluCisJCSAqIHRoZSBtYnVmIGNsdXN0ZXIgem9uZSB3aGVuIGl0IGZhaWxzLgorCQkg KiBBcyBzdWNoLCBhbiBtYnVmIHdpbGwgYWx3YXlzIGJlIGFsbG9jYXRlZCBhbmQgbW9zdCBsaWtl bHkKKwkJICogaXQgd2lsbCBoYXZlIGEgY2x1c3Rlci4KKwkJICovCisJCW0gPSBOVUxMOworCQlp ZiAoZG9fcGFnZXNpemUgIT0gMCkKKwkJCW0gPSBtX2dldGpjbChNX05PV0FJVCwgTVRfREFUQSwg MCwgTUpVTVBBR0VTSVpFKTsKKwkJaWYgKG0gPT0gTlVMTCkKKwkJCU5GU01DTEdFVChtLCBNX05P V0FJVCk7CiAJCW0tPm1fbGVuID0gMDsKIAkJc2l6ID0gbWluKE1fVFJBSUxJTkdTUEFDRShtKSwg bGVmdCk7CiAJCWxlZnQgLT0gc2l6OwpAQCAtMTY1MywxMCArMTY3MSwxMCBAQCBhZ2FpbjoKIAlp ZiAoc2l6ID09IDApIHsKIAkJdnB1dCh2cCk7CiAJCWlmIChuZC0+bmRfZmxhZyAmIE5EX05GU1Yy KSB7Ci0JCQlORlNNX0JVSUxEKHRsLCB1X2ludDMyX3QgKiwgMiAqIE5GU1hfVU5TSUdORUQpOwor CQkJTkZTTV9CVUlMRF9QQUdFTUJDTCh0bCwgdV9pbnQzMl90ICosIDIgKiBORlNYX1VOU0lHTkVE KTsKIAkJfSBlbHNlIHsKIAkJCW5mc3J2X3Bvc3RvcGF0dHIobmQsIGdldHJldCwgJmF0KTsKLQkJ CU5GU01fQlVJTEQodGwsIHVfaW50MzJfdCAqLCA0ICogTkZTWF9VTlNJR05FRCk7CisJCQlORlNN X0JVSUxEX1BBR0VNQkNMKHRsLCB1X2ludDMyX3QgKiwgNCAqIE5GU1hfVU5TSUdORUQpOwogCQkJ dHhkcl9oeXBlcihhdC5uYV9maWxlcmV2LCB0bCk7CiAJCQl0bCArPSAyOwogCQl9CkBAIC0xNzA4 LDcgKzE3MjYsNyBAQCBhZ2FpbjoKIAkgKi8KIAlpZiAobmQtPm5kX2ZsYWcgJiBORF9ORlNWMykg ewogCQluZnNydl9wb3N0b3BhdHRyKG5kLCBnZXRyZXQsICZhdCk7Ci0JCU5GU01fQlVJTEQodGws IHVfaW50MzJfdCAqLCAyICogTkZTWF9VTlNJR05FRCk7CisJCU5GU01fQlVJTERfUEFHRU1CQ0wo dGwsIHVfaW50MzJfdCAqLCAyICogTkZTWF9VTlNJR05FRCk7CiAJCXR4ZHJfaHlwZXIoYXQubmFf ZmlsZXJldiwgdGwpOwogCQlkaXJsZW4gPSBORlNYX1YzUE9TVE9QQVRUUiArIE5GU1hfVkVSRiAr IDIgKiBORlNYX1VOU0lHTkVEOwogCX0gZWxzZSB7CkBAIC0xNzM0LDIwICsxNzUyLDI0IEBAIGFn YWluOgogCQkJICogdGhlIGRpcmVudCBlbnRyeS4KIAkJCSAqLwogCQkJaWYgKG5kLT5uZF9mbGFn ICYgTkRfTkZTVjMpIHsKLQkJCQlORlNNX0JVSUxEKHRsLCB1X2ludDMyX3QgKiwgMyAqIE5GU1hf VU5TSUdORUQpOworCQkJCU5GU01fQlVJTERfUEFHRU1CQ0wodGwsIHVfaW50MzJfdCAqLAorCQkJ CSAgICAzICogTkZTWF9VTlNJR05FRCk7CiAJCQkJKnRsKysgPSBuZXduZnNfdHJ1ZTsKIAkJCQkq dGwrKyA9IDA7CiAJCQl9IGVsc2UgewotCQkJCU5GU01fQlVJTEQodGwsIHVfaW50MzJfdCAqLCAy ICogTkZTWF9VTlNJR05FRCk7CisJCQkJTkZTTV9CVUlMRF9QQUdFTUJDTCh0bCwgdV9pbnQzMl90 ICosCisJCQkJICAgIDIgKiBORlNYX1VOU0lHTkVEKTsKIAkJCQkqdGwrKyA9IG5ld25mc190cnVl OwogCQkJfQogCQkJKnRsID0gdHhkcl91bnNpZ25lZChkcC0+ZF9maWxlbm8pOwogCQkJKHZvaWQp IG5mc21fc3RydG9tKG5kLCBkcC0+ZF9uYW1lLCBubGVuKTsKIAkJCWlmIChuZC0+bmRfZmxhZyAm IE5EX05GU1YzKSB7Ci0JCQkJTkZTTV9CVUlMRCh0bCwgdV9pbnQzMl90ICosIDIgKiBORlNYX1VO U0lHTkVEKTsKKwkJCQlORlNNX0JVSUxEX1BBR0VNQkNMKHRsLCB1X2ludDMyX3QgKiwKKwkJCQkg ICAgMiAqIE5GU1hfVU5TSUdORUQpOwogCQkJCSp0bCsrID0gMDsKIAkJCX0gZWxzZQotCQkJCU5G U01fQlVJTEQodGwsIHVfaW50MzJfdCAqLCBORlNYX1VOU0lHTkVEKTsKKwkJCQlORlNNX0JVSUxE X1BBR0VNQkNMKHRsLCB1X2ludDMyX3QgKiwKKwkJCQkgICAgTkZTWF9VTlNJR05FRCk7CiAJCQkq dGwgPSB0eGRyX3Vuc2lnbmVkKCpjb29raWVwKTsKIAkJfQogCQljcG9zICs9IGRwLT5kX3JlY2xl bjsKQEAgLTE3NTcsNyArMTc3OSw3IEBAIGFnYWluOgogCX0KIAlpZiAoY3BvcyA8IGNlbmQpCiAJ CWVvZmZsYWcgPSAwOwotCU5GU01fQlVJTEQodGwsIHVfaW50MzJfdCAqLCAyICogTkZTWF9VTlNJ R05FRCk7CisJTkZTTV9CVUlMRF9QQUdFTUJDTCh0bCwgdV9pbnQzMl90ICosIDIgKiBORlNYX1VO U0lHTkVEKTsKIAkqdGwrKyA9IG5ld25mc19mYWxzZTsKIAlpZiAoZW9mZmxhZykKIAkJKnRsID0g bmV3bmZzX3RydWU7CkBAIC0xOTI4LDcgKzE5NTAsNyBAQCBhZ2FpbjoKIAkJdnB1dCh2cCk7CiAJ CWlmIChuZC0+bmRfZmxhZyAmIE5EX05GU1YzKQogCQkJbmZzcnZfcG9zdG9wYXR0cihuZCwgZ2V0 cmV0LCAmYXQpOwotCQlORlNNX0JVSUxEKHRsLCB1X2ludDMyX3QgKiwgNCAqIE5GU1hfVU5TSUdO RUQpOworCQlORlNNX0JVSUxEX1BBR0VNQkNMKHRsLCB1X2ludDMyX3QgKiwgNCAqIE5GU1hfVU5T SUdORUQpOwogCQl0eGRyX2h5cGVyKGF0Lm5hX2ZpbGVyZXYsIHRsKTsKIAkJdGwgKz0gMjsKIAkJ KnRsKysgPSBuZXduZnNfZmFsc2U7CkBAIC0yMDMxLDcgKzIwNTMsNyBAQCBhZ2FpbjoKIAl9IGVs c2UgewogCQlkaXJsZW4gPSBORlNYX1ZFUkYgKyAyICogTkZTWF9VTlNJR05FRDsKIAl9Ci0JTkZT TV9CVUlMRCh0bCwgdV9pbnQzMl90ICosIE5GU1hfVkVSRik7CisJTkZTTV9CVUlMRF9QQUdFTUJD TCh0bCwgdV9pbnQzMl90ICosIE5GU1hfVkVSRik7CiAJdHhkcl9oeXBlcihhdC5uYV9maWxlcmV2 LCB0bCk7CiAKIAkvKgpAQCAtMjE4NiwxMiArMjIwOCwxNCBAQCBhZ2FpbjoKIAkJCSAqIEJ1aWxk IHRoZSBkaXJlY3RvcnkgcmVjb3JkIHhkcgogCQkJICovCiAJCQlpZiAobmQtPm5kX2ZsYWcgJiBO RF9ORlNWMykgewotCQkJCU5GU01fQlVJTEQodGwsIHVfaW50MzJfdCAqLCAzICogTkZTWF9VTlNJ R05FRCk7CisJCQkJTkZTTV9CVUlMRF9QQUdFTUJDTCh0bCwgdV9pbnQzMl90ICosCisJCQkJICAg IDMgKiBORlNYX1VOU0lHTkVEKTsKIAkJCQkqdGwrKyA9IG5ld25mc190cnVlOwogCQkJCSp0bCsr ID0gMDsKIAkJCQkqdGwgPSB0eGRyX3Vuc2lnbmVkKGRwLT5kX2ZpbGVubyk7CiAJCQkJZGlybGVu ICs9IG5mc21fc3RydG9tKG5kLCBkcC0+ZF9uYW1lLCBubGVuKTsKLQkJCQlORlNNX0JVSUxEKHRs LCB1X2ludDMyX3QgKiwgMiAqIE5GU1hfVU5TSUdORUQpOworCQkJCU5GU01fQlVJTERfUEFHRU1C Q0wodGwsIHVfaW50MzJfdCAqLAorCQkJCSAgICAyICogTkZTWF9VTlNJR05FRCk7CiAJCQkJKnRs KysgPSAwOwogCQkJCSp0bCA9IHR4ZHJfdW5zaWduZWQoKmNvb2tpZXApOwogCQkJCW5mc3J2X3Bv c3RvcGF0dHIobmQsIDAsIG52YXApOwpAQCAtMjIwMCw3ICsyMjI0LDggQEAgYWdhaW46CiAJCQkJ aWYgKG52cCAhPSBOVUxMKQogCQkJCQl2cHV0KG52cCk7CiAJCQl9IGVsc2UgewotCQkJCU5GU01f QlVJTEQodGwsIHVfaW50MzJfdCAqLCAzICogTkZTWF9VTlNJR05FRCk7CisJCQkJTkZTTV9CVUlM RF9QQUdFTUJDTCh0bCwgdV9pbnQzMl90ICosCisJCQkJICAgIDMgKiBORlNYX1VOU0lHTkVEKTsK IAkJCQkqdGwrKyA9IG5ld25mc190cnVlOwogCQkJCSp0bCsrID0gMDsKIAkJCQkqdGwgPSB0eGRy X3Vuc2lnbmVkKCpjb29raWVwKTsKQEAgLTIyNjcsNyArMjI5Miw3IEBAIGFnYWluOgogCX0gZWxz ZSBpZiAoY3BvcyA8IGNlbmQpCiAJCWVvZmZsYWcgPSAwOwogCWlmICghbmQtPm5kX3JlcHN0YXQp IHsKLQkJTkZTTV9CVUlMRCh0bCwgdV9pbnQzMl90ICosIDIgKiBORlNYX1VOU0lHTkVEKTsKKwkJ TkZTTV9CVUlMRF9QQUdFTUJDTCh0bCwgdV9pbnQzMl90ICosIDIgKiBORlNYX1VOU0lHTkVEKTsK IAkJKnRsKysgPSBuZXduZnNfZmFsc2U7CiAJCWlmIChlb2ZmbGFnKQogCQkJKnRsID0gbmV3bmZz X3RydWU7Ci0tLSBmcy9uZnNjbGllbnQvbmZzX2NsY29tc3Vicy5jLnNhdjIJMjAxNC0wMi0wMSAy MDo0NzowNy4wMDAwMDAwMDAgLTA1MDAKKysrIGZzL25mc2NsaWVudC9uZnNfY2xjb21zdWJzLmMJ MjAxNC0wMy0xNiAyMzoyMjowNi4wMDAwMDAwMDAgLTA0MDAKQEAgLTE1NSw3ICsxNTUsNyBAQCBu ZnNjbF9yZXFzdGFydChzdHJ1Y3QgbmZzcnZfZGVzY3JpcHQgKm5kCiAJICogR2V0IHRoZSBmaXJz dCBtYnVmIGZvciB0aGUgcmVxdWVzdC4KIAkgKi8KIAlpZiAobmZzX2JpZ3JlcXVlc3RbcHJvY251 bV0pCi0JCU5GU01DTEdFVChtYiwgTV9XQUlUT0spOworCQlORlNNQ0xHRVQobWIsIE1fTk9XQUlU KTsKIAllbHNlCiAJCU5GU01HRVQobWIpOwogCW1idWZfc2V0bGVuKG1iLCAwKTsKQEAgLTI2Nyw5 ICsyNjcsMjkgQEAgbmZzbV91aW9tYnVmKHN0cnVjdCBuZnNydl9kZXNjcmlwdCAqbmQsIAogCQl3 aGlsZSAobGVmdCA+IDApIHsKIAkJCW1sZW4gPSBNX1RSQUlMSU5HU1BBQ0UobXApOwogCQkJaWYg KG1sZW4gPT0gMCkgewotCQkJCWlmIChjbGZsZykKLQkJCQkJTkZTTUNMR0VUKG1wLCBNX1dBSVRP Syk7Ci0JCQkJZWxzZQorCQkJCWlmIChjbGZsZyAhPSAwKSB7CisJCQkJCS8qCisJCQkJCSAqIEZv ciBsYXJnZSB3cml0ZXMsIHRyeSBhbmQgYWNxdWlyZQorCQkJCQkgKiBNSlVNUEFHRVNJWkUgY2x1 c3RlcnMuCisJCQkJCSAqIEhvd2V2ZXIsIGRvIHNvIHdpdGggTV9OT1dBSVQgc28gdGhlCisJCQkJ CSAqIHRocmVhZCBjYW4ndCBnZXQgc3R1Y2sgc2xlZXBpbmcgb24KKwkJCQkJICogImJ0YWxsb2Mi LiAgSWYgdGhpcyBmYWlscywgdXNlCisJCQkJCSAqIE5GU01DTEdFVCguLk1fTk9XQUlUKSwgd2hp Y2ggZG9lcyBhbgorCQkJCQkgKiBNR0VUKC4uTV9XQUlUT0spIGZvbGxvd2VkIGJ5IGEKKwkJCQkJ ICogTUNMR0UgVCguLk1fTk9XQUlUKS4gVGhpcyBtYXkgbm90IGdldAorCQkJCQkgKiBhIGNsdXN0 ZXIsIGJ1dCB3aWxsIGRyYWluIHRoZSBtYnVmCisJCQkJCSAqIGNsdXN0ZXIgem9uZSB3aGVuIGl0 IGZhaWxzLgorCQkJCQkgKiBBcyBzdWNoLCBhbiBtYnVmIHdpbGwgYWx3YXlzIGJlCisJCQkJCSAq IGFsbG9jYXRlZCBhbmQgbW9zdCBsaWtlbHkgaXQgd2lsbAorCQkJCQkgKiBoYXZlIGEgY2x1c3Rl ci4KKwkJCQkJICovCisjaWYgTUpVTVBBR0VTSVpFICE9IE1DTEJZVEVTCisJCQkJCW1wID0gbV9n ZXRqY2woTV9OT1dBSVQsIE1UX0RBVEEsIDAsCisJCQkJCSAgICBNSlVNUEFHRVNJWkUpOworCQkJ CQlpZiAobXAgPT0gTlVMTCkKKyNlbmRpZgorCQkJCQkJTkZTTUNMR0VUKG1wLCBNX05PV0FJVCk7 CisJCQkJfSBlbHNlCiAJCQkJCU5GU01HRVQobXApOwogCQkJCW1idWZfc2V0bGVuKG1wLCAwKTsK IAkJCQltYnVmX3NldG5leHQobXAyLCBtcCk7Ci0tLSBmcy9uZnMvbmZzbV9zdWJzLmguc2F2Mgky MDE0LTAyLTAxIDE5OjUxOjEyLjAwMDAwMDAwMCAtMDUwMAorKysgZnMvbmZzL25mc21fc3Vicy5o CTIwMTQtMDMtMTMgMTg6NTQ6MjcuMDAwMDAwMDAwIC0wNDAwCkBAIC04OSw2ICs4OSwzNyBAQCBu ZnNtX2J1aWxkKHN0cnVjdCBuZnNydl9kZXNjcmlwdCAqbmQsIGluCiAKICNkZWZpbmUJTkZTTV9C VUlMRChhLCBjLCBzKQkoKGEpID0gKGMpbmZzbV9idWlsZChuZCwgKHMpKSkKIAorLyoKKyAqIFNh bWUgYXMgYWJvdmUsIGJ1dCBhbGxvY2F0ZXMgTUpVTVBBR0VTSVpFIG1idWYgY2x1c3RlcnMsIGlm IHBvc3NpYmxlLgorICovCitzdGF0aWMgX19pbmxpbmUgdm9pZCAqCituZnNtX2J1aWxkX3BhZ2Vt YmNsKHN0cnVjdCBuZnNydl9kZXNjcmlwdCAqbmQsIGludCBzaXopCit7CisJdm9pZCAqcmV0cDsK KwlzdHJ1Y3QgbWJ1ZiAqbWIyOworCisJaWYgKHNpeiA+IE1fVFJBSUxJTkdTUEFDRShuZC0+bmRf bWIpKSB7CisJCW1iMiA9IE5VTEw7CisjaWYgTUpVTVBBR0VTSVpFICE9IE1DTEJZVEVTCisJCW1i MiA9IG1fZ2V0amNsKE1fTk9XQUlULCBNVF9EQVRBLCAwLCBNSlVNUEFHRVNJWkUpOworI2VuZGlm CisJCWlmIChtYjIgPT0gTlVMTCkKKwkJCU5GU01DTEdFVChtYjIsIE1fTk9XQUlUKTsKKwkJaWYg KHNpeiA+IE1MRU4pCisJCQlwYW5pYygiYnVpbGQgPiBNTEVOIik7CisJCW1idWZfc2V0bGVuKG1i MiwgMCk7CisJCW5kLT5uZF9icG9zID0gTkZTTVRPRChtYjIsIGNhZGRyX3QpOworCQluZC0+bmRf bWItPm1fbmV4dCA9IG1iMjsKKwkJbmQtPm5kX21iID0gbWIyOworCX0KKwlyZXRwID0gKHZvaWQg KikobmQtPm5kX2Jwb3MpOworCW5kLT5uZF9tYi0+bV9sZW4gKz0gc2l6OworCW5kLT5uZF9icG9z ICs9IHNpejsKKwlyZXR1cm4gKHJldHApOworfQorCisjZGVmaW5lCU5GU01fQlVJTERfUEFHRU1C Q0woYSwgYywgcykJKChhKSA9IChjKW5mc21fYnVpbGRfcGFnZW1iY2wobmQsIChzKSkpCisKIHN0 YXRpYyBfX2lubGluZSB2b2lkICoKIG5mc21fZGlzc2VjdChzdHJ1Y3QgbmZzcnZfZGVzY3JpcHQg Km5kLCBpbnQgc2l6KQogewotLS0gZnMvbmZzL25mc3BvcnQuaC5zYXYyCTIwMTQtMDItMTMgMTk6 MDM6MjIuMDAwMDAwMDAwIC0wNTAwCisrKyBmcy9uZnMvbmZzcG9ydC5oCTIwMTQtMDItMTMgMTk6 MTQ6MjQuMDAwMDAwMDAwIC0wNTAwCkBAIC0xMzgsNiArMTM4LDggQEAKIAogLyoKICAqIEFsbG9j YXRlIG1idWZzLiBNdXN0IHN1Y2NlZWQgYW5kIG5ldmVyIHNldCB0aGUgbWJ1ZiBwdHIgdG8gTlVM TC4KKyAqIE5vdGUgdGhhdCB3aGVuIE5GU01DTEdFVChtLCBNX05PV0FJVCkgaXMgZG9uZSwgaXQg c3RpbGwgbXVzdCBhbGxvY2F0ZQorICogYW4gbWJ1ZiAoYW5kIGNhbiBzbGVlcCksIGJ1dCBtaWdo dCBub3QgZ2V0IGEgY2x1c3RlciwgaW4gdGhlIHdvcnN0IGNhc2UuCiAgKi8KICNkZWZpbmUJTkZT TUdFVChtKQlkbyB7IAkJCQkJXAogCQlNR0VUKChtKSwgTV9XQUlUT0ssIE1UX0RBVEEpOyAJCQlc Cg== ------=_Part_1670344_208375117.1395619029041 Content-Type: text/x-patch; name=tsomaxseg.patch Content-Disposition: attachment; filename=tsomaxseg.patch Content-Transfer-Encoding: base64 LS0tIGtlcm4vdWlwY19zb2NrYnVmLmMuc2F2CTIwMTQtMDEtMzAgMjA6Mjc6MTcuMDAwMDAwMDAw IC0wNTAwCisrKyBrZXJuL3VpcGNfc29ja2J1Zi5jCTIwMTQtMDEtMzAgMjI6MTI6MDguMDAwMDAw MDAwIC0wNTAwCkBAIC05NjUsNiArOTY1LDM5IEBAIHNic25kcHRyKHN0cnVjdCBzb2NrYnVmICpz YiwgdV9pbnQgb2ZmLCAKIH0KIAogLyoKKyAqIFJldHVybiB0aGUgZmlyc3QgbWJ1ZiBmb3IgdGhl IHByb3ZpZGVkIG9mZnNldC4KKyAqLworc3RydWN0IG1idWYgKgorc2JzbmRtYnVmKHN0cnVjdCBz b2NrYnVmICpzYiwgdV9pbnQgb2ZmLCBsb25nICpmaXJzdF9sZW4pCit7CisJc3RydWN0IG1idWYg Km07CisKKwlLQVNTRVJUKHNiLT5zYl9tYiAhPSBOVUxMLCAoIiVzOiBzYl9tYiBpcyBOVUxMIiwg X19mdW5jX18pKTsKKworCSpmaXJzdF9sZW4gPSAwOworCS8qCisJICogSXMgb2ZmIGJlbG93IHN0 b3JlZCBvZmZzZXQ/IEhhcHBlbnMgb24gcmV0cmFuc21pdHMuCisJICogSWYgc28sIGp1c3QgdXNl IHNiX21iLgorCSAqLworCWlmIChzYi0+c2Jfc25kcHRyID09IE5VTEwgfHwgc2ItPnNiX3NuZHB0 cm9mZiA+IG9mZikKKwkJbSA9IHNiLT5zYl9tYjsKKwllbHNlIHsKKwkJbSA9IHNiLT5zYl9zbmRw dHI7CisJCW9mZiAtPSBzYi0+c2Jfc25kcHRyb2ZmOworCX0KKwl3aGlsZSAob2ZmID4gMCAmJiBt ICE9IE5VTEwpIHsKKwkJaWYgKG9mZiA8IG0tPm1fbGVuKQorCQkJYnJlYWs7CisJCW9mZiAtPSBt LT5tX2xlbjsKKwkJbSA9IG0tPm1fbmV4dDsKKwl9CisJaWYgKG0gIT0gTlVMTCkKKwkJKmZpcnN0 X2xlbiA9IG0tPm1fbGVuIC0gb2ZmOworCisJcmV0dXJuIChtKTsKK30KKworLyoKICAqIERyb3Ag YSByZWNvcmQgb2ZmIHRoZSBmcm9udCBvZiBhIHNvY2tidWYgYW5kIG1vdmUgdGhlIG5leHQgcmVj b3JkIHRvIHRoZQogICogZnJvbnQuCiAgKi8KLS0tIHN5cy9zb2NrYnVmLmguc2F2CTIwMTQtMDEt MzAgMjA6NDI6MjguMDAwMDAwMDAwIC0wNTAwCisrKyBzeXMvc29ja2J1Zi5oCTIwMTQtMDEtMzAg MjI6MDg6NDMuMDAwMDAwMDAwIC0wNTAwCkBAIC0xNTMsNiArMTUzLDggQEAgaW50CXNicmVzZXJ2 ZV9sb2NrZWQoc3RydWN0IHNvY2tidWYgKnNiLAogCSAgICBzdHJ1Y3QgdGhyZWFkICp0ZCk7CiBz dHJ1Y3QgbWJ1ZiAqCiAJc2JzbmRwdHIoc3RydWN0IHNvY2tidWYgKnNiLCB1X2ludCBvZmYsIHVf aW50IGxlbiwgdV9pbnQgKm1vZmYpOworc3RydWN0IG1idWYgKgorCXNic25kbWJ1ZihzdHJ1Y3Qg c29ja2J1ZiAqc2IsIHVfaW50IG9mZiwgbG9uZyAqZmlyc3RfbGVuKTsKIHZvaWQJc2J0b3hzb2Nr YnVmKHN0cnVjdCBzb2NrYnVmICpzYiwgc3RydWN0IHhzb2NrYnVmICp4c2IpOwogaW50CXNid2Fp dChzdHJ1Y3Qgc29ja2J1ZiAqc2IpOwogaW50CXNibG9jayhzdHJ1Y3Qgc29ja2J1ZiAqc2IsIGlu dCBmbGFncyk7Ci0tLSBuZXRpbmV0L3RjcF9pbnB1dC5jLnNhdgkyMDE0LTAxLTMwIDE5OjM3OjUy LjAwMDAwMDAwMCAtMDUwMAorKysgbmV0aW5ldC90Y3BfaW5wdXQuYwkyMDE0LTAxLTMwIDE5OjM5 OjA3LjAwMDAwMDAwMCAtMDUwMApAQCAtMzYyNyw2ICszNjI3LDcgQEAgdGNwX21zcyhzdHJ1Y3Qg dGNwY2IgKnRwLCBpbnQgb2ZmZXIpCiAJaWYgKGNhcC5pZmNhcCAmIENTVU1fVFNPKSB7CiAJCXRw LT50X2ZsYWdzIHw9IFRGX1RTTzsKIAkJdHAtPnRfdHNvbWF4ID0gY2FwLnRzb21heDsKKwkJdHAt PnRfdHNvbWF4c2VncyA9IGNhcC50c29tYXhzZWdzOwogCX0KIH0KIAotLS0gbmV0aW5ldC90Y3Bf b3V0cHV0LmMuc2F2CTIwMTQtMDEtMzAgMTg6NTU6MTUuMDAwMDAwMDAwIC0wNTAwCisrKyBuZXRp bmV0L3RjcF9vdXRwdXQuYwkyMDE0LTAxLTMwIDIyOjE4OjU2LjAwMDAwMDAwMCAtMDUwMApAQCAt MTY2LDggKzE2Niw4IEBAIGludAogdGNwX291dHB1dChzdHJ1Y3QgdGNwY2IgKnRwKQogewogCXN0 cnVjdCBzb2NrZXQgKnNvID0gdHAtPnRfaW5wY2ItPmlucF9zb2NrZXQ7Ci0JbG9uZyBsZW4sIHJl Y3dpbiwgc2VuZHdpbjsKLQlpbnQgb2ZmLCBmbGFncywgZXJyb3IgPSAwOwkvKiBLZWVwIGNvbXBp bGVyIGhhcHB5ICovCisJbG9uZyBsZW4sIHJlY3dpbiwgc2VuZHdpbiwgdHNvX3RsZW47CisJaW50 IGNudCwgb2ZmLCBmbGFncywgZXJyb3IgPSAwOwkvKiBLZWVwIGNvbXBpbGVyIGhhcHB5ICovCiAJ c3RydWN0IG1idWYgKm07CiAJc3RydWN0IGlwICppcCA9IE5VTEw7CiAJc3RydWN0IGlwb3ZseSAq aXBvdiA9IE5VTEw7CkBAIC03ODAsNiArNzgwLDI0IEBAIHNlbmQ6CiAJCQl9CiAKIAkJCS8qCisJ CQkgKiBMaW1pdCB0aGUgbnVtYmVyIG9mIFRTTyB0cmFuc21pdCBzZWdtZW50cyAobWJ1ZnMKKwkJ CSAqIGluIG1idWYgbGlzdCkgdG8gdHAtPnRfdHNvbWF4c2Vncy4KKwkJCSAqLworCQkJY250ID0g MDsKKwkJCW0gPSBzYnNuZG1idWYoJnNvLT5zb19zbmQsIG9mZiwgJnRzb190bGVuKTsKKwkJCXdo aWxlIChtICE9IE5VTEwgJiYgY250IDwgdHAtPnRfdHNvbWF4c2VncyAmJgorCQkJICAgIHRzb190 bGVuIDwgbGVuKSB7CisJCQkJaWYgKGNudCA+IDApCisJCQkJCXRzb190bGVuICs9IG0tPm1fbGVu OworCQkJCWNudCsrOworCQkJCW0gPSBtLT5tX25leHQ7CisJCQl9CisJCQlpZiAobSAhPSBOVUxM ICYmIHRzb190bGVuIDwgbGVuKSB7CisJCQkJbGVuID0gdHNvX3RsZW47CisJCQkJc2VuZGFsb3Qg PSAxOworCQkJfQorCisJCQkvKgogCQkJICogUHJldmVudCB0aGUgbGFzdCBzZWdtZW50IGZyb20g YmVpbmcKIAkJCSAqIGZyYWN0aW9uYWwgdW5sZXNzIHRoZSBzZW5kIHNvY2tidWYgY2FuCiAJCQkg KiBiZSBlbXB0aWVkLgotLS0gbmV0aW5ldC90Y3Bfc3Vici5jLnNhdgkyMDE0LTAxLTMwIDE5OjQ0 OjM1LjAwMDAwMDAwMCAtMDUwMAorKysgbmV0aW5ldC90Y3Bfc3Vici5jCTIwMTQtMDEtMzAgMjA6 NTY6MTIuMDAwMDAwMDAwIC0wNTAwCkBAIC0xODAwLDYgKzE4MDAsMTIgQEAgdGNwX21heG10dShz dHJ1Y3QgaW5fY29ubmluZm8gKmluYywgc3RydQogCQkJICAgIGlmcC0+aWZfaHdhc3Npc3QgJiBD U1VNX1RTTykKIAkJCQljYXAtPmlmY2FwIHw9IENTVU1fVFNPOwogCQkJCWNhcC0+dHNvbWF4ID0g aWZwLT5pZl9od190c29tYXg7CisjaWZkZWYgbm90eWV0CisJCQkJY2FwLT50c29tYXhzZWdzID0g aWZwLT5pZl9od190c29tYXhzZWdzOworI2VuZGlmCisJCQkJaWYgKGNhcC0+dHNvbWF4c2VncyA9 PSAwKQorCQkJCQljYXAtPnRzb21heHNlZ3MgPQorCQkJCQkgICAgVENQVFNPX01BWF9UWF9TRUdT X0RFRkFVTFQ7CiAJCX0KIAkJUlRGUkVFKHNyby5yb19ydCk7CiAJfQotLS0gbmV0aW5ldC90Y3Bf dmFyLmguc2F2CTIwMTQtMDEtMzAgMTk6Mzk6MjIuMDAwMDAwMDAwIC0wNTAwCisrKyBuZXRpbmV0 L3RjcF92YXIuaAkyMDE0LTAxLTMwIDIwOjUyOjU3LjAwMDAwMDAwMCAtMDUwMApAQCAtMjA5LDYg KzIwOSw3IEBAIHN0cnVjdCB0Y3BjYiB7CiAJdV9pbnQJdF9rZWVwY250OwkJLyogbnVtYmVyIG9m IGtlZXBhbGl2ZXMgYmVmb3JlIGNsb3NlICovCiAKIAl1X2ludAl0X3Rzb21heDsJCS8qIHRzbyBi dXJzdCBsZW5ndGggbGltaXQgKi8KKwl1X2ludAl0X3Rzb21heHNlZ3M7CQkvKiB0c28gYnVyc3Qg c2VnbWVudCBsaW1pdCAqLwogCiAJdWludDMyX3QgdF9pc3BhcmVbOF07CQkvKiA1IFVUTywgMyBU QkQgKi8KIAl2b2lkCSp0X3BzcGFyZTJbNF07CQkvKiA0IFRCRCAqLwpAQCAtMjY4LDYgKzI2OSwx MSBAQCBzdHJ1Y3QgdGNwY2IgewogI2RlZmluZQlUQ1BPT0JfSEFWRURBVEEJMHgwMQogI2RlZmlu ZQlUQ1BPT0JfSEFEREFUQQkweDAyCiAKKy8qCisgKiBEZWZhdWx0IHZhbHVlIGZvciBUU08gbWF4 aW11bSBudW1iZXIgb2YgdHJhbnNtaXQgc2VnbWVudHMgKGNvdW50IG9mIG1idWZzKS4KKyAqLwor I2RlZmluZQlUQ1BUU09fTUFYX1RYX1NFR1NfREVGQVVMVAkzMAorCiAjaWZkZWYgVENQX1NJR05B VFVSRQogLyoKICAqIERlZmluZXMgd2hpY2ggYXJlIG5lZWRlZCBieSB0aGUgeGZvcm1fdGNwIG1v ZHVsZSBhbmQgdGNwX1tpbnxvdXRdcHV0CkBAIC0zMzMsNiArMzM5LDcgQEAgc3RydWN0IGhjX21l dHJpY3NfbGl0ZSB7CS8qIG11c3Qgc3RheSBpbgogc3RydWN0IHRjcF9pZmNhcCB7CiAJaW50CWlm Y2FwOwogCXVfaW50CXRzb21heDsKKwl1X2ludAl0c29tYXhzZWdzOwogfTsKIAogI2lmbmRlZiBf TkVUSU5FVF9JTl9QQ0JfSF8K ------=_Part_1670344_208375117.1395619029041--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1936201708.1670348.1395619029045.JavaMail.root>