Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 19 Jul 2017 20:59:37 +1000 (EST)
From:      Bruce Evans <brde@optusnet.com.au>
To:        Ian Lepore <ian@freebsd.org>
Cc:        Bruce Evans <brde@optusnet.com.au>, src-committers@freebsd.org,  svn-src-all@freebsd.org, svn-src-head@freebsd.org
Subject:   Re: svn commit: r320901 - in head/sys: amd64/amd64 isa x86/isa
Message-ID:  <20170719183850.G1161@besplex.bde.org>
In-Reply-To: <1500389932.22314.151.camel@freebsd.org>
References:  <201707120242.v6C2gvDB026199@repo.freebsd.org>   <20170712224803.G1991@besplex.bde.org> <1500308973.22314.89.camel@freebsd.org> <20170718153823.K1105@besplex.bde.org> <1500389932.22314.151.camel@freebsd.org>

next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, 18 Jul 2017, Ian Lepore wrote:

> On Tue, 2017-07-18 at 18:08 +1000, Bruce Evans wrote:

>> [lots of stuff about locking and accurate time-setting]
>>
>> Good idea.
>>
>> My inittodr() waits for the seconds register to change.=A0 This
>> increases
>> boot time by an average of 0.5 seconds, but is needed to get a result
>> that is not out of date by an average of 0.5 seconds.=A0 After the
>> seconds
>> register changes, the window for reading the other registers is
>> almost
>> 1 second, but the reads must complete in a few microseconds anyway
>> else
>> the result is still too out of date.=A0 Debugging printf()s show that
>> the reads usually complete in 10-50 usec with low latency (faster on
>> old systems with faster ISA buses).
>>
>
> I was considering adding a tunable to do that for people who don't mind
> the extra time to boot/wakeup.=A0=A0It's interesting that you label an
> error of a few microseconds as "too out of date", I was thinking that
> reducing the average error from .5s to .005s was good enough.
>
> The .005s number comes from an idea of using a callout that begins as
> soon as interrupts are available that checks for rtc rollover every
> 10ms, with the hopes that it can catch the rollover between the time
> callouts start working and rootfs mount time when inittodr() gets
> called.=A0=A0If it can catch the rollover happening in that window, it ca=
n
> latch the monotonic clock and then use that at inittodr() time to
> calculate the fractional part of the time.=A0=A0Even if it doesn't catch
> the rollover in that time, it at least has a measurement that sets a
> floor value for the fractional part, reducing the error to something
> less than .5s.

Hmm, my rtcintr() already does essentially that without really trying.
It is designed to fix up the time after "suspension" in ddb breaks the
time by stopping timecounter updates.  The fixed up time is:

    (the rtc time) +
    (the last known good difference between the timecounter time and the
      rtc time) +
    (an adjustment for the drift rate)

My rtcintr() samples the difference and calibrates the drift every 64
seconds.  It uses rtc update interrupts every second to get precise
timing for the sampling and to avoid having to wait for updates to
complete before sampling.

Initially, the offset and drift are unknown, but using 0 for them would
give similar functionality to inittodr().

> I have no idea yet how much time elapses between interrupts coming
> online and the first inittodr() call on a typical modern x86
> system.=A0=A0For all my arm systems with nfsroot it's several seconds. =
=A0If
> it averages a few milliseconds on x86 systems, it isn't useful enough
> to justify the extra work. =A0If it's typically half a second or more it
> probably is worth it.

My fastest x86 system takes 6 seconds to boot from boot2 (no loader) to
mounting root.  Surprisingly, inittodr() runs even later than mountroot
(at least on an old system).  That must be for the old feature of
defaulting the time to a timestamp on the (ffs) root partition.  Normally
that is not used, but if inittodr() ran earlier than it would have to
run again if it couldn't find a hardware rtc.  Interrupt handling starts
working much earlier than that, but I wouldn't trust all time features
to be initialized on the first call to rtcintr().

However, it is unimportant to set the time very early in the boot.  This
just makes CLOCK_REALTIME timestamps valid earlier.

The time should be fixed up similarly after real suspensions.  Then it
is even less clear what the reinitialization order actually is.

>> Accuracy for reading is not very important because accuracy for
>> writing
>> is very low (1-2 seconds).=A0 My resettodr() only writes if the
>> difference
>> is more than 2 seconds.
>>
>
> The gross inaccuracy when writing is fixable now. =A0I'm not worried
> about the microsecond inaccuracies of accessing the hardware, what I
> want to fix is the fact that clock_settime() may get called at x.995
> and will record the time as x.0.

Very old old versions seemed to have complications related to the
hardware.  Something to do with half-second boundaries, because the
hardware is "smart" about this 0.995 error and schedules the next
update to 0.5 seconds after writing to the seconds register (this
reduces the average error to 0.5 seconds for sloppy settings)?
I can't find these in the 2004 version.

> Because settime is now called from a task thread, any clock driver is
> free to sleep as needed to set the clock hardware in a way that aligns
> its second rollover with kernel time. =A0Not efficient, but easy to do.

The syscall was always free to sleep, and it is a bug for it return before
setting the time.  Except, the syscall never claimed to even have hardware
to write to.  If there were an RTC driver, then it should support both sync
and async settings.

> Even better, most rtc hardware has unused storage. =A0Sometimes in the
> form of battery-backed ram, sometimes in the form of things like alarm
> registers that aren't used for anything and a driver can tuck away
> values in them for its own use. =A0If clock_settime() stores the
> fractional part of the second in such a location, clock_gettime() can
> use it to remove all of the inaccuracy that otherwise happens when the
> fraction is discarded. =A0If even a single bit is available the error is
> cut in half. =A0If a byte is available it drops to ~40ms.

I tried to find unused rtc registers for the purpose of fixing nextboot
and other boot loaders which do dangerous writes to disks.  A single bit
is almost enough for them too.  I didn't really succeed.  On x86, everythin=
g
is reserved or undocumented and most of the 256 (?) bytes of standard cmos
was used by BIOSes before flash RAM made this unecessary (I don't know
how much of it they use now).  The best that I could find was the alarm
registers.  On some of my systems, the BIOS does use these registers, but
the use is under user control and when non-use by the BIOS is configured,
the BIOS doesn't clobber the registers.

Bruce
From owner-svn-src-all@freebsd.org  Wed Jul 19 12:27:42 2017
Return-Path: <owner-svn-src-all@freebsd.org>
Delivered-To: svn-src-all@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0E710C7C48F;
 Wed, 19 Jul 2017 12:27:42 +0000 (UTC)
 (envelope-from tuexen@FreeBSD.org)
Received: from repo.freebsd.org (repo.freebsd.org
 [IPv6:2610:1c1:1:6068::e6a:0])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id A39F87D45A;
 Wed, 19 Jul 2017 12:27:41 +0000 (UTC)
 (envelope-from tuexen@FreeBSD.org)
Received: from repo.freebsd.org ([127.0.1.37])
 by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v6JCRetr020754;
 Wed, 19 Jul 2017 12:27:40 GMT (envelope-from tuexen@FreeBSD.org)
Received: (from tuexen@localhost)
 by repo.freebsd.org (8.15.2/8.15.2/Submit) id v6JCReUh020750;
 Wed, 19 Jul 2017 12:27:40 GMT (envelope-from tuexen@FreeBSD.org)
Message-Id: <201707191227.v6JCReUh020750@repo.freebsd.org>
X-Authentication-Warning: repo.freebsd.org: tuexen set sender to
 tuexen@FreeBSD.org using -f
From: Michael Tuexen <tuexen@FreeBSD.org>
Date: Wed, 19 Jul 2017 12:27:40 +0000 (UTC)
To: src-committers@freebsd.org, svn-src-all@freebsd.org,
 svn-src-head@freebsd.org
Subject: svn commit: r321197 - head/sys/netinet
X-SVN-Group: head
X-SVN-Commit-Author: tuexen
X-SVN-Commit-Paths: head/sys/netinet
X-SVN-Commit-Revision: 321197
X-SVN-Commit-Repository: base
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-BeenThere: svn-src-all@freebsd.org
X-Mailman-Version: 2.1.23
Precedence: list
List-Id: "SVN commit messages for the entire src tree \(except for &quot;
 user&quot; and &quot; projects&quot; \)" <svn-src-all.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/svn-src-all>,
 <mailto:svn-src-all-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/svn-src-all/>;
List-Post: <mailto:svn-src-all@freebsd.org>
List-Help: <mailto:svn-src-all-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/svn-src-all>,
 <mailto:svn-src-all-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 19 Jul 2017 12:27:42 -0000

Author: tuexen
Date: Wed Jul 19 12:27:40 2017
New Revision: 321197
URL: https://svnweb.freebsd.org/changeset/base/321197

Log:
  Fix the accounting and add code to detect errors in accounting.
  Joint work with rrs@
  MFC after:	1 week

Modified:
  head/sys/netinet/sctp_auth.c
  head/sys/netinet/sctp_indata.c
  head/sys/netinet/sctp_pcb.c
  head/sys/netinet/sctputil.c

Modified: head/sys/netinet/sctp_auth.c
==============================================================================
--- head/sys/netinet/sctp_auth.c	Wed Jul 19 09:59:32 2017	(r321196)
+++ head/sys/netinet/sctp_auth.c	Wed Jul 19 12:27:40 2017	(r321197)
@@ -1797,8 +1797,8 @@ sctp_notify_authentication(struct sctp_tcb *stcb, uint
 		sctp_m_freem(m_notify);
 		return;
 	}
-	control->spec_flags = M_NOTIFICATION;
 	control->length = SCTP_BUF_LEN(m_notify);
+	control->spec_flags = M_NOTIFICATION;
 	/* not that we need this */
 	control->tail_mbuf = m_notify;
 	sctp_add_to_readq(stcb->sctp_ep, stcb, control,

Modified: head/sys/netinet/sctp_indata.c
==============================================================================
--- head/sys/netinet/sctp_indata.c	Wed Jul 19 09:59:32 2017	(r321196)
+++ head/sys/netinet/sctp_indata.c	Wed Jul 19 12:27:40 2017	(r321197)
@@ -59,7 +59,7 @@ __FBSDID("$FreeBSD$");
  * This will cause sctp_service_queues() to get called on the top entry in
  * the list.
  */
-static void
+static uint32_t
 sctp_add_chk_to_control(struct sctp_queued_to_read *control,
     struct sctp_stream_in *strm,
     struct sctp_tcb *stcb,
@@ -92,6 +92,8 @@ sctp_calc_rwnd(struct sctp_tcb *stcb, struct sctp_asso
 	    asoc->size_on_reasm_queue == 0 &&
 	    asoc->size_on_all_streams == 0) {
 		/* Full rwnd granted */
+		KASSERT(asoc->cnt_on_reasm_queue == 0, ("cnt_on_reasm_queue is %u", asoc->cnt_on_reasm_queue));
+		KASSERT(asoc->cnt_on_all_streams == 0, ("cnt_on_all_streams is %u", asoc->cnt_on_all_streams));
 		calc = max(SCTP_SB_LIMIT_RCV(stcb->sctp_socket), SCTP_MINIMAL_RWND);
 		return (calc);
 	}
@@ -558,7 +560,15 @@ sctp_queue_data_to_stream(struct sctp_tcb *stcb,
 		}
 		/* EY it wont be queued if it could be delivered directly */
 		queue_needed = 0;
-		asoc->size_on_all_streams -= control->length;
+		if (asoc->size_on_all_streams >= control->length) {
+			asoc->size_on_all_streams -= control->length;
+		} else {
+#ifdef INVARIANTS
+			panic("size_on_all_streams = %u smaller than control length %u", asoc->size_on_all_streams, control->length);
+#else
+			asoc->size_on_all_streams = 0;
+#endif
+		}
 		sctp_ucount_decr(asoc->cnt_on_all_streams);
 		strm->last_mid_delivered++;
 		sctp_mark_non_revokable(asoc, control->sinfo_tsn);
@@ -571,11 +581,19 @@ sctp_queue_data_to_stream(struct sctp_tcb *stcb,
 			nxt_todel = strm->last_mid_delivered + 1;
 			if (SCTP_MID_EQ(asoc->idata_supported, nxt_todel, control->mid) &&
 			    (((control->sinfo_flags >> 8) & SCTP_DATA_NOT_FRAG) == SCTP_DATA_NOT_FRAG)) {
-				asoc->size_on_all_streams -= control->length;
-				sctp_ucount_decr(asoc->cnt_on_all_streams);
 				if (control->on_strm_q == SCTP_ON_ORDERED) {
 					TAILQ_REMOVE(&strm->inqueue, control, next_instrm);
+					if (asoc->size_on_all_streams >= control->length) {
+						asoc->size_on_all_streams -= control->length;
+					} else {
 #ifdef INVARIANTS
+						panic("size_on_all_streams = %u smaller than control length %u", asoc->size_on_all_streams, control->length);
+#else
+						asoc->size_on_all_streams = 0;
+#endif
+					}
+					sctp_ucount_decr(asoc->cnt_on_all_streams);
+#ifdef INVARIANTS
 				} else {
 					panic("Huh control: %p is on_strm_q: %d",
 					    control, control->on_strm_q);
@@ -671,7 +689,7 @@ sctp_setup_tail_pointer(struct sctp_queued_to_read *co
 }
 
 static void
-sctp_add_to_tail_pointer(struct sctp_queued_to_read *control, struct mbuf *m)
+sctp_add_to_tail_pointer(struct sctp_queued_to_read *control, struct mbuf *m, uint32_t *added)
 {
 	struct mbuf *prev = NULL;
 	struct sctp_tcb *stcb;
@@ -715,6 +733,7 @@ sctp_add_to_tail_pointer(struct sctp_queued_to_read *c
 			 */
 			sctp_sballoc(stcb, &stcb->sctp_socket->so_rcv, m);
 		}
+		*added += SCTP_BUF_LEN(m);
 		atomic_add_int(&control->length, SCTP_BUF_LEN(m));
 		m = SCTP_BUF_NEXT(m);
 	}
@@ -815,7 +834,15 @@ restart:
 					tchk = TAILQ_FIRST(&control->reasm);
 					if (tchk->rec.data.rcv_flags & SCTP_DATA_FIRST_FRAG) {
 						TAILQ_REMOVE(&control->reasm, tchk, sctp_next);
-						asoc->size_on_reasm_queue -= tchk->send_size;
+						if (asoc->size_on_reasm_queue >= tchk->send_size) {
+							asoc->size_on_reasm_queue -= tchk->send_size;
+						} else {
+#ifdef INVARIANTS
+							panic("size_on_reasm_queue = %u smaller than chunk length %u", asoc->size_on_reasm_queue, tchk->send_size);
+#else
+							asoc->size_on_reasm_queue = 0;
+#endif
+						}
 						sctp_ucount_decr(asoc->cnt_on_reasm_queue);
 						nc->first_frag_seen = 1;
 						nc->fsn_included = tchk->rec.data.fsn;
@@ -1127,6 +1154,16 @@ done_un:
 #endif
 				SCTP_STAT_INCR_COUNTER64(sctps_reasmusrmsgs);
 				TAILQ_REMOVE(&strm->inqueue, control, next_instrm);
+				if (asoc->size_on_all_streams >= control->length) {
+					asoc->size_on_all_streams -= control->length;
+				} else {
+#ifdef INVARIANTS
+					panic("size_on_all_streams = %u smaller than control length %u", asoc->size_on_all_streams, control->length);
+#else
+					asoc->size_on_all_streams = 0;
+#endif
+				}
+				sctp_ucount_decr(asoc->cnt_on_all_streams);
 				control->on_strm_q = 0;
 			}
 			if (strm->pd_api_started && control->pdapi_started) {
@@ -1173,6 +1210,16 @@ deliver_more:
 #endif
 					SCTP_STAT_INCR_COUNTER64(sctps_reasmusrmsgs);
 					TAILQ_REMOVE(&strm->inqueue, control, next_instrm);
+					if (asoc->size_on_all_streams >= control->length) {
+						asoc->size_on_all_streams -= control->length;
+					} else {
+#ifdef INVARIANTS
+						panic("size_on_all_streams = %u smaller than control length %u", asoc->size_on_all_streams, control->length);
+#else
+						asoc->size_on_all_streams = 0;
+#endif
+					}
+					sctp_ucount_decr(asoc->cnt_on_all_streams);
 					control->on_strm_q = 0;
 				}
 				ret++;
@@ -1219,7 +1266,7 @@ out:
 }
 
 
-void
+uint32_t
 sctp_add_chk_to_control(struct sctp_queued_to_read *control,
     struct sctp_stream_in *strm,
     struct sctp_tcb *stcb, struct sctp_association *asoc,
@@ -1229,6 +1276,7 @@ sctp_add_chk_to_control(struct sctp_queued_to_read *co
 	 * Given a control and a chunk, merge the data from the chk onto the
 	 * control and free up the chunk resources.
 	 */
+	uint32_t added = 0;
 	int i_locked = 0;
 
 	if (control->on_read_q && (hold_rlock == 0)) {
@@ -1242,7 +1290,7 @@ sctp_add_chk_to_control(struct sctp_queued_to_read *co
 		control->data = chk->data;
 		sctp_setup_tail_pointer(control);
 	} else {
-		sctp_add_to_tail_pointer(control, chk->data);
+		sctp_add_to_tail_pointer(control, chk->data, &added);
 	}
 	control->fsn_included = chk->rec.data.fsn;
 	asoc->size_on_reasm_queue -= chk->send_size;
@@ -1268,6 +1316,16 @@ sctp_add_chk_to_control(struct sctp_queued_to_read *co
 			} else if (control->on_strm_q == SCTP_ON_ORDERED) {
 				/* Ordered */
 				TAILQ_REMOVE(&strm->inqueue, control, next_instrm);
+				if (asoc->size_on_all_streams >= control->length) {
+					asoc->size_on_all_streams -= control->length;
+				} else {
+#ifdef INVARIANTS
+					panic("size_on_all_streams = %u smaller than control length %u", asoc->size_on_all_streams, control->length);
+#else
+					asoc->size_on_all_streams = 0;
+#endif
+				}
+				sctp_ucount_decr(asoc->cnt_on_all_streams);
 				control->on_strm_q = 0;
 #ifdef INVARIANTS
 			} else if (control->on_strm_q) {
@@ -1283,6 +1341,7 @@ sctp_add_chk_to_control(struct sctp_queued_to_read *co
 		SCTP_INP_READ_UNLOCK(stcb->sctp_ep);
 	}
 	sctp_free_a_chunk(stcb, chk, SCTP_SO_NOT_LOCKED);
+	return (added);
 }
 
 /*
@@ -1302,6 +1361,7 @@ sctp_queue_data_for_reasm(struct sctp_tcb *stcb, struc
 	struct sctp_tmit_chunk *at, *nat;
 	struct sctp_stream_in *strm;
 	int do_wakeup, unordered;
+	uint32_t lenadded;
 
 	strm = &asoc->strmin[control->sinfo_stream];
 	/*
@@ -1314,6 +1374,9 @@ sctp_queue_data_for_reasm(struct sctp_tcb *stcb, struc
 	}
 	/* Must be added to the stream-in queue */
 	if (created_control) {
+		if (unordered == 0) {
+			sctp_ucount_incr(asoc->cnt_on_all_streams);
+		}
 		if (sctp_place_control_in_stream(strm, asoc, control)) {
 			/* Duplicate SSN? */
 			sctp_clean_up_control(stcb, control);
@@ -1373,6 +1436,7 @@ sctp_queue_data_for_reasm(struct sctp_tcb *stcb, struc
 		chk->data = NULL;
 		sctp_free_a_chunk(stcb, chk, SCTP_SO_NOT_LOCKED);
 		sctp_setup_tail_pointer(control);
+		asoc->size_on_all_streams += control->length;
 	} else {
 		/* Place the chunk in our list */
 		int inserted = 0;
@@ -1529,7 +1593,8 @@ sctp_queue_data_for_reasm(struct sctp_tcb *stcb, struc
 				    at->rec.data.fsn,
 				    next_fsn, control->fsn_included);
 				TAILQ_REMOVE(&control->reasm, at, sctp_next);
-				sctp_add_chk_to_control(control, strm, stcb, asoc, at, SCTP_READ_LOCK_NOT_HELD);
+				lenadded = sctp_add_chk_to_control(control, strm, stcb, asoc, at, SCTP_READ_LOCK_NOT_HELD);
+				asoc->size_on_all_streams += lenadded;
 				if (control->on_read_q) {
 					do_wakeup = 1;
 				}
@@ -1600,7 +1665,7 @@ sctp_process_a_data_chunk(struct sctp_tcb *stcb, struc
 	uint16_t sid;
 	struct mbuf *op_err;
 	char msg[SCTP_DIAG_INFO_LEN];
-	struct sctp_queued_to_read *control = NULL;
+	struct sctp_queued_to_read *control, *ncontrol;
 	uint32_t ppid;
 	uint8_t chk_flags;
 	struct sctp_stream_reset_list *liste;
@@ -2006,7 +2071,13 @@ sctp_process_a_data_chunk(struct sctp_tcb *stcb, struc
 			return (0);
 		}
 		if ((chk_flags & SCTP_DATA_NOT_FRAG) == SCTP_DATA_NOT_FRAG) {
+			struct mbuf *m;
+
 			control->data = dmbuf;
+			m = control->data;
+			for (m = control->data; m; m = m->m_next) {
+				control->length += SCTP_BUF_LEN(m);
+			}
 			control->tail_mbuf = NULL;
 			control->end_added = 1;
 			control->last_frag_seen = 1;
@@ -2116,16 +2187,16 @@ sctp_process_a_data_chunk(struct sctp_tcb *stcb, struc
 				/* first one on */
 				TAILQ_INSERT_TAIL(&asoc->pending_reply_queue, control, next);
 			} else {
-				struct sctp_queued_to_read *ctlOn, *nctlOn;
+				struct sctp_queued_to_read *lcontrol, *nlcontrol;
 				unsigned char inserted = 0;
 
-				TAILQ_FOREACH_SAFE(ctlOn, &asoc->pending_reply_queue, next, nctlOn) {
-					if (SCTP_TSN_GT(control->sinfo_tsn, ctlOn->sinfo_tsn)) {
+				TAILQ_FOREACH_SAFE(lcontrol, &asoc->pending_reply_queue, next, nlcontrol) {
+					if (SCTP_TSN_GT(control->sinfo_tsn, lcontrol->sinfo_tsn)) {
 
 						continue;
 					} else {
 						/* found it */
-						TAILQ_INSERT_BEFORE(ctlOn, control, next);
+						TAILQ_INSERT_BEFORE(lcontrol, control, next);
 						inserted = 1;
 						break;
 					}
@@ -2216,8 +2287,6 @@ finish_express_del:
 		 * pending_reply space 3: distribute any chunks in
 		 * pending_reply_queue.
 		 */
-		struct sctp_queued_to_read *ctl, *nctl;
-
 		sctp_reset_in_stream(stcb, liste->number_entries, liste->list_of_streams);
 		TAILQ_REMOVE(&asoc->resetHead, liste, next_resp);
 		sctp_send_deferred_reset_response(stcb, liste, SCTP_STREAM_RESET_RESULT_PERFORMED);
@@ -2226,34 +2295,34 @@ finish_express_del:
 		liste = TAILQ_FIRST(&asoc->resetHead);
 		if (TAILQ_EMPTY(&asoc->resetHead)) {
 			/* All can be removed */
-			TAILQ_FOREACH_SAFE(ctl, &asoc->pending_reply_queue, next, nctl) {
-				TAILQ_REMOVE(&asoc->pending_reply_queue, ctl, next);
-				sctp_queue_data_to_stream(stcb, asoc, ctl, abort_flag, &need_reasm_check);
+			TAILQ_FOREACH_SAFE(control, &asoc->pending_reply_queue, next, ncontrol) {
+				TAILQ_REMOVE(&asoc->pending_reply_queue, control, next);
+				sctp_queue_data_to_stream(stcb, asoc, control, abort_flag, &need_reasm_check);
 				if (*abort_flag) {
 					return (0);
 				}
 				if (need_reasm_check) {
-					(void)sctp_deliver_reasm_check(stcb, asoc, &asoc->strmin[ctl->sinfo_stream], SCTP_READ_LOCK_NOT_HELD);
+					(void)sctp_deliver_reasm_check(stcb, asoc, &asoc->strmin[control->sinfo_stream], SCTP_READ_LOCK_NOT_HELD);
 					need_reasm_check = 0;
 				}
 			}
 		} else {
-			TAILQ_FOREACH_SAFE(ctl, &asoc->pending_reply_queue, next, nctl) {
-				if (SCTP_TSN_GT(ctl->sinfo_tsn, liste->tsn)) {
+			TAILQ_FOREACH_SAFE(control, &asoc->pending_reply_queue, next, ncontrol) {
+				if (SCTP_TSN_GT(control->sinfo_tsn, liste->tsn)) {
 					break;
 				}
 				/*
-				 * if ctl->sinfo_tsn is <= liste->tsn we can
-				 * process it which is the NOT of
-				 * ctl->sinfo_tsn > liste->tsn
+				 * if control->sinfo_tsn is <= liste->tsn we
+				 * can process it which is the NOT of
+				 * control->sinfo_tsn > liste->tsn
 				 */
-				TAILQ_REMOVE(&asoc->pending_reply_queue, ctl, next);
-				sctp_queue_data_to_stream(stcb, asoc, ctl, abort_flag, &need_reasm_check);
+				TAILQ_REMOVE(&asoc->pending_reply_queue, control, next);
+				sctp_queue_data_to_stream(stcb, asoc, control, abort_flag, &need_reasm_check);
 				if (*abort_flag) {
 					return (0);
 				}
 				if (need_reasm_check) {
-					(void)sctp_deliver_reasm_check(stcb, asoc, &asoc->strmin[ctl->sinfo_stream], SCTP_READ_LOCK_NOT_HELD);
+					(void)sctp_deliver_reasm_check(stcb, asoc, &asoc->strmin[control->sinfo_stream], SCTP_READ_LOCK_NOT_HELD);
 					need_reasm_check = 0;
 				}
 			}
@@ -5181,7 +5250,7 @@ static void
 sctp_kick_prsctp_reorder_queue(struct sctp_tcb *stcb,
     struct sctp_stream_in *strmin)
 {
-	struct sctp_queued_to_read *ctl, *nctl;
+	struct sctp_queued_to_read *control, *ncontrol;
 	struct sctp_association *asoc;
 	uint32_t mid;
 	int need_reasm_check = 0;
@@ -5192,43 +5261,51 @@ sctp_kick_prsctp_reorder_queue(struct sctp_tcb *stcb,
 	 * First deliver anything prior to and including the stream no that
 	 * came in.
 	 */
-	TAILQ_FOREACH_SAFE(ctl, &strmin->inqueue, next_instrm, nctl) {
-		if (SCTP_MID_GE(asoc->idata_supported, mid, ctl->mid)) {
+	TAILQ_FOREACH_SAFE(control, &strmin->inqueue, next_instrm, ncontrol) {
+		if (SCTP_MID_GE(asoc->idata_supported, mid, control->mid)) {
 			/* this is deliverable now */
-			if (((ctl->sinfo_flags >> 8) & SCTP_DATA_NOT_FRAG) == SCTP_DATA_NOT_FRAG) {
-				if (ctl->on_strm_q) {
-					if (ctl->on_strm_q == SCTP_ON_ORDERED) {
-						TAILQ_REMOVE(&strmin->inqueue, ctl, next_instrm);
-					} else if (ctl->on_strm_q == SCTP_ON_UNORDERED) {
-						TAILQ_REMOVE(&strmin->uno_inqueue, ctl, next_instrm);
+			if (((control->sinfo_flags >> 8) & SCTP_DATA_NOT_FRAG) == SCTP_DATA_NOT_FRAG) {
+				if (control->on_strm_q) {
+					if (control->on_strm_q == SCTP_ON_ORDERED) {
+						TAILQ_REMOVE(&strmin->inqueue, control, next_instrm);
+					} else if (control->on_strm_q == SCTP_ON_UNORDERED) {
+						TAILQ_REMOVE(&strmin->uno_inqueue, control, next_instrm);
 #ifdef INVARIANTS
 					} else {
 						panic("strmin: %p ctl: %p unknown %d",
-						    strmin, ctl, ctl->on_strm_q);
+						    strmin, control, control->on_strm_q);
 #endif
 					}
-					ctl->on_strm_q = 0;
+					control->on_strm_q = 0;
 				}
 				/* subtract pending on streams */
-				asoc->size_on_all_streams -= ctl->length;
+				if (asoc->size_on_all_streams >= control->length) {
+					asoc->size_on_all_streams -= control->length;
+				} else {
+#ifdef INVARIANTS
+					panic("size_on_all_streams = %u smaller than control length %u", asoc->size_on_all_streams, control->length);
+#else
+					asoc->size_on_all_streams = 0;
+#endif
+				}
 				sctp_ucount_decr(asoc->cnt_on_all_streams);
 				/* deliver it to at least the delivery-q */
 				if (stcb->sctp_socket) {
-					sctp_mark_non_revokable(asoc, ctl->sinfo_tsn);
+					sctp_mark_non_revokable(asoc, control->sinfo_tsn);
 					sctp_add_to_readq(stcb->sctp_ep, stcb,
-					    ctl,
+					    control,
 					    &stcb->sctp_socket->so_rcv,
 					    1, SCTP_READ_LOCK_HELD,
 					    SCTP_SO_NOT_LOCKED);
 				}
 			} else {
 				/* Its a fragmented message */
-				if (ctl->first_frag_seen) {
+				if (control->first_frag_seen) {
 					/*
 					 * Make it so this is next to
 					 * deliver, we restore later
 					 */
-					strmin->last_mid_delivered = ctl->mid - 1;
+					strmin->last_mid_delivered = control->mid - 1;
 					need_reasm_check = 1;
 					break;
 				}
@@ -5257,32 +5334,40 @@ sctp_kick_prsctp_reorder_queue(struct sctp_tcb *stcb,
 	 * now ready.
 	 */
 	mid = strmin->last_mid_delivered + 1;
-	TAILQ_FOREACH_SAFE(ctl, &strmin->inqueue, next_instrm, nctl) {
-		if (SCTP_MID_EQ(asoc->idata_supported, mid, ctl->mid)) {
-			if (((ctl->sinfo_flags >> 8) & SCTP_DATA_NOT_FRAG) == SCTP_DATA_NOT_FRAG) {
+	TAILQ_FOREACH_SAFE(control, &strmin->inqueue, next_instrm, ncontrol) {
+		if (SCTP_MID_EQ(asoc->idata_supported, mid, control->mid)) {
+			if (((control->sinfo_flags >> 8) & SCTP_DATA_NOT_FRAG) == SCTP_DATA_NOT_FRAG) {
 				/* this is deliverable now */
-				if (ctl->on_strm_q) {
-					if (ctl->on_strm_q == SCTP_ON_ORDERED) {
-						TAILQ_REMOVE(&strmin->inqueue, ctl, next_instrm);
-					} else if (ctl->on_strm_q == SCTP_ON_UNORDERED) {
-						TAILQ_REMOVE(&strmin->uno_inqueue, ctl, next_instrm);
+				if (control->on_strm_q) {
+					if (control->on_strm_q == SCTP_ON_ORDERED) {
+						TAILQ_REMOVE(&strmin->inqueue, control, next_instrm);
+					} else if (control->on_strm_q == SCTP_ON_UNORDERED) {
+						TAILQ_REMOVE(&strmin->uno_inqueue, control, next_instrm);
 #ifdef INVARIANTS
 					} else {
 						panic("strmin: %p ctl: %p unknown %d",
-						    strmin, ctl, ctl->on_strm_q);
+						    strmin, control, control->on_strm_q);
 #endif
 					}
-					ctl->on_strm_q = 0;
+					control->on_strm_q = 0;
 				}
 				/* subtract pending on streams */
-				asoc->size_on_all_streams -= ctl->length;
+				if (asoc->size_on_all_streams >= control->length) {
+					asoc->size_on_all_streams -= control->length;
+				} else {
+#ifdef INVARIANTS
+					panic("size_on_all_streams = %u smaller than control length %u", asoc->size_on_all_streams, control->length);
+#else
+					asoc->size_on_all_streams = 0;
+#endif
+				}
 				sctp_ucount_decr(asoc->cnt_on_all_streams);
 				/* deliver it to at least the delivery-q */
-				strmin->last_mid_delivered = ctl->mid;
+				strmin->last_mid_delivered = control->mid;
 				if (stcb->sctp_socket) {
-					sctp_mark_non_revokable(asoc, ctl->sinfo_tsn);
+					sctp_mark_non_revokable(asoc, control->sinfo_tsn);
 					sctp_add_to_readq(stcb->sctp_ep, stcb,
-					    ctl,
+					    control,
 					    &stcb->sctp_socket->so_rcv, 1,
 					    SCTP_READ_LOCK_HELD, SCTP_SO_NOT_LOCKED);
 
@@ -5290,12 +5375,12 @@ sctp_kick_prsctp_reorder_queue(struct sctp_tcb *stcb,
 				mid = strmin->last_mid_delivered + 1;
 			} else {
 				/* Its a fragmented message */
-				if (ctl->first_frag_seen) {
+				if (control->first_frag_seen) {
 					/*
 					 * Make it so this is next to
 					 * deliver
 					 */
-					strmin->last_mid_delivered = ctl->mid - 1;
+					strmin->last_mid_delivered = control->mid - 1;
 					need_reasm_check = 1;
 					break;
 				}
@@ -5347,7 +5432,15 @@ sctp_flush_reassm_for_str_seq(struct sctp_tcb *stcb,
 		}
 		cnt_removed++;
 		TAILQ_REMOVE(&control->reasm, chk, sctp_next);
-		asoc->size_on_reasm_queue -= chk->send_size;
+		if (asoc->size_on_reasm_queue >= chk->send_size) {
+			asoc->size_on_reasm_queue -= chk->send_size;
+		} else {
+#ifdef INVARIANTS
+			panic("size_on_reasm_queue = %u smaller than chunk length %u", asoc->size_on_reasm_queue, chk->send_size);
+#else
+			asoc->size_on_reasm_queue = 0;
+#endif
+		}
 		sctp_ucount_decr(asoc->cnt_on_reasm_queue);
 		if (chk->data) {
 			sctp_m_freem(chk->data);
@@ -5373,6 +5466,16 @@ sctp_flush_reassm_for_str_seq(struct sctp_tcb *stcb,
 	}
 	if (control->on_strm_q == SCTP_ON_ORDERED) {
 		TAILQ_REMOVE(&strm->inqueue, control, next_instrm);
+		if (asoc->size_on_all_streams >= control->length) {
+			asoc->size_on_all_streams -= control->length;
+		} else {
+#ifdef INVARIANTS
+			panic("size_on_all_streams = %u smaller than control length %u", asoc->size_on_all_streams, control->length);
+#else
+			asoc->size_on_all_streams = 0;
+#endif
+		}
+		sctp_ucount_decr(asoc->cnt_on_all_streams);
 		control->on_strm_q = 0;
 	} else if (control->on_strm_q == SCTP_ON_UNORDERED) {
 		TAILQ_REMOVE(&strm->uno_inqueue, control, next_instrm);
@@ -5416,7 +5519,7 @@ sctp_handle_forward_tsn(struct sctp_tcb *stcb,
 	unsigned int i, fwd_sz, m_size;
 	uint32_t str_seq;
 	struct sctp_stream_in *strm;
-	struct sctp_queued_to_read *ctl, *sv;
+	struct sctp_queued_to_read *control, *sv;
 
 	asoc = &stcb->asoc;
 	if ((fwd_sz = ntohs(fwd->ch.chunk_length)) < sizeof(struct sctp_forward_tsn_chunk)) {
@@ -5575,25 +5678,35 @@ sctp_handle_forward_tsn(struct sctp_tcb *stcb,
 			for (cur_mid = strm->last_mid_delivered; SCTP_MID_GE(asoc->idata_supported, mid, cur_mid); cur_mid++) {
 				sctp_flush_reassm_for_str_seq(stcb, asoc, sid, cur_mid, ordered, new_cum_tsn);
 			}
-			TAILQ_FOREACH(ctl, &stcb->sctp_ep->read_queue, next) {
-				if ((ctl->sinfo_stream == sid) &&
-				    (SCTP_MID_EQ(asoc->idata_supported, ctl->mid, mid))) {
+			TAILQ_FOREACH(control, &stcb->sctp_ep->read_queue, next) {
+				if ((control->sinfo_stream == sid) &&
+				    (SCTP_MID_EQ(asoc->idata_supported, control->mid, mid))) {
 					str_seq = (sid << 16) | (0x0000ffff & mid);
-					ctl->pdapi_aborted = 1;
+					control->pdapi_aborted = 1;
 					sv = stcb->asoc.control_pdapi;
-					ctl->end_added = 1;
-					if (ctl->on_strm_q == SCTP_ON_ORDERED) {
-						TAILQ_REMOVE(&strm->inqueue, ctl, next_instrm);
-					} else if (ctl->on_strm_q == SCTP_ON_UNORDERED) {
-						TAILQ_REMOVE(&strm->uno_inqueue, ctl, next_instrm);
+					control->end_added = 1;
+					if (control->on_strm_q == SCTP_ON_ORDERED) {
+						TAILQ_REMOVE(&strm->inqueue, control, next_instrm);
+						if (asoc->size_on_all_streams >= control->length) {
+							asoc->size_on_all_streams -= control->length;
+						} else {
 #ifdef INVARIANTS
-					} else if (ctl->on_strm_q) {
+							panic("size_on_all_streams = %u smaller than control length %u", asoc->size_on_all_streams, control->length);
+#else
+							asoc->size_on_all_streams = 0;
+#endif
+						}
+						sctp_ucount_decr(asoc->cnt_on_all_streams);
+					} else if (control->on_strm_q == SCTP_ON_UNORDERED) {
+						TAILQ_REMOVE(&strm->uno_inqueue, control, next_instrm);
+#ifdef INVARIANTS
+					} else if (control->on_strm_q) {
 						panic("strm: %p ctl: %p unknown %d",
-						    strm, ctl, ctl->on_strm_q);
+						    strm, control, control->on_strm_q);
 #endif
 					}
-					ctl->on_strm_q = 0;
-					stcb->asoc.control_pdapi = ctl;
+					control->on_strm_q = 0;
+					stcb->asoc.control_pdapi = control;
 					sctp_ulp_notify(SCTP_NOTIFY_PARTIAL_DELVIERY_INDICATION,
 					    stcb,
 					    SCTP_PARTIAL_DELIVERY_ABORTED,
@@ -5601,8 +5714,8 @@ sctp_handle_forward_tsn(struct sctp_tcb *stcb,
 					    SCTP_SO_NOT_LOCKED);
 					stcb->asoc.control_pdapi = sv;
 					break;
-				} else if ((ctl->sinfo_stream == sid) &&
-				    SCTP_MID_GT(asoc->idata_supported, ctl->mid, mid)) {
+				} else if ((control->sinfo_stream == sid) &&
+				    SCTP_MID_GT(asoc->idata_supported, control->mid, mid)) {
 					/* We are past our victim SSN */
 					break;
 				}

Modified: head/sys/netinet/sctp_pcb.c
==============================================================================
--- head/sys/netinet/sctp_pcb.c	Wed Jul 19 09:59:32 2017	(r321196)
+++ head/sys/netinet/sctp_pcb.c	Wed Jul 19 12:27:40 2017	(r321197)
@@ -4599,21 +4599,21 @@ void
 sctp_clean_up_stream(struct sctp_tcb *stcb, struct sctp_readhead *rh)
 {
 	struct sctp_tmit_chunk *chk, *nchk;
-	struct sctp_queued_to_read *ctl, *nctl;
+	struct sctp_queued_to_read *control, *ncontrol;
 
-	TAILQ_FOREACH_SAFE(ctl, rh, next_instrm, nctl) {
-		TAILQ_REMOVE(rh, ctl, next_instrm);
-		ctl->on_strm_q = 0;
-		if (ctl->on_read_q == 0) {
-			sctp_free_remote_addr(ctl->whoFrom);
-			if (ctl->data) {
-				sctp_m_freem(ctl->data);
-				ctl->data = NULL;
+	TAILQ_FOREACH_SAFE(control, rh, next_instrm, ncontrol) {
+		TAILQ_REMOVE(rh, control, next_instrm);
+		control->on_strm_q = 0;
+		if (control->on_read_q == 0) {
+			sctp_free_remote_addr(control->whoFrom);
+			if (control->data) {
+				sctp_m_freem(control->data);
+				control->data = NULL;
 			}
 		}
 		/* Reassembly free? */
-		TAILQ_FOREACH_SAFE(chk, &ctl->reasm, sctp_next, nchk) {
-			TAILQ_REMOVE(&ctl->reasm, chk, sctp_next);
+		TAILQ_FOREACH_SAFE(chk, &control->reasm, sctp_next, nchk) {
+			TAILQ_REMOVE(&control->reasm, chk, sctp_next);
 			if (chk->data) {
 				sctp_m_freem(chk->data);
 				chk->data = NULL;
@@ -4629,8 +4629,8 @@ sctp_clean_up_stream(struct sctp_tcb *stcb, struct sct
 		 * We don't free the address here since all the net's were
 		 * freed above.
 		 */
-		if (ctl->on_read_q == 0) {
-			sctp_free_a_readq(stcb, ctl);
+		if (control->on_read_q == 0) {
+			sctp_free_a_readq(stcb, control);
 		}
 	}
 }
@@ -6801,7 +6801,7 @@ sctp_drain_mbufs(struct sctp_tcb *stcb)
 	struct sctp_association *asoc;
 	struct sctp_tmit_chunk *chk, *nchk;
 	uint32_t cumulative_tsn_p1;
-	struct sctp_queued_to_read *ctl, *nctl;
+	struct sctp_queued_to_read *control, *ncontrol;
 	int cnt, strmat;
 	uint32_t gap, i;
 	int fnd = 0;
@@ -6818,88 +6818,124 @@ sctp_drain_mbufs(struct sctp_tcb *stcb)
 	cnt = 0;
 	/* Ok that was fun, now we will drain all the inbound streams? */
 	for (strmat = 0; strmat < asoc->streamincnt; strmat++) {
-		TAILQ_FOREACH_SAFE(ctl, &asoc->strmin[strmat].inqueue, next_instrm, nctl) {
+		TAILQ_FOREACH_SAFE(control, &asoc->strmin[strmat].inqueue, next_instrm, ncontrol) {
 #ifdef INVARIANTS
-			if (ctl->on_strm_q != SCTP_ON_ORDERED) {
+			if (control->on_strm_q != SCTP_ON_ORDERED) {
 				panic("Huh control: %p on_q: %d -- not ordered?",
-				    ctl, ctl->on_strm_q);
+				    control, control->on_strm_q);
 			}
 #endif
-			if (SCTP_TSN_GT(ctl->sinfo_tsn, cumulative_tsn_p1)) {
+			if (SCTP_TSN_GT(control->sinfo_tsn, cumulative_tsn_p1)) {
 				/* Yep it is above cum-ack */
 				cnt++;
-				SCTP_CALC_TSN_TO_GAP(gap, ctl->sinfo_tsn, asoc->mapping_array_base_tsn);
-				asoc->size_on_all_streams = sctp_sbspace_sub(asoc->size_on_all_streams, ctl->length);
+				SCTP_CALC_TSN_TO_GAP(gap, control->sinfo_tsn, asoc->mapping_array_base_tsn);
+				KASSERT(control->length > 0, ("control has zero length"));
+				if (asoc->size_on_all_streams >= control->length) {
+					asoc->size_on_all_streams -= control->length;
+				} else {
+#ifdef INVARIANTS
+					panic("size_on_all_streams = %u smaller than control length %u", asoc->size_on_all_streams, control->length);
+#else
+					asoc->size_on_all_streams = 0;
+#endif
+				}
 				sctp_ucount_decr(asoc->cnt_on_all_streams);
 				SCTP_UNSET_TSN_PRESENT(asoc->mapping_array, gap);
-				if (ctl->on_read_q) {
-					TAILQ_REMOVE(&stcb->sctp_ep->read_queue, ctl, next);
-					ctl->on_read_q = 0;
+				if (control->on_read_q) {
+					TAILQ_REMOVE(&stcb->sctp_ep->read_queue, control, next);
+					control->on_read_q = 0;
 				}
-				TAILQ_REMOVE(&asoc->strmin[strmat].inqueue, ctl, next_instrm);
-				ctl->on_strm_q = 0;
-				if (ctl->data) {
-					sctp_m_freem(ctl->data);
-					ctl->data = NULL;
+				TAILQ_REMOVE(&asoc->strmin[strmat].inqueue, control, next_instrm);
+				control->on_strm_q = 0;
+				if (control->data) {
+					sctp_m_freem(control->data);
+					control->data = NULL;
 				}
-				sctp_free_remote_addr(ctl->whoFrom);
+				sctp_free_remote_addr(control->whoFrom);
 				/* Now its reasm? */
-				TAILQ_FOREACH_SAFE(chk, &ctl->reasm, sctp_next, nchk) {
+				TAILQ_FOREACH_SAFE(chk, &control->reasm, sctp_next, nchk) {
 					cnt++;
 					SCTP_CALC_TSN_TO_GAP(gap, chk->rec.data.tsn, asoc->mapping_array_base_tsn);
-					asoc->size_on_reasm_queue = sctp_sbspace_sub(asoc->size_on_reasm_queue, chk->send_size);
+					KASSERT(chk->send_size > 0, ("chunk has zero length"));
+					if (asoc->size_on_reasm_queue >= chk->send_size) {
+						asoc->size_on_reasm_queue -= chk->send_size;
+					} else {
+#ifdef INVARIANTS
+						panic("size_on_reasm_queue = %u smaller than chunk length %u", asoc->size_on_reasm_queue, chk->send_size);
+#else
+						asoc->size_on_reasm_queue = 0;
+#endif
+					}
 					sctp_ucount_decr(asoc->cnt_on_reasm_queue);
 					SCTP_UNSET_TSN_PRESENT(asoc->mapping_array, gap);
-					TAILQ_REMOVE(&ctl->reasm, chk, sctp_next);
+					TAILQ_REMOVE(&control->reasm, chk, sctp_next);
 					if (chk->data) {
 						sctp_m_freem(chk->data);
 						chk->data = NULL;
 					}
 					sctp_free_a_chunk(stcb, chk, SCTP_SO_NOT_LOCKED);
 				}
-				sctp_free_a_readq(stcb, ctl);
+				sctp_free_a_readq(stcb, control);
 			}
 		}
-		TAILQ_FOREACH_SAFE(ctl, &asoc->strmin[strmat].uno_inqueue, next_instrm, nctl) {
+		TAILQ_FOREACH_SAFE(control, &asoc->strmin[strmat].uno_inqueue, next_instrm, ncontrol) {
 #ifdef INVARIANTS
-			if (ctl->on_strm_q != SCTP_ON_UNORDERED) {
+			if (control->on_strm_q != SCTP_ON_UNORDERED) {
 				panic("Huh control: %p on_q: %d -- not unordered?",
-				    ctl, ctl->on_strm_q);
+				    control, control->on_strm_q);
 			}
 #endif
-			if (SCTP_TSN_GT(ctl->sinfo_tsn, cumulative_tsn_p1)) {
+			if (SCTP_TSN_GT(control->sinfo_tsn, cumulative_tsn_p1)) {
 				/* Yep it is above cum-ack */
 				cnt++;
-				SCTP_CALC_TSN_TO_GAP(gap, ctl->sinfo_tsn, asoc->mapping_array_base_tsn);
-				asoc->size_on_all_streams = sctp_sbspace_sub(asoc->size_on_all_streams, ctl->length);
+				SCTP_CALC_TSN_TO_GAP(gap, control->sinfo_tsn, asoc->mapping_array_base_tsn);
+				KASSERT(control->length > 0, ("control has zero length"));
+				if (asoc->size_on_all_streams >= control->length) {
+					asoc->size_on_all_streams -= control->length;
+				} else {
+#ifdef INVARIANTS
+					panic("size_on_all_streams = %u smaller than control length %u", asoc->size_on_all_streams, control->length);
+#else
+					asoc->size_on_all_streams = 0;
+#endif
+				}
 				sctp_ucount_decr(asoc->cnt_on_all_streams);
 				SCTP_UNSET_TSN_PRESENT(asoc->mapping_array, gap);
-				if (ctl->on_read_q) {
-					TAILQ_REMOVE(&stcb->sctp_ep->read_queue, ctl, next);
-					ctl->on_read_q = 0;
+				if (control->on_read_q) {
+					TAILQ_REMOVE(&stcb->sctp_ep->read_queue, control, next);
+					control->on_read_q = 0;
 				}
-				TAILQ_REMOVE(&asoc->strmin[strmat].uno_inqueue, ctl, next_instrm);
-				ctl->on_strm_q = 0;
-				if (ctl->data) {
-					sctp_m_freem(ctl->data);
-					ctl->data = NULL;
+				TAILQ_REMOVE(&asoc->strmin[strmat].uno_inqueue, control, next_instrm);
+				control->on_strm_q = 0;
+				if (control->data) {
+					sctp_m_freem(control->data);
+					control->data = NULL;
 				}
-				sctp_free_remote_addr(ctl->whoFrom);
+				sctp_free_remote_addr(control->whoFrom);
 				/* Now its reasm? */
-				TAILQ_FOREACH_SAFE(chk, &ctl->reasm, sctp_next, nchk) {
+				TAILQ_FOREACH_SAFE(chk, &control->reasm, sctp_next, nchk) {
 					cnt++;
 					SCTP_CALC_TSN_TO_GAP(gap, chk->rec.data.tsn, asoc->mapping_array_base_tsn);
-					asoc->size_on_reasm_queue = sctp_sbspace_sub(asoc->size_on_reasm_queue, chk->send_size);
+					KASSERT(chk->send_size > 0, ("chunk has zero length"));
+					if (asoc->size_on_reasm_queue >= chk->send_size) {
+						asoc->size_on_reasm_queue -= chk->send_size;
+					} else {
+#ifdef INVARIANTS
+						panic("size_on_reasm_queue = %u smaller than chunk length %u", asoc->size_on_reasm_queue, chk->send_size);
+#else
+						asoc->size_on_reasm_queue = 0;
+#endif
+					}
 					sctp_ucount_decr(asoc->cnt_on_reasm_queue);
 					SCTP_UNSET_TSN_PRESENT(asoc->mapping_array, gap);
-					TAILQ_REMOVE(&ctl->reasm, chk, sctp_next);
+					TAILQ_REMOVE(&control->reasm, chk, sctp_next);
 					if (chk->data) {
 						sctp_m_freem(chk->data);
 						chk->data = NULL;
 					}
 					sctp_free_a_chunk(stcb, chk, SCTP_SO_NOT_LOCKED);
 				}
-				sctp_free_a_readq(stcb, ctl);
+				sctp_free_a_readq(stcb, control);
 			}
 		}
 	}

Modified: head/sys/netinet/sctputil.c
==============================================================================
--- head/sys/netinet/sctputil.c	Wed Jul 19 09:59:32 2017	(r321196)
+++ head/sys/netinet/sctputil.c	Wed Jul 19 12:27:40 2017	(r321197)
@@ -2754,9 +2754,9 @@ sctp_notify_assoc_change(uint16_t state, struct sctp_t
 		    m_notify);
 		if (control != NULL) {
 			control->length = SCTP_BUF_LEN(m_notify);
+			control->spec_flags = M_NOTIFICATION;
 			/* not that we need this */
 			control->tail_mbuf = m_notify;
-			control->spec_flags = M_NOTIFICATION;
 			sctp_add_to_readq(stcb->sctp_ep, stcb,
 			    control,
 			    &stcb->sctp_socket->so_rcv, 1, SCTP_READ_LOCK_NOT_HELD,
@@ -3039,7 +3039,10 @@ sctp_notify_send_failed(struct sctp_tcb *stcb, uint8_t
 		sctp_m_freem(m_notify);
 		return;
 	}
+	control->length = SCTP_BUF_LEN(m_notify);
 	control->spec_flags = M_NOTIFICATION;
+	/* not that we need this */
+	control->tail_mbuf = m_notify;
 	sctp_add_to_readq(stcb->sctp_ep, stcb,
 	    control,
 	    &stcb->sctp_socket->so_rcv, 1,
@@ -3139,7 +3142,10 @@ sctp_notify_send_failed2(struct sctp_tcb *stcb, uint32
 		sctp_m_freem(m_notify);
 		return;
 	}
+	control->length = SCTP_BUF_LEN(m_notify);
 	control->spec_flags = M_NOTIFICATION;
+	/* not that we need this */
+	control->tail_mbuf = m_notify;
 	sctp_add_to_readq(stcb->sctp_ep, stcb,
 	    control,
 	    &stcb->sctp_socket->so_rcv, 1, SCTP_READ_LOCK_NOT_HELD, so_locked);
@@ -3240,12 +3246,10 @@ sctp_notify_partial_delivery_indication(struct sctp_tc
 		sctp_m_freem(m_notify);
 		return;
 	}
-	control->spec_flags = M_NOTIFICATION;
 	control->length = SCTP_BUF_LEN(m_notify);
+	control->spec_flags = M_NOTIFICATION;
 	/* not that we need this */
 	control->tail_mbuf = m_notify;
-	control->held_length = 0;
-	control->length = 0;
 	sb = &stcb->sctp_socket->so_rcv;
 	if (SCTP_BASE_SYSCTL(sctp_logging_level) & SCTP_SB_LOGGING_ENABLE) {
 		sctp_sblog(sb, control->do_not_ref_stcb ? NULL : stcb, SCTP_LOG_SBALLOC, SCTP_BUF_LEN(m_notify));
@@ -3254,7 +3258,6 @@ sctp_notify_partial_delivery_indication(struct sctp_tc
 	if (SCTP_BASE_SYSCTL(sctp_logging_level) & SCTP_SB_LOGGING_ENABLE) {
 		sctp_sblog(sb, control->do_not_ref_stcb ? NULL : stcb, SCTP_LOG_SBRESULT, 0);
 	}
-	atomic_add_int(&control->length, SCTP_BUF_LEN(m_notify));
 	control->end_added = 1;
 	if (stcb->asoc.control_pdapi)
 		TAILQ_INSERT_AFTER(&stcb->sctp_ep->read_queue, stcb->asoc.control_pdapi, control, next);
@@ -3349,8 +3352,8 @@ sctp_notify_shutdown_event(struct sctp_tcb *stcb)
 		sctp_m_freem(m_notify);
 		return;
 	}
-	control->spec_flags = M_NOTIFICATION;
 	control->length = SCTP_BUF_LEN(m_notify);
+	control->spec_flags = M_NOTIFICATION;
 	/* not that we need this */
 	control->tail_mbuf = m_notify;
 	sctp_add_to_readq(stcb->sctp_ep, stcb,
@@ -3456,8 +3459,8 @@ sctp_notify_stream_reset_add(struct sctp_tcb *stcb, ui
 		sctp_m_freem(m_notify);
 		return;
 	}
-	control->spec_flags = M_NOTIFICATION;
 	control->length = SCTP_BUF_LEN(m_notify);
+	control->spec_flags = M_NOTIFICATION;
 	/* not that we need this */
 	control->tail_mbuf = m_notify;
 	sctp_add_to_readq(stcb->sctp_ep, stcb,
@@ -3506,8 +3509,8 @@ sctp_notify_stream_reset_tsn(struct sctp_tcb *stcb, ui
 		sctp_m_freem(m_notify);
 		return;
 	}
-	control->spec_flags = M_NOTIFICATION;
 	control->length = SCTP_BUF_LEN(m_notify);
+	control->spec_flags = M_NOTIFICATION;
 	/* not that we need this */
 	control->tail_mbuf = m_notify;
 	sctp_add_to_readq(stcb->sctp_ep, stcb,
@@ -3571,8 +3574,8 @@ sctp_notify_stream_reset(struct sctp_tcb *stcb,
 		sctp_m_freem(m_notify);
 		return;
 	}
-	control->spec_flags = M_NOTIFICATION;
 	control->length = SCTP_BUF_LEN(m_notify);
+	control->spec_flags = M_NOTIFICATION;
 	/* not that we need this */
 	control->tail_mbuf = m_notify;
 	sctp_add_to_readq(stcb->sctp_ep, stcb,
@@ -3627,9 +3630,9 @@ sctp_notify_remote_error(struct sctp_tcb *stcb, uint16
 	    m_notify);
 	if (control != NULL) {
 		control->length = SCTP_BUF_LEN(m_notify);
+		control->spec_flags = M_NOTIFICATION;
 		/* not that we need this */
 		control->tail_mbuf = m_notify;
-		control->spec_flags = M_NOTIFICATION;
 		sctp_add_to_readq(stcb->sctp_ep, stcb,
 		    control,
 		    &stcb->sctp_socket->so_rcv, 1,



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20170719183850.G1161>