From owner-svn-src-all@FreeBSD.ORG  Fri Feb  4 21:30:28 2011
Return-Path: <owner-svn-src-all@FreeBSD.ORG>
Delivered-To: svn-src-all@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id A7C8710656A8;
	Fri,  4 Feb 2011 21:30:28 +0000 (UTC)
	(envelope-from tuexen@fh-muenster.de)
Received: from mail-n.franken.de (drew.ipv6.franken.de
	[IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa])
	by mx1.freebsd.org (Postfix) with ESMTP id BF4838FC20;
	Fri,  4 Feb 2011 21:30:26 +0000 (UTC)
Received: from [192.168.1.113] (p508FA367.dip.t-dialin.net [80.143.163.103])
	(Authenticated sender: macmic)
	by mail-n.franken.de (Postfix) with ESMTP id D68901C0C0BD8;
	Fri,  4 Feb 2011 22:30:23 +0100 (CET)
Mime-Version: 1.0 (Apple Message framework v1082)
Content-Type: text/plain; charset=us-ascii
From: Michael Tuexen <tuexen@fh-muenster.de>
In-Reply-To: <alpine.BSF.2.00.1102040639440.18938@fledge.watson.org>
Date: Fri, 4 Feb 2011 22:30:22 +0100
Content-Transfer-Encoding: quoted-printable
Message-Id: <FC0EBA69-26C9-450D-9B18-F2B4461364F7@fh-muenster.de>
References: <201102031005.p13A5Vwi040803@svn.freebsd.org>
	<alpine.BSF.2.00.1102040639440.18938@fledge.watson.org>
To: Robert Watson <rwatson@freebsd.org>
X-Mailer: Apple Mail (2.1082)
Cc: svn-src-head@freebsd.org, Randall Stewart <rrs@FreeBSD.org>,
	svn-src-all@freebsd.org, src-committers@freebsd.org
Subject: Re: svn commit: r218211 - in head/sys: conf netinet
X-BeenThere: svn-src-all@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: "SVN commit messages for the entire src tree \(except for &quot;
	user&quot; and &quot; projects&quot; \)" <svn-src-all.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/svn-src-all>,
	<mailto:svn-src-all-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/svn-src-all>
List-Post: <mailto:svn-src-all@freebsd.org>
List-Help: <mailto:svn-src-all-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/svn-src-all>,
	<mailto:svn-src-all-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 04 Feb 2011 21:30:28 -0000

On Feb 4, 2011, at 7:40 AM, Robert Watson wrote:

>=20
> On Thu, 3 Feb 2011, Randall Stewart wrote:
>=20
>> Author: rrs
>> Date: Thu Feb  3 10:05:30 2011
>> New Revision: 218211
>> URL: http://svn.freebsd.org/changeset/base/218211
>>=20
>> Log:
>> Adds an experimental option to create a pool of
>> threads. These serve as input threads and are queued
>> packets based on the V-tag number. This is similar to
>> what a modern card can do with queue's for TCP... but
>> alas modern cards know nothing about SCTP.
>=20
> Hmm.  It might be better to add a new NETISR_SCTP and use netisr's =
support for multithreading?
That sounds really good.

Is it possible that different network cards put packets in the same =
queue?
That would be helpful in the case of SCTP.
>=20
> (I'm preparing a patch for review that enhances that a bit so that =
protocols can be a bit more expressive in terms of specifying dispatch =
policy, etc, currently).
Great!

Best regards
Michael
>=20
> Robert
>=20
>=20
>>=20
>> MFC after:	3 months (maybe)
>>=20
>> Modified:
>> head/sys/conf/options
>> head/sys/netinet/sctp_bsd_addr.c
>> head/sys/netinet/sctp_constants.h
>> head/sys/netinet/sctp_input.c
>> head/sys/netinet/sctp_lock_bsd.h
>> head/sys/netinet/sctp_os_bsd.h
>> head/sys/netinet/sctp_pcb.c
>> head/sys/netinet/sctp_pcb.h
>> head/sys/netinet/sctp_structs.h
>>=20
>> Modified: head/sys/conf/options
>> =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D
>> --- head/sys/conf/options	Thu Feb  3 08:55:45 2011	=
(r218210)
>> +++ head/sys/conf/options	Thu Feb  3 10:05:30 2011	=
(r218211)
>> @@ -439,6 +439,7 @@ SCTP_PACKET_LOGGING	opt_sctp.h # Log to
>> SCTP_LTRACE_CHUNKS	opt_sctp.h # Log to KTR chunks processed
>> SCTP_LTRACE_ERRORS	opt_sctp.h # Log to KTR error returns.
>> SCTP_USE_PERCPU_STAT    opt_sctp.h # Use per cpu stats.
>> +SCTP_MCORE_INPUT        opt_sctp.h # Have multiple input threads for =
input mbufs
>> #
>> #
>> #
>>=20
>> Modified: head/sys/netinet/sctp_bsd_addr.c
>> =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D
>> --- head/sys/netinet/sctp_bsd_addr.c	Thu Feb  3 08:55:45 2011	=
(r218210)
>> +++ head/sys/netinet/sctp_bsd_addr.c	Thu Feb  3 10:05:30 2011	=
(r218211)
>> @@ -68,6 +68,7 @@ MALLOC_DEFINE(SCTP_M_TIMW, "sctp_timw",
>> MALLOC_DEFINE(SCTP_M_MVRF, "sctp_mvrf", "sctp mvrf pcb list");
>> MALLOC_DEFINE(SCTP_M_ITER, "sctp_iter", "sctp iterator control");
>> MALLOC_DEFINE(SCTP_M_SOCKOPT, "sctp_socko", "sctp socket option");
>> +MALLOC_DEFINE(SCTP_M_MCORE, "sctp_mcore", "sctp mcore queue");
>>=20
>> /* Global NON-VNET structure that controls the iterator */
>> struct iterator_control sctp_it_ctl;
>>=20
>> Modified: head/sys/netinet/sctp_constants.h
>> =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D
>> --- head/sys/netinet/sctp_constants.h	Thu Feb  3 08:55:45 2011	=
(r218210)
>> +++ head/sys/netinet/sctp_constants.h	Thu Feb  3 10:05:30 2011	=
(r218211)
>> @@ -91,6 +91,8 @@ __FBSDID("$FreeBSD$");
>> #define SCTP_KTRHEAD_NAME "sctp_iterator"
>> #define SCTP_KTHREAD_PAGES 0
>>=20
>> +#define SCTP_MCORE_NAME "sctp_core_worker"
>> +
>>=20
>> /* If you support Multi-VRF how big to
>> * make the initial array of VRF's to.
>>=20
>> Modified: head/sys/netinet/sctp_input.c
>> =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D
>> --- head/sys/netinet/sctp_input.c	Thu Feb  3 08:55:45 2011	=
(r218210)
>> +++ head/sys/netinet/sctp_input.c	Thu Feb  3 10:05:30 2011	=
(r218211)
>> @@ -48,6 +48,7 @@ __FBSDID("$FreeBSD$");
>> #include <netinet/sctp_timer.h>
>> #include <netinet/sctp_crc32.h>
>> #include <netinet/udp.h>
>> +#include <sys/smp.h>
>>=20
>>=20
>>=20
>> @@ -5921,10 +5922,32 @@ bad:
>> 	}
>> 	return;
>> }
>> +
>> +
>> void
>> -sctp_input(i_pak, off)
>> -	struct mbuf *i_pak;
>> -	int off;
>> +sctp_input(struct mbuf *m, int off)
>> {
>> -	sctp_input_with_port(i_pak, off, 0);
>> +#if defined(__FreeBSD__) && defined(SCTP_MCORE_INPUT) && =
defined(SMP)
>> +	struct ip *ip;
>> +	struct sctphdr *sh;
>> +	int offset;
>> +	int cpu_to_use;
>> +
>> +	if (mp_ncpus > 1) {
>> +		ip =3D mtod(m, struct ip *);
>> +		offset =3D off + sizeof(*sh);
>> +		if (SCTP_BUF_LEN(m) < offset) {
>> +			if ((m =3D m_pullup(m, offset)) =3D=3D 0) {
>> +				SCTP_STAT_INCR(sctps_hdrops);
>> +				return;
>> +			}
>> +			ip =3D mtod(m, struct ip *);
>> +		}
>> +		sh =3D (struct sctphdr *)((caddr_t)ip + off);
>> +		cpu_to_use =3D ntohl(sh->v_tag) % mp_ncpus;
>> +		sctp_queue_to_mcore(m, off, cpu_to_use);
>> +		return;
>> +	}
>> +#endif
>> +	sctp_input_with_port(m, off, 0);
>> }
>>=20
>> Modified: head/sys/netinet/sctp_lock_bsd.h
>> =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D
>> --- head/sys/netinet/sctp_lock_bsd.h	Thu Feb  3 08:55:45 2011	=
(r218210)
>> +++ head/sys/netinet/sctp_lock_bsd.h	Thu Feb  3 10:05:30 2011	=
(r218211)
>> @@ -97,6 +97,48 @@ extern int sctp_logoff_stuff;
>>             rw_rlock(&SCTP_BASE_INFO(ipi_ep_mtx));                    =
     \
>> } while (0)
>>=20
>> +#define SCTP_MCORE_QLOCK_INIT(cpstr) do { \
>> +		mtx_init(&(cpstr)->que_mtx,	      \
>> +			 "sctp-mcore_queue","queue_lock",	\
>> +			 MTX_DEF|MTX_DUPOK);		\
>> +} while (0)
>> +
>> +#define SCTP_MCORE_QLOCK(cpstr)  do { \
>> +		mtx_lock(&(cpstr)->que_mtx);	\
>> +} while (0)
>> +
>> +#define SCTP_MCORE_QUNLOCK(cpstr)  do { \
>> +		mtx_unlock(&(cpstr)->que_mtx);	\
>> +} while (0)
>> +
>> +#define SCTP_MCORE_QDESTROY(cpstr)  do { \
>> +	if(mtx_owned(&(cpstr)->core_mtx)) {	\
>> +		mtx_unlock(&(cpstr)->que_mtx);	\
>> +        } \
>> +	mtx_destroy(&(cpstr)->que_mtx);	\
>> +} while (0)
>> +
>> +
>> +#define SCTP_MCORE_LOCK_INIT(cpstr) do { \
>> +		mtx_init(&(cpstr)->core_mtx,	      \
>> +			 "sctp-cpulck","cpu_proc_lock",	\
>> +			 MTX_DEF|MTX_DUPOK);		\
>> +} while (0)
>> +
>> +#define SCTP_MCORE_LOCK(cpstr)  do { \
>> +		mtx_lock(&(cpstr)->core_mtx);	\
>> +} while (0)
>> +
>> +#define SCTP_MCORE_UNLOCK(cpstr)  do { \
>> +		mtx_unlock(&(cpstr)->core_mtx);	\
>> +} while (0)
>> +
>> +#define SCTP_MCORE_DESTROY(cpstr)  do { \
>> +	if(mtx_owned(&(cpstr)->core_mtx)) {	\
>> +		mtx_unlock(&(cpstr)->core_mtx);	\
>> +        } \
>> +	mtx_destroy(&(cpstr)->core_mtx);	\
>> +} while (0)
>>=20
>> #define SCTP_INP_INFO_WLOCK()	do { 					=
\
>>            rw_wlock(&SCTP_BASE_INFO(ipi_ep_mtx));                     =
    \
>>=20
>> Modified: head/sys/netinet/sctp_os_bsd.h
>> =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D
>> --- head/sys/netinet/sctp_os_bsd.h	Thu Feb  3 08:55:45 2011	=
(r218210)
>> +++ head/sys/netinet/sctp_os_bsd.h	Thu Feb  3 10:05:30 2011	=
(r218211)
>> @@ -123,6 +123,7 @@ MALLOC_DECLARE(SCTP_M_TIMW);
>> MALLOC_DECLARE(SCTP_M_MVRF);
>> MALLOC_DECLARE(SCTP_M_ITER);
>> MALLOC_DECLARE(SCTP_M_SOCKOPT);
>> +MALLOC_DECLARE(SCTP_M_MCORE);
>>=20
>> #if defined(SCTP_LOCAL_TRACE_BUF)
>>=20
>>=20
>> Modified: head/sys/netinet/sctp_pcb.c
>> =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D
>> --- head/sys/netinet/sctp_pcb.c	Thu Feb  3 08:55:45 2011	=
(r218210)
>> +++ head/sys/netinet/sctp_pcb.c	Thu Feb  3 10:05:30 2011	=
(r218211)
>> @@ -47,6 +47,9 @@ __FBSDID("$FreeBSD$");
>> #include <netinet/sctp_bsd_addr.h>
>> #include <netinet/sctp_dtrace_define.h>
>> #include <netinet/udp.h>
>> +#include <sys/sched.h>
>> +#include <sys/smp.h>
>> +#include <sys/unistd.h>
>>=20
>>=20
>> VNET_DEFINE(struct sctp_base_info, system_base_info);
>> @@ -5435,6 +5438,148 @@ sctp_del_local_addr_restricted(struct sc
>> static int sctp_max_number_of_assoc =3D SCTP_MAX_NUM_OF_ASOC;
>> static int sctp_scale_up_for_address =3D SCTP_SCALE_FOR_ADDR;
>>=20
>> +
>> +
>> +#if defined(__FreeBSD__) && defined(SCTP_MCORE_INPUT) && =
defined(SMP)
>> +struct sctp_mcore_ctrl *sctp_mcore_workers =3D NULL;
>> +
>> +void
>> +sctp_queue_to_mcore(struct mbuf *m, int off, int cpu_to_use)
>> +{
>> +	/* Queue a packet to a processor for the specified core */
>> +	struct sctp_mcore_queue *qent;
>> +	struct sctp_mcore_ctrl *wkq;
>> +	int need_wake =3D 0;
>> +
>> +	if (sctp_mcore_workers =3D=3D NULL) {
>> +		/* Something went way bad during setup */
>> +		sctp_input_with_port(m, off, 0);
>> +		return;
>> +	}
>> +	SCTP_MALLOC(qent, struct sctp_mcore_queue *,
>> +	    (sizeof(struct sctp_mcore_queue)),
>> +	    SCTP_M_MCORE);
>> +	if (qent =3D=3D NULL) {
>> +		/* This is trouble  */
>> +		sctp_input_with_port(m, off, 0);
>> +		return;
>> +	}
>> +	qent->vn =3D curvnet;
>> +	qent->m =3D m;
>> +	qent->off =3D off;
>> +	qent->v6 =3D 0;
>> +	wkq =3D &sctp_mcore_workers[cpu_to_use];
>> +	SCTP_MCORE_QLOCK(wkq);
>> +
>> +	TAILQ_INSERT_TAIL(&wkq->que, qent, next);
>> +	if (wkq->running =3D=3D 0) {
>> +		need_wake =3D 1;
>> +	}
>> +	SCTP_MCORE_QUNLOCK(wkq);
>> +	if (need_wake) {
>> +		wakeup(&wkq->running);
>> +	}
>> +}
>> +
>> +static void
>> +sctp_mcore_thread(void *arg)
>> +{
>> +
>> +	struct sctp_mcore_ctrl *wkq;
>> +	struct sctp_mcore_queue *qent;
>> +
>> +	wkq =3D (struct sctp_mcore_ctrl *)arg;
>> +	struct mbuf *m;
>> +	int off, v6;
>> +
>> +	/* Wait for first tickle */
>> +	SCTP_MCORE_LOCK(wkq);
>> +	wkq->running =3D 0;
>> +	msleep(&wkq->running,
>> +	    &wkq->core_mtx,
>> +	    0, "wait for pkt", 0);
>> +	SCTP_MCORE_UNLOCK(wkq);
>> +
>> +	/* Bind to our cpu */
>> +	thread_lock(curthread);
>> +	sched_bind(curthread, wkq->cpuid);
>> +	thread_unlock(curthread);
>> +
>> +	/* Now lets start working */
>> +	SCTP_MCORE_LOCK(wkq);
>> +	/* Now grab lock and go */
>> +	while (1) {
>> +		SCTP_MCORE_QLOCK(wkq);
>> +skip_sleep:
>> +		wkq->running =3D 1;
>> +		qent =3D TAILQ_FIRST(&wkq->que);
>> +		if (qent) {
>> +			TAILQ_REMOVE(&wkq->que, qent, next);
>> +			SCTP_MCORE_QUNLOCK(wkq);
>> +			CURVNET_SET(qent->vn);
>> +			m =3D qent->m;
>> +			off =3D qent->off;
>> +			v6 =3D qent->v6;
>> +			SCTP_FREE(qent, SCTP_M_MCORE);
>> +			if (v6 =3D=3D 0) {
>> +				sctp_input_with_port(m, off, 0);
>> +			} else {
>> +				printf("V6 not yet supported\n");
>> +				sctp_m_freem(m);
>> +			}
>> +			CURVNET_RESTORE();
>> +			SCTP_MCORE_QLOCK(wkq);
>> +		}
>> +		wkq->running =3D 0;
>> +		if (!TAILQ_EMPTY(&wkq->que)) {
>> +			goto skip_sleep;
>> +		}
>> +		SCTP_MCORE_QUNLOCK(wkq);
>> +		msleep(&wkq->running,
>> +		    &wkq->core_mtx,
>> +		    0, "wait for pkt", 0);
>> +	};
>> +}
>> +
>> +static void
>> +sctp_startup_mcore_threads(void)
>> +{
>> +	int i;
>> +
>> +	if (mp_ncpus =3D=3D 1)
>> +		return;
>> +
>> +	SCTP_MALLOC(sctp_mcore_workers, struct sctp_mcore_ctrl *,
>> +	    (mp_ncpus * sizeof(struct sctp_mcore_ctrl)),
>> +	    SCTP_M_MCORE);
>> +	if (sctp_mcore_workers =3D=3D NULL) {
>> +		/* TSNH I hope */
>> +		return;
>> +	}
>> +	memset(sctp_mcore_workers, 0, (mp_ncpus *
>> +	    sizeof(struct sctp_mcore_ctrl)));
>> +	/* Init the structures */
>> +	for (i =3D 0; i < mp_ncpus; i++) {
>> +		TAILQ_INIT(&sctp_mcore_workers[i].que);
>> +		SCTP_MCORE_LOCK_INIT(&sctp_mcore_workers[i]);
>> +		SCTP_MCORE_QLOCK_INIT(&sctp_mcore_workers[i]);
>> +		sctp_mcore_workers[i].cpuid =3D i;
>> +	}
>> +	/* Now start them all */
>> +	for (i =3D 0; i < mp_ncpus; i++) {
>> +		(void)kproc_create(sctp_mcore_thread,
>> +		    (void *)&sctp_mcore_workers[i],
>> +		    &sctp_mcore_workers[i].thread_proc,
>> +		    RFPROC,
>> +		    SCTP_KTHREAD_PAGES,
>> +		    SCTP_MCORE_NAME);
>> +
>> +	}
>> +}
>> +
>> +#endif
>> +
>> +
>> void
>> sctp_pcb_init()
>> {
>> @@ -5565,6 +5710,10 @@ sctp_pcb_init()
>>=20
>> 	sctp_startup_iterator();
>>=20
>> +#if defined(__FreeBSD__) && defined(SCTP_MCORE_INPUT) && =
defined(SMP)
>> +	sctp_startup_mcore_threads();
>> +#endif
>> +
>> 	/*
>> 	 * INIT the default VRF which for BSD is the only one, other =
O/S's
>> 	 * may have more. But initially they must start with one and =
then
>>=20
>> Modified: head/sys/netinet/sctp_pcb.h
>> =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D
>> --- head/sys/netinet/sctp_pcb.h	Thu Feb  3 08:55:45 2011	=
(r218210)
>> +++ head/sys/netinet/sctp_pcb.h	Thu Feb  3 10:05:30 2011	=
(r218211)
>> @@ -624,6 +624,12 @@ sctp_initiate_iterator(inp_func inpf,
>>    struct sctp_inpcb *,
>>    uint8_t co_off);
>>=20
>> +#if defined(__FreeBSD__) && defined(SCTP_MCORE_INPUT) && =
defined(SMP)
>> +void
>> +     sctp_queue_to_mcore(struct mbuf *m, int off, int cpu_to_use);
>> +
>> +#endif
>> +
>> #ifdef INVARIANTS
>> void
>>     sctp_validate_no_locks(struct sctp_inpcb *inp);
>>=20
>> Modified: head/sys/netinet/sctp_structs.h
>> =
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D
>> --- head/sys/netinet/sctp_structs.h	Thu Feb  3 08:55:45 2011	=
(r218210)
>> +++ head/sys/netinet/sctp_structs.h	Thu Feb  3 10:05:30 2011	=
(r218211)
>> @@ -106,6 +106,31 @@ typedef void (*asoc_func) (struct sctp_i
>> typedef int (*inp_func) (struct sctp_inpcb *, void *ptr, uint32_t =
val);
>> typedef void (*end_func) (void *ptr, uint32_t val);
>>=20
>> +#if defined(__FreeBSD__) && defined(SCTP_MCORE_INPUT) && =
defined(SMP)
>> +/* whats on the mcore control struct */
>> +struct sctp_mcore_queue {
>> +	TAILQ_ENTRY(sctp_mcore_queue) next;
>> +	struct vnet *vn;
>> +	struct mbuf *m;
>> +	int off;
>> +	int v6;
>> +};
>> +
>> +TAILQ_HEAD(sctp_mcore_qhead, sctp_mcore_queue);
>> +
>> +struct sctp_mcore_ctrl {
>> +	SCTP_PROCESS_STRUCT thread_proc;
>> +	struct sctp_mcore_qhead que;
>> +	struct mtx core_mtx;
>> +	struct mtx que_mtx;
>> +	int running;
>> +	int cpuid;
>> +};
>> +
>> +
>> +#endif
>> +
>> +
>> struct sctp_iterator {
>> 	TAILQ_ENTRY(sctp_iterator) sctp_nxt_itr;
>> 	struct vnet *vn;
>>=20
>=20