Date: Tue, 19 Apr 2011 21:06:02 +0200 From: "K. Macy" <kmacy@freebsd.org> To: Freddie Cash <fjwcash@gmail.com> Cc: Nikolay Denev <ndenev@gmail.com>, Ingo Flaschberger <if@freebsd.org>, freebsd-net@freebsd.org Subject: Re: Routing enhancement - reduce routing table locking Message-ID: <BANLkTinvc%2BQitiBoi7f3agiHMdbCWdPvWQ@mail.gmail.com> In-Reply-To: <BANLkTimEvou2f-b8ggLpVDh6RVnnva=wUQ@mail.gmail.com> References: <alpine.LRH.2.00.1104050303140.2152@filebunker.xip.at> <alpine.LRH.2.00.1104061426350.2152@filebunker.xip.at> <alpine.LRH.2.00.1104180051450.8693@filebunker.xip.at> <BANLkTik39HvVire6Hzi9U6J2BwKV7apCCg@mail.gmail.com> <alpine.LRH.2.00.1104181852420.8693@filebunker.xip.at> <BANLkTim0hoHDnrweYz%2Bvc7zOvMubddJmGg@mail.gmail.com> <BANLkTim6HMGibDB4ucs%2BtEfqv-LBnF4O-w@mail.gmail.com> <E4328014-47D0-4EFC-8C70-5D28823179C8@gmail.com> <BANLkTi=q4d58LbeEo3=J8U7vAuXSmouEQA@mail.gmail.com> <BANLkTimEvou2f-b8ggLpVDh6RVnnva=wUQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Apr 19, 2011 at 8:19 PM, Freddie Cash <fjwcash@gmail.com> wrote:
> On Tue, Apr 19, 2011 at 7:42 AM, K. Macy <kmacy@freebsd.org> wrote:
>>> I'm not able to find IFNET_MULTIQUEUE in a recent 8.2-STABLE, is this s=
omething
>>> present only in HEAD?
>>
>> It looks like it is now EM_MULTIQUEUE.
>
> Just curious, how would one enable this to test it? =A0We have igb(4)
> interfaces in our new storage boxes, and it would be interesting to
> test whether or not it helps in our setup.
>
It should automatically allocate a queue per core up to the max
supported. Post 8.0 it should be enabled by default for igb:
#if __FreeBSD_version >=3D 800000
/*
** Multiqueue Transmit driver
**
*/
static int
igb_mq_start(struct ifnet *ifp, struct mbuf *m)
{
struct adapter *adapter =3D ifp->if_softc;
struct igb_queue *que;
struct tx_ring *txr;
int i =3D 0, err =3D 0;
/* Which queue to use */
if ((m->m_flags & M_FLOWID) !=3D 0)
i =3D m->m_pkthdr.flowid % adapter->num_queues;
txr =3D &adapter->tx_rings[i];
que =3D &adapter->queues[i];
if (IGB_TX_TRYLOCK(txr)) {
err =3D igb_mq_start_locked(ifp, txr, m);
IGB_TX_UNLOCK(txr);
} else {
err =3D drbr_enqueue(ifp, txr->br, m);
taskqueue_enqueue(que->tq, &que->que_task);
}
return (err);
}
static int
igb_mq_start_locked(struct ifnet *ifp, struct tx_ring *txr, struct mbuf *m)
{
struct adapter *adapter =3D txr->adapter;
struct mbuf *next;
int err =3D 0, enq;
IGB_TX_LOCK_ASSERT(txr);
if ((ifp->if_drv_flags & (IFF_DRV_RUNNING | IFF_DRV_OACTIVE)) !=3D
IFF_DRV_RUNNING || adapter->link_active =3D=3D 0) {
if (m !=3D NULL)
err =3D drbr_enqueue(ifp, txr->br, m);
return (err);
}
/* Call cleanup if number of TX descriptors low */
if (txr->tx_avail <=3D IGB_TX_CLEANUP_THRESHOLD)
igb_txeof(txr);
enq =3D 0;
if (m =3D=3D NULL) {
next =3D drbr_dequeue(ifp, txr->br);
} else if (drbr_needs_enqueue(ifp, txr->br)) {
if ((err =3D drbr_enqueue(ifp, txr->br, m)) !=3D 0)
return (err);
next =3D drbr_dequeue(ifp, txr->br);
} else
next =3D m;
/* Process the queue */
while (next !=3D NULL) {
if ((err =3D igb_xmit(txr, &next)) !=3D 0) {
if (next !=3D NULL)
err =3D drbr_enqueue(ifp, txr->br, next);
break;
}
enq++;
drbr_stats_update(ifp, next->m_pkthdr.len, next->m_flags);
ETHER_BPF_MTAP(ifp, next);
if ((ifp->if_drv_flags & IFF_DRV_RUNNING) =3D=3D 0)
break;
if (txr->tx_avail <=3D IGB_TX_OP_THRESHOLD) {
ifp->if_drv_flags |=3D IFF_DRV_OACTIVE;
break;
}
next =3D drbr_dequeue(ifp, txr->br);
}
if (enq > 0) {
/* Set the watchdog */
txr->queue_status =3D IGB_QUEUE_WORKING;
txr->watchdog_time =3D ticks;
}
return (err);
}
I haven't tested this to make sure there aren't any hidden locking
performance issues.
Cheers
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?BANLkTinvc%2BQitiBoi7f3agiHMdbCWdPvWQ>