Date: Wed, 25 Jan 2017 03:09:53 +1100 (EST) From: Bruce Evans <brde@optusnet.com.au> To: Sean Bruno <sbruno@freebsd.org> Cc: =?UTF-8?Q?Olivier_Cochard-Labb=c3=a9?= <olivier@freebsd.org>, "freebsd-net@freebsd.org" <freebsd-net@freebsd.org>, "freebsd-current@freebsd.org" <freebsd-current@freebsd.org>, Matthew Macy <mmacy@nextbsd.org> Subject: Re: HEADS-UP: IFLIB implementations of sys/dev/e1000 em, lem, igb pending Message-ID: <20170125023853.Q1809@besplex.bde.org> In-Reply-To: <b2714d3a-75f7-4959-e390-8ade11d77962@freebsd.org> References: <30f21c75-d3a2-edcd-1999-d5ed9f970c06@freebsd.org> <574a7ac7-4842-9518-8286-a4d89a9f7a27@freebsd.org> <CA%2Bq%2BTco-dcoU8EZnDEzgoK-v2Q2=U5GF6ASMSj0kwzd_wB5xig@mail.gmail.com> <6c6cb534-73c7-464b-8af1-7445a9c0188c@freebsd.org> <1598f29d379.ea6360351471.8752933472741761813@nextbsd.org> <CA%2Bq%2BTcpUXXPEQtdMFup6EZzyCKs9Ep%2BnS5SB%2Bfm6bSJSDs34_w@mail.gmail.com> <1598f3f8588.d20017893749.339651164872952258@nextbsd.org> <1598f42ad77.eeec05be4113.9201780237587761460@nextbsd.org> <CA%2Bq%2BTcp5LwrnXt75tNpYpAr1KWx9YpLx5kMHhPR%2BYgAs__n1eA@mail.gmail.com> <159902b73ed.10775291e21533.7488368455500235608@nextbsd.org> <CA%2Bq%2BTcpHmuOGyp5A290WmUvGTnOSse7v8gj4=R8kZ=m51-_s4A@mail.gmail.com> <18abdd64-08a6-50ca-fb6b-9c01a3d7b60c@freebsd.org> <CA%2Bq%2BTcptEN5pcScYo4j3O8OuJHEacZu9ugOz_6b2iFb-CzBFXA@mail.gmail.com> <ad7fdc31-b0dd-2105-1610-cf0f3de42245@freebsd.org> <CA%2Bq%2BTcpcZUJTmycPBF9kS-x4qcqN7V3LdHW=BQ3ttzXLyU3FWw@mail.gmail.com> <b2714d3a-75f7-4959-e390-8ade11d77962@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, 24 Jan 2017, Sean Bruno wrote:
> On 01/24/17 08:27, Olivier Cochard-Labb=C3=A9 wrote:
>> On Tue, Jan 24, 2017 at 3:17 PM, Sean Bruno <sbruno@freebsd.org
>> <mailto:sbruno@freebsd.org>> wrote:
>>
>> Did you increase the number of rx/tx rings to 8 and the number of
>> descriptors to 4k in your tests or just the defaults?
>>
>> Tuning are same as described in my previous email (rxd|txd=3D2048, rx|tx
>> process_limit=3D-1, max_interrupt_rate=3D16000).
>> [root@apu2]~# sysctl hw.igb.
>> hw.igb.tx_process_limit: -1
>> hw.igb.rx_process_limit: -1
>> hw.igb.num_queues: 0
>> hw.igb.header_split: 0
>> hw.igb.max_interrupt_rate: 16000
>> hw.igb.enable_msix: 1
>> hw.igb.enable_aim: 1
>> hw.igb.txd: 2048
>> hw.igb.rxd: 2048
>
> Oh, I think you missed my note on these. In order to adjust txd/rxd you
> need to tweak the iflib version of these numbers. nrxds/ntxds should be
> adjust upwards to your value of 2048. nrxqs/ntxqs should be adjust
> upwards to 8, I think, so you can test equivalent settings to the legacy
> driver.
>
> Specifically, you may want to adjust these:
>
> dev.em.0.iflib.override_nrxds: 0
> dev.em.0.iflib.override_ntxds: 0
>
> dev.em.0.iflib.override_nrxqs: 0
> dev.em.0.iflib.override_ntxqs: 0
That is painful.
My hack to increase the ifq length also no longer works:
X Index: if_em.c
X =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
X --- if_em.c=09(revision 312696)
X +++ if_em.c=09(working copy)
X @@ -1,3 +1,5 @@
X +int em_qlenadj =3D -1;
X +
-1 gives a null adjustment; 0 gives a default (very large ifq), and other
values give a non-null adustment.
X /*-
X * Copyright (c) 2016 Matt Macy <mmacy@nextbsd.org>
X * All rights reserved.
X @@ -2488,7 +2490,10 @@
X=20
X =09/* Single Queue */
X if (adapter->tx_num_queues =3D=3D 1) {
X -=09 if_setsendqlen(ifp, scctx->isc_ntxd[0] - 1);
X +=09 if (em_qlenadj =3D=3D 0)
X +=09 em_qlenadj =3D imax(2 * tick, 0) * 15 / 10;
X +=09 // lem_qlenadj =3D imax(2 * tick, 0) * 42 / 100;
X +=09 if_setsendqlen(ifp, scctx->isc_ntxd[0] + em_qlenadj);
X =09 if_setsendqready(ifp);
X =09}
X
I don't want larger hardware queues, but sometimes want larger software
queues. ifq's used to give them. The if_setsenqlen() call is still there.
but no longer gives them.
The large queues are needed for backet blasting benchmarks since select()
doesn't work for udp sockets, so if the queues fill up then the benchmarks
must busy-wait or sleep waiting for them to drain, and timeout granularity
tends to prevent short sleeps from working so the queues run dry while
sleeping unless the queues are very large.
Bruce
From owner-freebsd-current@freebsd.org Tue Jan 24 18:32:32 2017
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
[IPv6:2001:1900:2254:206a::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id C135ECC07EA
for <freebsd-current@mailman.ysv.freebsd.org>;
Tue, 24 Jan 2017 18:32:32 +0000 (UTC)
(envelope-from anindya49@hotmail.com)
Received: from SNT004-OMC4S6.hotmail.com (snt004-omc4s6.hotmail.com
[65.55.90.209])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits))
(Client CN "*.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 83861AE8
for <freebsd-current@freebsd.org>; Tue, 24 Jan 2017 18:32:32 +0000 (UTC)
(envelope-from anindya49@hotmail.com)
Received: from NAM04-SN1-obe.outbound.protection.outlook.com ([65.55.90.199])
by SNT004-OMC4S6.hotmail.com over TLS secured channel with Microsoft
SMTPSVC(7.5.7601.23008); Tue, 24 Jan 2017 10:31:24 -0800
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com;
s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
bh=5gPWfAV1ZRDqbbOHwIzUAWCHPrlzwQEB9Wit3eR69/4=;
b=bzUDXuQVGE11rVv5FFV2ofRBrB8uaTyDDF4V+6i9MS5Jn4lF+9hkB3I8vhE8T+oNzjlFrg8uOjDBPaqXf8McZUyhhKodIRkpjzkFP3P59WBljC3VbAVen7qn9F6y6xUvK+E+Mj/TMjvhrbGPMOlD0nfCLP4O710TIY23cqXsK1u+YqkovGwk8by3hhpWGqSH723HddEb+1vahzGOcNe0+nDb9Q6vUA+dEnE7P9YhT8fbLRpYGT+cvx/NOspOMG0HxVcrH1CfJSt+hZW4FZrkJHG/WKQPSDzxaNKM8NIjtA+POU2xV5m8ZrENSFB/yWeMUxtbuOCyBv63nXGgYXqffg==
Received: from CO1NAM04FT022.eop-NAM04.prod.protection.outlook.com
(10.152.90.57) by CO1NAM04HT194.eop-NAM04.prod.protection.outlook.com
(10.152.90.198) with Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.803.8; Tue, 24 Jan
2017 18:31:24 +0000
Received: from BN6PR22MB0802.namprd22.prod.outlook.com (10.152.90.56) by
CO1NAM04FT022.mail.protection.outlook.com (10.152.90.167) with Microsoft SMTP
Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id
15.1.803.8 via Frontend Transport; Tue, 24 Jan 2017 18:31:24 +0000
Received: from BN6PR22MB0802.namprd22.prod.outlook.com ([10.171.180.21]) by
BN6PR22MB0802.namprd22.prod.outlook.com ([10.171.180.21]) with mapi id
15.01.0860.021; Tue, 24 Jan 2017 18:31:24 +0000
From: Anindya Mukherjee <anindya49@hotmail.com>
To: Adrian Chadd <adrian.chadd@gmail.com>
CC: "freebsd-current@freebsd.org" <freebsd-current@freebsd.org>
Subject: RE: vt(4) chops off the leftmost three columns
Thread-Topic: vt(4) chops off the leftmost three columns
Thread-Index: AQHScpOc+dBATW1RlESizy/O7gA13qFB//UAgAX2e4k=
Date: Tue, 24 Jan 2017 18:31:23 +0000
Message-ID: <BN6PR22MB08024232C8CE4B3A05D0469CB6750@BN6PR22MB0802.namprd22.prod.outlook.com>
References: <BN6PR22MB0802DB657D97A1F6C8EB3F34B67E0@BN6PR22MB0802.namprd22.prod.outlook.com>,
<CAJ-Vmo=bRJvkX9JM0CmLcQexpPF+6X8GDB3urCTjAAS6Tzro1Q@mail.gmail.com>
In-Reply-To: <CAJ-Vmo=bRJvkX9JM0CmLcQexpPF+6X8GDB3urCTjAAS6Tzro1Q@mail.gmail.com>
Accept-Language: en-CA, en-US
Content-Language: en-CA
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: gmail.com; dkim=none (message not signed)
header.d=none;gmail.com; dmarc=none action=none header.from=hotmail.com;
x-incomingtopheadermarker: OriginalChecksum:BC693119B414D6BE428BAAE180743E3389722DE572E7D1793FF0FBD70CC7697D;
UpperCasedChecksum:2E79FC1AF5437021799E665BAFC2D779914FFDB0B4F5B03F10AF1374BE917C9E;
SizeAsReceived:7817; Count:40
x-ms-exchange-messagesentrepresentingtype: 1
x-tmn: [0Z3Mw/oDgRykQapTxd9POMABz/VGGNOd]
x-incomingheadercount: 40
x-eopattributedmessage: 0
x-microsoft-exchange-diagnostics: 1; CO1NAM04HT194;
7:S3/GZbRVMP+cDzyiaGjjN6EcR5bi641SunmFmZkzzc27pxBmSLI3zyXr7weWamjjc/ZvFZw75vyi0Wdn1sxxA9UVT5xaSEx0ir3gXqzYtqviBwIBvGpCj1PqX3g79IHozTJNbRh9NK5PKaeSegoQJVkdMrut1mSAjxXz5wq5ysd9ZQqx1KbRF/zArlv/m46ki3Vb8FgxhjlzInf+LRsYbx2Wdll0tjVy/x4bjG2rZQ71fb4bjAbTnrJSMa9gmFxX+ppRQ1uZDRYwNU/psfNp/p3+p4ZHhCSngb2sDoRvG6cd9GCD+XUq8fo3pE5X+i5Aq6k/ClCUXqKkSsQxRvw26P2OJ8ykHQRMTIX4Oul/wvgwXLfqtUMK7Cff7hWLJEAZ5T4C/ntpqYOpUPEzGgBMLQCqce2S3cSYNQ7W6awB5qlT/+wrJvMC5fVd6ldwXoieLMrK5jprv93h7rMbtburpg==
x-forefront-antispam-report: EFV:NLI; SFV:NSPM; SFS:(10019020)(98900005);
DIR:OUT; SFP:1102; SCL:1; SRVR:CO1NAM04HT194;
H:BN6PR22MB0802.namprd22.prod.outlook.com; FPR:; SPF:None; LANG:en;
x-ms-office365-filtering-correlation-id: 93a6e286-68b7-4e1e-7a71-08d44487338b
x-microsoft-antispam: UriScan:; BCL:0; PCL:0;
RULEID:(22001)(1601124038)(5061506344)(5061507293)(1603103113)(1603101340)(1601125047)(1701031023);
SRVR:CO1NAM04HT194;
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0;
RULEID:(444111334)(444112120)(432015012)(82015046); SRVR:CO1NAM04HT194; BCL:0;
PCL:0; RULEID:; SRVR:CO1NAM04HT194;
x-forefront-prvs: 0197AFBD92
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: hotmail.com
X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Jan 2017 18:31:23.8902 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Internet
X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1NAM04HT194
X-OriginalArrivalTime: 24 Jan 2017 18:31:24.0982 (UTC)
FILETIME=[1189BD60:01D27670]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.23
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
<freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>,
<mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>;
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>,
<mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 24 Jan 2017 18:32:32 -0000
Hi again,
I have been switching back and forth between exploring the code and reading=
J. Kong's Device Drivers book. As you explained, the first thing to tackle=
is the coupling between vesa and sc. Right now I can't even load vesa unle=
ss the kern.vty=3Dsc, and then I cannot unload it.
vesa depends on vesa.c and pulls in scvesactl.c for two functions: vesa_loa=
d_ioctl() and vesa_unload_ioctl(). Although these are used only in vesa.c, =
they depend on several structures (for example the virtual screen structure=
scr_stat, and others) defined in syscons.h, which makes these two systems =
coupled.
I was wondering what would be a good way to decouple these dependencies. I =
want to do the right thing even if it takes longer. Really appreciate you t=
aking the time to respond, thanks!
Anindya
________________________________________
From: Adrian Chadd [adrian.chadd@gmail.com]
Sent: January 20, 2017 3:11 PM
To: Anindya Mukherjee
Cc: freebsd-current@freebsd.org
Subject: Re: vt(4) chops off the leftmost three columns
hiya,
Mechanically it doesn't look /that/ hard:
* vesa.ko pulls in the vesa.c bits and the syscons vesa control bits.
Ideally we'd have them as two separate modules, so you could load
"vesa" without needing the syscons bits.
* Maybe then write a vt 'fb' interface to talk to the old-school
framebuffer interface
* Then (if we're lucky) we can have vt use the same VGA, VESA, (mach,
creator, etc!) through the fb interface, rather than reimplementing
its own.
I looked at it and it doesn't look /that/ hard. If you only cared
about vesa, then you could do something like what 'creator' and
'creator_vt' did in sys/dev/fb/ . It's just sad that the vt interface
to the screen buffer isn't as complete as the older school framebuffer
interface is.
-adrian
On 19 January 2017 at 12:35, Anindya Mukherjee <anindya49@hotmail.com> wrot=
e:
> Hi Adrian,
>
> I was looking at the source for the vt driver. Wondering how much work it=
is to add VESA support to the VGA backend? As you say ATM it's hardcoded t=
o use 640x480. Pardon my ignorance, but can we reuse any VESA code from sys=
cons?
>
> Also, how dependent is splash/screensaver support on the VESA implementat=
ion?
>
> Thanks,
> Anindya
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20170125023853.Q1809>