Date: Tue, 29 Nov 2016 11:26:50 +0000 From: Steven Crangle <Steven@stream-technologies.com> To: Vincenzo Maffione <v.maffione@gmail.com> Cc: FreeBSD Net <freebsd-net@freebsd.org> Subject: Re: Looking for some help with netmap/bhyve Message-ID: <DB5PR07MB168582129D05D52878D3DCE59B8D0@DB5PR07MB1685.eurprd07.prod.outlook.com> In-Reply-To: <CA%2B_eA9gggyo_ncSZDribOr%2BsWoFELbPKEeLdaZ8AZwgHAYjcRA@mail.gmail.com> References: <DB5PR07MB1685DAA40193595950A464FB9BB40@DB5PR07MB1685.eurprd07.prod.outlook.com>, <CA%2B_eA9gggyo_ncSZDribOr%2BsWoFELbPKEeLdaZ8AZwgHAYjcRA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--_004_DB5PR07MB168582129D05D52878D3DCE59B8D0DB5PR07MB1685eurp_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Vincenzo! Thank you so much for your help! With your instructions I've managed to get= a set of bhyves up and running with netmap working properly within them! I= chose to go with the recent version of head instead of 10.3. I will also = try and build a version on 11.0 if I get chance! I will make sure and ask any further questions on the github page for netma= p, but while I'm emailing, I've attached a text file containing a few kerne= l page fault errors I've ran into. They don't happen repeatedly. They seem = to just happen randomly if I try to shut the process down and possibly catc= h netmap in an incorrect state! Figured they might be useful to you. If I f= igure out what the error is myself I will try and help out! And thanks again for your help! Kind Regards ________________________________ [Stream Logo] Steven Crangle Systems Developer | Stream Technologies | Glasgow, UK [http://www.stream-technologies.com/img/phone.png] +44 (0)844 800 8520 | [http://www.stream-technologies.com/img/mouse.png] www.stream-technologies.com<http://www.stream-technologies.com/> ________________________________ From: Vincenzo Maffione <v.maffione@gmail.com> Sent: 23 November 2016 08:33:11 To: Steven Crangle Cc: FreeBSD Net Subject: Re: Looking for some help with netmap/bhyve 2016-11-22 11:31 GMT+01:00 Steven Crangle <Steven@stream-technologies.com<m= ailto:Steven@stream-technologies.com>>: Hi, I've recently been trying to boot up several bhyves so that I can test netm= ap communication between instances. The problem is, no matter what configur= ation I try, the guest vm running in bhyve completely hangs and becomes unu= sable as soon as a packet hits the netmap interface. When testing with pkt-= gen, the TX side successfully starts sending packets, but the RX side will = reliable freeze with the only option being killing the bhyve process. The bhyve command used for the above test was: bhyve -c 1 -s 0,hostbridge -s 1,lpc -s 2,virtio-blk,/dev/zvol/zroot/vip= er1vol -s 3,virtio-net,tap0,mac=3D00:01:23:45:67:83 -s 4,virtio-net,tap4= -l com1,/dev/nmdm0A -A -H -P -m 6g viper1 & bhyve -c 1 -s 0,hostbridge -s 1,lpc -s 2,virtio-blk,/dev/zvol/zroot/vip= er2vol -s 3,virtio-net,tap1,mac=3D00:01:23:45:67:84 -s 4,virtio-net,tap5= -l com1,/dev/nmdm1A -A -H -P -m 6g viper2 For this test the host OS was FreeBSD-11.0-p3 and the guest OS was FreeBSD-= 11.0-p3. After failing to get this solution working, I pulled down the source from t= he following url and installed it on the host box: https://svnweb.freebsd.org/socsvn/soc2016/vincenzo/head/ I then ran the following commands to try and bring up the machines using th= e ptnetmap interface (the guest still running 11.0p3) : bhyve -c 1 -s 0,hostbridge -s 1,lpc -s 1:1,virtio-blk,/dev/zvol/zroot/v= iper1vol -s 2:0,virtio-net,tap0,mac=3D00:01:23:45:67:83 -s 2:1,ptnetmap-= memdev -s 2:2,ptnet,vale0:0 -l com1,/dev/nmdm0A -A -H -P -m 6g viper1 & bhyve -c 1 -s 0,hostbridge -s 1,lpc -s 1:1,virtio-blk,/dev/zvol/zroot/v= iper2vol -s 2:0,virtio-net,tap1,mac=3D00:01:23:45:67:84 -s 2:1,ptnetmap-= memdev -s 2:2,ptnet,vale0:1 -l com1,/dev/nmdm1A -A -H -P -m 6g viper2 With the above commands the vm's fail to boot with the following message: ptnet_init: failed to get ptnetmap Output in /var/log/messages seems to just show the ptnetmap driver allocati= ng one RX/TX ring for each vm, while bringing the device up, the device the= n goes down and the above error is seen in the console. Is there something I'm doing wrong with regards to running netmap or ptnetm= ap within a bhyve? Any pointers in the right direction will be much appreci= ated! Kind Regards Steven Hi Steven, The code you are looking at is the final code released by my gsoc 2016 pr= oject at the end of August 2016. However, I've been working on that for a w= hile after the gsoc, so that code is not updated anymore. My modification b= asically involves two subsystems: netmap and bhyve. The updates to netmap are already available in HEAD. The updates to bhyve a= re not yet upstream, as we are in the process to review that with the bhyve= maintainers. Anyway, you need two sources to get the latest code: 1) https://github.com/luigirizzo/netmap for the latest netmap code, that yo= u could also compile as an external kernel module. 2) https://github.com/vmaffione/freebsd to get the updates to bhyve (and vm= m.ko). You can use this as the host system. There are two branches here: pt= net-10.3, and ptnet-head. My original code was developed under FreeBSD 10.3= , so a first possibility is to try this as the host system (using the ptnet= -10.3 branch). The other branch (ptnet-head) contains a porting of the work= to a recent version of HEAD. I would be very glad if you could test the co= de also on FreeBSD 11. We support two combinations of bhyve networking with netmap: (A) virtio-net + netmap: that is something like "-s 2:0,virtio-net,vale0:= 0" in the bhyve command line. (B) ptnet + ptnetmap: that is something like "-s 2:1,ptnetmap-memdev -s 2= :2,ptnet,vale0:0" in the bhyve command line. so you may want to try A first (netmap backend in user-space, slower) and t= hen B (netmap backend in kernel-space, faster). Sorry about the confusion on the code repositories, I'll try also to update= the wiki page (and/or the gsoc svn repo) to reflect these updates. It's pe= rfectly ok for me to discuss these issues here in the ML, however for more = detailed/low-level discussion and support about problems you are running in= to, feel free to open Github issues here https://github.com/luigirizzo/netm= ap. Cheers, Vincenzo ________________________________ [Stream Logo] Steven Crangle Systems Developer | Stream Technologies | Glasgow, UK [http://www.stream-technologies.com/img/phone.png] +44 (0)844 800 8520<tel:%2B44%20%280%29844%20800%208520> | [http://www.stream-technologies.com/img/mouse.png] www.stream-technologies.com<http://www.stream-technologies.com><http://www.= stream-technologies.com/> _______________________________________________ freebsd-net@freebsd.org<mailto:freebsd-net@freebsd.org> mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org<mailt= o:freebsd-net-unsubscribe@freebsd.org>" -- Vincenzo Maffione --_004_DB5PR07MB168582129D05D52878D3DCE59B8D0DB5PR07MB1685eurp_ Content-Type: text/plain; name="kernel_stacktraces.txt" Content-Description: kernel_stacktraces.txt Content-Disposition: attachment; filename="kernel_stacktraces.txt"; size=1049; creation-date="Tue, 29 Nov 2016 11:25:17 GMT"; modification-date="Tue, 29 Nov 2016 11:25:17 GMT" Content-Transfer-Encoding: base64 CkZhdGFsIHRyYXAgMTI6IHBhZ2UgZmF1bHQgd2hpbGUgaW4ga2VybmVsIG1vZGUKY3B1aWQgPSAw OyBhcGljIGlkID0gMDAKZmF1bHQgdmlydHVhbCBhZGRyZXNzCT0gMHgwCmZhdWx0IGNvZGUJCT0g c3VwZXJ2aXNvciByZWFkIGluc3RydWN0aW9uLCBwYWdlIG5vdCBwcmVzZW50Cmluc3RydWN0aW9u IHBvaW50ZXIJPSAweDIwOjB4MApzdGFjayBwb2ludGVyCSAgICAgICAgPSAweDI4OjB4ZmZmZmZl MDFhNGQ2ZDkzOApmcmFtZSBwb2ludGVyCSAgICAgICAgPSAweDI4OjB4ZmZmZmZlMDFhNGQ2ZDk2 MApjb2RlIHNlZ21lbnQJCT0gYmFzZSAweDAsIGxpbWl0IDB4ZmZmZmYsIHR5cGUgMHgxYgoJCQk9 IERQTCAwLCBwcmVzIDEsIGxvbmcgMSwgZGVmMzIgMCwgZ3JhbiAxCnByb2Nlc3NvciBlZmxhZ3MJ PSBpbnRlcnJ1cHQgZW5hYmxlZCwgcmVzdW1lLCBJT1BMID0gMApjdXJyZW50IHByb2Nlc3MJCT0g MTIgKHN3aTQ6IGNsb2NrICgwKSkKWyB0aHJlYWQgcGlkIDEyIHRpZCAxMDAwMDUgXQpTdG9wcGVk IGF0ICAgICAgMAoKCkZhdGFsIHRyYXAgMTI6IHBhZ2UgZmF1bHQgd2hpbGUgaW4ga2VybmVsIG1v ZGUKY3B1aWQgPSAwOyBhcGljIGlkID0gMDAKZmF1bHQgdmlydHVhbCBhZGRyZXNzCT0gMHgwCmZh dWx0IGNvZGUJCT0gc3VwZXJ2aXNvciByZWFkIGluc3RydWN0aW9uLCBwYWdlIG5vdCBwcmVzZW50 Cmluc3RydWN0aW9uIHBvaW50ZXIJPSAweDIwOjB4MApzdGFjayBwb2ludGVyCSAgICAgICAgPSAw eDA6MHhmZmZmZmUwMWE0ZDZkOTM4CmZyYW1lIHBvaW50ZXIJICAgICAgICA9IDB4MDoweGZmZmZm ZTAxYTRkNmQ5NjAKY29kZSBzZWdtZW50CQk9IGJhc2UgMHgwLCBsaW1pdCAweGZmZmZmLCB0eXBl IDB4MWIKCQkJPSBEUEwgMCwgcHJlcyAxLCBsb25nIDEsIGRlZjMyIDAsIGdyYW4gMQpwcm9jZXNz b3IgZWZsYWdzCT0gaW50ZXJydXB0IGVuYWJsZWQsIHJlc3VtZSwgSU9QTCA9IDAKY3VycmVudCBw cm9jZXNzCQk9IDEyIChzd2k0OiBjbG9jayAoMCkpClsgdGhyZWFkIHBpZCAxMiB0aWQgMTAwMDA1 IF0KU3RvcHBlZCBhdCAgICAgIDAKCgo= --_004_DB5PR07MB168582129D05D52878D3DCE59B8D0DB5PR07MB1685eurp_--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?DB5PR07MB168582129D05D52878D3DCE59B8D0>