From owner-freebsd-virtualization@freebsd.org Sun Apr 16 05:15:11 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 73DC2D407CD for ; Sun, 16 Apr 2017 05:15:11 +0000 (UTC) (envelope-from akgupt3@gmail.com) Received: from mail-qk0-x230.google.com (mail-qk0-x230.google.com [IPv6:2607:f8b0:400d:c09::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 40B247B1; Sun, 16 Apr 2017 05:15:11 +0000 (UTC) (envelope-from akgupt3@gmail.com) Received: by mail-qk0-x230.google.com with SMTP id p68so88078617qke.1; Sat, 15 Apr 2017 22:15:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=QoAA3os2QK6L5GpE5M33vqCse5lhcyyydBT+Q5Tsu2I=; b=ahcsRXpWIax9nibPix/guGxYJ8x1Y7kF5kw58gjhtwQpWwQGL+yQq93o3jO4ERU5V+ X7iIj7S4bFXoKC+8jG0LmyF/Sh985QfUuWPajHOI39uawoiiDh5/1rH+iQHxRlQ1jYYO mkHdeE1YbXG9yVuEprj40JXI4sNwDe/8HUHN1zsh+tyXl+W27KyiL9F9MCdwHYg5fnAA a9ZpN8o1mjocwVJm3ey4rehdK0qgDkjd0FXPAFSJAcVomv5A/5F++A/guTaQFYECpfQs 3OF4RsrZE+/7PgJ/JawlG7n/n+egQsc1brMT3HdeLrZTDSgorzbehBXf/WYmSYmIqQkV mt8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=QoAA3os2QK6L5GpE5M33vqCse5lhcyyydBT+Q5Tsu2I=; b=jsjPAq+GTGUlVoAcPfDzz2HzpVWAZNJQvVorR7Dw7eSpbipK8RWRcybuz7LY5v1KRY pK+Z2eC09dXg5lEQN+tTRkqXNVGAOoJ9oyWhsmIRdJwAD3OTPN+WWW4DCdD2cfNtBxFn UR1xYvXl4D4d2Jks9T+ugUOgnnLI8yJShk5YiXLacUCKEvH4/1JZVpvmVJ83icbADncM o70zE//NaP8C/g0kjR4c56gbXKSkaTWjKDybNzDaeJ5t8OEjVxWC/CLf8447g0wB/1Cw tkaagMxUmo60EFsVFiPFWM1BEv43bWX+SQ2r8XsMRIYk9s7Q0036utw+VAOsptLBB4xl Dwng== X-Gm-Message-State: AN3rC/56UYgSzcAwy+MsvY8NkvaotkSVA9aK+1jzYzq9apH2iqoxG2t1 xvhxOOBHe1rm/OIoodBHjQulJ8DLZ3t1 X-Received: by 10.55.17.149 with SMTP id 21mr4311534qkr.246.1492319710148; Sat, 15 Apr 2017 22:15:10 -0700 (PDT) MIME-Version: 1.0 Received: by 10.12.178.73 with HTTP; Sat, 15 Apr 2017 22:15:09 -0700 (PDT) In-Reply-To: References: From: Anish Date: Sat, 15 Apr 2017 22:15:09 -0700 Message-ID: Subject: Re: debugging FreeBSD BHyve guest To: Julian Elischer Cc: "freebsd-virtualization@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Apr 2017 05:15:11 -0000 I just tested bvmdebug with FreeBSD HEAD and it worked fine. -Anish *./vmrun.sh -g 6466 vm1* ** *root@bsdguest:~ # uname -a* *FreeBSD bsdguest 12.0-CURRENT FreeBSD 12.0-CURRENT #1 r314641M: Sat Apr 15 19:53:23 PDT 2017 root@svmhost:/usr/obj/root/head/sys/BHYVE amd64* *root@bsdguest:~ # sysctl debug.kdb.enter=1* *debug.kdb.enter: 0KDB: enter: sysctl debug.kdb.enter* *Received ^C; trying to switch back to ddb.* *using longjmp, hope it works!* *KDB: reentering* *KDB: stack backtrace:* *...* *Switching to ddb back-end* *[ thread pid 782 tid 100072 ]* *Stopped at kdb_sysctl_enter+0x98: movq $0,kdb_why* *db> gdb* *(ctrl-c will return control to ddb)* *Switching to gdb back-end* ----------------------------------------- >From other terminal (kgdb) target remote localhost:6466 ... (kgdb) bt #0 0xffffffff80a8c6b8 in kdb_sysctl_current (oidp=0x0, arg1=0x80, arg2=-2198296484032, req=0xfffffe002b51a930) at /root/FreeBSD/base/head/sys/kern/subr_kdb.c:146 #1 0xffffffff80a57820 in sysctl_new_user (req=0x100000000, p=0xfffffe002b51a930, l=18446741875413067952) at /root/FreeBSD/base/head/sys/kern/kern_sysctl.c:1729 #2 0xffffffff80a57016 in sysctl_root (arg1=0x0, arg2=-2120631584) at /root/FreeBSD/base/head/sys/kern/kern_sysctl.c:1911 #3 0xffffffff80a575a8 in sys___sysctl (td=, uap=0x3) at /root/FreeBSD/base/head/sys/kern/kern_sysctl.c:1944 #4 0xffffffff80a5741f in sysctl_old_user (req=, p=, l=) at /root/FreeBSD/base/head/sys/kern/kern_sysctl.c:1718 #5 0xffffffff80ec901a in trap (frame=0x0) at /root/FreeBSD/base/head/sys/amd64/amd64/trap.c:569 #6 0xffffffff80eaab1b in Xalign () at /root/FreeBSD/base/head/sys/amd64/amd64/exception.S:164 #7 0x000000080097cb6a in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) On Wed, Apr 12, 2017 at 10:14 AM, Julian Elischer wrote: > On 11/4/17 9:30 pm, Hannes Mehnert wrote: > >> Hello, >> >> I want to debug a FreeBSD BHyve guest kernel, and tried to follow the >> instructions on the wiki >> (https://wiki.freebsd.org/bhyve/DebuggingWithGdb) -- but already the >> "device bvmdebug" is no longer recognized by make buildkernel. >> > > I hope that is not true. I use it all the time on 10. > > > >> Is there a way (and a howto?) to kgdb a FreeBSD guest? >> >> >> Thanks, >> >> hannes >> _______________________________________________ >> freebsd-virtualization@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >> To unsubscribe, send any mail to "freebsd-virtualization-unsubs >> cribe@freebsd.org" >> >> > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubs > cribe@freebsd.org" > From owner-freebsd-virtualization@freebsd.org Sun Apr 16 21:00:42 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D1F14D41DEB for ; Sun, 16 Apr 2017 21:00:42 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AFF6219C9 for ; Sun, 16 Apr 2017 21:00:42 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v3GL01kN022636 for ; Sun, 16 Apr 2017 21:00:42 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <201704162100.v3GL01kN022636@kenobi.freebsd.org> From: bugzilla-noreply@FreeBSD.org To: freebsd-virtualization@FreeBSD.org Subject: Problem reports for freebsd-virtualization@FreeBSD.org that need special attention Date: Sun, 16 Apr 2017 21:00:42 +0000 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Apr 2017 21:00:42 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- New | 212820 | FreeBSD 10-STABLE from latest HEAD and 11-RELEASE 1 problems total for which you should take action. From owner-freebsd-virtualization@freebsd.org Mon Apr 17 16:34:34 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DDEDBD4132A for ; Mon, 17 Apr 2017 16:34:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C45DF110F for ; Mon, 17 Apr 2017 16:34:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v3HGYYNf084288 for ; Mon, 17 Apr 2017 16:34:34 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 215972] Bhyve crash more then 1 cpu AMD Date: Mon, 17 Apr 2017 16:34:34 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: jesper@monsted.dk X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Apr 2017 16:34:35 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D215972 jesper@monsted.dk changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jesper@monsted.dk --- Comment #13 from jesper@monsted.dk --- I appear to be running into the same problem with different circumstances. I am running a Windows 2012R2 VM with a little help from chyves. It works perfectly well for a 3-4 days, idling at about 1% CPU on my Xeon E5-2630v3. Then, the VM goes unresponsive and bhyve starts consuming ~100% of a core, regardless of the number of vCPUs assigned to the VM. I've tested this with both one and four cores assigned to the VM. One crash filled the screen with "atkbd data buffer full", but most don't. = The VNC console is blank and unresponsive. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Platform: =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D White box server Motherboard: Asrock X99/Extreme4 CPU: Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz (2399.35-MHz K8-class CPU) RAM: 8x 16 GB ECC (128GB) Intel NIC, IBM/LSI HBA, a few other odds and ends i doubt would make much difference =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D top reports one of 16 (8, hyperthreaded) cores in use: =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D last pid: 41496; load averages: 1.12, 1.13, 1.09=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20 up 44+19:40:00 17:55:06 59 processes: 1 running, 58 sleeping CPU: 0.0% user, 0.0% nice, 6.2% system, 0.0% interrupt, 93.8% idle Mem: 12M Active, 1281M Inact, 121G Wired, 2684M Free ARC: 92G Total, 39G MFU, 50G MRU, 300K Anon, 787M Header, 2850M Other Swap: PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COM= MAND 29470 root 23 20 0 17482M 6750M kqread 3 21.9H 101.21% bhy= ve =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D root@chef:~ # uname -a FreeBSD chef.bofh 11.0-STABLE FreeBSD 11.0-STABLE #0: Fri Mar 3 04:28:46 C= ET 2017 root@chef.bofh:/usr/obj/usr/src/sys/CHEF amd64 root@chef:~ # chyves ike get all Getting all ike's properties... bargs -A -H -P -S bhyve_disk_type ahci-hd bhyve_net_type e1000 bhyveload_flags chyves_guest_version 0300 cpu 4 creation Created on Fri Mar 24 20:26:53 CET 201= 7 by chyves v0.2.0 2016/09/11 using __create() description - eject_iso_on_n_reboot 3 loader uefi net_ifaces tap51 notes - os windows ram 16G rcboot 0 revert_to_snapshot revert_to_snapshot_method off serial nmdm51 template no uefi_console_output vnc uefi_firmware BHYVE_UEFI.fd uefi_vnc_client print uefi_vnc_client_custom_cmd uefi_vnc_ip 0.0.0.0 uefi_vnc_mouse_type usb3 uefi_vnc_pause_until_client_connect no uefi_vnc_port 5901 uefi_vnc_res 800x600 uuid d5302114-10c7-11e7-91c6-d05099803cdc =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D I get the same kdump output as Nils Beyer: =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 29470 vcpu 1 CALL ioctl(0x3,0xc0907601,0x7fffdd9eae30) 29470 vcpu 2 CALL ioctl(0x3,0xc0907601,0x7fffdd7e9e30) 29470 vcpu 3 CALL ioctl(0x3,0xc0907601,0x7fffdd5e8e30) 29470 vcpu 1 RET ioctl 0 29470 vcpu 3 RET ioctl 0 29470 vcpu 1 CALL ioctl(0x3,0xc0907601,0x7fffdd9eae30) 29470 vcpu 3 CALL ioctl(0x3,0xc0907601,0x7fffdd5e8e30) 29470 vcpu 2 RET ioctl 0 29470 vcpu 2 CALL ioctl(0x3,0xc0907601,0x7fffdd7e9e30) 29470 vcpu 1 RET ioctl 0 29470 vcpu 1 CALL ioctl(0x3,0xc0907601,0x7fffdd9eae30) 29470 vcpu 3 RET ioctl 0 29470 vcpu 3 CALL ioctl(0x3,0xc0907601,0x7fffdd5e8e30) 29470 vcpu 2 RET ioctl 0 29470 vcpu 2 CALL ioctl(0x3,0xc0907601,0x7fffdd7e9e30) 29470 vcpu 0 RET ioctl 0 29470 vcpu 0 CALL ioctl(0x3,0xc0907601,0x7fffddbebe30) --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Mon Apr 17 18:35:55 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F01ABD42503 for ; Mon, 17 Apr 2017 18:35:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DFAA61ED6 for ; Mon, 17 Apr 2017 18:35:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v3HIZtZW026769 for ; Mon, 17 Apr 2017 18:35:55 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 218715] e1000 emulation in bhyve will lock up under Windows guests Date: Mon, 17 Apr 2017 18:35:55 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: bhyve X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: grehan@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status keywords bug_severity priority component assigned_to reporter Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Apr 2017 18:35:56 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D218715 Bug ID: 218715 Summary: e1000 emulation in bhyve will lock up under Windows guests Product: Base System Version: CURRENT Hardware: amd64 OS: Any Status: New Keywords: bhyve Severity: Affects Some People Priority: --- Component: bin Assignee: freebsd-virtualization@FreeBSD.org Reporter: grehan@FreeBSD.org When using the e1000 emulation in bhyve with Windows guests, it's possible = to get a lockup where a single vCPU spins at 100%. This can be reproduced within 10 minutes or so by running iperf3 on Windows= to the host. In normal use, the lockup can take hours or even weeks, but seems= to eventually occur. A workaround is to use the virtio network drivers on Windows with the virtio-net emulation on bhyve. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Mon Apr 17 18:37:19 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EDC81D42584 for ; Mon, 17 Apr 2017 18:37:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DD76A1F50 for ; Mon, 17 Apr 2017 18:37:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v3HIbJ7k028399 for ; Mon, 17 Apr 2017 18:37:19 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 215972] Bhyve crash more then 1 cpu AMD Date: Mon, 17 Apr 2017 18:37:19 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: grehan@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Apr 2017 18:37:20 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D215972 --- Comment #14 from Peter Grehan --- >bhyve_net_type e1000 The lockup you are seeing is unrelated to the AMD one, and is a known one = with the e1000 under Windows. I've created 218715 to track the e1000 issue. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Tue Apr 18 17:47:04 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 85D29D44E00 for ; Tue, 18 Apr 2017 17:47:04 +0000 (UTC) (envelope-from vhpc.dist@gmail.com) Received: from mail-oi0-x242.google.com (mail-oi0-x242.google.com [IPv6:2607:f8b0:4003:c06::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 51C601C29 for ; Tue, 18 Apr 2017 17:47:04 +0000 (UTC) (envelope-from vhpc.dist@gmail.com) Received: by mail-oi0-x242.google.com with SMTP id t14so39667oif.1 for ; Tue, 18 Apr 2017 10:47:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=S5ro2EgFJyDIkn8pVRVpUv82ys0ezdjrGIGDZp5MujQ=; b=eX4XauBR4K94il0dsRwCw+M0s8DM7maI1sszmvXi/8E4dze23DtiqYKEaVXGAEtYo8 E41gmXWjMTj9oNvZsTZhSMajJmTJUSH24ZvhPOlwVLQb3OuGkMbmZTkbKWKRUh7GW8Kk uIRw0bYTDrHhSsF7sJGmHyT/K20eItIWSjLXhxzKU29LedhjfGs22R8v4SKNOd9UXDNm 4Nawf4PcjR4//3xB4m5CjolUUXRR3ohinYx/cd6JE0ShHNGP6Gu4PxVEUTtr3tW6ahMn 1BFMFn9ECBJcWEDq78s4s5pojl/PgQ0s3Dt2bwDP0nzKPuRBkO2A/OLBYS8WpCpPZrTZ KkBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=S5ro2EgFJyDIkn8pVRVpUv82ys0ezdjrGIGDZp5MujQ=; b=oHbbIkIsaAMatDiqe4blmKf3g982B1LJAuqYLkgwOcjy7SUJQUDnYRpy6QAtwKFQvX tzQuyWJxDCLIT66qMcR4gAT03u8E1zv2V532X13jeHCn8Ykr/vQnJ0L1cw67p17FVM08 GVW4+/L28C3p5uv+2t79lW6d6K7fkAAVdr2nEkkNBwFXSZc97lJYLF57a3y/IX10/iUu tOAy1K9AUZBG8unIL1RjiyZoIh7IE8p06kiop2GeG6xiCzOytapib3UTSxkvu5/h0uQz bOtGpmWCT3IdM4cr0dClOZ/UNRPaIGsKtqc8ZO47pTwlpUKxUI6QQWglhChk9J2Y/uXl wv3w== X-Gm-Message-State: AN3rC/6wPaCnirpFPd999iNiEDjLYvXfZyLa5ljDjKoL7eUttfKJ+dac 3+H87dQ4JCZivTpmESgbMgCGMxHvP2Ztr80= X-Received: by 10.202.220.3 with SMTP id t3mr8342316oig.121.1492537623113; Tue, 18 Apr 2017 10:47:03 -0700 (PDT) MIME-Version: 1.0 Received: by 10.202.237.7 with HTTP; Tue, 18 Apr 2017 10:47:01 -0700 (PDT) From: VHPC 17 Date: Tue, 18 Apr 2017 19:47:01 +0200 Message-ID: Subject: VHPC at ISC extension - Papers due May 2 To: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Apr 2017 17:47:04 -0000 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D CALL FOR PAPERS 12th Workshop on Virtualization in High=C2=AD-Performance Cloud Computing = (VHPC '17) held in conjunction with the International Supercomputing Conference - High Performance, June 18-22, 2017, Frankfurt, Germany. (Springer LNCS Proceedings) =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Date: June 22, 2017 Workshop URL: http://vhpc.org Paper Submission Deadline: May 2, 2017 (extended), Springer LNCS, rolling abstract submission Abstract/Paper Submission Link: https://edas.info/newPaper.php?c=3D23179 Keynotes: Satoshi Matsuoka, Professor of Computer Science, Tokyo Institute of Technology and John Goodacre, Professor in Computer Architectures University of Manchester, Director of Technology and Systems ARM Ltd. Research Group and Chief Scientific Officer Kaleao Ltd. Call for Papers Virtualization technologies constitute a key enabling factor for flexible resource management in modern data centers, and particularly in cloud environments. Cloud providers need to manage complex infrastructures in a seamless fashion to support the highly dynamic and heterogeneous workloads and hosted applications customers deploy. Similarly, HPC environments have been increasingly adopting techniques that enable flexible management of vast computing and networking resources, close to marginal provisioning cost, which is unprecedented in the history of scientific and commercial computing. Various virtualization technologies contribute to the overall picture in different ways: machine virtualization, with its capability to enable consolidation of multiple under=C2=ADutilized servers with heterogeneous software and operating systems (OSes), and its capability to live=C2=AD-migrate a fully operating virtual machine (VM) with a very short downtime, enables novel and dynamic ways to manage physical servers; OS-=C2=ADlevel virtualization (i.e., containerization), with its capability to isolate multiple user=C2=AD-space environments and to allow f= or their co=C2=ADexistence within the same OS kernel, promises to provide many of the advantages of machine virtualization with high levels of responsiveness and performance; I/O Virtualization allows physical NICs/HBAs to take traffic from multiple VMs or containers; network virtualization, with its capability to create logical network overlays that are independent of the underlying physical topology and IP addressing, provides the fundamental ground on top of which evolved network services can be realized with an unprecedented level of dynamicity and flexibility; the increasingly adopted paradigm of Software-=C2=ADDefined Networking (SDN) promises to extend this flexibility to the control and data planes of network paths. Publication Accepted papers will be published in a Springer LNCS proceedings volume. Topics of Interest The VHPC program committee solicits original, high-quality submissions related to virtualization across the entire software stack with a special focus on the intersection of HPC and the cloud. Major Topics - Virtualization in supercomputing environments, HPC clusters, HPC in the cloud and grids - OS-level virtualization and containers (Docker, rkt, Singularity, Shifter, i.a.) - Lightweight/specialized operating systems, unikernels - Optimizations of virtual machine monitor platforms and hypervisors - Hypervisor support for heterogenous resources (GPUs, co-processors, FPGAs, etc.) - Virtualization support for emerging memory technologies - Virtualization in enterprise HPC and microvisors - Software defined networks and network virtualization - Management, deployment of virtualized environments and orchestration (Kubernetes i.a.), - Workflow-pipeline container-based composability - Performance measurement, modelling and monitoring of virtualized/cloud workloads - Virtualization in data intensive computing and Big Data processing - HPC convergence - Adaptation of HPC technologies in the cloud (high performance networks, RDMA, etc.) - ARM-based hypervisors, ARM virtualization extensions - I/O virtualization and cloud based storage systems - GPU, FPGA and many-core accelerator virtualization - Job scheduling/control/policy and container placement in virtualized environments - Cloud reliability, fault-tolerance and high-availability - QoS and SLA in virtualized environments - IaaS platforms, cloud frameworks and APIs - Large-scale virtualization in domains such as finance and government - Energy-efficient and power-aware virtualization - Container security - Configuration management tools for containers (including CFEngine, Puppet, i.a.) - Emerging topics including multi-kernel approaches and,NUMA in hypervisors The Workshop on Virtualization in High=C2=AD-Performance Cloud Computing (V= HPC) aims to bring together researchers and industrial practitioners facing the challenges posed by virtualization in order to foster discussion, collaboration, mutual exchange of knowledge and experience, enabling research to ultimately provide novel solutions for virtualized computing systems of tomorrow. The workshop will be one day in length, composed of 20 min paper presentations, each followed by 10 min discussion sections, plus lightning talks that are limited to 5 minutes. Presentations may be accompanied by interactive demonstrations. Important Dates Rolling Abstract Submission May 2, 2017 - Paper submission deadline (extended) May 30, 2017 - Acceptance notification June 22, 2017 - Workshop Day July 18, 2017 - Camera-ready version due Chair Michael Alexander (chair), scaledinfra technologies, Austria Anastassios Nanos (co-=C2=ADchair), NTUA, Greece Balazs Gerofi (co-=C2=ADchair), =E2=80=8BRIKEN Advanced Institute for Compu= tational Science=E2=80=8B, Japan Program committee Stergios Anastasiadis, University of Ioannina, Greece Jakob Blomer, CERN, Europe Ron Brightwell, Sandia National Laboratories, USA Eduardo C=C3=A9sar, Universidad Autonoma de Barcelona, Spain Julian Chesterfield, OnApp, UK Stephen Crago, USC ISI, USA Christoffer Dall, Columbia University, USA Patrick Dreher, MIT, USA Robert Futrick, Cycle Computing, USA Maria Girone, CERN, Europe Kyle Hale, Northwestern University, USA Romeo Kinzler, IBM, Switzerland Brian Kocoloski, University of Pittsburgh, USA Nectarios Koziris, National Technical University of Athens, Greece John Lange, University of Pittsburgh, USA Che-Rung Lee, National Tsing Hua University, Taiwan Giuseppe Lettieri, University of Pisa, Italy Qing Liu, Oak Ridge National Laboratory, USA Nikos Parlavantzas, IRISA, France Kevin Pedretti, Sandia National Laboratories, USA Amer Qouneh, University of Florida, USA Carlos Rea=C3=B1o, Technical University of Valencia, Spain Thomas Ryd, CFEngine, Norway Na Zhang, VMWare, USA Borja Sotomayor, University of Chicago, USA Craig Stewart, Indiana University, USA Anata Tiwari, San Diego Supercomputer Center, USA Kurt Tutschku, Blekinge Institute of Technology, Sweden Yasuhiro Watashiba, Osaka University, Japan Nicholas Wright, Lawrence Berkeley National Laboratory, USA Chao-Tung Yang, Tunghai University, Taiwan Paper Submission-Publication Papers submitted to the workshop will be reviewed by at least two members of the program committee and external reviewers. Submissions should include abstract, key words, the e-mail address of the corresponding author, and must not exceed 10 pages, including tables and figures at a main font size no smaller than 11 point. Submission of a paper should be regarded as a commitment that, should the paper be accepted, at least one of the authors will register and attend the conference to present the work. Accepted papers will be published in a Springer LNCS volume. . The format must be according to the Springer LNCS Style. Initial submissions are in PDF; authors of accepted papers will be requested to provide source files. Format Guidelines: ftp://ftp.springer.de/pub/tex/latex/llncs/latex2e/llncs2e.zip Abstract, Paper Submission Link: https://edas.info/newPaper.php?c=3D23179 Lightning Talks Lightning Talks are non-paper track, synoptical in nature and are strictly limited to 5 minutes. They can be used to gain early feedback on ongoing research, for demonstrations, to present research results, early research ideas, perspectives and positions of interest to the community. Submit abstract via the main submission link. General Information The workshop is one day in length and will be held in conjunction with the International Supercomputing Conference - High Performance (ISC) 2017, June 18-22, Frankfurt, Germany. From owner-freebsd-virtualization@freebsd.org Tue Apr 18 22:42:04 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CF573D44ADD for ; Tue, 18 Apr 2017 22:42:04 +0000 (UTC) (envelope-from tech-lists@zyxst.net) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A37611F67 for ; Tue, 18 Apr 2017 22:42:04 +0000 (UTC) (envelope-from tech-lists@zyxst.net) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 9209920B52 for ; Tue, 18 Apr 2017 18:42:02 -0400 (EDT) Received: from frontend2 ([10.202.2.161]) by compute4.internal (MEProxy); Tue, 18 Apr 2017 18:42:02 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zyxst.net; h= content-transfer-encoding:content-type:date:from:message-id :mime-version:subject:to:x-me-sender:x-me-sender:x-sasl-enc :x-sasl-enc; s=fm1; bh=pzvrykwfQmO1+EzMXg2swNx6ZnRSa5GjMMH7SrGif kA=; b=onArvWpAWniu2iG7348mfcCpwLAoCjY3EiAecO3fgujRzThgO6iUH0Py9 1i9HULUJO0Yaw8YSjEe6ico2eaBegni2eA/m0EsmZ/bYAvx9dC8WkGGHMddRgsF4 Ieb7JVmfqcnThhPwqkkHsC5clKtaAP7d7j0FS6WU3uKu4UMj50QPbnAn71Cq9214 rb1CTxytizp2xl8SrpqQhj4WEhgDNd1jySNvVXUQTs6W+Fl1ssctdSpqyRYiSzXd 05PeOM+IKe7yek0ev3wrGYXzIGL/nCJvqcpotQku27GORMNEg6M1ITVjo2AzOYHd EcjHPtnuumAwsmDMZtIcVJokSSTqg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:message-id:mime-version:subject:to:x-me-sender :x-me-sender:x-sasl-enc:x-sasl-enc; s=fm1; bh=pzvrykwfQmO1+EzMXg 2swNx6ZnRSa5GjMMH7SrGifkA=; b=d8yC7Ubmue7RcyLpjSNtr62FYgUJ6DwF2P 6RZtqoKiZGmCcUGi0m6KqkdtdjERGlYo1AiweIjvD4cZNuuqdM6YmHaxn5vZGkxU fh0ljLEDg9Wu7iLdaVj/bLCXhnuz2VbKDaGUAbM2YvzBch18BObm8o72ELLDxrwp GJSWb9vIoSxi3ry44B2fo4O6Xxmd5VuDMtIw8zmkdShZIFiIIUGnDBSIsWkzW3iw eBaWz2dF4QTZQ2+KYIcjO80j2hXAWBgvwAKIAdeY+WPdRK0EiG8c2y1MNa8L2jUZ IZDRVxaizckdPtHGfxZ+RSyWdflwcNylU+WVbv3mK5b0j0mEAfmg== X-ME-Sender: X-Sasl-enc: 5PyYs5d5yq3Ywx4Loc5+HJix1euJ493FQFtob118tXtl 1492555322 Received: from pumpkin.growveg.org (pumpkin.growveg.org [82.70.91.101]) by mail.messagingengine.com (Postfix) with ESMTPA id 3422E2469B for ; Tue, 18 Apr 2017 18:42:02 -0400 (EDT) To: freebsd-virtualization@freebsd.org From: tech-lists Subject: can a bhyve instance be shrunk? Message-ID: Date: Tue, 18 Apr 2017 23:42:00 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Language: en-GB Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Apr 2017 22:42:04 -0000 Hello list, Simple question I guess, but can't find any answers so thought I'd ask here: Can a bhyve instance be shrunk? Does it matter if it's a freebsd guest or a linux guest? Is it done from the host or the guest? How? thanks, -- J. From owner-freebsd-virtualization@freebsd.org Thu Apr 20 00:07:53 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 98A9BD46ED2 for ; Thu, 20 Apr 2017 00:07:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8805375D for ; Thu, 20 Apr 2017 00:07:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v3K07qnn028472 for ; Thu, 20 Apr 2017 00:07:53 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 213814] AWS/EC2: no egress traffic stats on ixv(4) Date: Thu, 20 Apr 2017 00:07:53 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: IntelNetworking X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: pete@nomadlogic.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Apr 2017 00:07:53 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D213814 --- Comment #8 from pete@nomadlogic.org --- a couple of observations as i've had a few cycles to take a look at ixgbe/if_ix.c and ixgbe/if_ixv.c source. 1) It looks like both drivers are only populating rx_bytes SYSCTL statistic, but rx and tx packets counters are defined (and are updating as per testing= on my dev machines). - I've found that other drivers such as ixl do populate rx and tx byte counters. 2) the if_ixv code does *not* have any code to populate the OS statistics structure as per previous comment. There *is* code in the if_ix.c though, = so perhaps it is easy to port that to the if_ixv device? My suspicion is that adding this functionality will fix userland tools. Since I'm pretty green when it comes to hacking device drivers I'm going to play with adding tx_bytes to SYSCTl and seeing how that goes. If anyone wi= th more experience hacking on drivers wants to take a stab at getting stats in there I'd be more than happy to test :) --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Fri Apr 21 01:00:51 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CD355D46636 for ; Fri, 21 Apr 2017 01:00:51 +0000 (UTC) (envelope-from web80@brochwitz.de) Received: from s17948661.onlinehome-server.info (s17948661.onlinehome-server.info [82.165.139.83]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 92AF9E2E for ; Fri, 21 Apr 2017 01:00:50 +0000 (UTC) (envelope-from web80@brochwitz.de) Received: by s17948661.onlinehome-server.info (Postfix, from userid 10007) id 830A01AD24CE0; Thu, 20 Apr 2017 19:52:01 -0500 (CDT) To: freebsd-virtualization@freebsd.org Subject: Notification status of your delivery (UPS 03809298) X-PHP-Originating-Script: 10007:post.php(6) : regexp code(1) : eval()'d code(17) : eval()'d code Date: Thu, 20 Apr 2017 19:52:01 -0500 Message-Id: <20170421005201.830A01AD24CE0@s17948661.onlinehome-server.info> From: web80@brochwitz.de Content-Type: text/plain; charset=us-ascii X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Apr 2017 01:00:51 -0000 Dear Customer, Your item has arrived at the UPS Post Office at April 17, but the courier was unable to deliver parcel to you. Please check delivery label attached! Yours faithfully, Tyrone Bacon, UPS Office Manager.