Date: Sun, 25 Mar 2012 14:47:23 +1000 From: Da Rock <freebsd-emulation@herveybayaustralia.com.au> To: freebsd-emulation@freebsd.org Subject: Re: Fwd: Kernel memory usage Message-ID: <4F6EA35B.1050308@herveybayaustralia.com.au> In-Reply-To: <CA%2BWntOuyngb=cpTRyx_EjBjJ=mBOR6JuKX-PEDe%2BKNEJpu_scw@mail.gmail.com> References: <4F5D48E6.9010802@herveybayaustralia.com.au> <CA%2BWntOt=4P=%2BmdAX3G1s5jQHxt9HFSN6c%2BxTjo9cQAsaSgdJqw@mail.gmail.com> <4F5F2CC5.80608@herveybayaustralia.com.au> <CA%2BWntOv%2B%2BtkAms%2BOQtW-bH=3f7VnUhtA7SA1c8dQYwZho5aX4g@mail.gmail.com> <4F615D09.8030009@mittelstaedt.us> <CA%2BWntOth%2BQinHiU0GSq=ozbswnqx0v1Gui2D3mrjkZhke3Ounw@mail.gmail.com> <4F617C63.9030106@mittelstaedt.us> <4F618F2B.7040709@herveybayaustralia.com.au> <4F642D63.10600@mittelstaedt.us> <CA%2BWntOuyngb=cpTRyx_EjBjJ=mBOR6JuKX-PEDe%2BKNEJpu_scw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 03/18/12 00:49, Super Bisquit wrote: > On Sat, Mar 17, 2012 at 2:21 AM, Ted Mittelstaedt<tedm@mittelstaedt.us>wrote: > >> On 3/14/2012 11:41 PM, Da Rock wrote: >> >>> On 03/15/12 15:21, Ted Mittelstaedt wrote: >>> >>>> A number of FreeBSD kernel structures are dynamically created at >>>> boot time - based on the amount of ram in the system. I would guess >>>> if you boot your stripped BSD image on a virtual system with 64Mb of >>>> ram you wouldn't see the kernel grow to 32MB >>>> >>> So how do I find out how large the "kernel" is then in memory? >>> >> Boot it! From: >> >> http://www.freebsd.org/doc/en_**US.ISO8859-1/books/handbook/** >> configtuning-kernel-limits.**html<http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/configtuning-kernel-limits.html>; >> >> "...The variable kern.maxusers is automatically sized at boot based on >> >> the amount of memory available in the system, and may be determined at >> >> run-time by inspecting the value of the read-only kern.maxusers sysctl..." >> >> A whole bunch of kernel parameters are calculated at boot based on this >> figure. Those make the kernel bigger or smaller. >> >> As to how "large" it is, the file descriptors, clusters and so on that >> are calculated at boot are considered part of the kernel from a memory >> standpoint. I'll take a closer look at that- I missed the significance. >> >> >> And how >> >>> did this conversation make it to emulation@? :) >>> >>> >> beats me. > > From working with qemu, asking questions, researching. This seemed to be a > better place for your situation. Ahh. I see now... Thanks for the info guys. 'I'll be back...' I'm sure :) > >> >> Actually, another question comes to mind: is it possible to limit how >>> large the kernel structures grow? >>> >> The handbook seems to imply that once they are all setup the kernel >> doesn't grow. >> >> >> >>>> It would be interesting to see FreeBSD running on a Smartphone. >>>> >>> Thats what I'm interested in for the long term, but not necessarily in >>> this instance. This really has become an interesting thread... for every >>> answer there are more questions- a hydra ;) >>> >> Heh. >> >> Ted >> >> >> >>>> Ted >>>> >>>> On 3/14/2012 9:03 PM, Super Bisquit wrote: >>>> >>>>> On 3/14/12, Ted Mittelstaedt<tedm@**mittelstaedt.us<tedm@mittelstaedt.us>> >>>>> wrote: >>>>> >>>>>> What do you regard as a "low memory" machine? >>>>>> >>>>> On real hardware I've used: >>>>> HTC Apache 64/128 ROM/RAM >>>>> Embedded 486 with 5M RAM >>>>> Older tower maxed at 128M >>>>> >>>>> I'd say anything less than 64M RAM for normal hardware, less than 32 >>>>> for embedded. >>>>> >>>>> Some systems do not have small memory modules. I have two SunBlade >>>>> 1000s that can be stripped down to 128M. >>>>> >>>>> I'm also aware that the environment limits what the machine can have >>>>> installed. >>>>> >>>>> There are older systems and embedded devices which run/ran on 16kb of >>>>> RAM. I'm certain that the oldest electronic led calculators used even >>>>> less memory. >>>>> I digress. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>>> Ted >>>>>> >>>>>> On 3/13/2012 8:30 AM, Super Bisquit wrote: >>>>>> >>>>>>> ---------- Forwarded message ---------- >>>>>>> From: Da Rock<9Phackers@**herveybayaustralia.com.au<9Phackers@herveybayaustralia.com.au> >>>>>>> Date: Tue, Mar 13, 2012 at 7:17 AM >>>>>>> Subject: Re: Kernel memory usage >>>>>>> To: freebsd-hackers@freebsd.org >>>>>>> >>>>>>> >>>>>>> On 03/13/12 05:40, Super Bisquit wrote: >>>>>>> >>>>>>> CPU architecture and model have a lot to do with performance. >>>>>>>> You will also get different results if you used qemu in place of >>>>>>>> VirtualBox. Qemu allows you to choose different emulated >>>>>>>> architectures, CPUs, and machine bases. What's the downside? You have >>>>>>>> to use the command line. >>>>>>>> Install qemu and run a series of virtual machines. >>>>>>>> Embedded devices also include Power(PC), ARMv?, Coldfire, et al; >>>>>>>> you're only dealing with i386 and/or the 64 bit extension (AMD64). >>>>>>>> CISC- which does not contain any hardware modification to be a RISC >>>>>>>> replacement- runs fewer instructions than RISC due to the limited >>>>>>>> number of registers. Take this into consideration every time a >>>>>>>> program runs. >>>>>>>> Everything else also matters on real and emulated systems: >>>>>>>> Is it ide, scsi, sdd, flashdevice for the hard drive? >>>>>>>> What type of RAM? >>>>>>>> Dedicated or shared disk? >>>>>>>> >>>>>>>> I'm a little confused by the response, I was interested if someone >>>>>>> knew >>>>>>> what determines the size of kernel in memory (or wired); I only >>>>>>> considered >>>>>>> the embedded list because they have a greater interest in the >>>>>>> memory size >>>>>>> working with so little. >>>>>>> >>>>>>> It is academic as I'm trying to understand the kernel internals, as >>>>>>> well >>>>>>> as >>>>>>> understand what works with low memory so I can tune accordingly. I >>>>>>> understand the different CPU instruction sets (roughly), although I >>>>>>> would >>>>>>> be interested as to the size of the kernel in each. >>>>>>> >>>>>>> What my question was about was the wired memory size and what makes it >>>>>>> grow >>>>>>> (to put it super simply :) ). I know some growth would be expected, >>>>>>> but >>>>>>> this seems obese; how would I find out how much memory a process >>>>>>> structure >>>>>>> takes? Or else what am I missing? >>>>>>> >>>>>>> That said I'll have a crack at what you suggest as that follows a >>>>>>> whole >>>>>>> new >>>>>>> interesting tangent :) I have used qemu before, but found VBox a >>>>>>> bit more >>>>>>> responsive (and, I will admit, easier...) >>>>>>> >>>>>>> >>>>>>> On 3/11/12, Da >>>>>>>> Rock<9Phackers@**herveybayaust**ralia.com.au<http://herveybayaustralia.com.au>; >>>>>>>> <9Phackers@**herveybayaustralia.com.au<9Phackers@herveybayaustralia.com.au> >>>>>>>> wrote: >>>>>>>> >>>>>>>> I may be required to move this to embedded, but I am only looking >>>>>>>>> for >>>>>>>>> generalisation. >>>>>>>>> >>>>>>>>> Recently a thread came up on questions regarding memory usage, and a >>>>>>>>> post was made regarding wired memory being nearly all kernel- >>>>>>>>> something >>>>>>>>> I was ready to dispute, but then I thought I'd better make sure. >>>>>>>>> >>>>>>>>> So I tested a few theories first off: >>>>>>>>> >>>>>>>>> 1. Comparing memory usage across machines >>>>>>>>> >>>>>>>>> I checked servers and desktops as well as vm's for memory usage, >>>>>>>>> and I >>>>>>>>> found some interesting results. On a firewall with no apps installed >>>>>>>>> only 35M wired is used, on a desktop up to 700M~ can be used. >>>>>>>>> Even as a >>>>>>>>> dedicated server with a few services used it remains around 35M. >>>>>>>>> >>>>>>>>> Surely this means that the wired memory used is not just kernel? >>>>>>>>> But I >>>>>>>>> held off my assumptions as it was still plausible that the >>>>>>>>> structures >>>>>>>>> used by the kernel could balloon that far, too. >>>>>>>>> >>>>>>>>> 2. Stripped down, lean mean, kernel machine >>>>>>>>> >>>>>>>>> I then (using a vm I was building a kernel for anyway) stripped >>>>>>>>> down a >>>>>>>>> kernel in a VBox VM using le drivers for network to see what >>>>>>>>> could be >>>>>>>>> achieved. This is my kernel conf: >>>>>>>>> >>>>>>>>> include GENERIC >>>>>>>>> ident VPN >>>>>>>>> options IPSEC >>>>>>>>> options IPSEC_DEBUG >>>>>>>>> options IPSEC_NAT_T >>>>>>>>> device crypto >>>>>>>>> device enc >>>>>>>>> >>>>>>>>> # minimise kernel >>>>>>>>> nooptions UFS_GJOURNAL >>>>>>>>> nooptions MD_ROOT >>>>>>>>> nooptions NFSCL >>>>>>>>> nooptions NFSD >>>>>>>>> nooptions NFSLOCKD >>>>>>>>> nooptions NFS_ROOT >>>>>>>>> nooptions MSDOSFS >>>>>>>>> nooptions CD9660 >>>>>>>>> nooptions PROCFS >>>>>>>>> nooptions PSEUDOFS >>>>>>>>> nodevice fdc >>>>>>>>> nodevice mvs >>>>>>>>> nodevice siis >>>>>>>>> nodevice ahc >>>>>>>>> nodevice ahd >>>>>>>>> nodevice esp >>>>>>>>> nodevice hptiop >>>>>>>>> nodevice isp >>>>>>>>> nodevice mpt >>>>>>>>> nodevice mps >>>>>>>>> nodevice sym >>>>>>>>> nodevice trm >>>>>>>>> nodevice adv >>>>>>>>> nodevice adw >>>>>>>>> nodevice aic >>>>>>>>> nodevice bt >>>>>>>>> nodevice ses >>>>>>>>> nodevice amr >>>>>>>>> nodevice arcmsr >>>>>>>>> nodevice ciss >>>>>>>>> nodevice dpt >>>>>>>>> nodevice hptmv >>>>>>>>> nodevice hptrr >>>>>>>>> nodevice irr >>>>>>>>> nodevice ips >>>>>>>>> nodevice mly >>>>>>>>> nodevice twa >>>>>>>>> nodevice aac >>>>>>>>> nodevice aacp >>>>>>>>> nodevice ida >>>>>>>>> nodevice mfi >>>>>>>>> nodevice mlx >>>>>>>>> nodevice twe >>>>>>>>> nodevice tws >>>>>>>>> nodevice splash >>>>>>>>> nodevice cbb >>>>>>>>> nodevice pccard >>>>>>>>> nodevice cardbus >>>>>>>>> nodevice uart >>>>>>>>> nodevice ppc >>>>>>>>> nodevice ppbus >>>>>>>>> nodevice lpt >>>>>>>>> nodevice plip >>>>>>>>> nodevice ppi >>>>>>>>> nodevice puc >>>>>>>>> nodevice bxe >>>>>>>>> nodevice de >>>>>>>>> nodevice em >>>>>>>>> nodevice igb >>>>>>>>> nodevice ixgbe >>>>>>>>> nodevice ti >>>>>>>>> nodevice txp >>>>>>>>> nodevice vx >>>>>>>>> nodevice miibus >>>>>>>>> nodevice ae >>>>>>>>> nodevice age >>>>>>>>> nodevice alc >>>>>>>>> nodevice ale >>>>>>>>> nodevice bce >>>>>>>>> nodevice bfe >>>>>>>>> nodevice bge >>>>>>>>> nodevice dc >>>>>>>>> nodevice et >>>>>>>>> nodevice fxp >>>>>>>>> nodevice jme >>>>>>>>> nodevice lge >>>>>>>>> nodevice msk >>>>>>>>> nodevice nfe >>>>>>>>> nodevice nge >>>>>>>>> nodevice pcn >>>>>>>>> nodevice re >>>>>>>>> nodevice rl >>>>>>>>> nodevice sf >>>>>>>>> nodevice sge >>>>>>>>> nodevice sis >>>>>>>>> nodevice sk >>>>>>>>> nodevice ste >>>>>>>>> nodevice stge >>>>>>>>> nodevice tl >>>>>>>>> nodevice tx >>>>>>>>> nodevice vge >>>>>>>>> nodevice vr >>>>>>>>> nodevice wb >>>>>>>>> nodevice xl >>>>>>>>> nodevice cs >>>>>>>>> nodevice ed >>>>>>>>> nodevice ex >>>>>>>>> nodevice ep >>>>>>>>> nodevice fe >>>>>>>>> nodevice sn >>>>>>>>> nodevice xe >>>>>>>>> nodevice wlan >>>>>>>>> nooptions IEEE80211_DEBUG >>>>>>>>> nooptions IEEE80211_AMPDU_AGE >>>>>>>>> nooptions IEEE80211_SUPPORT_MESH >>>>>>>>> nodevice wlan_wep >>>>>>>>> nodevice wlan_ccmp >>>>>>>>> nodevice wlan_tkip >>>>>>>>> nodevice wlan_amrr >>>>>>>>> nodevice an >>>>>>>>> nodevice ath >>>>>>>>> nodevice ath_pci >>>>>>>>> nodevice ath_hal >>>>>>>>> nooptions AH_SUPPORT_AR5416 >>>>>>>>> nodevice ath_rate_sample >>>>>>>>> nodevice ipw >>>>>>>>> nodevice iwi >>>>>>>>> nodevice iwn >>>>>>>>> nodevice malo >>>>>>>>> nodevice mwl >>>>>>>>> nodevice ral >>>>>>>>> nodevice wi >>>>>>>>> nodevice wpi >>>>>>>>> nodevice md >>>>>>>>> nooption USB_DEBUG >>>>>>>>> nodevice uhci >>>>>>>>> nodevice ohci >>>>>>>>> nodevice ehci >>>>>>>>> nodevice xhci >>>>>>>>> nodevice usb >>>>>>>>> nodevice uhid >>>>>>>>> nodevice ukbd >>>>>>>>> nodevice ulpt >>>>>>>>> nodevice umass >>>>>>>>> nodevice ums >>>>>>>>> nodevice urio >>>>>>>>> nodevice u3g >>>>>>>>> nodevice uark >>>>>>>>> nodevice ubsa >>>>>>>>> nodevice uftdi >>>>>>>>> nodevice uipaq >>>>>>>>> nodevice uplcom >>>>>>>>> nodevice uslcom >>>>>>>>> nodevice uvisor >>>>>>>>> nodevice uvscom >>>>>>>>> nodevice aue >>>>>>>>> nodevice axe >>>>>>>>> nodevice cdce >>>>>>>>> nodevice cue >>>>>>>>> nodevice kue >>>>>>>>> nodevice rue >>>>>>>>> nodevice udav >>>>>>>>> nodevice rum >>>>>>>>> nodevice run >>>>>>>>> nodevice uath >>>>>>>>> nodevice upgt >>>>>>>>> nodevice ural >>>>>>>>> nodevice urtw >>>>>>>>> nodevice zyd >>>>>>>>> #nodevice firewire >>>>>>>>> nodevice fwe >>>>>>>>> nodevice fwip >>>>>>>>> #nodevice dcons >>>>>>>>> #nodevice dcons_rom >>>>>>>>> nodevice sound >>>>>>>>> nodevice snd_es137x >>>>>>>>> nodevice snd_hda >>>>>>>>> nodevice snd_ich >>>>>>>>> nodevice snd_uaudio >>>>>>>>> nodevice snd_via8233 >>>>>>>>> >>>>>>>>> World was also rebuilt as recommended by the handbook. As you can >>>>>>>>> see I >>>>>>>>> was rebuilding for IPSEC (for testing purposes only note). I >>>>>>>>> couldn't >>>>>>>>> remove dcons (not sure why- missing defined references), and that >>>>>>>>> pulled >>>>>>>>> in firewire too. >>>>>>>>> >>>>>>>>> The result was surprising: a 14M kernel became 6.3M, and when >>>>>>>>> running I >>>>>>>>> found it only used 19M used for wired (whoopee!) as opposed to >>>>>>>>> 35M. No >>>>>>>>> services were running per se, only the usual suspects in base: >>>>>>>>> >>>>>>>>> vpn-test# ps ax >>>>>>>>> PID TT STAT TIME COMMAND >>>>>>>>> 0 ?? DLs 0:00.11 [kernel] >>>>>>>>> 1 ?? ILs 0:00.01 /sbin/init -- >>>>>>>>> 2 ?? DL 0:00.00 [crypto] >>>>>>>>> 3 ?? DL 0:00.00 [crypto returns] >>>>>>>>> 4 ?? DL 0:00.00 [sctp_iterator] >>>>>>>>> 5 ?? DL 0:00.00 [xpt_thrd] >>>>>>>>> 6 ?? DL 0:00.10 [pagedaemon] >>>>>>>>> 7 ?? DL 0:00.00 [vmdaemon] >>>>>>>>> 8 ?? DL 0:00.01 [pagezero] >>>>>>>>> 9 ?? DL 0:00.30 [bufdaemon] >>>>>>>>> 10 ?? DL 0:00.00 [audit] >>>>>>>>> 11 ?? RL 589:34.00 [idle] >>>>>>>>> 12 ?? WL 0:38.64 [intr] >>>>>>>>> 13 ?? DL 0:12.21 [geom] >>>>>>>>> 14 ?? DL 0:03.30 [yarrow] >>>>>>>>> 15 ?? DL 0:04.40 [syncer] >>>>>>>>> 16 ?? DL 0:00.63 [vnlru] >>>>>>>>> 17 ?? DL 0:00.53 [softdepflush] >>>>>>>>> 104 ?? Is 0:00.00 adjkerntz -i >>>>>>>>> 582 ?? Is 0:00.00 /sbin/devd >>>>>>>>> 737 ?? Ss 0:00.09 /usr/sbin/syslogd -s >>>>>>>>> 912 ?? Ss 0:02.68 /usr/sbin/ntpd -c /etc/ntp.conf -p >>>>>>>>> /var/run/ntpd.pid >>>>>>>>> 977 ?? Is 0:00.00 /usr/sbin/sshd >>>>>>>>> 984 ?? Ss 0:00.81 sendmail: accepting connections (sendmail) >>>>>>>>> 988 ?? Is 0:00.01 sendmail: Queue runner@00:30:00 for >>>>>>>>> /var/spool/client >>>>>>>>> 994 ?? Ss 0:00.12 /usr/sbin/cron -s >>>>>>>>> 2061 ?? Is 0:00.03 sshd: admin [priv] (sshd) >>>>>>>>> 2064 ?? S 0:00.04 sshd: admin@pts/0 (sshd) >>>>>>>>> 1052 v0 Is 0:00.01 login [pam] (login) >>>>>>>>> 1060 v0 I+ 0:00.02 -csh (csh) >>>>>>>>> 1053 v1 Is+ 0:00.00 /usr/libexec/getty Pc ttyv1 >>>>>>>>> 1054 v2 Is+ 0:00.00 /usr/libexec/getty Pc ttyv2 >>>>>>>>> 1055 v3 Is+ 0:00.00 /usr/libexec/getty Pc ttyv3 >>>>>>>>> 1056 v4 Is+ 0:00.00 /usr/libexec/getty Pc ttyv4 >>>>>>>>> 1057 v5 Is+ 0:00.00 /usr/libexec/getty Pc ttyv5 >>>>>>>>> 1058 v6 Is+ 0:00.00 /usr/libexec/getty Pc ttyv6 >>>>>>>>> 1059 v7 Is+ 0:00.00 /usr/libexec/getty Pc ttyv7 >>>>>>>>> 2065 0 Is 0:00.02 -csh (csh) >>>>>>>>> 2068 0 I 0:00.01 su >>>>>>>>> 2069 0 S 0:00.04 _su (csh) >>>>>>>>> 2414 0 R+ 0:00.00 ps ax >>>>>>>>> >>>>>>>>> After some minutes though, the usage went up to 39M. WTF? >>>>>>>>> >>>>>>>>> So I'm now thoroughly confused. Google didn't really show up any >>>>>>>>> real >>>>>>>>> answers either. Aside from discovering the true nature of what was >>>>>>>>> discussed in the posts, my hypothetical theorising is for my own >>>>>>>>> academics as well; and I'm trying to slim down my kernels to see >>>>>>>>> if I >>>>>>>>> can improve performance with low memory systems. >>>>>>>>> >>>>>>>>> Can process structures cause this much ballooning? I find that >>>>>>>>> hard to >>>>>>>>> believe given not much is running here. My only other thought >>>>>>>>> would be >>>>>>>>> pipe or network structures? Anything I'm missing? >>>>>>>>> >>>>>>>>> FWIW, my trimmed down, slim and slender, aerodynamic kernel can >>>>>>>>> boot in >>>>>>>>> seconds as opposed to minutes. Sweet! Although compiling clang took >>>>>>>>> years... and gcc was built as well :/ >>>>>>>>> >>>>>>>>> Cheers >>>>>>>>> ______________________________****_________________ >>>>>>>>> freebsd-hackers@freebsd.org mailing list >>>>>>>>> http://lists.freebsd.org/****mailman/listinfo/freebsd-****hackers<http://lists.freebsd.org/**mailman/listinfo/freebsd-**hackers>; >>>>>>>>> <http://lists.freebsd.**org/mailman/listinfo/freebsd-**hackers<http://lists.freebsd.org/mailman/listinfo/freebsd-hackers>; >>>>>>>>> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@**** >>>>>>>>> freebsd.org<freebsd-hackers-**unsubscribe@freebsd.org<freebsd-hackers-unsubscribe@freebsd.org> >>>>>>>>>> " >>>>>>>>> ______________________________****_________________ >>>>>>>>> >>>>>>>> freebsd-hackers@freebsd.org mailing list >>>>>>>> http://lists.freebsd.org/****mailman/listinfo/freebsd-****hackers<http://lists.freebsd.org/**mailman/listinfo/freebsd-**hackers>; >>>>>>>> <http://lists.freebsd.**org/mailman/listinfo/freebsd-**hackers<http://lists.freebsd.org/mailman/listinfo/freebsd-hackers>; >>>>>>>> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@**** >>>>>>>> freebsd.org<freebsd-hackers-**unsubscribe@freebsd.org<freebsd-hackers-unsubscribe@freebsd.org> >>>>>>>>> " >>>>>>>> >>>>>>> ______________________________****_________________ >>>>>>> freebsd-hackers@freebsd.org mailing list >>>>>>> http://lists.freebsd.org/****mailman/listinfo/freebsd-****hackers<http://lists.freebsd.org/**mailman/listinfo/freebsd-**hackers>; >>>>>>> <http://lists.freebsd.**org/mailman/listinfo/freebsd-**hackers<http://lists.freebsd.org/mailman/listinfo/freebsd-hackers>; >>>>>>> To unsubscribe, send any mail to >>>>>>> "freebsd-hackers-unsubscribe@**freebsd.org<freebsd-hackers-unsubscribe@freebsd.org> >>>>>>> " >>>>>>> >>>>>>> I'm fowarding this thread in the event the individual, "da Rock" >>>>>>> will be >>>>>>> able to receive more help. >>>>>>> ______________________________**_________________ >>>>>>> freebsd-emulation@freebsd.org mailing list >>>>>>> http://lists.freebsd.org/**mailman/listinfo/freebsd-**emulation<http://lists.freebsd.org/mailman/listinfo/freebsd-emulation>; >>>>>>> To unsubscribe, send any mail to >>>>>>> "freebsd-emulation-**unsubscribe@freebsd.org<freebsd-emulation-unsubscribe@freebsd.org> >>>>>>> " >>>>>>> >>>>>>> >>>>>>> ----- >>>>>>> No virus found in this message. >>>>>>> Checked by AVG - www.avg.com >>>>>>> Version: 2012.0.1913 / Virus Database: 2114/4871 - Release Date: >>>>>>> 03/14/12 >>>>>>> >>>>>> >>>>>> >>>>> ----- >>>>> No virus found in this message. >>>>> Checked by AVG - www.avg.com >>>>> Version: 2012.0.1913 / Virus Database: 2114/4871 - Release Date: >>>>> 03/14/12 >>>>> >>>> ______________________________**_________________ >>>> freebsd-emulation@freebsd.org mailing list >>>> http://lists.freebsd.org/**mailman/listinfo/freebsd-**emulation<http://lists.freebsd.org/mailman/listinfo/freebsd-emulation>; >>>> To unsubscribe, send any mail to >>>> "freebsd-emulation-**unsubscribe@freebsd.org<freebsd-emulation-unsubscribe@freebsd.org> >>>> " >>>> >>> ______________________________**_________________ >>> freebsd-emulation@freebsd.org mailing list >>> http://lists.freebsd.org/**mailman/listinfo/freebsd-**emulation<http://lists.freebsd.org/mailman/listinfo/freebsd-emulation>; >>> To unsubscribe, send any mail to >>> "freebsd-emulation-**unsubscribe@freebsd.org<freebsd-emulation-unsubscribe@freebsd.org> >>> " >>> >>> >>> ----- >>> No virus found in this message. >>> Checked by AVG - www.avg.com >>> Version: 2012.0.1913 / Virus Database: 2114/4875 - Release Date: 03/16/12 >>> >> ______________________________**_________________ >> freebsd-emulation@freebsd.org mailing list >> http://lists.freebsd.org/**mailman/listinfo/freebsd-**emulation<http://lists.freebsd.org/mailman/listinfo/freebsd-emulation>; >> To unsubscribe, send any mail to "freebsd-emulation-** >> unsubscribe@freebsd.org<freebsd-emulation-unsubscribe@freebsd.org>" >> > _______________________________________________ > freebsd-emulation@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-emulation > To unsubscribe, send any mail to "freebsd-emulation-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4F6EA35B.1050308>