Date: Sat, 17 Mar 2012 10:49:14 -0400 From: Super Bisquit <superbisquit@gmail.com> To: Ted Mittelstaedt <tedm@mittelstaedt.us> Cc: freebsd-emulation@freebsd.org Subject: Re: Fwd: Kernel memory usage Message-ID: <CA%2BWntOuyngb=cpTRyx_EjBjJ=mBOR6JuKX-PEDe%2BKNEJpu_scw@mail.gmail.com> In-Reply-To: <4F642D63.10600@mittelstaedt.us> References: <4F5D48E6.9010802@herveybayaustralia.com.au> <CA%2BWntOt=4P=%2BmdAX3G1s5jQHxt9HFSN6c%2BxTjo9cQAsaSgdJqw@mail.gmail.com> <4F5F2CC5.80608@herveybayaustralia.com.au> <CA%2BWntOv%2B%2BtkAms%2BOQtW-bH=3f7VnUhtA7SA1c8dQYwZho5aX4g@mail.gmail.com> <4F615D09.8030009@mittelstaedt.us> <CA%2BWntOth%2BQinHiU0GSq=ozbswnqx0v1Gui2D3mrjkZhke3Ounw@mail.gmail.com> <4F617C63.9030106@mittelstaedt.us> <4F618F2B.7040709@herveybayaustralia.com.au> <4F642D63.10600@mittelstaedt.us>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, Mar 17, 2012 at 2:21 AM, Ted Mittelstaedt <tedm@mittelstaedt.us>wrote: > On 3/14/2012 11:41 PM, Da Rock wrote: > >> On 03/15/12 15:21, Ted Mittelstaedt wrote: >> >>> A number of FreeBSD kernel structures are dynamically created at >>> boot time - based on the amount of ram in the system. I would guess >>> if you boot your stripped BSD image on a virtual system with 64Mb of >>> ram you wouldn't see the kernel grow to 32MB >>> >> So how do I find out how large the "kernel" is then in memory? >> > > Boot it! From: > > http://www.freebsd.org/doc/en_**US.ISO8859-1/books/handbook/** > configtuning-kernel-limits.**html<http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/configtuning-kernel-limits.html>; > > "...The variable kern.maxusers is automatically sized at boot based on > > the amount of memory available in the system, and may be determined at > > run-time by inspecting the value of the read-only kern.maxusers sysctl..." > > A whole bunch of kernel parameters are calculated at boot based on this > figure. Those make the kernel bigger or smaller. > > As to how "large" it is, the file descriptors, clusters and so on that > are calculated at boot are considered part of the kernel from a memory > standpoint. > > > And how > >> did this conversation make it to emulation@? :) >> >> > beats me. >From working with qemu, asking questions, researching. This seemed to be a better place for your situation. > > > Actually, another question comes to mind: is it possible to limit how >> large the kernel structures grow? >> > > The handbook seems to imply that once they are all setup the kernel > doesn't grow. > > > >>> It would be interesting to see FreeBSD running on a Smartphone. >>> >> Thats what I'm interested in for the long term, but not necessarily in >> this instance. This really has become an interesting thread... for every >> answer there are more questions- a hydra ;) >> > > Heh. > > Ted > > > >>> Ted >>> >>> On 3/14/2012 9:03 PM, Super Bisquit wrote: >>> >>>> On 3/14/12, Ted Mittelstaedt<tedm@**mittelstaedt.us<tedm@mittelstaedt.us>> >>>> wrote: >>>> >>>>> What do you regard as a "low memory" machine? >>>>> >>>> On real hardware I've used: >>>> HTC Apache 64/128 ROM/RAM >>>> Embedded 486 with 5M RAM >>>> Older tower maxed at 128M >>>> >>>> I'd say anything less than 64M RAM for normal hardware, less than 32 >>>> for embedded. >>>> >>>> Some systems do not have small memory modules. I have two SunBlade >>>> 1000s that can be stripped down to 128M. >>>> >>>> I'm also aware that the environment limits what the machine can have >>>> installed. >>>> >>>> There are older systems and embedded devices which run/ran on 16kb of >>>> RAM. I'm certain that the oldest electronic led calculators used even >>>> less memory. >>>> I digress. >>>> >>>> >>>> >>>> >>>> >>>>> Ted >>>>> >>>>> On 3/13/2012 8:30 AM, Super Bisquit wrote: >>>>> >>>>>> ---------- Forwarded message ---------- >>>>>> From: Da Rock<9Phackers@**herveybayaustralia.com.au<9Phackers@herveybayaustralia.com.au> >>>>>> > >>>>>> Date: Tue, Mar 13, 2012 at 7:17 AM >>>>>> Subject: Re: Kernel memory usage >>>>>> To: freebsd-hackers@freebsd.org >>>>>> >>>>>> >>>>>> On 03/13/12 05:40, Super Bisquit wrote: >>>>>> >>>>>> CPU architecture and model have a lot to do with performance. >>>>>>> You will also get different results if you used qemu in place of >>>>>>> VirtualBox. Qemu allows you to choose different emulated >>>>>>> architectures, CPUs, and machine bases. What's the downside? You have >>>>>>> to use the command line. >>>>>>> Install qemu and run a series of virtual machines. >>>>>>> Embedded devices also include Power(PC), ARMv?, Coldfire, et al; >>>>>>> you're only dealing with i386 and/or the 64 bit extension (AMD64). >>>>>>> CISC- which does not contain any hardware modification to be a RISC >>>>>>> replacement- runs fewer instructions than RISC due to the limited >>>>>>> number of registers. Take this into consideration every time a >>>>>>> program runs. >>>>>>> Everything else also matters on real and emulated systems: >>>>>>> Is it ide, scsi, sdd, flashdevice for the hard drive? >>>>>>> What type of RAM? >>>>>>> Dedicated or shared disk? >>>>>>> >>>>>>> I'm a little confused by the response, I was interested if someone >>>>>> knew >>>>>> what determines the size of kernel in memory (or wired); I only >>>>>> considered >>>>>> the embedded list because they have a greater interest in the >>>>>> memory size >>>>>> working with so little. >>>>>> >>>>>> It is academic as I'm trying to understand the kernel internals, as >>>>>> well >>>>>> as >>>>>> understand what works with low memory so I can tune accordingly. I >>>>>> understand the different CPU instruction sets (roughly), although I >>>>>> would >>>>>> be interested as to the size of the kernel in each. >>>>>> >>>>>> What my question was about was the wired memory size and what makes it >>>>>> grow >>>>>> (to put it super simply :) ). I know some growth would be expected, >>>>>> but >>>>>> this seems obese; how would I find out how much memory a process >>>>>> structure >>>>>> takes? Or else what am I missing? >>>>>> >>>>>> That said I'll have a crack at what you suggest as that follows a >>>>>> whole >>>>>> new >>>>>> interesting tangent :) I have used qemu before, but found VBox a >>>>>> bit more >>>>>> responsive (and, I will admit, easier...) >>>>>> >>>>>> >>>>>> On 3/11/12, Da >>>>>>> Rock<9Phackers@**herveybayaust**ralia.com.au<http://herveybayaustralia.com.au>; >>>>>>> <9Phackers@**herveybayaustralia.com.au<9Phackers@herveybayaustralia.com.au> >>>>>>> >> >>>>>>> >>>>>>> wrote: >>>>>>> >>>>>>> I may be required to move this to embedded, but I am only looking >>>>>>>> for >>>>>>>> generalisation. >>>>>>>> >>>>>>>> Recently a thread came up on questions regarding memory usage, and a >>>>>>>> post was made regarding wired memory being nearly all kernel- >>>>>>>> something >>>>>>>> I was ready to dispute, but then I thought I'd better make sure. >>>>>>>> >>>>>>>> So I tested a few theories first off: >>>>>>>> >>>>>>>> 1. Comparing memory usage across machines >>>>>>>> >>>>>>>> I checked servers and desktops as well as vm's for memory usage, >>>>>>>> and I >>>>>>>> found some interesting results. On a firewall with no apps installed >>>>>>>> only 35M wired is used, on a desktop up to 700M~ can be used. >>>>>>>> Even as a >>>>>>>> dedicated server with a few services used it remains around 35M. >>>>>>>> >>>>>>>> Surely this means that the wired memory used is not just kernel? >>>>>>>> But I >>>>>>>> held off my assumptions as it was still plausible that the >>>>>>>> structures >>>>>>>> used by the kernel could balloon that far, too. >>>>>>>> >>>>>>>> 2. Stripped down, lean mean, kernel machine >>>>>>>> >>>>>>>> I then (using a vm I was building a kernel for anyway) stripped >>>>>>>> down a >>>>>>>> kernel in a VBox VM using le drivers for network to see what >>>>>>>> could be >>>>>>>> achieved. This is my kernel conf: >>>>>>>> >>>>>>>> include GENERIC >>>>>>>> ident VPN >>>>>>>> options IPSEC >>>>>>>> options IPSEC_DEBUG >>>>>>>> options IPSEC_NAT_T >>>>>>>> device crypto >>>>>>>> device enc >>>>>>>> >>>>>>>> # minimise kernel >>>>>>>> nooptions UFS_GJOURNAL >>>>>>>> nooptions MD_ROOT >>>>>>>> nooptions NFSCL >>>>>>>> nooptions NFSD >>>>>>>> nooptions NFSLOCKD >>>>>>>> nooptions NFS_ROOT >>>>>>>> nooptions MSDOSFS >>>>>>>> nooptions CD9660 >>>>>>>> nooptions PROCFS >>>>>>>> nooptions PSEUDOFS >>>>>>>> nodevice fdc >>>>>>>> nodevice mvs >>>>>>>> nodevice siis >>>>>>>> nodevice ahc >>>>>>>> nodevice ahd >>>>>>>> nodevice esp >>>>>>>> nodevice hptiop >>>>>>>> nodevice isp >>>>>>>> nodevice mpt >>>>>>>> nodevice mps >>>>>>>> nodevice sym >>>>>>>> nodevice trm >>>>>>>> nodevice adv >>>>>>>> nodevice adw >>>>>>>> nodevice aic >>>>>>>> nodevice bt >>>>>>>> nodevice ses >>>>>>>> nodevice amr >>>>>>>> nodevice arcmsr >>>>>>>> nodevice ciss >>>>>>>> nodevice dpt >>>>>>>> nodevice hptmv >>>>>>>> nodevice hptrr >>>>>>>> nodevice irr >>>>>>>> nodevice ips >>>>>>>> nodevice mly >>>>>>>> nodevice twa >>>>>>>> nodevice aac >>>>>>>> nodevice aacp >>>>>>>> nodevice ida >>>>>>>> nodevice mfi >>>>>>>> nodevice mlx >>>>>>>> nodevice twe >>>>>>>> nodevice tws >>>>>>>> nodevice splash >>>>>>>> nodevice cbb >>>>>>>> nodevice pccard >>>>>>>> nodevice cardbus >>>>>>>> nodevice uart >>>>>>>> nodevice ppc >>>>>>>> nodevice ppbus >>>>>>>> nodevice lpt >>>>>>>> nodevice plip >>>>>>>> nodevice ppi >>>>>>>> nodevice puc >>>>>>>> nodevice bxe >>>>>>>> nodevice de >>>>>>>> nodevice em >>>>>>>> nodevice igb >>>>>>>> nodevice ixgbe >>>>>>>> nodevice ti >>>>>>>> nodevice txp >>>>>>>> nodevice vx >>>>>>>> nodevice miibus >>>>>>>> nodevice ae >>>>>>>> nodevice age >>>>>>>> nodevice alc >>>>>>>> nodevice ale >>>>>>>> nodevice bce >>>>>>>> nodevice bfe >>>>>>>> nodevice bge >>>>>>>> nodevice dc >>>>>>>> nodevice et >>>>>>>> nodevice fxp >>>>>>>> nodevice jme >>>>>>>> nodevice lge >>>>>>>> nodevice msk >>>>>>>> nodevice nfe >>>>>>>> nodevice nge >>>>>>>> nodevice pcn >>>>>>>> nodevice re >>>>>>>> nodevice rl >>>>>>>> nodevice sf >>>>>>>> nodevice sge >>>>>>>> nodevice sis >>>>>>>> nodevice sk >>>>>>>> nodevice ste >>>>>>>> nodevice stge >>>>>>>> nodevice tl >>>>>>>> nodevice tx >>>>>>>> nodevice vge >>>>>>>> nodevice vr >>>>>>>> nodevice wb >>>>>>>> nodevice xl >>>>>>>> nodevice cs >>>>>>>> nodevice ed >>>>>>>> nodevice ex >>>>>>>> nodevice ep >>>>>>>> nodevice fe >>>>>>>> nodevice sn >>>>>>>> nodevice xe >>>>>>>> nodevice wlan >>>>>>>> nooptions IEEE80211_DEBUG >>>>>>>> nooptions IEEE80211_AMPDU_AGE >>>>>>>> nooptions IEEE80211_SUPPORT_MESH >>>>>>>> nodevice wlan_wep >>>>>>>> nodevice wlan_ccmp >>>>>>>> nodevice wlan_tkip >>>>>>>> nodevice wlan_amrr >>>>>>>> nodevice an >>>>>>>> nodevice ath >>>>>>>> nodevice ath_pci >>>>>>>> nodevice ath_hal >>>>>>>> nooptions AH_SUPPORT_AR5416 >>>>>>>> nodevice ath_rate_sample >>>>>>>> nodevice ipw >>>>>>>> nodevice iwi >>>>>>>> nodevice iwn >>>>>>>> nodevice malo >>>>>>>> nodevice mwl >>>>>>>> nodevice ral >>>>>>>> nodevice wi >>>>>>>> nodevice wpi >>>>>>>> nodevice md >>>>>>>> nooption USB_DEBUG >>>>>>>> nodevice uhci >>>>>>>> nodevice ohci >>>>>>>> nodevice ehci >>>>>>>> nodevice xhci >>>>>>>> nodevice usb >>>>>>>> nodevice uhid >>>>>>>> nodevice ukbd >>>>>>>> nodevice ulpt >>>>>>>> nodevice umass >>>>>>>> nodevice ums >>>>>>>> nodevice urio >>>>>>>> nodevice u3g >>>>>>>> nodevice uark >>>>>>>> nodevice ubsa >>>>>>>> nodevice uftdi >>>>>>>> nodevice uipaq >>>>>>>> nodevice uplcom >>>>>>>> nodevice uslcom >>>>>>>> nodevice uvisor >>>>>>>> nodevice uvscom >>>>>>>> nodevice aue >>>>>>>> nodevice axe >>>>>>>> nodevice cdce >>>>>>>> nodevice cue >>>>>>>> nodevice kue >>>>>>>> nodevice rue >>>>>>>> nodevice udav >>>>>>>> nodevice rum >>>>>>>> nodevice run >>>>>>>> nodevice uath >>>>>>>> nodevice upgt >>>>>>>> nodevice ural >>>>>>>> nodevice urtw >>>>>>>> nodevice zyd >>>>>>>> #nodevice firewire >>>>>>>> nodevice fwe >>>>>>>> nodevice fwip >>>>>>>> #nodevice dcons >>>>>>>> #nodevice dcons_rom >>>>>>>> nodevice sound >>>>>>>> nodevice snd_es137x >>>>>>>> nodevice snd_hda >>>>>>>> nodevice snd_ich >>>>>>>> nodevice snd_uaudio >>>>>>>> nodevice snd_via8233 >>>>>>>> >>>>>>>> World was also rebuilt as recommended by the handbook. As you can >>>>>>>> see I >>>>>>>> was rebuilding for IPSEC (for testing purposes only note). I >>>>>>>> couldn't >>>>>>>> remove dcons (not sure why- missing defined references), and that >>>>>>>> pulled >>>>>>>> in firewire too. >>>>>>>> >>>>>>>> The result was surprising: a 14M kernel became 6.3M, and when >>>>>>>> running I >>>>>>>> found it only used 19M used for wired (whoopee!) as opposed to >>>>>>>> 35M. No >>>>>>>> services were running per se, only the usual suspects in base: >>>>>>>> >>>>>>>> vpn-test# ps ax >>>>>>>> PID TT STAT TIME COMMAND >>>>>>>> 0 ?? DLs 0:00.11 [kernel] >>>>>>>> 1 ?? ILs 0:00.01 /sbin/init -- >>>>>>>> 2 ?? DL 0:00.00 [crypto] >>>>>>>> 3 ?? DL 0:00.00 [crypto returns] >>>>>>>> 4 ?? DL 0:00.00 [sctp_iterator] >>>>>>>> 5 ?? DL 0:00.00 [xpt_thrd] >>>>>>>> 6 ?? DL 0:00.10 [pagedaemon] >>>>>>>> 7 ?? DL 0:00.00 [vmdaemon] >>>>>>>> 8 ?? DL 0:00.01 [pagezero] >>>>>>>> 9 ?? DL 0:00.30 [bufdaemon] >>>>>>>> 10 ?? DL 0:00.00 [audit] >>>>>>>> 11 ?? RL 589:34.00 [idle] >>>>>>>> 12 ?? WL 0:38.64 [intr] >>>>>>>> 13 ?? DL 0:12.21 [geom] >>>>>>>> 14 ?? DL 0:03.30 [yarrow] >>>>>>>> 15 ?? DL 0:04.40 [syncer] >>>>>>>> 16 ?? DL 0:00.63 [vnlru] >>>>>>>> 17 ?? DL 0:00.53 [softdepflush] >>>>>>>> 104 ?? Is 0:00.00 adjkerntz -i >>>>>>>> 582 ?? Is 0:00.00 /sbin/devd >>>>>>>> 737 ?? Ss 0:00.09 /usr/sbin/syslogd -s >>>>>>>> 912 ?? Ss 0:02.68 /usr/sbin/ntpd -c /etc/ntp.conf -p >>>>>>>> /var/run/ntpd.pid >>>>>>>> 977 ?? Is 0:00.00 /usr/sbin/sshd >>>>>>>> 984 ?? Ss 0:00.81 sendmail: accepting connections (sendmail) >>>>>>>> 988 ?? Is 0:00.01 sendmail: Queue runner@00:30:00 for >>>>>>>> /var/spool/client >>>>>>>> 994 ?? Ss 0:00.12 /usr/sbin/cron -s >>>>>>>> 2061 ?? Is 0:00.03 sshd: admin [priv] (sshd) >>>>>>>> 2064 ?? S 0:00.04 sshd: admin@pts/0 (sshd) >>>>>>>> 1052 v0 Is 0:00.01 login [pam] (login) >>>>>>>> 1060 v0 I+ 0:00.02 -csh (csh) >>>>>>>> 1053 v1 Is+ 0:00.00 /usr/libexec/getty Pc ttyv1 >>>>>>>> 1054 v2 Is+ 0:00.00 /usr/libexec/getty Pc ttyv2 >>>>>>>> 1055 v3 Is+ 0:00.00 /usr/libexec/getty Pc ttyv3 >>>>>>>> 1056 v4 Is+ 0:00.00 /usr/libexec/getty Pc ttyv4 >>>>>>>> 1057 v5 Is+ 0:00.00 /usr/libexec/getty Pc ttyv5 >>>>>>>> 1058 v6 Is+ 0:00.00 /usr/libexec/getty Pc ttyv6 >>>>>>>> 1059 v7 Is+ 0:00.00 /usr/libexec/getty Pc ttyv7 >>>>>>>> 2065 0 Is 0:00.02 -csh (csh) >>>>>>>> 2068 0 I 0:00.01 su >>>>>>>> 2069 0 S 0:00.04 _su (csh) >>>>>>>> 2414 0 R+ 0:00.00 ps ax >>>>>>>> >>>>>>>> After some minutes though, the usage went up to 39M. WTF? >>>>>>>> >>>>>>>> So I'm now thoroughly confused. Google didn't really show up any >>>>>>>> real >>>>>>>> answers either. Aside from discovering the true nature of what was >>>>>>>> discussed in the posts, my hypothetical theorising is for my own >>>>>>>> academics as well; and I'm trying to slim down my kernels to see >>>>>>>> if I >>>>>>>> can improve performance with low memory systems. >>>>>>>> >>>>>>>> Can process structures cause this much ballooning? I find that >>>>>>>> hard to >>>>>>>> believe given not much is running here. My only other thought >>>>>>>> would be >>>>>>>> pipe or network structures? Anything I'm missing? >>>>>>>> >>>>>>>> FWIW, my trimmed down, slim and slender, aerodynamic kernel can >>>>>>>> boot in >>>>>>>> seconds as opposed to minutes. Sweet! Although compiling clang took >>>>>>>> years... and gcc was built as well :/ >>>>>>>> >>>>>>>> Cheers >>>>>>>> ______________________________****_________________ >>>>>>>> freebsd-hackers@freebsd.org mailing list >>>>>>>> http://lists.freebsd.org/****mailman/listinfo/freebsd-****hackers<http://lists.freebsd.org/**mailman/listinfo/freebsd-**hackers>; >>>>>>>> <http://lists.freebsd.**org/mailman/listinfo/freebsd-**hackers<http://lists.freebsd.org/mailman/listinfo/freebsd-hackers>; >>>>>>>> > >>>>>>>> >>>>>>>> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@**** >>>>>>>> freebsd.org<freebsd-hackers-**unsubscribe@freebsd.org<freebsd-hackers-unsubscribe@freebsd.org> >>>>>>>> >" >>>>>>>> >>>>>>>> ______________________________****_________________ >>>>>>>> >>>>>>> freebsd-hackers@freebsd.org mailing list >>>>>>> http://lists.freebsd.org/****mailman/listinfo/freebsd-****hackers<http://lists.freebsd.org/**mailman/listinfo/freebsd-**hackers>; >>>>>>> <http://lists.freebsd.**org/mailman/listinfo/freebsd-**hackers<http://lists.freebsd.org/mailman/listinfo/freebsd-hackers>; >>>>>>> > >>>>>>> >>>>>>> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@**** >>>>>>> freebsd.org<freebsd-hackers-**unsubscribe@freebsd.org<freebsd-hackers-unsubscribe@freebsd.org> >>>>>>> >" >>>>>>> >>>>>>> >>>>>> ______________________________****_________________ >>>>>> freebsd-hackers@freebsd.org mailing list >>>>>> http://lists.freebsd.org/****mailman/listinfo/freebsd-****hackers<http://lists.freebsd.org/**mailman/listinfo/freebsd-**hackers>; >>>>>> <http://lists.freebsd.**org/mailman/listinfo/freebsd-**hackers<http://lists.freebsd.org/mailman/listinfo/freebsd-hackers>; >>>>>> > >>>>>> >>>>>> To unsubscribe, send any mail to >>>>>> "freebsd-hackers-unsubscribe@**freebsd.org<freebsd-hackers-unsubscribe@freebsd.org> >>>>>> " >>>>>> >>>>>> I'm fowarding this thread in the event the individual, "da Rock" >>>>>> will be >>>>>> able to receive more help. >>>>>> ______________________________**_________________ >>>>>> freebsd-emulation@freebsd.org mailing list >>>>>> http://lists.freebsd.org/**mailman/listinfo/freebsd-**emulation<http://lists.freebsd.org/mailman/listinfo/freebsd-emulation>; >>>>>> To unsubscribe, send any mail to >>>>>> "freebsd-emulation-**unsubscribe@freebsd.org<freebsd-emulation-unsubscribe@freebsd.org> >>>>>> " >>>>>> >>>>>> >>>>>> ----- >>>>>> No virus found in this message. >>>>>> Checked by AVG - www.avg.com >>>>>> Version: 2012.0.1913 / Virus Database: 2114/4871 - Release Date: >>>>>> 03/14/12 >>>>>> >>>>> >>>>> >>>>> >>>> >>>> ----- >>>> No virus found in this message. >>>> Checked by AVG - www.avg.com >>>> Version: 2012.0.1913 / Virus Database: 2114/4871 - Release Date: >>>> 03/14/12 >>>> >>> >>> ______________________________**_________________ >>> freebsd-emulation@freebsd.org mailing list >>> http://lists.freebsd.org/**mailman/listinfo/freebsd-**emulation<http://lists.freebsd.org/mailman/listinfo/freebsd-emulation>; >>> To unsubscribe, send any mail to >>> "freebsd-emulation-**unsubscribe@freebsd.org<freebsd-emulation-unsubscribe@freebsd.org> >>> " >>> >> >> ______________________________**_________________ >> freebsd-emulation@freebsd.org mailing list >> http://lists.freebsd.org/**mailman/listinfo/freebsd-**emulation<http://lists.freebsd.org/mailman/listinfo/freebsd-emulation>; >> To unsubscribe, send any mail to >> "freebsd-emulation-**unsubscribe@freebsd.org<freebsd-emulation-unsubscribe@freebsd.org> >> " >> >> >> ----- >> No virus found in this message. >> Checked by AVG - www.avg.com >> Version: 2012.0.1913 / Virus Database: 2114/4875 - Release Date: 03/16/12 >> > > ______________________________**_________________ > freebsd-emulation@freebsd.org mailing list > http://lists.freebsd.org/**mailman/listinfo/freebsd-**emulation<http://lists.freebsd.org/mailman/listinfo/freebsd-emulation>; > To unsubscribe, send any mail to "freebsd-emulation-** > unsubscribe@freebsd.org <freebsd-emulation-unsubscribe@freebsd.org>" >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CA%2BWntOuyngb=cpTRyx_EjBjJ=mBOR6JuKX-PEDe%2BKNEJpu_scw>