Date: Mon, 12 Mar 2012 10:52:54 +1000 From: Da Rock <9Phackers@herveybayaustralia.com.au> To: "hackers@freebsd.org" <hackers@freebsd.org> Subject: Kernel memory usage Message-ID: <4F5D48E6.9010802@herveybayaustralia.com.au>
next in thread | raw e-mail | index | archive | help
I may be required to move this to embedded, but I am only looking for
generalisation.
Recently a thread came up on questions regarding memory usage, and a
post was made regarding wired memory being nearly all kernel- something
I was ready to dispute, but then I thought I'd better make sure.
So I tested a few theories first off:
1. Comparing memory usage across machines
I checked servers and desktops as well as vm's for memory usage, and I
found some interesting results. On a firewall with no apps installed
only 35M wired is used, on a desktop up to 700M~ can be used. Even as a
dedicated server with a few services used it remains around 35M.
Surely this means that the wired memory used is not just kernel? But I
held off my assumptions as it was still plausible that the structures
used by the kernel could balloon that far, too.
2. Stripped down, lean mean, kernel machine
I then (using a vm I was building a kernel for anyway) stripped down a
kernel in a VBox VM using le drivers for network to see what could be
achieved. This is my kernel conf:
include GENERIC
ident VPN
options IPSEC
options IPSEC_DEBUG
options IPSEC_NAT_T
device crypto
device enc
# minimise kernel
nooptions UFS_GJOURNAL
nooptions MD_ROOT
nooptions NFSCL
nooptions NFSD
nooptions NFSLOCKD
nooptions NFS_ROOT
nooptions MSDOSFS
nooptions CD9660
nooptions PROCFS
nooptions PSEUDOFS
nodevice fdc
nodevice mvs
nodevice siis
nodevice ahc
nodevice ahd
nodevice esp
nodevice hptiop
nodevice isp
nodevice mpt
nodevice mps
nodevice sym
nodevice trm
nodevice adv
nodevice adw
nodevice aic
nodevice bt
nodevice ses
nodevice amr
nodevice arcmsr
nodevice ciss
nodevice dpt
nodevice hptmv
nodevice hptrr
nodevice irr
nodevice ips
nodevice mly
nodevice twa
nodevice aac
nodevice aacp
nodevice ida
nodevice mfi
nodevice mlx
nodevice twe
nodevice tws
nodevice splash
nodevice cbb
nodevice pccard
nodevice cardbus
nodevice uart
nodevice ppc
nodevice ppbus
nodevice lpt
nodevice plip
nodevice ppi
nodevice puc
nodevice bxe
nodevice de
nodevice em
nodevice igb
nodevice ixgbe
nodevice ti
nodevice txp
nodevice vx
nodevice miibus
nodevice ae
nodevice age
nodevice alc
nodevice ale
nodevice bce
nodevice bfe
nodevice bge
nodevice dc
nodevice et
nodevice fxp
nodevice jme
nodevice lge
nodevice msk
nodevice nfe
nodevice nge
nodevice pcn
nodevice re
nodevice rl
nodevice sf
nodevice sge
nodevice sis
nodevice sk
nodevice ste
nodevice stge
nodevice tl
nodevice tx
nodevice vge
nodevice vr
nodevice wb
nodevice xl
nodevice cs
nodevice ed
nodevice ex
nodevice ep
nodevice fe
nodevice sn
nodevice xe
nodevice wlan
nooptions IEEE80211_DEBUG
nooptions IEEE80211_AMPDU_AGE
nooptions IEEE80211_SUPPORT_MESH
nodevice wlan_wep
nodevice wlan_ccmp
nodevice wlan_tkip
nodevice wlan_amrr
nodevice an
nodevice ath
nodevice ath_pci
nodevice ath_hal
nooptions AH_SUPPORT_AR5416
nodevice ath_rate_sample
nodevice ipw
nodevice iwi
nodevice iwn
nodevice malo
nodevice mwl
nodevice ral
nodevice wi
nodevice wpi
nodevice md
nooption USB_DEBUG
nodevice uhci
nodevice ohci
nodevice ehci
nodevice xhci
nodevice usb
nodevice uhid
nodevice ukbd
nodevice ulpt
nodevice umass
nodevice ums
nodevice urio
nodevice u3g
nodevice uark
nodevice ubsa
nodevice uftdi
nodevice uipaq
nodevice uplcom
nodevice uslcom
nodevice uvisor
nodevice uvscom
nodevice aue
nodevice axe
nodevice cdce
nodevice cue
nodevice kue
nodevice rue
nodevice udav
nodevice rum
nodevice run
nodevice uath
nodevice upgt
nodevice ural
nodevice urtw
nodevice zyd
#nodevice firewire
nodevice fwe
nodevice fwip
#nodevice dcons
#nodevice dcons_rom
nodevice sound
nodevice snd_es137x
nodevice snd_hda
nodevice snd_ich
nodevice snd_uaudio
nodevice snd_via8233
World was also rebuilt as recommended by the handbook. As you can see I
was rebuilding for IPSEC (for testing purposes only note). I couldn't
remove dcons (not sure why- missing defined references), and that pulled
in firewire too.
The result was surprising: a 14M kernel became 6.3M, and when running I
found it only used 19M used for wired (whoopee!) as opposed to 35M. No
services were running per se, only the usual suspects in base:
vpn-test# ps ax
PID TT STAT TIME COMMAND
0 ?? DLs 0:00.11 [kernel]
1 ?? ILs 0:00.01 /sbin/init --
2 ?? DL 0:00.00 [crypto]
3 ?? DL 0:00.00 [crypto returns]
4 ?? DL 0:00.00 [sctp_iterator]
5 ?? DL 0:00.00 [xpt_thrd]
6 ?? DL 0:00.10 [pagedaemon]
7 ?? DL 0:00.00 [vmdaemon]
8 ?? DL 0:00.01 [pagezero]
9 ?? DL 0:00.30 [bufdaemon]
10 ?? DL 0:00.00 [audit]
11 ?? RL 589:34.00 [idle]
12 ?? WL 0:38.64 [intr]
13 ?? DL 0:12.21 [geom]
14 ?? DL 0:03.30 [yarrow]
15 ?? DL 0:04.40 [syncer]
16 ?? DL 0:00.63 [vnlru]
17 ?? DL 0:00.53 [softdepflush]
104 ?? Is 0:00.00 adjkerntz -i
582 ?? Is 0:00.00 /sbin/devd
737 ?? Ss 0:00.09 /usr/sbin/syslogd -s
912 ?? Ss 0:02.68 /usr/sbin/ntpd -c /etc/ntp.conf -p
/var/run/ntpd.pid
977 ?? Is 0:00.00 /usr/sbin/sshd
984 ?? Ss 0:00.81 sendmail: accepting connections (sendmail)
988 ?? Is 0:00.01 sendmail: Queue runner@00:30:00 for
/var/spool/client
994 ?? Ss 0:00.12 /usr/sbin/cron -s
2061 ?? Is 0:00.03 sshd: admin [priv] (sshd)
2064 ?? S 0:00.04 sshd: admin@pts/0 (sshd)
1052 v0 Is 0:00.01 login [pam] (login)
1060 v0 I+ 0:00.02 -csh (csh)
1053 v1 Is+ 0:00.00 /usr/libexec/getty Pc ttyv1
1054 v2 Is+ 0:00.00 /usr/libexec/getty Pc ttyv2
1055 v3 Is+ 0:00.00 /usr/libexec/getty Pc ttyv3
1056 v4 Is+ 0:00.00 /usr/libexec/getty Pc ttyv4
1057 v5 Is+ 0:00.00 /usr/libexec/getty Pc ttyv5
1058 v6 Is+ 0:00.00 /usr/libexec/getty Pc ttyv6
1059 v7 Is+ 0:00.00 /usr/libexec/getty Pc ttyv7
2065 0 Is 0:00.02 -csh (csh)
2068 0 I 0:00.01 su
2069 0 S 0:00.04 _su (csh)
2414 0 R+ 0:00.00 ps ax
After some minutes though, the usage went up to 39M. WTF?
So I'm now thoroughly confused. Google didn't really show up any real
answers either. Aside from discovering the true nature of what was
discussed in the posts, my hypothetical theorising is for my own
academics as well; and I'm trying to slim down my kernels to see if I
can improve performance with low memory systems.
Can process structures cause this much ballooning? I find that hard to
believe given not much is running here. My only other thought would be
pipe or network structures? Anything I'm missing?
FWIW, my trimmed down, slim and slender, aerodynamic kernel can boot in
seconds as opposed to minutes. Sweet! Although compiling clang took
years... and gcc was built as well :/
Cheers
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4F5D48E6.9010802>