From owner-freebsd-questions Sat Apr 15 19:16:35 2000 Delivered-To: freebsd-questions@freebsd.org Received: from www.bellnetworks.net (www.bellnetworks.net [208.177.187.234]) by hub.freebsd.org (Postfix) with ESMTP id BA3B337B8A0; Sat, 15 Apr 2000 19:16:14 -0700 (PDT) (envelope-from jerry@bellnetworks.net) Received: from itbell (dhcp-247.bellnetworks.net [208.177.187.247]) by www.bellnetworks.net (8.9.3/8.9.3) with SMTP id WAA44384; Sat, 15 Apr 2000 22:16:32 -0400 (EDT) (envelope-from jerry@bellnetworks.net) Message-ID: <008c01bfa749$aca13340$f7bbb1d0@netrex.com> From: "Jerry Bell" To: Cc: Subject: ps, top, et. al problems with 4.0 stable Date: Sat, 15 Apr 2000 22:15:45 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2314.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300 Sender: owner-freebsd-questions@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG This is somewhat long - I apologize in advance. db# top kvm_open: proc size mismatch (28188 total, 1056 chunks) top: Out of memory. db# ps -x ps: proc size mismatch (24012 total, 1056 chunks) db# Summary of my findings (detail follows): Starting with a fresh 4.0 RELEASE installation. cvsup 4.0 stable source. make world, make and install new kernel. The 4.0 stable copy of libkvm appears to not work on my system (built today from source supped today). The 4.0 release copy of libkvm does work. The 4.0 stable copy of top does not work on my system (built today from source supped today). The 4.0 release copy of top does work. The 4.0 stable copy of ps does not work on my system (built today from source supped today). The 4.0 release copy of ps does work. The 4.0 stable copy of iostat does not work on my system (built today from source supped today). The 4.0 release copy of iostat also does not work. This *is not* due to an inconsistancy between the binaries and the kernel. (Please don't email me telling me to build a new kernel) I have rebuilt and installed libkvm. I still get the same problem. I've done this about 10 times on 2 different computers with the exact same results. I have run mergemaster - it did not help. I did copy the new MAKEDEV in and remade all devices. I have tried copying /dev/null on top of utmp and wtmp. I have tried removing procfs from the kernel (same problem) I have tried removing procfs from fstab, reboot, add it back, reboot (same problem). I am REALLY REALLY sure that I am running on the new kernel. I am REALLY REALLY sure that I am running the new binaries. I am sorry if I'm coming off as rude. I posted an email a few weeks ago about the same problem and got about 50 responses telling me to build a new kernel and to make sure I wasn't running an old copy of ps. (the old copy works, anyway) I really need some help getting this straightened out. Any ideas? Here's the situation: I installed 4.0 release on a box. It works wonderfully. I cvsup the source with the following supfile (supped 4/15/2000): *default host=cvsup4.FreeBSD.org *default base=/usr *default prefix=/usr *default release=cvs tag=RELENG_4 *default delete use-rel-suffix src-all Now, I do the following: cd /usr/src make world -DNOGAMES -DNOAOUT -DNOPROFILE -DNOINFO cd /usr/src/sys/i386/conf config GENERIC cd ../../compile/GENERIC make depend make make install reboot After it boots, running on the new kernel, most everything is good, except ps, top and iostat. Here is an example of what I am seeing: db# top kvm_open: proc size mismatch (28188 total, 1056 chunks) top: Out of memory. db# ps -x ps: proc size mismatch (24012 total, 1056 chunks) db# iostat 1 1000 tty da0 da0 cpu tin tout KB/t tps MB/s KB/t tps MB/s us ni sy in id 0 323 0.00 0 0.00 0.00 0 0.00 0 0 0 0100 0 60 0.00 0 0.00 0.00 0 0.00 0 0 0 0100 351121 0.00 0 0.00 0.00 0 0.00 3 0 6 1 90 074525 0.00 4253280890 0.00 0.00 4253280890 0.00 4 0 5 2 90 046553 0.00 0 0.00 0.00 0 0.00 2 0 8 2 88 129533 0.00 18263922208111388672 0.00 0.00 18263922208111388672 0.00 5 0 9 0 86 0 93 3.81 901459873129 3355227269.53 3.81 901459873129 3355227269.53 0 0 1 0 99 0 93 0.00 0 3255805582.43 0.00 0 3255805582.43 0 0 0 0100 0 75 0.00 0 431896853.51 0.00 0 431896853.51 0 0 0 0100 0 73 0.00 0 0.00 0.00 0 0.00 0 0 0 0100 0 60 0.00 0 0.00 0.00 0 0.00 0 0 0 0100 0 60 0.00 0 0.00 0.00 0 0.00 0 0 0 0100 ^C db# Here is the kernel I'm running: db# ls -l / total 7514 -rw-r--r-- 2 root wheel 574 Mar 20 21:32 .cshrc -rw-r--r-- 2 root wheel 251 Mar 20 21:32 .profile -r--r--r-- 1 root wheel 4735 Mar 20 21:32 COPYRIGHT drwxr-xr-x 2 root wheel 1024 Apr 16 00:13 bin drwxr-xr-x 3 root wheel 512 Apr 9 01:18 boot lrwxr-xr-x 1 root wheel 11 Apr 9 01:19 compat -> /usr/compat drwxr-xr-x 3 root wheel 12288 Apr 16 00:51 dev drwxr-xr-x 15 root wheel 2048 Apr 16 01:11 etc lrwxr-xr-x 1 root wheel 9 Apr 9 01:32 home -> /usr/home -r-xr-xr-x 1 root wheel 2574609 Apr 16 00:48 kernel -rwxr-xr-x 1 root wheel 2777025 Mar 20 22:50 kernel.GENERIC -rwxr-xr-x 1 root wheel 2274463 Apr 8 08:16 kernel.old drwxr-xr-x 2 root wheel 512 Mar 20 21:26 mnt drwxr-xr-x 2 root wheel 2048 Apr 9 00:46 modules dr-xr-xr-x 1 root wheel 512 Apr 16 02:04 proc drwxr-xr-x 2 root wheel 512 Apr 9 01:33 root drwxr-xr-x 2 root wheel 2048 Apr 16 00:17 sbin drwxr-xr-x 4 root wheel 512 Apr 9 00:42 stand lrwxr-xr-x 1 root wheel 11 Apr 16 00:11 sys -> usr/src/sys drwxrwxrwt 2 root wheel 512 Apr 16 00:52 tmp drwxr-xr-x 18 root wheel 512 Apr 16 00:11 usr drwxr-xr-x 19 root wheel 512 Apr 15 15:55 var db# uname -a FreeBSD db.bellnetworks.net 4.0-STABLE FreeBSD 4.0-STABLE #0: Sun Apr 16 00:47:48 GMT 2000 jbell@db.bellnetworks.net:/usr/src/sys/compile/GENERIC i386 db# ls -l /bin/ps /usr/sbin/iostat /usr/bin/top -r-xr-xr-x 1 root wheel 222648 Apr 16 00:13 /bin/ps -r-xr-sr-x 1 root kmem 34240 Apr 16 02:08 /usr/bin/top -r-xr-sr-x 1 root kmem 10084 Apr 16 02:10 /usr/sbin/iostat All of these programs functioned normally under 4.0 release on this system. So, after some research, I find that libkvm is probably the culprit behind top not working. Several threads in the mail archives suggest building and installing libkvm. So I go to the libkvm directory under /usr/src, run make clean, make all, make install. Same thing. Ahhh, but since I've now rebuilt this computer approximately 10000000 times trying to solve this problem, I know to keep a backup copy of all the 4.0 release binaries :) So, I copy the 4.0 RELEASE (yes 4.0 release) version of /usr/lib/libkvm* back in. Guess what? top works! But the output looks like this: last pid: 528; load averages: 0.00, 0.00, 0.00 up 0+01:22:09 02:13:14 4 processes: 1 starting, 1 running CPU states: % user, % nice, % system, % interrupt, % idle Mem: 5908K Active, 28M Inact, 12M Wired, 52K Cache, 29M Buf, 141M Free Swap: 400M Total, 400M Free PID USERNAME PRI NICE SIZE RES STATE TIME WCPU CPU COMMAND 0 news -22 52481 0K 0K ? 20 57:18 1429.93% 1429.93% <> 0 daemon -22 52489 3279M 513G ? -32 0:00 0.00% 138.23% <> 528 root 30 0 0K 32K RUN 0:00 1.00% 0.05% top 0 root -22 -52 0K 0K START 0:00 0.00% 157443857.81% <> If I run the 4.0 release top binary with the 4.0 release libkvm's, top works properly. Unfortunately, ps and iostat (and who knows what else) still doesn't work - same problem. Ok, so now I focus on ps. This is the message I get: db# ps -x ps: proc size mismatch (26100 total, 1056 chunks) db# Ok, so lets to a bit of research: db# truss /bin/ps syscall ioctl(1,TIOCGWINSZ,0xbfbfefe0) returns 0 (0x0) syscall readlink("/etc/malloc.conf",0xbfbfeeb8,63) errno 2 'No such file or directory' syscall mmap(0x0,4096,0x3,0x1002,-1,0x0) returns 671588352 (0x2807a000) syscall break(0x8087000) returns 0 (0x0) syscall break(0x8088000) returns 0 (0x0) syscall open("/dev/null",0,00) returns 3 (0x3) syscall fstat(3,0xbfbfeeb8) returns 0 (0x0) syscall open("/dev/null",0,00) returns 4 (0x4) syscall break(0x8089000) returns 0 (0x0) syscall getuid() returns 0 (0x0) syscall __sysctl(0xbfbfef58,0x4,0x0,0xbfbfef24,0x0,0x0) returns 0 (0x0) syscall break(0x8092000) returns 0 (0x0) syscall __sysctl(0xbfbfef58,0x4,0x8089000,0xbfbfef24,0x0,0x0) returns 0 (0x0) ps: syscall write(2,0xbfbfe8b8,4) returns 4 (0x4) proc size mismatch (27144 total, 1056 chunks)syscall write(2,0xbfbfe8d8,45) returns 45 (0x2d) syscall write(2,0xbfbfe8b8,1) returns 1 (0x1) syscall exit(0x1) process exit, rval = 256 db# Nothing looks out of the ordinary. But wait. I still have a copy of the 4.0 release copy of ps. lets try it: db# /var/build/tmp/bin/ps -x PID TT STAT TIME COMMAND 0 ?? DLs 0:00.00 (swapper) 1 ?? ILs 0:00.02 /sbin/init -- 2 ?? DL 0:00.00 (pagedaemon) 3 ?? DL 0:00.00 (vmdaemon) 4 ?? DL 0:00.00 (bufdaemon) 5 ?? DL 0:00.15 (syncer) 73 ?? Is 0:00.05 syslogd 94 ?? Is 0:00.03 inetd -wW 96 ?? Is 0:00.02 cron 99 ?? Is 0:00.02 sendmail: accepting connections on port 25 (sendmail) 186 ?? Ss 0:00.24 telnetd 245 ?? Is 0:00.41 telnetd 224 p0 S 0:00.08 _su (csh) 286 p0 R+ 0:00.00 /var/build/tmp/bin/ps -x 248 p1 I+ 0:00.06 _su (csh) 219 v0 Is+ 0:00.01 /usr/libexec/getty Pc ttyv0 156 v1 Is 0:00.05 -csh (csh) 216 v1 I+ 0:00.04 _su (csh) 157 v2 Is+ 0:00.01 /usr/libexec/getty Pc ttyv2 158 v3 Is+ 0:00.01 /usr/libexec/getty Pc ttyv3 159 v4 Is+ 0:00.01 /usr/libexec/getty Pc ttyv4 160 v5 Is+ 0:00.01 /usr/libexec/getty Pc ttyv5 161 v6 Is+ 0:00.01 /usr/libexec/getty Pc ttyv6 162 v7 Is+ 0:00.01 /usr/libexec/getty Pc ttyv7 134 con- I 0:00.01 /bin/sh /usr/local/bin/safe_mysqld --user=mysql db# Woah! pretty interesting. The old copy of ps works. So, lets look at this: truss /var/build/tmp/bin/ps -x syscall ioctl(1,TIOCGWINSZ,0xbfbfefdc) returns 0 (0x0) syscall readlink("/etc/malloc.conf",0xbfbfeed4,63) errno 2 'No such file or directory' syscall mmap(0x0,4096,0x3,0x1002,-1,0x0) returns 671567872 (0x28075000) syscall break(0x8083000) returns 0 (0x0) syscall break(0x8084000) returns 0 (0x0) syscall break(0x8085000) returns 0 (0x0) syscall open("/dev/null",0,00) returns 3 (0x3) syscall fstat(3,0xbfbfeec4) returns 0 (0x0) syscall open("/dev/null",0,00) returns 4 (0x4) syscall getuid() returns 0 (0x0) syscall __sysctl(0xbfbfef64,0x4,0x0,0xbfbfef30,0x0,0x0) returns 0 (0x0) syscall break(0x808e000) returns 0 (0x0) syscall __sysctl(0xbfbfef64,0x4,0x8085000,0xbfbfef30,0x0,0x0) returns 0 (0x0) syscall break(0x8090000) returns 0 (0x0) syscall __sysctl(0xbfbfee88,0x2,0xbfbfee90,0xbfbfee84,0x80712e8,0x17) returns 0 (0x0) syscall __sysctl(0xbfbfee90,0x2,0x8077d70,0xbfbfef04,0x0,0x0) returns 0 (0x0) syscall break(0x8091000) returns 0 (0x0) syscall break(0x8092000) returns 0 (0x0) syscall __sysctl(0xbfbfef08,0x4,0x8090000,0xbfbfef04,0x0,0x0) returns 0 (0x0) syscall __sysctl(0xbfbfeeac,0x2,0xbfbfeea4,0xbfbfeea8,0x0,0x0) returns 0 (0x0) syscall break(0x80d3000) returns 0 (0x0) syscall break(0x80d4000) returns 0 (0x0) syscall __sysctl(0xbfbfef08,0x4,0x8090000,0xbfbfef04,0x0,0x0) returns 0 (0x0) syscall __sysctl(0xbfbfef08,0x4,0x8090000,0xbfbfef04,0x0,0x0) returns 0 (0x0) syscall __sysctl(0xbfbfef08,0x4,0x8090000,0xbfbfef04,0x0,0x0) returns 0 (0x0) syscall __sysctl(0xbfbfef08,0x4,0x8090000,0xbfbfef04,0x0,0x0) returns 0 (0x0) syscall __sysctl(0xbfbfef08,0x4,0x8090000,0xbfbfef04,0x0,0x0) returns 0 (0x0) syscall __sysctl(0xbfbfef08,0x4,0x8090000,0xbfbfef04,0x0,0x0) returns 0 (0x0) syscall __sysctl(0xbfbfef08,0x4,0x8090000,0xbfbfef04,0x0,0x0) returns 0 (0x0) syscall __sysctl(0xbfbfef08,0x4,0x8090000,0xbfbfef04,0x0,0x0) returns 0 (0x0) syscall __sysctl(0xbfbfef08,0x4,0x8090000,0xbfbfef04,0x0,0x0) returns 0 (0x0) syscall __sysctl(0xbfbfef08,0x4,0x8090000,0xbfbfef04,0x0,0x0) returns 0 (0x0) syscall __sysctl(0xbfbfef08,0x4,0x8090000,0xbfbfef04,0x0,0x0) returns 0 (0x0) syscall __sysctl(0xbfbfef08,0x4,0x8090000,0xbfbfef04,0x0,0x0) returns 0 (0x0) syscall __sysctl(0xbfbfef08,0x4,0x8090000,0xbfbfef04,0x0,0x0) returns 0 (0x0) syscall __sysctl(0xbfbfef08,0x4,0x8090000,0xbfbfef04,0x0,0x0) returns 0 (0x0) syscall __sysctl(0xbfbfef08,0x4,0x8090000,0xbfbfef04,0x0,0x0) returns 0 (0x0) syscall __sysctl(0xbfbfef08,0x4,0x8090000,0xbfbfef04,0x0,0x0) returns 0 (0x0) syscall __sysctl(0xbfbfef08,0x4,0x8090000,0xbfbfef04,0x0,0x0) returns 0 (0x0) syscall __sysctl(0xbfbfef08,0x4,0x8090000,0xbfbfef04,0x0,0x0) returns 0 (0x0) syscall __sysctl(0xbfbfef08,0x4,0x8090000,0xbfbfef04,0x0,0x0) returns 0 (0x0) syscall __sysctl(0xbfbfef08,0x4,0x8090000,0xbfbfef04,0x0,0x0) returns 0 (0x0) syscall __sysctl(0xbfbfef08,0x4,0x8090000,0xbfbfef04,0x0,0x0) returns 0 (0x0) syscall __sysctl(0xbfbfef08,0x4,0x8090000,0xbfbfef04,0x0,0x0) returns 0 (0x0) syscall __sysctl(0xbfbfef08,0x4,0x8090000,0xbfbfef04,0x0,0x0) returns 0 (0x0) syscall __sysctl(0xbfbfef08,0x4,0x8090000,0xbfbfef04,0x0,0x0) returns 0 (0x0) syscall __sysctl(0xbfbfef08,0x4,0x8090000,0xbfbfef04,0x0,0x0) returns 0 (0x0) syscall fstat(1,0xbfbfec04) returns 0 (0x0) syscall break(0x80d5000) returns 0 (0x0) syscall ioctl(1,TIOCGETA,0xbfbfec38) returns 0 (0x0) PID TT STAT TIME COMMAND syscall write(1,0x80d4000,34) returns 34 (0x22) 0 ?? DLs 0:00.00 (swapper) syscall write(1,0x80d4000,37) returns 37 (0x25) 1 ?? ILs 0:00.02 /sbin/init -- syscall write(1,0x80d4000,40) returns 40 (0x28) 2 ?? DL 0:00.00 (pagedaemon) syscall write(1,0x80d4000,40) returns 40 (0x28) 3 ?? DL 0:00.00 (vmdaemon) syscall write(1,0x80d4000,38) returns 38 (0x26) 4 ?? DL 0:00.00 (bufdaemon) syscall write(1,0x80d4000,39) returns 39 (0x27) 5 ?? DL 0:00.16 (syncer) syscall write(1,0x80d4000,36) returns 36 (0x24) 73 ?? Ss 0:00.05 syslogd syscall write(1,0x80d4000,34) returns 34 (0x22) 94 ?? Is 0:00.03 inetd -wW syscall write(1,0x80d4000,36) returns 36 (0x24) 96 ?? Is 0:00.02 cron syscall write(1,0x80d4000,31) returns 31 (0x1f) 99 ?? Is 0:00.02 sendmail: accepting connections on port 25 (sendmail) syscall write(1,0x80d4000,80) returns 80 (0x50) 186 ?? Ss 0:00.24 telnetd syscall write(1,0x80d4000,34) returns 34 (0x22) 245 ?? Is 0:00.41 telnetd syscall write(1,0x80d4000,34) returns 34 (0x22) syscall break(0x80d6000) returns 0 (0x0) syscall stat("/var/run/dev.db",0xbfbfee24) returns 0 (0x0) syscall open("/var/run/dev.db",0,00) returns 5 (0x5) syscall fcntl(0x5,0x2,0x1) returns 0 (0x0) syscall read(0x5,0x80d5000,0x104) returns 260 (0x104) syscall break(0x80d8000) returns 0 (0x0) syscall lseek(5,0x4000,0) returns 16384 (0x4000) syscall read(0x5,0x80d6000,0x2000) returns 8192 (0x2000) 224 p0 S 0:00.08 _su (csh) syscall write(1,0x80d4000,36) returns 36 (0x24) 290 p0 R+ 0:00.01 truss /var/build/tmp/bin/ps -x syscall write(1,0x80d4000,57) returns 57 (0x39) 291 p0 R+ 0:00.00 /var/build/tmp/bin/ps -x syscall write(1,0x80d4000,51) returns 51 (0x33) syscall break(0x80da000) returns 0 (0x0) syscall lseek(5,0x2000,0) returns 8192 (0x2000) syscall read(0x5,0x80d8000,0x2000) returns 8192 (0x2000) 248 p1 I+ 0:00.06 _su (csh) syscall write(1,0x80d4000,36) returns 36 (0x24) 219 v0 Is+ 0:00.01 /usr/libexec/getty Pc ttyv0 syscall write(1,0x80d4000,54) returns 54 (0x36) 156 v1 Is 0:00.05 -csh (csh) syscall write(1,0x80d4000,37) returns 37 (0x25) 216 v1 I+ 0:00.04 _su (csh) syscall write(1,0x80d4000,36) returns 36 (0x24) 157 v2 Is+ 0:00.01 /usr/libexec/getty Pc ttyv2 syscall write(1,0x80d4000,54) returns 54 (0x36) 158 v3 Is+ 0:00.01 /usr/libexec/getty Pc ttyv3 syscall write(1,0x80d4000,54) returns 54 (0x36) 159 v4 Is+ 0:00.01 /usr/libexec/getty Pc ttyv4 syscall write(1,0x80d4000,54) returns 54 (0x36) 160 v5 Is+ 0:00.01 /usr/libexec/getty Pc ttyv5 syscall write(1,0x80d4000,54) returns 54 (0x36) 161 v6 Is+ 0:00.01 /usr/libexec/getty Pc ttyv6 syscall write(1,0x80d4000,54) returns 54 (0x36) 162 v7 Is+ 0:00.01 /usr/libexec/getty Pc ttyv7 syscall write(1,0x80d4000,54) returns 54 (0x36) 134 con- I 0:00.01 /bin/sh /usr/local/bin/safe_mysqld --user=mysql syscall write(1,0x80d4000,74) returns 74 (0x4a) syscall exit(0x0) process exit, rval = 0 db# So, the 4.0 release copy of ps works on the 4.0 stable system. The 4.0 release copy of iostat produces the same results as the 4.0 stable (see above). Thanks for your time! Jerry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message