Date: Thu, 9 Nov 2000 21:35:12 +0100 From: Gerhard Sittig <Gerhard.Sittig@gmx.net> To: FreeBSD-gnats-submit@freebsd.org Subject: kern/22723: panic in socket operation inside jails Message-ID: <20001109213512.E25237@speedy.gsinet>
next in thread | raw e-mail | index | archive | help
>Number: 22723
>Category: kern
>Synopsis: panic in socket operation inside jails
>Confidential: no
>Severity: serious
>Priority: high
>Responsible: freebsd-bugs
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: sw-bug
>Submitter-Id: current-users
>Arrival-Date: Thu Nov 09 12:50:01 PST 2000
>Closed-Date:
>Last-Modified:
>Originator: Gerhard Sittig
>Release: FreeBSD 4.2-BETA i386
>Organization:
in private
>Environment:
FreeBSD 4.2-BETA environment and networked applications inside
jails (seen with sshd and ftpd; others probably affected, too)
>Description:
short version:
sshd(8) panics the system at startup when run inside a jail with
no ListenAddress specified in sshd_config (which is the default
after installation and is perfectly legitimate since jails only
have one address).
ftpd(8) panics the system at connection establishment time when
run inside a jail without the -a option (invocation from inetd
leaves no alternative AFAICT, and again this is the default
situation after installation).
I haven't tried other services, since constant panics are no fun
when one actually just wants to install a server machine ... :)
Since all of this happens in a configuration as installed by
default (apart from building the world, following jail(8) to
setup the file hierarchy, and enabling the services) I consider
this problem to be serious. To add to this the release of 4.2-R
will be real soon.
longer description:
When setting up jails and booting, the system immediately panics
in sshd. The only cure was to specify ListenAddresses in
sshd_config. And these panics happened at sshd(8) startup time
in contrast to what I experienced with ftpd below. Maybe this
will give some clues.
Then I had further panics in ftpd (started from inetd). These
did _not_ happen when I started the daemon by hand like
"/usr/libexec/ftpd -D -a $IP" and contacted them by "telnet $IP
ftp". But as soon as "inetd -wW -a $IP" with a line "ftp ...
/usr/libexec/ftpd ftpd -l" is run, any connection with the above
telnet command or ftp(1) frontend will result in a panic. This
only went away when setting the jail.socket_unixiproute_only
sysctl to 0.
I understand that the mentioned sysctl will limit accessible
protocol families to PF_UNIX, PF_INET and PF_ROUTE (that's what I
get from reading the jail(8) manpage and remebering the commit
message). Since this sysctl is set to 1 by default, the test
condition in uipc_socket.c at lines 140f gets evaluated further
than else and tries to dereference the NULL prp pointer. Boom!
Since I don't believe in simple solutions (just move the NULL
test of prp above dereferencing it) I could very well imagine
that this is not everything to do -- I don't believe that
EPROTONOSUPPORT is the correct answer when the service is running
and tries to bind(?) to "all the interfaces the jail has" ...
Here's a sample debugging session which led me to the impression
that uipc_socket.c has the problem of following the NULL pointer
(I failed to demonstrate it below by means of "print", but prp is
known to be NULL there -- from a debugging session I didn't
record by script(1) means; this explains, too, the curious order
my research actions are in ... :).
----- script(1) snippet -----------------------------------------
Script started on Thu Nov 9 06:55:23 2000
raider# cd /usr/src/sys/kern
raider# cvs diff -rRELENG_4_1_1_RELEASE -rRELENG_4 uipc_socket.craider
Index: uipc_socket.c
===================================================================
RCS file: /home/ncvs/src/sys/kern/uipc_socket.c,v
retrieving revision 1.68.2.7
retrieving revision 1.68.2.9
diff -u -r1.68.2.7 -r1.68.2.9
--- uipc_socket.c 2000/09/07 19:13:37 1.68.2.7
+++ uipc_socket.c 2000/10/29 19:25:38 1.68.2.9
@@ -31,7 +31,7 @@
* SUCH DAMAGE.
*
* @(#)uipc_socket.c 8.3 (Berkeley) 4/15/94
- * $FreeBSD: src/sys/kern/uipc_socket.c,v 1.68.2.7 2000/09/07 19:13:37 truckman Exp $
+ * $FreeBSD: src/sys/kern/uipc_socket.c,v 1.68.2.9 2000/10/29 19:25:38 rwatson Exp $
*/
#include <sys/param.h>
@@ -53,6 +53,7 @@
#include <sys/signalvar.h>
#include <sys/sysctl.h>
#include <sys/uio.h>
+#include <sys/jail.h>
#include <vm/vm_zone.h>
#include <machine/limits.h>
@@ -135,6 +136,14 @@
prp = pffindproto(dom, proto, type);
else
prp = pffindtype(dom, type);
+
+ if (p->p_prison && jail_socket_unixiproute_only &&
+ prp->pr_domain->dom_family != PF_LOCAL &&
+ prp->pr_domain->dom_family != PF_INET &&
+ prp->pr_domain->dom_family != PF_ROUTE) {
+ return (EPROTONOSUPPORT);
+ }
+
if (prp == 0 || prp->pr_usrreqs->pru_attach == 0)
return (EPROTONOSUPPORT);
if (prp->pr_type != type)
@@ -1557,7 +1566,9 @@
kn->kn_flags |= EV_EOF;
return (1);
}
- return (kn->kn_data > 0);
+ if (so->so_error) /* temporary udp error */
+ return (1);
+ return (kn->kn_data >= so->so_rcv.sb_lowat);
}
static int
@@ -1595,6 +1606,8 @@
kn->kn_flags |= EV_EOF;
return (1);
}
+ if (so->so_error) /* temporary udp error */
+ return (1);
if (((so->so_state & SS_ISCONNECTED) == 0) &&
(so->so_proto->pr_flags & PR_CONNREQUIRED))
return (0);
raider# cd /var/crash
raider# gdb -k kernel.debug vmcore.1
GNU gdb 4.18
Copyright 1998 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-unknown-freebsd"...
IdlePTD 3985408
initial pcb at 32bf00
panicstr: page fault
panic messages:
---
Fatal trap 12: page fault while in kernel mode
fault virtual address = 0x4
fault code = supervisor read, page not present
instruction pointer = 0x8:0xc0196531
stack pointer = 0x10:0xcd441ee8
frame pointer = 0x10:0xcd441ef4
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, def32 1, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 1256 (ftpd)
interrupt mask = none
trap number = 12
panic: page fault
syncing disks... 48 48 2
done
Uptime: 1m29s
amr0: flushing cache...done
dumping to dev #ad/0x40001, offset 524288
dump ata0: resetting devices .. done
256 255 254 253 252 251 250 249 248 247 246 245 244 243 242 241 240 239 238 237 236 235 234 233 232 231 230 229 228 227 226 225 224 223 222 221 220 219 218 217 216 215 214 213 212 211 210 209 208 207 206 205 204 203 202 201 200 199 198 197 196 195 194 193 192 191 190 189 188 187 186 185 184 183 182 181 180 179 178 177 176 175 174 173 172 171 170 169 168 167 166 165 164 163 162 161 160 159 158 157 156 155 154 153 152 151 150 149 148 147 146 145 144 143 142 141 140 139 138 137 136 135 134 133 132 131 130 129 128 127 126 125 124 123 122 121 120 119 118 117 116 115 114 113 112 111 110 109 108 107 106 105 104 103 102 101 100 99 98 97 96 95 94 93 92 91 90 89 88 87 86 85 84 83 82 81 80 79 78 77 76 75 74 73 72 71 70 69 68 67 66 65 64 63 62 61 60 59 58 57 56 55 54 53 52 51 50 49 48 47 46 45 44 43 42 41 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1
---
#0 dumpsys () at /usr/src-releng4/sys/kern/kern_shutdown.c:469
469 if (dumping++) {
(kgdb) where
#0 dumpsys () at /usr/src-releng4/sys/kern/kern_shutdown.c:469
#1 0xc017a6d7 in boot (howto=256)
at /usr/src-releng4/sys/kern/kern_shutdown.c:309
#2 0xc017aa6d in panic (fmt=0xc02d0b2f "page fault")
at /usr/src-releng4/sys/kern/kern_shutdown.c:556
#3 0xc027e056 in trap_fatal (frame=0xcd441ea8, eva=4)
at /usr/src-releng4/sys/i386/i386/trap.c:951
#4 0xc027dd09 in trap_pfault (frame=0xcd441ea8, usermode=0, eva=4)
at /usr/src-releng4/sys/i386/i386/trap.c:844
#5 0xc027d8ab in trap (frame={tf_fs = 16, tf_es = 16, tf_ds = -851378160,
tf_edi = 2, tf_esi = 0, tf_ebp = -851173644, tf_isp = -851173676,
tf_ebx = 0, tf_edx = -851173592, tf_ecx = 0, tf_eax = -851466432,
tf_trapno = 12, tf_err = 0, tf_eip = -1072077519, tf_cs = 8,
tf_eflags = 66050, tf_esp = 0, tf_ss = -851173504})
at /usr/src-releng4/sys/i386/i386/trap.c:443
#6 0xc0196531 in socreate (dom=28, aso=0xcd441f20, type=2, proto=0,
p=0xcd3fa740) at /usr/src-releng4/sys/kern/uipc_socket.c:140
#7 0xc0199e0e in socket (p=0xcd3fa740, uap=0xcd441f80)
at /usr/src-releng4/sys/kern/uipc_syscalls.c:119
#8 0xc027e309 in syscall2 (frame={tf_fs = 47, tf_es = 47, tf_ds = 47,
tf_edi = -1077940392, tf_esi = 0, tf_ebp = -1077940456,
tf_isp = -851173420, tf_ebx = 672314280, tf_edx = 672307592,
tf_ecx = -1077940360, tf_eax = 97, tf_trapno = 12, tf_err = 2,
tf_eip = 672026040, tf_cs = 31, tf_eflags = 647, tf_esp = -1077940500,
tf_ss = 47}) at /usr/src-releng4/sys/i386/i386/trap.c:1150
#9 0xc0271cd5 in Xint0x80_syscall ()
#10 0x280f71b1 in ?? ()
#11 0x804b299 in ?? ()
#12 0x804aa51 in ?? ()
#13 0x804a781 in ?? ()
(kgdb) up 5
#5 0xc027d8ab in trap (frame={tf_fs = 16, tf_es = 16, tf_ds = -851378160,
tf_edi = 2, tf_esi = 0, tf_ebp = -851173644, tf_isp = -851173676,
tf_ebx = 0, tf_edx = -851173592, tf_ecx = 0, tf_eax = -851466432,
tf_trapno = 12, tf_err = 0, tf_eip = -1072077519, tf_cs = 8,
tf_eflags = 66050, tf_esp = 0, tf_ss = -851173504})
at /usr/src-releng4/sys/i386/i386/trap.c:443
443 (void) trap_pfault(&frame, FALSE, eva);
(kgdb) list
438 kernel_trap:
439 /* kernel trap */
440
441 switch (type) {
442 case T_PAGEFLT: /* page fault */
443 (void) trap_pfault(&frame, FALSE, eva);
444 return;
445
446 case T_DNA:
447 #if NNPX > 0
(kgdb) up
#6 0xc0196531 in socreate (dom=28, aso=0xcd441f20, type=2, proto=0,
p=0xcd3fa740) at /usr/src-releng4/sys/kern/uipc_socket.c:140
140 if (p->p_prison && jail_socket_unixiproute_only &&
(kgdb) list
135 if (proto)
136 prp = pffindproto(dom, proto, type);
137 else
138 prp = pffindtype(dom, type);
139
140 if (p->p_prison && jail_socket_unixiproute_only &&
141 prp->pr_domain->dom_family != PF_LOCAL &&
142 prp->pr_domain->dom_family != PF_INET &&
143 prp->pr_domain->dom_family != PF_ROUTE) {
144 return (EPROTONOSUPPORT);
(kgdb) raider#
raider# uname -a
FreeBSD raider.some.domain.de 4.2-BETA FreeBSD 4.2-BETA #0: Wed Nov 8 13:25:17 CET 2000 root@raider.some.domain.de:/usr/obj/usr/src-releng4/sys/RAIDER i386
raider# strings /kernel | grep ___ | sed 's/^___//'
# ----- /usr/src/sys/i386/conf/RAIDER ---------------------------
machine i386
cpu I586_CPU
cpu I686_CPU
ident RAIDER
maxusers 64
options INET #InterNETworking
options FFS #Berkeley Fast Filesystem
options FFS_ROOT #FFS usable as root device [keep this!]
options SOFTUPDATES #Enable FFS soft updates support
options MFS #Memory Filesystem
options MSDOSFS #MSDOS Filesystem
options CD9660 #ISO 9660 Filesystem
options PROCFS #Process filesystem
options COMPAT_43 #Compatible with BSD 4.3 [KEEP THIS!]
options SCSI_DELAY=15000 #Delay (in ms) before probing SCSI
# options UCONSOLE #Allow users to grab the console
options USERCONFIG #boot -c editor
options VISUAL_USERCONFIG #visual boot -c editor
options KTRACE #ktrace(1) support
options SYSVSHM #SYSV-style shared memory
options SYSVMSG #SYSV-style message queues
options SYSVSEM #SYSV-style semaphores
options P1003_1B #Posix P1003_1B real-time extensions
options _KPOSIX_PRIORITY_SCHEDULING
options ICMP_BANDLIM #Rate limit bad replies
options KBD_INSTALL_CDEV # install a CDEV entry in /dev
device isa
device pci
device fdc0 at isa? port IO_FD1 irq 6 drq 2
device fd0 at fdc0 drive 0
device ata0 at isa? port IO_WD1 irq 14
device ata1 at isa? port IO_WD2 irq 15
device ata
device atadisk # ATA disk drives
device atapicd # ATAPI CDROM drives
options ATA_STATIC_ID #Static device numbering
options ATA_ENABLE_ATAPI_DMA #Enable DMA on ATAPI devices
device ahc # AHA2940 and onboard AIC7xxx devices
device scbus # SCSI bus (required)
device da # Direct Access (disks)
device sa # Sequential Access (tape etc)
device cd # CD
device pass # Passthrough device (direct SCSI access)
device amr # AMI MegaRAID
device atkbdc0 at isa? port IO_KBD
device atkbd0 at atkbdc? irq 1 flags 0x1
device psm0 at atkbdc? irq 12
device vga0 at isa?
pseudo-device splash
device sc0 at isa? flags 0x100
device npx0 at nexus? port IO_NPX irq 13
# device apm0 at nexus? disable flags 0x20 # Advanced Power Management
device sio0 at isa? port IO_COM1 flags 0x10 irq 4
device sio1 at isa? port IO_COM2 irq 3
device ppc0 at isa? irq 7
device ppbus # Parallel port bus (required)
device lpt # Printer
device vpo # Requires scbus and da
device miibus # MII bus support
device rl # RealTek 8129/8139
device xl # 3Com 3c90x (``Boomerang'', ``Cyclone'')
pseudo-device loop # Network loopback
pseudo-device ether # Ethernet support
pseudo-device tun # Packet tunnel.
pseudo-device pty # Pseudo-ttys (telnet etc)
pseudo-device md # Memory "disks"
pseudo-device bpf #Berkeley packet filter
device uhci # UHCI PCI->USB interface
device ohci # OHCI PCI->USB interface
device usb # USB Bus (required)
device ugen # Generic
device uhid # "Human Interface Devices"
device ukbd # Keyboard
device ulpt # Printer
device umass # Disks/Mass storage - Requires scbus and da
device ums # Mouse
options INCLUDE_CONFIG_FILE # Include this file in kernel
options CPU_FASTER_5X86_FPU
options CPU_SUSP_HLT
options NO_F00F_HACK
options DDB
options DDB_UNATTENDED
makeoptions DEBUG=-g
options PERFMON
options IPSEC #IP security
options IPSEC_ESP #IP security (crypto; define w/ IPSEC)
options NETGRAPH #netgraph(4) system
options NETGRAPH_ASYNC
options NETGRAPH_BPF
options NETGRAPH_ECHO
options NETGRAPH_IFACE
options NETGRAPH_KSOCKET
options NETGRAPH_SOCKET
options NETGRAPH_TEE
options NETGRAPH_TTY
options NETGRAPH_UI
options IPFILTER #ipfilter support
options IPFILTER_LOG #ipfilter logging
options IPFILTER_DEFAULT_BLOCK #ipfilter logging
options ACCEPT_FILTER_DATA
options ACCEPT_FILTER_HTTP
options TCP_DROP_SYNFIN #drop TCP packets with SYN+FIN
options TCP_RESTRICT_RST #restrict emission of TCP RST
options QUOTA #enable disk quotas
options CODA #CODA filesystem.
pseudo-device vcoda 4 #coda minicache <-> venus comm.
pseudo-device speaker #Play IBM BASIC-style noises out your speaker
pseudo-device gzip #Exec gzipped a.out's
pseudo-device vn #Vnode driver (turns a file into a device)
pseudo-device ccd 4 #Concatenated disk driver
options MSGBUF_SIZE=40960
options VESA
options SC_ALT_MOUSE_IMAGE # simplified mouse cursor in text mode
options SC_HISTORY_SIZE=200 # number of history buffer lines
options SC_MOUSE_CHAR=0x3 # char code for text mode mouse cursor
options SC_PIXEL_MODE # add support for the raster text mode
options SC_TWOBUTTON_MOUSE
options IDE_DELAY=8000 # Be optimistic about Joe IDE device
device pcm
device sbc0
device pca0 at isa? port IO_TIMER1
device smbus # Bus support, required for smb below.
device intpm
device alpm
device smb
device iicbus # Bus support, required for ic/iic/iicsmb below.
device iicbb
device ic
device iic
device iicsmb # smb over i2c bridge
options PPC_PROBE_CHIPSET # Enable chipset specific detection
# options PNPBIOS
# ----- E O F ---------------------------------------------------
raider# sysctl -a | grep jail
jail.set_hostname_allowed: 0
jail.socket_unixiproute_only: 0
jail.sysvipc_allowed: 0
raider# exit
Script done on Thu Nov 9 07:00:54 2000
----- script(1) snippet -----------------------------------------
The jail.socket_unixiproute_only was set to 1 at the panic
moment, setting it to 0 (as it was when I debugged the crash
dump) made the setup work as mentioned above.
>How-To-Repeat:
Install 4.2-BETA as of 2000/11/07 (haven't seen a commit to
uipc_socket.c since then), setup a jailed process group as
described in jail(8) and have "sshd_enabled=YES" in the jail's
rc.conf. Starting the jail will panic the machine due to
dereferencing a NULL pointer.
Alternatively:
Install the above jail environment and start inetd(8) with the
ftp line enabled. See panics when doing "telnet $JAIL ftp" while
contacting an ftpd started by "/usr/libexec/ftpd -D -a $JAIL" has
no problems.
>Fix:
none known yet -- moving the (prp == 0) check upwards will result
in EPROTONOTSUPPORT which I feel to be the wrong reaction to
something as usual as "just not specifying the ip address to bind
the service to".
Setting the jail.socket_unixiproute_only to 0 in /etc/sysctl.conf
will workaround the panics, but somehow defeats the idea of a
jail(2) and the sysctl's default setting or the reason for
introducing it in the first place.
Binding the services to the jail's IP address seems to help, but
some services might not be this configurable (I fail to see how
this could be done with ftpd in the inetd scenario, for
instance).
virtually yours 82D1 9B9C 01DC 4FB4 D7B4 61BE 3F49 4F77 72DE DA76
Gerhard Sittig true | mail -s "get gpg key" Gerhard.Sittig@gmx.net
--
If you don't understand or are scared by any of the above
ask your parents or an adult to help you.
>Release-Note:
>Audit-Trail:
>Unformatted:
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-bugs" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20001109213512.E25237>