From owner-svn-src-head@FreeBSD.ORG Tue May 5 10:56:13 2009 Return-Path: Delivered-To: svn-src-head@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8BE0F106566B; Tue, 5 May 2009 10:56:13 +0000 (UTC) (envelope-from zec@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 762DB8FC13; Tue, 5 May 2009 10:56:13 +0000 (UTC) (envelope-from zec@FreeBSD.org) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id n45AuDj0079330; Tue, 5 May 2009 10:56:13 GMT (envelope-from zec@svn.freebsd.org) Received: (from zec@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id n45AuCho079307; Tue, 5 May 2009 10:56:12 GMT (envelope-from zec@svn.freebsd.org) Message-Id: <200905051056.n45AuCho079307@svn.freebsd.org> From: Marko Zec Date: Tue, 5 May 2009 10:56:12 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r191816 - in head/sys: conf dev/cxgb/ulp/tom kern net net80211 netgraph netinet netinet6 netipsec nfsclient sys X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 May 2009 10:56:14 -0000 Author: zec Date: Tue May 5 10:56:12 2009 New Revision: 191816 URL: http://svn.freebsd.org/changeset/base/191816 Log: Change the curvnet variable from a global const struct vnet *, previously always pointing to the default vnet context, to a dynamically changing thread-local one. The currvnet context should be set on entry to networking code via CURVNET_SET() macros, and reverted to previous state via CURVNET_RESTORE(). Recursions on curvnet are permitted, though strongly discuouraged. This change should have no functional impact on nooptions VIMAGE kernel builds, where CURVNET_* macros expand to whitespace. The curthread->td_vnet (aka curvnet) variable's purpose is to be an indicator of the vnet context in which the current network-related operation takes place, in case we cannot deduce the current vnet context from any other source, such as by looking at mbuf's m->m_pkthdr.rcvif->if_vnet, sockets's so->so_vnet etc. Moreover, so far curvnet has turned out to be an invaluable consistency checking aid: it helps to catch cases when sockets, ifnets or any other vnet-aware structures may have leaked from one vnet to another. The exact placement of the CURVNET_SET() / CURVNET_RESTORE() macros was a result of an empirical iterative process, whith an aim to reduce recursions on CURVNET_SET() to a minimum, while still reducing the scope of CURVNET_SET() to networking only operations - the alternative would be calling CURVNET_SET() on each system call entry. In general, curvnet has to be set in three typicall cases: when processing socket-related requests from userspace or from within the kernel; when processing inbound traffic flowing from device drivers to upper layers of the networking stack, and when executing timer-driven networking functions. This change also introduces a DDB subcommand to show the list of all vnet instances. Approved by: julian (mentor) Modified: head/sys/conf/files head/sys/dev/cxgb/ulp/tom/cxgb_cpl_io.c head/sys/kern/init_main.c head/sys/kern/kern_fork.c head/sys/kern/kern_linker.c head/sys/kern/kern_vimage.c head/sys/kern/subr_pcpu.c head/sys/kern/sys_socket.c head/sys/kern/uipc_socket.c head/sys/kern/uipc_syscalls.c head/sys/kern/uipc_usrreq.c head/sys/net/bpf.c head/sys/net/if.c head/sys/net/if_clone.c head/sys/net/if_ethersubr.c head/sys/net/if_var.h head/sys/net/netisr.c head/sys/net/rtsock.c head/sys/net80211/ieee80211_freebsd.c head/sys/netgraph/netgraph.h head/sys/netinet/if_ether.c head/sys/netinet/igmp.c head/sys/netinet/in_rmx.c head/sys/netinet/ip_input.c head/sys/netinet/tcp_hostcache.c head/sys/netinet6/in6_rmx.c head/sys/netinet6/ip6_mroute.c head/sys/netinet6/mld6.c head/sys/netinet6/nd6.c head/sys/netinet6/nd6_nbr.c head/sys/netipsec/xform_tcp.c head/sys/nfsclient/nfs_vnops.c head/sys/sys/proc.h head/sys/sys/ucred.h head/sys/sys/vimage.h Modified: head/sys/conf/files ============================================================================== --- head/sys/conf/files Tue May 5 10:46:49 2009 (r191815) +++ head/sys/conf/files Tue May 5 10:56:12 2009 (r191816) @@ -1952,8 +1952,8 @@ kern/kern_time.c standard kern/kern_timeout.c standard kern/kern_umtx.c standard kern/kern_uuid.c standard -kern/kern_xxx.c standard kern/kern_vimage.c standard +kern/kern_xxx.c standard kern/link_elf.c standard kern/linker_if.m standard kern/md4c.c optional netsmb Modified: head/sys/dev/cxgb/ulp/tom/cxgb_cpl_io.c ============================================================================== --- head/sys/dev/cxgb/ulp/tom/cxgb_cpl_io.c Tue May 5 10:46:49 2009 (r191815) +++ head/sys/dev/cxgb/ulp/tom/cxgb_cpl_io.c Tue May 5 10:56:12 2009 (r191816) @@ -1217,7 +1217,7 @@ install_offload_ops(struct socket *so) * receive window. */ static __inline int -select_rcv_wscale(int space) +select_rcv_wscale(int space, struct vnet *vnet) { INIT_VNET_INET(so->so_vnet); int wscale = 0; @@ -1326,7 +1326,7 @@ static inline unsigned int calc_opt0h(struct socket *so, int mtu_idx) { struct tcpcb *tp = so_sototcpcb(so); - int wscale = select_rcv_wscale(tp->rcv_wnd); + int wscale = select_rcv_wscale(tp->rcv_wnd, so->so_vnet); return V_NAGLE((tp->t_flags & TF_NODELAY) == 0) | V_KEEP_ALIVE((so_options_get(so) & SO_KEEPALIVE) != 0) | F_TCAM_BYPASS | Modified: head/sys/kern/init_main.c ============================================================================== --- head/sys/kern/init_main.c Tue May 5 10:46:49 2009 (r191815) +++ head/sys/kern/init_main.c Tue May 5 10:56:12 2009 (r191816) @@ -74,6 +74,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include @@ -452,6 +453,9 @@ proc0_init(void *dummy __unused) p->p_ucred->cr_uidinfo = uifind(0); p->p_ucred->cr_ruidinfo = uifind(0); p->p_ucred->cr_prison = NULL; /* Don't jail it. */ +#ifdef VIMAGE + p->p_ucred->cr_vnet = LIST_FIRST(&vnet_head); +#endif #ifdef AUDIT audit_cred_kproc0(p->p_ucred); #endif Modified: head/sys/kern/kern_fork.c ============================================================================== --- head/sys/kern/kern_fork.c Tue May 5 10:46:49 2009 (r191815) +++ head/sys/kern/kern_fork.c Tue May 5 10:56:12 2009 (r191816) @@ -68,6 +68,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include @@ -523,6 +524,11 @@ again: td2->td_sigmask = td->td_sigmask; td2->td_flags = TDF_INMEM; +#ifdef VIMAGE + td2->td_vnet = NULL; + td2->td_vnet_lpush = NULL; +#endif + /* * Duplicate sub-structures as needed. * Increase reference counts on shared objects. Modified: head/sys/kern/kern_linker.c ============================================================================== --- head/sys/kern/kern_linker.c Tue May 5 10:46:49 2009 (r191815) +++ head/sys/kern/kern_linker.c Tue May 5 10:56:12 2009 (r191816) @@ -993,6 +993,12 @@ kern_kldload(struct thread *td, const ch return (error); /* + * It's possible that kldloaded module will attach a new ifnet, + * so vnet context must be set when this ocurs. + */ + CURVNET_SET(TD_TO_VNET(td)); + + /* * If file does not contain a qualified name or any dot in it * (kldname.ko, or kldname.ver.ko) treat it as an interface * name. @@ -1019,6 +1025,7 @@ kern_kldload(struct thread *td, const ch *fileid = lf->id; unlock: KLD_UNLOCK(); + CURVNET_RESTORE(); return (error); } @@ -1056,6 +1063,7 @@ kern_kldunload(struct thread *td, int fi if ((error = priv_check(td, PRIV_KLD_UNLOAD)) != 0) return (error); + CURVNET_SET(TD_TO_VNET(td)); KLD_LOCK(); lf = linker_find_file_by_id(fileid); if (lf) { @@ -1092,6 +1100,7 @@ kern_kldunload(struct thread *td, int fi PMC_CALL_HOOK(td, PMC_FN_KLD_UNLOAD, (void *) &pkm); #endif KLD_UNLOCK(); + CURVNET_RESTORE(); return (error); } Modified: head/sys/kern/kern_vimage.c ============================================================================== --- head/sys/kern/kern_vimage.c Tue May 5 10:46:49 2009 (r191815) +++ head/sys/kern/kern_vimage.c Tue May 5 10:56:12 2009 (r191816) @@ -31,6 +31,8 @@ #include __FBSDID("$FreeBSD$"); +#include "opt_ddb.h" + #include #include #include @@ -38,6 +40,9 @@ __FBSDID("$FreeBSD$"); #include #include #include +#ifdef DDB +#include +#endif #ifndef VIMAGE_GLOBALS @@ -51,8 +56,6 @@ static int vnet_mod_constructor(struct v static int vnet_mod_destructor(struct vnet_modlink *); #ifdef VIMAGE -/* curvnet should be thread-local - this is only a temporary step. */ -struct vnet *curvnet; struct vnet_list_head vnet_head; #endif @@ -183,7 +186,8 @@ vnet_mod_deregister_multi(const struct v free(vml, M_VIMAGE); } -static int vnet_mod_constructor(struct vnet_modlink *vml) +static int +vnet_mod_constructor(struct vnet_modlink *vml) { const struct vnet_modinfo *vmi = vml->vml_modinfo; @@ -303,7 +307,9 @@ vi_init(void *unused) if (vnet == NULL) panic("vi_alloc: malloc failed"); LIST_INSERT_HEAD(&vnet_head, vnet, vnet_le); + vnet->vnet_magic_n = VNET_MAGIC_N; + /* We MUST clear curvnet in vi_init_done before going SMP. */ curvnet = LIST_FIRST(&vnet_head); #endif } @@ -313,6 +319,10 @@ vi_init_done(void *unused) { struct vnet_modlink *vml_iter; +#ifdef VIMAGE + curvnet = NULL; +#endif + if (TAILQ_EMPTY(&vnet_modpending_head)) return; @@ -327,5 +337,45 @@ vi_init_done(void *unused) SYSINIT(vimage, SI_SUB_VIMAGE, SI_ORDER_FIRST, vi_init, NULL); SYSINIT(vimage_done, SI_SUB_VIMAGE_DONE, SI_ORDER_FIRST, vi_init_done, NULL); - #endif /* !VIMAGE_GLOBALS */ + +#ifdef VIMAGE +#ifdef DDB +static void +db_vnet_ptr(void *arg) +{ + + if (arg) + db_printf(" %p", arg); + else +#if SIZE_MAX == UINT32_MAX /* 32-bit arch */ + db_printf(" 0"); +#else /* 64-bit arch, most probaly... */ + db_printf(" 0"); +#endif +} + +DB_SHOW_COMMAND(vnets, db_show_vnets) +{ + VNET_ITERATOR_DECL(vnet_iter); + +#if SIZE_MAX == UINT32_MAX /* 32-bit arch */ + db_printf(" vnet ifs socks"); + db_printf(" net inet inet6 ipsec netgraph\n"); +#else /* 64-bit arch, most probaly... */ + db_printf(" vnet ifs socks"); + db_printf(" net inet inet6 ipsec netgraph\n"); +#endif + VNET_FOREACH(vnet_iter) { + db_printf("%p %3d %5d", + vnet_iter, vnet_iter->ifccnt, vnet_iter->sockcnt); + db_vnet_ptr(vnet_iter->mod_data[VNET_MOD_NET]); + db_vnet_ptr(vnet_iter->mod_data[VNET_MOD_INET]); + db_vnet_ptr(vnet_iter->mod_data[VNET_MOD_INET6]); + db_vnet_ptr(vnet_iter->mod_data[VNET_MOD_IPSEC]); + db_vnet_ptr(vnet_iter->mod_data[VNET_MOD_NETGRAPH]); + db_printf("\n"); + } +} +#endif +#endif /* VIMAGE */ Modified: head/sys/kern/subr_pcpu.c ============================================================================== --- head/sys/kern/subr_pcpu.c Tue May 5 10:46:49 2009 (r191815) +++ head/sys/kern/subr_pcpu.c Tue May 5 10:56:12 2009 (r191816) @@ -135,6 +135,10 @@ show_pcpu(struct pcpu *pc) db_printf("none\n"); db_show_mdpcpu(pc); +#ifdef VIMAGE + db_printf("curvnet = %p\n", pc->pc_curthread->td_vnet); +#endif + #ifdef WITNESS db_printf("spin locks held:\n"); witness_list_locks(&pc->pc_spinlocks); Modified: head/sys/kern/sys_socket.c ============================================================================== --- head/sys/kern/sys_socket.c Tue May 5 10:46:49 2009 (r191815) +++ head/sys/kern/sys_socket.c Tue May 5 10:56:12 2009 (r191816) @@ -50,6 +50,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include @@ -74,16 +75,19 @@ soo_read(struct file *fp, struct uio *ui int flags, struct thread *td) { struct socket *so = fp->f_data; -#ifdef MAC int error; +#ifdef MAC SOCK_LOCK(so); error = mac_socket_check_receive(active_cred, so); SOCK_UNLOCK(so); if (error) return (error); #endif - return (soreceive(so, 0, uio, 0, 0, 0)); + CURVNET_SET(so->so_vnet); + error = soreceive(so, 0, uio, 0, 0, 0); + CURVNET_RESTORE(); + return (error); } /* ARGSUSED */ @@ -125,6 +129,7 @@ soo_ioctl(struct file *fp, u_long cmd, v struct socket *so = fp->f_data; int error = 0; + CURVNET_SET(so->so_vnet); switch (cmd) { case FIONBIO: SOCK_LOCK(so); @@ -205,6 +210,7 @@ soo_ioctl(struct file *fp, u_long cmd, v (so, cmd, data, 0, td)); break; } + CURVNET_RESTORE(); return (error); } Modified: head/sys/kern/uipc_socket.c ============================================================================== --- head/sys/kern/uipc_socket.c Tue May 5 10:46:49 2009 (r191815) +++ head/sys/kern/uipc_socket.c Tue May 5 10:56:12 2009 (r191816) @@ -264,7 +264,7 @@ SYSINIT(param, SI_SUB_TUNABLES, SI_ORDER * soalloc() returns a socket with a ref count of 0. */ static struct socket * -soalloc(void) +soalloc(struct vnet *vnet) { struct socket *so; @@ -286,7 +286,8 @@ soalloc(void) so->so_gencnt = ++so_gencnt; ++numopensockets; #ifdef VIMAGE - so->so_vnet = curvnet; + ++vnet->sockcnt; /* locked with so_global_mtx */ + so->so_vnet = vnet; #endif mtx_unlock(&so_global_mtx); return (so); @@ -307,6 +308,9 @@ sodealloc(struct socket *so) mtx_lock(&so_global_mtx); so->so_gencnt = ++so_gencnt; --numopensockets; /* Could be below, but faster here. */ +#ifdef VIMAGE + --so->so_vnet->sockcnt; +#endif mtx_unlock(&so_global_mtx); if (so->so_rcv.sb_hiwat) (void)chgsbsize(so->so_cred->cr_uidinfo, @@ -356,7 +360,7 @@ socreate(int dom, struct socket **aso, i if (prp->pr_type != type) return (EPROTOTYPE); - so = soalloc(); + so = soalloc(TD_TO_VNET(td)); if (so == NULL) return (ENOBUFS); @@ -382,7 +386,9 @@ socreate(int dom, struct socket **aso, i * Auto-sizing of socket buffers is managed by the protocols and * the appropriate flags must be set in the pru_attach function. */ + CURVNET_SET(so->so_vnet); error = (*prp->pr_usrreqs->pru_attach)(so, proto, td); + CURVNET_RESTORE(); if (error) { KASSERT(so->so_count == 1, ("socreate: so_count %d", so->so_count)); @@ -424,7 +430,8 @@ sonewconn(struct socket *head, int conns if (over) #endif return (NULL); - so = soalloc(); + VNET_ASSERT(head->so_vnet); + so = soalloc(head->so_vnet); if (so == NULL) return (NULL); if ((head->so_options & SO_ACCEPTFILTER) != 0) @@ -496,8 +503,12 @@ sonewconn(struct socket *head, int conns int sobind(struct socket *so, struct sockaddr *nam, struct thread *td) { + int error; - return ((*so->so_proto->pr_usrreqs->pru_bind)(so, nam, td)); + CURVNET_SET(so->so_vnet); + error = (*so->so_proto->pr_usrreqs->pru_bind)(so, nam, td); + CURVNET_RESTORE(); + return error; } /* @@ -645,6 +656,7 @@ soclose(struct socket *so) KASSERT(!(so->so_state & SS_NOFDREF), ("soclose: SS_NOFDREF on enter")); + CURVNET_SET(so->so_vnet); funsetown(&so->so_sigio); if (so->so_state & SS_ISCONNECTED) { if ((so->so_state & SS_ISDISCONNECTING) == 0) { @@ -696,6 +708,7 @@ drop: KASSERT((so->so_state & SS_NOFDREF) == 0, ("soclose: NOFDREF")); so->so_state |= SS_NOFDREF; sorele(so); + CURVNET_RESTORE(); return (error); } @@ -771,7 +784,9 @@ soconnect(struct socket *so, struct sock * biting us. */ so->so_error = 0; + CURVNET_SET(so->so_vnet); error = (*so->so_proto->pr_usrreqs->pru_connect)(so, nam, td); + CURVNET_RESTORE(); } return (error); @@ -1287,9 +1302,13 @@ int sosend(struct socket *so, struct sockaddr *addr, struct uio *uio, struct mbuf *top, struct mbuf *control, int flags, struct thread *td) { + int error; - return (so->so_proto->pr_usrreqs->pru_sosend(so, addr, uio, top, - control, flags, td)); + CURVNET_SET(so->so_vnet); + error = so->so_proto->pr_usrreqs->pru_sosend(so, addr, uio, top, + control, flags, td); + CURVNET_RESTORE(); + return (error); } /* @@ -2037,6 +2056,7 @@ int soshutdown(struct socket *so, int how) { struct protosw *pr = so->so_proto; + int error; if (!(how == SHUT_RD || how == SHUT_WR || how == SHUT_RDWR)) return (EINVAL); @@ -2045,8 +2065,12 @@ soshutdown(struct socket *so, int how) } if (how != SHUT_WR) sorflush(so); - if (how != SHUT_RD) - return ((*pr->pr_usrreqs->pru_shutdown)(so)); + if (how != SHUT_RD) { + CURVNET_SET(so->so_vnet); + error = (*pr->pr_usrreqs->pru_shutdown)(so); + CURVNET_RESTORE(); + return (error); + } return (0); } @@ -2070,6 +2094,7 @@ sorflush(struct socket *so) * socket buffer. Don't let our acquire be interrupted by a signal * despite any existing socket disposition on interruptable waiting. */ + CURVNET_SET(so->so_vnet); socantrcvmore(so); (void) sblock(sb, SBL_WAIT | SBL_NOINTR); @@ -2093,6 +2118,7 @@ sorflush(struct socket *so) if (pr->pr_flags & PR_RIGHTS && pr->pr_domain->dom_dispose != NULL) (*pr->pr_domain->dom_dispose)(asb.sb_mb); sbrelease_internal(&asb, so); + CURVNET_RESTORE(); } /* Modified: head/sys/kern/uipc_syscalls.c ============================================================================== --- head/sys/kern/uipc_syscalls.c Tue May 5 10:46:49 2009 (r191815) +++ head/sys/kern/uipc_syscalls.c Tue May 5 10:56:12 2009 (r191816) @@ -64,6 +64,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #ifdef KTRACE #include #endif @@ -264,7 +265,9 @@ listen(td, uap) if (error) goto done; #endif + CURVNET_SET(so->so_vnet); error = solisten(so, uap->backlog, td); + CURVNET_RESTORE(); #ifdef MAC done: #endif @@ -429,7 +432,9 @@ kern_accept(struct thread *td, int s, st tmp = fflag & FASYNC; (void) fo_ioctl(nfp, FIOASYNC, &tmp, td->td_ucred, td); sa = 0; + CURVNET_SET(so->so_vnet); error = soaccept(so, &sa); + CURVNET_RESTORE(); if (error) { /* * return a namelen of zero for older code which might @@ -976,9 +981,11 @@ kern_recvit(td, s, mp, fromseg, controlp ktruio = cloneuio(&auio); #endif len = auio.uio_resid; + CURVNET_SET(so->so_vnet); error = soreceive(so, &fromsa, &auio, (struct mbuf **)0, (mp->msg_control || controlp) ? &control : (struct mbuf **)0, &mp->msg_flags); + CURVNET_RESTORE(); if (error) { if (auio.uio_resid != (int)len && (error == ERESTART || error == EINTR || error == EWOULDBLOCK)) @@ -1322,7 +1329,9 @@ kern_setsockopt(td, s, level, name, val, error = getsock(td->td_proc->p_fd, s, &fp, NULL); if (error == 0) { so = fp->f_data; + CURVNET_SET(so->so_vnet); error = sosetopt(so, &sopt); + CURVNET_RESTORE(); fdrop(fp, td); } return(error); @@ -1400,7 +1409,9 @@ kern_getsockopt(td, s, level, name, val, error = getsock(td->td_proc->p_fd, s, &fp, NULL); if (error == 0) { so = fp->f_data; + CURVNET_SET(so->so_vnet); error = sogetopt(so, &sopt); + CURVNET_RESTORE(); *valsize = sopt.sopt_valsize; fdrop(fp, td); } @@ -1463,7 +1474,9 @@ kern_getsockname(struct thread *td, int return (error); so = fp->f_data; *sa = NULL; + CURVNET_SET(so->so_vnet); error = (*so->so_proto->pr_usrreqs->pru_sockaddr)(so, sa); + CURVNET_RESTORE(); if (error) goto bad; if (*sa == NULL) @@ -1564,7 +1577,9 @@ kern_getpeername(struct thread *td, int goto done; } *sa = NULL; + CURVNET_SET(so->so_vnet); error = (*so->so_proto->pr_usrreqs->pru_peeraddr)(so, sa); + CURVNET_RESTORE(); if (error) goto bad; if (*sa == NULL) @@ -2176,9 +2191,11 @@ retry_space: goto done; } SOCKBUF_UNLOCK(&so->so_snd); + CURVNET_SET(so->so_vnet); /* Avoid error aliasing. */ err = (*so->so_proto->pr_usrreqs->pru_send) (so, 0, m, NULL, NULL, td); + CURVNET_RESTORE(); if (err == 0) { /* * We need two counters to get the Modified: head/sys/kern/uipc_usrreq.c ============================================================================== --- head/sys/kern/uipc_usrreq.c Tue May 5 10:46:49 2009 (r191815) +++ head/sys/kern/uipc_usrreq.c Tue May 5 10:56:12 2009 (r191816) @@ -90,6 +90,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #ifdef DDB #include @@ -1647,6 +1648,10 @@ static void unp_init(void) { +#ifdef VIMAGE + if (!IS_DEFAULT_VNET(curvnet)) + return; +#endif unp_zone = uma_zcreate("unpcb", sizeof(struct unpcb), NULL, NULL, NULL, NULL, UMA_ALIGN_PTR, 0); if (unp_zone == NULL) Modified: head/sys/net/bpf.c ============================================================================== --- head/sys/net/bpf.c Tue May 5 10:46:49 2009 (r191815) +++ head/sys/net/bpf.c Tue May 5 10:56:12 2009 (r191816) @@ -873,11 +873,10 @@ bpfwrite(struct cdev *dev, struct uio *u m->m_len -= hlen; m->m_data += hlen; /* XXX */ + CURVNET_SET(ifp->if_vnet); #ifdef MAC BPFD_LOCK(d); - CURVNET_SET(ifp->if_vnet); mac_bpfdesc_create_mbuf(d, m); - CURVNET_RESTORE(); if (mc != NULL) mac_bpfdesc_create_mbuf(d, mc); BPFD_UNLOCK(d); @@ -893,6 +892,7 @@ bpfwrite(struct cdev *dev, struct uio *u else m_freem(mc); } + CURVNET_RESTORE(); return (error); } Modified: head/sys/net/if.c ============================================================================== --- head/sys/net/if.c Tue May 5 10:46:49 2009 (r191815) +++ head/sys/net/if.c Tue May 5 10:56:12 2009 (r191816) @@ -53,6 +53,7 @@ #include #include #include +#include #include #include #include @@ -126,7 +127,6 @@ static void if_attachdomain(void *); static void if_attachdomain1(struct ifnet *); static int ifconf(u_long, caddr_t); static void if_freemulti(struct ifmultiaddr *); -static void if_grow(void); static void if_init(void *); static void if_check(void *); static void if_route(struct ifnet *, int flag, int fam); @@ -202,7 +202,7 @@ MALLOC_DEFINE(M_IFNET, "ifnet", "interfa MALLOC_DEFINE(M_IFADDR, "ifaddr", "interface address"); MALLOC_DEFINE(M_IFMADDR, "ether_multi", "link-level multicast address"); -static struct ifnet * +struct ifnet * ifnet_byindex_locked(u_short idx) { INIT_VNET_NET(curvnet); @@ -239,7 +239,7 @@ ifnet_byindex_ref(u_short idx) return (ifp); } -static void +void ifnet_setbyindex(u_short idx, struct ifnet *ifp) { INIT_VNET_NET(curvnet); @@ -445,7 +445,7 @@ vnet_net_iattach(const void *unused __un return (0); } -static void +void if_grow(void) { INIT_VNET_NET(curvnet); @@ -696,11 +696,13 @@ if_attach(struct ifnet *ifp) mac_ifnet_create(ifp); #endif - ifdev_setbyindex(ifp->if_index, make_dev(&net_cdevsw, - ifp->if_index, UID_ROOT, GID_WHEEL, 0600, "%s/%s", - net_cdevsw.d_name, ifp->if_xname)); - make_dev_alias(ifdev_byindex(ifp->if_index), "%s%d", - net_cdevsw.d_name, ifp->if_index); + if (IS_DEFAULT_VNET(curvnet)) { + ifdev_setbyindex(ifp->if_index, make_dev(&net_cdevsw, + ifp->if_index, UID_ROOT, GID_WHEEL, 0600, "%s/%s", + net_cdevsw.d_name, ifp->if_xname)); + make_dev_alias(ifdev_byindex(ifp->if_index), "%s%d", + net_cdevsw.d_name, ifp->if_index); + } ifq_attach(&ifp->if_snd, ifp); @@ -742,13 +744,17 @@ if_attach(struct ifnet *ifp) IFNET_WLOCK(); TAILQ_INSERT_TAIL(&V_ifnet, ifp, if_link); +#ifdef VIMAGE + curvnet->ifccnt++; +#endif IFNET_WUNLOCK(); if (domain_init_status >= 2) if_attachdomain1(ifp); EVENTHANDLER_INVOKE(ifnet_arrival_event, ifp); - devctl_notify("IFNET", ifp->if_xname, "ATTACH", NULL); + if (IS_DEFAULT_VNET(curvnet)) + devctl_notify("IFNET", ifp->if_xname, "ATTACH", NULL); /* Announce the interface. */ rt_ifannouncemsg(ifp, IFAN_ARRIVAL); @@ -895,6 +901,10 @@ if_detach(struct ifnet *ifp) found = 1; break; } +#ifdef VIMAGE + if (found) + curvnet->ifccnt--; +#endif IFNET_WUNLOCK(); if (!found) return; @@ -943,7 +953,8 @@ if_detach(struct ifnet *ifp) * Clean up all addresses. */ ifp->if_addr = NULL; - destroy_dev(ifdev_byindex(ifp->if_index)); + if (IS_DEFAULT_VNET(curvnet)) + destroy_dev(ifdev_byindex(ifp->if_index)); ifdev_setbyindex(ifp->if_index, NULL); /* We can now free link ifaddr. */ @@ -972,7 +983,8 @@ if_detach(struct ifnet *ifp) /* Announce that the interface is gone. */ rt_ifannouncemsg(ifp, IFAN_DEPARTURE); EVENTHANDLER_INVOKE(ifnet_departure_event, ifp); - devctl_notify("IFNET", ifp->if_xname, "DETACH", NULL); + if (IS_DEFAULT_VNET(curvnet)) + devctl_notify("IFNET", ifp->if_xname, "DETACH", NULL); if_delgroups(ifp); IF_AFDATA_LOCK(ifp); @@ -1701,8 +1713,10 @@ do_link_state_change(void *arg, int pend (*lagg_linkstate_p)(ifp, link_state); } - devctl_notify("IFNET", ifp->if_xname, - (link_state == LINK_STATE_UP) ? "LINK_UP" : "LINK_DOWN", NULL); + if (IS_DEFAULT_VNET(curvnet)) + devctl_notify("IFNET", ifp->if_xname, + (link_state == LINK_STATE_UP) ? "LINK_UP" : "LINK_DOWN", + NULL); if (pending > 1) if_printf(ifp, "%d link states coalesced\n", pending); if (log_link_state_change) Modified: head/sys/net/if_clone.c ============================================================================== --- head/sys/net/if_clone.c Tue May 5 10:46:49 2009 (r191815) +++ head/sys/net/if_clone.c Tue May 5 10:56:12 2009 (r191816) @@ -39,6 +39,7 @@ #include #include #include +#include #include #include @@ -49,6 +50,7 @@ #include #include #include +#include static void if_clone_free(struct if_clone *ifc); static int if_clone_createif(struct if_clone *ifc, char *name, size_t len, @@ -203,15 +205,14 @@ if_clone_destroyif(struct if_clone *ifc, { int err; - if (ifc->ifc_destroy == NULL) { - err = EOPNOTSUPP; - goto done; - } + if (ifc->ifc_destroy == NULL) + return(EOPNOTSUPP); IF_CLONE_LOCK(ifc); IFC_IFLIST_REMOVE(ifc, ifp); IF_CLONE_UNLOCK(ifc); + CURVNET_SET_QUIET(ifp->if_vnet); if_delgroup(ifp, ifc->ifc_name); err = (*ifc->ifc_destroy)(ifc, ifp); @@ -223,8 +224,7 @@ if_clone_destroyif(struct if_clone *ifc, IFC_IFLIST_INSERT(ifc, ifp); IF_CLONE_UNLOCK(ifc); } - -done: + CURVNET_RESTORE(); return (err); } Modified: head/sys/net/if_ethersubr.c ============================================================================== --- head/sys/net/if_ethersubr.c Tue May 5 10:46:49 2009 (r191815) +++ head/sys/net/if_ethersubr.c Tue May 5 10:56:12 2009 (r191816) @@ -602,6 +602,8 @@ ether_input(struct ifnet *ifp, struct mb } #endif + CURVNET_SET_QUIET(ifp->if_vnet); + if (ETHER_IS_MULTICAST(eh->ether_dhost)) { if (ETHER_IS_BROADCAST(eh->ether_dhost)) m->m_flags |= M_BCAST; @@ -638,6 +640,7 @@ ether_input(struct ifnet *ifp, struct mb /* Allow monitor mode to claim this frame, after stats are updated. */ if (ifp->if_flags & IFF_MONITOR) { m_freem(m); + CURVNET_RESTORE(); return; } @@ -686,8 +689,10 @@ ether_input(struct ifnet *ifp, struct mb ("%s: ng_ether_input_p is NULL", __func__)); m->m_flags &= ~M_PROMISC; (*ng_ether_input_p)(ifp, &m); - if (m == NULL) + if (m == NULL) { + CURVNET_RESTORE(); return; + } } /* @@ -698,8 +703,10 @@ ether_input(struct ifnet *ifp, struct mb if (ifp->if_bridge != NULL) { m->m_flags &= ~M_PROMISC; BRIDGE_INPUT(ifp, m); - if (m == NULL) + if (m == NULL) { + CURVNET_RESTORE(); return; + } } #ifdef DEV_CARP @@ -735,6 +742,7 @@ ether_input(struct ifnet *ifp, struct mb random_harvest(m, 16, 3, 0, RANDOM_NET); ether_demux(ifp, m); + CURVNET_RESTORE(); } /* Modified: head/sys/net/if_var.h ============================================================================== --- head/sys/net/if_var.h Tue May 5 10:46:49 2009 (r191815) +++ head/sys/net/if_var.h Tue May 5 10:56:12 2009 (r191816) @@ -731,7 +731,9 @@ struct ifindex_entry { * to call ifnet_byindex() instead if ifnet_byindex_ref(). */ struct ifnet *ifnet_byindex(u_short idx); +struct ifnet *ifnet_byindex_locked(u_short idx); struct ifnet *ifnet_byindex_ref(u_short idx); +void ifnet_setbyindex(u_short idx, struct ifnet *ifp); /* * Given the index, ifaddr_byindex() returns the one and only @@ -755,6 +757,7 @@ int if_allmulti(struct ifnet *, int); struct ifnet* if_alloc(u_char); void if_attach(struct ifnet *); void if_dead(struct ifnet *); +void if_grow(void); int if_delmulti(struct ifnet *, struct sockaddr *); void if_delmulti_ifma(struct ifmultiaddr *); void if_detach(struct ifnet *); Modified: head/sys/net/netisr.c ============================================================================== --- head/sys/net/netisr.c Tue May 5 10:46:49 2009 (r191815) +++ head/sys/net/netisr.c Tue May 5 10:56:12 2009 (r191816) @@ -43,6 +43,7 @@ #include #include #include +#include #include #include #include @@ -142,7 +143,10 @@ netisr_processqueue(struct netisr *ni) IF_DEQUEUE(ni->ni_queue, m); if (m == NULL) break; + VNET_ASSERT(m->m_pkthdr.rcvif != NULL); + CURVNET_SET(m->m_pkthdr.rcvif->if_vnet); ni->ni_handler(m); + CURVNET_RESTORE(); } } Modified: head/sys/net/rtsock.c ============================================================================== --- head/sys/net/rtsock.c Tue May 5 10:46:49 2009 (r191815) +++ head/sys/net/rtsock.c Tue May 5 10:56:12 2009 (r191816) @@ -1206,6 +1206,7 @@ rt_ifannouncemsg(struct ifnet *ifp, int static void rt_dispatch(struct mbuf *m, const struct sockaddr *sa) { + INIT_VNET_NET(curvnet); struct m_tag *tag; /* @@ -1223,6 +1224,14 @@ rt_dispatch(struct mbuf *m, const struct *(unsigned short *)(tag + 1) = sa->sa_family; m_tag_prepend(m, tag); } +#ifdef VIMAGE + if (V_loif) + m->m_pkthdr.rcvif = V_loif; + else { + m_freem(m); + return; + } +#endif netisr_queue(NETISR_ROUTE, m); /* mbuf is free'd on failure. */ } Modified: head/sys/net80211/ieee80211_freebsd.c ============================================================================== --- head/sys/net80211/ieee80211_freebsd.c Tue May 5 10:46:49 2009 (r191815) +++ head/sys/net80211/ieee80211_freebsd.c Tue May 5 10:56:12 2009 (r191816) @@ -41,6 +41,7 @@ __FBSDID("$FreeBSD$"); #include #include +#include #include #include @@ -498,9 +499,11 @@ notify_macaddr(struct ifnet *ifp, int op { struct ieee80211_join_event iev; + CURVNET_SET(ifp->if_vnet); memset(&iev, 0, sizeof(iev)); IEEE80211_ADDR_COPY(iev.iev_addr, mac); rt_ieee80211msg(ifp, op, &iev, sizeof(iev)); + CURVNET_RESTORE(); } void @@ -509,6 +512,7 @@ ieee80211_notify_node_join(struct ieee80 struct ieee80211vap *vap = ni->ni_vap; struct ifnet *ifp = vap->iv_ifp; + CURVNET_SET_QUIET(ifp->if_vnet); IEEE80211_NOTE(vap, IEEE80211_MSG_NODE, ni, "%snode join", (ni == vap->iv_bss) ? "bss " : ""); @@ -520,6 +524,7 @@ ieee80211_notify_node_join(struct ieee80 notify_macaddr(ifp, newassoc ? RTM_IEEE80211_JOIN : RTM_IEEE80211_REJOIN, ni->ni_macaddr); } + CURVNET_RESTORE(); } void @@ -528,6 +533,7 @@ ieee80211_notify_node_leave(struct ieee8 struct ieee80211vap *vap = ni->ni_vap; struct ifnet *ifp = vap->iv_ifp; + CURVNET_SET_QUIET(ifp->if_vnet); IEEE80211_NOTE(vap, IEEE80211_MSG_NODE, ni, "%snode leave", (ni == vap->iv_bss) ? "bss " : ""); @@ -538,6 +544,7 @@ ieee80211_notify_node_leave(struct ieee8 /* fire off wireless event station leaving */ notify_macaddr(ifp, RTM_IEEE80211_LEAVE, ni->ni_macaddr); } + CURVNET_RESTORE(); } void @@ -548,7 +555,9 @@ ieee80211_notify_scan_done(struct ieee80 IEEE80211_DPRINTF(vap, IEEE80211_MSG_SCAN, "%s\n", "notify scan done"); /* dispatch wireless event indicating scan completed */ + CURVNET_SET(ifp->if_vnet); rt_ieee80211msg(ifp, RTM_IEEE80211_SCAN, NULL, 0); + CURVNET_RESTORE(); } void @@ -576,7 +585,9 @@ ieee80211_notify_replay_failure(struct i iev.iev_keyix = k->wk_keyix; iev.iev_keyrsc = k->wk_keyrsc[0]; /* XXX need tid */ iev.iev_rsc = rsc; + CURVNET_SET(ifp->if_vnet); rt_ieee80211msg(ifp, RTM_IEEE80211_REPLAY, &iev, sizeof(iev)); + CURVNET_RESTORE(); } } @@ -597,7 +608,9 @@ ieee80211_notify_michael_failure(struct IEEE80211_ADDR_COPY(iev.iev_src, wh->i_addr2); iev.iev_cipher = IEEE80211_CIPHER_TKIP; iev.iev_keyix = keyix; + CURVNET_SET(ifp->if_vnet); rt_ieee80211msg(ifp, RTM_IEEE80211_MICHAEL, &iev, sizeof(iev)); + CURVNET_RESTORE(); } } Modified: head/sys/netgraph/netgraph.h ============================================================================== --- head/sys/netgraph/netgraph.h Tue May 5 10:46:49 2009 (r191815) +++ head/sys/netgraph/netgraph.h Tue May 5 10:56:12 2009 (r191816) @@ -352,6 +352,7 @@ struct ng_node { LIST_ENTRY(ng_node) nd_idnodes; /* ID hash collision list */ struct ng_queue nd_input_queue; /* input queue for locking */ int nd_refs; /* # of references to this node */ + struct vnet *nd_vnet; /* network stack instance */ #ifdef NETGRAPH_DEBUG /*----------------------------------------------*/ #define ND_MAGIC 0x59264837 int nd_magic; Modified: head/sys/netinet/if_ether.c ============================================================================== --- head/sys/netinet/if_ether.c Tue May 5 10:46:49 2009 (r191815) +++ head/sys/netinet/if_ether.c Tue May 5 10:56:12 2009 (r191816) @@ -143,10 +143,12 @@ arp_ifscrub(struct ifnet *ifp, uint32_t addr4.sin_len = sizeof(addr4); addr4.sin_family = AF_INET; addr4.sin_addr.s_addr = addr; + CURVNET_SET(ifp->if_vnet); IF_AFDATA_LOCK(ifp); lla_lookup(LLTABLE(ifp), (LLE_DELETE | LLE_IFADDR), (struct sockaddr *)&addr4); IF_AFDATA_UNLOCK(ifp); + CURVNET_RESTORE(); } #endif Modified: head/sys/netinet/igmp.c ============================================================================== --- head/sys/netinet/igmp.c Tue May 5 10:46:49 2009 (r191815) +++ head/sys/netinet/igmp.c Tue May 5 10:56:12 2009 (r191816) @@ -1117,6 +1117,9 @@ igmp_input_v3_group_query(struct in_mult nsrc = ntohs(igmpv3->igmp_numsrc); + if (!IS_DEFAULT_VNET(curvnet)) + return (retval); + /* * Deal with group-specific queries upfront. * If any group query is already pending, purge any recorded @@ -3372,7 +3375,7 @@ igmp_intr(struct mbuf *m) * indexes to guard against interface detach, they are * unique to each VIMAGE and must be retrieved. */ - CURVNET_SET(m->m_pkthdr.header); + CURVNET_SET((struct vnet *)(m->m_pkthdr.header)); INIT_VNET_NET(curvnet); INIT_VNET_INET(curvnet); ifindex = igmp_restore_context(m); @@ -3654,9 +3657,7 @@ igmp_modevent(module_t mod, int type, vo break; case MOD_UNLOAD: #ifndef VIMAGE_GLOBALS *** DIFF OUTPUT TRUNCATED AT 1000 LINES ***