From owner-svn-src-all@FreeBSD.ORG Sat Jan 8 16:13:45 2011 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3D8AC1065670; Sat, 8 Jan 2011 16:13:45 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 2AE928FC19; Sat, 8 Jan 2011 16:13:45 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id p08GDjaQ097005; Sat, 8 Jan 2011 16:13:45 GMT (envelope-from kib@svn.freebsd.org) Received: (from kib@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id p08GDijL096994; Sat, 8 Jan 2011 16:13:44 GMT (envelope-from kib@svn.freebsd.org) Message-Id: <201101081613.p08GDijL096994@svn.freebsd.org> From: Konstantin Belousov Date: Sat, 8 Jan 2011 16:13:44 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r217151 - in head/sys: amd64/amd64 amd64/ia32 amd64/include compat/freebsd32 compat/ia32 kern sys X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Jan 2011 16:13:45 -0000 Author: kib Date: Sat Jan 8 16:13:44 2011 New Revision: 217151 URL: http://svn.freebsd.org/changeset/base/217151 Log: Create shared (readonly) page. Each ABI may specify the use of page by setting SV_SHP flag and providing pointer to the vm object and mapping address. Provide simple allocator to carve space in the page, tailored to put the code with alignment restrictions. Enable shared page use for amd64, both native and 32bit FreeBSD binaries. Page is private mapped at the top of the user address space, moving a start of the stack one page down. Move signal trampoline code from the top of the stack to the shared page. Reviewed by: alc Modified: head/sys/amd64/amd64/elf_machdep.c head/sys/amd64/amd64/machdep.c head/sys/amd64/ia32/ia32_signal.c head/sys/amd64/include/vmparam.h head/sys/compat/freebsd32/freebsd32_misc.c head/sys/compat/ia32/ia32_sysvec.c head/sys/compat/ia32/ia32_util.h head/sys/kern/kern_exec.c head/sys/sys/imgact.h head/sys/sys/sysent.h Modified: head/sys/amd64/amd64/elf_machdep.c ============================================================================== --- head/sys/amd64/amd64/elf_machdep.c Sat Jan 8 16:02:14 2011 (r217150) +++ head/sys/amd64/amd64/elf_machdep.c Sat Jan 8 16:13:44 2011 (r217151) @@ -75,11 +75,14 @@ struct sysentvec elf64_freebsd_sysvec = .sv_setregs = exec_setregs, .sv_fixlimit = NULL, .sv_maxssiz = NULL, - .sv_flags = SV_ABI_FREEBSD | SV_LP64, + .sv_flags = SV_ABI_FREEBSD | SV_LP64 | SV_SHP, .sv_set_syscall_retval = cpu_set_syscall_retval, .sv_fetch_syscall_args = cpu_fetch_syscall_args, .sv_syscallnames = syscallnames, + .sv_shared_page_base = SHAREDPAGE, + .sv_shared_page_len = PAGE_SIZE, }; +INIT_SYSENTVEC(elf64_sysvec, &elf64_freebsd_sysvec); static Elf64_Brandinfo freebsd_brand_info = { .brand = ELFOSABI_FREEBSD, @@ -129,7 +132,6 @@ SYSINIT(kelf64, SI_SUB_EXEC, SI_ORDER_AN (sysinit_cfunc_t) elf64_insert_brand_entry, &kfreebsd_brand_info); - void elf64_dump_thread(struct thread *td __unused, void *dst __unused, size_t *off __unused) Modified: head/sys/amd64/amd64/machdep.c ============================================================================== --- head/sys/amd64/amd64/machdep.c Sat Jan 8 16:02:14 2011 (r217150) +++ head/sys/amd64/amd64/machdep.c Sat Jan 8 16:13:44 2011 (r217151) @@ -386,7 +386,7 @@ sendsig(sig_t catcher, ksiginfo_t *ksi, } regs->tf_rsp = (long)sfp; - regs->tf_rip = PS_STRINGS - *(p->p_sysent->sv_szsigcode); + regs->tf_rip = p->p_sysent->sv_sigcode_base; regs->tf_rflags &= ~(PSL_T | PSL_D); regs->tf_cs = _ucodesel; regs->tf_ds = _udatasel; Modified: head/sys/amd64/ia32/ia32_signal.c ============================================================================== --- head/sys/amd64/ia32/ia32_signal.c Sat Jan 8 16:02:14 2011 (r217150) +++ head/sys/amd64/ia32/ia32_signal.c Sat Jan 8 16:13:44 2011 (r217151) @@ -393,7 +393,8 @@ freebsd4_ia32_sendsig(sig_t catcher, ksi } regs->tf_rsp = (uintptr_t)sfp; - regs->tf_rip = p->p_sysent->sv_psstrings - sz_freebsd4_ia32_sigcode; + regs->tf_rip = p->p_sysent->sv_sigcode_base + sz_ia32_sigcode - + sz_freebsd4_ia32_sigcode; regs->tf_rflags &= ~(PSL_T | PSL_D); regs->tf_cs = _ucode32sel; regs->tf_ss = _udatasel; @@ -514,7 +515,7 @@ ia32_sendsig(sig_t catcher, ksiginfo_t * } regs->tf_rsp = (uintptr_t)sfp; - regs->tf_rip = p->p_sysent->sv_psstrings - *(p->p_sysent->sv_szsigcode); + regs->tf_rip = p->p_sysent->sv_sigcode_base; regs->tf_rflags &= ~(PSL_T | PSL_D); regs->tf_cs = _ucode32sel; regs->tf_ss = _udatasel; Modified: head/sys/amd64/include/vmparam.h ============================================================================== --- head/sys/amd64/include/vmparam.h Sat Jan 8 16:02:14 2011 (r217150) +++ head/sys/amd64/include/vmparam.h Sat Jan 8 16:13:44 2011 (r217151) @@ -186,7 +186,8 @@ #define VM_MAXUSER_ADDRESS UVADDR(NUPML4E, 0, 0, 0) -#define USRSTACK VM_MAXUSER_ADDRESS +#define SHAREDPAGE (VM_MAXUSER_ADDRESS - PAGE_SIZE) +#define USRSTACK SHAREDPAGE #define VM_MAX_ADDRESS UPT_MAX_ADDRESS #define VM_MIN_ADDRESS (0) Modified: head/sys/compat/freebsd32/freebsd32_misc.c ============================================================================== --- head/sys/compat/freebsd32/freebsd32_misc.c Sat Jan 8 16:02:14 2011 (r217150) +++ head/sys/compat/freebsd32/freebsd32_misc.c Sat Jan 8 16:13:44 2011 (r217151) @@ -2546,7 +2546,10 @@ freebsd32_copyout_strings(struct image_p execpath_len = 0; arginfo = (struct freebsd32_ps_strings *)curproc->p_sysent-> sv_psstrings; - szsigcode = *(imgp->proc->p_sysent->sv_szsigcode); + if (imgp->proc->p_sysent->sv_sigcode_base == 0) + szsigcode = *(imgp->proc->p_sysent->sv_szsigcode); + else + szsigcode = 0; destp = (caddr_t)arginfo - szsigcode - SPARE_USRSPACE - roundup(execpath_len, sizeof(char *)) - roundup(sizeof(canary), sizeof(char *)) - @@ -2556,7 +2559,7 @@ freebsd32_copyout_strings(struct image_p /* * install sigcode */ - if (szsigcode) + if (szsigcode != 0) copyout(imgp->proc->p_sysent->sv_sigcode, ((caddr_t)arginfo - szsigcode), szsigcode); Modified: head/sys/compat/ia32/ia32_sysvec.c ============================================================================== --- head/sys/compat/ia32/ia32_sysvec.c Sat Jan 8 16:02:14 2011 (r217150) +++ head/sys/compat/ia32/ia32_sysvec.c Sat Jan 8 16:13:44 2011 (r217151) @@ -129,7 +129,7 @@ struct sysentvec ia32_freebsd_sysvec = { .sv_minsigstksz = MINSIGSTKSZ, .sv_pagesize = IA32_PAGE_SIZE, .sv_minuser = 0, - .sv_maxuser = FREEBSD32_USRSTACK, + .sv_maxuser = FREEBSD32_MAXUSER, .sv_usrstack = FREEBSD32_USRSTACK, .sv_psstrings = FREEBSD32_PS_STRINGS, .sv_stackprot = VM_PROT_ALL, @@ -137,12 +137,20 @@ struct sysentvec ia32_freebsd_sysvec = { .sv_setregs = ia32_setregs, .sv_fixlimit = ia32_fixlimit, .sv_maxssiz = &ia32_maxssiz, - .sv_flags = SV_ABI_FREEBSD | SV_IA32 | SV_ILP32, + .sv_flags = SV_ABI_FREEBSD | SV_IA32 | SV_ILP32 | +#ifdef __amd64__ + SV_SHP +#else + 0 +#endif + , .sv_set_syscall_retval = ia32_set_syscall_retval, .sv_fetch_syscall_args = ia32_fetch_syscall_args, .sv_syscallnames = freebsd32_syscallnames, + .sv_shared_page_base = FREEBSD32_SHAREDPAGE, + .sv_shared_page_len = PAGE_SIZE, }; - +INIT_SYSENTVEC(elf_ia32_sysvec, &ia32_freebsd_sysvec); static Elf32_Brandinfo ia32_brand_info = { .brand = ELFOSABI_FREEBSD, @@ -191,7 +199,6 @@ SYSINIT(kia32, SI_SUB_EXEC, SI_ORDER_ANY (sysinit_cfunc_t) elf32_insert_brand_entry, &kia32_brand_info); - void elf32_dump_thread(struct thread *td __unused, void *dst __unused, size_t *off __unused) Modified: head/sys/compat/ia32/ia32_util.h ============================================================================== --- head/sys/compat/ia32/ia32_util.h Sat Jan 8 16:02:14 2011 (r217150) +++ head/sys/compat/ia32/ia32_util.h Sat Jan 8 16:13:44 2011 (r217151) @@ -41,9 +41,13 @@ #include #ifdef __ia64__ -#define FREEBSD32_USRSTACK ((1ul << 32) - IA32_PAGE_SIZE * 2) +#define FREEBSD32_MAXUSER ((1ul << 32) - IA32_PAGE_SIZE * 2) +#define FREEBSD32_SHAREDPAGE 0 +#define FREEBSD32_USRSTACK FREEBSD32_MAXUSER #else -#define FREEBSD32_USRSTACK ((1ul << 32) - IA32_PAGE_SIZE) +#define FREEBSD32_MAXUSER ((1ul << 32) - IA32_PAGE_SIZE) +#define FREEBSD32_SHAREDPAGE (FREEBSD32_MAXUSER - IA32_PAGE_SIZE) +#define FREEBSD32_USRSTACK FREEBSD32_SHAREDPAGE #endif #define IA32_PAGE_SIZE 4096 Modified: head/sys/kern/kern_exec.c ============================================================================== --- head/sys/kern/kern_exec.c Sat Jan 8 16:02:14 2011 (r217150) +++ head/sys/kern/kern_exec.c Sat Jan 8 16:13:44 2011 (r217151) @@ -389,6 +389,7 @@ do_execve(td, args, mac_p) imgp->canarylen = 0; imgp->pagesizes = 0; imgp->pagesizeslen = 0; + imgp->stack_prot = 0; #ifdef MAC error = mac_execve_enter(imgp, mac_p); @@ -996,6 +997,7 @@ exec_new_vmspace(imgp, sv) int error; struct proc *p = imgp->proc; struct vmspace *vmspace = p->p_vmspace; + vm_object_t obj; vm_offset_t sv_minuser, stack_addr; vm_map_t map; u_long ssiz; @@ -1029,6 +1031,20 @@ exec_new_vmspace(imgp, sv) map = &vmspace->vm_map; } + /* Map a shared page */ + obj = sv->sv_shared_page_obj; + if (obj != NULL) { + vm_object_reference(obj); + error = vm_map_fixed(map, obj, 0, + sv->sv_shared_page_base, sv->sv_shared_page_len, + VM_PROT_READ | VM_PROT_EXECUTE, VM_PROT_ALL, + MAP_COPY_ON_WRITE | MAP_ACC_NO_CHARGE); + if (error) { + vm_object_deallocate(obj); + return (error); + } + } + /* Allocate a new stack */ if (sv->sv_maxssiz != NULL) ssiz = *sv->sv_maxssiz; @@ -1036,7 +1052,9 @@ exec_new_vmspace(imgp, sv) ssiz = maxssiz; stack_addr = sv->sv_usrstack - ssiz; error = vm_map_stack(map, stack_addr, (vm_size_t)ssiz, - sv->sv_stackprot, VM_PROT_ALL, MAP_STACK_GROWS_DOWN); + obj != NULL && imgp->stack_prot != 0 ? imgp->stack_prot : + sv->sv_stackprot, + VM_PROT_ALL, MAP_STACK_GROWS_DOWN); if (error) return (error); @@ -1208,8 +1226,10 @@ exec_copyout_strings(imgp) p = imgp->proc; szsigcode = 0; arginfo = (struct ps_strings *)p->p_sysent->sv_psstrings; - if (p->p_sysent->sv_szsigcode != NULL) - szsigcode = *(p->p_sysent->sv_szsigcode); + if (p->p_sysent->sv_sigcode_base == 0) { + if (p->p_sysent->sv_szsigcode != NULL) + szsigcode = *(p->p_sysent->sv_szsigcode); + } destp = (caddr_t)arginfo - szsigcode - SPARE_USRSPACE - roundup(execpath_len, sizeof(char *)) - roundup(sizeof(canary), sizeof(char *)) - @@ -1219,7 +1239,7 @@ exec_copyout_strings(imgp) /* * install sigcode */ - if (szsigcode) + if (szsigcode != 0) copyout(p->p_sysent->sv_sigcode, ((caddr_t)arginfo - szsigcode), szsigcode); @@ -1459,3 +1479,64 @@ exec_unregister(execsw_arg) execsw = newexecsw; return (0); } + +static vm_object_t shared_page_obj; +static int shared_page_free; + +int +shared_page_fill(int size, int align, const char *data) +{ + vm_page_t m; + struct sf_buf *s; + vm_offset_t sk; + int res; + + VM_OBJECT_LOCK(shared_page_obj); + m = vm_page_grab(shared_page_obj, 0, VM_ALLOC_RETRY); + res = roundup(shared_page_free, align); + if (res + size >= IDX_TO_OFF(shared_page_obj->size)) + res = -1; + else { + VM_OBJECT_UNLOCK(shared_page_obj); + s = sf_buf_alloc(m, SFB_DEFAULT); + sk = sf_buf_kva(s); + bcopy(data, (void *)(sk + res), size); + shared_page_free = res + size; + sf_buf_free(s); + VM_OBJECT_LOCK(shared_page_obj); + } + vm_page_wakeup(m); + VM_OBJECT_UNLOCK(shared_page_obj); + return (res); +} + +static void +shared_page_init(void *dummy __unused) +{ + vm_page_t m; + + shared_page_obj = vm_pager_allocate(OBJT_PHYS, 0, PAGE_SIZE, + VM_PROT_DEFAULT, 0, NULL); + VM_OBJECT_LOCK(shared_page_obj); + m = vm_page_grab(shared_page_obj, 0, VM_ALLOC_RETRY | VM_ALLOC_NOBUSY | + VM_ALLOC_ZERO); + m->valid = VM_PAGE_BITS_ALL; + VM_OBJECT_UNLOCK(shared_page_obj); +} + +SYSINIT(shp, SI_SUB_EXEC, SI_ORDER_FIRST, (sysinit_cfunc_t)shared_page_init, + NULL); + +void +exec_sysvec_init(void *param) +{ + struct sysentvec *sv; + + sv = (struct sysentvec *)param; + + if ((sv->sv_flags & SV_SHP) == 0) + return; + sv->sv_shared_page_obj = shared_page_obj; + sv->sv_sigcode_base = sv->sv_shared_page_base + + shared_page_fill(*(sv->sv_szsigcode), 16, sv->sv_sigcode); +} Modified: head/sys/sys/imgact.h ============================================================================== --- head/sys/sys/imgact.h Sat Jan 8 16:02:14 2011 (r217150) +++ head/sys/sys/imgact.h Sat Jan 8 16:13:44 2011 (r217151) @@ -34,6 +34,8 @@ #include +#include + #define MAXSHELLCMDLEN PAGE_SIZE struct image_args { @@ -75,6 +77,7 @@ struct image_params { int canarylen; unsigned long pagesizes; int pagesizeslen; + vm_prot_t stack_prot; }; #ifdef _KERNEL Modified: head/sys/sys/sysent.h ============================================================================== --- head/sys/sys/sysent.h Sat Jan 8 16:02:14 2011 (r217150) +++ head/sys/sys/sysent.h Sat Jan 8 16:13:44 2011 (r217151) @@ -116,12 +116,17 @@ struct sysentvec { int (*sv_fetch_syscall_args)(struct thread *, struct syscall_args *); const char **sv_syscallnames; + vm_offset_t sv_shared_page_base; + vm_offset_t sv_shared_page_len; + vm_offset_t sv_sigcode_base; + void *sv_shared_page_obj; }; #define SV_ILP32 0x000100 #define SV_LP64 0x000200 #define SV_IA32 0x004000 #define SV_AOUT 0x008000 +#define SV_SHP 0x010000 #define SV_ABI_MASK 0xff #define SV_CURPROC_FLAG(x) (curproc->p_sysent->sv_flags & (x)) @@ -222,6 +227,13 @@ int lkmressys(struct thread *, struct no int syscall_thread_enter(struct thread *td, struct sysent *se); void syscall_thread_exit(struct thread *td, struct sysent *se); +int shared_page_fill(int size, int align, const char *data); +void exec_sysvec_init(void *param); + +#define INIT_SYSENTVEC(name, sv) \ + SYSINIT(name, SI_SUB_EXEC, SI_ORDER_ANY, \ + (sysinit_cfunc_t)exec_sysvec_init, sv); + #endif /* _KERNEL */ #endif /* !_SYS_SYSENT_H_ */