From owner-svn-src-projects@FreeBSD.ORG Sun May 20 08:58:30 2012 Return-Path: Delivered-To: svn-src-projects@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EC901106564A; Sun, 20 May 2012 08:58:30 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from fallbackmx10.syd.optusnet.com.au (fallbackmx10.syd.optusnet.com.au [211.29.132.251]) by mx1.freebsd.org (Postfix) with ESMTP id 2DF458FC08; Sun, 20 May 2012 08:58:29 +0000 (UTC) Received: from mail03.syd.optusnet.com.au (mail03.syd.optusnet.com.au [211.29.132.184]) by fallbackmx10.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id q4K8wNDd030784; Sun, 20 May 2012 18:58:23 +1000 Received: from c122-106-171-232.carlnfd1.nsw.optusnet.com.au (c122-106-171-232.carlnfd1.nsw.optusnet.com.au [122.106.171.232]) by mail03.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id q4K8wCef028222 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 20 May 2012 18:58:14 +1000 Date: Sun, 20 May 2012 18:58:12 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Andrew Turner In-Reply-To: <201205192351.q4JNpnAq053531@svn.freebsd.org> Message-ID: <20120520165107.D822@besplex.bde.org> References: <201205192351.q4JNpnAq053531@svn.freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: svn-src-projects@FreeBSD.org, src-committers@FreeBSD.org Subject: Re: svn commit: r235672 - in projects/arm_eabi/sys: amd64/include i386/include ia64/include mips/include pc98/include powerpc/include sparc64/include x86/include X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 May 2012 08:58:31 -0000 On Sat, 19 May 2012, Andrew Turner wrote: > Log: > Fix wchar support in the not ARM case. > > * Add machine/_wchar.h to define WCHAR_{MIN,MAX} and include it from > machine/_stdint.h, it is already in wchar.h. This adds 2 layers of include pessimizations to x86, though only 1 layer to other arches. The pessimization is most noticeable over nfs, where the RPCs for reopening include files can take 10-100 times longer than actually reading and parsing of the files for small files. This has some style bugs. > * Add the typedef for __wchar_t to machine/_types.h. The limits should be defined (with leading underscores) here too, so that no new includes are needed. > Added: projects/arm_eabi/sys/amd64/include/_wchar.h > ============================================================================== > --- /dev/null 00:00:00 1970 (empty, because file is newly added) > +++ projects/arm_eabi/sys/amd64/include/_wchar.h Sat May 19 23:51:48 2012 (r235672) > @@ -0,0 +1,6 @@ > +/*- > + * This file is in the public domain. > + */ > +/* $FreeBSD$ */ > + > +#include Going through x86 gives the second layer. Having an extra layer is especially ugly for new files (nothing to be compatible with), but seems to be needed even for them so that upper layers can keep using the generic pathname ). > Modified: projects/arm_eabi/sys/ia64/include/_stdint.h > ============================================================================== > --- projects/arm_eabi/sys/ia64/include/_stdint.h Sat May 19 23:25:57 2012 (r235671) > +++ projects/arm_eabi/sys/ia64/include/_stdint.h Sat May 19 23:51:48 2012 (r235672) > @@ -52,6 +52,8 @@ > > #if !defined(__cplusplus) || defined(__STDC_LIMIT_MACROS) > > +#include > + This should be done at a higher, MI level (hopefully just in and ). > /* > * ISO/IEC 9899:1999 > * 7.18.2.1 Limits of exact-width integer types > @@ -149,12 +151,6 @@ > /* Limit of size_t. */ > #define SIZE_MAX UINT64_MAX > > -#ifndef WCHAR_MIN /* Also possibly defined in */ > -/* Limits of wchar_t. */ > -#define WCHAR_MIN INT32_MIN > -#define WCHAR_MAX INT32_MAX > -#endif > - Once the limits are defined (with leading underscores) in machine/_types.h, everywhere that needs to define them without leading underscores can simply repeat the definitions without the ifdefs. This is not such a place, since the application-visible definition is only needed in sys/stdint.h (where you moved it). > /* Limits of wint_t. */ > #define WINT_MIN INT32_MIN > #define WINT_MAX INT32_MAX WINT_* will probably need the same treatment eventually. For the other limits, most are still declared without going through underscores in this file (machine/_stdint.h). Declaring them (with leading underscores) next to their typedefs in machine/_types.h doesn't work so well since it replaces a layer of include nesting/ obfuscation by a layer of #define nesting/obfuscation for no reason. For WCHAR_*, we need the layer of #define nesting for technical (anti-namespace-pollution) reasons. Elsewhere I complained about missing anti-namespace pollution in sys/time.h. It would be painful to do correctly. This commit does it correctly for just 2 symbols, but takes about 400 lines. That is too painful by a factor of about 100. The existence of sys/_stdint.h is an older quality of implementation bug for . It exists just to avoid duplicating (twice) 11 ifdefs that used to be in and (some ifdefs are needed since typedefs can't be repeated, unlike #defines). There is still a maze of ifdefs, and sys/_stdint.h increases the maze for the stdint.h includes too. The existence if sys/_null.h is an even older pessimization related to this one. It is not even clean, since it has MD ifdefs at the MI level. Better yet, its original reason for existence has been subverted so that it has been reduced to just C++ support, with a different and worse unconditional definition of NULL for C. Originally it gave a mixture of (conditional) definitions for C, with the mixture being intended primarily for hiding bugs on LP64 systems, but also helping to detect bugs by compiling the same sources on different arches. Now it gives a similar mixture for C++, but only for compatibility with old compilers. > Modified: projects/arm_eabi/sys/ia64/include/_types.h > ============================================================================== > --- projects/arm_eabi/sys/ia64/include/_types.h Sat May 19 23:25:57 2012 (r235671) > +++ projects/arm_eabi/sys/ia64/include/_types.h Sat May 19 23:51:48 2012 (r235672) > @@ -115,4 +115,6 @@ typedef char * __va_list; /* non-funct > #endif /* lint */ > #endif /* __GNUCLIKE_BUILTIN_VARARGS */ > > +typedef int __wchar_t; > + Style bug: new type in a new section after the gnu section, instead of sorted into the old general section. Similarly elsewhere. > #endif /* !_MACHINE__TYPES_H_ */ > > Added: projects/arm_eabi/sys/ia64/include/_wchar.h > ============================================================================== > --- /dev/null 00:00:00 1970 (empty, because file is newly added) > +++ projects/arm_eabi/sys/ia64/include/_wchar.h Sat May 19 23:51:48 2012 (r235672) > @@ -0,0 +1,37 @@ > +/* Style bug: missing indent protection. The copyright template /usr/share/examples/etc/bsd-style-copyright is supposed to prevent this style bug by not having it. > + * Copyright (C) 2011 Andrew Turner The copyright template also has a lower case (C) here, but otherwise seems to be identical with this one. > ... > + * $FreeBSD$ > + */ These little include files aren't so little when they have big copyrights in them. They may take hundreds if not thousands of nanoseconds to process. Include nesting gives tens if not hundreds of them per top-level include. > + > +#ifndef _IA64_INCLUDE__WCHAR_H_ > +#define _IA64_INCLUDE__WCHAR_H_ Style bug: space instead of tab after #define. Style bug: the idempotency guard for files in the directory is conventionally named _MACHINE_FOO_H_, not _${ARCH}_INCLUDE_FOO_H. This rule seems to have been followed in most cases. However, in x86/include, copying from the directory has resulted in _MACHINE_ being used for many files that aren't in the directory. itself is intentionally generic, but especially with include convolutions like the ones for and 32-bit sub-APIs and maybe arm endianness sub-APIs, you may want non-generic names to help untangle the mess. > + > +/* Limits of wchar_t. */ > +#define WCHAR_MIN __INT_MIN > +#define WCHAR_MAX __INT_MAX There is a minor scope problem here. This file is not self-sufficient, since it depends on machine/_limits.h. Moving the definitions to machine/_types.h doesn't change this problem, but gives another reason why _all_ the limits should be defined near their typedefs there. > + > +#endif /* _IA64_INCLUDE__WCHAR_H_ */ Style bug: backwards comment on #endif. > + Style bug: extra blank line before EOF. But this file is not needed. Similarly elsewhere. Bruce From owner-svn-src-projects@FreeBSD.ORG Tue May 22 02:37:04 2012 Return-Path: Delivered-To: svn-src-projects@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E8660106566C; Tue, 22 May 2012 02:37:04 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id D47578FC0A; Tue, 22 May 2012 02:37:04 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.4/8.14.4) with ESMTP id q4M2b4GG083154; Tue, 22 May 2012 02:37:04 GMT (envelope-from linimon@svn.freebsd.org) Received: (from linimon@localhost) by svn.freebsd.org (8.14.4/8.14.4/Submit) id q4M2b49e083152; Tue, 22 May 2012 02:37:04 GMT (envelope-from linimon@svn.freebsd.org) Message-Id: <201205220237.q4M2b49e083152@svn.freebsd.org> From: Mark Linimon Date: Tue, 22 May 2012 02:37:04 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org X-SVN-Group: projects MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r235770 - projects/portbuild/scripts X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 May 2012 02:37:05 -0000 Author: linimon (doc,ports committer) Date: Tue May 22 02:37:04 2012 New Revision: 235770 URL: http://svn.freebsd.org/changeset/base/235770 Log: Add two more cases to "fetch". These have arisen due to our new restrictions on package builds which were previously able to access the network post-"make extract". Several ports fail due to not obeying this restriction. While here, add a backstop to check to see if the log is incomplete. Since this rarely happens, put it all the way at the end. Modified: projects/portbuild/scripts/processonelog Modified: projects/portbuild/scripts/processonelog ============================================================================== --- projects/portbuild/scripts/processonelog Tue May 22 02:24:52 2012 (r235769) +++ projects/portbuild/scripts/processonelog Tue May 22 02:37:04 2012 (r235770) @@ -122,7 +122,7 @@ elif bzgrep -qE "((Can't|unable to) open reason="DISPLAY"; tag="display" elif bzgrep -qE '(No checksum recorded for|(Maybe|Either) .* is out of date, or)' $1; then reason="distinfo_update"; tag="distinfo" -elif bzgrep -qE "Member name contains .\.\." $1; then +elif bzgrep -qE "(error.*hostname nor servname provided|fetch:.*No address record|Member name contains .\.\.)" $1; then reason="fetch"; tag="fetch" elif bzgrep -qE "(pnohang: killing make checksum|fetch: transfer timed out)" $1; then reason="fetch_timeout"; tag="fetch-timeout" @@ -177,6 +177,10 @@ elif bzgrep -q "pkg_create: make_dist: t elif bzgrep -q "Segmentation fault" $1; then reason="segfault"; tag="segfault" +# note: searching for string-not-found here (builds that terminated early) +elif [ ! bzgrep -qE "^build of .* ended at" $1 ]; then + reason="cluster"; tag="cluster" + else reason="???"; tag="unknown" fi From owner-svn-src-projects@FreeBSD.ORG Wed May 23 09:38:38 2012 Return-Path: Delivered-To: svn-src-projects@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4302D1065672; Wed, 23 May 2012 09:38:38 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 2D9598FC19; Wed, 23 May 2012 09:38:38 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.4/8.14.4) with ESMTP id q4N9ccbR060836; Wed, 23 May 2012 09:38:38 GMT (envelope-from glebius@svn.freebsd.org) Received: (from glebius@localhost) by svn.freebsd.org (8.14.4/8.14.4/Submit) id q4N9cbVV060832; Wed, 23 May 2012 09:38:37 GMT (envelope-from glebius@svn.freebsd.org) Message-Id: <201205230938.q4N9cbVV060832@svn.freebsd.org> From: Gleb Smirnoff Date: Wed, 23 May 2012 09:38:37 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org X-SVN-Group: projects MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r235826 - projects/pf/head/sys/contrib/pf/net X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 May 2012 09:38:38 -0000 Author: glebius Date: Wed May 23 09:38:37 2012 New Revision: 235826 URL: http://svn.freebsd.org/changeset/base/235826 Log: Add some strictness to tableaddr and dynaddr setup and destroy functions. Instead of silently ignoring invalid parameters, assert their correctness. Don't call these functions for all addresses, but only for table addresses or dynamic addresses, respectively. Change may be arguable, since increases number of lines, but imho, make code easier to understand and easier for future modifications. Modified: projects/pf/head/sys/contrib/pf/net/pf_if.c projects/pf/head/sys/contrib/pf/net/pf_ioctl.c projects/pf/head/sys/contrib/pf/net/pfvar.h Modified: projects/pf/head/sys/contrib/pf/net/pf_if.c ============================================================================== --- projects/pf/head/sys/contrib/pf/net/pf_if.c Wed May 23 09:10:46 2012 (r235825) +++ projects/pf/head/sys/contrib/pf/net/pf_if.c Wed May 23 09:38:37 2012 (r235826) @@ -392,8 +392,9 @@ pfi_dynaddr_setup(struct pf_addr_wrap *a struct pf_ruleset *ruleset = NULL; int rv = 0; - if (aw->type != PF_ADDR_DYNIFTL) - return (0); + KASSERT(aw->type == PF_ADDR_DYNIFTL, ("%s: type %u", + __func__, aw->type)); + KASSERT(aw->p.dyn == NULL, ("%s: dyn is %p", __func__, aw->p.dyn)); if ((dyn = uma_zalloc(V_pfi_addr_z, M_NOWAIT | M_ZERO)) == NULL) return (ENOMEM); @@ -627,27 +628,26 @@ pfi_address_add(struct sockaddr *sa, int } void -pfi_dynaddr_remove(struct pf_addr_wrap *aw) +pfi_dynaddr_remove(struct pfi_dynaddr *dyn) { - if (aw->type != PF_ADDR_DYNIFTL || aw->p.dyn == NULL || - aw->p.dyn->pfid_kif == NULL || aw->p.dyn->pfid_kt == NULL) - return; + KASSERT(dyn->pfid_kif != NULL, ("%s: null pfid_kif", __func__)); + KASSERT(dyn->pfid_kt != NULL, ("%s: null pfid_kt", __func__)); - TAILQ_REMOVE(&aw->p.dyn->pfid_kif->pfik_dynaddrs, aw->p.dyn, entry); - pfi_kif_unref(aw->p.dyn->pfid_kif, PFI_KIF_REF_RULE); - aw->p.dyn->pfid_kif = NULL; - pfr_detach_table(aw->p.dyn->pfid_kt); - aw->p.dyn->pfid_kt = NULL; - uma_zfree(V_pfi_addr_z, aw->p.dyn); - aw->p.dyn = NULL; + TAILQ_REMOVE(&dyn->pfid_kif->pfik_dynaddrs, dyn, entry); + pfi_kif_unref(dyn->pfid_kif, PFI_KIF_REF_RULE); + pfr_detach_table(dyn->pfid_kt); + uma_zfree(V_pfi_addr_z, dyn); } void pfi_dynaddr_copyout(struct pf_addr_wrap *aw) { - if (aw->type != PF_ADDR_DYNIFTL || aw->p.dyn == NULL || - aw->p.dyn->pfid_kif == NULL) + + KASSERT(aw->type == PF_ADDR_DYNIFTL, + ("%s: type %u", __func__, aw->type)); + + if (aw->p.dyn == NULL || aw->p.dyn->pfid_kif == NULL) return; aw->p.dyncnt = aw->p.dyn->pfid_acnt4 + aw->p.dyn->pfid_acnt6; } Modified: projects/pf/head/sys/contrib/pf/net/pf_ioctl.c ============================================================================== --- projects/pf/head/sys/contrib/pf/net/pf_ioctl.c Wed May 23 09:10:46 2012 (r235825) +++ projects/pf/head/sys/contrib/pf/net/pf_ioctl.c Wed May 23 09:38:37 2012 (r235826) @@ -172,9 +172,6 @@ struct cdev *pf_dev; static void pf_clear_states(void); static int pf_clear_tables(void); static void pf_clear_srcnodes(struct pf_src_node *); -static int pf_tbladdr_setup(struct pf_ruleset *, - struct pf_addr_wrap *); -static void pf_tbladdr_remove(struct pf_addr_wrap *); static void pf_tbladdr_copyout(struct pf_addr_wrap *); /* @@ -371,14 +368,20 @@ pf_mv_pool(struct pf_palist *poola, stru static void pf_empty_pool(struct pf_palist *poola) { - struct pf_pooladdr *empty_pool_pa; + struct pf_pooladdr *pa; - while ((empty_pool_pa = TAILQ_FIRST(poola)) != NULL) { - pfi_dynaddr_remove(&empty_pool_pa->addr); - pf_tbladdr_remove(&empty_pool_pa->addr); - pfi_kif_unref(empty_pool_pa->kif, PFI_KIF_REF_RULE); - TAILQ_REMOVE(poola, empty_pool_pa, entries); - uma_zfree(V_pf_pooladdr_z, empty_pool_pa); + while ((pa = TAILQ_FIRST(poola)) != NULL) { + switch (pa->addr.type) { + case PF_ADDR_DYNIFTL: + pfi_dynaddr_remove(pa->addr.p.dyn); + break; + case PF_ADDR_TABLE: + pfr_detach_table(pa->addr.p.tbl); + break; + } + pfi_kif_unref(pa->kif, PFI_KIF_REF_RULE); + TAILQ_REMOVE(poola, pa, entries); + uma_zfree(V_pf_pooladdr_z, pa); } } @@ -407,10 +410,22 @@ pf_free_rule(struct pf_rule *rule) pf_qid_unref(rule->pqid); pf_qid_unref(rule->qid); #endif - pfi_dynaddr_remove(&rule->src.addr); - pfi_dynaddr_remove(&rule->dst.addr); - pf_tbladdr_remove(&rule->src.addr); - pf_tbladdr_remove(&rule->dst.addr); + switch (rule->src.addr.type) { + case PF_ADDR_DYNIFTL: + pfi_dynaddr_remove(rule->src.addr.p.dyn); + break; + case PF_ADDR_TABLE: + pfr_detach_table(rule->src.addr.p.tbl); + break; + } + switch (rule->dst.addr.type) { + case PF_ADDR_DYNIFTL: + pfi_dynaddr_remove(rule->dst.addr.p.dyn); + break; + case PF_ADDR_TABLE: + pfr_detach_table(rule->dst.addr.p.tbl); + break; + } if (rule->overload_tbl) pfr_detach_table(rule->overload_tbl); pfi_kif_unref(rule->kif, PFI_KIF_REF_RULE); @@ -968,11 +983,18 @@ static int pf_addr_setup(struct pf_ruleset *ruleset, struct pf_addr_wrap *addr, sa_family_t af) { - int error; + int error = 0; - error = pfi_dynaddr_setup(addr, af); - if (error == 0) - error = pf_tbladdr_setup(ruleset, addr); + switch (addr->type) { + case PF_ADDR_TABLE: + addr->p.tbl = pfr_attach_table(ruleset, addr->v.tblname); + if (addr->p.tbl == NULL) + error = ENOMEM; + break; + case PF_ADDR_DYNIFTL: + error = pfi_dynaddr_setup(addr, af); + break; + } return (error); } @@ -980,8 +1002,15 @@ pf_addr_setup(struct pf_ruleset *ruleset static void pf_addr_copyout(struct pf_addr_wrap *addr) { - pfi_dynaddr_copyout(addr); - pf_tbladdr_copyout(addr); + + switch (addr->type) { + case PF_ADDR_DYNIFTL: + pfi_dynaddr_copyout(addr); + break; + case PF_ADDR_TABLE: + pf_tbladdr_copyout(addr); + break; + } } static int @@ -1268,14 +1297,18 @@ pfioctl(struct cdev *dev, u_long cmd, ca error = EINVAL; #endif if (pf_addr_setup(ruleset, &rule->src.addr, rule->af)) - error = EINVAL; + error = ENOMEM; if (pf_addr_setup(ruleset, &rule->dst.addr, rule->af)) - error = EINVAL; + error = ENOMEM; if (pf_anchor_setup(rule, ruleset, pr->anchor_call)) error = EINVAL; TAILQ_FOREACH(pa, &V_pf_pabuf, entries) - if (pf_tbladdr_setup(ruleset, &pa->addr)) - error = EINVAL; + if (pa->addr.type == PF_ADDR_TABLE) { + pa->addr.p.tbl = pfr_attach_table(ruleset, + pa->addr.v.tblname); + if (pa->addr.p.tbl == NULL) + error = ENOMEM; + } if (rule->overload_tblname[0]) { if ((rule->overload_tbl = pfr_attach_table(ruleset, @@ -1528,14 +1561,19 @@ pfioctl(struct cdev *dev, u_long cmd, ca error = EINVAL; #endif if (pf_addr_setup(ruleset, &newrule->src.addr, newrule->af)) - error = EINVAL; + error = ENOMEM; if (pf_addr_setup(ruleset, &newrule->dst.addr, newrule->af)) - error = EINVAL; + error = ENOMEM; if (pf_anchor_setup(newrule, ruleset, pcr->anchor_call)) error = EINVAL; TAILQ_FOREACH(pa, &V_pf_pabuf, entries) - if (pf_tbladdr_setup(ruleset, &pa->addr)) - error = EINVAL; + if (pa->addr.type == PF_ADDR_TABLE) { + pa->addr.p.tbl = + pfr_attach_table(ruleset, + pa->addr.v.tblname); + if (pa->addr.p.tbl == NULL) + error = ENOMEM; + } if (newrule->overload_tblname[0]) { if ((newrule->overload_tbl = pfr_attach_table( @@ -2236,9 +2274,8 @@ DIOCGETSTATES_full: } pfi_kif_ref(pa->kif, PFI_KIF_REF_RULE); } - error = pfi_dynaddr_setup(&pa->addr, pp->af); - if (error) { - pfi_dynaddr_remove(&pa->addr); + if (pa->addr.type == PF_ADDR_DYNIFTL && ((error = + pfi_dynaddr_setup(&pa->addr, pp->af)) != 0)) { pfi_kif_unref(pa->kif, PFI_KIF_REF_RULE); PF_UNLOCK(); uma_zfree(V_pf_pooladdr_z, pa); @@ -2362,10 +2399,20 @@ DIOCGETSTATES_full: pfi_kif_ref(newpa->kif, PFI_KIF_REF_RULE); } else newpa->kif = NULL; - if ((error = pfi_dynaddr_setup(&newpa->addr, - pca->af)) != 0 || ((error = - pf_tbladdr_setup(ruleset, &newpa->addr)) != 0 )) { - pfi_dynaddr_remove(&newpa->addr); + + switch (newpa->addr.type) { + case PF_ADDR_DYNIFTL: + error = pfi_dynaddr_setup(&newpa->addr, + pca->af); + break; + case PF_ADDR_TABLE: + newpa->addr.p.tbl = pfr_attach_table(ruleset, + newpa->addr.v.tblname); + if (newpa->addr.p.tbl == NULL) + error = ENOMEM; + break; + } + if (error) { pfi_kif_unref(newpa->kif, PFI_KIF_REF_RULE); PF_UNLOCK(); uma_zfree(V_pf_pooladdr_z, newpa); @@ -2394,8 +2441,14 @@ DIOCGETSTATES_full: if (pca->action == PF_CHANGE_REMOVE) { TAILQ_REMOVE(&pool->list, oldpa, entries); - pfi_dynaddr_remove(&oldpa->addr); - pf_tbladdr_remove(&oldpa->addr); + switch (oldpa->addr.type) { + case PF_ADDR_DYNIFTL: + pfi_dynaddr_remove(oldpa->addr.p.dyn); + break; + case PF_ADDR_TABLE: + pfr_detach_table(oldpa->addr.p.tbl); + break; + } pfi_kif_unref(oldpa->kif, PFI_KIF_REF_RULE); uma_zfree(V_pf_pooladdr_z, oldpa); } else { @@ -3342,32 +3395,14 @@ pfsync_state_export(struct pfsync_state } -static int -pf_tbladdr_setup(struct pf_ruleset *rs, struct pf_addr_wrap *aw) -{ - if (aw->type != PF_ADDR_TABLE) - return (0); - if ((aw->p.tbl = pfr_attach_table(rs, aw->v.tblname)) == NULL) - return (ENOMEM); - return (0); -} - -static void -pf_tbladdr_remove(struct pf_addr_wrap *aw) -{ - if (aw->type != PF_ADDR_TABLE || aw->p.tbl == NULL) - return; - pfr_detach_table(aw->p.tbl); - aw->p.tbl = NULL; -} - static void pf_tbladdr_copyout(struct pf_addr_wrap *aw) { - struct pfr_ktable *kt = aw->p.tbl; + struct pfr_ktable *kt; - if (aw->type != PF_ADDR_TABLE || kt == NULL) - return; + KASSERT(aw->type == PF_ADDR_TABLE, ("%s: type %u", __func__, aw->type)); + + kt = aw->p.tbl; if (!(kt->pfrkt_flags & PFR_TFLAG_ACTIVE) && kt->pfrkt_root != NULL) kt = kt->pfrkt_root; aw->p.tbl = NULL; Modified: projects/pf/head/sys/contrib/pf/net/pfvar.h ============================================================================== --- projects/pf/head/sys/contrib/pf/net/pfvar.h Wed May 23 09:10:46 2012 (r235825) +++ projects/pf/head/sys/contrib/pf/net/pfvar.h Wed May 23 09:38:37 2012 (r235826) @@ -1934,7 +1934,7 @@ int pfi_kif_match(struct pfi_kif *, st int pfi_match_addr(struct pfi_dynaddr *, struct pf_addr *, sa_family_t); int pfi_dynaddr_setup(struct pf_addr_wrap *, sa_family_t); -void pfi_dynaddr_remove(struct pf_addr_wrap *); +void pfi_dynaddr_remove(struct pfi_dynaddr *); void pfi_dynaddr_copyout(struct pf_addr_wrap *); void pfi_update_status(const char *, struct pf_status *); void pfi_get_ifaces(const char *, struct pfi_kif *, int *); From owner-svn-src-projects@FreeBSD.ORG Thu May 24 12:02:11 2012 Return-Path: Delivered-To: svn-src-projects@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6C400106567B; Thu, 24 May 2012 12:02:11 +0000 (UTC) (envelope-from cherry@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 54AC88FC23; Thu, 24 May 2012 12:02:11 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.4/8.14.4) with ESMTP id q4OC2BIm036892; Thu, 24 May 2012 12:02:11 GMT (envelope-from cherry@svn.freebsd.org) Received: (from cherry@localhost) by svn.freebsd.org (8.14.4/8.14.4/Submit) id q4OC2BFp036889; Thu, 24 May 2012 12:02:11 GMT (envelope-from cherry@svn.freebsd.org) Message-Id: <201205241202.q4OC2BFp036889@svn.freebsd.org> From: "Cherry G. Mathew" Date: Thu, 24 May 2012 12:02:11 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org X-SVN-Group: projects MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r235904 - in projects/amd64_xen_pv/sys: amd64/xen conf X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 May 2012 12:02:11 -0000 Author: cherry Date: Thu May 24 12:02:10 2012 New Revision: 235904 URL: http://svn.freebsd.org/changeset/base/235904 Log: This API is an attempt to abstract the MMU state in an MI fashion. It is heavily wip and may or may not go away, from amd64/ depending on how things go with the direct mapped implementation Added: projects/amd64_xen_pv/sys/amd64/xen/mmu_map.c projects/amd64_xen_pv/sys/amd64/xen/mmu_map.h Modified: projects/amd64_xen_pv/sys/conf/files.amd64 Added: projects/amd64_xen_pv/sys/amd64/xen/mmu_map.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ projects/amd64_xen_pv/sys/amd64/xen/mmu_map.c Thu May 24 12:02:10 2012 (r235904) @@ -0,0 +1,389 @@ +/* $FreeBSD$ */ +/*- + * Copyright (c) 2011-2012 Spectra Logic Corporation + * All rights reserved. + * + * This software was developed by Cherry G. Mathew + * under sponsorship from Spectra Logic Corporation. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions, and the following disclaimer, + * without modification. + * 2. Redistributions in binary form must reproduce at minimum a disclaimer + * substantially similar to the "NO WARRANTY" disclaimer below + * ("Disclaimer") and any redistribution must be conditioned upon + * including a substantially similar Disclaimer requirement for further + * binary redistribution. + * + * NO WARRANTY + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * HOLDERS OR CONTRIBUTORS BE LIABLE FOR SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, + * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING + * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE + * POSSIBILITY OF SUCH DAMAGES. + */ + + +/* + * This file implements the API that manages the page table + * hierarchy for the amd64 Xen pmap. + */ +#include +__FBSDID("$FreeBSD$"); + +#include "opt_cpu.h" +#include "opt_pmap.h" +#include "opt_smp.h" + + +#include +#include +#include + +#include +#include +#include + +#include +#include + +#include + +static int +pml4t_index(vm_offset_t va) +{ + /* amd64 sign extends 48th bit and upwards */ + const uint64_t SIGNMASK = (1UL << 48) - 1; + va &= SIGNMASK; /* Remove sign extension */ + + return (va >> PML4SHIFT); +} + +static int +pdpt_index(vm_offset_t va) +{ + /* amd64 sign extends 48th bit and upwards */ + const uint64_t SIGNMASK = (1UL << 48) - 1; + va &= SIGNMASK; /* Remove sign extension */ + + return ((va & PML4MASK) >> PDPSHIFT); +} + +static int +pdt_index(vm_offset_t va) +{ + /* amd64 sign extends 48th bit and upwards */ + const uint64_t SIGNMASK = (1UL << 48) - 1; + va &= SIGNMASK; /* Remove sign extension */ + + return ((va & PDPMASK) >> PDRSHIFT); +} + +/* + * The table get functions below assume that a table cannot exist at + * address 0 + */ +static pml4_entry_t * +pmap_get_pml4t(struct pmap *pm) +{ + KASSERT(pm != NULL, + ("NULL pmap passed in!\n")); + + pml4_entry_t *pm_pml4 = pm->pm_pml4; + + KASSERT(pm_pml4 != NULL, + ("pmap has NULL pml4!\n")); + + return pm->pm_pml4; +} + +/* Returns physical address */ +static vm_paddr_t +pmap_get_pdpt(vm_offset_t va, pml4_entry_t *pml4t) +{ + pml4_entry_t pml4e; + + KASSERT(va <= VM_MAX_KERNEL_ADDRESS, + ("invalid address requested")); + KASSERT(pml4t != 0, ("pml4t cannot be zero")); + + pml4e = pml4t[pml4t_index(va)]; + + if (!(pml4e & PG_V)) { + return 0; + } + + return xpmap_mtop(pml4e & PG_FRAME); +} + +/* Returns physical address */ +static vm_paddr_t +pmap_get_pdt(vm_offset_t va, pdp_entry_t *pdpt) +{ + pdp_entry_t pdpe; + + KASSERT(va <= VM_MAX_KERNEL_ADDRESS, + ("invalid address requested")); + KASSERT(pdpt != 0, ("pdpt cannot be zero")); + + pdpe = pdpt[pdpt_index(va)]; + + if (!(pdpe & PG_V)) { + return 0; + } + + return xpmap_mtop(pdpe & PG_FRAME); +} + +/* Returns physical address */ +static vm_paddr_t +pmap_get_pt(vm_offset_t va, pd_entry_t *pdt) +{ + pd_entry_t pdte; + + KASSERT(va <= VM_MAX_KERNEL_ADDRESS, + ("invalid address requested")); + + KASSERT(pdt != 0, ("pdt cannot be zero")); + + pdte = pdt[pdt_index(va)]; + + if (!(pdte & PG_V)) { + return 0; + } + + return xpmap_mtop(pdte & PG_FRAME); +} + +/* + * This structure defines the 4 indices that a given virtual + * address lookup would traverse. + * + * Note: this structure is opaque to API customers. Callers give us an + * untyped array which is marshalled/unmarshalled inside of the + * stateful api. + */ + +static const uint64_t SANE = 0xcafebabe; + +struct mmu_map_index { + pml4_entry_t *pml4t; /* Page Map Level 4 Table */ + pdp_entry_t *pdpt; /* Page Directory Pointer Table */ + pd_entry_t *pdt; /* Page Directory Table */ + pt_entry_t *pt; /* Page Table */ + + struct mmu_map_mbackend ptmb; /* Backend info */ + + uint32_t sanity; /* 32 bit (for alignment) magic XXX: + * Make optional on DEBUG */ +}; + +size_t +mmu_map_t_size(void) +{ + return sizeof (struct mmu_map_index); +} + +void +mmu_map_t_init(void *addr, struct mmu_map_mbackend *mb) +{ + KASSERT((addr != NULL) && (mb != NULL), ("NULL args given!")); + struct mmu_map_index *pti = addr; + KASSERT(pti->sanity != SANE, ("index initialised twice!")); + KASSERT(mb->alloc != NULL && + mb->ptov != NULL && + mb->vtop != NULL, + ("initialising with pre-registered alloc routine active")); + + pti->ptmb = *mb; + + /* Backend page allocation should provide default VA mapping */ + pti->sanity = SANE; +} + +void mmu_map_t_fini(void *addr) +{ + KASSERT(addr != NULL, ("NULL args given!")); + + struct mmu_map_index *pti = addr; + KASSERT(pti->sanity == SANE, ("Uninitialised index cookie used")); + struct mmu_map_mbackend *mb = &pti->ptmb; + + pti->sanity = 0; + + if (mb->free != NULL) { + /* XXX: go through PT hierarchy and free + unmap + * unused tables */ + } +} + +pd_entry_t * +mmu_map_pml4t(void *addr) +{ + KASSERT(addr != NULL, ("NULL args given!")); + struct mmu_map_index *pti = addr; + + KASSERT(pti->sanity == SANE, ("Uninitialised index cookie used")); + + return pti->pml4t; +} + +pd_entry_t * +mmu_map_pdpt(void *addr) +{ + KASSERT(addr != NULL, ("NULL args given!")); + struct mmu_map_index *pti = addr; + + KASSERT(pti->sanity == SANE, ("Uninitialised index cookie used")); + + return pti->pdpt; +} + +pd_entry_t * +mmu_map_pdt(void *addr) +{ + KASSERT(addr != NULL, ("NULL args given!")); + struct mmu_map_index *pti = addr; + + KASSERT(pti->sanity == SANE, ("Uninitialised index cookie used")); + + return pti->pdt; +} + +pd_entry_t * +mmu_map_pt(void *addr) +{ + KASSERT(addr != NULL, ("NULL args given!")); + struct mmu_map_index *pti = addr; + + KASSERT(pti->sanity == SANE, ("Uninitialised index cookie used")); + + return pti->pt; +} + +bool +mmu_map_inspect_va(struct pmap *pm, void *addr, vm_offset_t va) +{ + KASSERT(addr != NULL && pm != NULL, ("NULL arg(s) given")); + + struct mmu_map_index *pti = addr; + KASSERT(pti->sanity == SANE, ("Uninitialised index cookie used")); + + vm_paddr_t pt; + + pti->pml4t = pmap_get_pml4t(pm); + + pt = pmap_get_pdpt(va, pti->pml4t); + + if (pt == 0) { + return false; + } else { + pti->pdpt = (pdp_entry_t *) pti->ptmb.ptov(pt); + } + + pt = pmap_get_pdt(va, pti->pdpt); + + if (pt == 0) { + return false; + } else { + pti->pdt = (pd_entry_t *) pti->ptmb.ptov(pt); + } + + pt = pmap_get_pt(va, pti->pdt); + + if (pt == 0) { + return false; + } else { + pti->pt = (pt_entry_t *)pti->ptmb.ptov(pt); + } + + return true; +} +extern uint64_t xenstack; /* The stack Xen gives us at boot */ +void +mmu_map_hold_va(struct pmap *pm, void *addr, vm_offset_t va) +{ + KASSERT(addr != NULL && pm != NULL, ("NULL arg(s) given")); + + struct mmu_map_index *pti = addr; + KASSERT(pti->sanity == SANE, ("Uninitialised index cookie used")); + + vm_paddr_t pt; + + pti->pml4t = pmap_get_pml4t(pm); + + pt = pmap_get_pdpt(va, pti->pml4t); + + if (pt == 0) { + pml4_entry_t *pml4tep; + vm_paddr_t pml4tep_ma; + pml4_entry_t pml4te; + + pti->pdpt = (pdp_entry_t *)pti->ptmb.alloc(PAGE_SIZE); + + pml4tep = &pti->pml4t[pml4t_index(va)]; + pml4tep_ma = xpmap_ptom(pti->ptmb.vtop((vm_offset_t)pml4tep)); + pml4te = xpmap_ptom(pti->ptmb.vtop((vm_offset_t)pti->pdpt)) | PG_RW | PG_V | PG_U; /* XXX: revisit flags */ + xen_queue_pt_update(pml4tep_ma, pml4te); + + } else { + pti->pdpt = (pdp_entry_t *) pti->ptmb.ptov(pt); + } + + pt = pmap_get_pdt(va, pti->pdpt); + + if (pt == 0) { + pdp_entry_t *pdptep; + vm_paddr_t pdptep_ma; + pdp_entry_t pdpte; + + pti->pdt = (pd_entry_t *)pti->ptmb.alloc(PAGE_SIZE); + + pdptep = &pti->pdpt[pdpt_index(va)]; + pdptep_ma = xpmap_ptom(pti->ptmb.vtop((vm_offset_t)pdptep)); + pdpte = xpmap_ptom(pti->ptmb.vtop((vm_offset_t)pti->pdt)) | PG_RW | PG_V | PG_U; /* XXX: revisit flags */ + xen_queue_pt_update(pdptep_ma, pdpte); + + } else { + pti->pdt = (pd_entry_t *) pti->ptmb.ptov(pt); + } + + pt = pmap_get_pt(va, pti->pdt); + + if (pt == 0) { + pd_entry_t *pdtep; + vm_paddr_t pdtep_ma; + pd_entry_t pdte; + + pti->pt = (pt_entry_t *) pti->ptmb.alloc(PAGE_SIZE); + + pdtep = &pti->pdt[pdt_index(va)]; + pdtep_ma = xpmap_ptom(pti->ptmb.vtop((vm_offset_t)pdtep)); + pdte = xpmap_ptom(pti->ptmb.vtop((vm_offset_t)pti->pt)) | PG_RW | PG_V | PG_U; /* XXX: revisit flags */ + xen_queue_pt_update(pdtep_ma, pdte); + + } else { + pti->pt = (pt_entry_t *) pti->ptmb.ptov(pt); + } +} + +void +mmu_map_release_va(struct pmap *pm, void *addr, vm_offset_t va) +{ + + KASSERT(addr != NULL && pm != NULL, ("NULL arg(s) given")); + + struct mmu_map_index *pti = addr; + KASSERT(pti->sanity == SANE, ("Uninitialised index cookie used")); + + /* XXX: */ +} Added: projects/amd64_xen_pv/sys/amd64/xen/mmu_map.h ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ projects/amd64_xen_pv/sys/amd64/xen/mmu_map.h Thu May 24 12:02:10 2012 (r235904) @@ -0,0 +1,152 @@ +/* $FreeBSD$ */ +/*- + * Copyright (c) 2011-2012 Spectra Logic Corporation + * All rights reserved. + * + * This software was developed by Cherry G. Mathew + * under sponsorship from Spectra Logic Corporation. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions, and the following disclaimer, + * without modification. + * 2. Redistributions in binary form must reproduce at minimum a disclaimer + * substantially similar to the "NO WARRANTY" disclaimer below + * ("Disclaimer") and any redistribution must be conditioned upon + * including a substantially similar Disclaimer requirement for further + * binary redistribution. + * + * NO WARRANTY + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * HOLDERS OR CONTRIBUTORS BE LIABLE FOR SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, + * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING + * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE + * POSSIBILITY OF SUCH DAMAGES. + */ + +#ifndef _XEN_MMU_MAP_H_ +#define _XEN_MMU_MAP_H_ + +#include + +#include + +/* + * + * This API abstracts, in an MI fashion, the paging mechanism of an + * arbitrary CPU architecture as an opaque FSM, which may then be + * subject to inspection in MD ways. + * + * Use of this API can have the following effects on the VM system and + * the kernel address space: + * + * - physical memory pages may be allocated. + * - physical memory pages may be de-allocated. + * - kernel virtual address space may be allocated. + * - kernel virtual address space may be de-allocated. + * - The page table hierarchy may be modified. + * - TLB entries may be invalidated. + * + * The API is stateful, and designed around the following principles: + * - Simplicity + * - Object orientation + * - Code reuse. + */ + +/* + * We hide the page table structure behind an opaque "index" cookie + * which acts as the "key" to a given va->pa mapping being inspected. + */ +typedef void * mmu_map_t; + +/* + * Memory backend types: + * + * We provide a means to allocate ad-hoc memory/physical page + * requirements to the paging mechanism by means of a "backend" + * alloc function + * + * The memory backend is required to provide physical pages that are + * at least temporarily mapped into the kernel VA space and whose + * contents are thus accessible by a simple pointer indirection from + * within the kernel. This requirement may be revoked after conclusion + * of an instance of stateful usage of the API ( See: + * mmu_map_t_fini() below ), at which point the backend + * implementation is free to unmap any temporary mappings if so + * desired. (XXX: review this for non-x86) + * + * Note: Only the mappings may be revoked - any physical pages + * themselves allocated by the backend are considered allocated and + * part of the paging mechanism. + */ + +struct mmu_map_mbackend { /* Callbacks */ + + vm_offset_t (*alloc)(size_t); + void (*free)(vm_offset_t); /* May be NULL */ + + /* + * vtop()/ptov() conversion functions: + * These callbacks typically provide conversions for mapped + * pages allocated via the alloc()/free() callbacks (above). + * The API implementation is free to cache the mappings across + * multiple instances of use; ie; mappings may persist across + * one pair of mmu_map_t_init()/.._finit() calls. + */ + vm_offset_t (*ptov)(vm_paddr_t); + vm_paddr_t (*vtop)(vm_offset_t); +}; + +/* + * Return sizeof (mmu_map_t) as implemented within the api + * This may then be used to allocate untyped memory for the cookie + * which can then be operated on opaquely behind the API in a machine + * specific manner. + */ +size_t mmu_map_t_size(void); + +/* + * Initialise the API state to use a specified memory backend + */ +void mmu_map_t_init(mmu_map_t, struct mmu_map_mbackend *); + +/* Conclude this instance of use of the API */ +void mmu_map_t_fini(mmu_map_t); + +/* Set "index" cookie state based on va lookup. This state may then be + * inspected in MD ways ( See below ). Note that every call to the + * following functions can change the state of the backing paging + * mechanism FSM. + */ +bool mmu_map_inspect_va(struct pmap *, mmu_map_t, vm_offset_t); +/* + * Unconditionally allocate resources to setup and "inspect" (as + * above) a given va->pa mapping + */ +void mmu_map_hold_va(struct pmap *, mmu_map_t, vm_offset_t); + +/* Optionally release resources after tear down of a va->pa mapping */ +void mmu_map_release_va(struct pmap *, mmu_map_t, vm_offset_t); + +/* + * Machine dependant "view" into the page table hierarchy FSM. + * On amd64, there are four tables that are consulted for a va->pa + * translation. This information may be extracted by the MD functions + * below and is only considered valid between a successful call to + * mmu_map_inspect_va() or mmu_map_hold_va() and a subsequent + * call to mmu_map_release_va() + */ +pd_entry_t * mmu_map_pml4t(mmu_map_t); /* Page Map Level 4 Table */ +pd_entry_t * mmu_map_pdpt(mmu_map_t); /* Page Directory Pointer Table */ +pd_entry_t * mmu_map_pdt(mmu_map_t); /* Page Directory Table */ +pd_entry_t * mmu_map_pt(mmu_map_t); /* Page Table */ + +#endif /* !_XEN_MMU_MAP_H_ */ Modified: projects/amd64_xen_pv/sys/conf/files.amd64 ============================================================================== --- projects/amd64_xen_pv/sys/conf/files.amd64 Thu May 24 11:52:57 2012 (r235903) +++ projects/amd64_xen_pv/sys/conf/files.amd64 Thu May 24 12:02:10 2012 (r235904) @@ -128,6 +128,7 @@ amd64/amd64/mpboot.S optional native sm amd64/xen/mpboot.c optional xen smp amd64/amd64/pmap.c optional native amd64/xen/pmap.c optional xen +amd64/xen/mmu_map.c optional xen amd64/amd64/prof_machdep.c optional profiling-routine amd64/amd64/ptrace_machdep.c standard amd64/amd64/sigtramp.S standard From owner-svn-src-projects@FreeBSD.ORG Thu May 24 12:08:35 2012 Return-Path: Delivered-To: svn-src-projects@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 35C13106568A; Thu, 24 May 2012 12:08:35 +0000 (UTC) (envelope-from cherry@zyx.in) Received: from xsmtp10.mail2web.com (xsmtp10.mail2web.com [168.144.250.227]) by mx1.freebsd.org (Postfix) with ESMTP id 0943B8FC1A; Thu, 24 May 2012 12:08:35 +0000 (UTC) Received: from [10.5.41.26] (helo=M2W126.mail2web.com) by xsmtp10.mail2web.com with smtp (Exim 4.63) (envelope-from ) id 1SXWqH-00040U-9I; Thu, 24 May 2012 08:08:34 -0400 Message-ID: <380-22012542412927450@M2W126.mail2web.com> X-Priority: 3 X-Originating-IP: 81-102-251-192 X-URL: http://mail2web.com/ From: "cherry@zyx.in" To: cherry@freebsd.org, src-committers@freebsd.org, svn-src-projects@freebsd.org Date: Thu, 24 May 2012 08:09:27 -0400 MIME-Version: 1.0 Content-type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Spam-Relays-Trusted: [ ip=10.5.41.26 rdns= helo=M2W126.mail2web.com by=xsmtp10.mail2web.com ident= envfrom=cherry@zyx.in intl=1 id=1SXWqH-00040U-9I auth= msa=0 ] X-Spam-Relays-Untrusted: X-Spam-Relays-Internal: [ ip=10.5.41.26 rdns= helo=M2W126.mail2web.com by=xsmtp10.mail2web.com ident= envfrom=cherry@zyx.in intl=1 id=1SXWqH-00040U-9I auth= msa=0 ] X-Spam-Relays-External: Cc: Subject: RE: svn commit: r235904 - in projects/amd64_xen_pv/sys: amd64/xen conf X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: cherry@zyx.in List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 May 2012 12:08:35 -0000 Apologies, I've not used the proper commit template - is there a way to fi= x this, or shall I write an email with the correct log subsequently into thi= s thread =3F Cheers, Cherry -------------------------------------------------------------------- myhosting=2Ecom - Premium Microsoft=AE Windows=AE and Linux web and applic= ation hosting - http://link=2Emyhosting=2Ecom/myhosting From owner-svn-src-projects@FreeBSD.ORG Thu May 24 12:27:28 2012 Return-Path: Delivered-To: svn-src-projects@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 02D201065686; Thu, 24 May 2012 12:27:28 +0000 (UTC) (envelope-from cherry@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id E16D18FC12; Thu, 24 May 2012 12:27:27 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.4/8.14.4) with ESMTP id q4OCRR1p038000; Thu, 24 May 2012 12:27:27 GMT (envelope-from cherry@svn.freebsd.org) Received: (from cherry@localhost) by svn.freebsd.org (8.14.4/8.14.4/Submit) id q4OCRRhI037998; Thu, 24 May 2012 12:27:27 GMT (envelope-from cherry@svn.freebsd.org) Message-Id: <201205241227.q4OCRRhI037998@svn.freebsd.org> From: "Cherry G. Mathew" Date: Thu, 24 May 2012 12:27:27 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org X-SVN-Group: projects MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r235905 - projects/amd64_xen_pv/sys/amd64/xen X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 May 2012 12:27:28 -0000 Author: cherry Date: Thu May 24 12:27:27 2012 New Revision: 235905 URL: http://svn.freebsd.org/changeset/base/235905 Log: Bring in more very early thread0 startup code from native Approved by: gibbs (implicit) Modified: projects/amd64_xen_pv/sys/amd64/xen/machdep.c Modified: projects/amd64_xen_pv/sys/amd64/xen/machdep.c ============================================================================== --- projects/amd64_xen_pv/sys/amd64/xen/machdep.c Thu May 24 12:02:10 2012 (r235904) +++ projects/amd64_xen_pv/sys/amd64/xen/machdep.c Thu May 24 12:27:27 2012 (r235905) @@ -60,6 +60,8 @@ #include #include #include +#include +#include #include #include #include @@ -98,7 +100,10 @@ start_info_t *xen_start_info; shared_info_t *HYPERVISOR_shared_info; xen_pfn_t *xen_machine_phys = machine_to_phys_mapping; xen_pfn_t *xen_phys_machine; -vm_paddr_t phys_avail[0]; /* XXX: todo */ + +#define PHYSMAP_SIZE (2 * VM_PHYSSEG_MAX) +vm_offset_t pa_index = 0; +vm_paddr_t phys_avail[PHYSMAP_SIZE + 2]; vm_paddr_t dump_avail[0]; /* XXX: todo */ struct pcpu __pcpu[MAXCPU]; @@ -111,6 +116,8 @@ struct mtx dt_lock; /* lock for GDT and vm_paddr_t initxen(struct start_info *); +extern void identify_cpu(void); + static void get_fpcontext(struct thread *td, mcontext_t *mcp); static int set_fpcontext(struct thread *td, const mcontext_t *mcp, char *xfpustate, size_t xfpustate_len); @@ -268,6 +275,7 @@ initxen(struct start_info *si) KASSERT(si != NULL, ("start_info invalid")); + /* global variables */ xen_start_info = si; /* xen variables */ @@ -275,6 +283,7 @@ initxen(struct start_info *si) physmem = si->nr_pages; Maxmem = si->nr_pages + 1; + memset(phys_avail, 0, sizeof phys_avail); /* * Setup kernel tls registers. pcpu needs them, and other @@ -297,21 +306,31 @@ initxen(struct start_info *si) /* Address of lowest unused page */ physfree = VTOP(si->pt_base + si->nr_pt_frames * PAGE_SIZE); + /* page tables */ pmap_bootstrap(&physfree); + /* Setup thread context */ thread0.td_kstack = PTOV(physfree); thread0.td_kstack_pages = KSTACK_PAGES; - kstack0_sz = thread0.td_kstack_pages * PAGE_SIZE; + kstack0_sz = ptoa(thread0.td_kstack_pages); bzero((void *)thread0.td_kstack, kstack0_sz); + thread0.td_pcb = get_pcb_td(&thread0); + physfree += kstack0_sz; - thread0.td_pcb = (struct pcb *)(thread0.td_kstack + kstack0_sz) - 1; + + /* Make sure we are still inside of available mapped va. */ + KASSERT(PTOV(physfree) <= (xenstack + 512 * 1024), + ("Attempt to use unmapped va\n")); + + /* Register the rest of free physical memory with phys_avail[] */ + phys_avail[pa_index++] = physfree; + phys_avail[pa_index++] = ptoa(physmem); /* * This may be done better later if it gets more high level * components in it. If so just link td->td_proc here. */ proc_linkup0(&proc0, &thread0); - KASSERT(si->mod_start == 0, ("MISMATCH")); if (si->mod_start != 0) { /* we have a ramdisk or kernel module */ @@ -367,16 +386,54 @@ initxen(struct start_info *si) HYPERVISOR_set_segment_base (SEGBASE_GS_KERNEL, (uint64_t) pc); HYPERVISOR_set_segment_base (SEGBASE_GS_USER, (uint64_t) 0); + /* per cpu structures for cpu0 */ + pcpu_init(pc, 0, sizeof(struct pcpu)); + PCPU_SET(prvspace, pc); + PCPU_SET(curthread, &thread0); + + /* + * Initialize mutexes. + * + * icu_lock: in order to allow an interrupt to occur in a critical + * section, to set pcpu->ipending (etc...) properly, we + * must be able to get the icu lock, so it can't be + * under witness. + */ + mutex_init(); + mtx_init(&icu_lock, "icu", NULL, MTX_SPIN | MTX_NOWITNESS); + mtx_init(&dt_lock, "descriptor tables", NULL, MTX_DEF); + /* exception handling */ init_exception_table(); - /* page tables */ + identify_cpu(); /* Final stage of CPU initialization */ + + //msgbufinit(msgbufp, msgbufsize); + //fpuinit(); + + /* + * Set up thread0 pcb after fpuinit calculated pcb + fpu save + * area size. Zero out the extended state header in fpu save + * area. + */ + thread0.td_pcb = get_pcb_td(&thread0); + bzero(get_pcb_user_save_td(&thread0), cpu_max_ext_state_size); + + PCPU_SET(rsp0, (vm_offset_t) thread0.td_pcb & ~0xFul /* 16 byte aligned */); + PCPU_SET(curpcb, thread0.td_pcb); + + /* setup user mode selector glue */ + _ucodesel = GSEL(GUCODE_SEL, SEL_UPL); + _udatasel = GSEL(GUDATA_SEL, SEL_UPL); + _ufssel = GSEL(GUFS32_SEL, SEL_UPL); + _ugssel = GSEL(GUGS32_SEL, SEL_UPL); + + gdtset = 1; + /* console */ printk("Hello world!\n"); - while(1); - KASSERT(0, ("TODO")); - return thread0.td_kstack; + return (u_int64_t) thread0.td_pcb; } /* From owner-svn-src-projects@FreeBSD.ORG Thu May 24 12:45:01 2012 Return-Path: Delivered-To: svn-src-projects@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E20641065670; Thu, 24 May 2012 12:45:01 +0000 (UTC) (envelope-from cherry@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id CA7978FC12; Thu, 24 May 2012 12:45:01 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.4/8.14.4) with ESMTP id q4OCj1pH038844; Thu, 24 May 2012 12:45:01 GMT (envelope-from cherry@svn.freebsd.org) Received: (from cherry@localhost) by svn.freebsd.org (8.14.4/8.14.4/Submit) id q4OCj1h6038843; Thu, 24 May 2012 12:45:01 GMT (envelope-from cherry@svn.freebsd.org) Message-Id: <201205241245.q4OCj1h6038843@svn.freebsd.org> From: "Cherry G. Mathew" Date: Thu, 24 May 2012 12:45:01 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org X-SVN-Group: projects MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r235909 - projects/amd64_xen_pv/sys/amd64/xen X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 May 2012 12:45:02 -0000 Author: cherry Date: Thu May 24 12:45:01 2012 New Revision: 235909 URL: http://svn.freebsd.org/changeset/base/235909 Log: The following changes go into this diff: i) Add copyright information from the native file ii) Rename some variables to be more accurately descriptive of their function. iii) Add a boottime implementation of pmap_kenter() which uses mmu_map.h Please note that this is heavily wip and therefore probably very broken. The idea behind this commit is to publish the changes so that the branch can be pulled up to HEAD, and progress noted. The ideal way to get pmap operating would be to use 2MB or greater page sized pages to direct map all the physical/machine pages and use direct map. This is slightly more complicated by the fact that xen disallows write access to pages that are part of the page table hierarchy and therefore the native implementation of dmap will not work directly. Approved by: gibbs (implicit) Modified: projects/amd64_xen_pv/sys/amd64/xen/pmap.c Modified: projects/amd64_xen_pv/sys/amd64/xen/pmap.c ============================================================================== --- projects/amd64_xen_pv/sys/amd64/xen/pmap.c Thu May 24 12:41:57 2012 (r235908) +++ projects/amd64_xen_pv/sys/amd64/xen/pmap.c Thu May 24 12:45:01 2012 (r235909) @@ -1,4 +1,115 @@ -/* XXX: Copyright */ +/*- + * + * Copyright (c) 1991 Regents of the University of California. + * All rights reserved. + * Copyright (c) 1994 John S. Dyson + * All rights reserved. + * Copyright (c) 1994 David Greenman + * All rights reserved. + * Copyright (c) 2005 Alan L. Cox + * All rights reserved. + * Copyright (c) 2012 Spectra Logic Corporation + * All rights reserved. + * + * This code is derived from software contributed to Berkeley by + * the Systems Programming Group of the University of Utah Computer + * Science Department and William Jolitz of UUNET Technologies Inc. + * + * Portions of this software were developed by + * Cherry G. Mathew under sponsorship + * from Spectra Logic Corporation. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. All advertising materials mentioning features or use of this software + * must display the following acknowledgement: + * This product includes software developed by the University of + * California, Berkeley and its contributors. + * 4. Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * from: @(#)pmap.c 7.7 (Berkeley) 5/12/91 + */ +/*- + * Copyright (c) 2003 Networks Associates Technology, Inc. + * All rights reserved. + * + * This software was developed for the FreeBSD Project by Jake Burkholder, + * Safeport Network Services, and Network Associates Laboratories, the + * Security Research Division of Network Associates, Inc. under + * DARPA/SPAWAR contract N66001-01-C-8035 ("CBOSS"), as part of the DARPA + * CHATS research program. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include +__FBSDID("$FreeBSD$"); + +/* + * Manages physical address maps. + * + * In addition to hardware address maps, this + * module is called upon to provide software-use-only + * maps which may or may not be stored in the same + * form as hardware maps. These pseudo-maps are + * used to store intermediate results from copy + * operations to and from address spaces. + * + * Since the information managed by this module is + * also stored by the logical address mapping module, + * this module may throw away valid virtual-to-physical + * mappings at almost any time. However, invalidations + * of virtual-to-physical mappings must be done as + * requested. + * + * In order to cope with hardware architectures which + * make virtual-to-physical map invalidates expensive, + * this module may delay invalidate or reduced protection + * operations until such time as they are actually + * necessary. This module is given full information as + * to which processors are currently using which maps, + * and to when physical maps must be made correct. + */ + #include __FBSDID("$FreeBSD$"); @@ -8,6 +119,9 @@ __FBSDID("$FreeBSD$"); #include #include +#include +#include +#include #ifdef SMP #include @@ -22,6 +136,11 @@ __FBSDID("$FreeBSD$"); #include #include +#include + +extern vm_offset_t pa_index; /* from machdep.c */ +extern unsigned long physfree; /* from machdep.c */ + struct pmap kernel_pmap_store; vm_offset_t virtual_avail; /* VA of first avail page (after kernel bss) */ @@ -54,17 +173,32 @@ static vm_paddr_t boot_ptendphys; /* phy * bootstrap page tables */ +static size_t tsz; /* mmu_map.h opaque cookie size */ +static vm_offset_t (*ptmb_mappedalloc)(size_t) = NULL; +static void (*ptmb_mappedfree)(size_t) = NULL; +static vm_offset_t ptmb_ptov(vm_paddr_t p) +{ + return PTOV(p); +} +static vm_paddr_t ptmb_vtop(vm_offset_t v) +{ + return VTOP(v); +} + extern uint64_t xenstack; /* The stack Xen gives us at boot */ /* return kernel virtual address of 'n' claimed physical pages at boot. */ static vm_offset_t vallocpages(vm_paddr_t *firstaddr, int n) { - u_int64_t ret; - - ret = *firstaddr + KERNBASE; + u_int64_t ret = *firstaddr + KERNBASE; bzero((void *)ret, n * PAGE_SIZE); *firstaddr += n * PAGE_SIZE; + + /* Make sure we are still inside of available mapped va. */ + KASSERT(PTOV(*firstaddr) <= (xenstack + 512 * 1024), + ("Attempt to use unmapped va\n")); + return (ret); } @@ -102,23 +236,14 @@ extern int end; /* End of kernel binary static pt_entry_t pmap_xen_kernel_vaflags(vm_offset_t va) { - /* See: interface/xen.h */ - /* - * i) si->mfn_list - * ii) start info page - * iii) bootstrap stack - * iv) Everything in and above the "scratch area - * Everything else is r/o - */ - if ((va > (vm_offset_t) &etext && /* .data, .bss et. al */ (va < (vm_offset_t) &end)) || - (va > (vm_offset_t)(xen_start_info->pt_base + - xen_start_info->nr_pt_frames * PAGE_SIZE) && - va < xenstack) + ((va > (vm_offset_t)(xen_start_info->pt_base + + xen_start_info->nr_pt_frames * PAGE_SIZE)) && + va < PTOV(boot_ptphys)) || - va >= PTOV(boot_ptendphys + 1)) { + va > PTOV(boot_ptendphys)) { return PG_RW; } @@ -130,13 +255,14 @@ create_boot_pagetables(vm_paddr_t *first { int i; int nkpt, nkpdpe; - int mapspan = (xenstack - KERNBASE + - 512 * 1024 + PAGE_SIZE) / PAGE_SIZE; + int nkmapped = atop(VTOP(xenstack + 512 * 1024 + PAGE_SIZE)); + + kernel_vm_end = PTOV(ptoa(nkmapped - 1)); boot_ptphys = *firstaddr; /* lowest available r/w area */ /* Allocate pseudo-physical pages for kernel page tables. */ - nkpt = howmany(mapspan, NPTEPG); + nkpt = howmany(nkmapped, NPTEPG); nkpdpe = howmany(nkpt, NPDEPG); KPML4phys = vallocpages(firstaddr, 1); KPDPphys = vallocpages(firstaddr, NKPML4E); @@ -148,13 +274,14 @@ create_boot_pagetables(vm_paddr_t *first ndmpdp = (ptoa(Maxmem) + NBPDP - 1) >> PDPSHIFT; if (ndmpdp < 4) /* Minimum 4GB of dirmap */ ndmpdp = 4; - DMPDPphys = vallocpages(firstaddr, NDMPML4E); + DMPDPphys = vallocpages(firstaddr, NDMPML4E, VALLOC_MAPPED); ndm1g = 0; if ((amd_feature & AMDID_PAGE1GB) != 0) ndm1g = ptoa(Maxmem) >> PDPSHIFT; if (ndm1g < ndmpdp) - DMPDphys = vallocpages(firstaddr, ndmpdp - ndm1g); + DMPDphys = vallocpages(firstaddr, ndmpdp - ndm1g, + VALLOC_MAPPED); dmaplimit = (vm_paddr_t)ndmpdp << PDPSHIFT; #endif /* SUPERPAGESUPPORT */ @@ -166,9 +293,7 @@ create_boot_pagetables(vm_paddr_t *first ("bootstrap mapped memory insufficient.\n")); /* Fill in the underlying page table pages */ - /* Read-only from zero to physfree */ - /* XXX not fully used, underneath 2M pages */ - for (i = 0; (i << PAGE_SHIFT) < (mapspan * PAGE_SIZE); i++) { + for (i = 0; ptoa(i) < ptoa(nkmapped); i++) { ((pt_entry_t *)KPTphys)[i] = phystomach(i << PAGE_SHIFT); ((pt_entry_t *)KPTphys)[i] |= PG_V | PG_G | PG_U; ((pt_entry_t *)KPTphys)[i] |= @@ -321,10 +446,35 @@ pmap_xen_bootpages(vm_paddr_t *firstaddr HYPERVISOR_shared_info = (void *) va; } +/* alloc from linear mapped boot time virtual address space */ +static vm_offset_t +mmu_alloc(size_t size) +{ + KASSERT(size != 0, ("mmu_alloc size must not be zero\n")); + KASSERT(physfree != 0, + ("physfree must have been set before using mmu_alloc")); + + size = round_page(size); /* We can allocate only in page sizes */ + + vm_offset_t va = vallocpages(&physfree, atop(size)); + + /* + * Xen requires the page table hierarchy to be R/O. + */ + + pmap_xen_setpages_ro(va, atop(size)); + + return va; +} + void pmap_bootstrap(vm_paddr_t *firstaddr) { + /* setup mmu_map backend function pointers for boot */ + ptmb_mappedalloc = mmu_alloc; + ptmb_mappedfree = NULL; + create_boot_pagetables(firstaddr); /* Switch to the new kernel tables */ @@ -335,17 +485,65 @@ pmap_bootstrap(vm_paddr_t *firstaddr) pmap_xen_setpages_rw(xen_start_info->pt_base, xen_start_info->nr_pt_frames); - /* Map in Xen related pages into VA space */ - pmap_xen_bootpages(firstaddr); - /* * gc newly free pages (bootstrap PTs and bootstrap stack, * mostly, I think.). + * Record the pages as available to the VM via phys_avail[] */ - virtual_avail = (vm_offset_t) KERNBASE + *firstaddr; - virtual_end = VM_MAX_KERNEL_ADDRESS; /* XXX: Check we don't - overlap xen pgdir entries. */ + /* This is the first free phys segment. see: xen.h */ + KASSERT(pa_index == 0, + ("reclaimed page table pages are not the lowest available!")); + + phys_avail[pa_index] = VTOP(xen_start_info->pt_base); + phys_avail[pa_index + 1] = phys_avail[pa_index] + + ptoa(xen_start_info->nr_pt_frames - 1); + pa_index += 2; + + /* Map in Xen related pages into VA space */ + pmap_xen_bootpages(firstaddr); + + /* + * Xen guarantees mapped virtual addresses at boot time upto + * xenstack + 512KB. We want to use these for vallocpages() + * and therefore don't want to touch these mappings since + * they're scarce resources. Move along to the end of + * guaranteed mapping. + * + * Note: Xen *may* provide mappings upto xenstack + 2MB, but + * this is not guaranteed. We therefore assum that only 512KB + * is available. + */ + + virtual_avail = (vm_offset_t) xenstack + 512 * 1024; + /* XXX: Check we don't overlap xen pgdir entries. */ + virtual_end = VM_MAX_KERNEL_ADDRESS; + + virtual_avail = xenstack + 1052 * 1024; + + /* + * Initialize the kernel pmap (which is statically allocated). + */ + PMAP_LOCK_INIT(kernel_pmap); + kernel_pmap->pm_pml4 = (pdp_entry_t *)KPML4phys; + kernel_pmap->pm_root = NULL; + CPU_FILL(&kernel_pmap->pm_active); /* don't allow deactivation */ + TAILQ_INIT(&kernel_pmap->pm_pvchunk); + + tsz = mmu_map_t_size(); + +#if 0 + /* XXX test */ + vm_offset_t va = virtual_avail + 4096 * 1024; + + vm_paddr_t pa = phys_avail[pa_index - 1]; + + pmap_kenter(va, pa); + + memset((void *)va, 0, PAGE_SIZE); + while(1); + /* test XXX */ +#endif } void @@ -476,23 +674,107 @@ pmap_extract_and_hold(pmap_t pmap, vm_of return 0; } +/*************************************************** + * Low level mapping routines..... + ***************************************************/ + +/* + * Add a wired page to the kva. + * Note: not SMP coherent. + * + * This function may be used before pmap_bootstrap() is called. + */ + void pmap_kenter(vm_offset_t va, vm_paddr_t pa) { - KASSERT(0, ("XXX: TODO\n")); + + char tbuf[tsz]; /* Safe to do this on the stack since tsz is + * effectively const. + */ + + mmu_map_t tptr = tbuf; + + struct mmu_map_mbackend mb = { + ptmb_mappedalloc, + ptmb_mappedfree, + ptmb_ptov, + ptmb_vtop + }; + + mmu_map_t_init(tptr, &mb); + + if (!mmu_map_inspect_va(kernel_pmap, tptr, va)) { + mmu_map_hold_va(kernel_pmap, tptr, va); /* PT hierarchy */ + xen_flush_queue(); + } + + /* Backing page tables are in place, let xen do the maths */ + + PT_SET_MA(va, xpmap_ptom(pa) | PG_RW | PG_V | PG_U); + PT_UPDATES_FLUSH(); + + mmu_map_release_va(kernel_pmap, tptr, va); + mmu_map_t_fini(tptr); + } +/* + * Remove a page from the kernel pagetables. + * Note: not SMP coherent. + * + * This function may be used before pmap_bootstrap() is called. + */ + __inline void pmap_kremove(vm_offset_t va) { - KASSERT(0, ("XXX: TODO\n")); + pt_entry_t *pte; + + pte = vtopte(va); + PT_CLEAR_VA(pte, FALSE); } +/* + * Used to map a range of physical addresses into kernel + * virtual address space. + * + * The value passed in '*virt' is a suggested virtual address for + * the mapping. Architectures which can support a direct-mapped + * physical to virtual region can return the appropriate address + * within that region, leaving '*virt' unchanged. Other + * architectures should map the pages starting at '*virt' and + * update '*virt' with the first usable address after the mapped + * region. + */ + vm_offset_t pmap_map(vm_offset_t *virt, vm_paddr_t start, vm_paddr_t end, int prot) { - KASSERT(0, ("XXX: TODO\n")); - return -1; + vm_offset_t va, sva; + + va = sva = *virt; + CTR4(KTR_PMAP, "pmap_map: va=0x%x start=0x%jx end=0x%jx prot=0x%x", + va, start, end, prot); + + while (start < end) { +#if 0 + if (PTOV(start) < xenstack + 512 * 1024) { /* XXX: + remove me */ + continue; + } +#endif + + pmap_kenter(va, start); + va += PAGE_SIZE; + start += PAGE_SIZE; + + while(1); + } + + // XXX: pmap_invalidate_range(kernel_pmap, sva, va); + *virt = va; + return (sva); } void From owner-svn-src-projects@FreeBSD.ORG Thu May 24 17:02:22 2012 Return-Path: Delivered-To: svn-src-projects@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ED7BB1065672; Thu, 24 May 2012 17:02:22 +0000 (UTC) (envelope-from cherry@zyx.in) Received: from xsmtp10.mail2web.com (xsmtp30.mail2web.com [168.144.250.249]) by mx1.freebsd.org (Postfix) with ESMTP id A34138FC0A; Thu, 24 May 2012 17:02:22 +0000 (UTC) Received: from [10.5.41.3] (helo=M2W103.mail2web.com) by xsmtp10.mail2web.com with smtp (Exim 4.63) (envelope-from ) id 1SXbQM-0007h5-SJ; Thu, 24 May 2012 13:02:21 -0400 Message-ID: <380-2201254241730954@M2W103.mail2web.com> X-Priority: 3 X-Originating-IP: 81-102-250-43 X-URL: http://mail2web.com/ From: "cherry@zyx.in" To: cherry@freebsd.org, src-committers@freebsd.org, svn-src-projects@freebsd.org Date: Thu, 24 May 2012 13:03:00 -0400 MIME-Version: 1.0 Content-type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Subject: mmu_map.h review/justification - was => RE: svn commit: r235904 - in projects/amd64_xen_pv/sys: amd64/xen conf X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: cherry@zyx.in List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 May 2012 17:02:23 -0000 Hi, Apologies for the top-post - I'm having mail client difficulties at the moment=2E I have recieved private feedback about the contents of the commit log message, so I thought I would address them here, and keep it in mind for future commits=2E The main question is "rationale/justification" for the a= pi in the diff below, and I will try to address that here: Facts: - The x86 MMU hardware works with a hierarchical page table=2E Pages in th= e hierarchy are referred to each other by physical addresses=2E - The processor itself however accesses all memory via virtual addresses=2E= What this means is that in order for the kernel to manipulate page tables entries, it needs to have the pages in the hierarchy themselves to be mapped into its virtual address space=2E However, this requires that the p= age tables be setup already to facilitate this - clearly a bootstrap problem=2E= In order to address this problem, I've attempted to come up with an API that is easy to use, who's objective is to do "whatever it takes" to the MMU, to make the pages in the physical hierarchy visible to the kernel virtual address space=2E The use of the api is very intuitive=2E One basically says to it: "Do whatever it takes to make the mapping va->pa viable"=2E On x86, this basically means setting up the page table hierarchy (if this has not already been done) and providing a means for the caller to inspect the contents of the backing pages=2E On other architectures, with software TLB= s, for eg:, I would envisage the implementation returning a direct mapped segment offset to the page, so that the caller can then insert this into the soft TLB=2E Admittedly I haven't given too much thought to it beyond t= hat - I've only attempted to not restrict the API design to the x86 paging architecture alone, as far as is possible=2E In order for the implementation to do "whatever it takes", the caller is provided with a set of callbacks that allocate resources if required=2E On= x86, the callbacks need to make sure that the returned memory which will b= e used for backing page tables is already mapped into the kernel address space=2E On xen, we have the additional requirement that page tables are n= ot mapped in into the KVA space as writeable pages=2E On the upside, xen provides us with about 512kb of already mapped in memory at boot - so we just lop off chunks of it via vallocpages() (see pmap=2Ec)=2E That's it, really=2E It's a fairly intuitive api once the background is in place=2E O= n the down side, it's bound to be ridiculously unoptimal, and I imagine there ar= e other subsystems already in the kernel which provide this functionality - albeit unavailable at boot ( the reason I wrote this api )=2E As I get alo= ng with the port, I will make a decision ( open to feedback here ) about the future of the api within the amd64/xen subdirectory=2E I don't expect this api to be used outside of the x86 architecture, or outside of the amd64/xen port at this point, but I've written it with a view to it being useful, if possible=2E I hope that was a useful explanation=2E Many Thanks, Cherry=2E Original Message: ----------------- From: Cherry G=2E Mathew cherry@FreeBSD=2Eorg Date: Thu, 24 May 2012 12:02:11 +0000 (UTC) To: src-committers@freebsd=2Eorg, svn-src-projects@freebsd=2Eorg Subject: svn commit: r235904 - in projects/amd64=5Fxen=5Fpv/sys: amd64/xen= conf Author: cherry Date: Thu May 24 12:02:10 2012 New Revision: 235904 URL: http://svn=2Efreebsd=2Eorg/changeset/base/235904 Log: This API is an attempt to abstract the MMU state in an MI fashion=2E It = is heavily wip and may or may not go away, from amd64/ depending on how thing= s go with the direct mapped implementation Added: projects/amd64=5Fxen=5Fpv/sys/amd64/xen/mmu=5Fmap=2Ec projects/amd64=5Fxen=5Fpv/sys/amd64/xen/mmu=5Fmap=2Eh Modified: projects/amd64=5Fxen=5Fpv/sys/conf/files=2Eamd64 Added: projects/amd64=5Fxen=5Fpv/sys/amd64/xen/mmu=5Fmap=2Ec =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D =3D=3D --- /dev/null=0900:00:00 1970=09(empty, because file is newly added) +++ projects/amd64=5Fxen=5Fpv/sys/amd64/xen/mmu=5Fmap=2Ec=09Thu May 24 12:= 02:10 2012=09(r235904) @@ -0,0 +1,389 @@ +/* $FreeBSD$ */ +/*- + * Copyright (c) 2011-2012 Spectra Logic Corporation + * All rights reserved=2E + * + * This software was developed by Cherry G=2E Mathew + * under sponsorship from Spectra Logic Corporation=2E + *=20 + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1=2E Redistributions of source code must retain the above copyright + * notice, this list of conditions, and the following disclaimer, + * without modification=2E + * 2=2E Redistributions in binary form must reproduce at minimum a discla= imer + * substantially similar to the "NO WARRANTY" disclaimer below + * ("Disclaimer") and any redistribution must be conditioned upon + * including a substantially similar Disclaimer requirement for furthe= r + * binary redistribution=2E + * + * NO WARRANTY + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED=2E IN NO EVENT SHALL THE COPYRIGHT= + * HOLDERS OR CONTRIBUTORS BE LIABLE FOR SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOOD= S + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, + * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING + * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE + * POSSIBILITY OF SUCH DAMAGES=2E + */ + + +/* + * This file implements the API that manages the page table + * hierarchy for the amd64 Xen pmap=2E + */ +#include +=5F=5FFBSDID("$FreeBSD$"); + +#include "opt=5Fcpu=2Eh" +#include "opt=5Fpmap=2Eh" +#include "opt=5Fsmp=2Eh" + + +#include +#include +#include + +#include +#include +#include + +#include +#include + +#include + +static int +pml4t=5Findex(vm=5Foffset=5Ft va) +{ +=09/* amd64 sign extends 48th bit and upwards */ +=09const uint64=5Ft SIGNMASK =3D (1UL << 48) - 1; +=09va &=3D SIGNMASK; /* Remove sign extension */ + +=09return (va >> PML4SHIFT);=20 +} + +static int +pdpt=5Findex(vm=5Foffset=5Ft va) +{ +=09/* amd64 sign extends 48th bit and upwards */ +=09const uint64=5Ft SIGNMASK =3D (1UL << 48) - 1; +=09va &=3D SIGNMASK; /* Remove sign extension */ + +=09return ((va & PML4MASK) >> PDPSHIFT); +} + +static int +pdt=5Findex(vm=5Foffset=5Ft va) +{ +=09/* amd64 sign extends 48th bit and upwards */ +=09const uint64=5Ft SIGNMASK =3D (1UL << 48) - 1; +=09va &=3D SIGNMASK; /* Remove sign extension */ + +=09return ((va & PDPMASK) >> PDRSHIFT); +} + +/*=20 + * The table get functions below assume that a table cannot exist at + * address 0 + */ +static pml4=5Fentry=5Ft * +pmap=5Fget=5Fpml4t(struct pmap *pm) +{ +=09KASSERT(pm !=3D NULL, +=09=09("NULL pmap passed in!\n")); + +=09pml4=5Fentry=5Ft *pm=5Fpml4 =3D pm->pm=5Fpml4; +=09 +=09KASSERT(pm=5Fpml4 !=3D NULL, +=09=09("pmap has NULL pml4!\n")); + +=09return pm->pm=5Fpml4; +} + +/* Returns physical address */ +static vm=5Fpaddr=5Ft +pmap=5Fget=5Fpdpt(vm=5Foffset=5Ft va, pml4=5Fentry=5Ft *pml4t) +{ +=09pml4=5Fentry=5Ft pml4e; + +=09KASSERT(va <=3D VM=5FMAX=5FKERNEL=5FADDRESS, +=09=09("invalid address requested")); +=09KASSERT(pml4t !=3D 0, ("pml4t cannot be zero")); + +=09pml4e =3D pml4t[pml4t=5Findex(va)]; + +=09if (!(pml4e & PG=5FV)) { +=09=09return 0; +=09} + +=09return xpmap=5Fmtop(pml4e & PG=5FFRAME); +} + +/* Returns physical address */ +static vm=5Fpaddr=5Ft +pmap=5Fget=5Fpdt(vm=5Foffset=5Ft va, pdp=5Fentry=5Ft *pdpt) +{ +=09pdp=5Fentry=5Ft pdpe; + +=09KASSERT(va <=3D VM=5FMAX=5FKERNEL=5FADDRESS, +=09=09("invalid address requested")); +=09KASSERT(pdpt !=3D 0, ("pdpt cannot be zero")); + +=09pdpe =3D pdpt[pdpt=5Findex(va)]; + +=09if (!(pdpe & PG=5FV)) { +=09=09return 0; +=09} + +=09return xpmap=5Fmtop(pdpe & PG=5FFRAME); +} + +/* Returns physical address */ +static vm=5Fpaddr=5Ft +pmap=5Fget=5Fpt(vm=5Foffset=5Ft va, pd=5Fentry=5Ft *pdt) +{ +=09pd=5Fentry=5Ft pdte; + +=09KASSERT(va <=3D VM=5FMAX=5FKERNEL=5FADDRESS, +=09=09("invalid address requested")); + +=09KASSERT(pdt !=3D 0, ("pdt cannot be zero")); + +=09pdte =3D pdt[pdt=5Findex(va)]; + +=09if (!(pdte & PG=5FV)) { +=09=09return 0; +=09} + +=09return xpmap=5Fmtop(pdte & PG=5FFRAME); +} + +/*=20 + * This structure defines the 4 indices that a given virtual + * address lookup would traverse=2E + * + * Note: this structure is opaque to API customers=2E Callers give us an + * untyped array which is marshalled/unmarshalled inside of the + * stateful api=2E + */ + +static const uint64=5Ft SANE =3D 0xcafebabe; + +struct mmu=5Fmap=5Findex { +=09pml4=5Fentry=5Ft *pml4t; /* Page Map Level 4 Table */ +=09pdp=5Fentry=5Ft *pdpt; /* Page Directory Pointer Table */ +=09pd=5Fentry=5Ft *pdt; /* Page Directory Table */ +=09pt=5Fentry=5Ft *pt; /* Page Table */ + +=09struct mmu=5Fmap=5Fmbackend ptmb; /* Backend info */ + +=09uint32=5Ft sanity; /* 32 bit (for alignment) magic XXX: +=09=09=09 * Make optional on DEBUG */ +}; + +size=5Ft +mmu=5Fmap=5Ft=5Fsize(void) +{ +=09return sizeof (struct mmu=5Fmap=5Findex); +} + +void +mmu=5Fmap=5Ft=5Finit(void *addr, struct mmu=5Fmap=5Fmbackend *mb) +{ +=09KASSERT((addr !=3D NULL) && (mb !=3D NULL), ("NULL args given!")); +=09struct mmu=5Fmap=5Findex *pti =3D addr; +=09KASSERT(pti->sanity !=3D SANE, ("index initialised twice!")); +=09KASSERT(mb->alloc !=3D NULL && +=09=09mb->ptov !=3D NULL && +=09=09mb->vtop !=3D NULL,=20 +=09=09("initialising with pre-registered alloc routine active")); + +=09pti->ptmb =3D *mb; + +=09/* Backend page allocation should provide default VA mapping */ +=09pti->sanity =3D SANE; +} + +void mmu=5Fmap=5Ft=5Ffini(void *addr) +{ +=09KASSERT(addr !=3D NULL, ("NULL args given!")); + +=09struct mmu=5Fmap=5Findex *pti =3D addr; +=09KASSERT(pti->sanity =3D=3D SANE, ("Uninitialised index cookie used"));= +=09struct mmu=5Fmap=5Fmbackend *mb =3D &pti->ptmb; + +=09pti->sanity =3D 0; + +=09if (mb->free !=3D NULL) { +=09=09/* XXX: go through PT hierarchy and free + unmap +=09=09 * unused tables */=20 +=09} +} + +pd=5Fentry=5Ft * +mmu=5Fmap=5Fpml4t(void *addr) +{ +=09KASSERT(addr !=3D NULL, ("NULL args given!")); +=09struct mmu=5Fmap=5Findex *pti =3D addr; + +=09KASSERT(pti->sanity =3D=3D SANE, ("Uninitialised index cookie used"));= + +=09return pti->pml4t; +} + +pd=5Fentry=5Ft * +mmu=5Fmap=5Fpdpt(void *addr) +{ +=09KASSERT(addr !=3D NULL, ("NULL args given!")); +=09struct mmu=5Fmap=5Findex *pti =3D addr; + +=09KASSERT(pti->sanity =3D=3D SANE, ("Uninitialised index cookie used"));= + +=09return pti->pdpt; +} + +pd=5Fentry=5Ft * +mmu=5Fmap=5Fpdt(void *addr) +{ +=09KASSERT(addr !=3D NULL, ("NULL args given!")); +=09struct mmu=5Fmap=5Findex *pti =3D addr; + +=09KASSERT(pti->sanity =3D=3D SANE, ("Uninitialised index cookie used"));= + +=09return pti->pdt; +} + +pd=5Fentry=5Ft * +mmu=5Fmap=5Fpt(void *addr) +{ +=09KASSERT(addr !=3D NULL, ("NULL args given!")); +=09struct mmu=5Fmap=5Findex *pti =3D addr; + +=09KASSERT(pti->sanity =3D=3D SANE, ("Uninitialised index cookie used"));= + +=09return pti->pt; +} + +bool +mmu=5Fmap=5Finspect=5Fva(struct pmap *pm, void *addr, vm=5Foffset=5Ft va)= +{ +=09KASSERT(addr !=3D NULL && pm !=3D NULL, ("NULL arg(s) given")); + +=09struct mmu=5Fmap=5Findex *pti =3D addr; +=09KASSERT(pti->sanity =3D=3D SANE, ("Uninitialised index cookie used"));= + +=09vm=5Fpaddr=5Ft pt; + +=09pti->pml4t =3D pmap=5Fget=5Fpml4t(pm); + +=09pt =3D pmap=5Fget=5Fpdpt(va, pti->pml4t); + +=09if (pt =3D=3D 0) { +=09=09return false; +=09} else { +=09=09pti->pdpt =3D (pdp=5Fentry=5Ft *) pti->ptmb=2Eptov(pt); +=09} + +=09pt =3D pmap=5Fget=5Fpdt(va, pti->pdpt); + +=09if (pt =3D=3D 0) { +=09=09return false; +=09} else { +=09=09pti->pdt =3D (pd=5Fentry=5Ft *) pti->ptmb=2Eptov(pt); +=09} + +=09pt =3D pmap=5Fget=5Fpt(va, pti->pdt); + +=09if (pt =3D=3D 0) { +=09=09return false; +=09} else { +=09=09pti->pt =3D (pt=5Fentry=5Ft *)pti->ptmb=2Eptov(pt); +=09} + +=09return true; +} +extern uint64=5Ft xenstack; /* The stack Xen gives us at boot */ +void +mmu=5Fmap=5Fhold=5Fva(struct pmap *pm, void *addr, vm=5Foffset=5Ft va) +{ +=09KASSERT(addr !=3D NULL && pm !=3D NULL, ("NULL arg(s) given")); + +=09struct mmu=5Fmap=5Findex *pti =3D addr; +=09KASSERT(pti->sanity =3D=3D SANE, ("Uninitialised index cookie used"));= + +=09vm=5Fpaddr=5Ft pt; + +=09pti->pml4t =3D pmap=5Fget=5Fpml4t(pm); + +=09pt =3D pmap=5Fget=5Fpdpt(va, pti->pml4t); + +=09if (pt =3D=3D 0) { +=09=09pml4=5Fentry=5Ft *pml4tep; +=09=09vm=5Fpaddr=5Ft pml4tep=5Fma; +=09=09pml4=5Fentry=5Ft pml4te; + +=09=09pti->pdpt =3D (pdp=5Fentry=5Ft *)pti->ptmb=2Ealloc(PAGE=5FSIZE); + +=09=09pml4tep =3D &pti->pml4t[pml4t=5Findex(va)]; +=09=09pml4tep=5Fma =3D xpmap=5Fptom(pti->ptmb=2Evtop((vm=5Foffset=5Ft)pml= 4tep)); +=09=09pml4te =3D xpmap=5Fptom(pti->ptmb=2Evtop((vm=5Foffset=5Ft)pti->pdpt= )) | PG=5FRW | PG=5FV | PG=5FU; /* XXX: revisit flags */ +=09=09xen=5Fqueue=5Fpt=5Fupdate(pml4tep=5Fma, pml4te); + +=09} else { +=09=09pti->pdpt =3D (pdp=5Fentry=5Ft *) pti->ptmb=2Eptov(pt); +=09} + +=09pt =3D pmap=5Fget=5Fpdt(va, pti->pdpt); + +=09if (pt =3D=3D 0) { +=09=09pdp=5Fentry=5Ft *pdptep; +=09=09vm=5Fpaddr=5Ft pdptep=5Fma; +=09=09pdp=5Fentry=5Ft pdpte; + +=09=09pti->pdt =3D (pd=5Fentry=5Ft *)pti->ptmb=2Ealloc(PAGE=5FSIZE); + +=09=09pdptep =3D &pti->pdpt[pdpt=5Findex(va)]; +=09=09pdptep=5Fma =3D xpmap=5Fptom(pti->ptmb=2Evtop((vm=5Foffset=5Ft)pdpt= ep)); +=09=09pdpte =3D xpmap=5Fptom(pti->ptmb=2Evtop((vm=5Foffset=5Ft)pti->pdt))= | PG=5FRW | PG=5FV | PG=5FU; /*=09XXX: revisit flags */ +=09=09xen=5Fqueue=5Fpt=5Fupdate(pdptep=5Fma, pdpte); +=09=09 +=09} else { +=09=09pti->pdt =3D (pd=5Fentry=5Ft *) pti->ptmb=2Eptov(pt); +=09} + +=09pt =3D pmap=5Fget=5Fpt(va, pti->pdt); + +=09if (pt =3D=3D 0) { +=09=09pd=5Fentry=5Ft *pdtep; +=09=09vm=5Fpaddr=5Ft pdtep=5Fma; +=09=09pd=5Fentry=5Ft pdte; + +=09=09pti->pt =3D (pt=5Fentry=5Ft *) pti->ptmb=2Ealloc(PAGE=5FSIZE); + +=09=09pdtep =3D &pti->pdt[pdt=5Findex(va)]; +=09=09pdtep=5Fma =3D xpmap=5Fptom(pti->ptmb=2Evtop((vm=5Foffset=5Ft)pdtep= )); +=09=09pdte =3D xpmap=5Fptom(pti->ptmb=2Evtop((vm=5Foffset=5Ft)pti->pt)) |= PG=5FRW | PG=5FV | PG=5FU; /*=09XXX: revisit flags */ +=09=09xen=5Fqueue=5Fpt=5Fupdate(pdtep=5Fma, pdte); + +=09} else { +=09=09pti->pt =3D (pt=5Fentry=5Ft *) pti->ptmb=2Eptov(pt); +=09} +} + +void +mmu=5Fmap=5Frelease=5Fva(struct pmap *pm, void *addr, vm=5Foffset=5Ft va)= +{ + +=09KASSERT(addr !=3D NULL && pm !=3D NULL, ("NULL arg(s) given")); + +=09struct mmu=5Fmap=5Findex *pti =3D addr; +=09KASSERT(pti->sanity =3D=3D SANE, ("Uninitialised index cookie used"));= + +=09/* XXX: */ +} Added: projects/amd64=5Fxen=5Fpv/sys/amd64/xen/mmu=5Fmap=2Eh =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D =3D=3D --- /dev/null=0900:00:00 1970=09(empty, because file is newly added) +++ projects/amd64=5Fxen=5Fpv/sys/amd64/xen/mmu=5Fmap=2Eh=09Thu May 24 12:= 02:10 2012=09(r235904) @@ -0,0 +1,152 @@ +/* $FreeBSD$ */ +/*- + * Copyright (c) 2011-2012 Spectra Logic Corporation + * All rights reserved=2E + * + * This software was developed by Cherry G=2E Mathew + * under sponsorship from Spectra Logic Corporation=2E + *=20 + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1=2E Redistributions of source code must retain the above copyright + * notice, this list of conditions, and the following disclaimer, + * without modification=2E + * 2=2E Redistributions in binary form must reproduce at minimum a discla= imer + * substantially similar to the "NO WARRANTY" disclaimer below + * ("Disclaimer") and any redistribution must be conditioned upon + * including a substantially similar Disclaimer requirement for furthe= r + * binary redistribution=2E + * + * NO WARRANTY + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED=2E IN NO EVENT SHALL THE COPYRIGHT= + * HOLDERS OR CONTRIBUTORS BE LIABLE FOR SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOOD= S + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, + * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING + * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE + * POSSIBILITY OF SUCH DAMAGES=2E + */ + +#ifndef =5FXEN=5FMMU=5FMAP=5FH=5F +#define =5FXEN=5FMMU=5FMAP=5FH=5F + +#include + +#include + +/*=20 + * + * This API abstracts, in an MI fashion, the paging mechanism of an + * arbitrary CPU architecture as an opaque FSM, which may then be + * subject to inspection in MD ways=2E + * + * Use of this API can have the following effects on the VM system and + * the kernel address space: + * + * - physical memory pages may be allocated=2E + * - physical memory pages may be de-allocated=2E + * - kernel virtual address space may be allocated=2E + * - kernel virtual address space may be de-allocated=2E + * - The page table hierarchy may be modified=2E + * - TLB entries may be invalidated=2E + * + * The API is stateful, and designed around the following principles: + * - Simplicity + * - Object orientation + * - Code reuse=2E + */ + +/*=20 + * We hide the page table structure behind an opaque "index" cookie + * which acts as the "key" to a given va->pa mapping being inspected=2E + */ +typedef void * mmu=5Fmap=5Ft; + +/* + * Memory backend types: + *=20 + * We provide a means to allocate ad-hoc memory/physical page + * requirements to the paging mechanism by means of a "backend" + * alloc function + * + * The memory backend is required to provide physical pages that are + * at least temporarily mapped into the kernel VA space and whose + * contents are thus accessible by a simple pointer indirection from + * within the kernel=2E This requirement may be revoked after conclusion + * of an instance of stateful usage of the API ( See: + * mmu=5Fmap=5Ft=5Ffini() below ), at which point the backend + * implementation is free to unmap any temporary mappings if so + * desired=2E (XXX: review this for non-x86) + * + * Note: Only the mappings may be revoked - any physical pages + * themselves allocated by the backend are considered allocated and + * part of the paging mechanism=2E + */ + +struct mmu=5Fmap=5Fmbackend { /* Callbacks */ + +=09vm=5Foffset=5Ft (*alloc)(size=5Ft); +=09void (*free)(vm=5Foffset=5Ft); /* May be NULL */ + +=09/*=20 +=09 * vtop()/ptov() conversion functions: +=09 * These callbacks typically provide conversions for mapped +=09 * pages allocated via the alloc()/free() callbacks (above)=2E +=09 * The API implementation is free to cache the mappings across +=09 * multiple instances of use; ie; mappings may persist across=20 +=09 * one pair of mmu=5Fmap=5Ft=5Finit()/=2E=2E=5Ffinit() calls=2E +=09 */ +=09vm=5Foffset=5Ft (*ptov)(vm=5Fpaddr=5Ft); +=09vm=5Fpaddr=5Ft (*vtop)(vm=5Foffset=5Ft); +}; + +/*=20 + * Return sizeof (mmu=5Fmap=5Ft) as implemented within the api + * This may then be used to allocate untyped memory for the cookie + * which can then be operated on opaquely behind the API in a machine + * specific manner=2E + */ +size=5Ft mmu=5Fmap=5Ft=5Fsize(void); + +/* + * Initialise the API state to use a specified memory backend=20 + */ +void mmu=5Fmap=5Ft=5Finit(mmu=5Fmap=5Ft, struct mmu=5Fmap=5Fmbackend *); + +/* Conclude this instance of use of the API */ +void mmu=5Fmap=5Ft=5Ffini(mmu=5Fmap=5Ft); + +/* Set "index" cookie state based on va lookup=2E This state may then be + * inspected in MD ways ( See below )=2E Note that every call to the + * following functions can change the state of the backing paging + * mechanism FSM=2E + */ +bool mmu=5Fmap=5Finspect=5Fva(struct pmap *, mmu=5Fmap=5Ft, vm=5Foffset=5F= t); +/*=20 + * Unconditionally allocate resources to setup and "inspect" (as + * above) a given va->pa mapping=20 + */ +void mmu=5Fmap=5Fhold=5Fva(struct pmap *, mmu=5Fmap=5Ft, vm=5Foffset=5Ft= ); + +/* Optionally release resources after tear down of a va->pa mapping */ +void mmu=5Fmap=5Frelease=5Fva(struct pmap *, mmu=5Fmap=5Ft, vm=5Foffset=5F= t); + +/*=20 + * Machine dependant "view" into the page table hierarchy FSM=2E + * On amd64, there are four tables that are consulted for a va->pa + * translation=2E This information may be extracted by the MD functions + * below and is only considered valid between a successful call to + * mmu=5Fmap=5Finspect=5Fva() or mmu=5Fmap=5Fhold=5Fva() and a subsequent= + * call to mmu=5Fmap=5Frelease=5Fva() + */ +pd=5Fentry=5Ft * mmu=5Fmap=5Fpml4t(mmu=5Fmap=5Ft); /* Page Map Level 4 Ta= ble */ +pd=5Fentry=5Ft * mmu=5Fmap=5Fpdpt(mmu=5Fmap=5Ft); /* Page Directory Poin= ter Table */ +pd=5Fentry=5Ft * mmu=5Fmap=5Fpdt(mmu=5Fmap=5Ft); /* Page Directory Tabl= e */ +pd=5Fentry=5Ft * mmu=5Fmap=5Fpt(mmu=5Fmap=5Ft); /* Page Table */ + +#endif /* !=5FXEN=5FMMU=5FMAP=5FH=5F */ Modified: projects/amd64=5Fxen=5Fpv/sys/conf/files=2Eamd64 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D =3D=3D --- projects/amd64=5Fxen=5Fpv/sys/conf/files=2Eamd64=09Thu May 24 11:52:57= 2012=09(r235903) +++ projects/amd64=5Fxen=5Fpv/sys/conf/files=2Eamd64=09Thu May 24 12:02:10= 2012=09(r235904) @@ -128,6 +128,7 @@ amd64/amd64/mpboot=2ES=09=09optional=09native sm amd64/xen/mpboot=2Ec=09=09optional=09xen smp amd64/amd64/pmap=2Ec=09=09optional=09native amd64/xen/pmap=2Ec=09=09optional=09xen +amd64/xen/mmu=5Fmap=2Ec=09=09optional=09xen amd64/amd64/prof=5Fmachdep=2Ec=09optional=09profiling-routine amd64/amd64/ptrace=5Fmachdep=2Ec=09standard amd64/amd64/sigtramp=2ES=09=09standard -------------------------------------------------------------------- mail2web LIVE =96 Free email based on Microsoft=AE Exchange technology - http://link=2Email2web=2Ecom/LIVE From owner-svn-src-projects@FreeBSD.ORG Thu May 24 21:55:37 2012 Return-Path: Delivered-To: svn-src-projects@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5143C106564A; Thu, 24 May 2012 21:55:37 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 3C1F68FC0A; Thu, 24 May 2012 21:55:37 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.4/8.14.4) with ESMTP id q4OLtbkm062569; Thu, 24 May 2012 21:55:37 GMT (envelope-from linimon@svn.freebsd.org) Received: (from linimon@localhost) by svn.freebsd.org (8.14.4/8.14.4/Submit) id q4OLtbnq062565; Thu, 24 May 2012 21:55:37 GMT (envelope-from linimon@svn.freebsd.org) Message-Id: <201205242155.q4OLtbnq062565@svn.freebsd.org> From: Mark Linimon Date: Thu, 24 May 2012 21:55:37 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org X-SVN-Group: projects MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r235940 - projects/portbuild/scripts X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 May 2012 21:55:37 -0000 Author: linimon (doc,ports committer) Date: Thu May 24 21:55:36 2012 New Revision: 235940 URL: http://svn.freebsd.org/changeset/base/235940 Log: - Allow for per-build src.conf as ${builddir}/src.conf.server. - allow 'latest' to be assumed as an argument to makeworld and mkbindist. - standardize on 'builddir' instead of 'here'. - some formatting cleanup. Modified: projects/portbuild/scripts/buildenv projects/portbuild/scripts/makeworld projects/portbuild/scripts/mkbindist Modified: projects/portbuild/scripts/buildenv ============================================================================== --- projects/portbuild/scripts/buildenv Thu May 24 21:44:46 2012 (r235939) +++ projects/portbuild/scripts/buildenv Thu May 24 21:55:36 2012 (r235940) @@ -135,6 +135,7 @@ buildenv () { # set make.conf. Consumed by both 'make index' and 'makeworld'. # First, must explicitly _unset_ pointyhat's native one. export __MAKE_CONF=/dev/null + # allow for per-build make.conf if [ -f ${builddir}/make.conf.server ]; then export __MAKE_CONF=${builddir}/make.conf.server @@ -145,6 +146,13 @@ buildenv () { # export __MAKE_CONF=${pbd}/${arch}/make.conf # fi fi + + # allow for per-build src.conf + if [ -f ${builddir}/src.conf.server ]; then + export SRCCONF=${builddir}/src.conf.server + else + export SRCCONF=/dev/null + fi } # Modified: projects/portbuild/scripts/makeworld ============================================================================== --- projects/portbuild/scripts/makeworld Thu May 24 21:44:46 2012 (r235939) +++ projects/portbuild/scripts/makeworld Thu May 24 21:55:36 2012 (r235940) @@ -4,26 +4,43 @@ # XXX lockfile and interlock with mkbindist to avoid overlapping # builds -if [ $# -lt 3 ]; then - echo "usage: makeworld arch branch buildid [args]" - exit 1 +if [ $# -lt 2 ]; then + echo "usage: makeworld [] [args]" + exit 1 fi arch=$1 branch=$2 -buildid=$3 -shift 3 +shift 2 + +buildid="latest" +if [ $# -gt 0 ]; then + case "$1" in + -client) + ;; + -nocvs|-novcs) + ;; + *) + buildid="$1" + shift + ;; + esac +fi pbc=${PORTBUILD_CHECKOUT:-/var/portbuild} pbd=${PORTBUILD_DATA:-/var/portbuild} builddir=${pbd}/${arch}/${branch}/builds/${buildid} +if [ ! -d ${builddir} ]; then + echo "build directory ${builddir} does not exist!" + exit 1 +fi . ${pbc}/conf/server.conf . ${pbc}/conf/common.conf . ${pbd}/${arch}/portbuild.conf -if [ -f ${pbd}/${arch}/${branch}/builds/${buildid}/portbuild.conf ]; then - . ${pbd}/${arch}/${branch}/builds/${buildid}/portbuild.conf +if [ -f ${builddir}/portbuild.conf ]; then + . ${builddir}/portbuild.conf fi # NB: we can't use buildenv because it sets ARCH and MACHINE_ARCH that # confuses cross-builds @@ -57,14 +74,23 @@ if [ "$client" = "1" ]; then shift 1 else SRC_BASE=${builddir}/src + # allow for per-build make.conf if [ -f ${builddir}/make.conf.server ]; then export __MAKE_CONF=${builddir}/make.conf.server else export __MAKE_CONF=/dev/null fi + + # allow for per-build src.conf + if [ -f ${builddir}/src.conf.server ]; then + export SRCCONF=${builddir}/src.conf.server + else + export SRCCONF=/dev/null + fi fi -cd ${SRC_BASE} + +cd ${SRC_BASE} || exit $? if [ "$novcs" = "0" ]; then echo "==> Updating source tree" Modified: projects/portbuild/scripts/mkbindist ============================================================================== --- projects/portbuild/scripts/mkbindist Thu May 24 21:44:46 2012 (r235939) +++ projects/portbuild/scripts/mkbindist Thu May 24 21:55:36 2012 (r235940) @@ -3,7 +3,7 @@ # XXX merge with makeworld? usage () { - echo "usage: mkbindist " + echo "usage: mkbindist []" exit 1 } @@ -16,22 +16,33 @@ cleandir() { fi } -if [ $# -lt 3 ]; then +if [ $# -lt 2 ]; then usage fi arch=$1 branch=$2 -buildid=$3 -shift 3 +shift 2 + +buildid="latest" +if [ $# -gt 0 ]; then + buildid="$1" + shift +fi pbc=${PORTBUILD_CHECKOUT:-/var/portbuild} pbd=${PORTBUILD_DATA:-/var/portbuild} +builddir=${pbd}/${arch}/${branch}/builds/${buildid} +if [ ! -d ${builddir} ]; then + echo "build directory ${builddir} does not exist!" + exit 1 +fi + . ${pbc}/conf/server.conf . ${pbd}/${arch}/portbuild.conf -if [ -f ${pbd}/${arch}/${branch}/builds/${buildid}/portbuild.conf ]; then - . ${pbd}/${arch}/${branch}/builds/${buildid}/portbuild.conf +if [ -f ${builddir}/portbuild.conf ]; then + . ${builddir}/portbuild.conf fi . ${pbc}/scripts/buildenv @@ -41,13 +52,7 @@ if ! validate_env ${arch} ${branch}; the exit 1 fi -here=${pbd}/${arch}/${branch}/builds/${buildid} -if [ ! -d ${here} ]; then - echo "Invalid build ID ${buildid}" - exit 1 -fi - -tmpdir=${here}/bindist/tmp +tmpdir=${builddir}/bindist/tmp # Clean up ${tmpdir} cleandir ${tmpdir} @@ -64,25 +69,25 @@ cd ${destdir}; find -dx . | \ cd ${tmpdir} # Customize the tmpdir -if [ -s "${here}/bindist/delete" ]; then - sed -e "s,^,${tmpdir}," ${here}/bindist/delete | xargs rm -rf +if [ -s "${builddir}/bindist/delete" ]; then + sed -e "s,^,${tmpdir}," ${builddir}/bindist/delete | xargs rm -rf fi -if [ -s "${here}/bindist/dirlist" ]; then - cat "${here}/bindist/dirlist" | xargs mkdir -p +if [ -s "${builddir}/bindist/dirlist" ]; then + cat "${builddir}/bindist/dirlist" | xargs mkdir -p fi # XXX MCL seems to be obsoleted by individual files in clients/? -if [ -d ${here}/bindist/files ]; then - cd ${here}/bindist/files; find -dx . | cpio -dump ${tmpdir} +if [ -d ${builddir}/bindist/files ]; then + cd ${builddir}/bindist/files; find -dx . | cpio -dump ${tmpdir} fi # Post-processing of installed world date '+%Y%m%d' > ${tmpdir}/var/db/port.mkversion # Create the tarball -tar cfCj ${here}/.bindist.tbz ${tmpdir} . -mv -f ${here}/.bindist.tbz ${here}/bindist.tbz -md5 ${here}/bindist.tbz > ${here}/bindist.tbz.md5 +tar cfCj ${builddir}/.bindist.tbz ${tmpdir} . +mv -f ${builddir}/.bindist.tbz ${builddir}/bindist.tbz +md5 ${builddir}/bindist.tbz > ${builddir}/bindist.tbz.md5 # Clean up -cd ${here} +cd ${builddir} cleandir ${tmpdir} From owner-svn-src-projects@FreeBSD.ORG Fri May 25 14:11:03 2012 Return-Path: Delivered-To: svn-src-projects@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 64B971065678; Fri, 25 May 2012 14:11:03 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 4DC5F8FC15; Fri, 25 May 2012 14:11:03 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.4/8.14.4) with ESMTP id q4PEB3li014296; Fri, 25 May 2012 14:11:03 GMT (envelope-from glebius@svn.freebsd.org) Received: (from glebius@localhost) by svn.freebsd.org (8.14.4/8.14.4/Submit) id q4PEB3Dt014289; Fri, 25 May 2012 14:11:03 GMT (envelope-from glebius@svn.freebsd.org) Message-Id: <201205251411.q4PEB3Dt014289@svn.freebsd.org> From: Gleb Smirnoff Date: Fri, 25 May 2012 14:11:03 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org X-SVN-Group: projects MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r235991 - projects/pf/head/sys/contrib/pf/net X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 May 2012 14:11:03 -0000 Author: glebius Date: Fri May 25 14:11:02 2012 New Revision: 235991 URL: http://svn.freebsd.org/changeset/base/235991 Log: 1) Locking kifs. In r234651 I made safe expiring of rules. However, states may also reference kifs directly. Rules, and some structures hanging off rules may also reference kifs. Now, states no longer update refcount on kifs, only rules do. When refcount on a kif reaches zero, and kif isn't representing an existing interface or group, then it is moved to a list of unlinked kifs locked by a separate mutex. These unlinked kifs are purged via naive mark-and-sweep run, similarly to unlinked rules expiry. 2) Apart from rules we've got some more structures that a hanging off the rules, and are read by the packet processing path: struct pfi_kif, struct pfi_dynaddr, struct pfr_ktable, struct pf_pool, struct pf_pooladdr. Reading these should require reader lock on rules, and modifying them writer lock. - Convert PF_LOCK to PF_RULES_WLOCK() in appropriate ioctls. - Acquire PF_RULES_WLOCK() in pf_free_rule(). To avoid LOR with unlinked rules mutex, use a temporary list on stack. - Assert pf rules lock in many functions that operate on tables or struct pf_addr_wrap. - Remove separate uma(9) zone for dynaddrs, no reason for it. 3) In many ioctl paths we used to obtain locks quite early, and thus wasn't able to use M_WAITOK. Make a poor attempt to make situation here better: - Do some parameter validation prior to actually processing the request - Pre-allocate struct pf_rule, struct pfi_kif prior to obtaining locks with M_WAITOK. Free them on failure. Unfortunately, fixing all pf(4) ioctls to use M_WAITOK is quite difficult, especially configuring pf tables, where rn_inithead() is called, which uses M_NOWAIT. Modified: projects/pf/head/sys/contrib/pf/net/if_pfsync.c projects/pf/head/sys/contrib/pf/net/pf.c projects/pf/head/sys/contrib/pf/net/pf_if.c projects/pf/head/sys/contrib/pf/net/pf_ioctl.c projects/pf/head/sys/contrib/pf/net/pf_table.c projects/pf/head/sys/contrib/pf/net/pfvar.h Modified: projects/pf/head/sys/contrib/pf/net/if_pfsync.c ============================================================================== --- projects/pf/head/sys/contrib/pf/net/if_pfsync.c Fri May 25 11:14:08 2012 (r235990) +++ projects/pf/head/sys/contrib/pf/net/if_pfsync.c Fri May 25 14:11:02 2012 (r235991) @@ -419,7 +419,7 @@ pfsync_state_import(struct pfsync_state struct pfi_kif *kif; int error; - PF_LOCK_ASSERT(); + PF_RULES_RASSERT(); if (sp->creatorid == 0 && V_pf_status.debug >= PF_DEBUG_MISC) { printf("%s: invalid creator id: %08x\n", __func__, @@ -427,7 +427,7 @@ pfsync_state_import(struct pfsync_state return (EINVAL); } - if ((kif = pfi_kif_get(sp->ifname)) == NULL) { + if ((kif = pfi_kif_find(sp->ifname)) == NULL) { if (V_pf_status.debug >= PF_DEBUG_MISC) printf("%s: unknown interface: %s\n", __func__, sp->ifname); @@ -629,6 +629,12 @@ pfsync_input(struct mbuf *m, __unused in pkt.src = ip->ip_src; pkt.flags = 0; + PF_LOCK(); + /* + * Trusting pf_chksum during packet processing, as well as seeking + * in interface name tree, require holding PF_RULES_RLOCK(). + */ + PF_RULES_RLOCK(); if (!bcmp(&ph->pfcksum, &V_pf_status.pf_chksum, PF_MD5_DIGEST_LENGTH)) pkt.flags |= PFSYNC_SI_CKSUM; @@ -639,17 +645,24 @@ pfsync_input(struct mbuf *m, __unused in if (subh.action >= PFSYNC_ACT_MAX) { V_pfsyncstats.pfsyncs_badact++; + PF_RULES_RUNLOCK(); + PF_UNLOCK(); goto done; } count = ntohs(subh.count); V_pfsyncstats.pfsyncs_iacts[subh.action] += count; rv = (*pfsync_acts[subh.action])(&pkt, m, offset, count); - if (rv == -1) + if (rv == -1) { + PF_RULES_RUNLOCK(); + PF_UNLOCK(); return; + } offset += rv; } + PF_RULES_RUNLOCK(); + PF_UNLOCK(); done: m_freem(m); @@ -671,12 +684,11 @@ pfsync_in_clr(struct pfsync_pkt *pkt, st } clr = (struct pfsync_clr *)(mp->m_data + offp); - PF_LOCK(); for (i = 0; i < count; i++) { creatorid = clr[i].creatorid; if (clr[i].ifname[0] != '\0' && - pfi_kif_get(clr[i].ifname) == NULL) + pfi_kif_find(clr[i].ifname) == NULL) continue; for (int i = 0; i <= V_pf_hashmask; i++) { @@ -694,7 +706,6 @@ relock: PF_HASHROW_UNLOCK(ih); } } - PF_UNLOCK(); return (len); } @@ -714,7 +725,6 @@ pfsync_in_ins(struct pfsync_pkt *pkt, st } sa = (struct pfsync_state *)(mp->m_data + offp); - PF_LOCK(); for (i = 0; i < count; i++) { sp = &sa[i]; @@ -734,7 +744,6 @@ pfsync_in_ins(struct pfsync_pkt *pkt, st /* Drop out, but process the rest of the actions. */ break; } - PF_UNLOCK(); return (len); } @@ -756,7 +765,6 @@ pfsync_in_iack(struct pfsync_pkt *pkt, s } iaa = (struct pfsync_ins_ack *)(mp->m_data + offp); - PF_LOCK(); for (i = 0; i < count; i++) { ia = &iaa[i]; @@ -771,7 +779,6 @@ pfsync_in_iack(struct pfsync_pkt *pkt, s } PF_STATE_UNLOCK(st); } - PF_UNLOCK(); /* * XXX this is not yet implemented, but we know the size of the * message so we can skip it. @@ -835,7 +842,6 @@ pfsync_in_upd(struct pfsync_pkt *pkt, st } sa = (struct pfsync_state *)(mp->m_data + offp); - PF_LOCK(); for (i = 0; i < count; i++) { sp = &sa[i]; @@ -903,7 +909,6 @@ pfsync_in_upd(struct pfsync_pkt *pkt, st st->pfsync_time = time_uptime; PF_STATE_UNLOCK(st); } - PF_UNLOCK(); return (len); } @@ -928,7 +933,6 @@ pfsync_in_upd_c(struct pfsync_pkt *pkt, } ua = (struct pfsync_upd_c *)(mp->m_data + offp); - PF_LOCK(); for (i = 0; i < count; i++) { up = &ua[i]; @@ -997,7 +1001,6 @@ pfsync_in_upd_c(struct pfsync_pkt *pkt, st->pfsync_time = time_uptime; PF_STATE_UNLOCK(st); } - PF_UNLOCK(); return (len); } @@ -1019,7 +1022,6 @@ pfsync_in_ureq(struct pfsync_pkt *pkt, s } ura = (struct pfsync_upd_req *)(mp->m_data + offp); - PF_LOCK(); for (i = 0; i < count; i++) { ur = &ura[i]; @@ -1040,7 +1042,6 @@ pfsync_in_ureq(struct pfsync_pkt *pkt, s PF_STATE_UNLOCK(st); } } - PF_UNLOCK(); return (len); } @@ -1061,7 +1062,6 @@ pfsync_in_del(struct pfsync_pkt *pkt, st } sa = (struct pfsync_state *)(mp->m_data + offp); - PF_LOCK(); for (i = 0; i < count; i++) { sp = &sa[i]; @@ -1073,7 +1073,6 @@ pfsync_in_del(struct pfsync_pkt *pkt, st st->state_flags |= PFSTATE_NOSYNC; pf_unlink_state(st, PF_ENTER_LOCKED); } - PF_UNLOCK(); return (len); } @@ -1094,7 +1093,6 @@ pfsync_in_del_c(struct pfsync_pkt *pkt, } sa = (struct pfsync_del_c *)(mp->m_data + offp); - PF_LOCK(); for (i = 0; i < count; i++) { sp = &sa[i]; @@ -1107,7 +1105,6 @@ pfsync_in_del_c(struct pfsync_pkt *pkt, st->state_flags |= PFSTATE_NOSYNC; pf_unlink_state(st, PF_ENTER_LOCKED); } - PF_UNLOCK(); return (len); } @@ -1193,10 +1190,8 @@ pfsync_in_tdb(struct pfsync_pkt *pkt, st } tp = (struct pfsync_tdb *)(mp->m_data + offp); - PF_LOCK(); for (i = 0; i < count; i++) pfsync_update_net_tdb(&tp[i]); - PF_UNLOCK(); #endif return (len); @@ -1662,8 +1657,6 @@ pfsync_insert_state(struct pf_state *st) { struct pfsync_softc *sc = V_pfsyncif; - PF_LOCK_ASSERT(); - if (st->state_flags & PFSTATE_NOSYNC) return; Modified: projects/pf/head/sys/contrib/pf/net/pf.c ============================================================================== --- projects/pf/head/sys/contrib/pf/net/pf.c Fri May 25 11:14:08 2012 (r235990) +++ projects/pf/head/sys/contrib/pf/net/pf.c Fri May 25 14:11:02 2012 (r235991) @@ -730,8 +730,6 @@ pf_initialize() sizeof(struct pfr_kentry), NULL, NULL, NULL, NULL, UMA_ALIGN_PTR, 0); V_pf_limits[PF_LIMIT_TABLE_ENTRIES].zone = V_pfr_kentry_z; - V_pfi_addr_z = uma_zcreate("pf pfi_dynaddr", sizeof(struct pfi_dynaddr), - NULL, NULL, NULL, NULL, UMA_ALIGN_PTR, 0); } void @@ -778,7 +776,6 @@ pf_cleanup() uma_zdestroy(V_pf_pooladdr_z); uma_zdestroy(V_pfr_ktable_z); uma_zdestroy(V_pfr_kentry_z); - uma_zdestroy(V_pfi_addr_z); } static int @@ -1051,7 +1048,6 @@ pf_state_insert(struct pfi_kif *kif, str V_pf_status.fcounters[FCNT_STATE_INSERT]++; V_pf_status.states++; - pfi_kif_ref(kif, PFI_KIF_REF_STATE); if (pfsync_insert_state_ptr != NULL) pfsync_insert_state_ptr(s); @@ -1312,10 +1308,15 @@ pf_purge_thread(void *v) /* Purge other expired types every PFTM_INTERVAL seconds. */ if (fullrun) { - /* Order important: rules should be the last. */ + /* + * Order is important: + * - states and src nodes reference rules + * - states and rules reference kifs + */ pf_purge_expired_fragments(); pf_purge_expired_src_nodes(); pf_purge_unlinked_rules(); + pfi_kif_purge(); } PF_UNLOCK(); @@ -1471,7 +1472,6 @@ pf_free_state(struct pf_state *cur) if (cur->anchor.ptr != NULL) --cur->anchor.ptr->states_cur; pf_normalize_tcp_cleanup(cur); - pfi_kif_unref(cur->kif, PFI_KIF_REF_STATE); if (cur->tag) pf_tag_unref(cur->tag); uma_zfree(V_pf_state_z, cur); @@ -1515,7 +1515,9 @@ relock: s->nat_rule.ptr->rule_flag |= PFRULE_REFS; if (s->anchor.ptr != NULL) s->anchor.ptr->rule_flag |= PFRULE_REFS; - + s->kif->pfik_flags |= PFI_IFLAG_REFS; + if (s->rt_kif) + s->rt_kif->pfik_flags |= PFI_IFLAG_REFS; } PF_HASHROW_UNLOCK(ih); i++; @@ -1528,22 +1530,36 @@ relock: static void pf_purge_unlinked_rules() { + struct pf_rulequeue tmpq; struct pf_rule *r, *r1; /* * Do naive mark-and-sweep garbage collecting of old rules. * Reference flag is raised by pf_purge_expired_states() * and pf_purge_expired_src_nodes(). + * + * To avoid LOR between PF_UNLNKDRULES_LOCK/PF_RULES_WLOCK, + * use a temporary queue. */ + TAILQ_INIT(&tmpq); PF_UNLNKDRULES_LOCK(); TAILQ_FOREACH_SAFE(r, &V_pf_unlinked_rules, entries, r1) { if (!(r->rule_flag & PFRULE_REFS)) { TAILQ_REMOVE(&V_pf_unlinked_rules, r, entries); - pf_free_rule(r); + TAILQ_INSERT_TAIL(&tmpq, r, entries); } else r->rule_flag &= ~PFRULE_REFS; } PF_UNLNKDRULES_UNLOCK(); + + if (!TAILQ_EMPTY(&tmpq)) { + PF_RULES_WLOCK(); + TAILQ_FOREACH_SAFE(r, &tmpq, entries, r1) { + TAILQ_REMOVE(&tmpq, r, entries); + pf_free_rule(r); + } + PF_RULES_WUNLOCK(); + } } void Modified: projects/pf/head/sys/contrib/pf/net/pf_if.c ============================================================================== --- projects/pf/head/sys/contrib/pf/net/pf_if.c Fri May 25 11:14:08 2012 (r235990) +++ projects/pf/head/sys/contrib/pf/net/pf_if.c Fri May 25 14:11:02 2012 (r235991) @@ -65,17 +65,18 @@ __FBSDID("$FreeBSD$"); #endif /* INET6 */ VNET_DEFINE(struct pfi_kif *, pfi_all); -VNET_DEFINE(uma_zone_t, pfi_addr_z); -VNET_DEFINE(struct pfi_ifhead, pfi_ifs); -#define V_pfi_ifs VNET(pfi_ifs) -VNET_DEFINE(long, pfi_update); -#define V_pfi_update VNET(pfi_update) -VNET_DEFINE(struct pfr_addr *, pfi_buffer); +static VNET_DEFINE(long, pfi_update); +#define V_pfi_update VNET(pfi_update) +#define PFI_BUFFER_MAX 0x10000 + +/* XXXGL */ +static VNET_DEFINE(struct pfr_addr *, pfi_buffer); +static VNET_DEFINE(int, pfi_buffer_cnt); +static VNET_DEFINE(int, pfi_buffer_max); #define V_pfi_buffer VNET(pfi_buffer) -VNET_DEFINE(int, pfi_buffer_cnt); #define V_pfi_buffer_cnt VNET(pfi_buffer_cnt) -VNET_DEFINE(int, pfi_buffer_max); #define V_pfi_buffer_max VNET(pfi_buffer_max) + eventhandler_tag pfi_attach_cookie; eventhandler_tag pfi_detach_cookie; eventhandler_tag pfi_attach_group_cookie; @@ -84,16 +85,12 @@ eventhandler_tag pfi_detach_group_cooki eventhandler_tag pfi_ifaddr_event_cookie; static void pfi_attach_ifnet(struct ifnet *); -static void pfi_detach_ifnet(struct ifnet *); static void pfi_attach_ifgroup(struct ifg_group *); -static void pfi_detach_ifgroup(struct ifg_group *); -static void pfi_group_change(const char *); static void pfi_kif_update(struct pfi_kif *); static void pfi_dynaddr_update(struct pfi_dynaddr *dyn); static void pfi_table_update(struct pfr_ktable *, struct pfi_kif *, int, int); -static void pfi_kifaddr_update(void *); static void pfi_instance_add(struct ifnet *, int, int); static void pfi_address_add(struct sockaddr *, int, int); static int pfi_if_compare(struct pfi_kif *, struct pfi_kif *); @@ -106,26 +103,37 @@ static void pfi_change_group_event(void static void pfi_detach_group_event(void *, struct ifg_group *); static void pfi_ifaddr_event(void * __unused, struct ifnet *); -RB_PROTOTYPE(pfi_ifhead, pfi_kif, pfik_tree, pfi_if_compare); -RB_GENERATE(pfi_ifhead, pfi_kif, pfik_tree, pfi_if_compare); - -#define PFI_BUFFER_MAX 0x10000 -#define PFI_MTYPE M_IFADDR +RB_HEAD(pfi_ifhead, pfi_kif); +static RB_PROTOTYPE(pfi_ifhead, pfi_kif, pfik_tree, pfi_if_compare); +static RB_GENERATE(pfi_ifhead, pfi_kif, pfik_tree, pfi_if_compare); +static VNET_DEFINE(struct pfi_ifhead, pfi_ifs); +#define V_pfi_ifs VNET(pfi_ifs) + +#define PFI_BUFFER_MAX 0x10000 +MALLOC_DEFINE(PFI_MTYPE, "pf ifnets", "pf interface database"); + +LIST_HEAD(pfi_list, pfi_kif); +static VNET_DEFINE(struct pfi_list, pfi_unlinked_kifs); +#define V_pfi_unlinked_kifs VNET(pfi_unlinked_kifs) +static struct mtx pfi_unlnkdkifs_mtx; void pfi_initialize(void) { - if (V_pfi_all != NULL) /* already initialized */ - return; + struct ifg_group *ifg; + struct ifnet *ifp; + struct pfi_kif *kif; V_pfi_buffer_max = 64; V_pfi_buffer = malloc(V_pfi_buffer_max * sizeof(*V_pfi_buffer), PFI_MTYPE, M_WAITOK); - if ((V_pfi_all = pfi_kif_get(IFG_ALL)) == NULL) - panic("pfi_kif_get for pfi_all failed"); - struct ifg_group *ifg; - struct ifnet *ifp; + mtx_init(&pfi_unlnkdkifs_mtx, "pf unlinked interfaces", NULL, MTX_DEF); + + kif = malloc(sizeof(*kif), PFI_MTYPE, M_WAITOK); + PF_RULES_WLOCK(); + V_pfi_all = pfi_kif_attach(kif, IFG_ALL); + PF_RULES_WUNLOCK(); IFNET_RLOCK(); TAILQ_FOREACH(ifg, &V_ifg_head, ifg_next) @@ -153,53 +161,59 @@ pfi_cleanup(void) { struct pfi_kif *p; - PF_UNLOCK(); EVENTHANDLER_DEREGISTER(ifnet_arrival_event, pfi_attach_cookie); EVENTHANDLER_DEREGISTER(ifnet_departure_event, pfi_detach_cookie); EVENTHANDLER_DEREGISTER(group_attach_event, pfi_attach_group_cookie); EVENTHANDLER_DEREGISTER(group_change_event, pfi_change_group_cookie); EVENTHANDLER_DEREGISTER(group_detach_event, pfi_detach_group_cookie); EVENTHANDLER_DEREGISTER(ifaddr_event, pfi_ifaddr_event_cookie); - PF_LOCK(); V_pfi_all = NULL; while ((p = RB_MIN(pfi_ifhead, &V_pfi_ifs))) { - if (p->pfik_rules || p->pfik_states) { - printf("pfi_cleanup: dangling refs for %s\n", - p->pfik_name); - } - RB_REMOVE(pfi_ifhead, &V_pfi_ifs, p); free(p, PFI_MTYPE); } + while ((p = LIST_FIRST(&V_pfi_unlinked_kifs))) { + LIST_REMOVE(p, pfik_list); + free(p, PFI_MTYPE); + } + + mtx_destroy(&pfi_unlnkdkifs_mtx); + free(V_pfi_buffer, PFI_MTYPE); } struct pfi_kif * -pfi_kif_get(const char *kif_name) +pfi_kif_find(const char *kif_name) { - struct pfi_kif *kif; - struct pfi_kif_cmp s; + struct pfi_kif_cmp s; + + PF_RULES_ASSERT(); bzero(&s, sizeof(s)); strlcpy(s.pfik_name, kif_name, sizeof(s.pfik_name)); - if ((kif = RB_FIND(pfi_ifhead, &V_pfi_ifs, (struct pfi_kif *)&s)) != NULL) - return (kif); - /* create new one */ - if ((kif = malloc(sizeof(*kif), PFI_MTYPE, M_NOWAIT | M_ZERO)) == NULL) - return (NULL); + return (RB_FIND(pfi_ifhead, &V_pfi_ifs, (struct pfi_kif *)&s)); +} + +struct pfi_kif * +pfi_kif_attach(struct pfi_kif *kif, const char *kif_name) +{ + struct pfi_kif *kif1; + PF_RULES_WASSERT(); + KASSERT(kif != NULL, ("%s: null kif", __func__)); + + kif1 = pfi_kif_find(kif_name); + if (kif1 != NULL) { + free(kif, PFI_MTYPE); + return (kif1); + } + + bzero(kif, sizeof(*kif)); strlcpy(kif->pfik_name, kif_name, sizeof(kif->pfik_name)); - /* - * It seems that the value of time_second is in unintialzied state - * when pf sets interface statistics clear time in boot phase if pf - * was statically linked to kernel. Instead of setting the bogus - * time value have pfi_get_ifaces handle this case. In - * pfi_get_ifaces it uses boottime.tv_sec if it sees the time is 0. - */ - kif->pfik_tzero = time_second > 1 ? time_second : 0; + kif->pfik_tzero = time_second; TAILQ_INIT(&kif->pfik_dynaddrs); RB_INSERT(pfi_ifhead, &V_pfi_ifs, kif); @@ -208,55 +222,56 @@ pfi_kif_get(const char *kif_name) } void -pfi_kif_ref(struct pfi_kif *kif, enum pfi_kif_refs what) +pfi_kif_ref(struct pfi_kif *kif) { - switch (what) { - case PFI_KIF_REF_RULE: - kif->pfik_rules++; - break; - case PFI_KIF_REF_STATE: - kif->pfik_states++; - break; - default: - panic("pfi_kif_ref with unknown type"); - } + + PF_RULES_WASSERT(); + kif->pfik_rulerefs++; } void -pfi_kif_unref(struct pfi_kif *kif, enum pfi_kif_refs what) +pfi_kif_unref(struct pfi_kif *kif) { - if (kif == NULL) - return; - switch (what) { - case PFI_KIF_REF_NONE: - break; - case PFI_KIF_REF_RULE: - if (kif->pfik_rules <= 0) { - printf("pfi_kif_unref: rules refcount <= 0\n"); - return; - } - kif->pfik_rules--; - break; - case PFI_KIF_REF_STATE: - if (kif->pfik_states <= 0) { - printf("pfi_kif_unref: state refcount <= 0\n"); - return; - } - kif->pfik_states--; - break; - default: - panic("pfi_kif_unref with unknown type"); - } + PF_RULES_WASSERT(); + KASSERT(kif->pfik_rulerefs > 0, ("%s: %p has zero refs", __func__, kif)); - if (kif->pfik_ifp != NULL || kif->pfik_group != NULL || kif == V_pfi_all) + kif->pfik_rulerefs--; + + if (kif->pfik_rulerefs > 0) return; - if (kif->pfik_rules || kif->pfik_states) + /* kif referencing an existing ifnet or group should exist. */ + if (kif->pfik_ifp != NULL || kif->pfik_group != NULL || kif == V_pfi_all) return; RB_REMOVE(pfi_ifhead, &V_pfi_ifs, kif); - free(kif, PFI_MTYPE); + + kif->pfik_flags |= PFI_IFLAG_REFS; + + mtx_lock(&pfi_unlnkdkifs_mtx); + LIST_INSERT_HEAD(&V_pfi_unlinked_kifs, kif, pfik_list); + mtx_unlock(&pfi_unlnkdkifs_mtx); +} + +void +pfi_kif_purge(void) +{ + struct pfi_kif *kif, *kif1; + + /* + * Do naive mark-and-sweep garbage collecting of old kifs. + * Reference flag is raised by pf_purge_expired_states(). + */ + mtx_lock(&pfi_unlnkdkifs_mtx); + LIST_FOREACH_SAFE(kif, &V_pfi_unlinked_kifs, pfik_list, kif1) { + if (!(kif->pfik_flags & PFI_IFLAG_REFS)) { + LIST_REMOVE(kif, pfik_list); + free(kif, PFI_MTYPE); + } else + kif->pfik_flags &= ~PFI_IFLAG_REFS; + } + mtx_unlock(&pfi_unlnkdkifs_mtx); } int @@ -268,6 +283,7 @@ pfi_kif_match(struct pfi_kif *rule_kif, return (1); if (rule_kif->pfik_group != NULL) + /* XXXGL: locking? */ TAILQ_FOREACH(p, &packet_kif->pfik_ifp->if_groups, ifgl_next) if (p->ifgl_group == rule_kif->pfik_group) return (1); @@ -278,75 +294,35 @@ pfi_kif_match(struct pfi_kif *rule_kif, static void pfi_attach_ifnet(struct ifnet *ifp) { - struct pfi_kif *kif; + struct pfi_kif *kif; + + kif = malloc(sizeof(*kif), PFI_MTYPE, M_WAITOK); - pfi_initialize(); + PF_RULES_WLOCK(); V_pfi_update++; - if ((kif = pfi_kif_get(ifp->if_xname)) == NULL) - panic("pfi_kif_get failed"); + kif = pfi_kif_attach(kif, ifp->if_xname); kif->pfik_ifp = ifp; - ifp->if_pf_kif = (caddr_t)kif; - - - pfi_kif_update(kif); -} - -static void -pfi_detach_ifnet(struct ifnet *ifp) -{ - struct pfi_kif *kif; - - if ((kif = (struct pfi_kif *)ifp->if_pf_kif) == NULL) - return; + ifp->if_pf_kif = kif; - V_pfi_update++; pfi_kif_update(kif); - - kif->pfik_ifp = NULL; - ifp->if_pf_kif = NULL; - pfi_kif_unref(kif, PFI_KIF_REF_NONE); + PF_RULES_WUNLOCK(); } static void pfi_attach_ifgroup(struct ifg_group *ifg) { - struct pfi_kif *kif; - - pfi_initialize(); - V_pfi_update++; - if ((kif = pfi_kif_get(ifg->ifg_group)) == NULL) - panic("pfi_kif_get failed"); - - kif->pfik_group = ifg; - ifg->ifg_pf_kif = (caddr_t)kif; -} - -static void -pfi_detach_ifgroup(struct ifg_group *ifg) -{ - struct pfi_kif *kif; - - if ((kif = (struct pfi_kif *)ifg->ifg_pf_kif) == NULL) - return; - - V_pfi_update++; - - kif->pfik_group = NULL; - ifg->ifg_pf_kif = NULL; - pfi_kif_unref(kif, PFI_KIF_REF_NONE); -} + struct pfi_kif *kif; -static void -pfi_group_change(const char *group) -{ - struct pfi_kif *kif; + kif = malloc(sizeof(*kif), PFI_MTYPE, M_WAITOK); + PF_RULES_WLOCK(); V_pfi_update++; - if ((kif = pfi_kif_get(group)) == NULL) - panic("pfi_kif_get failed"); + kif = pfi_kif_attach(kif, ifg->ifg_group); - pfi_kif_update(kif); + kif->pfik_group = ifg; + ifg->ifg_pf_kif = kif; + PF_RULES_WUNLOCK(); } int @@ -390,24 +366,27 @@ pfi_dynaddr_setup(struct pf_addr_wrap *a struct pfi_dynaddr *dyn; char tblname[PF_TABLE_NAME_SIZE]; struct pf_ruleset *ruleset = NULL; + struct pfi_kif *kif; int rv = 0; + PF_RULES_WASSERT(); KASSERT(aw->type == PF_ADDR_DYNIFTL, ("%s: type %u", __func__, aw->type)); KASSERT(aw->p.dyn == NULL, ("%s: dyn is %p", __func__, aw->p.dyn)); - if ((dyn = uma_zalloc(V_pfi_addr_z, M_NOWAIT | M_ZERO)) == NULL) + if ((dyn = malloc(sizeof(*dyn), PFI_MTYPE, M_NOWAIT | M_ZERO)) == NULL) return (ENOMEM); + if ((kif = malloc(sizeof(*kif), PFI_MTYPE, M_NOWAIT)) == NULL) { + free(dyn, PFI_MTYPE); + return (ENOMEM); + } + if (!strcmp(aw->v.ifname, "self")) - dyn->pfid_kif = pfi_kif_get(IFG_ALL); + dyn->pfid_kif = pfi_kif_attach(kif, IFG_ALL); else - dyn->pfid_kif = pfi_kif_get(aw->v.ifname); - if (dyn->pfid_kif == NULL) { - rv = ENOENT; - goto _bad; - } - pfi_kif_ref(dyn->pfid_kif, PFI_KIF_REF_RULE); + dyn->pfid_kif = pfi_kif_attach(kif, aw->v.ifname); + pfi_kif_ref(dyn->pfid_kif); dyn->pfid_net = pfi_unmask(&aw->v.a.mask); if (af == AF_INET && dyn->pfid_net == 32) @@ -450,8 +429,8 @@ _bad: if (ruleset != NULL) pf_remove_if_empty_ruleset(ruleset); if (dyn->pfid_kif != NULL) - pfi_kif_unref(dyn->pfid_kif, PFI_KIF_REF_RULE); - uma_zfree(V_pfi_addr_z, dyn); + pfi_kif_unref(dyn->pfid_kif); + free(dyn, PFI_MTYPE); return (rv); } @@ -462,15 +441,20 @@ pfi_kif_update(struct pfi_kif *kif) struct ifg_list *ifgl; struct pfi_dynaddr *p; + PF_RULES_WASSERT(); + /* update all dynaddr */ TAILQ_FOREACH(p, &kif->pfik_dynaddrs, entry) pfi_dynaddr_update(p); /* again for all groups kif is member of */ - if (kif->pfik_ifp != NULL) + if (kif->pfik_ifp != NULL) { + IF_ADDR_RLOCK(kif->pfik_ifp); TAILQ_FOREACH(ifgl, &kif->pfik_ifp->if_groups, ifgl_next) pfi_kif_update((struct pfi_kif *) ifgl->ifgl_group->ifg_pf_kif); + IF_ADDR_RUNLOCK(kif->pfik_ifp); + } } static void @@ -479,8 +463,9 @@ pfi_dynaddr_update(struct pfi_dynaddr *d struct pfi_kif *kif; struct pfr_ktable *kt; - if (dyn == NULL || dyn->pfid_kif == NULL || dyn->pfid_kt == NULL) - panic("pfi_dynaddr_update"); + PF_RULES_WASSERT(); + KASSERT(dyn && dyn->pfid_kif && dyn->pfid_kt, + ("%s: bad argument", __func__)); kif = dyn->pfid_kif; kt = dyn->pfid_kt; @@ -503,14 +488,17 @@ pfi_table_update(struct pfr_ktable *kt, if (kif->pfik_ifp != NULL) pfi_instance_add(kif->pfik_ifp, net, flags); - else if (kif->pfik_group != NULL) + else if (kif->pfik_group != NULL) { + IFNET_RLOCK(); TAILQ_FOREACH(ifgm, &kif->pfik_group->ifg_members, ifgm_next) pfi_instance_add(ifgm->ifgm_ifp, net, flags); + IFNET_RUNLOCK(); + } if ((e = pfr_set_addrs(&kt->pfrkt_t, V_pfi_buffer, V_pfi_buffer_cnt, &size2, NULL, NULL, NULL, 0, PFR_TFLAG_ALLMASK))) - printf("pfi_table_update: cannot set %d new addresses " - "into table %s: %d\n", V_pfi_buffer_cnt, kt->pfrkt_name, e); + printf("%s: cannot set %d new addresses into table %s: %d\n", + __func__, V_pfi_buffer_cnt, kt->pfrkt_name, e); } static void @@ -520,9 +508,8 @@ pfi_instance_add(struct ifnet *ifp, int int got4 = 0, got6 = 0; int net2, af; - if (ifp == NULL) - return; - TAILQ_FOREACH(ia, &ifp->if_addrlist, ifa_list) { + IF_ADDR_RLOCK(ifp); + TAILQ_FOREACH(ia, &ifp->if_addrhead, ifa_list) { if (ia->ifa_addr == NULL) continue; af = ia->ifa_addr->sa_family; @@ -578,6 +565,7 @@ pfi_instance_add(struct ifnet *ifp, int else pfi_address_add(ia->ifa_addr, af, net2); } + IF_ADDR_RUNLOCK(ifp); } static void @@ -590,15 +578,15 @@ pfi_address_add(struct sockaddr *sa, int int new_max = V_pfi_buffer_max * 2; if (new_max > PFI_BUFFER_MAX) { - printf("pfi_address_add: address buffer full (%d/%d)\n", + printf("%s: address buffer full (%d/%d)\n", __func__, V_pfi_buffer_cnt, PFI_BUFFER_MAX); return; } p = malloc(new_max * sizeof(*V_pfi_buffer), PFI_MTYPE, M_NOWAIT); if (p == NULL) { - printf("pfi_address_add: no memory to grow buffer " - "(%d/%d)\n", V_pfi_buffer_cnt, PFI_BUFFER_MAX); + printf("%s: no memory to grow buffer (%d/%d)\n", + __func__, V_pfi_buffer_cnt, PFI_BUFFER_MAX); return; } memcpy(V_pfi_buffer, p, V_pfi_buffer_cnt * sizeof(*V_pfi_buffer)); @@ -635,9 +623,9 @@ pfi_dynaddr_remove(struct pfi_dynaddr *d KASSERT(dyn->pfid_kt != NULL, ("%s: null pfid_kt", __func__)); TAILQ_REMOVE(&dyn->pfid_kif->pfik_dynaddrs, dyn, entry); - pfi_kif_unref(dyn->pfid_kif, PFI_KIF_REF_RULE); + pfi_kif_unref(dyn->pfid_kif); pfr_detach_table(dyn->pfid_kt); - uma_zfree(V_pfi_addr_z, dyn); + free(dyn, PFI_MTYPE); } void @@ -652,15 +640,6 @@ pfi_dynaddr_copyout(struct pf_addr_wrap aw->p.dyncnt = aw->p.dyn->pfid_acnt4 + aw->p.dyn->pfid_acnt6; } -static void -pfi_kifaddr_update(void *v) -{ - struct pfi_kif *kif = (struct pfi_kif *)v; - - V_pfi_update++; - pfi_kif_update(kif); -} - static int pfi_if_compare(struct pfi_kif *p, struct pfi_kif *q) { @@ -808,8 +787,8 @@ pfi_attach_ifnet_event(void *arg __unuse { CURVNET_SET(ifp->if_vnet); - PF_LOCK(); pfi_attach_ifnet(ifp); + PF_LOCK(); #ifdef ALTQ pf_altq_ifnet_event(ifp, 0); #endif @@ -820,10 +799,18 @@ pfi_attach_ifnet_event(void *arg __unuse static void pfi_detach_ifnet_event(void *arg __unused, struct ifnet *ifp) { + struct pfi_kif *kif = (struct pfi_kif *)ifp->if_pf_kif; CURVNET_SET(ifp->if_vnet); + PF_RULES_WLOCK(); + V_pfi_update++; + pfi_kif_update(kif); + + kif->pfik_ifp = NULL; + ifp->if_pf_kif = NULL; + PF_RULES_WUNLOCK(); + PF_LOCK(); - pfi_detach_ifnet(ifp); #ifdef ALTQ pf_altq_ifnet_event(ifp, 1); #endif @@ -836,31 +823,38 @@ pfi_attach_group_event(void *arg , struc { CURVNET_SET((struct vnet *)arg); - PF_LOCK(); pfi_attach_ifgroup(ifg); - PF_UNLOCK(); CURVNET_RESTORE(); } static void pfi_change_group_event(void *arg, char *gname) { + struct pfi_kif *kif; + + kif = malloc(sizeof(*kif), PFI_MTYPE, M_WAITOK); CURVNET_SET((struct vnet *)arg); - PF_LOCK(); - pfi_group_change(gname); - PF_UNLOCK(); + PF_RULES_WLOCK(); + V_pfi_update++; + kif = pfi_kif_attach(kif, gname); + pfi_kif_update(kif); + PF_RULES_WUNLOCK(); CURVNET_RESTORE(); } static void pfi_detach_group_event(void *arg, struct ifg_group *ifg) { + struct pfi_kif *kif = (struct pfi_kif *)ifg->ifg_pf_kif; CURVNET_SET((struct vnet *)arg); - PF_LOCK(); - pfi_detach_ifgroup(ifg); - PF_UNLOCK(); + PF_RULES_WLOCK(); + V_pfi_update++; + + kif->pfik_group = NULL; + ifg->ifg_pf_kif = NULL; + PF_RULES_WUNLOCK(); CURVNET_RESTORE(); } @@ -869,9 +863,11 @@ pfi_ifaddr_event(void *arg __unused, str { CURVNET_SET(ifp->if_vnet); - PF_LOCK(); - if (ifp && ifp->if_pf_kif) - pfi_kifaddr_update(ifp->if_pf_kif); - PF_UNLOCK(); + PF_RULES_WLOCK(); + if (ifp && ifp->if_pf_kif) { + V_pfi_update++; + pfi_kif_update(ifp->if_pf_kif); + } + PF_RULES_WUNLOCK(); CURVNET_RESTORE(); } Modified: projects/pf/head/sys/contrib/pf/net/pf_ioctl.c ============================================================================== --- projects/pf/head/sys/contrib/pf/net/pf_ioctl.c Fri May 25 11:14:08 2012 (r235990) +++ projects/pf/head/sys/contrib/pf/net/pf_ioctl.c Fri May 25 14:11:02 2012 (r235991) @@ -379,7 +379,8 @@ pf_empty_pool(struct pf_palist *poola) pfr_detach_table(pa->addr.p.tbl); break; } - pfi_kif_unref(pa->kif, PFI_KIF_REF_RULE); + if (pa->kif) + pfi_kif_unref(pa->kif); TAILQ_REMOVE(poola, pa, entries); uma_zfree(V_pf_pooladdr_z, pa); } @@ -403,6 +404,8 @@ void pf_free_rule(struct pf_rule *rule) { + PF_RULES_WASSERT(); + pf_tag_unref(rule->tag); pf_tag_unref(rule->match_tag); #ifdef ALTQ @@ -428,7 +431,8 @@ pf_free_rule(struct pf_rule *rule) } if (rule->overload_tbl) pfr_detach_table(rule->overload_tbl); - pfi_kif_unref(rule->kif, PFI_KIF_REF_RULE); + if (rule->kif) + pfi_kif_unref(rule->kif); pf_anchor_remove(rule); pf_empty_pool(&rule->rpool.list); uma_zfree(V_pf_rule_z, rule); @@ -1016,8 +1020,6 @@ pf_addr_copyout(struct pf_addr_wrap *add static int pfioctl(struct cdev *dev, u_long cmd, caddr_t addr, int flags, struct thread *td) { - struct pf_pooladdr *pa = NULL; - struct pf_pool *pool = NULL; int error = 0; CURVNET_SET(TD_TO_VNET(td)); @@ -1179,75 +1181,57 @@ pfioctl(struct cdev *dev, u_long cmd, ca struct pf_ruleset *ruleset; struct pf_rule *rule, *tail; struct pf_pooladdr *pa; + struct pfi_kif *kif = NULL; int rs_num; - PF_RULES_WLOCK(); - pr->anchor[sizeof(pr->anchor) - 1] = 0; - ruleset = pf_find_ruleset(pr->anchor); - if (ruleset == NULL) { - PF_RULES_WUNLOCK(); + if (pr->rule.return_icmp >> 8 > ICMP_MAXTYPE) { error = EINVAL; *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-projects@FreeBSD.ORG Fri May 25 14:49:52 2012 Return-Path: Delivered-To: svn-src-projects@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 420AC106566C; Fri, 25 May 2012 14:49:52 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 227778FC1C; Fri, 25 May 2012 14:49:52 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.4/8.14.4) with ESMTP id q4PEnpF8016122; Fri, 25 May 2012 14:49:51 GMT (envelope-from glebius@svn.freebsd.org) Received: (from glebius@localhost) by svn.freebsd.org (8.14.4/8.14.4/Submit) id q4PEnpYd016118; Fri, 25 May 2012 14:49:51 GMT (envelope-from glebius@svn.freebsd.org) Message-Id: <201205251449.q4PEnpYd016118@svn.freebsd.org> From: Gleb Smirnoff Date: Fri, 25 May 2012 14:49:51 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org X-SVN-Group: projects MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r235993 - projects/pf/head/sys/contrib/pf/net X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 May 2012 14:49:52 -0000 Author: glebius Date: Fri May 25 14:49:51 2012 New Revision: 235993 URL: http://svn.freebsd.org/changeset/base/235993 Log: - Stop referencing tag names by states. Not a big deal if tag name disappears. - Make tag code more private to pf_ioctl.c Modified: projects/pf/head/sys/contrib/pf/net/pf.c projects/pf/head/sys/contrib/pf/net/pf_ioctl.c projects/pf/head/sys/contrib/pf/net/pfvar.h Modified: projects/pf/head/sys/contrib/pf/net/pf.c ============================================================================== --- projects/pf/head/sys/contrib/pf/net/pf.c Fri May 25 14:40:56 2012 (r235992) +++ projects/pf/head/sys/contrib/pf/net/pf.c Fri May 25 14:49:51 2012 (r235993) @@ -1472,8 +1472,6 @@ pf_free_state(struct pf_state *cur) if (cur->anchor.ptr != NULL) --cur->anchor.ptr->states_cur; pf_normalize_tcp_cleanup(cur); - if (cur->tag) - pf_tag_unref(cur->tag); uma_zfree(V_pf_state_z, cur); V_pf_status.fcounters[FCNT_STATE_REMOVALS]++; V_pf_status.states--; @@ -3446,10 +3444,8 @@ pf_create_state(struct pf_rule *r, struc *sm = s; pf_set_rt_ifp(s, pd->src); /* needs s->state_key set */ - if (tag > 0) { - pf_tag_ref(tag); + if (tag > 0) s->tag = tag; - } if (pd->proto == IPPROTO_TCP && (th->th_flags & (TH_SYN|TH_ACK)) == TH_SYN && r->keep_state == PF_STATE_SYNPROXY) { s->src.state = PF_TCPS_PROXY_SRC; Modified: projects/pf/head/sys/contrib/pf/net/pf_ioctl.c ============================================================================== --- projects/pf/head/sys/contrib/pf/net/pf_ioctl.c Fri May 25 14:40:56 2012 (r235992) +++ projects/pf/head/sys/contrib/pf/net/pf_ioctl.c Fri May 25 14:49:51 2012 (r235993) @@ -135,8 +135,6 @@ static int pf_addr_setup(struct pf_rul static void pf_addr_copyout(struct pf_addr_wrap *); static void pf_pkt_addr_changed(struct mbuf *); -#define TAGID_MAX 50000 - VNET_DEFINE(struct pf_rule, pf_default_rule); VNET_DEFINE(struct sx, pf_consistency_lock); #define V_pf_consistency_lock VNET(pf_consistency_lock) @@ -146,13 +144,20 @@ static VNET_DEFINE(int, pf_altq_running #define V_pf_altq_running VNET(pf_altq_running) #endif -TAILQ_HEAD(pf_tags, pf_tagname); +#define TAGID_MAX 50000 +struct pf_tagname { + TAILQ_ENTRY(pf_tagname) entries; + char name[PF_TAG_NAME_SIZE]; + uint16_t tag; + int ref; +}; +TAILQ_HEAD(pf_tags, pf_tagname); #define V_pf_tags VNET(pf_tags) VNET_DEFINE(struct pf_tags, pf_tags); #define V_pf_qids VNET(pf_qids) VNET_DEFINE(struct pf_tags, pf_qids); - +MALLOC_DEFINE(M_PFTAG, "pf tags", "pf tags"); #if (PF_QNAME_SIZE != PF_TAG_NAME_SIZE) #error PF_QNAME_SIZE must be equal to PF_TAG_NAME_SIZE @@ -160,7 +165,7 @@ VNET_DEFINE(struct pf_tags, pf_qids); static u_int16_t tagname2tag(struct pf_tags *, char *); static u_int16_t pf_tagname2tag(char *); -void tag_unref(struct pf_tags *, u_int16_t); +static void tag_unref(struct pf_tags *, u_int16_t); #define DPFPRINTF(n, x) if (V_pf_status.debug >= (n)) printf x @@ -406,8 +411,10 @@ pf_free_rule(struct pf_rule *rule) PF_RULES_WASSERT(); - pf_tag_unref(rule->tag); - pf_tag_unref(rule->match_tag); + if (rule->tag) + tag_unref(&V_pf_tags, rule->tag); + if (rule->match_tag) + tag_unref(&V_pf_tags, rule->match_tag); #ifdef ALTQ if (rule->pqid != rule->qid) pf_qid_unref(rule->pqid); @@ -444,6 +451,8 @@ tagname2tag(struct pf_tags *head, char * struct pf_tagname *tag, *p = NULL; u_int16_t new_tagid = 1; + PF_RULES_WASSERT(); + TAILQ_FOREACH(tag, head, entries) if (strcmp(tagname, tag->name) == 0) { tag->ref++; @@ -466,7 +475,7 @@ tagname2tag(struct pf_tags *head, char * return (0); /* allocate and fill new struct pf_tagname */ - tag = malloc(sizeof(*tag), M_TEMP, M_NOWAIT|M_ZERO); + tag = malloc(sizeof(*tag), M_PFTAG, M_NOWAIT|M_ZERO); if (tag == NULL) return (0); strlcpy(tag->name, tagname, sizeof(tag->name)); @@ -481,20 +490,19 @@ tagname2tag(struct pf_tags *head, char * return (tag->tag); } -void +static void tag_unref(struct pf_tags *head, u_int16_t tag) { struct pf_tagname *p, *next; - if (tag == 0) - return; + PF_RULES_WASSERT(); for (p = TAILQ_FIRST(head); p != NULL; p = next) { next = TAILQ_NEXT(p, entries); if (tag == p->tag) { if (--p->ref == 0) { TAILQ_REMOVE(head, p, entries); - free(p, M_TEMP); + free(p, M_PFTAG); } break; } @@ -507,24 +515,6 @@ pf_tagname2tag(char *tagname) return (tagname2tag(&V_pf_tags, tagname)); } -void -pf_tag_ref(u_int16_t tag) -{ - struct pf_tagname *t; - - TAILQ_FOREACH(t, &V_pf_tags, entries) - if (t->tag == tag) - break; - if (t != NULL) - t->ref++; -} - -void -pf_tag_unref(u_int16_t tag) -{ - tag_unref(&V_pf_tags, tag); -} - #ifdef ALTQ static u_int32_t pf_qname2qid(char *qname) Modified: projects/pf/head/sys/contrib/pf/net/pfvar.h ============================================================================== --- projects/pf/head/sys/contrib/pf/net/pfvar.h Fri May 25 14:40:56 2012 (r235992) +++ projects/pf/head/sys/contrib/pf/net/pfvar.h Fri May 25 14:49:51 2012 (r235993) @@ -1410,13 +1410,6 @@ struct pf_altq { u_int32_t qid; /* return value */ }; -struct pf_tagname { - TAILQ_ENTRY(pf_tagname) entries; - char name[PF_TAG_NAME_SIZE]; - u_int16_t tag; - int ref; -}; - struct pf_divert { union { struct in_addr ipv4; @@ -1931,8 +1924,6 @@ int pfi_clear_flags(const char *, int) int pf_match_tag(struct mbuf *, struct pf_rule *, int *, struct pf_mtag *); -void pf_tag_ref(u_int16_t); -void pf_tag_unref(u_int16_t); int pf_tag_packet(struct mbuf *, int, int, struct pf_mtag *); void pf_qid2qname(u_int32_t, char *); From owner-svn-src-projects@FreeBSD.ORG Fri May 25 15:49:22 2012 Return-Path: Delivered-To: svn-src-projects@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DDD4A1065672; Fri, 25 May 2012 15:49:21 +0000 (UTC) (envelope-from ermal.luci@gmail.com) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id 6FDBA8FC08; Fri, 25 May 2012 15:49:21 +0000 (UTC) Received: by yhgm50 with SMTP id m50so842587yhg.13 for ; Fri, 25 May 2012 08:49:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=QmmKIoG/8ipUoE08Kawa7feYe/hMU+zrkWgolPuP3to=; b=XFh9lVe7tVCKqFwCbn5Wj8kBqXkd+Nl1AJTJOzVVts+rwCDF55iJNWAOGgjR4ypFF/ FEnEyjtu+JUd3Qi3HyAxA+j+KNaXalvtF0EjPNzDZzYdhfuTWYH4L6hjEpm3VaLBDBRt YYYDwe/RmJeZa/tONVrJlmyl7eGUiB5WvTW521USNIELhruEtQDVlRriSq+dbQeLwskq i8ngUaVmwcuENQE7+xLUTXsaSpRklKfbvUqqjqsM7JKQ7qajADjgSUpJD46ICpi8zPM8 P5kAHaw2eMT5PAoJw8aZYLMhikgCJ9/NeNfLVo3or6U38qi/6X6j8bvBRznDO4uqHjzE 9SoA== MIME-Version: 1.0 Received: by 10.50.195.234 with SMTP id ih10mr3546216igc.0.1337960960137; Fri, 25 May 2012 08:49:20 -0700 (PDT) Sender: ermal.luci@gmail.com Received: by 10.231.35.202 with HTTP; Fri, 25 May 2012 08:49:20 -0700 (PDT) In-Reply-To: <201205251449.q4PEnpYd016118@svn.freebsd.org> References: <201205251449.q4PEnpYd016118@svn.freebsd.org> Date: Fri, 25 May 2012 17:49:20 +0200 X-Google-Sender-Auth: WHoLP9wPZ9y1cnLN6aAzqdUtsqQ Message-ID: From: =?ISO-8859-1?Q?Ermal_Lu=E7i?= To: Gleb Smirnoff Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: svn-src-projects@freebsd.org, src-committers@freebsd.org Subject: Re: svn commit: r235993 - projects/pf/head/sys/contrib/pf/net X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 May 2012 15:49:22 -0000 I am not sure you understand the implications here and previous commit. But word CONSISTENCY and wrong behviour is the first to look after. On Fri, May 25, 2012 at 4:49 PM, Gleb Smirnoff wrote: > Author: glebius > Date: Fri May 25 14:49:51 2012 > New Revision: 235993 > URL: http://svn.freebsd.org/changeset/base/235993 > > Log: > =A0- Stop referencing tag names by states. Not a big deal if tag name > =A0 =A0disappears. > =A0- Make tag code more private to pf_ioctl.c > > Modified: > =A0projects/pf/head/sys/contrib/pf/net/pf.c > =A0projects/pf/head/sys/contrib/pf/net/pf_ioctl.c > =A0projects/pf/head/sys/contrib/pf/net/pfvar.h > > Modified: projects/pf/head/sys/contrib/pf/net/pf.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D > --- projects/pf/head/sys/contrib/pf/net/pf.c =A0 =A0Fri May 25 14:40:56 2= 012 =A0 =A0 =A0 =A0(r235992) > +++ projects/pf/head/sys/contrib/pf/net/pf.c =A0 =A0Fri May 25 14:49:51 2= 012 =A0 =A0 =A0 =A0(r235993) > @@ -1472,8 +1472,6 @@ pf_free_state(struct pf_state *cur) > =A0 =A0 =A0 =A0if (cur->anchor.ptr !=3D NULL) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0--cur->anchor.ptr->states_cur; > =A0 =A0 =A0 =A0pf_normalize_tcp_cleanup(cur); > - =A0 =A0 =A0 if (cur->tag) > - =A0 =A0 =A0 =A0 =A0 =A0 =A0 pf_tag_unref(cur->tag); > =A0 =A0 =A0 =A0uma_zfree(V_pf_state_z, cur); > =A0 =A0 =A0 =A0V_pf_status.fcounters[FCNT_STATE_REMOVALS]++; > =A0 =A0 =A0 =A0V_pf_status.states--; > @@ -3446,10 +3444,8 @@ pf_create_state(struct pf_rule *r, struc > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0*sm =3D s; > > =A0 =A0 =A0 =A0pf_set_rt_ifp(s, pd->src); =A0 =A0 =A0/* needs s->state_ke= y set */ > - =A0 =A0 =A0 if (tag > 0) { > - =A0 =A0 =A0 =A0 =A0 =A0 =A0 pf_tag_ref(tag); > + =A0 =A0 =A0 if (tag > 0) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0s->tag =3D tag; > - =A0 =A0 =A0 } > =A0 =A0 =A0 =A0if (pd->proto =3D=3D IPPROTO_TCP && (th->th_flags & (TH_SY= N|TH_ACK)) =3D=3D > =A0 =A0 =A0 =A0 =A0 =A0TH_SYN && r->keep_state =3D=3D PF_STATE_SYNPROXY) = { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0s->src.state =3D PF_TCPS_PROXY_SRC; > > Modified: projects/pf/head/sys/contrib/pf/net/pf_ioctl.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D > --- projects/pf/head/sys/contrib/pf/net/pf_ioctl.c =A0 =A0 =A0Fri May 25 = 14:40:56 2012 =A0 =A0 =A0 =A0(r235992) > +++ projects/pf/head/sys/contrib/pf/net/pf_ioctl.c =A0 =A0 =A0Fri May 25 = 14:49:51 2012 =A0 =A0 =A0 =A0(r235993) > @@ -135,8 +135,6 @@ static int =A0 =A0 =A0 =A0 =A0 pf_addr_setup(struct p= f_rul > =A0static void =A0 =A0 =A0 =A0 =A0 =A0 pf_addr_copyout(struct pf_addr_wra= p *); > =A0static void =A0 =A0 =A0 =A0 =A0 =A0 pf_pkt_addr_changed(struct mbuf *)= ; > > -#define =A0 =A0 =A0 =A0TAGID_MAX =A0 =A0 =A0 =A050000 > - > =A0VNET_DEFINE(struct pf_rule, =A0 =A0pf_default_rule); > =A0VNET_DEFINE(struct sx, =A0 =A0 =A0 =A0 pf_consistency_lock); > =A0#define V_pf_consistency_lock =A0VNET(pf_consistency_lock) > @@ -146,13 +144,20 @@ static VNET_DEFINE(int, =A0 =A0 =A0 =A0 =A0 pf_altq= _running > =A0#define =A0 =A0 =A0 =A0V_pf_altq_running =A0 =A0 =A0 VNET(pf_altq_runn= ing) > =A0#endif > > -TAILQ_HEAD(pf_tags, pf_tagname); > +#define =A0 =A0 =A0 =A0TAGID_MAX =A0 =A0 =A0 =A050000 > +struct pf_tagname { > + =A0 =A0 =A0 TAILQ_ENTRY(pf_tagname) entries; > + =A0 =A0 =A0 char =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0name[PF_TAG_NAM= E_SIZE]; > + =A0 =A0 =A0 uint16_t =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0tag; > + =A0 =A0 =A0 int =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 ref; > +}; > > +TAILQ_HEAD(pf_tags, pf_tagname); > =A0#define =A0 =A0 =A0 =A0V_pf_tags =A0 =A0 =A0 =A0 =A0 =A0 =A0 VNET(pf_t= ags) > =A0VNET_DEFINE(struct pf_tags, pf_tags); > =A0#define =A0 =A0 =A0 =A0V_pf_qids =A0 =A0 =A0 =A0 =A0 =A0 =A0 VNET(pf_q= ids) > =A0VNET_DEFINE(struct pf_tags, pf_qids); > - > +MALLOC_DEFINE(M_PFTAG, "pf tags", "pf tags"); > > =A0#if (PF_QNAME_SIZE !=3D PF_TAG_NAME_SIZE) > =A0#error PF_QNAME_SIZE must be equal to PF_TAG_NAME_SIZE > @@ -160,7 +165,7 @@ VNET_DEFINE(struct pf_tags, pf_qids); > > =A0static u_int16_t =A0 =A0 =A0 =A0tagname2tag(struct pf_tags *, char *); > =A0static u_int16_t =A0 =A0 =A0 =A0pf_tagname2tag(char *); > -void =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0tag_unref(struct pf_tags *, = u_int16_t); > +static void =A0 =A0 =A0 =A0 =A0 =A0 tag_unref(struct pf_tags *, u_int16_= t); > > =A0#define DPFPRINTF(n, x) if (V_pf_status.debug >=3D (n)) printf x > > @@ -406,8 +411,10 @@ pf_free_rule(struct pf_rule *rule) > > =A0 =A0 =A0 =A0PF_RULES_WASSERT(); > > - =A0 =A0 =A0 pf_tag_unref(rule->tag); > - =A0 =A0 =A0 pf_tag_unref(rule->match_tag); > + =A0 =A0 =A0 if (rule->tag) > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 tag_unref(&V_pf_tags, rule->tag); > + =A0 =A0 =A0 if (rule->match_tag) > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 tag_unref(&V_pf_tags, rule->match_tag); > =A0#ifdef ALTQ > =A0 =A0 =A0 =A0if (rule->pqid !=3D rule->qid) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0pf_qid_unref(rule->pqid); > @@ -444,6 +451,8 @@ tagname2tag(struct pf_tags *head, char * > =A0 =A0 =A0 =A0struct pf_tagname =A0 =A0 =A0 *tag, *p =3D NULL; > =A0 =A0 =A0 =A0u_int16_t =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0new_tagid =3D 1; > > + =A0 =A0 =A0 PF_RULES_WASSERT(); > + > =A0 =A0 =A0 =A0TAILQ_FOREACH(tag, head, entries) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if (strcmp(tagname, tag->name) =3D=3D 0) { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0tag->ref++; > @@ -466,7 +475,7 @@ tagname2tag(struct pf_tags *head, char * > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0return (0); > > =A0 =A0 =A0 =A0/* allocate and fill new struct pf_tagname */ > - =A0 =A0 =A0 tag =3D malloc(sizeof(*tag), M_TEMP, M_NOWAIT|M_ZERO); > + =A0 =A0 =A0 tag =3D malloc(sizeof(*tag), M_PFTAG, M_NOWAIT|M_ZERO); > =A0 =A0 =A0 =A0if (tag =3D=3D NULL) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0return (0); > =A0 =A0 =A0 =A0strlcpy(tag->name, tagname, sizeof(tag->name)); > @@ -481,20 +490,19 @@ tagname2tag(struct pf_tags *head, char * > =A0 =A0 =A0 =A0return (tag->tag); > =A0} > > -void > +static void > =A0tag_unref(struct pf_tags *head, u_int16_t tag) > =A0{ > =A0 =A0 =A0 =A0struct pf_tagname =A0 =A0 =A0 *p, *next; > > - =A0 =A0 =A0 if (tag =3D=3D 0) > - =A0 =A0 =A0 =A0 =A0 =A0 =A0 return; > + =A0 =A0 =A0 PF_RULES_WASSERT(); > > =A0 =A0 =A0 =A0for (p =3D TAILQ_FIRST(head); p !=3D NULL; p =3D next) { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0next =3D TAILQ_NEXT(p, entries); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if (tag =3D=3D p->tag) { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if (--p->ref =3D=3D 0) { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0TAILQ_REMO= VE(head, p, entries); > - =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 free(p, M_T= EMP); > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 free(p, M_P= FTAG); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0} > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0break; > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0} > @@ -507,24 +515,6 @@ pf_tagname2tag(char *tagname) > =A0 =A0 =A0 =A0return (tagname2tag(&V_pf_tags, tagname)); > =A0} > > -void > -pf_tag_ref(u_int16_t tag) > -{ > - =A0 =A0 =A0 struct pf_tagname *t; > - > - =A0 =A0 =A0 TAILQ_FOREACH(t, &V_pf_tags, entries) > - =A0 =A0 =A0 =A0 =A0 =A0 =A0 if (t->tag =3D=3D tag) > - =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 break; > - =A0 =A0 =A0 if (t !=3D NULL) > - =A0 =A0 =A0 =A0 =A0 =A0 =A0 t->ref++; > -} > - > -void > -pf_tag_unref(u_int16_t tag) > -{ > - =A0 =A0 =A0 tag_unref(&V_pf_tags, tag); > -} > - > =A0#ifdef ALTQ > =A0static u_int32_t > =A0pf_qname2qid(char *qname) > > Modified: projects/pf/head/sys/contrib/pf/net/pfvar.h > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D > --- projects/pf/head/sys/contrib/pf/net/pfvar.h Fri May 25 14:40:56 2012 = =A0 =A0 =A0 =A0(r235992) > +++ projects/pf/head/sys/contrib/pf/net/pfvar.h Fri May 25 14:49:51 2012 = =A0 =A0 =A0 =A0(r235993) > @@ -1410,13 +1410,6 @@ struct pf_altq { > =A0 =A0 =A0 =A0u_int32_t =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0qid; =A0 =A0 =A0 = =A0 =A0 /* return value */ > =A0}; > > -struct pf_tagname { > - =A0 =A0 =A0 TAILQ_ENTRY(pf_tagname) entries; > - =A0 =A0 =A0 char =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0name[PF_TAG_NAM= E_SIZE]; > - =A0 =A0 =A0 u_int16_t =A0 =A0 =A0 =A0 =A0 =A0 =A0 tag; > - =A0 =A0 =A0 int =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 ref; > -}; > - > =A0struct pf_divert { > =A0 =A0 =A0 =A0union { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0struct in_addr =A0ipv4; > @@ -1931,8 +1924,6 @@ int =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0pfi_clear_flags(= const char *, int) > > =A0int =A0 =A0 =A0 =A0 =A0 =A0 pf_match_tag(struct mbuf *, struct pf_rule= *, int *, > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0struct pf_mtag *); > -void =A0 =A0 =A0 =A0 =A0 =A0pf_tag_ref(u_int16_t); > -void =A0 =A0 =A0 =A0 =A0 =A0pf_tag_unref(u_int16_t); > =A0int =A0 =A0 =A0 =A0 =A0 =A0 pf_tag_packet(struct mbuf *, int, int, str= uct pf_mtag *); > =A0void =A0 =A0 =A0 =A0 =A0 =A0pf_qid2qname(u_int32_t, char *); > --=20 Ermal From owner-svn-src-projects@FreeBSD.ORG Fri May 25 23:41:22 2012 Return-Path: Delivered-To: svn-src-projects@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id DA9AF106564A; Fri, 25 May 2012 23:41:22 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id C6C018FC0A; Fri, 25 May 2012 23:41:22 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.4/8.14.4) with ESMTP id q4PNfMUv039935; Fri, 25 May 2012 23:41:22 GMT (envelope-from linimon@svn.freebsd.org) Received: (from linimon@localhost) by svn.freebsd.org (8.14.4/8.14.4/Submit) id q4PNfM9i039933; Fri, 25 May 2012 23:41:22 GMT (envelope-from linimon@svn.freebsd.org) Message-Id: <201205252341.q4PNfM9i039933@svn.freebsd.org> From: Mark Linimon Date: Fri, 25 May 2012 23:41:22 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org X-SVN-Group: projects MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r236031 - projects/portbuild/scripts X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 May 2012 23:41:22 -0000 Author: linimon (doc,ports committer) Date: Fri May 25 23:41:22 2012 New Revision: 236031 URL: http://svn.freebsd.org/changeset/base/236031 Log: Fix test for "incomplete build". Spotted by: jilles Modified: projects/portbuild/scripts/processonelog Modified: projects/portbuild/scripts/processonelog ============================================================================== --- projects/portbuild/scripts/processonelog Fri May 25 23:24:16 2012 (r236030) +++ projects/portbuild/scripts/processonelog Fri May 25 23:41:22 2012 (r236031) @@ -178,7 +178,7 @@ elif bzgrep -q "Segmentation fault" $1; reason="segfault"; tag="segfault" # note: searching for string-not-found here (builds that terminated early) -elif [ ! bzgrep -qE "^build of .* ended at" $1 ]; then +elif ! bzgrep -qE "^build of .* ended at" $1; then reason="cluster"; tag="cluster" else