From owner-freebsd-current@freebsd.org Sun Jun 9 06:13:02 2019 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 856AC15BEB5E for ; Sun, 9 Jun 2019 06:13:02 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-QB1-obe.outbound.protection.outlook.com (mail-eopbgr660061.outbound.protection.outlook.com [40.107.66.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "GlobalSign Organization Validation CA - SHA256 - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F12EF8B03E for ; Sun, 9 Jun 2019 06:13:01 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from YQXPR01MB3128.CANPRD01.PROD.OUTLOOK.COM (52.132.93.160) by YQXPR01MB3879.CANPRD01.PROD.OUTLOOK.COM (52.132.94.18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1965.12; Sun, 9 Jun 2019 06:12:59 +0000 Received: from YQXPR01MB3128.CANPRD01.PROD.OUTLOOK.COM ([fe80::f9fe:559f:fdc:9e5a]) by YQXPR01MB3128.CANPRD01.PROD.OUTLOOK.COM ([fe80::f9fe:559f:fdc:9e5a%3]) with mapi id 15.20.1965.017; Sun, 9 Jun 2019 06:12:59 +0000 From: Rick Macklem To: Konstantin Belousov CC: "freebsd-current@FreeBSD.org" Subject: Re: adding a syscall to libc? Thread-Topic: adding a syscall to libc? Thread-Index: AQHVHaUq4CSrPmO6/E+nutIFTmCjBKaRjnkAgAFG004= Date: Sun, 9 Jun 2019 06:12:59 +0000 Message-ID: References: , <20190608102816.GR75280@kib.kiev.ua> In-Reply-To: <20190608102816.GR75280@kib.kiev.ua> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 5097e902-840c-4726-82c1-08d6eca18596 x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(5600148)(711020)(4605104)(1401327)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7193020); SRVR:YQXPR01MB3879; x-ms-traffictypediagnostic: YQXPR01MB3879: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-forefront-prvs: 006339698F x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(136003)(376002)(346002)(366004)(39850400004)(396003)(199004)(189003)(186003)(9686003)(55016002)(6916009)(14454004)(99286004)(46003)(305945005)(2906002)(71190400001)(6436002)(53936002)(66476007)(476003)(11346002)(66946007)(73956011)(86362001)(66556008)(66446008)(786003)(486006)(64756008)(76116006)(446003)(316002)(229853002)(74316002)(8936002)(6506007)(52536014)(74482002)(478600001)(8676002)(102836004)(81166006)(4326008)(33656002)(81156014)(71200400001)(14444005)(256004)(68736007)(25786009)(6246003)(1411001)(7696005)(5660300002)(76176011)(21314003); DIR:OUT; SFP:1101; SCL:1; SRVR:YQXPR01MB3879; H:YQXPR01MB3128.CANPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: uoguelph.ca does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: q4p5PPVsQmdVUEccufz+YKiwwaoyGMyOcbsAItRt4NMxcwzNyxVncln7tmzXhXKxFY6+4jX9d8km8VIBrpvQTeafor+JOTbgb9uYgkLIhm++Kf2A1NtPdCaqVGGQQzwfyTOJUiUbQP25+hXLiTzfUSLBQIEB+G+c0EN01nMylErZdy++86ubtM281Ll7CPA6JgRFXSX7/hpyCrRU1S1CZbMczI9r9CmpGxv+086jx1PjXv01LqVTuMOrKPqrhBtPfshlZo9dzW6+TkT62NJMeLMVnmyWYSBktkIXDdhQhTzhwVzvQh5TTXkUB79FjFspQcqhMhbSbPdzWMFOWYWsrMPeMR2RvcxYGMjuui1sksGzAP8m9VtFwThCUhaEHvqJa3mqWuMpMlQSsl1SkBXjebzCasJSMx8qGLdqpGxUfkU= Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-Network-Message-Id: 5097e902-840c-4726-82c1-08d6eca18596 X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Jun 2019 06:12:59.3377 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: rmacklem@uoguelph.ca X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQXPR01MB3879 X-Rspamd-Queue-Id: F12EF8B03E X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.97 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[]; NEURAL_HAM_SHORT(-0.97)[-0.974,0] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Jun 2019 06:13:02 -0000 Konstantin Belousov wrote: >On Sat, Jun 08, 2019 at 02:57:27AM +0000, Rick Macklem wrote: >> Hi, >> First off, thanks Kostik for the fine explanation. I agree with Oliver that= it should be captured somewhere like the wiki. I'm no wiki guy, so hopefully someone = else will do this? >> I've started working of a copy_file_range() syscall for FreeBSD. I think= I have the >> kernel patched and ready for some testing. >> However, I'm confused about what I need to do in src/lib/libc/sys? >> - Some syscalls have little .c files, but other ones do not. >> When is one of these little .c files needed and, when not needed, what= else >> needs to be done? (I notice that syscall.mk in src/sys/sys automagical= ly, but >> I can't see what else, if anything, needs to be done?) >Most important is to add the new syscall public symbol to sys/Symbol.map >into the correct version, FBSD_1.6 for CURRENT-13. Do no bother with >__sys_XXX and __XXX aliases. I could only find a Symbol.map in src/lib/libc/sys. I added it there and it= seems to work. (I am using a stable/12 source tree for testing the build/userland. I= 'll check head in case it has moved.) >'Tiny .c files' are typically used for one of two purposes: >- Convert raw kernel interface into something expected by userspace, > often this coversion uses more generic and non-standard interface to > implement more usual function. Examples are open(2) or waitid(2) > which are really tiny wrappers around openat(2) and wait6(2) in > today libc. >- Allow libthr to hook into libc to provide additional services. Libthr > often has to modify semantic of raw syscall, and libc contains the > tables redirecting to implementation, the tables are patched on libthr > load. Since tables must fill entries with some address in case libthr > is not loaded, tiny functions which wrap syscalls are created for > use in that tables. > >I think you do not need anything that complications for start, in which >case adding new syscall consists of the following steps: Yes, I don't think I need the above. >- Add the syscall to sys/kern/syscalls.master, and if reasonable, > to sys/compat/freebsd32/syscalls.master. I don't think a 32bit binary on a 64bit system needs this for now. (At least that's my understanding of what this is used for?) >- Consider if the syscall makes sense in capsicumized environment, > and if yes, list the syscall in sys/kern/capabilities.conf. Typically, > if syscall provides access to the global files namespace, it must be not > allowed. On the other hand, if syscall only operates on already opened > file descriptors, then it is suitable (but of course there are lot of > nuances). It uses open fds, but I think I'll leave it out of capabilities.conf for no= w. If there is a need, someone more familiar with capsicum can check it. >- Add syscall prototype to the user-visible portion of header, > hiding it under the proper visibility check. Hmm, not quite sure what you mean here. It ends up in sys/sysproto.h automagically. Does it need to go somewhere else too? >- Add syscall symbol to lib/libc/sys/Symbol.ver. All I found was lib/libc/sys/Symbol.map and I've added it there. >- Implement the syscall. There are some additional details that might > require attention: > - If compat32 syscall going to be implemented, or you know > that Linuxolator needs to implement same syscall and would > like to take advantage of the code, provide > int kern_YOURSYSCALL(); > wrapper and declare it in sys/syscallsubr.h. Real implementatio= ns > of host-native and compat32 sys_YOURSYSCALL() should be just > decoding of uap members and call into kern_YOURSYSCALL. I think it might be useful for the Linuxolator, since it is meant to be Lin= ux compatible, so I've done this. > - Consider the need to add auditing for new syscall. This one I need to look at more closely. I may end up posting to the list w.r.t. what to do about this. I think I'll leave it out of the first draft = for phabricator. >- Add man page for the syscall, at lib/libc/sys/YOURSYSCALL.2, and connect > it to the build in lib/libc/sys/Makefile.inc. Yea, I know I have to write a man page. Maybe get to that tomorrow. >- When creating review for the change, do not include diff for generated > files after make sysent. Similarly, when doing the commit, first commit > everything non-generated, then do make -C sys/kern sysent (and > make sysent -C sys/compat/freebsd32 sysent if appropriate) and commit > the generated files in follow-up. Righto, I'll do this when it gets to that stage. Thanks again for the useful answer, rick From owner-freebsd-current@freebsd.org Sun Jun 9 13:17:33 2019 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1161915C736C for ; Sun, 9 Jun 2019 13:17:33 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DC1716FD0B for ; Sun, 9 Jun 2019 13:17:31 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id x59DHM1b083163 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Sun, 9 Jun 2019 16:17:26 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua x59DHM1b083163 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id x59DHM93083157; Sun, 9 Jun 2019 16:17:22 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 9 Jun 2019 16:17:22 +0300 From: Konstantin Belousov To: Rick Macklem Cc: "freebsd-current@FreeBSD.org" Subject: Re: adding a syscall to libc? Message-ID: <20190609131722.GV75280@kib.kiev.ua> References: <20190608102816.GR75280@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.12.0 (2019-05-25) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on tom.home X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Jun 2019 13:17:33 -0000 On Sun, Jun 09, 2019 at 06:12:59AM +0000, Rick Macklem wrote: > Konstantin Belousov wrote: > >On Sat, Jun 08, 2019 at 02:57:27AM +0000, Rick Macklem wrote: > >> Hi, > >> > First off, thanks Kostik for the fine explanation. I agree with Oliver that it should > be captured somewhere like the wiki. I'm no wiki guy, so hopefully someone else > will do this? There is apparently https://wiki.freebsd.org/AddingSyscalls which is already quite comprehensive. I am adding missed information right now. > > >> I've started working of a copy_file_range() syscall for FreeBSD. I think I have the > >> kernel patched and ready for some testing. > >> However, I'm confused about what I need to do in src/lib/libc/sys? > >> - Some syscalls have little .c files, but other ones do not. > >> When is one of these little .c files needed and, when not needed, what else > >> needs to be done? (I notice that syscall.mk in src/sys/sys automagically, but > >> I can't see what else, if anything, needs to be done?) > >Most important is to add the new syscall public symbol to sys/Symbol.map > >into the correct version, FBSD_1.6 for CURRENT-13. Do no bother with > >__sys_XXX and __XXX aliases. > I could only find a Symbol.map in src/lib/libc/sys. I added it there and it seems to > work. (I am using a stable/12 source tree for testing the build/userland. I'll check > head in case it has moved.) Yes, Symbol.map, it was thinko. > > >'Tiny .c files' are typically used for one of two purposes: > >- Convert raw kernel interface into something expected by userspace, > > often this coversion uses more generic and non-standard interface to > > implement more usual function. Examples are open(2) or waitid(2) > > which are really tiny wrappers around openat(2) and wait6(2) in > > today libc. > >- Allow libthr to hook into libc to provide additional services. Libthr > > often has to modify semantic of raw syscall, and libc contains the > > tables redirecting to implementation, the tables are patched on libthr > > load. Since tables must fill entries with some address in case libthr > > is not loaded, tiny functions which wrap syscalls are created for > > use in that tables. > > > >I think you do not need anything that complications for start, in which > >case adding new syscall consists of the following steps: > Yes, I don't think I need the above. > > >- Add the syscall to sys/kern/syscalls.master, and if reasonable, > > to sys/compat/freebsd32/syscalls.master. > I don't think a 32bit binary on a 64bit system needs this for now. > (At least that's my understanding of what this is used for?) Right. > > >- Consider if the syscall makes sense in capsicumized environment, > > and if yes, list the syscall in sys/kern/capabilities.conf. Typically, > > if syscall provides access to the global files namespace, it must be not > > allowed. On the other hand, if syscall only operates on already opened > > file descriptors, then it is suitable (but of course there are lot of > > nuances). > It uses open fds, but I think I'll leave it out of capabilities.conf for now. If > there is a need, someone more familiar with capsicum can check it. > > >- Add syscall prototype to the user-visible portion of header, > > hiding it under the proper visibility check. > Hmm, not quite sure what you mean here. It ends up in sys/sysproto.h > automagically. Does it need to go somewhere else too? sys/sysproto.h is not used by userspace. If your syscall is going to be useful for normal userspace (it should be, othewise why adding it at all ?) you have to declare it in some header typically used by apps. Take for example include/unistd.h, where you should only add the syscall to appropriate namespace, allowing strict standard-compliance compiler option to hide symbols not specified by choosen standard. I suspect that you need #if __BSD_VISIBLE int your_syscall(args); #endif if the header is defined e.g. by POSIX. > > >- Add syscall symbol to lib/libc/sys/Symbol.ver. > All I found was lib/libc/sys/Symbol.map and I've added it there. > > >- Implement the syscall. There are some additional details that might > > require attention: > > - If compat32 syscall going to be implemented, or you know > > that Linuxolator needs to implement same syscall and would > > like to take advantage of the code, provide > > int kern_YOURSYSCALL(); > > wrapper and declare it in sys/syscallsubr.h. Real implementations > > of host-native and compat32 sys_YOURSYSCALL() should be just > > decoding of uap members and call into kern_YOURSYSCALL. > I think it might be useful for the Linuxolator, since it is meant to be Linux > compatible, so I've done this. > > > - Consider the need to add auditing for new syscall. > This one I need to look at more closely. I may end up posting to the list > w.r.t. what to do about this. I think I'll leave it out of the first draft for phabricator. > > >- Add man page for the syscall, at lib/libc/sys/YOURSYSCALL.2, and connect > > it to the build in lib/libc/sys/Makefile.inc. > Yea, I know I have to write a man page. Maybe get to that tomorrow. > > >- When creating review for the change, do not include diff for generated > > files after make sysent. Similarly, when doing the commit, first commit > > everything non-generated, then do make -C sys/kern sysent (and > > make sysent -C sys/compat/freebsd32 sysent if appropriate) and commit > > the generated files in follow-up. > Righto, I'll do this when it gets to that stage. > > Thanks again for the useful answer, rick From owner-freebsd-current@freebsd.org Mon Jun 10 01:32:43 2019 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5505D15B04B5 for ; Mon, 10 Jun 2019 01:32:43 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-QB1-obe.outbound.protection.outlook.com (mail-eopbgr660046.outbound.protection.outlook.com [40.107.66.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "GlobalSign Organization Validation CA - SHA256 - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0D7FA8D38E for ; Mon, 10 Jun 2019 01:32:41 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from YQXPR01MB3128.CANPRD01.PROD.OUTLOOK.COM (52.132.93.160) by YQXPR01MB3048.CANPRD01.PROD.OUTLOOK.COM (52.132.93.21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1965.14; Mon, 10 Jun 2019 01:32:40 +0000 Received: from YQXPR01MB3128.CANPRD01.PROD.OUTLOOK.COM ([fe80::f9fe:559f:fdc:9e5a]) by YQXPR01MB3128.CANPRD01.PROD.OUTLOOK.COM ([fe80::f9fe:559f:fdc:9e5a%3]) with mapi id 15.20.1965.017; Mon, 10 Jun 2019 01:32:40 +0000 From: Rick Macklem To: "freebsd-current@FreeBSD.org" Subject: patch to add a Linux compatible copy_file_range(2) syscall Thread-Topic: patch to add a Linux compatible copy_file_range(2) syscall Thread-Index: AQHVHyu9o5p1TII5CEeKHWgU+iUSsQ== Date: Mon, 10 Jun 2019 01:32:40 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 8dcc6099-10ba-4839-3f58-08d6ed4386fe x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(711020)(4605104)(1401327)(2017052603328)(7193020); SRVR:YQXPR01MB3048; x-ms-traffictypediagnostic: YQXPR01MB3048: x-ms-exchange-purlcount: 1 x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-forefront-prvs: 0064B3273C x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(376002)(346002)(136003)(396003)(366004)(39850400004)(189003)(199004)(186003)(46003)(478600001)(476003)(55016002)(5640700003)(486006)(6436002)(305945005)(66446008)(64756008)(14454004)(966005)(76116006)(66556008)(66476007)(316002)(786003)(66946007)(53936002)(74482002)(2351001)(73956011)(25786009)(33656002)(2906002)(2501003)(68736007)(8676002)(7696005)(8936002)(81166006)(6506007)(81156014)(102836004)(74316002)(9686003)(99286004)(52536014)(6916009)(5660300002)(256004)(86362001)(71190400001)(71200400001)(6306002)(4744005); DIR:OUT; SFP:1101; SCL:1; SRVR:YQXPR01MB3048; H:YQXPR01MB3128.CANPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: uoguelph.ca does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: KafOLWQH3tDt6sjWgpBBLigpqO2OkUF9zFCzAJQXvj60F1wXM107YAGKdYs5vsJEaURk7dmBXbARk1+HlUowq4tSjoZEqHyILkVGUlq+aN4I0zXjahkU2xQ0wC7KGZ8P1+/QU4e5F757712Cmb8CwN4kATWSA1hsEGX1VMQmGXQGWlaZo7fog/Gct6n0N88nrJF0Gxa91fNa0oonwXYlzrQjsLlIQLJUpL3E7Rxzyde4o6KB0egj92m4Zf9Bp3DqZ9272iJxzPHxfnKWUPKZ3puCc3oXjcTlWDsjNQUHuIoHROJ5iAdN1tRilEwE3BFSvLPaLoYCg/OvnBNps0BgqOwdpz7GEfB+XUn01ELO7tQ2/ontTOQy3P/WYDd6jTrBd6NGlFWb7t0XBzE4QkclF27qUVXUuEu5rYggQK1rrSo= Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-Network-Message-Id: 8dcc6099-10ba-4839-3f58-08d6ed4386fe X-MS-Exchange-CrossTenant-originalarrivaltime: 10 Jun 2019 01:32:40.2245 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: rmacklem@uoguelph.ca X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQXPR01MB3048 X-Rspamd-Queue-Id: 0D7FA8D38E X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of rmacklem@uoguelph.ca designates 40.107.66.46 as permitted sender) smtp.mailfrom=rmacklem@uoguelph.ca X-Spamd-Result: default: False [-2.60 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.97)[-0.967,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:40.107.0.0/16]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[uoguelph.ca]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[3]; MX_GOOD(-0.01)[mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com]; NEURAL_HAM_SHORT(-0.32)[-0.325,0]; RCVD_IN_DNSWL_NONE(0.00)[46.66.107.40.list.dnswl.org : 127.0.3.0]; TO_DN_EQ_ADDR_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Jun 2019 01:32:43 -0000 Hi, I just put a patch in phabricator that is intended to add a Linux compatibl= e copy_file_range(2) syscall. My main interest in having this is that NFSv4.2= will know how to do file copying locally on the NFS server, saving all the reads= /writes across the wire. It copies the file byte range in the kernel. I don't know how the performan= ce compares with a userland file copy done to a local file system on the machi= ne. (It would save syscalls, but I have no idea if that will result in a notice= able performance difference?) It is at https://reviews.freebsd.org/D20584 I've listed a few guys as possible reviewers, but if anyone else would like= to review it, feel free to add yourself. If anyone is able to test this, it would be appreciated and let me know how= it goes, rick. From owner-freebsd-current@freebsd.org Tue Jun 11 06:59:59 2019 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5ED0F15B03E8 for ; Tue, 11 Jun 2019 06:59:59 +0000 (UTC) (envelope-from guru@unixarea.de) Received: from smh-06.1blu.de (smh-06.1blu.de [178.254.0.206]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4E56876FB3 for ; Tue, 11 Jun 2019 06:59:57 +0000 (UTC) (envelope-from guru@unixarea.de) Received: from [172.16.29.5] (helo=sh4-5.1blu.de) by smh-06.1blu.de with esmtp (Exim 4.86_2) (envelope-from ) id 1haal1-00089M-27; Tue, 11 Jun 2019 08:59:47 +0200 Received: from ftp51246-2575596 by sh4-5.1blu.de with local (Exim 4.86_2) (envelope-from ) id 1haal0-0007Ts-Vt; Tue, 11 Jun 2019 08:59:46 +0200 Date: Tue, 11 Jun 2019 08:59:46 +0200 From: Matthias Apitz To: freebsd-current@freebsd.org Cc: kde-freebsd@kde.org Subject: building x11-wm/plasma5-kwin w/ poudriere results in out of swap space Message-ID: <20190611065946.GA25933@sh4-5.1blu.de> Reply-To: Matthias Apitz Mail-Followup-To: freebsd-current@freebsd.org, kde-freebsd@kde.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit X-Operating-System: FreeBSD 12.0-CURRENT r314251 (amd64) X-message-flag: Mails containing HTML will not be read! Please send only plain text. User-Agent: Mutt/1.5.24 (2015-08-30) X-Rspamd-Queue-Id: 4E56876FB3 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-3.18 / 15.00]; ARC_NA(0.00)[]; HAS_REPLYTO(0.00)[guru@unixarea.de]; REPLYTO_EQ_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[unixarea.de]; AUTH_NA(1.00)[]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: mail.unixarea.de]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.80)[-0.799,0]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:42730, ipnet:178.254.0.0/19, country:DE]; RCVD_TLS_LAST(0.00)[]; IP_SCORE(-1.27)[ipnet: 178.254.0.0/19(-3.59), asn: 42730(-2.77), country: DE(-0.00)] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Jun 2019 06:59:59 -0000 Hello, I'm build the latest port svn checkout (June 10) on r342378 with poudriere and (after isolating it) have the problem that building x11-wm/plasma5-kwin runs reproducible out of swap during 'configure' phase: [00:00:49] Cleaning the build queue [00:00:49] Sanity checking build queue [00:00:49] Processing PRIORITY_BOOST [00:00:50] Balancing pool [00:00:50] Recording filesystem state for prepkg... done [00:00:52] Building 1 packages using 1 builders [00:00:52] Starting/Cloning builders [00:00:54] Hit CTRL+t at any time to see build progress and stats [00:00:54] [01] [00:00:00] Building x11-wm/plasma5-kwin | plasma5-kwin-5.15.5 load: 0.95 cmd: sh 19376 [piperd] 67.68r 0.00u 0.00s 0% 3932k [freebsd-r342378-ports-20190610] [2019-06-11_07h49m10s] [parallel_build:] Queued: 1 Built: 0 Failed: 0 Skipped: 0 Ignored: 0 Tobuild: 1 Time: 00:01:40 [01]: x11-wm/plasma5-kwin | plasma5-kwin-5.15.5 build-depends (00:01:06 / 00:01:08) [00:02:02] Logs: /usr/local/poudriere/data/logs/bulk/freebsd-r342378-ports-20190610/2019-06-11_07h49m10s load: 0.52 cmd: wc 26535 [piperd] 0.10r 0.00u 0.00s 0% 2656k [freebsd-r342378-ports-20190610] [2019-06-11_07h49m10s] [parallel_build:] Queued: 1 Built: 0 Failed: 0 Skipped: 0 Ignored: 0 Tobuild: 1 Time: 00:03:27 [01]: x11-wm/plasma5-kwin | plasma5-kwin-5.15.5 configure (00:01:17 / 00:02:55) [00:03:49] Logs: /usr/local/poudriere/data/logs/bulk/freebsd-r342378-ports-20190610/2019-06-11_07h49m10s ^C[00:04:13] Error: Signal SIGINT caught, cleaning up and exiting Is this a known issue? Or should I investigate deeper which process it is? matthias -- Matthias Apitz, ✉ guru@unixarea.de, http://www.unixarea.de/ +49-176-38902045 Public GnuPG key: http://www.unixarea.de/key.pub May, 9: Спаси́бо освободители! Thank you very much, Russian liberators! ----- End forwarded message ----- -- Matthias Apitz, ✉ guru@unixarea.de, http://www.unixarea.de/ +49-176-38902045 Public GnuPG key: http://www.unixarea.de/key.pub May, 9: Спаси́бо освободители! Thank you very much, Russian liberators! From owner-freebsd-current@freebsd.org Tue Jun 11 15:52:55 2019 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1CC4915BF4A8; Tue, 11 Jun 2019 15:52:55 +0000 (UTC) (envelope-from jbwlists@hilltopgroup.com) Received: from equinox.hilltopgroup.com (equinox.hilltopgroup.com [204.109.63.175]) by mx1.freebsd.org (Postfix) with ESMTP id A60FF6B96E; Tue, 11 Jun 2019 15:52:53 +0000 (UTC) (envelope-from jbwlists@hilltopgroup.com) Received: from mail.relativity.hilltopgroup.com (unknown [104.185.205.155]) by equinox.hilltopgroup.com (Postfix) with ESMTP id C7ED637BDB8; Tue, 11 Jun 2019 11:52:43 -0400 (EDT) Received: from cloud.example.com (unknown [192.168.6.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: jbwlists@hilltopgroup.com) by mail.relativity.hilltopgroup.com (Postfix) with ESMTPSA id B958010DB2; Tue, 11 Jun 2019 11:52:43 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=hilltopgroup.com; s=mail; t=1560268363; bh=zzcZS9ZbSJBd1+P4ISnmMLcotCWn1K87OJ7Dn+5HrX8=; h=Date:From:To:Cc:Subject:In-Reply-To:References; b=DCNgM49SXWJnkSoGlzyfPp3OToWEqWpfe1TxEce/u7wKpUCINVFQCKbU/Gg1+OVrz 9SoctKeYOqQIubIItSiDBOo14cPd+4uWWh14h/lBtxCmEzxvc0MsrslCJAZIl1INPg 2B4Udo4RpKhuavoyzkjCgJCkhKvi8RJAWiVU5LLU= MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Tue, 11 Jun 2019 11:52:43 -0400 From: jbwlists@hilltopgroup.com To: Johannes Lundberg Cc: Baptiste Daroussin , freebsd-current , owner-freebsd-current@freebsd.org Subject: Re: pkg: Cannot open /dev/null:No such file or directory In-Reply-To: References: <20190604073209.4e42a0eb@freyja> <20190604054409.4anei2ljzimqc75m@ivaldir.net> Message-ID: <935a44aa7587cdc07fadc2e33caed1f7@hilltopgroup.com> X-Sender: jbwlists@hilltopgroup.com User-Agent: Roundcube Webmail/1.3.9 X-Rspamd-Queue-Id: A60FF6B96E X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=hilltopgroup.com header.s=mail header.b=DCNgM49S; spf=pass (mx1.freebsd.org: domain of jbwlists@hilltopgroup.com designates 204.109.63.175 as permitted sender) smtp.mailfrom=jbwlists@hilltopgroup.com X-Spamd-Result: default: False [-4.37 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[hilltopgroup.com:s=mail]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+a]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[hilltopgroup.com]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[hilltopgroup.com:+]; MX_GOOD(-0.01)[equinox.hilltopgroup.com,mail2.hilltopgroup.com,mail.hilltopgroup.com]; FROM_NO_DN(0.00)[]; NEURAL_HAM_SHORT(-0.84)[-0.843,0]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; IP_SCORE(-1.11)[ipnet: 204.109.60.0/22(-1.62), asn: 36236(-3.88), country: US(-0.06)]; ASN(0.00)[asn:36236, ipnet:204.109.60.0/22, country:US]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Jun 2019 15:52:55 -0000 I'm having the same issue with poudriere image; could you please let me know what you did to fix it? I'm assuming the image.sh you're referring to is /usr/local/share/poudriere/image.sh, but I'm not sure where the change would need to be made. Thanks in advance, Joseph On 2019-06-04 13:02, Johannes Lundberg wrote: > On 6/3/19 10:44 PM, Baptiste Daroussin wrote: >> On Tue, Jun 04, 2019 at 07:32:16AM +0200, O. Hartmann wrote: >>> Hello List, >>> >>> lately I ran into a serious problem installing packages in a nanoBSD >>> environment, in which the package repository server is "remotely" on >>> site. The >>> issue as documented below occurs on both 12-STABLE r348529 and >>> CURRENT r348600 >>> and must have been introduced shortly, since the last known good >>> installation >>> with the environment of ours was on 21st May 2019. >>> >>> As far as I know,, the package installation is performed via >>> "chroot'ed" >>> environment and somehow /dev/null is out of a sudden not accessible >>> anymore >>> while pkg tries to delegate some output to /dev/null. >>> >>> What happened here? >>> >>> Kind regards and thanks in advance, >>> >>> oh >>> >>> [...] >>> All repositories are up to date. >>> The following 10 package(s) will be affected (of 0 checked): >>> >>> New packages to be INSTALLED: >>> python3: 3_3 [zeit4] >>> sudo: 1.8.27_1 [zeit4] >>> devcpu-data: 1.22 [zeit4] >>> python36: 3.6.8_2 [zeit4] >>> readline: 8.0.0 [zeit4] >>> indexinfo: 0.3.1 [zeit4] >>> libffi: 3.2.1_3 [zeit4] >>> gettext-runtime: 0.19.8.1_2 [zeit4] >>> openldap-sasl-client: 2.4.47 [zeit4] >>> cyrus-sasl: 2.1.27 [zeit4] >>> >>> Number of packages to be installed: 10 >>> >> What is new is that pkg is using /dev/null as input when running >> script? this is >> new since pkg 1.11 . Somehow this does not seems to be avaalaible in >> your >> environement. > > Hi > > Same things applies to poudriere-image. I had to add a mount devfs > command to the image.sh script. > >> >> Best regards, >> Bapt > > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to > "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-current@freebsd.org Tue Jun 11 18:52:35 2019 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F329E15C392B for ; Tue, 11 Jun 2019 18:52:34 +0000 (UTC) (envelope-from johalun@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 972377295F; Tue, 11 Jun 2019 18:52:34 +0000 (UTC) (envelope-from johalun@FreeBSD.org) Received: from [10.46.14.95] (wsip-72-212-151-146.ph.ph.cox.net [72.212.151.146]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: johalun) by smtp.freebsd.org (Postfix) with ESMTPSA id 147224DA4; Tue, 11 Jun 2019 18:52:33 +0000 (UTC) (envelope-from johalun@FreeBSD.org) Subject: Re: pkg: Cannot open /dev/null:No such file or directory To: jbwlists@hilltopgroup.com, Baptiste Daroussin Cc: freebsd-current References: <20190604073209.4e42a0eb@freyja> <20190604054409.4anei2ljzimqc75m@ivaldir.net> <935a44aa7587cdc07fadc2e33caed1f7@hilltopgroup.com> From: Johannes Lundberg X-Tagtoolbar-Keys: D20190611115231840 Message-ID: Date: Tue, 11 Jun 2019 11:52:31 -0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.7.0 MIME-Version: 1.0 In-Reply-To: <935a44aa7587cdc07fadc2e33caed1f7@hilltopgroup.com> Content-Type: multipart/mixed; boundary="------------F545AD8270A6D91ADDA344E0" Content-Language: en-US X-Rspamd-Queue-Id: 972377295F X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.95 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.996,0]; NEURAL_HAM_SHORT(-0.95)[-0.954,0]; ASN(0.00)[asn:11403, ipnet:96.47.64.0/20, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Jun 2019 18:52:35 -0000 This is a multi-part message in MIME format. --------------F545AD8270A6D91ADDA344E0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Hi This is probably overkill but I've attached a diff that show my patches to image.sh. It's just a hack so far to make it do what I want and not meant as general purpose. Use the changes you need for your application. Changes (only meant to work for building usb images on amd64 and i386) - mount devfs so pkg will work properly - add "install packages from official repo" option so you won't have to build your own packages for each jail - downloaded packages from official repo is cached locally to avoid excessive downloads - hack for i386 so packages are installed properly - increase swap space to allow for kernel core dumps (for usb image) - execute post-install script "overlay.sh" in the jail if provided in the overlay folder Bapt: Maybe you'll find some of these changes useful? :) /Johannes On 6/11/19 8:52 AM, jbwlists@hilltopgroup.com wrote: > I'm having the same issue with poudriere image; could you please let > me know what you did to fix it?  I'm assuming the image.sh you're > referring to is /usr/local/share/poudriere/image.sh, but I'm not sure > where the change would need to be made. > > Thanks in advance, > Joseph > > > On 2019-06-04 13:02, Johannes Lundberg wrote: >> On 6/3/19 10:44 PM, Baptiste Daroussin wrote: >>> On Tue, Jun 04, 2019 at 07:32:16AM +0200, O. Hartmann wrote: >>>> Hello List, >>>> >>>> lately I ran into a serious problem installing packages in a nanoBSD >>>> environment, in which the package repository server is "remotely" >>>> on site. The >>>> issue as documented below occurs on both 12-STABLE r348529 and >>>> CURRENT r348600 >>>> and must have been introduced shortly, since the last known good >>>> installation >>>> with the environment of ours was on 21st May 2019. >>>> >>>> As far as I know,, the package installation is performed via >>>> "chroot'ed" >>>> environment and somehow /dev/null is out of a sudden not accessible >>>> anymore >>>> while pkg tries to delegate some output to /dev/null. >>>> >>>> What happened here? >>>> >>>> Kind regards and thanks in advance, >>>> >>>> oh >>>> >>>> [...] >>>> All repositories are up to date. >>>> The following 10 package(s) will be affected (of 0 checked): >>>> >>>> New packages to be INSTALLED: >>>>         python3: 3_3 [zeit4] >>>>         sudo: 1.8.27_1 [zeit4] >>>>         devcpu-data: 1.22 [zeit4] >>>>         python36: 3.6.8_2 [zeit4] >>>>         readline: 8.0.0 [zeit4] >>>>         indexinfo: 0.3.1 [zeit4] >>>>         libffi: 3.2.1_3 [zeit4] >>>>         gettext-runtime: 0.19.8.1_2 [zeit4] >>>>         openldap-sasl-client: 2.4.47 [zeit4] >>>>         cyrus-sasl: 2.1.27 [zeit4] >>>> >>>> Number of packages to be installed: 10 >>>> >>> What is new is that pkg is using /dev/null as input when running >>> script? this is >>> new since pkg 1.11 . Somehow this does not seems to be avaalaible in >>> your >>> environement. >> >> Hi >> >> Same things applies to poudriere-image. I had to add a mount devfs >> command to the image.sh script. >> >>> >>> Best regards, >>> Bapt >> >> _______________________________________________ >> freebsd-current@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-current >> To unsubscribe, send any mail to >> "freebsd-current-unsubscribe@freebsd.org" --------------F545AD8270A6D91ADDA344E0 Content-Type: text/x-patch; name="image.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="image.diff" --- image.sh.orig 2019-06-06 08:24:36.445264000 -0700 +++ image.sh 2019-06-08 17:44:57.852022000 -0700 @@ -40,6 +40,7 @@ -n imagename -- The name of the generated image -o outputdir -- Image destination directory -p portstree -- Ports tree + -P -- Install packages from official repo -s size -- Set the image size -t type -- Type of image can be one of (default iso+zmfs): -- iso, iso+mfs, iso+zmfs, usb, usb+mfs, usb+zmfs, @@ -112,7 +113,7 @@ . ${SCRIPTPREFIX}/common.sh HOSTNAME=poudriere-image -while getopts "c:f:h:j:m:n:o:p:s:t:X:z:" FLAG; do +while getopts "c:f:h:j:m:n:o:Pp:s:t:X:z:" FLAG; do case "${FLAG}" in c) [ -d "${OPTARG}" ] || err 1 "No such extract directory: ${OPTARG}" @@ -146,6 +147,9 @@ OPTARG="${SAVED_PWD}/${OPTARG}" OUTPUTDIR=${OPTARG} ;; + P) + PKGREPO=1 + ;; p) PTNAME=${OPTARG} ;; @@ -239,7 +243,6 @@ mkdir -p ${WRKDIR}/out [ -z "${EXCLUDELIST}" ] || cat ${EXCLUDELIST} > ${excludelist} cat >> ${excludelist} << EOF -usr/src var/db/freebsd-update var/db/etcupdate boot/kernel.old @@ -351,8 +354,11 @@ make -C ${mnt}/usr/src DESTDIR=${WRKDIR}/world BATCH_DELETE_OLD_FILES=yes SRCCONF=${WRKDIR}/src.conf delete-old delete-old-libs [ ! -d "${EXTRADIR}" ] || cp -fRPp ${EXTRADIR}/ ${WRKDIR}/world/ -mv ${WRKDIR}/world/etc/login.conf.orig ${WRKDIR}/world/etc/login.conf -cap_mkdb ${WRKDIR}/world/etc/login.conf +if [ -e "${WRKDIR}/world/etc/login.conf.orig" ]; then + # No login.conf.orig on 13.0-CURRENT-i386 + mv ${WRKDIR}/world/etc/login.conf.orig ${WRKDIR}/world/etc/login.conf + cap_mkdb ${WRKDIR}/world/etc/login.conf +fi # Set hostname if [ -n "${HOSTNAME}" ]; then @@ -367,12 +373,15 @@ local REPOS_DIR=$(mktemp -dt poudriere_repo) local ABI_FILE - # This pkg rquery is always ran in host so we need a host-centric - # repo.conf always. - cat > "${REPOS_DIR}/repo.conf" <<-EOF - FreeBSD: { enabled: false } - local: { url: file:///${WRKDIR}/world/tmp/packages } - EOF + if [ -n "${PKGREPO}" ]; then + else + # This pkg rquery is always ran in host so we need a host-centric + # repo.conf always. + cat > "${REPOS_DIR}/repo.conf" <<-EOF + FreeBSD: { enabled: false } + local: { url: file:///${WRKDIR}/world/tmp/packages } + EOF + fi export REPOS_DIR PKG_DBDIR # Always need this from host. @@ -387,36 +396,83 @@ # install packages if any is needed if [ -n "${PACKAGELIST}" ]; then mkdir -p ${WRKDIR}/world/tmp/packages - ${NULLMOUNT} ${POUDRIERE_DATA}/packages/${MASTERNAME} ${WRKDIR}/world/tmp/packages + mount -t devfs devfs ${WRKDIR}/world/dev + chroot "${WRKDIR}/world" env /usr/sbin/pwd_mkdb -p /etc/master.passwd + [ -n "${RESOLV_CONF}" ] && cp -v "${RESOLV_CONF}" "${WRKDIR}/world/etc/" + mkdir -p ${POUDRIERE_DATA}/pkgcache/${MASTERNAME} + mkdir -p ${WRKDIR}/world/var/cache/pkg if [ "${arch}" == "${host_arch}" ]; then - cat > "${WRKDIR}/world/tmp/repo.conf" <<-EOF - FreeBSD: { enabled: false } - local: { url: file:///tmp/packages } - EOF - convert_package_list "${PACKAGELIST}" | \ - xargs chroot "${WRKDIR}/world" env \ - REPOS_DIR=/tmp ASSUME_ALWAYS_YES=yes \ - pkg install + if [ -n "${PKGREPO}" ]; then + ${NULLMOUNT} ${POUDRIERE_DATA}/pkgcache/${MASTERNAME} \ + ${WRKDIR}/world/var/cache/pkg + convert_package_list "${PACKAGELIST}" | \ + xargs chroot "${WRKDIR}/world" env \ + ASSUME_ALWAYS_YES=yes \ + pkg install + umount ${WRKDIR}/world/var/cache/pkg + else + ${NULLMOUNT} ${POUDRIERE_DATA}/packages/${MASTERNAME} ${WRKDIR}/world/tmp/packages + cat > "${WRKDIR}/world/tmp/repo.conf" <<-EOF + FreeBSD: { enabled: false } + local: { url: file:///tmp/packages } + EOF + convert_package_list "${PACKAGELIST}" | \ + xargs chroot "${WRKDIR}/world" env \ + REPOS_DIR=/tmp ASSUME_ALWAYS_YES=yes \ + pkg install + umount ${WRKDIR}/world/tmp/packages + fi else - cat > "${WRKDIR}/world/tmp/repo.conf" <<-EOF - FreeBSD: { enabled: false } - local: { url: file:///${WRKDIR}/world/tmp/packages } - EOF - ( - export ASSUME_ALWAYS_YES=yes SYSLOG=no \ - REPOS_DIR="${WRKDIR}/world/tmp/" \ - ABI_FILE="${WRKDIR}/world/usr/lib/crt1.o" - pkg -r "${WRKDIR}/world/" install pkg + if [ -n "${PKGREPO}" ]; then + # Try do same as for amd64 (we only care about i386) + ${NULLMOUNT} ${POUDRIERE_DATA}/pkgcache/${MASTERNAME} \ + ${WRKDIR}/world/var/cache/pkg convert_package_list "${PACKAGELIST}" | \ - xargs pkg -r "${WRKDIR}/world/" install - ) + xargs chroot "${WRKDIR}/world" env \ + ASSUME_ALWAYS_YES=yes \ + pkg install + umount ${WRKDIR}/world/var/cache/pkg + # ${NULLMOUNT} ${POUDRIERE_DATA}/pkgcache/${MASTERNAME} \ + # ${WRKDIR}/world/var/cache/pkg + # ( + # export ASSUME_ALWAYS_YES=yes SYSLOG=no \ + # ABI_FILE="${WRKDIR}/world/usr/lib/crt1.o" + # pkg -r "${WRKDIR}/world/" install pkg + # convert_package_list "${PACKAGELIST}" | \ + # xargs pkg -r "${WRKDIR}/world/" install + # ) + # umount ${WRKDIR}/world/var/cache/pkg + else + ${NULLMOUNT} ${POUDRIERE_DATA}/packages/${MASTERNAME} ${WRKDIR}/world/tmp/packages + cat > "${WRKDIR}/world/tmp/repo.conf" <<-EOF + FreeBSD: { enabled: false } + local: { url: file:///${WRKDIR}/world/tmp/packages } + EOF + ( + export ASSUME_ALWAYS_YES=yes SYSLOG=no \ + REPOS_DIR="${WRKDIR}/world/tmp/" \ + ABI_FILE="${WRKDIR}/world/usr/lib/crt1.o" + pkg -r "${WRKDIR}/world/" install pkg + convert_package_list "${PACKAGELIST}" | \ + xargs pkg -r "${WRKDIR}/world/" install + ) + umount ${WRKDIR}/world/tmp/packages + fi fi - rm -rf ${WRKDIR}/world/var/cache/pkg - umount ${WRKDIR}/world/tmp/packages + umount ${WRKDIR}/world/dev rmdir ${WRKDIR}/world/tmp/packages + rm -rf ${WRKDIR}/world/var/cache/pkg rm ${WRKDIR}/world/var/db/pkg/repo-* 2>/dev/null || : fi + +if [ -e "${WRKDIR}/world/overlay.sh" ]; then + export MASTERNAME + mount -t devfs devfs ${WRKDIR}/world/dev + chroot "${WRKDIR}/world" env LD_LIBRARY_PATH=/lib:/usr/lib:/usr/local/lib /overlay.sh + umount ${WRKDIR}/world/dev +fi + case ${MEDIATYPE} in *mfs) cat >> ${WRKDIR}/world/etc/fstab <<-EOF @@ -574,8 +630,8 @@ mkimg -s gpt -b ${mnt}/boot/pmbr \ -p efi:=${mnt}/boot/boot1.efifat \ -p freebsd-boot:=${mnt}/boot/gptboot \ - -p freebsd-ufs:=${WRKDIR}/raw.img \ - -p freebsd-swap::1M \ + -p freebsd-ufs/gfx-root:=${WRKDIR}/raw.img \ + -p freebsd-swap/gfx-swap::1G \ -o ${OUTPUTDIR}/${FINALIMAGE} ;; tar) --------------F545AD8270A6D91ADDA344E0-- From owner-freebsd-current@freebsd.org Wed Jun 12 00:22:45 2019 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0F2FE15CAF5C for ; Wed, 12 Jun 2019 00:22:45 +0000 (UTC) (envelope-from jbwlists@hilltopgroup.com) Received: from equinox.hilltopgroup.com (equinox.hilltopgroup.com [204.109.63.175]) by mx1.freebsd.org (Postfix) with ESMTP id 5305986CF1; Wed, 12 Jun 2019 00:22:43 +0000 (UTC) (envelope-from jbwlists@hilltopgroup.com) Received: from mail.relativity.hilltopgroup.com (unknown [104.185.205.155]) by equinox.hilltopgroup.com (Postfix) with ESMTP id 72F7B37B4C2; Tue, 11 Jun 2019 20:22:41 -0400 (EDT) Received: from [192.168.8.200] (unknown [104.185.205.155]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: jbwlists@hilltopgroup.com) by mail.relativity.hilltopgroup.com (Postfix) with ESMTPSA id 7596F11703; Tue, 11 Jun 2019 20:22:41 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=hilltopgroup.com; s=mail; t=1560298961; bh=da6RX47vTusnzv1Mx47e8rpBRcc1sDFG5KAHuJIOPzU=; h=Subject:To:Cc:References:From:Date:In-Reply-To; b=YExCtLyjT/bGeDm6cqZ4fx3b1VVQfrxsieBZXG9bt13To+ukWh1IaMksJvMS2kMLG QyXKwcipoJUd7DCtj71245t/nIG1WqAUXjN+6ifxqKUqcD9NIpAEiMK8waSsQkMftN qc/H77tpTik/1luPiCuZQfYiHalk3jiRGu6N0Nxo= Subject: Re: pkg: Cannot open /dev/null:No such file or directory To: Johannes Lundberg , Baptiste Daroussin Cc: freebsd-current References: <20190604073209.4e42a0eb@freyja> <20190604054409.4anei2ljzimqc75m@ivaldir.net> <935a44aa7587cdc07fadc2e33caed1f7@hilltopgroup.com> From: Joseph Ward Message-ID: Date: Tue, 11 Jun 2019 20:22:40 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 5305986CF1 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=hilltopgroup.com header.s=mail header.b=YExCtLyj; spf=pass (mx1.freebsd.org: domain of jbwlists@hilltopgroup.com designates 204.109.63.175 as permitted sender) smtp.mailfrom=jbwlists@hilltopgroup.com X-Spamd-Result: default: False [-4.50 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[hilltopgroup.com:s=mail]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+a]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[hilltopgroup.com]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: equinox.hilltopgroup.com]; DKIM_TRACE(0.00)[hilltopgroup.com:+]; NEURAL_HAM_SHORT(-0.96)[-0.959,0]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; IP_SCORE(-1.13)[ipnet: 204.109.60.0/22(-1.72), asn: 36236(-3.88), country: US(-0.06)]; ASN(0.00)[asn:36236, ipnet:204.109.60.0/22, country:US]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Jun 2019 00:22:45 -0000 Thank you!  I only added the mount line, but it seemed to work. On 2019-06-11 14:52, Johannes Lundberg wrote: > Hi > > This is probably overkill but I've attached a diff that show my patches > to image.sh. It's just a hack so far to make it do what I want and not > meant as general purpose. Use the changes you need for your application. > > Changes (only meant to work for building usb images on amd64 and i386) > > - mount devfs so pkg will work properly > - add "install packages from official repo" option so you won't have to > build your own packages for each jail > - downloaded packages from official repo is cached locally to avoid > excessive downloads > - hack for i386 so packages are installed properly > - increase swap space to allow for kernel core dumps (for usb image) > - execute post-install script "overlay.sh" in the jail if provided in > the overlay folder > > Bapt: Maybe you'll find some of these changes useful? :) > > /Johannes > > On 6/11/19 8:52 AM, jbwlists@hilltopgroup.com wrote: >> I'm having the same issue with poudriere image; could you please let >> me know what you did to fix it?  I'm assuming the image.sh you're >> referring to is /usr/local/share/poudriere/image.sh, but I'm not sure >> where the change would need to be made. >> >> Thanks in advance, >> Joseph >> >> >> On 2019-06-04 13:02, Johannes Lundberg wrote: >>> On 6/3/19 10:44 PM, Baptiste Daroussin wrote: >>>> On Tue, Jun 04, 2019 at 07:32:16AM +0200, O. Hartmann wrote: >>>>> Hello List, >>>>> >>>>> lately I ran into a serious problem installing packages in a nanoBSD >>>>> environment, in which the package repository server is "remotely" >>>>> on site. The >>>>> issue as documented below occurs on both 12-STABLE r348529 and >>>>> CURRENT r348600 >>>>> and must have been introduced shortly, since the last known good >>>>> installation >>>>> with the environment of ours was on 21st May 2019. >>>>> >>>>> As far as I know,, the package installation is performed via >>>>> "chroot'ed" >>>>> environment and somehow /dev/null is out of a sudden not accessible >>>>> anymore >>>>> while pkg tries to delegate some output to /dev/null. >>>>> >>>>> What happened here? >>>>> >>>>> Kind regards and thanks in advance, >>>>> >>>>> oh >>>>> >>>>> [...] >>>>> All repositories are up to date. >>>>> The following 10 package(s) will be affected (of 0 checked): >>>>> >>>>> New packages to be INSTALLED: >>>>>         python3: 3_3 [zeit4] >>>>>         sudo: 1.8.27_1 [zeit4] >>>>>         devcpu-data: 1.22 [zeit4] >>>>>         python36: 3.6.8_2 [zeit4] >>>>>         readline: 8.0.0 [zeit4] >>>>>         indexinfo: 0.3.1 [zeit4] >>>>>         libffi: 3.2.1_3 [zeit4] >>>>>         gettext-runtime: 0.19.8.1_2 [zeit4] >>>>>         openldap-sasl-client: 2.4.47 [zeit4] >>>>>         cyrus-sasl: 2.1.27 [zeit4] >>>>> >>>>> Number of packages to be installed: 10 >>>>> >>>> What is new is that pkg is using /dev/null as input when running >>>> script? this is >>>> new since pkg 1.11 . Somehow this does not seems to be avaalaible in >>>> your >>>> environement. >>> Hi >>> >>> Same things applies to poudriere-image. I had to add a mount devfs >>> command to the image.sh script. >>> >>>> Best regards, >>>> Bapt >>> _______________________________________________ >>> freebsd-current@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-current >>> To unsubscribe, send any mail to >>> "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-current@freebsd.org Wed Jun 12 06:21:47 2019 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 730D315A845D for ; Wed, 12 Jun 2019 06:21:47 +0000 (UTC) (envelope-from bapt@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 174778F549; Wed, 12 Jun 2019 06:21:47 +0000 (UTC) (envelope-from bapt@freebsd.org) Received: from ivaldir.etoilebsd.net (etoilebsd.net [178.32.217.76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: bapt) by smtp.freebsd.org (Postfix) with ESMTPSA id D43409EE0; Wed, 12 Jun 2019 06:21:46 +0000 (UTC) (envelope-from bapt@freebsd.org) Received: by ivaldir.etoilebsd.net (Postfix, from userid 1001) id BD15CB2A41; Wed, 12 Jun 2019 08:21:45 +0200 (CEST) Date: Wed, 12 Jun 2019 08:21:45 +0200 From: Baptiste Daroussin To: Johannes Lundberg Cc: jbwlists@hilltopgroup.com, freebsd-current Subject: Re: pkg: Cannot open /dev/null:No such file or directory Message-ID: <20190612062145.ssvzsstb7rc3ubdm@ivaldir.net> References: <20190604073209.4e42a0eb@freyja> <20190604054409.4anei2ljzimqc75m@ivaldir.net> <935a44aa7587cdc07fadc2e33caed1f7@hilltopgroup.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="gfkfnwlwfwv6nu24" Content-Disposition: inline In-Reply-To: User-Agent: NeoMutt/20180716 X-Rspamd-Queue-Id: 174778F549 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.99 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.99)[-0.993,0]; ASN(0.00)[asn:11403, ipnet:96.47.64.0/20, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Jun 2019 06:21:47 -0000 --gfkfnwlwfwv6nu24 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jun 11, 2019 at 11:52:31AM -0700, Johannes Lundberg wrote: > Hi >=20 > This is probably overkill but I've attached a diff that show my patches > to image.sh. It's just a hack so far to make it do what I want and not > meant as general purpose. Use the changes you need for your application. >=20 > Changes (only meant to work for building usb images on amd64 and i386) >=20 > - mount devfs so pkg will work properly > - add "install packages from official repo" option so you won't have to > build your own packages for each jail > - downloaded packages from official repo is cached locally to avoid > excessive downloads > - hack for i386 so packages are installed properly > - increase swap space to allow for kernel core dumps (for usb image) > - execute post-install script "overlay.sh" in the jail if provided in > the overlay folder >=20 > Bapt: Maybe you'll find some of these changes useful? :) >=20 The first I already committed days ago in poudriere, just I should have add= ed it as a patch to the poudriere version in ports, I will do it asap. As for other improvements, sure they sounds like useful, would be happy to review them if you make pull requests out of them! Thanks a lot, Bapt --gfkfnwlwfwv6nu24 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEgOTj3suS2urGXVU3Y4mL3PG3PloFAl0AmfMACgkQY4mL3PG3 PloUUg/6A3+jkc0imUEwQeVcG1h/2/AjRc2lrTweiovuKBs75lthUxSyBht3fdpw QroKQWFKV3LXWsfPIQa4cuCoTmqoF8GntZZjnUQeUwNBuXvTKrNP3H8lVkx9jNYl pODH3bve32jcPzvjSJIiDKOGLntVbPE/A08BYHXrFN3MprOt9kJ20pZ2T/RzWJ1V LHwqVbXNkG1o6N7lQZL9ty7bgT7hILA9H5hK2ikZQydk1pHTK2e2/H7pZGqT7D5f /CDwagdTUqaZXcNiQTQzyyalpUntieXZbwNYKJ/bZ0ltir9lx86L1gX3JcEksK+j qXpsv/8eL5lzs/T4g3950e4dV53KR3x8CnorNKHTb/QbdN6qZoQes5DC3co+PnAV GaRfbrguiEUflgknoxkl2nGKMJX4eejVyxXk659Sgex3rdj5ajwlgidpS6GN0d6r nxiHgDD3hew2XJlb8ZoH1AZKHO47IngXvxaE0FBAz5kzepG+CWwpiHPVDMVj5afi vlRsvkewvM9vgtkoPiQ9h1kmIBp3HvhjiMpYVctgaXb5GDOPalojUor00h0FD1Hu ddJ+kJJqzFy6gpNZAonNxh+smoDsa2P+VRQg8QocBTXMBNuSBynMfjmuEZ3WxihX lYByzkIbwDT5H06uEPXTyd2o65qTzeP2ZBxqxeFjiIYSjTta4TE= =wnyA -----END PGP SIGNATURE----- --gfkfnwlwfwv6nu24-- From owner-freebsd-current@freebsd.org Wed Jun 12 15:13:30 2019 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C9C5015B9302; Wed, 12 Jun 2019 15:13:30 +0000 (UTC) (envelope-from lwhsu.freebsd@gmail.com) Received: from mail-yw1-f65.google.com (mail-yw1-f65.google.com [209.85.161.65]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7772680F47; Wed, 12 Jun 2019 15:13:29 +0000 (UTC) (envelope-from lwhsu.freebsd@gmail.com) Received: by mail-yw1-f65.google.com with SMTP id t2so6939809ywe.10; Wed, 12 Jun 2019 08:13:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=HgXElayOnBH1Ca1XjFBa3vAyFilXIHncrM7KvWSdyQY=; b=UlTpsOTw25FjKiIjxX6nRyl8GeQcNKa5P+E0KkT8+k1VKYRz016asj8MJRlm8tglNm Cr0De/2Xi9dxxsNh/OiPxAr4t7M/0KMEXvIPDfVRO7O2AT6KsJFh+iSSEjYCLivT/mfc lBILT3AcB049iE6O09ijOUJzYJkQM3uolSzYPYJUmH4dgXWDTnF7q+elaelsNGp3jUbB ZHfMnXnytIOh8rHnBGRGp4lG+eXiI4jj1TXMrSZcm0LgWWrzjaFO8sy7kzykNzXMzfaD msevxdZ2AAiI5Sm0CJMAhm0TvwNdIv+VcN5pjMZ3cbnjkOc1Dkk3yZ5frwO/C+yoi24L iJnA== X-Gm-Message-State: APjAAAUIW+KleuQIdUjvw2CoHdFE9HARlsAcGyhgNvC1ZCmn6Hw66phc c57h6cJH5Lxemt98N6YA41T2gDxGwJ3W1YqmJZyxYOK/ X-Google-Smtp-Source: APXvYqzudl0/0Ba/w9Xuh7IuwR+mnH4IorVgVSiwTAY7dZ1S8pIOH6aDlVAxJisPMVPNf1OWJGFvAnqWMy7sn4vMLr4= X-Received: by 2002:a0d:ce86:: with SMTP id q128mr13485802ywd.360.1560350963656; Wed, 12 Jun 2019 07:49:23 -0700 (PDT) MIME-Version: 1.0 From: Li-Wen Hsu Date: Wed, 12 Jun 2019 10:49:13 -0400 Message-ID: Subject: FreeBSD CI Weekly Report 2019-06-09 To: freebsd-testing@freebsd.org Cc: freebsd-current@freebsd.org, freebsd-stable@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 7772680F47 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of lwhsufreebsd@gmail.com designates 209.85.161.65 as permitted sender) smtp.mailfrom=lwhsufreebsd@gmail.com X-Spamd-Result: default: False [-4.20 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_NEQ_ENVFROM(0.00)[lwhsu@freebsd.org,lwhsufreebsd@gmail.com]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_TLS_LAST(0.00)[]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; NEURAL_HAM_SHORT(-0.89)[-0.890,0]; RCVD_IN_DNSWL_NONE(0.00)[65.161.85.209.list.dnswl.org : 127.0.5.0]; IP_SCORE(-1.30)[ip: (-0.74), ipnet: 209.85.128.0/17(-3.41), asn: 15169(-2.30), country: US(-0.06)]; FORGED_SENDER(0.30)[lwhsu@freebsd.org,lwhsufreebsd@gmail.com]; MIME_TRACE(0.00)[0:+]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; TAGGED_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Jun 2019 15:13:31 -0000 (Please send the followup discussions to freebsd-testing@ list.) FreeBSD CI Weekly Report 2019-06-09 =================================== Here is a summary of the FreeBSD Continuous Integration results for the period from 2019-06-03 to 2019-06-09. During this period, we have: * 2444 builds (92.9% passed, 7.1% failed) were executed on aarch64, amd64, armv6, armv7, i386, mips, mips64, powerpc, powerpc64, powerpcspe, riscv64, sparc64 architectures for head, stable/12, stable/11 branches. * 247 test runs (51% passed, 49% unstable) were executed on amd64, i386, riscv64 architectures for head, stable/12, stable/11 branches. * 76 doc builds (100% passed) (The statistics from experimental jobs are omitted) If any of the issues found by CI are in your area of interest or expertise please investigate the PRs listed below. The latest web version of this report is available at https://hackmd.io/s/BJI5k8RRV and archive is available at http://hackfoldr.org/freebsd-ci-report/, any help is welcome. ## Fixed Tests * https://ci.freebsd.org/job/FreeBSD-head-i386-test/ * [r348808](https://reviews.freebsd.org/rS348808) fixed i386 kernel panics on loading ipsec(4) kernel module, which is needed after [347410](https://svnweb.freebsd.org/changeset/base/347410). For more information, see: * https://bugs.freebsd.org/238012 * https://bugs.freebsd.org/230857 * https://reviews.freebsd.org/D17512 ## Failing builds * https://ci.freebsd.org/job/FreeBSD-head-amd64-gcc/ * See https://reviews.freebsd.org/D20622 * pkg(8) now requires `/dev/null` which doesn't exist on jail builds test VM image. Fix is under development now. ## Failing Tests * https://ci.freebsd.org/job/FreeBSD-head-amd64-test/ * sys.netinet.socket_afinet.socket_afinet_bind_zero Affected by mac_portacl(4), which is loaded by MAC tests. Need to specify AF_INET to workaround and fix is being discussed. * https://ci.freebsd.org/job/FreeBSD-head-i386-test/ * Same as amd64: * sys.netinet.socket_afinet.socket_afinet_bind_zero * Others: * sys.netpfil.pf.forward.v6 * sys.netpfil.pf.forward.v4 * sys.netpfil.pf.set_tos.v4 * sys.opencrypto.runtests.main * https://ci.freebsd.org/job/FreeBSD-stable-12-i386-test/ * sys.netpfil.pf.forward.v6 * sys.netpfil.pf.forward.v4 * sys.netpfil.pf.set_tos.v4 * lib.libc.regex.exhaust_test.regcomp_too_big * lib.libregex.exhaust_test.regcomp_too_big * sys.opencrypto.runtests.main Failed with: ``` File "/usr/tests/sys/opencrypto/cryptodev.py", line 179, in __init__ ioctl(_cryptodev, CIOCGSESSION2, s, 1) IOError: [Errno 22] Invalid argument ``` * https://ci.freebsd.org/job/FreeBSD-stable-11-i386-test/ * local.kyua.* (31 cases) * local.lutok.* (3 cases) ## Failing and Flaky Tests (from experimental jobs) * https://ci.freebsd.org/job/FreeBSD-head-amd64-dtrace_test/ * Flakey test case: common.misc.t_dtrace_contrib.tst_dynopt_d https://bugs.freebsd.org/237641 * https://ci.freebsd.org/job/FreeBSD-head-amd64-test_zfs/ * This job is currently suffering from timeout because of https://bugs.freebsd.org/237652 * There are ~60 failing cases, including flakey ones, see https://ci.freebsd.org/job/FreeBSD-head-amd64-test_zfs/lastCompletedBuild/testReport/ for more details ## Disabled Tests * lib.libc.sys.mmap_test.mmap_truncate_signal https://bugs.freebsd.org/211924 * sys.fs.tmpfs.mount_test.large https://bugs.freebsd.org/212862 * sys.fs.tmpfs.link_test.kqueue https://bugs.freebsd.org/213662 * sys.kqueue.libkqueue.kqueue_test.main https://bugs.freebsd.org/233586 * usr.bin.procstat.procstat_test.command_loogle.com/ine_arguments https://bugs.freebsd.org/233587 * usr.bin.procstat.procstat_test.environment https://bugs.freebsd.org/233588 ## Open Issues * https://bugs.freebsd.org/237077 possible race in build: /usr/src/sys/amd64/linux/linux_support.s:38:2: error: expected relocatable expression * https://bugs.freebsd.org/237403 Tests in sys/opencrypto should be converted to Python3 * https://bugs.freebsd.org/237641 Flakey test case: common.misc.t_dtrace_contrib.tst_dynopt_d * https://bugs.freebsd.org/237652 tests.hotspare.hotspare_test.hotspare_snapshot_001_pos timeout since somewhere in (r346814, r 346845] * https://bugs.freebsd.org/237655 Non-deterministic panic when running pf tests in interface ioctl code (NULL passed to strncmp) * https://bugs.freebsd.org/237656 "Freed UMA keg (rtentry) was not empty (18 items). Lost 1 pages of memory." seen when running sys/netipsec tests * https://bugs.freebsd.org/237657 sys.kern.pdeathsig.signal_delivered_ptrace timing out periodically on i386 ### Cause build fails * [233735: Possible build race: genoffset.o /usr/src/sys/sys/types.h: error: machine/endian.h: No such file or directory](https://bugs.freebsd.org/233735) * [233769: Possible build race: ld: error: unable to find library -lgcc_s](https://bugs.freebsd.org/233769) ### Others * [Tickets related to testing@](https://preview.tinyurl.com/y9maauwg) * https://issues.tmatesoft.com/issue/SVNKIT-740 The patch is asked to be updated and help wanted. * https://bugs.freebsd.org/235356 Help on how to reproduce and analyze is wanted. From owner-freebsd-current@freebsd.org Wed Jun 12 16:06:28 2019 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0500A15BAFCF for ; Wed, 12 Jun 2019 16:06:28 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Received: from spindle.one-eyed-alien.net (spindle.one-eyed-alien.net [199.48.129.229]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B09F283827 for ; Wed, 12 Jun 2019 16:06:26 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Received: by spindle.one-eyed-alien.net (Postfix, from userid 3001) id E192B3C0182; Wed, 12 Jun 2019 16:06:24 +0000 (UTC) Date: Wed, 12 Jun 2019 16:06:24 +0000 From: Brooks Davis To: Oliver Pinter Cc: Konstantin Belousov , Rick Macklem , "freebsd-current@FreeBSD.org" Subject: Re: adding a syscall to libc? Message-ID: <20190612160624.GG64641@spindle.one-eyed-alien.net> References: <20190608102816.GR75280@kib.kiev.ua> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="2qXFWqzzG3v1+95a" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) X-Rspamd-Queue-Id: B09F283827 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.26 / 15.00]; ARC_NA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; IP_SCORE(-3.57)[ip: (-9.30), ipnet: 199.48.128.0/22(-4.63), asn: 36236(-3.88), country: US(-0.06)]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; MIME_TRACE(0.00)[0:+,1:+]; DMARC_NA(0.00)[freebsd.org]; AUTH_NA(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: spindle.one-eyed-alien.net]; NEURAL_HAM_SHORT(-0.78)[-0.776,0]; R_SPF_NA(0.00)[]; SIGNED_PGP(-2.00)[]; FORGED_SENDER(0.30)[brooks@freebsd.org,brooks@spindle.one-eyed-alien.net]; R_DKIM_NA(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:36236, ipnet:199.48.128.0/22, country:US]; FROM_NEQ_ENVFROM(0.00)[brooks@freebsd.org,brooks@spindle.one-eyed-alien.net]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_CC(0.00)[gmail.com] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Jun 2019 16:06:28 -0000 --2qXFWqzzG3v1+95a Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Jun 08, 2019 at 01:47:39PM +0200, Oliver Pinter wrote: > On Saturday, June 8, 2019, Konstantin Belousov wrot= e: >=20 > > On Sat, Jun 08, 2019 at 02:57:27AM +0000, Rick Macklem wrote: > > > Hi, > > > > > > I've started working of a copy_file_range() syscall for FreeBSD. I th= ink > > I have the > > > kernel patched and ready for some testing. > > > However, I'm confused about what I need to do in src/lib/libc/sys? > > > - Some syscalls have little .c files, but other ones do not. > > > When is one of these little .c files needed and, when not needed, w= hat > > else > > > needs to be done? (I notice that syscall.mk in src/sys/sys > > automagically, but > > > I can't see what else, if anything, needs to be done?) > > Most important is to add the new syscall public symbol to sys/Symbol.map > > into the correct version, FBSD_1.6 for CURRENT-13. Do no bother with > > __sys_XXX and __XXX aliases. > > > > 'Tiny .c files' are typically used for one of two purposes: > > - Convert raw kernel interface into something expected by userspace, > > often this coversion uses more generic and non-standard interface to > > implement more usual function. Examples are open(2) or waitid(2) > > which are really tiny wrappers around openat(2) and wait6(2) in > > today libc. > > - Allow libthr to hook into libc to provide additional services. Libthr > > often has to modify semantic of raw syscall, and libc contains the > > tables redirecting to implementation, the tables are patched on libthr > > load. Since tables must fill entries with some address in case libthr > > is not loaded, tiny functions which wrap syscalls are created for > > use in that tables. > > > > I think you do not need anything that complications for start, in which > > case adding new syscall consists of the following steps: > > - Add the syscall to sys/kern/syscalls.master, and if reasonable, > > to sys/compat/freebsd32/syscalls.master. > > - Consider if the syscall makes sense in capsicumized environment, > > and if yes, list the syscall in sys/kern/capabilities.conf. Typicall= y, > > if syscall provides access to the global files namespace, it must be = not > > allowed. On the other hand, if syscall only operates on already open= ed > > file descriptors, then it is suitable (but of course there are lot of > > nuances). > > - Add syscall prototype to the user-visible portion of header, > > hiding it under the proper visibility check. > > - Add syscall symbol to lib/libc/sys/Symbol.ver. > > - Implement the syscall. There are some additional details that might > > require attention: > > - If compat32 syscall going to be implemented, or you know > > that Linuxolator needs to implement same syscall and would > > like to take advantage of the code, provide > > int kern_YOURSYSCALL(); > > wrapper and declare it in sys/syscallsubr.h. Real > > implementations > > of host-native and compat32 sys_YOURSYSCALL() should be just > > decoding of uap members and call into kern_YOURSYSCALL. > > - Consider the need to add auditing for new syscall. > > - Add man page for the syscall, at lib/libc/sys/YOURSYSCALL.2, and conn= ect > > it to the build in lib/libc/sys/Makefile.inc. > > - When creating review for the change, do not include diff for generated > > files after make sysent. Similarly, when doing the commit, first com= mit > > everything non-generated, then do make -C sys/kern sysent (and > > make sysent -C sys/compat/freebsd32 sysent if appropriate) and commit > > the generated files in follow-up. >=20 >=20 > The best place for this little writeup would be in the wiki. ;) I wrote the initial version of this page long ago: https://wiki.freebsd.org/AddingSyscalls -- Brooks --2qXFWqzzG3v1+95a Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJdASL/AAoJEKzQXbSebgfADGwIAINGFN721khWIVUi7C/P50n9 ZG5D2UG3jih+ufPiTyEA7fGfrSCudowB2O8cl98V4b+o0m0LlrqoyONj+EIWKo9w NsfdIbjxDx5EWyZU+m1iUNzlyAhvNVYxQ7Vz5PB4zxJiuI3XftVlTAJZQ7+KqEdH +Xesizml5rQpb6Sv7Zpzq0KORzPRk8iwu/BfnuqPvM2wSVN+UpLpdiBnHTZBcOFu yofLfFmcM1TQE/wZ/hw0wvCl4fQwb6rnP8zvDtNLOnHKkKfkaKpxFiY1S2hLa3Y7 D3KoCw5/lBBq6gznib+sTumQ/6YvjcS3ldrmYVE0m/ys8dmePosdMDme69AXG6E= =eXhQ -----END PGP SIGNATURE----- --2qXFWqzzG3v1+95a-- From owner-freebsd-current@freebsd.org Thu Jun 13 08:38:41 2019 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 58D8915D0329; Thu, 13 Jun 2019 08:38:41 +0000 (UTC) (envelope-from miwi@FreeBSD.org) Received: from mail.fbsd.io (miwi.fbsd.io [149.28.138.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 930FC71C20; Thu, 13 Jun 2019 08:38:40 +0000 (UTC) (envelope-from miwi@FreeBSD.org) Received: from localhost ([127.0.0.1] helo=fbsd.io) by mail.fbsd.io with esmtpa (Exim 4.92 (FreeBSD)) (envelope-from ) id 1hbLFZ-000PHW-SA; Thu, 13 Jun 2019 16:38:31 +0800 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Thu, 13 Jun 2019 16:38:25 +0800 From: Martin Wilke To: freebsd-current@freebsd.org, kde-freebsd@kde.org Cc: owner-freebsd-current@freebsd.org Subject: Re: building x11-wm/plasma5-kwin w/ poudriere results in out of swap space In-Reply-To: <20190611065946.GA25933@sh4-5.1blu.de> References: <20190611065946.GA25933@sh4-5.1blu.de> Message-ID: <1e5d8f7b2360d65b8e8bfd2c3acd0388@FreeBSD.org> X-Sender: miwi@FreeBSD.org User-Agent: Roundcube Webmail/1.3.9 X-SA-Exim-Connect-IP: 127.0.0.1 X-SA-Exim-Mail-From: miwi@FreeBSD.org X-SA-Exim-Scanned: No (on mail.fbsd.io); SAEximRunCond expanded to false X-Rspamd-Queue-Id: 930FC71C20 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; NEURAL_HAM_SHORT(-0.98)[-0.985,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; ASN(0.00)[asn:20473, ipnet:149.28.128.0/19, country:US] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Jun 2019 08:38:41 -0000 On 2019-06-11 14:59, Matthias Apitz wrote: > Hello, > > I'm build the latest port svn checkout (June 10) on r342378 with > poudriere and (after isolating it) have the problem that building > x11-wm/plasma5-kwin runs reproducible out of swap during 'configure' > phase: > > [00:00:49] Cleaning the build queue > [00:00:49] Sanity checking build queue > [00:00:49] Processing PRIORITY_BOOST > [00:00:50] Balancing pool > [00:00:50] Recording filesystem state for prepkg... done > [00:00:52] Building 1 packages using 1 builders > [00:00:52] Starting/Cloning builders > [00:00:54] Hit CTRL+t at any time to see build progress and stats > [00:00:54] [01] [00:00:00] Building x11-wm/plasma5-kwin | > plasma5-kwin-5.15.5 > load: 0.95 cmd: sh 19376 [piperd] 67.68r 0.00u 0.00s 0% 3932k > [freebsd-r342378-ports-20190610] [2019-06-11_07h49m10s] > [parallel_build:] Queued: 1 Built: 0 Failed: 0 Skipped: 0 Ignored: > 0 Tobuild: 1 Time: 00:01:40 > [01]: x11-wm/plasma5-kwin | plasma5-kwin-5.15.5 > build-depends (00:01:06 / 00:01:08) > [00:02:02] Logs: > /usr/local/poudriere/data/logs/bulk/freebsd-r342378-ports-20190610/2019-06-11_07h49m10s > load: 0.52 cmd: wc 26535 [piperd] 0.10r 0.00u 0.00s 0% 2656k > [freebsd-r342378-ports-20190610] [2019-06-11_07h49m10s] > [parallel_build:] Queued: 1 Built: 0 Failed: 0 Skipped: 0 Ignored: > 0 Tobuild: 1 Time: 00:03:27 > [01]: x11-wm/plasma5-kwin | plasma5-kwin-5.15.5 configure > (00:01:17 / 00:02:55) > [00:03:49] Logs: > /usr/local/poudriere/data/logs/bulk/freebsd-r342378-ports-20190610/2019-06-11_07h49m10s > ^C[00:04:13] Error: Signal SIGINT caught, cleaning up and exiting > > Is this a known issue? Or should I investigate deeper which process it > is? This problem is not only for plasma5 its random happened since import of clang8. I have not found any real solution for it. > > matthias > > > -- > Matthias Apitz, ✉ guru@unixarea.de, http://www.unixarea.de/ > +49-176-38902045 > Public GnuPG key: http://www.unixarea.de/key.pub > May, 9: Спаси́бо освободители! Thank you very much, Russian liberators! > > > > ----- End forwarded message ----- From owner-freebsd-current@freebsd.org Thu Jun 13 09:39:39 2019 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A8F3D15A8E1D for ; Thu, 13 Jun 2019 09:39:39 +0000 (UTC) (envelope-from guru@unixarea.de) Received: from smh-06.1blu.de (smh-06.1blu.de [178.254.0.206]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 335F0746C5; Thu, 13 Jun 2019 09:39:37 +0000 (UTC) (envelope-from guru@unixarea.de) Received: from [172.16.29.5] (helo=sh4-5.1blu.de) by smh-06.1blu.de with esmtp (Exim 4.86_2) (envelope-from ) id 1hbMCd-0001fm-SZ; Thu, 13 Jun 2019 11:39:28 +0200 Received: from ftp51246-2575596 by sh4-5.1blu.de with local (Exim 4.86_2) (envelope-from ) id 1hbMCd-00087p-QM; Thu, 13 Jun 2019 11:39:27 +0200 Date: Thu, 13 Jun 2019 11:39:27 +0200 From: Matthias Apitz To: Martin Wilke Cc: freebsd-current@freebsd.org, kde-freebsd@kde.org Subject: Re: building x11-wm/plasma5-kwin w/ poudriere results in out of swap space Message-ID: <20190613093927.GA4977@sh4-5.1blu.de> Reply-To: Matthias Apitz Mail-Followup-To: Martin Wilke , freebsd-current@freebsd.org, kde-freebsd@kde.org References: <20190611065946.GA25933@sh4-5.1blu.de> <1e5d8f7b2360d65b8e8bfd2c3acd0388@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <1e5d8f7b2360d65b8e8bfd2c3acd0388@FreeBSD.org> X-Operating-System: FreeBSD 12.0-CURRENT r314251 (amd64) X-message-flag: Mails containing HTML will not be read! Please send only plain text. User-Agent: Mutt/1.5.24 (2015-08-30) X-Rspamd-Queue-Id: 335F0746C5 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-3.30 / 15.00]; ARC_NA(0.00)[]; HAS_REPLYTO(0.00)[guru@unixarea.de]; REPLYTO_EQ_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[unixarea.de]; AUTH_NA(1.00)[]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: mail.unixarea.de]; NEURAL_HAM_SHORT(-0.94)[-0.937,0]; NEURAL_HAM_MEDIUM(-0.99)[-0.994,0]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:42730, ipnet:178.254.0.0/19, country:DE]; RCVD_TLS_LAST(0.00)[]; IP_SCORE(-1.26)[ipnet: 178.254.0.0/19(-3.54), asn: 42730(-2.74), country: DE(-0.01)] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Jun 2019 09:39:39 -0000 El día Thursday, June 13, 2019 a las 04:38:25PM +0800, Martin Wilke escribió: > On 2019-06-11 14:59, Matthias Apitz wrote: > >Hello, > > > >I'm build the latest port svn checkout (June 10) on r342378 with > >poudriere and (after isolating it) have the problem that building > >x11-wm/plasma5-kwin runs reproducible out of swap during 'configure' > >phase: > > > >[00:00:49] Cleaning the build queue > >[00:00:49] Sanity checking build queue > >[00:00:49] Processing PRIORITY_BOOST > >[00:00:50] Balancing pool > >[00:00:50] Recording filesystem state for prepkg... done > >[00:00:52] Building 1 packages using 1 builders > >[00:00:52] Starting/Cloning builders > >[00:00:54] Hit CTRL+t at any time to see build progress and stats > >[00:00:54] [01] [00:00:00] Building x11-wm/plasma5-kwin | > >plasma5-kwin-5.15.5 > >load: 0.95 cmd: sh 19376 [piperd] 67.68r 0.00u 0.00s 0% 3932k > >[freebsd-r342378-ports-20190610] [2019-06-11_07h49m10s] > >[parallel_build:] Queued: 1 Built: 0 Failed: 0 Skipped: 0 Ignored: > >0 Tobuild: 1 Time: 00:01:40 > > [01]: x11-wm/plasma5-kwin | plasma5-kwin-5.15.5 > >build-depends (00:01:06 / 00:01:08) > >[00:02:02] Logs: > >/usr/local/poudriere/data/logs/bulk/freebsd-r342378-ports-20190610/2019-06-11_07h49m10s > >load: 0.52 cmd: wc 26535 [piperd] 0.10r 0.00u 0.00s 0% 2656k > >[freebsd-r342378-ports-20190610] [2019-06-11_07h49m10s] > >[parallel_build:] Queued: 1 Built: 0 Failed: 0 Skipped: 0 Ignored: > >0 Tobuild: 1 Time: 00:03:27 > > [01]: x11-wm/plasma5-kwin | plasma5-kwin-5.15.5 configure > > (00:01:17 / 00:02:55) > >[00:03:49] Logs: > >/usr/local/poudriere/data/logs/bulk/freebsd-r342378-ports-20190610/2019-06-11_07h49m10s > >^C[00:04:13] Error: Signal SIGINT caught, cleaning up and exiting > > > >Is this a known issue? Or should I investigate deeper which process it > >is? > > This problem is not only for plasma5 its random happened since import of > clang8. I have not found any real solution for it. Hello Martin, Thanks for replying. The issue is not randomly, but affects all ports using a QML tool 'qmlplugindump' which allocates 12 GByte memory. It is this bug issue: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=232736 I attached yesterday a poudriere log file for the port misc/kf5-purpose which also shows the same problem with 'qmlplugindump'. The poudriere log also says, that clang7 is used: ... =================================================== ===> Configuring for kf5-purpose-5.58.0 ===> Performing out-of-source build /bin/mkdir -p /wrkdirs/usr/ports/misc/kf5-purpose/work/.build -- The C compiler identification is Clang 7.0.1 -- The CXX compiler identification is Clang 7.0.1 -- Check for working C compiler: /usr/bin/clang -- Check for working C compiler: /usr/bin/clang -- works ... matthias -- Matthias Apitz, ✉ guru@unixarea.de, http://www.unixarea.de/ +49-176-38902045 Public GnuPG key: http://www.unixarea.de/key.pub May, 9: Спаси́бо освободители! Thank you very much, Russian liberators! From owner-freebsd-current@freebsd.org Sat Jun 15 04:11:11 2019 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0D23D15BAA1F for ; Sat, 15 Jun 2019 04:11:11 +0000 (UTC) (envelope-from bcran@freebsd.org) Received: from muon.bluestop.org (muon.bluestop.org [96.73.9.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 395D68C231; Sat, 15 Jun 2019 04:11:10 +0000 (UTC) (envelope-from bcran@freebsd.org) Received: from muon.bluestop.org (localhost [127.0.0.1]) by muon.bluestop.org (Postfix) with ESMTP id 32E726AE33; Fri, 14 Jun 2019 22:15:03 -0600 (MDT) Received: from muon.bluestop.org ([127.0.0.1]) by muon.bluestop.org (muon.bluestop.org [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 0G_72uu8kh96; Fri, 14 Jun 2019 22:15:02 -0600 (MDT) Received: from photon.int.bluestop.org (unknown [96.73.9.5]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by muon.bluestop.org (Postfix) with ESMTPSA; Fri, 14 Jun 2019 22:15:01 -0600 (MDT) To: freebsd-current@freebsd.org Cc: D Scott Phillips From: Rebecca Cran Subject: CTF: UEFI HTTP boot support Openpgp: preference=signencrypt Autocrypt: addr=bcran@freebsd.org; keydata= mQINBFrUMZ4BEADI1yUEGeZeXeTCPay1ZpTBdDEpGPAw1dq2VCSTc1VhsnrEBa1iZxAfaeSv Uu5Ti7jlhQ/3sQMl0bJMKGB/RtmIW7k8h2w476oZmG8gChk8su5ZEx/pV1gdqInyFmmJKTYc gabJz8pL+m82w07qPv+oalepZ4dbj+HF++RAK/iEju+q9UHlsjj8e3mMNsvtrOz1K6bnpveO jZ+ms/2H3Hs5a4k8y6buwe2RvwhJQaXa13cR3LhzL+nwj4B9PHZZEa2WpEyYpw/bI0V9YSQN QgC1CYRzDyakZge6BCM6wHOgZSUzRPufGilrNKUwIVbRoIBR9/85+0wR+PlFUOUOfOc6ox7T dWcIx6PuPhek48rh4uwmmwsPtPiH4Z3T5p+GmWQ9NLFZKA1YnEdaSkWtYZsDxwVZZeYG2plt MfhXP0Hj4rf9Y3eoUenCaGioxAbUOBCtXdTGNAhNjz1g5NGDBVyhjKkzwJQvt9UrYTseERit 5dX2CMTy8hYLvSXd/Ivy+HylUS5IslfZxW5z9LgWX7Z97kILgkH3N0ewtLkygkG+Y+x7uaAV dFqp9ASOyzaiwKbJdeOI+WxRSh+AqeCR0S+bpkcLudLmbjrPmaFwjKycy1H85Z5R2J3YHyXY oT6OYjD8vLbUU2GWp6Onkcy1Pu8EMbRuzKil6HnpYg3BexbPFwARAQABtCBSZWJlY2NhIENy YW4gPGJjcmFuQGZyZWVic2Qub3JnPokCVAQTAQgAPhYhBB+5fZtkTdO940Yr4g0CK1MRvhAg BQJa2B8pAhsjBQkJZgGABQsJCAcCBhUKCQgLAgQWAgMBAh4BAheAAAoJEA0CK1MRvhAgAe0P /R65umdPBVFCYKPZ91HMqlZtn0EWOGwycWEK/feWI+jaYOa+8+VVxFau4gwnBmgCdf5XOAJW QugUlPte9T+dP9QXmgm8z3KMLCj2PATYlmqmQfvIleJPf8w7BFBw/kkd6ZxoEQXaEyZwWuJc vY58uFYizZ8s1gMjD7uV3eg2UuGYd4loBZ3MSanWrhE6mmxAjzcYYb0KTsaTH9ON4uctcTYG 4FN0KzRx5d4nAhnS/yaL+3OI23vUDt+XnOCx8tIOczScOEN5NFChgyvTxzwi7hTVNB7uUCha mN6vcjtrrzi03zHXooldE4gRJ5G+SzuH9yHKrrwYXUeKi8sG8uXVoWwzsLbCxHRe7T9Ow7Im 92Aep3DEIE9whG6Fg3hRQ4/d/9OVCGrV3XwRMEstCvamJQc69ZsULo1ssGmPvcLn8fNaLNeP ICCLQj4JLcYvKKfLIQ/Cm0Orsy8rJGhwF4W1mBUbTdR6pk2azEkrhE7KZDylgikpNNqshKV3 1nD/5SNrTDl0P8rTnu0OKT0IbozIsaz9FD2xMPHPUMPnVSTB96+PhgoBIQlHcys19gftotuN 1tlLs4Ny93xWLSjKWoW5l9E9LbIh+M8gD3A7JyyV9DRZkHdbOt3pGjpaozchDPCpRkcsFXp4 9zSbXtxnbAWfZwoSYQTvznmXpzHMzoLMWf/guQINBFrUMZ4BEADkc4mvMcMcDF1tdNxNQuIB E1F243oZamG3LACCKfc1Yur3CPzHwIk5LXCUmbq23iE5bowxMWw3mlVT0p5xM0WnUidIBwCK u4kRyy/fY4NyWWBuwy9srpTdmUcKRBRNB8zEZE8xIlidD1ijjgqLBfeM7n9ylawAxHLxwU96 sdpdHFzb7Z0yKY2e/bzDaHiG0fUvcCmkgLf+uwKKZid1j8zR5PzKpgPqfy/PF01eKyGV3MNu 8Y90xMoiEMWfCI2IB1m+hTuzZoboFvGV54SiMuvfWK/VMQjhsL6K2ddOqwVuy2nIMI4G3xDQ W/v8KVyn43OSIAyW1eaklhzu0Ir2sO60PXRkvbTUrouvmSvpJfIQS49rU0M/X6FSDgXQLKrZ 3my94+g8ptz9KoVml6s4OAwYVz+sb49nuSxipFKkU5FwhKOLmzbsBxCtytcUJoLmjuJPJPDQ ue6YJiIXyc86GVY2pH3DjemKdbB4dSgqAJIp+lCzKSJzz7bgueh2Ox8vzx1tSxKj7V8Nal+U TKKbkxPmMh+e20YZ4esAVifO3bS6IJP/aDnfagghB71vA7+aWGXPbjPlc2UHpCBiRSsl+Igo QXvdvZBsKRyfBx8neODa2C6JIE5vcaCjilSeKF8SzsFXvimnndhQNhAPU/DwQwSXdCl4gTsF Vi5d8Oxq1sce+wARAQABiQI8BBgBCAAmFiEEH7l9m2RN073jRiviDQIrUxG+ECAFAlrUMZ4C GwwFCQlmAYAACgkQDQIrUxG+ECAWnRAAsmZX+KgNxW3v7R/76Tz4Wjmh4AGeE+Ji3p5QsdTY ny1B6vYBL9vCzPJ/AK8pgKMDRaweUP5eZQpfrdWC8Q7SNGgi4Q+97KEs+i2xZLQ+WJb8a+WE EIc716u0y4ITiHfOgM5jWcFO4MXQATbJgv0drLLesa+LQCvZgPBqupt307EsCubQs+Sxt+RV jf6rOUolp1GJXEQYwGsKklVd6yqLC8M1BSG53/WE5tSv5GzBZ8fp6EtmjT7leuidFtEvKYHQ z4DqG9ELpHUF0X0UUCBK/MgXe3kCVLKE060UrJ4M6uPSx57rmVFA2MvwQR8M7GsWC5UsSM4P YwPWBhwxE7vcx0691YKAHT/5q8LxRVBdUyzPSprMhSQFttsBt+ygm6wRi3Pi3TuCEARNubPk QefyeC34yr40SAUCkOl3eWxSXPF4NfXFQb4AAzZSE5hv3qbDuwo3lrL0LqpIpEQPAz+JZ1QZ 6mMFQ5/JD9Gukj54kZc0X8w3sQt0a8vyE/qrJg8vKgv2rCHrPc5MeDkEUEFiiJiCEDdkJtMy oRlU3S4NrnbyLOLEcHE8fGe3hStPX8hY62id2ecdQ5WZ7vLZW5SFeLarbUciuHIkVL6MHnUj bV7XlY50N7ebeFCIdlCWhdum2FJs/Ni+SSxbZC564vrokwlBBGSo6WTPQTa8IWx1DtU= Message-ID: <5cbfe8da-25d3-123f-52a1-08cde4e83f67@freebsd.org> Date: Fri, 14 Jun 2019 22:10:52 -0600 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.7.0 MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="WeVzBv5N2bdMWKXS0XeaEQQe2pYyPTWli" X-Rspamd-Queue-Id: 395D68C231 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.99 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.99)[-0.988,0]; ASN(0.00)[asn:7922, ipnet:96.64.0.0/11, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Jun 2019 04:11:11 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --WeVzBv5N2bdMWKXS0XeaEQQe2pYyPTWli Content-Type: multipart/mixed; boundary="KYzcXz774v3UhKQ3IjfIJ379KXqaFbiG9"; protected-headers="v1" From: Rebecca Cran To: freebsd-current@freebsd.org Cc: D Scott Phillips Message-ID: <5cbfe8da-25d3-123f-52a1-08cde4e83f67@freebsd.org> Subject: CTF: UEFI HTTP boot support --KYzcXz774v3UhKQ3IjfIJ379KXqaFbiG9 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Content-Language: en-US I've been working with D Scott Phillips to test the UEFI HTTP loader code he's written, and we're now ready for wider testing. In the testing I've done I've discovered a couple of bugs in the TianoCore EDK2 firmware, so I'd appreciate any testing people can do on various different systems to check how well they work. In theory HTTPS is also supported, but at least on TianoCore firmware it looks like TLS has a bug which means the second connection causes the system to crash. Also, this current work is IPv4 only: I hope to add IPv6 in the near futu= re. The patches can be found at: https://reviews.freebsd.org/D20642 https://reviews.freebsd.org/D20643 There's a good guide to setting up the DHCP and HTTP servers at https://en.opensuse.org/UEFI_HTTPBoot_Server_Setup . --=20 Rebecca Cran --KYzcXz774v3UhKQ3IjfIJ379KXqaFbiG9-- --WeVzBv5N2bdMWKXS0XeaEQQe2pYyPTWli Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEH7l9m2RN073jRiviDQIrUxG+ECAFAl0Eb9MACgkQDQIrUxG+ ECDK1hAAm56fsJcPA4GIWH7ctR9vgXKztIFZdn9zHHf0zLyiTDX9t4/Uo4Ichusu K0WUt+L9ZeBy9qXdDhjvhYgHqsPE/hUPo3koRBiHMWuyaBELdUYS6x0gcjBCRkLC K9dpuXExQC54OxyXwR8vCSMZC49ez2srtKPRHTF6MblroqNMFhsRfO0RYTpvhmKI /VB67+MJc/NkbTQUF5N3KFLM5rcdCCJbeUroM53cFSOmg6Ux/iq1/FV6leh6DWfF qsEJW1FtOjVg70DYIXJGZT+7yXDbiYsYVNBfy0yXtJvJWc/VVYhA3gBElnDQ+xUN Zb5vcQoRM4PqrjLoPVmj2az9+hsWqOi4D0HoNibbXjAiBYASFJ7xEC0ig+0j/ZJL wmwvfNf53YOGbrbr7AIGGMRC0hwrZ3P2taXrBcFwYqMixTMGVtHTb7KE3DT0vsqD kFa8fujQGDitnUvQHDzZ8CKJNREta9KGXhz/aHzCEqjvqx22Ao+Yp+ReRQyvWm2C dndvomMbOFlDI6ZoQqoVwZ1P2emV+X510Ff6kj76kk8MxWmBNd9W2ZMLqwwafbWd 3drSv4zVjGKvbNq6KYCZtb/XKxSdrszomh63jmNq1N58s0IFBp3BjvetTruHJ1Yf WDsjWyZqEzl3I0pg+AkGvHhtZe5ZvRErOGeRT1FUe3FqUyTEX6E= =+1G9 -----END PGP SIGNATURE----- --WeVzBv5N2bdMWKXS0XeaEQQe2pYyPTWli-- From owner-freebsd-current@freebsd.org Sat Jun 15 09:36:02 2019 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9122015C2082; Sat, 15 Jun 2019 09:36:02 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 33BBC6EE94; Sat, 15 Jun 2019 09:36:02 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from venus.codepro.be (venus.codepro.be [5.9.86.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.codepro.be", Issuer "Let's Encrypt Authority X3" (verified OK)) (Authenticated sender: kp) by smtp.freebsd.org (Postfix) with ESMTPSA id E6C14BD11; Sat, 15 Jun 2019 09:36:01 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from [10.10.132.5] (vpn.bally-wulff.de [212.144.118.13]) (Authenticated sender: kp) by venus.codepro.be (Postfix) with ESMTPSA id 840861F35C; Sat, 15 Jun 2019 11:36:00 +0200 (CEST) From: "Kristof Provost" To: "Li-Wen Hsu" Cc: freebsd-testing@freebsd.org, freebsd-current@freebsd.org, freebsd-stable@freebsd.org Subject: Re: FreeBSD CI Weekly Report 2019-06-09 Date: Sat, 15 Jun 2019 11:35:59 +0200 X-Mailer: MailMate (2.0BETAr6137) Message-ID: In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 33BBC6EE94 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.99 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.99)[-0.988,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Jun 2019 09:36:02 -0000 On 12 Jun 2019, at 16:49, Li-Wen Hsu wrote: > * https://ci.freebsd.org/job/FreeBSD-head-i386-test/ > * Same as amd64: > * sys.netinet.socket_afinet.socket_afinet_bind_zero > * Others: > * sys.netpfil.pf.forward.v6 > * sys.netpfil.pf.forward.v4 > * sys.netpfil.pf.set_tos.v4 I’ve finally gotten around to taking a look at this, and it appears to not be a pf problem. forward:v4 already fails at its sanity check, before it configures pf. It creates a vnet jail, telling it to route traffic through, and then we run a sanity check with pft_ping.py. Scapy tries to resolve the MAC address of the gateway (jail, 192.0.2.1). The jail replies, but scapy never picks up the reply, so the traffic looks like this: 13:19:29.953468 02:be:b4:57:9f:0a > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 192.0.2.2 tell 192.0.2.1, length 28 13:19:29.953572 02:be:b4:57:9f:0b > 00:a0:98:b2:48:59, ethertype ARP (0x0806), length 42: Reply 192.0.2.2 is-at 02:be:b4:57:9f:0b, length 28 13:19:32.082843 02:be:b4:57:9f:0a > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 52: 192.0.2.1 > 198.51.100.3: ICMP echo request, id 0, seq 0, length 18 The jail doesn’t forward the broadcast ICMP echo request and the test fails. My current guess is that it’s related to bpf. It’s interesting to note that it fails on i386, but succeeds on amd64. -- Kristof From owner-freebsd-current@freebsd.org Sat Jun 15 13:34:58 2019 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1B33515C79AB; Sat, 15 Jun 2019 13:34:58 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 84178759F9; Sat, 15 Jun 2019 13:34:57 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from venus.codepro.be (venus.codepro.be [5.9.86.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.codepro.be", Issuer "Let's Encrypt Authority X3" (verified OK)) (Authenticated sender: kp) by smtp.freebsd.org (Postfix) with ESMTPSA id 29656D936; Sat, 15 Jun 2019 13:34:57 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from [172.26.30.5] (vpn.bally-wulff.de [212.144.118.13]) (Authenticated sender: kp) by venus.codepro.be (Postfix) with ESMTPSA id BD6E31F626; Sat, 15 Jun 2019 15:34:53 +0200 (CEST) From: "Kristof Provost" To: "Li-Wen Hsu" Cc: freebsd-testing@freebsd.org, freebsd-current@freebsd.org, freebsd-stable@freebsd.org Subject: Re: FreeBSD CI Weekly Report 2019-06-09 Date: Sat, 15 Jun 2019 15:34:53 +0200 X-Mailer: MailMate (2.0BETAr6137) Message-ID: <356AE1B0-B3D7-4BE2-A052-7A1F996E3F37@FreeBSD.org> In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 84178759F9 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.978,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Jun 2019 13:34:58 -0000 On 15 Jun 2019, at 11:35, Kristof Provost wrote: > On 12 Jun 2019, at 16:49, Li-Wen Hsu wrote: >> * https://ci.freebsd.org/job/FreeBSD-head-i386-test/ >> * Same as amd64: >> * sys.netinet.socket_afinet.socket_afinet_bind_zero >> * Others: >> * sys.netpfil.pf.forward.v6 >> * sys.netpfil.pf.forward.v4 >> * sys.netpfil.pf.set_tos.v4 > > I’ve finally gotten around to taking a look at this, and it appears > to not be a pf problem. forward:v4 already fails at its sanity check, > before it configures pf. > > It creates a vnet jail, telling it to route traffic through, and then > we run a sanity check with pft_ping.py. > Scapy tries to resolve the MAC address of the gateway (jail, > 192.0.2.1). The jail replies, but scapy never picks up the reply, so > the traffic looks like this: > > 13:19:29.953468 02:be:b4:57:9f:0a > ff:ff:ff:ff:ff:ff, ethertype ARP > (0x0806), length 42: Request who-has 192.0.2.2 tell 192.0.2.1, length > 28 > 13:19:29.953572 02:be:b4:57:9f:0b > 00:a0:98:b2:48:59, ethertype ARP > (0x0806), length 42: Reply 192.0.2.2 is-at 02:be:b4:57:9f:0b, length > 28 > 13:19:32.082843 02:be:b4:57:9f:0a > ff:ff:ff:ff:ff:ff, ethertype IPv4 > (0x0800), length 52: 192.0.2.1 > 198.51.100.3: ICMP echo request, id > 0, seq 0, length 18 > > The jail doesn’t forward the broadcast ICMP echo request and the > test fails. > > My current guess is that it’s related to bpf. It’s interesting to > note that it fails on i386, but succeeds on amd64. > I’ve done a little dtracing, and I think that points at bpf too: #!/usr/sbin/dtrace -s fbt:kernel:bpf_buffer_uiomove:entry { tracemem(arg1, 1500, arg2); stack(); } Results in: 1 49539 bpf_buffer_uiomove:entry 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef 0: ce 0e 05 5d 17 ea 00 00 2a 00 00 00 2a 00 00 00 ...]....*...*... 10: 12 00 ff ff ff ff ff ff 02 fd 10 30 e6 0a 08 06 ...........0.... 20: 00 01 08 00 06 04 00 01 00 a0 98 b2 48 59 c0 00 ............HY.. 30: 02 01 00 00 00 00 00 00 c0 00 02 02 ce 0e 05 5d ...............] 40: 60 ea 00 00 2a 00 00 00 2a 00 00 00 12 00 00 a0 `...*...*....... 50: 98 b2 48 59 02 fd 10 30 e6 0b 08 06 00 01 08 00 ..HY...0........ 60: 06 04 00 02 02 fd 10 30 e6 0b c0 00 02 02 00 a0 .......0........ 70: 98 b2 48 59 c0 00 02 01 ..HY.... kernel`bpfread+0x137 kernel`dofileread+0x6d kernel`kern_readv+0x3b kernel`sys_read+0x48 kernel`syscall+0x2b4 0xffc033b7 So, we see the ARP request through bpf, but we don’t see the reply, despite tcpdump capturing it. I have no idea how that’d happen, so I’d very much like someone more familiar with bpf to take a look at this problem. Regards, Kristof