Date: Sat, 5 Dec 2015 16:24:03 +0200 From: Konstantin Belousov <kostikbel@gmail.com> To: Gavin Mu <gavin.mu@qq.com> Cc: freebsd-stable <freebsd-stable@freebsd.org> Subject: Re: application coredump behavior differences between FreeBSD 7.0and FreeBSD 10.1 Message-ID: <20151205142403.GE2202@kib.kiev.ua> In-Reply-To: <tencent_532DCE3C7376849271198715@qq.com> References: <tencent_16AD659A73E203C91A0E112F@qq.com> <20151204094550.GO2405@kib.kiev.ua> <tencent_532DCE3C7376849271198715@qq.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, Dec 05, 2015 at 01:09:31PM +0800, Gavin Mu wrote:
> Hi, kib,
>
>
> Please see my testing on FreeBSD 7.0.
> freebsd7# sysctl kern.ipc.shmall
> kern.ipc.shmall: 819200
> freebsd7# sysctl kern.ipc.shmmax
> kern.ipc.shmmax: 3355443200
> freebsd7# uname -a
> FreeBSD freebsd7.localdomain 7.0-RELEASE FreeBSD 7.0-RELEASE #0: Sun Feb 24 10:35:36 UTC 2008 root@driscoll.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64
>
>
>
> testing code:
> freebsd7# cat tt.c
> #include <stdio.h>
> #include <stdlib.h>
> #include <machine/param.h>
> #include <sys/types.h>
> #include <sys/ipc.h>
> #include <sys/shm.h>
>
>
> int
> main(int argc, char **argv)
> {
> char **p;
> int size;
> int i;
> char *c = NULL;
> int shmid;
> void *shm_handle;
> size = atoi(argv[1]);
> printf("will alloc %dGB\n", size);
>
>
> shmid = shmget(100, size * 1024 * 1024 * 1024, 0644 | IPC_CREAT);
> if (shmid == -1) {
> printf("shmid = %d\n", shmid);
> }
>
>
> shm_handle = shmat(shmid, NULL, 0);
(shm_handle is not a handle).
> if (shm_handle == -1) {
> printf("null shm_handle\n");
> }
>
What if you add
madvise(shm_handle, size, MADV_SEQUENTIAL);
there ? Does 10.x behaviour become similar to that of the 7.x ?
>
> *c = 0;
> return 0;
> }
>
>
>
> freebsd7# ./a.out 1
> will alloc 1GB
> Segmentation fault (core dumped)
>
>
>
> when a.out is running, the RES keeps being 2024K without increasing:
>
>
> last pid: 735; load averages: 0.00, 0.01, 0.03 up 0+00:15:11 04:43:35
> 25 processes: 1 running, 24 sleeping
> CPU states: 0.0% user, 0.0% nice, 22.6% system, 0.8% interrupt, 76.7% idle
> Mem: 13M Active, 6380K Inact, 52M Wired, 32K Cache, 39M Buf, 910M Free
> Swap: 2015M Total, 2015M Free
>
>
> PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND
> 734 root 1 -16 0 1027M 2024K wdrain 0:02 13.27% a.out
>
>
>
> but when same code is running on FreeBSD 10.1, the RES keeps increasing to 1GB. From my testing, if the memory is allocated by malloc(), then RES will keep increasing in both 7.0 and 10.1. only sysv_shm in 7.0 has different behavior. I have checked coredump() code but did not find any clue why it is different.
>
>
> Regards,
> Gavin Mu
>
>
> ------------------ Original ------------------
> From: "Konstantin Belousov";<kostikbel@gmail.com>;
> Date: Fri, Dec 4, 2015 05:45 PM
> To: "Gavin Mu"<gavin.mu@qq.com>;
> Cc: "freebsd-stable"<freebsd-stable@freebsd.org>;
> Subject: Re: application coredump behavior differences between FreeBSD 7.0and FreeBSD 10.1
>
>
>
> On Fri, Dec 04, 2015 at 09:35:54AM +0800, Gavin Mu wrote:
> > Hi,
> >
> > We have an application running on old FreeBSD 7.0, and we are upgrading the base system to FreeBSD 10.1. The application uses sysv_shm, and will allocate a lot of share memory, though most of time only a part of the allocated memory is used. aka. large SIZE and small RES from /usr/bin/top view.
> >
> > When the application core dump, the core dump file will be large, and in FreeBSD 7.0, it uses only a little more memory to do core dump, but in FreeBSD 10.1, it seems all share memory are touched and uses a lot of physical memory (RES in /usr/bin/top output will increase very much) and cause memory drain.
> >
> > I have been debugging but can not find any clue yet. Could someone provide some points where the issue happen? Thanks.
>
> Both stable/7 and latest HEAD do read the whole mapped segment to write
> the coredump. This behaviour did not changed, since probably introduction
> of the ELF support into FreeBSD. And, how otherwise could coredump file
> contain the content of the mapped segments ?
>
> What in the FreeBSD 10 changed in this regard, is a deadlock fix which
> could occur in some scenarious, including the coredumping. In stable/7,
> the page instantiation or swap-in for pages accessed by the core write,
> was done while owning several VFS locks. This sometimes caused deadlock.
> In stable/10 the deadlock avoidance code is enabled by default, and
> when kernel detects the possibility of the deadlock, it changes to reading
> carefully by small chunks.
>
> Still, this does not explain the effect that you describe. In fact, I
> am more suspicious to the claim that stable/7 did not increase RSS of
> the dumping process or did not accessed the whole mapped shared segment,
> then the claim that there is a regression in stable/10.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20151205142403.GE2202>