From owner-freebsd-hackers Wed Sep 11 19:05:47 1996 Return-Path: owner-hackers Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id TAA22112 for hackers-outgoing; Wed, 11 Sep 1996 19:05:47 -0700 (PDT) Received: from mx.serv.net (mx.serv.net [199.201.191.10]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id TAA22105 for ; Wed, 11 Sep 1996 19:05:36 -0700 (PDT) Received: from MindBender.serv.net by mx.serv.net (8.7.5/SERV Revision: 2.30 † id TAA05857; Wed, 11 Sep 1996 19:00:16 -0700 (PDT) Received: from localhost.HeadCandy.com (michaelv@localhost.HeadCandy.com [127.0.0.1]) by MindBender.serv.net (8.7.5/8.7.3) with SMTP id TAA23852; Wed, 11 Sep 1996 19:00:02 -0700 (PDT) Message-Id: <199609120200.TAA23852@MindBender.serv.net> X-Authentication-Warning: MindBender.serv.net: Host michaelv@localhost.HeadCandy.com [127.0.0.1] didn't use HELO protocol To: Mark Mayo cc: hackers@freebsd.org Subject: Re: forking program ? In-reply-to: Your message of Wed, 11 Sep 96 16:27:15 -0400. Date: Wed, 11 Sep 1996 19:00:00 -0700 From: "Michael L. VanLoon -- HeadCandy.com" Sender: owner-hackers@freebsd.org X-Loop: FreeBSD.org Precedence: bulk >I remember some time back someone posted a little piece of code that >forked and allocated memory until the OS gave up.. and the same program >appeared with an NT version (using CreateThread instead of fork()). Appended first is the original article that started the thread. Appended second is the NT version that I wrote. You should be able to find anything else by searching the FreeBSD archives using the information in these two messages... ---------- >8 ----- Begin article ----- 8< ---------- (Message freebsd/hackers:3826) Return-Path: owner-freebsd-hackers@freefall.freebsd.org Received: from smyrno.sol.net (smyrno.sol.net [206.55.64.117]) by MindBender.HeadCandy.com (8.7.5/8.7.3) with SMTP id LAA00987 for ; Thu, 8 Aug 1996 11:54:43 -0700 (PDT) Received: from freefall.freebsd.org (freefall.FreeBSD.ORG [204.216.27.4]) by smyrno.sol.net (8.6.11/8.6.12) with ESMTP id NAA29676; Thu, 8 Aug 1996 13:50:13 -0500 Received: from localhost (daemon@localhost) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id LAA04359; Thu, 8 Aug 1996 11:35:10 -0700 (PDT) Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id LAA04161 for hackers-outgoing; Thu, 8 Aug 1996 11:32:20 -0700 (PDT) Received: from cmr.kiev.ua (cmr.kiev.ua [193.193.193.50]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id LAA04130 for ; Thu, 8 Aug 1996 11:31:09 -0700 (PDT) Received: (archer@localhost) by cmr.kiev.ua (Sendmail 8.who.cares/5) id TAA15058; Thu, 8 Aug 1996 19:19:17 GMT Date: Thu, 8 Aug 1996 22:19:17 +0300 (EET DST) From: "Litvin Alexander B. " To: hackers@freebsd.org Subject: "Panick" - help needed... Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-hackers@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hello, All! I was advised by J.K.Habbard to send this into freebsd.hackers, so here my "problem" goes... I tried to figure out how would FreeBSD behave under heavy load. For this purpose I wrote the simple program. It creates 200 copies of itself, and then each copy tries to give system some work, that is opens a file and writes into it randomly and also allocates memory, uses it and frees memory afterwards. Here it is: #include #include #include int main() { int incarnation; /* Program incarnation */ int i,j; /* Counters */ int descriptor; /* File descriptor */ int PID,status; /* Self-explaining */ char buffer[512]; /* Buffer for file operations */ int written; /* Bytes written */ char name[10]; /* For file name */ double t; /* Temporary */ char* p=NULL; /* Dinamic allocated memory pointer */ for(incarnation=0;incarnation<200;incarnation++) { PID=fork(); /* Let's create another incarnation */ if(PID==-1) { printf("Cannot create another process, %d\n",incarnation); break; } if(!PID) break; } sprintf(name,"%d",getpid()); descriptor=open(name,O_CREAT|O_WRONLY,0777); /* Create temporary file */ if(descriptor==-1) { printf("Cannot create another file, %d\n",incarnation); exit(-1); } srand(incarnation); /* Make some randomization */ for(i=0;i<1000;i++) /* Do some work */ { written=write(descriptor,buffer,512); if(written!=512) { printf("Cannot write file, %d\n",incarnation); close(descriptor); if(p) free(p); exit(-1); } /* We'll write into file at random position */ t=((double)rand()/(double)RAND_MAX)*32255.; lseek(descriptor,(int)t,SEEK_SET); if(p) /* Use some VM as well */ { for(j=0;j<0xffff;j++) p[j]='Z'; free(p); p=NULL; } else { p=(char*)malloc(0xffff*sizeof(char)); if(!p) { printf("Cannot allocate memory, %d\n",incarnation); close(descriptor); exit(-1); } for(j=0;j<0xffff;j++) p[j]='z'; } } close(descriptor); if(p) free(p); wait(&status); return 0; } I ran at at one 2.1 box: AMD 5x86-133, ISA/PCI, 24M ram, 1.2G IDE hard disk, and at two 2.1.5 boxes: Pentium 100, ISA/PCI, 16M ram, Adaptec 7850 SCSI adapter, Seagate 2G SCSI hard disk, and Pentium 75, ISA/PCI, 16M ram, 1.2G hard disk. Each system crashes with message like: kernel page directory invalid pdir=0x52e063 va=0xc000 ^^^^^^^^ actual numbers may differ panic: invalid page directory Crash is more or less severe from time to time. Sometimes system will work well with 200 processes, but crash with 300 - it depends. I don't see any reason for such system behaviour. I'm not sure it's melfunction, but will be grateful to hear any comments. For we would like to use FreeBSD as a server for http, ftp, nntp, and other services, and we're very concerned with it's stability. I of course know that ftp.freebsd.org sustains very heavy load, but... At least, system should refuse to give servicies when overloaded, but (from my point of view) it shouldn't crash! P.S. May be the program is simply buggy - forgive the dummy in that case. -- Litvin Alexander +--------------------+ |"Must-die" must die!| +--------------------+ ---------- >8 ----- End article ----- 8< ---------- ---------- >8 ----- Begin NT code ----- 8< ---------- (Message carbons:1943) To: "Ron G. Minnich" , "Litvin Alexander B. " cc: hackers@freebsd.org Subject: Re: "Panick" - help needed... In-reply-to: Your message of Thu, 08 Aug 96 16:45:42 -0400. Date: Fri, 09 Aug 1996 01:17:55 -0700 From: "Michael L. VanLoon -- HeadCandy.com" >i just tried this on linux 1.3.99 for grins. It does not crash the >system. It just makes the system totally unusable. X locks up, my 'top' >window stops updating, etc. [details of Linux biting for anything other than benchmarks, deleted...] >If anyone runs this on a freebsd desktop i'd be interested in what you >observe -- how does interactive response function as this program runs. >Does the system become unusable for interactive use as does linux or does >the system just keep plugging along. Same for NT if anyone wants to fool >with it. I ran it on my NetBSD system. NetBSD-current (1.2_BETA), AMD 5x86 133MHz, 24MB RAM, BusLogic BT747s EISA SCSI, filesystem on a two SCSI drive ccd, etc. It ran for fifteen minutes, approximately with no noticeable weirdness. Response was *very* sluggish, taking a few minutes to log into another PCVT virtual terminal. This is probably partially because 200 processes were writing non-stop on the same disks it was trying to swap my process back in off of. I have a feeling it would have responded much quicker if the tmp files were written to a disk that had none of my swap allocated on it. At around fifteen minutes, I got a panic: vm_map_entry_create: out of map entries. Note that this is not exactly a bug. It's just a design deficiency in the old mach memory system NetBSD still uses. I. e. pmap entries are allocated in a fixed table, and I just plain ran out of them. The FreeBSD VM system has an advantage in this particular area. I also ran a modified version of this program on NT Server 4.0 Beta 2 (haven't upgraded to the release bits at home, yet), ASUS Pentium 100MHz, BusLogic BT956c PCI SCSI controller, etc. It ran for twenty minutes straight before I finally killed it. Absolutely nothing bad happened, as I had expected (we push NT machines a *lot* harder than this in the Exchange stress labs). The machine, like NetBSD, was very sluggish while the whole thing was in progress, taking from 30 to 60 seconds to switch back to a program that had gotten pushed out to swap. Since the filesystem and the swap, once again, were on the same drive, that probably was a significant impact. Here is my NT version of your program: -------------------------------------------------------------------- #include #include #include #include #define MAX_THREADS 200 int thrash(int iParam) { DWORD dwThreadID = 0; int i, j; /* Counters */ HANDLE hFile; DWORD dwStatus; BOOL bRet; char buffer[512]; /* Buffer for file operations */ int written; /* Bytes written */ char name[256]; /* For file name */ double t; /* Temporary */ char* p = NULL; /* Dinamic allocated memory pointer */ Sleep(rand() * 9000 / RAND_MAX + 1000); // 1 to 10 seconds printf("Starting thread %d.\n", iParam); dwThreadID = GetCurrentThreadId(); sprintf(name, "thrasher-tmp-%d", dwThreadID); hFile = CreateFile(name, GENERIC_WRITE, FILE_SHARE_WRITE, NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL); if (hFile == INVALID_HANDLE_VALUE) { dwStatus = GetLastError(); printf("Cannot create another file, %d: 0x%08x/%d\n", dwThreadID, dwStatus, dwStatus); goto Exit; } for (i = 0; i < 1000; i++) /* Do some work */ { bRet = WriteFile(hFile, buffer, 512, &written, NULL); if (!bRet || written != 512) { dwStatus = GetLastError(); printf("Cannot write file, %d: 0x%08x/%d\n", dwThreadID, dwStatus, dwStatus); goto Exit; } /* We'll write into file at random position */ t = ((double)rand() / (double)RAND_MAX) * 32255.; dwStatus = SetFilePointer(hFile, (LONG)t, NULL, FILE_BEGIN); if (dwStatus < 0) { dwStatus = GetLastError(); printf("Cannot seek file, %d: 0x%08x/%d\n", dwThreadID, dwStatus, dwStatus); goto Exit; } /* Use some VM as well */ if (p) { for (j = 0; j < 0xffff; j++) p[j] = (char)j; free(p); p = NULL; } else { p = (char*)malloc(0xffff * sizeof(char)); if (!p) { dwStatus = GetLastError(); printf("Cannot allocate memory, %d: 0x%08x/%d\n", dwThreadID, dwStatus, dwStatus); goto Exit; } for (j = 0; j < 0xffff; j++) p[j] = ~(char)j; } } dwStatus = 0; Exit: if (hFile != INVALID_HANDLE_VALUE) { CloseHandle(hFile); hFile = INVALID_HANDLE_VALUE; } if (p) { free(p); p = NULL; } return dwStatus; } int main() { int nThreads; HANDLE hThread; DWORD dwStatus; srand(GetTickCount()); for (nThreads = 0; nThreads < MAX_THREADS; nThreads++) { hThread = CreateThread(NULL, 0, (LPTHREAD_START_ROUTINE)thrash, (LPVOID)nThreads, 0, NULL); if (hThread == NULL) { dwStatus = GetLastError(); printf("Cannot create another thread, %d: 0x%08x/%d\n", nThreads, dwStatus, dwStatus); break; } } thrash(nThreads); return 0; } -------------------------------------------------------------------- ---------- >8 ----- End NT code ----- 8< ---------- ----------------------------------------------------------------------------- Michael L. VanLoon michaelv@MindBender.serv.net --< Free your mind and your machine -- NetBSD free un*x >-- NetBSD working ports: 386+PC, Mac 68k, Amiga, Atari 68k, HP300, Sun3, Sun4/4c/4m, DEC MIPS, DEC Alpha, PC532, VAX, MVME68k, arm32... NetBSD ports in progress: PICA, others... -----------------------------------------------------------------------------