From owner-freebsd-hackers@FreeBSD.ORG Thu Jul 25 14:57:47 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id DB3C59CA for ; Thu, 25 Jul 2013 14:57:46 +0000 (UTC) (envelope-from h.tugrul.erdogan@gmail.com) Received: from mail-ve0-x22a.google.com (mail-ve0-x22a.google.com [IPv6:2607:f8b0:400c:c01::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9A41E2CF4 for ; Thu, 25 Jul 2013 14:57:46 +0000 (UTC) Received: by mail-ve0-f170.google.com with SMTP id 14so541428vea.29 for ; Thu, 25 Jul 2013 07:57:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=iq4ID92iyvrDlGDbq96s6Jq1VCvtVwKcUuKeB+v5Srw=; b=a7QYPln3vOEiTzw8ycNaxy5fEMGp3CjGAoFCWyfrV8odQO6v16uM9THTQT1SUTa7IH uzZUe2waZrL0ZHOLZUo4Se28E4Xh2ZIkkrc/b2ypNN2llXU5ncM2scUbkIviSC20J20V ZlzTWPLLTfsDeHUIxGmxJNEhqwl9IksZx0cWjbUGx1y/1Y2RTtoDvakSOfyjzxTDqheM ZYTbWVnJqAIQ/P44Ytqm/K9hyzcKsNlZZuVItNBlOWUrvrhSKpaTi89nX5H5R9/ML3zA BMTUOq7mR668v2jGZIGsh/K5cM/r81IBGHu2okaw3U5Kk5EenpsagTNU2aPmMoIDX+h4 Kl0A== MIME-Version: 1.0 X-Received: by 10.58.69.65 with SMTP id c1mr17569375veu.88.1374764265738; Thu, 25 Jul 2013 07:57:45 -0700 (PDT) Received: by 10.58.54.103 with HTTP; Thu, 25 Jul 2013 07:57:45 -0700 (PDT) Date: Thu, 25 Jul 2013 17:57:45 +0300 Message-ID: Subject: panic: kmem_map too small at heavy packet traffic From: Tugrul Erdogan To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Jul 2013 14:57:47 -0000 howdy all, At my work, I am using 10.0-CURRENT on Intel(R) Xeon(R) E5620 with 16GB ram. I am taking "panic: kmem_malloc(-548663296): kmem_map too small: 539459584 total allocated" message with configuration below: [root@ ~]# sysctl vm.kmem_size_min vm.kmem_size_max vm.kmem_size vm.kmem_size_scale vm.kmem_size_min: 0 vm.kmem_size_max: 329853485875 vm.kmem_size: 16686845952 vm.kmem_size_scale: 1 [root@ ~]# sysctl hw.physmem hw.usermem hw.realmem hw.physmem: 17151787008 hw.usermem: 8282652672 hw.realmem: 18253611008 [root@ ~]# sysctl hw.pagesize hw.pagesizes hw.availpages hw.pagesize: 4096 hw.pagesizes: 4096 2097152 0 hw.availpages: 4187448 When I compare vmstat and netstat output of boot time result and subsequent result, the major difference are seemed at: pf_temp 0 0K - 79309736 128 | pf_temp 1077640 134705K - 84330076 128 and after the panic at the core dump file the major vmstat difference is: temp 110 15K - 76212305 16,32,64,128,256 | temp 117 6742215K - 655115 16,32,64,128,2 When I explore the source code of kernel (at vm_kern.c and vm_map.c), I see that the panic can occur with the cases at below: * negative malloc size parameter * longer than free buffer respect to kmem_map min_offset and max_offset values * try to allocate when the root entry of map is the rightmost entry of map * try to allocate bigger than map's max_free value I think the panic occurs at mbuf creation process when calling malloc() as a result of couldn't be able to allocate memory; but I don't understand why one of this panic case activating? The memory is almost empty but the device is saying kmem_map small when using about 0.5GB memory purely. How can i solve this panic problem? Thank you all for your time. -- Best Wishes, Tugrul Erdogan