From owner-freebsd-stable@FreeBSD.ORG Mon Dec 11 20:34:44 2006 Return-Path: X-Original-To: freebsd-stable@freebsd.org Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9A07F16A412 for ; Mon, 11 Dec 2006 20:34:44 +0000 (UTC) (envelope-from uspoerlein@gmail.com) Received: from ug-out-1314.google.com (ug-out-1314.google.com [66.249.92.172]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1A08044A79 for ; Mon, 11 Dec 2006 20:10:02 +0000 (GMT) (envelope-from uspoerlein@gmail.com) Received: by ug-out-1314.google.com with SMTP id o2so1301962uge for ; Mon, 11 Dec 2006 12:10:51 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:date:from:to:subject:message-id:mail-followup-to:references:mime-version:content-type:content-disposition:in-reply-to; b=h0BF50LHfxgvC3YssPfT47bx0mzBG7eGCyFtWTarwAwxkD3LpglmV3EYu+LQqg40acqYo8AMWzYgxzThV5PnOAddxMT1uuQVBiK1lXjtVzDyI+7MOZzIC9u8HYaEKy8+wus+WGGfH92YitwmwepBIJU9vBJNXwN9zRz8GbFAAc0= Received: by 10.66.244.11 with SMTP id r11mr10454500ugh.1165867849568; Mon, 11 Dec 2006 12:10:49 -0800 (PST) Received: from roadrunner.q.local ( [85.180.168.118]) by mx.google.com with ESMTP id y7sm7711201ugc.2006.12.11.12.10.48; Mon, 11 Dec 2006 12:10:49 -0800 (PST) Received: from roadrunner.q.local (localhost [127.0.0.1]) by roadrunner.q.local (8.13.8/8.13.8) with ESMTP id kBBKAjgG001625; Mon, 11 Dec 2006 21:10:45 +0100 (CET) (envelope-from uspoerlein@gmail.com) Received: (from q@localhost) by roadrunner.q.local (8.13.8/8.13.8/Submit) id kBBKAg0i001624; Mon, 11 Dec 2006 21:10:42 +0100 (CET) (envelope-from uspoerlein@gmail.com) Date: Mon, 11 Dec 2006 21:10:42 +0100 From: Ulrich Spoerlein To: Craig Boston , David Gilbert , freebsd-stable@freebsd.org Message-ID: <20061211201042.GA1569@roadrunner.q.local> Mail-Followup-To: Craig Boston , David Gilbert , freebsd-stable@freebsd.org References: <17774.32960.176956.52924@canoe.dclg.ca> <20061203171221.GB2369@roadrunner.q.local> <20061211024353.GA1220@nowhere> <20061211183728.GA1525@roadrunner.q.local> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20061211183728.GA1525@roadrunner.q.local> Cc: Subject: Re: ggate still broken on 6.2-RC1 for amd64. X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Dec 2006 20:34:44 -0000 Ulrich Spoerlein wrote: > But I'll whip up a ggate test case. Very strange ... I thought I would work through different buffer sizes, starting with some low value. Here's what gives: igor# ggated -a localhost -v -R8k -S8k /tmp/ggate_exports igor# ggatec create -v -R8k -S8k localhost /tmp/ggate_test info: Reading exports file (/tmp/ggate_exports). info: Connected to the server: localhost:3080. debug: Added 127.0.0.1/32 /tmp/ggate_test RW to exports list. debug: Sending version packet. info: Exporting 1 object(s). info: Listen on port: 3080. info: Connection from: 127.0.0.1. debug: Receiving version packet. debug: Version packet received. debug: Receiving initial packet. debug: Initial packet received. debug: Sending initial packet. debug: Connection created [127.0.0.1, /tmp/ggate_test]. debug: Receiving initial packet. debug: New connection created (token=226910802). debug: Received initial packet. debug: Sending initial packet. info: Connected to the server: localhost:3080. debug: Sending version packet. g_gate_send: EAGAIN g_gate_send: EAGAIN g_gate_send: EAGAIN g_gate_send: EAGAIN info: Connection from: 127.0.0.1. ^C debug: Receiving version packet. ^C Now try with 16k. igor# ggated -a localhost -v -R16k -S16k /tmp/ggate_exports igor# ggatec create -v -R16k -S16k localhost /tmp/ggate_test info: Reading exports file (/tmp/ggate_exports). info: Connected to the server: localhost:3080. debug: Added 127.0.0.1/32 /tmp/ggate_test RW to exports list. debug: Sending version packet. info: Exporting 1 object(s). info: Listen on port: 3080. info: Connection from: 127.0.0.1. debug: Receiving version packet. debug: Version packet received. debug: Receiving initial packet. debug: Initial packet received. debug: Sending initial packet. debug: Connection created [127.0.0.1, /tmp/ggate_test]. debug: Receiving initial packet. debug: New connection created (token=2294332471). debug: Received initial packet. debug: Sending initial packet. info: Connected to the server: localhost:3080. info: Connection from: 127.0.0.1. debug: Sending version packet. debug: Receiving version packet. debug: Version packet received. debug: Receiving initial packet. debug: Initial packet received. debug: Sending initial packet. debug: Found existing connection (token=2294332471). debug: Receiving initial packet. debug: Connection added [127.0.0.1, /tmp/ggate_test]. debug: Received initial packet. debug: Sending initial packet. ggate5 debug: Connection removed [127.0.0.1 /tmp/ggate_test]. notice: send_thread: started! debug: Process created [/tmp/ggate_test]. notice: recv_thread: started! notice: disk_thread: started [/tmp/ggate_test]! notice: send_thread: started [/tmp/ggate_test]! notice: recv_thread: started [/tmp/ggate_test]! debug: Process 1140 exiting. ^C I wanted to use something like the following, for first draft of a benchmark, but I just I/O deadlocked the system (6.2 and CURRENT). Simply by running ggated/ggatec in various combinations. db> show alllocks Process 1333 (ggatel) thread 0xc2767510 (100081) exclusive sx sysctl lock r = 0 (0xc078c420) locked @ /vol/src/sys/kern/kern_sysctl.c:1376 db> trace 1333 Tracing pid 1333 tid 100081 td 0xc2767510 sched_switch(c2767510,0,1) at sched_switch+0xe7 mi_switch(1,0) at mi_switch+0x27c sleepq_switch(c2b3e680,c078bdd0,0,c070e413,236,...) at sleepq_switch+0xc9 sleepq_timedwait(c2b3e680) at sleepq_timedwait+0x4a msleep(c2b3e680,0,4c,c07028f3,64) at msleep+0x281 g_waitfor_event(c050d120,c2b6c300,2,0,0,0,0,1) at g_waitfor_event+0x73 sysctl_kern_geom_confxml(c07485e0,0,0,d1781b9c,c07485e0,...) at sysctl_kern_geom_confxml+0x26 sysctl_root(0,d1781c1c,3,d1781b9c) at sysctl_root+0x12f userland_sysctl(c2767510,d1781c1c,3,8300000,bfbfe3d8,0,0,0,d1781c18,0,c078bde8,0,c070bc1f,522) at userland_sysctl+0xf4 __sysctl(c2767510,d1781d04) at __sysctl+0x77 syscall(3b,3b,3b,3,bfbfe3d8,...) at syscall+0x27e Xint0x80_syscall() at Xint0x80_syscall+0x1f --- syscall (202, FreeBSD ELF32, __sysctl), eip = 0x2816ba7f, esp = 0xbfbfe2bc, ebp = 0xbfbfe2f8 --- db> ps pid ppid pgrp uid state wmesg wchan cmd 1348 800 800 0 S sysctl l 0xc078c444 cron 1347 800 800 0 S sysctl l 0xc078c444 cron 1346 800 800 0 S sysctl l 0xc078c444 cron 1345 795 795 0 S sysctl l 0xc078c444 sshd 1344 800 800 0 S sysctl l 0xc078c444 cron 1343 800 800 0 S sysctl l 0xc078c444 cron 1342 800 800 0 S sysctl l 0xc078c444 cron 1341 800 800 0 S sysctl l 0xc078c444 cron 1340 800 800 0 S sysctl l 0xc078c444 cron 1339 800 800 0 S sysctl l 0xc078c444 cron 1338 800 800 0 S sysctl l 0xc078c444 cron 1337 800 800 0 S sysctl l 0xc078c444 cron 1336 800 800 0 S sysctl l 0xc078c444 cron 1335 262 40 0 S sysctl l 0xc078c444 sh 1334 925 1334 0 S+ sysctl l 0xc078c444 ls 1333 1078 1333 0 S+ g_waitfo 0xc2b3e680 ggatel 1078 1077 1078 0 S+ pause 0xc2b06264 csh 1077 1068 1077 0 S+ wait 0xc2b05b40 su 1068 884 1068 1000 Ss+ pause 0xc2b07024 zsh 969 899 969 0 Ss+ ttyin 0xc24a2c10 csh 925 899 925 0 Ss+ pause 0xc2765da4 csh 900 899 900 0 Ss+ ttyin 0xc2479810 csh 899 898 899 0 Ss select 0xc07d69ec screen 898 895 898 0 S+ pause 0xc2b06da4 screen 895 894 895 0 S+ pause 0xc2765024 csh 894 885 894 1000 S+ wait 0xc27656c0 su 885 884 885 1000 Ss+ pause 0xc26e64a4 zsh 884 882 882 1000 S sysctl l 0xc078c444 sshd 882 795 882 0 Ss sbwait 0xc27591e8 sshd 864 1 864 0 Ss+ ttyin 0xc2491410 getty 863 1 863 0 Ss+ ttyin 0xc24a2810 getty As you see, the crons are slowly piling up, waiting on the sysctl lock. ---- cut here ---- #!/bin/sh ge="/tmp/ggate_exports" tf="/tmp/ggate_test" ts="64m" u=6 if [ "$#" -lt 1 ]; then set -- 32k 64k 128k 256k 16k 8k fi kldload geom_gate truncate -s $ts $tf echo "localhost RW $tf" > $ge for buf; do ggated -a localhost -R $buf -S $buf $ge sleep 3 ggatec create -u $u -R $buf -S $buf localhost $tf sleep 3 dd if=/dev/ggate$u of=/dev/zero ggatec destroy -fu$u killall ggated sleep 3 done ---- cut here ---- Too bad, I don't have a 6.1 or 5.x system lying around that I could mess with. Could you guys try the script above on your test systems? Ulrich Spoerlein -- A: Yes. >Q: Are you sure? > >A: Because it reverses the logical flow of conversation. > >>Q: Why is top posting frowned upon?