From owner-freebsd-fs@FreeBSD.ORG Mon Mar 30 15:45:40 2015 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8AEC2C72 for ; Mon, 30 Mar 2015 15:45:40 +0000 (UTC) Received: from smtp21.mail.ru (smtp21.mail.ru [94.100.179.250]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5DA293B6 for ; Mon, 30 Mar 2015 15:45:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mail.ru; s=mail2; h=Content-Type:In-Reply-To:References:Subject:To:MIME-Version:From:Date:Message-ID; bh=EvG6BJ8oY3+arXzPURoJjq9z9OPR8Xr0iJDL0ItbKc0=; b=NBCusfHC503in5in3pUPwQhKB0L3BT6WgCM/nNs5wdqv5h/hTe1Bn4UFhPp8PQigLZUcWmWMMyiqmKgaV7yNDOZxqGN5bhgrC9XWCNETJ3fEZfv0pBHBiWu4F//bGjXPTflst+ao3OBHZZB6/eU5RZEzG23SdOM0fEOaYF+Gl84=; Received: from [109.188.127.13] (port=64864 helo=[192.168.0.12]) by smtp21.mail.ru with esmtpa (envelope-from ) id 1Ycbsd-0002q5-66 for freebsd-fs@freebsd.org; Mon, 30 Mar 2015 18:45:35 +0300 Message-ID: <55196FC7.8090107@artem.ru> Date: Mon, 30 Mar 2015 18:46:15 +0300 From: Artem Kuchin User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: freebsd-fs@freebsd.org Subject: Re: Little research how rm -rf and tar kill server References: <55170D9C.1070107@artem.ru> <1427727936.293597.247070269.5CE0D411@webmail.messagingengine.com> In-Reply-To: <1427727936.293597.247070269.5CE0D411@webmail.messagingengine.com> X-Spam: Not detected X-Mras: Ok Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Mar 2015 15:45:40 -0000 30.03.2015 18:05, Mark Felder пишет: > > > On Sat, Mar 28, 2015, at 15:22, Artem Kuchin wrote: >> >> So, questions and thoughts: 1) Why i had no problem such this in >> fbsd 9? I think the reason for the problem is in kernel, not in >> hardware or mariadb+nginx because server load did not increase at >> all, even decreases a little. > > This is only anecdotal until you run the exact same OS version and > load on the old hardware which had a different motherboard and > (probably) disk controller. Can you provide us with any further > hardware details between the two systems? My point was that the change what causes it was in freebsd kernel or ufs driver or sata driver or somewhere there, not the hardware.I even left the Kernel config file the same. I cannot provide exact hardware and dmesg for the old server (it is down and not available), but it was Intel Xeon E3-1245 /Quad Core/ 16 GB DDR3 ECC RAM 2x 3 TB SATA III Enterprise HDD (seagate ST33000650NS) New server kernel: Copyright (c) 1992-2015 The FreeBSD Project. kernel: Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 kernel: The Regents of the University of California. All rights reserved. kernel: FreeBSD is a registered trademark of The FreeBSD Foundation. kernel: FreeBSD 10.1-STABLE #0 r279278: Wed Feb 25 15:17:48 MSK 2015 kernel: FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 kernel: CPU: Intel(R) Xeon(R) CPU E3-1270 v3 @ 3.50GHz (3500.07-MHz K8-class CPU) kernel: Origin="GenuineIntel" Id=0x306c3 Family=0x6 Model=0x3c Stepping=3 kernel: Features=0xbfebfbff kernel: Features2=0x7ffafbff,FMA,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AE kernel: AMD Features=0x2c100800 kernel: AMD Features2=0x21 kernel: Structured Extended Features=0x2fbb kernel: XSAVE Features=0x1 kernel: VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID kernel: TSC: P-state invariant, performance statistics kernel: real memory = 34359738368 (32768 MB) kernel: avail memory = 33271001088 (31729 MB) kernel: Event timer "LAPIC" quality 600 kernel: ACPI APIC Table: kernel: FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs kernel: FreeBSD/SMP: 1 package(s) x 4 core(s) x 2 SMT threads kernel: cpu0 (BSP): APIC ID: 0 kernel: cpu1 (AP): APIC ID: 1 kernel: cpu2 (AP): APIC ID: 2 kernel: cpu3 (AP): APIC ID: 3 kernel: cpu4 (AP): APIC ID: 4 kernel: cpu5 (AP): APIC ID: 5 kernel: cpu6 (AP): APIC ID: 6 kernel: cpu7 (AP): APIC ID: 7 kernel: ioapic0 irqs 0-23 on motherboard kernel: random: initialized kernel: module_register_init: MOD_LOAD (vesa, 0xffffffff80b6a090, 0) error 19 kernel: kbd1 at kbdmux0 kernel: module_register_init: MOD_LOAD (accf_data, 0xffffffff807e9af0, 0xffffffff8155d0c8) error 17 kernel: acpi0: on motherboard kernel: acpi0: Power Button (fixed) kernel: cpu0: on acpi0 kernel: cpu1: on acpi0 kernel: cpu2: on acpi0 kernel: cpu3: on acpi0 kernel: cpu4: on acpi0 kernel: cpu5: on acpi0 kernel: cpu6: on acpi0 kernel: cpu7: on acpi0 kernel: hpet0: iomem 0xfed00000-0xfed003ff on acpi0 kernel: Timecounter "HPET" frequency 14318180 Hz quality 950 kernel: Event timer "HPET" frequency 14318180 Hz quality 550 kernel: atrtc0: port 0x70-0x77 irq 8 on acpi0 kernel: Event timer "RTC" frequency 32768 Hz quality 0 kernel: attimer0: port 0x40-0x43,0x50-0x53 irq 0 on acpi0 kernel: Timecounter "i8254" frequency 1193182 Hz quality 0 kernel: Event timer "i8254" frequency 1193182 Hz quality 100 kernel: Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 kernel: acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1808-0x180b on acpi0 kernel: pcib0: port 0xcf8-0xcff on acpi0 kernel: pci0: on pcib0 kernel: xhci0: mem 0xf7220000-0xf722ffff irq 16 at device 20.0 on pci0 kernel: xhci0: 32 bytes context size, 64-bit DMA kernel: xhci0: Port routing mask set to 0xffffffff kernel: usbus0 on xhci0 kernel: em0: port 0xf020-0xf03f mem 0xf7200000-0xf721ffff,0xf7235000-0xf7235fff irq 20 at device 25.0 on pci0 kernel: em0: Using an MSI interrupt kernel: em0: Ethernet address: 00:25:90:47:42:57 kernel: ehci0: mem 0xf7234000-0xf72343ff irq 16 at device 26.0 on pci0 kernel: usbus1: EHCI version 1.0 kernel: usbus1 on ehci0 kernel: pcib1: irq 16 at device 28.0 on pci0 kernel: pci1: on pcib1 kernel: pcib2: at device 0.0 on pci1 kernel: pci2: on pcib2 kernel: vgapci0: port 0xe000-0xe07f mem 0xf6000000-0xf6ffffff,0xf7000000-0xf701ffff irq 16 at device 0.0 on pci2 kernel: vgapci0: Boot video device kernel: pcib3: irq 17 at device 28.1 on pci0 kernel: pci3: on pcib3 kernel: igb0: port 0xd000-0xd01f mem 0xf7100000-0xf717ffff,0xf7180000-0xf7183fff irq 17 at device 0.0 on pci3 kernel: igb0: Using MSIX interrupts with 5 vectors kernel: igb0: Ethernet address: 00:25:90:47:42:56 kernel: igb0: Bound queue 0 to cpu 0 kernel: igb0: Bound queue 1 to cpu 1 kernel: igb0: Bound queue 2 to cpu 2 kernel: igb0: Bound queue 3 to cpu 3 kernel: ehci1: mem 0xf7233000-0xf72333ff irq 23 at device 29.0 on pci0 kernel: usbus2: EHCI version 1.0 kernel: usbus2 on ehci1 kernel: isab0: at device 31.0 on pci0 kernel: isa0: on isab0 kernel: ahci0: port 0xf070-0xf077,0xf060-0xf063,0xf050-0xf057,0xf040-0xf043,0xf000-0xf01f mem 0xf7232000-0xf72327ff irq 19 at device 31.2 on pci0 kernel: ahci0: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported kernel: ahcich0: at channel 0 on ahci0 kernel: ahcich1: at channel 1 on ahci0 kernel: ahcich2: at channel 2 on ahci0 kernel: ahcich3: at channel 3 on ahci0 kernel: ahcich4: at channel 4 on ahci0 kernel: ahcich5: at channel 5 on ahci0 kernel: ahciem0: on ahci0 kernel: acpi_button0: on acpi0 kernel: acpi_button1: on acpi0 kernel: acpi_tz0: on acpi0 kernel: acpi_tz1: on acpi0 kernel: uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 kernel: uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 kernel: orm0: at iomem 0xc0000-0xc7fff,0xc8000-0xc8fff on isa0 kernel: sc0: at flags 0x100 on isa0 kernel: sc0: CGA <16 virtual consoles, flags=0x300> kernel: vga0: at port 0x3d0-0x3db iomem 0xb8000-0xbffff on isa0 kernel: ppc0: cannot reserve I/O port range kernel: est0: on cpu0 kernel: est1: on cpu1 kernel: est2: on cpu2 kernel: est3: on cpu3 kernel: est4: on cpu4 kernel: est5: on cpu5 kernel: est6: on cpu6 kernel: est7: on cpu7 kernel: random: unblocking device. kernel: usbus0: 5.0Gbps Super Speed USB v3.0 kernel: Timecounters tick every 1.000 msec kernel: ipfw2 (+ipv6) initialized, divert enabled, nat loadable, default to accept, logging disabled kernel: DUMMYNET 0 with IPv6 initialized (100409) kernel: load_dn_sched dn_sched FIFO loaded kernel: load_dn_sched dn_sched PRIO loaded kernel: load_dn_sched dn_sched QFQ loaded kernel: load_dn_sched dn_sched RR loaded kernel: load_dn_sched dn_sched WF2Q+ loaded kernel: usbus1: 480Mbps High Speed USB v2.0 kernel: usbus2: 480Mbps High Speed USB v2.0 kernel: ugen1.1: at usbus1 kernel: uhub0: on usbus1 kernel: ugen0.1: <0x8086> at usbus0 kernel: ugen2.1: at usbus2 kernel: uhub1: on usbus2 kernel: uhub2: <0x8086 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0 kernel: ses0 at ahciem0 bus 0 scbus6 target 0 lun 0 kernel: ses0: SEMB S-E-S 2.00 device kernel: ses0: SEMB SES Device kernel: ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 kernel: ada0: ATA-8 SATA 3.x device kernel: ada0: Serial Number 64O4281GS kernel: ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) kernel: ada0: Command Queueing enabled kernel: ada0: 2861588MB (5860533168 512 byte sectors: 16H 63S/T 16383C) kernel: ada0: Previously was known as ad4 kernel: ada1 at ahcich1 bus 0 scbus1 target 0 lun 0 kernel: ada1: ATA-8 SATA 3.x device kernel: ada1: Serial Number 64O43WZGS kernel: ada1: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) kernel: ada1: Command Queueing enabled kernel: ada1: 2861588MB (5860533168 512 byte sectors: 16H 63S/T 16383C) kernel: ada1: Previously was known as ad6 kernel: SMP: AP CPU #1 Launched! kernel: SMP: AP CPU #4 Launched! kernel: SMP: AP CPU #2 Launched! kernel: SMP: AP CPU #3 Launched! kernel: SMP: AP CPU #6 Launched! kernel: SMP: AP CPU #7 Launched! kernel: SMP: AP CPU #5 Launched! kernel: Timecounter "TSC-low" frequency 1750036200 Hz quality 1000 kernel: GEOM_MIRROR: Device mirror/boot launched (2/2). kernel: GEOM_MIRROR: Device mirror/swap launched (2/2). kernel: GEOM_MIRROR: Device mirror/root launched (2/2). kernel: Root mount waiting for: usbus2 usbus1 usbus0 kernel: uhub2: 17 ports with 17 removable, self powered kernel: uhub0: 2 ports with 2 removable, self powered kernel: uhub1: 2 ports with 2 removable, self powered kernel: Root mount waiting for: usbus2 usbus1 usbus0 kernel: ugen0.2: at usbus0 kernel: uhub3: on usbus0 kernel: uhub3: 4 ports with 3 removable, self powered kernel: ugen1.2: at usbus1 kernel: uhub4: on usbus1 kernel: ugen2.2: at usbus2 kernel: uhub5: on usbus2 kernel: uhub4: 4 ports with 4 removable, self powered kernel: uhub5: 6 ports with 6 removable, self powered kernel: ugen0.3: at usbus0 kernel: ukbd0: on usbus0 kernel: kbd0 at ukbd0 kernel: ums0: on usbus0 kernel: ums0: 3 buttons and [Z] coordinates ID=0 kernel: Trying to mount root from ufs:/dev/mirror/root [rw,async,groupquota,noatime]... > > >> 2) I consider it a sever bug, because even normal used (and i have >> plenty of them using ssh) can eventually do rm -rf and kill all >> sites. Which means there are must be some way to limit io usage per >> user >> > > ZFS has far superior disk IO scheduling. You may have more success > with that. Unfortunately I'm not aware of a way to limit user/process > disk IO on FreeBSD. I hesitatated migrating to ZFS because "don't fix it if it ain't broken". And it was not broken.. until now. But now i cannot migrate to ZFS - server is too busy and actually i am afraid something bad will come up with ZFS eventually too. Well, if not limit, then maybe just monitor and kill user processes that create problems. But the problem is how to determine which processes overuse the io system. I have very little clue how to collect such information. Artem