From owner-freebsd-stable@FreeBSD.ORG Sat Sep 29 02:19:37 2012 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id F078E1065675 for ; Sat, 29 Sep 2012 02:19:36 +0000 (UTC) (envelope-from spawk@acm.poly.edu) Received: from acm.poly.edu (acm.poly.edu [64.147.113.26]) by mx1.freebsd.org (Postfix) with ESMTP id 924158FC18 for ; Sat, 29 Sep 2012 02:19:36 +0000 (UTC) Received: (qmail 22010 invoked from network); 29 Sep 2012 02:12:54 -0000 Received: from unknown (HELO ?192.168.67.24?) (spawk@64.147.100.14) by acm.poly.edu with CAMELLIA256-SHA encrypted SMTP; 29 Sep 2012 02:12:54 -0000 Message-ID: <50665914.8080301@acm.poly.edu> Date: Fri, 28 Sep 2012 22:12:36 -0400 From: Boris Kochergin User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:10.0.5) Gecko/20120716 Thunderbird/10.0.5 MIME-Version: 1.0 To: FreeBSD-STABLE Mailing List Content-Type: multipart/mixed; boundary="------------000803050604010303050807" Subject: qmail-remote uses 100% CPU when delivering to certain hosts X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Sep 2012 02:19:37 -0000 This is a multi-part message in MIME format. --------------000803050604010303050807 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Hi. Firstly, my environment: FreeBSD isis.poly.edu 9.1-RC1 FreeBSD 9.1-RC1 #0: Thu Sep 20 23:42:23 EDT 2012 boris@jails.isis.poly.edu:/usr/obj/usr/src/sys/GENERIC amd64 qmail-tls-1.03.20021228_4 I have noticed that qmail-remote spins around using 100% CPU forever when delivering mail to a select few hosts. I believe this all to be in userspace as top reports that the CPU is busy with only userspace code. This has never happened when the machine was running 8.0-RELEASE, but happens with 8.2-RELEASE and 9.1-RC1. The environment this happens in is a jail. Calling truss on the offending processes produces no output, with truss in the "wait" state forever. I've tcpdumped the conversation between the qmail-remote process and the remote host. It is attached with this e-mail. Thoughts? -Boris --------------000803050604010303050807 Content-Type: application/octet-stream; name="qmail-remote.pcap" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="qmail-remote.pcap" 1MOyoQIABAAAAAAAAAAAAP//AAABAAAAIlFmUCdfDQBKAAAASgAAAAADn4lgCAAwSLpuJAgA RQAAPG1nQABABrQsgO5CDxIHRCSn0QAZeU8N3QAAAACgAv//v2cAAAIEBbQBAwMGBAIICiUi GzgAAAAAIlFmUEC/DQBKAAAASgAAAAAwSLpuJAADn4lgCAgARQAAPAAAQAAyBi+UEgdEJIDu Qg8AGafRUUNhXnlPDd6gEhaguE4AAAIEBWQEAggKM/8KFiUiGzgBAwMHIlFmUAXADQBCAAAA QgAAAAADn4lgCAAwSLpuJAgARQAANG1oQABABrQzgO5CDxIHRCSn0QAZeU8N3lFDYV+AEAQC +U8AAAEBCAolIhtRM/8KFiJRZlBCNg4AhgAAAIYAAAAAMEi6biQAA5+JYAgIAEUAAHiqQkAA MgaFFRIHRCSA7kIPABmn0VFDYV95Tw3egBgALsmYAAABAQgKM/8KNCUiG1EyMjAgZG16LW1h aWxzZWMtc2Nhbm5lci03Lm1pdC5lZHUgRVNNVFAgU3ltYW50ZWMgTWVzc2FnaW5nIEdhdGV3 YXkNCiJRZlCcNw4AVgAAAFYAAAAAA5+JYAgAMEi6biQIAEUAAEhtakAAQAa0HYDuQg8SB0Qk p9EAGXlPDd5RQ2GjgBgEAuNMAAABAQgKJSIbcDP/CjRFSExPIGlzaXMucG9seS5lZHUNCiJR ZlDhmQ4AQgAAAEIAAAAAMEi6biQAA5+JYAgIAEUAADSqQ0AAMgaFWBIHRCSA7kIPABmn0VFD YaN5Tw3ygBAALvx1AAABAQgKM/8KTSUiG3AiUWZQMpoOANEAAADRAAAAADBIum4kAAOfiWAI CABFAADDqkRAADIGhMgSB0QkgO5CDwAZp9FRQ2GjeU8N8oAYAC6bQQAAAQEICjP/Ck4lIhtw MjUwLWRtei1tYWlsc2VjLXNjYW5uZXItNy5taXQuZWR1IHNheXMgRUhMTyB0byAxMjguMjM4 LjY2LjE1OjQyOTYxDQoyNTAtOEJJVE1JTUUNCjI1MC1TSVpFICA0MTk0MzA0MA0KMjUwLUVO SEFOQ0VEU1RBVFVTQ09ERVMNCjI1MCBQSVBFTElOSU5HDQojUWZQON0AAEIAAABCAAAAAAOf iWAIADBIum4kCABFAAA0bW5AAEAGtC2A7kIPEgdEJKfRABl5Tw3yUUNiMoAQBAL3lAAAAQEI CiUiG+0z/wpOM1FmUKnRCABKAAAASgAAAAADn4lgCAAwSLpuJAgARQAAPHyTQABABtATgO5C DxIJGQ9EHQAZfQo7gAAAAACgAv//25IAAAIEBbQBAwMGBAIICiUiXHYAAAAAM1FmUPoyCQBK AAAASgAAAAAwSLpuJAADn4lgCAgARQAAPAAAQAAyBlqnEgkZD4DuQg8AGUQdOYLrkX0KO4Gg Ehag73EAAAIEBWQEAggKNAh8oiUiXHYBAwMHM1FmUAUzCQBCAAAAQgAAAAADn4lgCAAwSLpu JAgARQAANHycQABABtASgO5CDxIJGQ9EHQAZfQo7gTmC65KAEAQCMHMAAAEBCAolIlyPNAh8 ojNRZlD6qQoASgAAAEoAAAAAA5+JYAgAMEi6biQIAEUAADx8tkAAQAbP8IDuQg8SCRkPfN0A GezlMg8AAAAAoAL//zvvAAACBAW0AQMDBgQCCAolIlzvAAAAADNRZlBhCwsASgAAAEoAAAAA MEi6biQAA5+JYAgIAEUAADwAAEAAMgZapxIJGQ+A7kIPABl83WnU6kLs5TIQoBIWoCBSAAAC BAVkBAIICjQIfRslIlzvAQMDBzNRZlBuCwsAQgAAAEIAAAAAA5+JYAgAMEi6biQIAEUAADR8 vEAAQAbP8oDuQg8SCRkPfN0AGezlMhBp1OpDgBAEAmFTAAABAQgKJSJdCDQIfRszUWZQyJsL AIYAAACGAAAAADBIum4kAAOfiWAICABFAAB4iSxAADIG0T4SCRkPgO5CDwAZRB05guuSfQo7 gYAYAC4DPQAAAQEICjQIfT8lIlyPMjIwIGRtei1tYWlsc2VjLXNjYW5uZXItNC5taXQuZWR1 IEVTTVRQIFN5bWFudGVjIE1lc3NhZ2luZyBHYXRld2F5DQozUWZQ5JsLAFYAAABWAAAAAAOf iWAIADBIum4kCABFAABIfMRAAEAGz9aA7kIPEgkZD0QdABl9CjuBOYLr1oAYBAIZcgAAAQEI CiUiXS00CH0/RUhMTyBpc2lzLnBvbHkuZWR1DQozUWZQrZ8LAF0AAABdAAAAADBIum4kAAOf iWAICABFAABPX/ZAADIG+p0SCRkPgO5CDwAZfN1p1OpD7OUyEIAYAC7rtAAAAQEICjQIfUAl Il0INDUwIDQuMy4yIHRyeSBhZ2FpbiBsYXRlcg0KM1FmULSfCwBCAAAAQgAAAAAwSLpuJAAD n4lgCAgARQAANF/3QAAyBvq3EgkZD4DuQg8AGXzdadTqXuzlMhCAEQAuZOYAAAEBCAo0CH1A JSJdCDNRZlC8nwsAQgAAAEIAAAAAA5+JYAgAMEi6biQIAEUAADR8xUAAQAbP6YDuQg8SCRkP fN0AGezlMhBp1OpfgBAEAWDtAAABAQgKJSJdLjQIfUAzUWZQx58LAEgAAABIAAAAAAOfiWAI ADBIum4kCABFAAA6fMZAAEAGz+KA7kIPEgkZD3zdABns5TIQadTqX4AYBAK5KgAAAQEICiUi XS40CH1AUVVJVA0KM1FmUOifCwBCAAAAQgAAAAADn4lgCAAwSLpuJAgARQAANHzHQABABs/n gO5CDxIJGQ983QAZ7OUyFmnU6l+AEQQCYOUAAAEBCAolIl0uNAh9QDNRZlBg/QsAQgAAAEIA AAAAMEi6biQAA5+JYAgIAEUAADSJLUAAMgbRgRIJGQ+A7kIPABlEHTmC69Z9CjuVgBAALjKa AAABAQgKNAh9WSUiXS0zUWZQZv0LANEAAADRAAAAADBIum4kAAOfiWAICABFAADDiS5AADIG 0PESCRkPgO5CDwAZRB05guvWfQo7lYAYAC6hmQAAAQEICjQIfVklIl0tMjUwLWRtei1tYWls c2VjLXNjYW5uZXItNC5taXQuZWR1IHNheXMgRUhMTyB0byAxMjguMjM4LjY2LjE1OjE3NDM3 DQoyNTAtOEJJVE1JTUUNCjI1MC1TSVpFICA0MTk0MzA0MA0KMjUwLVBJUEVMSU5JTkcNCjI1 MCBFTkhBTkNFRFNUQVRVU0NPREVTDQozUWZQRgEMADwAAAA8AAAAADBIum4kAAOfiWAICABF AAAoAABAADIGWrsSCRkPgO5CDwAZfN1p1Opf8zWSFVAEAABrVQAAAAAAAAAAM1FmUPGCDQBC AAAAQgAAAAADn4lgCAAwSLpuJAgARQAANHziQABABs/MgO5CDxIJGQ9EHQAZfQo7lTmC7GWA EAQCLboAAAEBCAolIl2qNAh9WU5SZlC6oA4AYAAAAGAAAAAAMEi6biQAA5+JYAgIAEUAAFKq RUAAMgaFOBIHRCSA7kIPABmn0VFDYjJ5Tw3ygBgALnybAAABAQgKNAOeJSUiG+00MjEgNC40 LjIgc2VydmljZSB0aW1lZCBvdXQuDQpOUmZQwqAOAEIAAABCAAAAADBIum4kAAOfiWAICABF AAA0qkZAADIGhVUSB0QkgO5CDwAZp9FRQ2JQeU8N8oARAC5nbgAAAQEICjQDniUlIhvtTlJm UMqgDgBCAAAAQgAAAAADn4lgCAAwSLpuJAgARQAANIjKQABABpjRgO5CDxIHRCSn0QAZeU8N 8lFDYlGAEAQB0BgAAAEBCAolJq9rNAOeJV9SZlCFAAwAYAAAAGAAAAAAMEi6biQAA5+JYAgI AEUAAFKJL0AAMgbRYRIJGQ+A7kIPABlEHTmC7GV9CjuVgBgALrLEAAABAQgKNA0RLCUiXao0 MjEgNC40LjIgc2VydmljZSB0aW1lZCBvdXQuDQpfUmZQjQAMAEIAAABCAAAAADBIum4kAAOf iWAICABFAAA0iTBAADIG0X4SCRkPgO5CDwAZRB05guyDfQo7lYARAC6dlwAAAQEICjQNESwl Il2qX1JmUJUADABCAAAAQgAAAAADn4lgCAAwSLpuJAgARQAANJwVQABABrCZgO5CDxIJGQ9E HQAZfQo7lTmC7ISAEAQBBkMAAAEBCAolJvEnNA0RLJRTZlCIygUAQgAAAEIAAAAAA5+JYAgA MEi6biQIAEUAADQs2EAAQAYf14DuQg8SCRkPRB0AGX0KO5U5guyEgBEEAlDLAAABAQgKJSum mDQNESyUU2ZQ8soFAEIAAABCAAAAAAOfiWAIADBIum4kCABFAAA0LNlAAEAG9MKA7kIPEgdE JKfRABl5Tw3yUUNiUYARBALY5AAAAQEICiUrppg0A54llFNmUN4rBgA8AAAAPAAAAAAwSLpu JAADn4lgCAgARQAAKAAAQAAyBi+oEgdEJIDuQg8AGafRUUNiURgDghtQBAAAoRoAAAAAAAAA AJRTZlDnKwYAPAAAADwAAAAAMEi6biQAA5+JYAgIAEUAACgAAEAAMgZauxIJGQ+A7kIPABlE HTmC7ITa5c4wUAQAAK53AAAAAAAAAAA= --------------000803050604010303050807--