From owner-freebsd-hackers@FreeBSD.ORG Tue Dec 14 19:47:56 2004 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 57C4016A4CE for ; Tue, 14 Dec 2004 19:47:56 +0000 (GMT) Received: from rproxy.gmail.com (rproxy.gmail.com [64.233.170.202]) by mx1.FreeBSD.org (Postfix) with ESMTP id BD6A943D1F for ; Tue, 14 Dec 2004 19:47:26 +0000 (GMT) (envelope-from sferris@gmail.com) Received: by rproxy.gmail.com with SMTP id q1so431224rnf for ; Tue, 14 Dec 2004 11:47:15 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=QYXsACkluDaYocwELLxjISXfhWb+Cyk/HmlMMeSI1i0p/c4ixkncmZVzlygfKxHFAn51Bu+YPFz09tOsm7/Zn3feFheRl6PWB3lDItK8Usv9dA+h6PahHEwX06XlE0QUvae6d++4zlwz+DZZWk5xrOy9eggg+dKh+MBF0KWx+bc= Received: by 10.38.83.22 with SMTP id g22mr2392564rnb; Tue, 14 Dec 2004 11:47:15 -0800 (PST) Received: by 10.38.83.5 with HTTP; Tue, 14 Dec 2004 11:47:15 -0800 (PST) Message-ID: <1eea89cd041214114766fd34dc@mail.gmail.com> Date: Tue, 14 Dec 2004 13:47:15 -0600 From: "Scott M. Ferris" To: Danny Braniss In-Reply-To: <20041214072922.2604543D1D@mx1.FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit References: <20041214072922.2604543D1D@mx1.FreeBSD.org> cc: hackers@freebsd.org cc: Peter Blok Subject: Re: My project wish-list for the next 12 months X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: "Scott M. Ferris" List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Dec 2004 19:47:56 -0000 On Tue, 14 Dec 2004 09:29:19 +0200, Danny Braniss wrote: > Great!, we seem to be on the same wavelength, im now writing (at about one > char a minute) the login user program, and somehow - to be discovered -, the > socket will be passed to the kernel. > my main efford, at the moment, is a) to &^%$$## understand the RFC (i think > they used a scrambler) and b) define the data structures. How do you plan on handling cases where the user program blocks and can't login again (because of a page fault for code or data, allocating a new socket in the kernel, allocating a new socket buffer in the kernel, etc)? One of the major problems any software-only iSCSI initiator has is dealing with memory deadlocks. The OS may try to write out one or more pages in order to free up memory. If the iSCSI initiator needs to allocate memory (directly or indirectly in the TCP stack) in order to complete that write, you've got a circular dependency where in order to get free memory you need to have free memory. Hardware-based iSCSI HBAs solve this by having their own memory and TCP stack separate from the OS. Software-only iSCSI initiators such as linux-iscsi usually just hope it doesn't happen, and that's why I don't usually recommend software-only iSCSI initiators to anyone. Having a user program login is fine for SendTargets discovery and testing, but if you're planning on getting a reliable iSCSI initiator, you'll probably need to do the login from within the kernel, and make sure you have all of the resources needed to do that reserved (wire pages into RAM, etc). The hard part of that is making sure you have all the resources needed to send and receive TCP packets reserved, as well as the resources to establish new connections in case the existing connections is closed or aborted. The linux-iscsi initiator doesn't even attempt this, and just hopes deadlocks don't happen very often. I used to be the main (and for most of that time, only) developer for the linux-iscsi initiator. When I first took over the initiator, it used the approach of doing iSCSI session login from a daemon, and passing the socket down into a kernel module. I had to take that out because it deadlocked too easily. I wouldn't recommend that approach. -- Scott M. Ferris