Date: Fri, 19 Jul 2002 09:09:59 -0500 From: "Jay West" <jwest@classiccmp.org> To: "Dan Nelson" <dnelson@allantgroup.com> Cc: <freebsd-questions@FreeBSD.ORG> Subject: Re: Repost: Why are IPC message queues incredibly slow? Message-ID: <007601c22f2d$f7d37760$9701a8c0@HPLAPTOP> References: <01b501c22e72$acb7a0a0$9701a8c0@HPLAPTOP> <20020718165230.GC41869@dan.emsphone.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Dan wrote... > You can try freebsd-hackers, but I don't think sysv queues are used > very often at all. You say there's a 1-sec delay when sending a > message? Have you ktrace'd both client and server and verified that > msgsnd/msgrcv are really the cause of the delay? Can you reproduce the > problem with a pair of minimal programs? I have not ktraced both client and server. However, I did copy the programs in question and scale them down. The programs are doing so little else that I can't believe it is anything other than the IPC stuff. Here's a more verbose description of what happens in the code... The executive process comes up and opens a message queue. It then reads a config file that lists the child processes which need to be started, and what "driver" program they should execute. In my stripped down example there is only one child process. So the executive forks the child, and when the child comes up it opens the same message queue as the executive process. Then the executive waits for commands from the keyboard. One of the commands selects which child process to communicate with, and from that point on, all keyboard entries are sent to the message queue. The child process continually checks the message queue with IPC_NOWAIT until it finds a non-zero sized message. Once it finds one (in the stripped down example), it sends a single reply message to the queue tagged with the parent process id which says "OK". The conversations are always driven from the executive process, so the sequence is as follows... executive sends a message to the queue, and then waits for a response. This is a timed wait, up to 5 seconds. When it gets a response message from the queue, it prints it to the console. child constantly scans the message queue with NO_WAIT in a loop. Soon as it sees a message it gets it, then puts a message on the queue with the ppid of "OK". It then returns to the waiting for messages loop. Here's the slight wrinkle... in the executive, there is a command which calls a "command input file". It then reads the commands in that file one line at a time and sends them to the child process, waiting for each reply and then printing it. In my scaled down example, this file is just about 30 lines of text. When I type "call comfile.com" it starts reading each line, sending it to the child, and then the child sends back "OK" which it prints on the screen of the executive. When this command file is run, the only things involved are reading a file line at a time, put on queue, wait on queue, and print. I find it hard to believe that it is the file I/O or print functions taking so long. That pretty much only leaves the IPC stuff. When the command file is running, It is about .5 seconds per line up to 1 second per line. That just seems way slow. Hence, I was wondering simply if SYSV IPC message queues on FreeBSD is known to be slow. If that's the case, I was going to switch to sockets perhaps. Somewhere on the web I came across a program that benchmarks sending the same data several ways - sockets, pipes, several IPC methods, etc. Unfortunately I lost the URL and can't find that benchmark program anymore. Because the data being transferred between processes is at times being used to simulate DMA transfers between "virtual computer peripherals", I really need about 2mbits per second of throughput just to match the original hardware, and I'm afraid the message queues just aren't fast enough. To get that speed I may have to write my own message exchange structures with semaphores and shared memory (ie. build my own messaging mechanism), but I'd rather not do that if there is just something that can be tweaked with message queues. More to the point, if message queues are slow only on FreeBSD, I would think the core team might should take a look at that because there are certainly some packages which make use of them that would be hindered on FreeBSD. Thanks for any and all advice! Jay West --- [This E-mail scanned for viruses by Declude Virus] To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?007601c22f2d$f7d37760$9701a8c0>