Date: Fri, 8 Mar 2002 08:45:21 +0200 From: Dimitar Peikov <mitko@rila.bg>(by way of Dimitar Peikov <mitko@rila.bg>) To: Terry Lambert <tlambert2@mindspring.com> Subject: Re: Swapping performance Message-ID: <20020308084521.65ef47a8.mitko@rila.bg> In-Reply-To: <3C87BD3D.49BE8105@mindspring.com> References: <20020307095452.D18855-100000@frond.minions.com> <5.1.0.14.0.20020307193205.01c3b0f0@mail.drwilco.net> <3C87BD3D.49BE8105@mindspring.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, 07 Mar 2002 11:19:25 -0800
Terry Lambert <tlambert2@mindspring.com> wrote:
> > #include <stdlib.h>
> > #include <string.h>
> > #include <stdio.h>
> >
> > #define MALLOC_SIZE 1024*1024*256
> >
> > int main(int argc, char **argv) {
> > char *ptr;
> > int i, i_count;
> > int j;
> >
> > ptr = (char *) malloc(MALLOC_SIZE);
> > bzero(ptr, MALLOC_SIZE);
>
> The bzero is unnecessary on FreeBSD. Allocated pages start out
> zero'ed. Part of the performance issue might be that FreeBSD is
> being asked to zero the pages twice, instead of once.
>
> > i_count = MALLOC_SIZE / 16;
> > fprintf(stderr, "*");
> > for (i = 0; i < i_count; i ++) {
> > ptr[i >> 4] = ptr[(i >> 3) + 1]++;
> **
> Is this what was intended? This will keep the lvalue in the
> first 256th of memory, and the rvalue in the first half.
>
> > }
> > fprintf(stderr, "#");
> > for (j = 0; j < i_count; j ++) {
> > ptr[j << 4] = ptr[(j >> 3) + 1]++;
> > }
>
> This looks more right... the lvalue hits every page, and the
> rvalue is (still) limited to the first half of memory.
>
> >
> > free(ptr);
>
> Try just exiting; free() might be trying to be too clever
> on FreeBSD. Doing the same on both platforms should not
> invalidate the test.
>
> > return 0;
> > }
>
> It might be more interesting to mmap() anonymous memory
> (e.g. out of /dev/zero), rather than using malloc, and
> then use that memory the same way, instead (it's swap
> backed, as well). Giving it an madvise, to tell it your
> intended access pattern would also be useful.
>
> I'm also not sure if the Linux VM "prefers" code pages to
> data pages. It could very well be the case that it does;
> that would result in a potentiall much better performance
> on this test. It would also be useful to know the relative
> load addresses in the code; FreeBSD might end up having the
> code compiled to be split across cache lines, etc, where
> the Linux version of the code does not.
>
> As other people pointed out, the "speed" of the swap
> partition is also potentially an issue. Make sure that
> FreBSD and Linux are using the same region relative to the
> drive spindle for swap.
>
> In addition, you should verify that the hardware is, in
> fact, identical. The easiest way to do this would be to
> swap drives in the machines to see if the affects the results,
> and -- while they are out -- verify that the drives themselves
> are as identical as possible.
>
> You might also want to turn *off* the FreeBSD "harvesting
> entropy", if it's doing it on disk interrupts or other
> interrupts inportant to the code path.
>
> If all this still has the FreeBSD slower, it still,
> unfortunately, isn't conclusive, one way or the other; it
> could still be that what you are testing is not the swap
> performance of FreeBSD, but, perhaps, the differences in
> the disk device drivers for the particular controller you
> ended up with in the motherboard lottery.
>
> It would also be useful to see the verbose dmesg output
> for both systems for the devices involved in the test code
> path. It may be that FreeBSD is working around a known
> controller issue (e.g. the CMD640B IDE controller), etc.,
> where interleaved I/O's are being permitted by Linux, but
> not by FreeBSD, for fear of data corruption.
>
> Finally... you might want to try the Linux version of the
> program on FreeBSD, rather than using a native FreeBSD
> version. THis should account for many things, including
> the sync'ing of mapping regions, if that's a factor.
>
> All this information is useless to me personally; please
> post to the list, if you post at all, thanks.
>
> -- Terry
In fact I was starting these tests as a result of heavy mathematical
operations (composing, compacting, minimizing automata's with a lot of
states and much much more arcs between them). Our application that deals
with this is written in C and is not optimized to any particular OS. If I
had to optimize it for Linux or FreeBSD there certainly would be major
changes, but our team uses not only *BSD. So when we start to build our
automata's some of them require more than 1G to be created and most of the
memory has temporary usage (there are most malloc, realloc, calloc, free
operatons) because of atomic operations in algorithms for making automata's.
And when we saw that one think on Linux was done for a few hours, but on
FreeBSD much, much more, I start tests to investigate where the problem is.
This test (malloc_test) is the first think that gives serious difference and
I thought that the problem is in swapping or paging. I understand that there
are a lot of optimizations that can be made if have more time for that, but
even in this case I don't think that the things would be much more bright
for the FreeBSD.
--
Dimitar Peikov
Programmer Analyst
Globalization Group
"We Build e-Business"
RILA Solutions
27 Building, Acad.G.Bonchev Str.
1113 Sofia, Bulgaria
phone: (+359 2) 9797320
phone: (+359 2) 9797300
fax: (+359 2) 9733355
http://www.rila.com
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20020308084521.65ef47a8.mitko>