Date: Tue, 3 Nov 1998 16:22:57 -0800 (PST) From: Archie Cobbs <archie@whistle.com> To: FreeBSD-gnats-submit@FreeBSD.ORG Subject: bin/8566: mergesort() core dumps if number of elements is zero Message-ID: <199811040022.QAA16848@bubba.whistle.com>
next in thread | raw e-mail | index | archive | help
>Number: 8566
>Category: bin
>Synopsis: mergesort() core dumps if number of elements is zero
>Confidential: no
>Severity: serious
>Priority: medium
>Responsible: freebsd-bugs
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: sw-bug
>Submitter-Id: current-users
>Arrival-Date: Tue Nov 3 16:30:01 PST 1998
>Last-Modified:
>Originator: Archie Cobbs
>Organization:
Whistle Communications, Inc.
>Release: FreeBSD 2.2.6-RELEASE i386
>Environment:
FreeBSD 2.2-stable AND 3.0-current
>Description:
If meregsort() is called with number of elements == 0,
it will still try to compare them (!), causing a core
dump or worse.
>How-To-Repeat:
Compile and run this program:
#include <stdlib.h>
#include <stdio.h>
static int
compare(const void *v1, const void *v2)
{
return(*((int *) v1) - *((int *) v2));
}
int
main(void)
{
int *array = NULL;
if (mergesort(array, 0, 4, compare) < 0)
err(1, "mergesort");
}
>Fix:
NOTE: Please fix this in BOTH 2.2 and 3.0.. thanks!
Index: merge.c
===================================================================
RCS file: /cvs/freebsd/src/lib/libc/stdlib/merge.c,v
retrieving revision 1.2
diff -u -r1.2 merge.c
--- merge.c 1995/05/30 05:41:50 1.2
+++ merge.c 1998/11/04 00:19:56
@@ -111,6 +111,9 @@
return (-1);
}
+ if (nmemb == 0)
+ return (0);
+
/*
* XXX
* Stupid subtraction for the Cray.
>Audit-Trail:
>Unformatted:
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-bugs" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199811040022.QAA16848>