Date: Sun, 12 Dec 2010 22:40:12 +0200 From: Andriy Gapon <avg@freebsd.org> To: =?x-viet-vps?Q?Martin_Matus=28ka?= <mm@freebsd.org>, freebsd-multimedia@freebsd.org Subject: ffmpeg and mmap Message-ID: <4D05332C.7010208@freebsd.org>
next in thread | raw e-mail | index | archive | help
I've been getting some crash dumps in libswscale.so code.
The stack trace is always like this:
#0 0x000000083a6abf10 in ?? ()
#1 0x000000080a717dc6 in hyscale_fast_MMX2
#2 0x000000080a71bd64 in swScale_MMX2
#3 0x000000080a71ebf9 in sws_scale
...
>From disassembling I've identified that the crash happens as soon as inline
assembly in hyscale_fast_MMX2 calls code pointed to by lumMmx2FilterCode pointer.
The following code in libswscale/utils.c, function sws_getContext() is of interest:
#if ARCH_X86 && (HAVE_MMX2 || CONFIG_RUNTIME_CPUDETECT)
// can't downscale !!!
if (c->canMMX2BeUsed && (flags & SWS_FAST_BILINEAR)) {
c->lumMmx2FilterCodeSize = initMMX2HScaler( dstW, c->lumXInc,
NULL, NULL, NULL, 8);
c->chrMmx2FilterCodeSize = initMMX2HScaler(c->chrDstW, c->chrXInc,
NULL, NULL, NULL, 4);
#ifdef MAP_ANONYMOUS
c->lumMmx2FilterCode = mmap(NULL, c->lumMmx2FilterCodeSize,
PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
c->chrMmx2FilterCode = mmap(NULL, c->chrMmx2FilterCodeSize,
PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
#elif HAVE_VIRTUALALLOC
c->lumMmx2FilterCode = VirtualAlloc(NULL, c->lumMmx2FilterCodeSize,
MEM_COMMIT, PAGE_EXECUTE_READWRITE);
c->chrMmx2FilterCode = VirtualAlloc(NULL, c->chrMmx2FilterCodeSize,
MEM_COMMIT, PAGE_EXECUTE_READWRITE);
#else
c->lumMmx2FilterCode = av_malloc(c->lumMmx2FilterCodeSize);
c->chrMmx2FilterCode = av_malloc(c->chrMmx2FilterCodeSize);
#endif
if (!c->lumMmx2FilterCode || !c->chrMmx2FilterCode)
goto fail;
FF_ALLOCZ_OR_GOTO(c, c->hLumFilter , (dstW
/8+8)*sizeof(int16_t), fail);
FF_ALLOCZ_OR_GOTO(c, c->hChrFilter , (c->chrDstW
/4+8)*sizeof(int16_t), fail);
FF_ALLOCZ_OR_GOTO(c, c->hLumFilterPos, (dstW
/2/8+8)*sizeof(int32_t), fail);
FF_ALLOCZ_OR_GOTO(c, c->hChrFilterPos,
(c->chrDstW/2/4+8)*sizeof(int32_t), fail);
initMMX2HScaler( dstW, c->lumXInc, c->lumMmx2FilterCode,
c->hLumFilter, c->hLumFilterPos, 8);
initMMX2HScaler(c->chrDstW, c->chrXInc, c->chrMmx2FilterCode,
c->hChrFilter, c->hChrFilterPos, 4);
#ifdef MAP_ANONYMOUS
mprotect(c->lumMmx2FilterCode, c->lumMmx2FilterCodeSize, PROT_EXEC |
PROT_READ);
mprotect(c->chrMmx2FilterCode, c->chrMmx2FilterCodeSize, PROT_EXEC |
PROT_READ);
#endif
} else
#endif /* ARCH_X86 && (HAVE_MMX2 || CONFIG_RUNTIME_CPUDETECT) */
I've noticed that in our port lumMmx2FilterCode and chrMmx2FilterCode buffers
are allocated using av_malloc() (which is a wrapper around libc malloc) and
that's the source of the problem - on amd64 heap memory is not executable. So
the attempt to execute code in the heap buffer results in a page fault and in
the crash (SIGBUS).
The buffer should be actually allocated via mmap() and then get proper
permissions via mprotect(). That doesn't happen because MAP_ANONYMOUS is not
defined in that context.
MAP_ANONYMOUS/MAP_ANON is not defined, because it is not specified in POSIX and
it is under _BSD_VISIBLE in sys/mman.h, but the code at hand is compiled with
_POSIX_SOURCE defined. On FreeBSD _POSIX_SOURCE implies !_BSD_VISIBLE.
To get around that issue _on Linux_, ffmpeg folks have put explicit _SVID_SOURCE
definition right into the libswscale/utils.c file:
#define _SVID_SOURCE //needed for MAP_ANONYMOUS
For much the same reason we need to add the following for FreeBSD (as hackish as
it is):
#define __BSD_VISIBLE 1
With that addition the code buffer is allocated properly and the code in it can
be executed.
P.S. this page talks about a similar issue:
http://ubuntuforums.org/showthread.php?t=1547726
--
Andriy Gapon
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4D05332C.7010208>