Skip site navigation (1)Skip section navigation (2)
Date:      Sun, 02 Jul 2006 20:27:57 -0700
From:      "Philip M. Gollucci" <pgollucci@p6m7g8.com>
To:        questions@freebsd.org
Cc:        dev@perl.apache.org, dev@httpd.apache.org
Subject:   gdb in realloc(): error: junk pointer, too high to make sense on FreeBSD 5.2+
Message-ID:  <44A88EBD.80407@p6m7g8.com>

next in thread | raw e-mail | index | archive | help
This is a multi-part message in MIME format.
--------------080509030709030100050209
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Hi,

2 years ago, PR: http://www.freebsd.org/cgi/query-pr.cgi?pr=bin/68727
was filed about gdb itself segfaulting when trying to debug httpd.
This PR hasn't seen any action AFAIK.

This has been slowly driving me mad as you can not debug any httpd startup code.
Which coincidentally also prevents you from debugging mod_perl startup code and most likely
mod_* (php,ruby,tcl,java,python)

I did some painstakingly slow "stepping" through httpd code today:

Basically after apr_dso_load() calls dlopen()  (OS level Freebsd version) the
SECOND time through (httpd parses the config twice)
The first LoadModule line (doesn't matter which) crashes gdb.

I can reliably reproduce this on _ALL_ versions of FreeBSD from 5.2 through 6.1-STABLE and on -CURRENT.
and ALL versions of httpd 2.0.48+ or 2.2.x

Attached is the information I was able to collect, but at this point, I'm in WAY over my head.

[Apologies for the cross post, feel free to spam me in return]

gdb -v
GNU gdb 6.1.1 [FreeBSD]

gcc -v
gcc version 3.4.4 [FreeBSD] 20050518

I've also seen it on EVERY version of gdb in the ports tree combined with any version of gcc in the ports tree.
/usr/local/bin/gcc32
/usr/local/bin/gcc33
/usr/local/bin/gcc34
/usr/local/bin/gcc40
/usr/local/bin/gcc41
/usr/local/bin/gcc42

------------------------------------------------------------------------
Philip M. Gollucci (pgollucci@p6m7g8.com) 323.219.4708
Consultant / http://p6m7g8.net/Resume/resume.shtml
Senior Software Engineer - TicketMaster - http://ticketmaster.com
1024D/A79997FA F357 0FDD 2301 6296 690F  6A47 D55A 7172 A799 97F

"In all that I've done wrong I know I must have done something right to
deserve a hug every morning and butterfly kisses at night."

--------------080509030709030100050209
Content-Type: text/plain;
 name="gdb"
Content-Transfer-Encoding: 8bit
Content-Disposition: inline;
 filename="gdb"

(gdb) bt
#0  apr_dso_load (res_handle=0xbfbfe15c, path=0x80f8760 "/usr/local/software/frebsd-6.1-stable/3.4.4/httpd/2.2.2/prefork/modules/mod_authn_file.so", pool=0x80bd018)
    at /usr/home/pgollucci/dev/src/httpd-2.2.2/srclib/apr/dso/unix/dso.c:126
#1  0x08089b73 in load_module (cmd=0xbfbfe4d0, dummy=0xbfbfe348, modname=0x80f8728 "authn_file_module", filename=0x80f8740 "modules/mod_authn_file.so") at mod_so.c:238
#2  0x0807824d in invoke_cmd (cmd=0x809da80, parms=0xbfbfe4d0, mconfig=0xbfbfe348, args=0x80f104e "") at config.c:778
#3  0x0807969b in execute_now (cmd_line=0x80f86b8 "LoadModule", args=0x80f1023 "authn_file_module modules/mod_authn_file.so", parms=0xbfbfe4d0, p=0x80bd018, ptemp=0x80ef018, 
    sub_tree=0xbfbfe348, parent=0x0) at config.c:1419
#4  0x08078c6c in ap_build_config_sub (p=0x80bd018, temp_pool=0x80ef018, l=0x80f1018 "LoadModule authn_file_module modules/mod_authn_file.so", parms=0xbfbfe4d0, current=0xbfbfe3a4, 
    curr_parent=0xbfbfe3a0, conftree=0x80a4674) at config.c:990
#5  0x080791ad in ap_build_config (parms=0xbfbfe4d0, p=0x80bd018, temp_pool=0x80ef018, conftree=0x80a4674) at config.c:1202
#6  0x08079b2a in process_resource_config_nofnmatch (s=0x80beeb8, fname=0x80f7408 "/usr/local/software/frebsd-6.1-stable/3.4.4/httpd/2.2.2/prefork/conf/httpd.conf", 
    conftree=0x80a4674, p=0x80bd018, ptemp=0x80ef018, depth=0) at config.c:1612
#7  0x08079c25 in ap_process_resource_config (s=0x80beeb8, fname=0x80f7408 "/usr/local/software/frebsd-6.1-stable/3.4.4/httpd/2.2.2/prefork/conf/httpd.conf", conftree=0x80a4674, 
    p=0x80bd018, ptemp=0x80ef018) at config.c:1644
#8  0x0807a730 in ap_read_config (process=0x80bb098, ptemp=0x80ef018, filename=0x8095a01 "conf/httpd.conf", conftree=0x80a4674) at config.c:2002
#9  0x08062083 in main (argc=2, argv=0xbfbfe778) at main.c:681


apr_dso_load (res_handle=0xbfbfe15c, path=0x80f8760 "/usr/local/software/frebsd-6.1-stable/3.4.4/httpd/2.2.2/prefork/modules/mod_authn_file.so", pool=0x80bd018)
    at /usr/home/pgollucci/dev/src/httpd-2.2.2/srclib/apr/dso/unix/dso.c:126
126         int flags = RTLD_NOW | RTLD_GLOBAL;
(gdb) s
139         os_handle = dlopen(path, flags);
(gdb) s
0x28173e24 in pthread_rwlock_timedwrlock () from /usr/lib/libpthread.so.2
(gdb) s
Single stepping until exit from function pthread_rwlock_timedwrlock, 
which has no line number information.
0x28179a44 in pthread_mutexattr_init () from /usr/lib/libpthread.so.2
(gdb) s
Single stepping until exit from function pthread_mutexattr_init, 
which has no line number information.
0x28173cc5 in pthread_rwlock_timedwrlock () from /usr/lib/libpthread.so.2
(gdb) s
Single stepping until exit from function pthread_rwlock_timedwrlock, 
which has no line number information.
0x281833c0 in __error () from /usr/lib/libpthread.so.2
(gdb) s
Single stepping until exit from function __error, 
which has no line number information.
0x28173d34 in pthread_rwlock_timedwrlock () from /usr/lib/libpthread.so.2
(gdb) s
Single stepping until exit from function pthread_rwlock_timedwrlock, 
which has no line number information.
s
gdb in realloc(): error: junk pointer, too high to make sense
Abort (core dumped)


#0  apr_dso_load (res_handle=0xbfbfe15c, path=0x80f8760 "/usr/local/software/frebsd-6.1-stable/3.4.4/httpd/2.2.2/prefork/modules/mod_authn_file.so", pool=0x80bd018)
    at /usr/home/pgollucci/dev/src/httpd-2.2.2/srclib/apr/dso/unix/dso.c:126
        flags = 135235424
        os_handle = (void *) 0x10
#1  0x08089b73 in load_module (cmd=0xbfbfe4d0, dummy=0xbfbfe348, modname=0x80f8728 "authn_file_module", filename=0x80f8740 "modules/mod_authn_file.so") at mod_so.c:238
        modhandle = (apr_dso_handle_t *) 0x80f8740
        modsym = 0x6d000000
        modp = (module *) 0x80a2400
        szModuleFile = 0x80f8760 "/usr/local/software/frebsd-6.1-stable/3.4.4/httpd/2.2.2/prefork/modules/mod_authn_file.so"
        sconf = (so_server_conf *) 0x80c1228
        modi = (ap_module_symbol_t *) 0x80c1248
        modie = (ap_module_symbol_t *) 0x80c1248
        i = 4
        error = 0x8064a8b "\203Ä\020\211Eô\213Eø\2008"
#2  0x0807824d in invoke_cmd (cmd=0x809da80, parms=0xbfbfe4d0, mconfig=0xbfbfe348, args=0x80f104e "") at config.c:778
        w = 0x80f8728 "authn_file_module"
        w2 = 0x80f8740 "modules/mod_authn_file.so"
        w3 = 0x80f8718 "loadmodule"
        errmsg = 0x0
#3  0x0807969b in execute_now (cmd_line=0x80f86b8 "LoadModule", args=0x80f1023 "authn_file_module modules/mod_authn_file.so", parms=0xbfbfe4d0, p=0x80bd018, ptemp=0x80ef018, 
    sub_tree=0xbfbfe348, parent=0x0) at config.c:1419
        retval = 0x2c <Error reading address 0x2c: Bad address>
        cmd = (const command_rec *) 0x809da80
        ml = (ap_mod_list *) 0x80f7380
        dir = 0x80f8718 "loadmodule"
#4  0x08078c6c in ap_build_config_sub (p=0x80bd018, temp_pool=0x80ef018, l=0x80f1018 "LoadModule authn_file_module modules/mod_authn_file.so", parms=0xbfbfe4d0, current=0xbfbfe3a4, 
    curr_parent=0xbfbfe3a0, conftree=0x80a4674) at config.c:990
        sub_tree = (ap_directive_t *) 0x0
        retval = 0x0
        args = 0x80f1023 "authn_file_module modules/mod_authn_file.so"
        cmd_name = 0x80f86b8 "LoadModule"
        newdir = (ap_directive_t *) 0x80f86c8
        mod = (module *) 0x80a2400
        cmd = (const command_rec *) 0x809da80
#5  0x080791ad in ap_build_config (parms=0xbfbfe4d0, p=0x80bd018, temp_pool=0x80ef018, conftree=0x80a4674) at config.c:1202
        current = (ap_directive_t *) 0x80f8690
        curr_parent = (ap_directive_t *) 0x0
        l = 0x80f1018 "LoadModule authn_file_module modules/mod_authn_file.so"
        errmsg = 0x0
#6  0x08079b2a in process_resource_config_nofnmatch (s=0x80beeb8, fname=0x80f7408 "/usr/local/software/frebsd-6.1-stable/3.4.4/httpd/2.2.2/prefork/conf/httpd.conf", 
    conftree=0x80a4674, p=0x80bd018, ptemp=0x80ef018, depth=0) at config.c:1612
        parms = {info = 0x0, override = 150, limited = -1, limited_xmethods = 0x0, xlimited = 0x0, config_file = 0x80f8508, directive = 0x0, pool = 0x80bd018, temp_pool = 0x80ef018, 
  server = 0x80beeb8, path = 0x0, cmd = 0x809da80, context = 0x0, err_directive = 0x80f86c8, override_opts = 239}
        cfp = (ap_configfile_t *) 0x80f8508
        error = 0x80a8a80 "P\223\n\bP\223\n\b\002"
        rv = 0
#7  0x08079c25 in ap_process_resource_config (s=0x80beeb8, fname=0x80f7408 "/usr/local/software/frebsd-6.1-stable/3.4.4/httpd/2.2.2/prefork/conf/httpd.conf", conftree=0x80a4674, 
    p=0x80bd018, ptemp=0x80ef018) at config.c:1644
No locals.
#8  0x0807a730 in ap_read_config (process=0x80bb098, ptemp=0x80ef018, filename=0x8095a01 "conf/httpd.conf", conftree=0x80a4674) at config.c:2002
        confname = 0x80f7408 "/usr/local/software/frebsd-6.1-stable/3.4.4/httpd/2.2.2/prefork/conf/httpd.conf"
        error = 0x0
        p = (apr_pool_t *) 0x80bd018
        s = (server_rec *) 0x80beeb8
#9  0x08062083 in main (argc=2, argv=0xbfbfe778) at main.c:681
        c = 88 'X'
        configtestonly = 0
        confname = 0x8095a01 "conf/httpd.conf"
        def_server_root = 0x8095a14 "/usr/local/software/frebsd-6.1-stable/3.4.4/httpd/2.2.2/prefork"
        temp_error_log = 0x0
        error = 0x0
        process = (process_rec *) 0x80bb098
        server_conf = (server_rec *) 0x80c2988
        pglobal = (apr_pool_t *) 0x80bb018
        pconf = (apr_pool_t *) 0x80bd018
        plog = (apr_pool_t *) 0x80eb018
        ptemp = (apr_pool_t *) 0x80ef018
        pcommands = (apr_pool_t *) 0x80bf018
        opt = (apr_getopt_t *) 0x80bf0b0
        rv = 0
        mod = (module **) 0x80a1290
        optarg = 0x0
        signal_server = (apr_OFN_ap_signal_server_t *) 0


(gdb) x os_handle
0x10:   Error accessing memory address 0x10: Bad address.
(gdb) p os_handle
$1 = (void *) 0x10
(gdb) 

(gdb) s
139         os_handle = dlopen(path, flags);
(gdb) p path
$2 = 0x80f8760 "/usr/local/software/frebsd-6.1-stable/3.4.4/httpd/2.2.2/prefork/modules/mod_authn_file.so"
(gdb) p flags
$3 = 258
(gdb) x flags
0x102:  Error accessing memory address 0x102: Bad address.




--------------080509030709030100050209--



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?44A88EBD.80407>