Date: Sat, 23 Feb 2013 22:31:10 -0800 From: Jeremy Chadwick <jdc@koitsu.org> To: Michael Ross <gmx@ross.cx> Cc: freebsd-stable@freebsd.org, John Mehr <jcm@visi.com> Subject: Re: svn - but smaller? Message-ID: <20130224063110.GA53348@icarus.home.lan> In-Reply-To: <op.wszt3wh2g7njmm@michael-think> References: <1359320641-6493504.60501067.fr0RL3aYw027137@rs149.luxsci.com> <web-10418670@mailback3.g2host.com> <1359380582-6256705.77592125.fr0SDgrYH000991@rs149.luxsci.com> <web-10502111@mailback3.g2host.com> <web-12014638@mailback4.g2host.com> <op.wszomvfyg7njmm@michael-think> <20130224031509.GA47838@icarus.home.lan> <op.wszrv9k5g7njmm@michael-think> <20130224041638.GA51493@icarus.home.lan> <op.wszt3wh2g7njmm@michael-think>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, Feb 24, 2013 at 05:44:10AM +0100, Michael Ross wrote: > On Sun, 24 Feb 2013 05:16:38 +0100, Jeremy Chadwick <jdc@koitsu.org> wrote: > > >On Sun, Feb 24, 2013 at 04:56:23AM +0100, Michael Ross wrote: > >>On Sun, 24 Feb 2013 04:15:09 +0100, Jeremy Chadwick > >><jdc@koitsu.org> wrote: > >> > >>>On Sun, Feb 24, 2013 at 03:45:57AM +0100, Michael Ross wrote: > >>>>On Sun, 24 Feb 2013 01:36:36 +0100, John Mehr <jcm@visi.com> wrote: > >>>> > >>>>> Hello all, > >>>>> I've believe I've made just about all of the progress > >>>>optimizing svnup > >>>>> as I can and I've just submitted it as a new port. With my > >>>>~ 350kb/s > >>>>> DSL connection, it now takes just under 30 minutes to > >>>>download a fresh > >>>>> base/releng/8.3 tree using svnup (Subversion's svn takes > >>>>approximately > >>>>> 12 minutes). Incremental updates, such as tracking one of > >>>>the stable > >>>>> branches takes only 2-3 minutes. > >>>>> For anyone that wants to preview the port before it gets > >>>>added to the > >>>>> ports tree (assuming I got the send-pr correct), the tarball is > >>>>>located > >>>>> at: > >>>>> http://jcm.dsl.visi.com/freebsd/svnup/svnup-0.5.tar.xz > >>>>> Please let me know if you find any issues. > >>>> > >>>> > >>>>Maybe it's me, but: > >>>> > >>>>No Makefile. > >>>>So I try manually: > >>>> > >>>>gurder> cc svnup.c > >>>>/tmp//cconKEOv.o: In function `compare_md5': > >>>>svnup.c:(.text+0x175e): undefined reference to `MD5Init' > >>>>svnup.c:(.text+0x1774): undefined reference to `MD5Update' > >>>>svnup.c:(.text+0x1785): undefined reference to `MD5End' > >>>>/tmp//cconKEOv.o: In function `get_files': > >>>>svnup.c:(.text+0x1f20): undefined reference to `MD5Init' > >>>>svnup.c:(.text+0x1f4e): undefined reference to `MD5Update' > >>>>svnup.c:(.text+0x1f5f): undefined reference to `MD5End' > >>>> > >>>> > >>>>9.0-STABLE FreeBSD 9.0-STABLE #17: Fri May 4 02:53:49 CEST 2012 > >>> > >>>Those are all defined in libmd (see MD5Init(3) man page). Thus: > >>> > >>>cc -o svnup svnup.c -lmd > >>> > >> > >>Thanks. > >> > >>gurder> ./svnup -h svn0.us-west.FreeBSD.org -b ports/head -l test > >> > >>Dumps core: http://gurder.ross.cx/misc/svnup.core > > > >Should be easy enough to debug: > > > >$ cc -g3 -ggdb -o svnup svnup.c -lmd > >$ gdb svnup > >(gdb) run -h svn0.us-west.FreeBSD.org -b ports/head -l test > > > >Then once it cores, provide the output from "bt" and/or "bt full". > > > >Might also be useful to compile with -Wall to see if there are any > >compile-time warnings. > > > > gurder> cc -Wall -g3 -ggdb -o svnup svnup.c -lmd > svnup.c: In function 'main': > svnup.c:1002: warning: zero-length printf format string > svnup.c:1020: warning: zero-length printf format string > svnup.c:1027: warning: zero-length printf format string > svnup.c:1065: warning: zero-length printf format string > > > (gdb) run -h svn0.us-west.FreeBSD.org -b ports/head -l test > Starting program: /usr/ports/sysutils/svnup-0.5/svnup -h > svn0.us-west.FreeBSD.org -b ports/head -l test > ####### Fetching revision: 312860 > ? test/archivers > Program received signal SIGSEGV, Segmentation fault. > 0x0000000800b22950 in strchr () from /lib/libc.so.7 > > Backtrace: http://gurder.ross.cx/misc/backtrace.txt I downloaded it and looked at the source. svnup.c:1002: warning: zero-length printf format string svnup.c:1020: warning: zero-length printf format string svnup.c:1027: warning: zero-length printf format string svnup.c:1065: warning: zero-length printf format string 1002 sprintf(command, ""); 1020 sprintf(command, ""); 1027 sprintf(command, ""); 1065 sprintf(command, ""); I'm not sure what the intention is here. If it's to terminate the buffer, then all of these should be: command[0] = '\0'; Or if the entire command[] buffer needs to be zeroed, use memset(3). Doing this does not fix the segfault. The segfault experienced is caused by the following problem: Program received signal SIGSEGV, Segmentation fault. 0x00000000a0b2eb10 in strchr () from /lib/libc.so.7 (gdb) bt #0 0x00000000a0b2eb10 in strchr () from /lib/libc.so.7 #1 0x00000000004021e8 in build_source_directory_tree (connection=0x7fffffff4820, command=0x7ffffffc3650 "", file=0xa1135000, file_count=0x7fffffff48f8, max_file=0x7fffffff48fc, path_target=0xa1009058 "test", revision=0) at svnup.c:412 #2 0x0000000000000000 in ?? () (gdb) f 1 #1 0x00000000004021e8 in build_source_directory_tree (connection=0x7fffffff4820, command=0x7ffffffc3650 "", file=0xa1135000, file_count=0x7fffffff48f8, max_file=0x7fffffff48fc, path_target=0xa1009058 "test", revision=0) at svnup.c:412 412 end = strchr(directory, '\n'); (gdb) p directory $1 = 0x0 It's a wee bit hard to do strchr() on something that points to NULL. At line 412 I inserted this, which is almost certainly not sufficient/correct: if (directory == NULL) { continue; } This allows the program to get a bit further than before (same goes if I use break instead of continue), but not before segfaulting again. And segfaults after this point contain a completely smashed stack: Program received signal SIGSEGV, Segmentation fault. 0x0000000000000000 in ?? () (gdb) bt #0 0x0000000000000000 in ?? () #1 0x0000000000000000 in ?? () John will need to look into these. They all reek of string parsing anomalies and so on. I would need to sit down and read/understand the code in full to know how to fix this one. Also, John, please consider using malloc(3) instead of heap-allocated buffers like file_buffer[6][] (196608 bytes) and command[] (32769 bytes). I'm referring to this: 47 #define COMMAND_BUFFER 32768 386 char new_path_target[BUFFER_UNIT], file_buffer[6][COMMAND_BUFFER], *path_source; 836 char *start, *value, *addr, *branch, *path_target, temp_file[BUFFER_UNIT], command[COMMAND_BUFFER + 1]; I should also point out that watching this thing in top(1) shows RES/RSS increasing until the segfault (for me dies out around 4.5MBytes). I don't know if that's by design or not, but I don't see why this thing would need so much memory given what it's doing. You'd know better than I though. You may want to consider running this under valgrind. It's remarkable what you can find with that during debugging. -- | Jeremy Chadwick jdc@koitsu.org | | UNIX Systems Administrator http://jdc.koitsu.org/ | | Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB |
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20130224063110.GA53348>