Date: Fri, 19 Jun 1998 16:20:58 -0400 (EDT) From: Simon Shapiro <shimon@simon-shapiro.org> To: Steven Yang <syang@DIRHIT.COM> Cc: "freebsd-database@FreeBSD.org" <freebsd-database@FreeBSD.ORG> Subject: RE: gdbm scalability question Message-ID: <XFMail.980619162058.shimon@simon-shapiro.org> In-Reply-To: <839A86AB6CE4D111A52200104B938D435B41@MOE>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --] Try the ulimit command. Then try to glean from my kernel configuration below what is it that I do to get your problem solved. I simnply do not remember. Simon On 19-Jun-98 Steven Yang wrote: > Hi, I am having trouble creating a gdbm database larger than 16.9MB. I > have 128MB of RAM, and would like my database to be as large as 2GB. > However, as I generate a test database, when the file reaches 16.9MB in > size, I get: > > gdbm fatal: malloc error > > It appears that there is a per-process limitation of 16MB of RAM with a > generic kernel. But even if I were to get around that per-process > thing, I wouldn't be able to generate a 2GB database file with only > 128MB of RAM given what I am doing. Does gdbm have to load the entire > database into RAM in order to insert a record? Is there a setting I > have to change? > > Does anybody out there know how to make a large gdbm database, like one > with 10 million records where the content of each record is 1-2KB? Once > the database is created, it is meant to be read-only. > > - Steven Yang > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-database" in the body of the message ---------- Sincerely Yours, Simon Shapiro Shimon@Simon-Shapiro.ORG Voice: 503.799.2313 [-- Attachment #2 --] # # NOMIS-smp -- Test DPT SMP kernel for NOMIS-like machines # # $Id: NOMIS-smp,v 1.16 1998/06/15 16:00:12 ShimonR Exp ShimonR $ machine "i386" cpu "I686_CPU" ident NOMIS-smp maxusers 64 options INET #InterNETworking options FFS #Berkeley Fast Filesystem options NFS #Network Filesystem options MSDOSFS #MSDOS Filesystem options "CD9660" #ISO 9660 Filesystem options PROCFS #Process filesystem options "COMPAT_43" #Compatible with BSD 4.3 [KEEP THIS!] options BOUNCE_BUFFERS #include support for DMA bounce buffers options UCONSOLE #Allow users to grab the console options FAILSAFE #Be conservative options USERCONFIG #boot -c editor options VISUAL_USERCONFIG #visual boot -c editor options DDB options KTRACE options PERFMON options SMP # Symmetric MultiProcessor Kernel options APIC_IO # Symmetric (APIC) I/O config kernel root on wd0 controller isa0 controller pci0 controller fdc0 at isa? port "IO_FD1" bio irq 6 drq 2 vector fdintr disk fd0 at fdc0 drive 0 disk fd1 at fdc0 drive 1 controller wdc0 at isa? port "IO_WD1" bio irq 14 vector wdintr disk wd0 at wdc0 drive 0 disk wd1 at wdc0 drive 1 controller wdc1 at isa? port "IO_WD2" bio irq 15 vector wdintr disk wd2 at wdc1 drive 0 disk wd3 at wdc1 drive 1 options ATAPI options ATAPI_STATIC device wcd0 # We assume wide SCSI, not FCAL, with this release. # There is never a target 7, nor target 15; They are reserved for initiators! # Each ADAPTER takes 4 busses but uses only busses 0-2. # Each bus consumes 16 addresses, but uses no more than 13 devices per bus. # Devices with no LUNs have only LUN 0. Lun != 0 means secondary LUNS 1-7 # Non-zero LUNs have the same ID as LUN 0 but bit 7 is set on (64 + target) controller dpt0 controller dpt1 options DPT_MEASURE_PERFORMANCE options DPT_LOST_IRQ controller scbus0 at dpt0 bus 0 controller scbus1 at dpt0 bus 1 controller scbus2 at dpt0 bus 2 controller scbus4 at dpt1 bus 0 controller scbus5 at dpt1 bus 1 controller scbus6 at dpt1 bus 2 device ch0 device sd0 device od0 device st0 device cd0 # First DPT starts here # On bus 0, you can have both a Cd and a worm but no multi-initiator device worm0 at scbus0 target 4 unit 0 tape st0 at scbus0 target 5 unit 0 device cd0 at scbus0 target 6 unit 0 disk sd0 at scbus0 target 0 unit 0 disk sd1 at scbus0 target 1 unit 0 disk sd2 at scbus0 target 2 unit 0 disk sd3 at scbus0 target 3 unit 0 disk sd4 at scbus0 target 4 unit 0 disk sd5 at scbus0 target 5 unit 0 disk sd6 at scbus0 target 6 unit 0 disk sd8 at scbus0 target 8 unit 0 disk sd9 at scbus0 target 9 unit 0 disk sd10 at scbus0 target 10 unit 0 disk sd11 at scbus0 target 11 unit 0 disk sd12 at scbus0 target 12 unit 0 disk sd13 at scbus0 target 13 unit 0 disk sd14 at scbus0 target 14 unit 0 # On bus 1 you can have either a CD or a WORM, and multi-initiators device worm1 at scbus1 target 4 unit 0 device cd1 at scbus1 target 4 unit 0 tape st1 at scbus1 target 5 unit 0 device pt1 at scbus1 target 6 unit 0 # SCSI processor type device pt1 at scbus1 target 7 unit 0 # SCSI processor type disk sd16 at scbus1 target 0 unit 0 disk sd17 at scbus1 target 1 unit 0 disk sd18 at scbus1 target 2 unit 0 disk sd19 at scbus1 target 3 unit 0 disk sd20 at scbus1 target 4 unit 0 disk sd21 at scbus1 target 5 unit 0 disk sd22 at scbus1 target 6 unit 0 disk sd24 at scbus1 target 8 unit 0 disk sd25 at scbus1 target 9 unit 0 disk sd26 at scbus1 target 10 unit 0 disk sd27 at scbus1 target 11 unit 0 disk sd28 at scbus1 target 12 unit 0 disk sd29 at scbus1 target 13 unit 0 disk sd30 at scbus1 target 14 unit 0 # On bus 2 you can have either a CD or a WORM, and multi-initiators device worm2 at scbus2 target 4 unit 0 device cd2 at scbus2 target 4 unit 0 tape st2 at scbus2 target 5 unit 0 device pt2 at scbus2 target 6 unit 0 # SCSI processor type device pt2 at scbus2 target 7 unit 0 # SCSI processor type disk sd32 at scbus2 target 0 unit 0 disk sd33 at scbus2 target 1 unit 0 disk sd34 at scbus2 target 2 unit 0 disk sd35 at scbus2 target 3 unit 0 disk sd36 at scbus2 target 4 unit 0 disk sd37 at scbus2 target 5 unit 0 disk sd38 at scbus2 target 6 unit 0 disk sd41 at scbus2 target 9 unit 0 disk sd42 at scbus2 target 10 unit 0 disk sd43 at scbus2 target 11 unit 0 disk sd44 at scbus2 target 12 unit 0 disk sd45 at scbus2 target 13 unit 0 disk sd46 at scbus2 target 14 unit 0 # Second DPT starts here # On bus 0, you can have both a CD and a worm but no multi-initiator device worm4 at scbus4 target 4 unit 0 tape st4 at scbus4 target 5 unit 0 device cd4 at scbus4 target 6 unit 0 disk sd64 at scbus4 target 0 unit 0 disk sd65 at scbus4 target 1 unit 0 disk sd66 at scbus4 target 2 unit 0 disk sd67 at scbus4 target 3 unit 0 disk sd68 at scbus4 target 4 unit 0 disk sd69 at scbus4 target 5 unit 0 disk sd70 at scbus4 target 6 unit 0 disk sd72 at scbus4 target 8 unit 0 disk sd73 at scbus4 target 9 unit 0 disk sd74 at scbus4 target 10 unit 0 disk sd75 at scbus4 target 11 unit 0 disk sd76 at scbus4 target 12 unit 0 disk sd77 at scbus4 target 13 unit 0 disk sd78 at scbus4 target 14 unit 0 disk sd79 at scbus4 target 15 unit 0 # On bus 1 you can have either a CD or a WORM, and multi-initiators device worm5 at scbus5 target 4 unit 0 device cd5 at scbus5 target 4 unit 0 tape st5 at scbus5 target 5 unit 0 device pt5 at scbus1 target 6 unit 0 # SCSI processor type device pt5 at scbus1 target 7 unit 0 # SCSI processor type disk sd80 at scbus5 target 0 unit 0 disk sd81 at scbus5 target 1 unit 0 disk sd82 at scbus5 target 2 unit 0 disk sd83 at scbus5 target 3 unit 0 disk sd84 at scbus5 target 4 unit 0 disk sd85 at scbus5 target 5 unit 0 disk sd86 at scbus5 target 6 unit 0 disk sd88 at scbus5 target 8 unit 0 disk sd89 at scbus5 target 9 unit 0 disk sd90 at scbus5 target 10 unit 0 disk sd91 at scbus5 target 11 unit 0 disk sd92 at scbus5 target 12 unit 0 disk sd93 at scbus5 target 13 unit 0 disk sd94 at scbus5 target 14 unit 0 disk sd95 at scbus5 target 15 unit 0 # On bus 2 you can have either a CD or a WORM, and multi-initiators device worm6 at scbus6 target 4 unit 0 device cd6 at scbus6 target 4 unit 0 tape st6 at scbus6 target 5 unit 0 device pt6 at scbus1 target 6 unit 0 # SCSI processor type device pt6 at scbus1 target 7 unit 0 # SCSI processor type disk sd96 at scbus6 target 0 unit 0 disk sd97 at scbus6 target 1 unit 0 disk sd98 at scbus6 target 2 unit 0 disk sd99 at scbus6 target 3 unit 0 disk sd100 at scbus6 target 4 unit 0 disk sd101 at scbus6 target 5 unit 0 disk sd102 at scbus6 target 6 unit 0 disk sd104 at scbus6 target 8 unit 0 disk sd105 at scbus6 target 9 unit 0 disk sd106 at scbus6 target 10 unit 0 disk sd107 at scbus6 target 11 unit 0 disk sd108 at scbus6 target 12 unit 0 disk sd109 at scbus6 target 13 unit 0 disk sd110 at scbus6 target 14 unit 0 disk sd111 at scbus6 target 15 unit 0 controller ahc0 options AHC_TAGENABLE options AHC_SCBPAGING_ENABLE options AHC_ALLOW_MEMIO controller scbus8 at ahc0 # On an Adaptec you can have both a CD or a WORM, but no multi-initiators device worm8 at scbus6 target 4 unit 0 tape st8 at scbus6 target 5 unit 0 device cd8 at scbus6 target 6 unit 0 disk sd128 at scbus8 target 0 unit 0 disk sd129 at scbus8 target 1 unit 0 disk sd130 at scbus8 target 2 unit 0 disk sd131 at scbus8 target 3 unit 0 disk sd132 at scbus8 target 4 unit 0 disk sd133 at scbus8 target 5 unit 0 disk sd134 at scbus8 target 6 unit 0 disk sd136 at scbus8 target 8 unit 0 disk sd137 at scbus8 target 9 unit 0 disk sd138 at scbus8 target 10 unit 0 disk sd139 at scbus8 target 11 unit 0 disk sd140 at scbus8 target 12 unit 0 disk sd141 at scbus8 target 13 unit 0 disk sd142 at scbus8 target 14 unit 0 disk sd143 at scbus8 target 15 unit 0 device sc0 at isa? port "IO_KBD" tty irq 1 vector scintr device psm0 at isa? port "IO_KBD" conflicts tty irq 12 vector psmintr device npx0 at isa? port "IO_NPX" irq 13 vector npxintr options CONSPEED=9600 #default speed for serial console (default 9600) device sio0 at isa? port "IO_COM1" flags 0x10 tty irq 4 vector siointr device sio1 at isa? port "IO_COM2" tty irq 3 vector siointr device sio2 at isa? port "IO_COM3" tty irq 5 vector siointr device sio3 at isa? port "IO_COM4" tty irq 9 vector siointr device lpt0 at isa? port? tty irq 7 vector lptintr # Order is important here due to intrusive probes, do *not* alphabetize. device de0 device fxp0 device fpa0 device ed0 at isa? port 0x300 net irq 10 iomem 0xd8000 vector edintr pseudo-device loop pseudo-device ether pseudo-device fddi pseudo-device ppp 8 pseudo-device bpfilter 16 pseudo-device tun 4 pseudo-device pty 16 pseudo-device gzip options MROUTING # Multicast routing options IPFIREWALL #firewall options IPFIREWALL_VERBOSE #print information about # dropped packets options IPDIVERT #divert sockets options SYSVSHM options SYSVSEM options SYSVMSG #pseudo-device dlm #options DLM_MAX_CHANNELS=4 #options DLM_LOCKS_PER_CHANNEL=4096 #options DLM_STATE_BITS=8 #options DLM_MAX_LOCKERS=128 #options DLM_KEY_LEN=16 pseudo-device sppp pseudo-device disc pseudo-device speaker pseudo-device vn #Vnode driver (turns a file into a device) pseudo-device snp 3 #Snoop device - to look at pty/vty/etc.. pseudo-device ccd 64 #Concatenated disk driver options XSERVER # include code for XFree86 options MAXCONS=12 # number of virtual consoles options COMPAT_LINUX options "EXT2FS" options "IBCS2" options SHOW_BUSYBUFS controller snd0 device sb0 at isa? port 0x220 irq 7 drq 1 vector sbintr device sbxvi0 at isa? drq 5 device sbmidi0 at isa? port 0x300 device opl0 at isa? port 0x388help
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?XFMail.980619162058.shimon>