From nobody Sun Jun 26 23:46:40 2022 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4272586E40E for ; Sun, 26 Jun 2022 23:47:13 +0000 (UTC) (envelope-from karl@denninger.net) Received: from colo1.denninger.net (colo1.denninger.net [104.236.120.189]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4LWSFW56Klz4YyJ for ; Sun, 26 Jun 2022 23:47:11 +0000 (UTC) (envelope-from karl@denninger.net) Received: from denninger.net (097-081-026-048.res.spectrum.com [97.81.26.48]) by colo1.denninger.net (Postfix) with ESMTP id 5393B2110A8 for ; Sun, 26 Jun 2022 19:46:41 -0400 (EDT) Received: from [192.168.10.25] (D15.Denninger.Net [192.168.10.25]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by denninger.net (Postfix) with ESMTPSA id D896F3EDE1E for ; Sun, 26 Jun 2022 19:46:40 -0400 (EDT) Message-ID: <7c96bf50-42d1-b799-ec8d-146e26f1f014@denninger.net> Date: Sun, 26 Jun 2022 19:46:40 -0400 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.10.0 From: Karl Denninger Subject: Programing question regarding FCGI apps (in "C") To: freebsd-hackers@freebsd.org Content-Language: en-US Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms080304010900090100080303" X-Rspamd-Queue-Id: 4LWSFW56Klz4YyJ X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=pass (policy=none) header.from=denninger.net; spf=pass (mx1.freebsd.org: domain of karl@denninger.net designates 104.236.120.189 as permitted sender) smtp.mailfrom=karl@denninger.net X-Spamd-Result: default: False [-5.90 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FREEFALL_USER(0.00)[karl]; FROM_HAS_DN(0.00)[]; SIGNED_SMIME(-2.00)[]; R_SPF_ALLOW(-0.20)[+mx:c]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,multipart/alternative,text/plain]; HAS_ATTACHMENT(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_NONE(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[denninger.net,none]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; MLMMJ_DEST(0.00)[freebsd-hackers]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:~,4:~]; ASN(0.00)[asn:14061, ipnet:104.236.64.0/18, country:US]; RCVD_TLS_LAST(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[97.81.26.48:received] X-ThisMailContainsUnwantedMimeParts: N This is a cryptographically signed message in MIME format. --------------ms080304010900090100080303 Content-Type: multipart/alternative; boundary="------------w9EMxmDxfOVQz0PHHO0peT0M" --------------w9EMxmDxfOVQz0PHHO0peT0M Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit I have an application that has been a "traditional" CGI app for a long time and am converting it to use the fcgi-dev "wrapper" under Apache along with writing a little program that manages the daemons (so if something goes wrong they get restarted with logging to the admin, I can issue a kill to all of them to update the code or a restart request, etc.) What I found is that with that wrapper in use the code runs fine -- but it leaks memory.  It is quite complex and uses the libpq library to talk to Postgres, but I've never seen problems with libpq leaking memory before.  Nonetheless I spent the rather-considerable time to make darn sure via wrapper functions that I wasn't leaking memory through inappropriate use of malloc/realloc/free or unpaired calls to the libpq functions that create allocations, including using wrappers to maintain a list of "active" allocations and outstanding requests, checking for dangling file handles, pollution of sentinel space in malloc'd memory and similar, tossing any discontinuities in the syslog if it finds any.  The code is clean and when it comes back for the next "accept()" there's nothing outstanding. This implies that either libpq has a problem or the fastcgi wrapper library does.  The former I've used a great deal over a long period of time (years) and have never had it cause trouble. The application does some "straight through" stuff out of cron as well and there I can run valgrind against it, which I have and there are no complaints with it unable to find any dangling allocations -- albeit not every function that is used in serving web requests in that library is used in the cron job. That leaves the fastcgi wrapper library.  There is a caution in there about anything that modifies the environment; I removed all the setproctitle() calls which I use to identify status and such, and there are no calls to setenv or putenv anywhere, nor any attempts to write back to any of the environment pointers -- that much I'm quite sure of. Anyone seen anything like this?  For obvious reasons this is not good behavior but tracking it down has proved elusive. Thanks in advance! -- Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------w9EMxmDxfOVQz0PHHO0peT0M Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit

I have an application that has been a "traditional" CGI app for a long time and am converting it to use the fcgi-dev "wrapper" under Apache along with writing a little program that manages the daemons (so if something goes wrong they get restarted with logging to the admin, I can issue a kill to all of them to update the code or a restart request, etc.)

What I found is that with that wrapper in use the code runs fine -- but it leaks memory.  It is quite complex and uses the libpq library to talk to Postgres, but I've never seen problems with libpq leaking memory before.  Nonetheless I spent the rather-considerable time to make darn sure via wrapper functions that I wasn't leaking memory through inappropriate use of malloc/realloc/free or unpaired calls to the libpq functions that create allocations, including using wrappers to maintain a list of "active" allocations and outstanding requests, checking for dangling file handles, pollution of sentinel space in malloc'd memory and similar, tossing any discontinuities in the syslog if it finds any.  The code is clean and when it comes back for the next "accept()" there's nothing outstanding.

This implies that either libpq has a problem or the fastcgi wrapper library does.  The former I've used a great deal over a long period of time (years) and have never had it cause trouble.  The application does some "straight through" stuff out of cron as well and there I can run valgrind against it, which I have and there are no complaints with it unable to find any dangling allocations -- albeit not every function that is used in serving web requests in that library is used in the cron job.

That leaves the fastcgi wrapper library.  There is a caution in there about anything that modifies the environment; I removed all the setproctitle() calls which I use to identify status and such, and there are no calls to setenv or putenv anywhere, nor any attempts to write back to any of the environment pointers -- that much I'm quite sure of.

Anyone seen anything like this?  For obvious reasons this is not good behavior but tracking it down has proved elusive.

Thanks in advance!

--
Karl Denninger
karl@denninger.net
The Market Ticker
[S/MIME encrypted email preferred]
--------------w9EMxmDxfOVQz0PHHO0peT0M-- --------------ms080304010900090100080303 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC DdgwggagMIIEiKADAgECAhMA5EiKghDOXrvfxYxjITXYDdhIMA0GCSqGSIb3DQEBCwUAMIGL MQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJTmljZXZpbGxlMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExITAf BgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQTAeFw0xNzA4MTcxNjQyMTdaFw0yNzA4 MTUxNjQyMTdaMHsxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkwFwYDVQQKDBBD dWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExJTAjBgNVBAMMHEN1 ZGEgU3lzdGVtcyBMTEMgMjAxNyBJbnQgQ0EwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK AoICAQC1aJotNUI+W4jP7xQDO8L/b4XiF4Rss9O0B+3vMH7Njk85fZ052QhZpMVlpaaO+sCI KqG3oNEbuOHzJB/NDJFnqh7ijBwhdWutdsq23Ux6TvxgakyMPpT6TRNEJzcBVQA0kpby1DVD 0EKSK/FrWWBiFmSxg7qUfmIq/mMzgE6epHktyRM3OGq3dbRdOUgfumWrqHXOrdJz06xE9NzY vc9toqZnd79FUtE/nSZVm1VS3Grq7RKV65onvX3QOW4W1ldEHwggaZxgWGNiR/D4eosAGFxn uYeWlKEC70c99Mp1giWux+7ur6hc2E+AaTGh+fGeijO5q40OGd+dNMgK8Es0nDRw81lRcl24 SWUEky9y8DArgIFlRd6d3ZYwgc1DMTWkTavx3ZpASp5TWih6yI8ACwboTvlUYeooMsPtNa9E 6UQ1nt7VEi5syjxnDltbEFoLYcXBcqhRhFETJe9CdenItAHAtOya3w5+fmC2j/xJz29og1KH YqWHlo3Kswi9G77an+zh6nWkMuHs+03DU8DaOEWzZEav3lVD4u76bKRDTbhh0bMAk4eXriGL h4MUoX3Imfcr6JoyheVrAdHDL/BixbMH1UUspeRuqQMQ5b2T6pabXP0oOB4FqldWiDgJBGRd zWLgCYG8wPGJGYgHibl5rFiI5Ix3FQncipc6SdUzOQIDAQABo4IBCjCCAQYwHQYDVR0OBBYE FF3AXsKnjdPND5+bxVECGKtc047PMIHABgNVHSMEgbgwgbWAFBu1oRhUMNEzjODolDka5k4Q EDBioYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJ TmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5 c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYIJAKxAy1WBo2kY MBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4IC AQCB5686UCBVIT52jO3sz9pKuhxuC2npi8ZvoBwt/IH9piPA15/CGF1XeXUdu2qmhOjHkVLN gO7XB1G8CuluxofOIUce0aZGyB+vZ1ylHXlMeB0R82f5dz3/T7RQso55Y2Vog2Zb7PYTC5B9 oNy3ylsnNLzanYlcW3AAfzZcbxYuAdnuq0Im3EpGm8DoItUcf1pDezugKm/yKtNtY6sDyENj tExZ377cYA3IdIwqn1Mh4OAT/Rmh8au2rZAo0+bMYBy9C11Ex0hQ8zWcvPZBDn4v4RtO8g+K uQZQcJnO09LJNtw94W3d2mj4a7XrsKMnZKvm6W9BJIQ4Nmht4wXAtPQ1xA+QpxPTmsGAU0Cv HmqVC7XC3qxFhaOrD2dsvOAK6Sn3MEpH/YrfYCX7a7cz5zW3DsJQ6o3pYfnnQz+hnwLlz4MK 17NIA0WOdAF9IbtQqarf44+PEyUbKtz1r0KGeGLs+VGdd2FLA0e7yuzxJDYcaBTVwqaHhU2/ Fna/jGU7BhrKHtJbb/XlLeFJ24yvuiYKpYWQSSyZu1R/gvZjHeGb344jGBsZdCDrdxtQQcVA 6OxsMAPSUPMrlg9LWELEEYnVulQJerWxpUecGH92O06wwmPgykkz//UmmgjVSh7ErNvL0lUY UMfunYVO/O5hwhW+P4gviCXzBFeTtDZH259O7TCCBzAwggUYoAMCAQICEwCg0WvVwekjGFiO 62SckFwepz0wDQYJKoZIhvcNAQELBQAwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3Jp ZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBD QTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExMQyAyMDE3IEludCBDQTAeFw0xNzA4MTcyMTIx MjBaFw0yMjA4MTYyMTIxMjBaMFcxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRswGQYDVQQDDBJrYXJsQGRlbm5pbmdlci5uZXQw ggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQC+HVSyxVtJhy3Ohs+PAGRuO//Dha9A 16l5FPATr6wude9zjX5f2lrkRyU8vhCXTZW7WbvWZKpcZ8r0dtZmiK9uF58Ec6hhvfkxJzbg 96WHBw5Fumd5ahZzuCJDtCAWW8R7/KN+zwzQf1+B3MVLmbaXAFBuKzySKhKMcHbK3/wjUYTg y+3UK6v2SBrowvkUBC+jxNg3Wy12GsTXcUS/8FYIXgVVPgfZZrbJJb5HWOQpvvhILpPCD3xs YJFNKEPltXKWHT7Qtc2HNqikgNwj8oqOb+PeZGMiWapsatKm8mxuOOGOEBhAoTVTwUHlMNTg 6QUCJtuWFCK38qOCyk9Haj+86lUU8RG6FkRXWgMbNQm1mWREQhw3axgGLSntjjnznJr5vsvX SYR6c+XKLd5KQZcS6LL8FHYNjqVKHBYM+hDnrTZMqa20JLAF1YagutDiMRURU23iWS7bA9tM cXcqkclTSDtFtxahRifXRI7Epq2GSKuEXe/1Tfb5CE8QsbCpGsfSwv2tZ/SpqVG08MdRiXxN 5tmZiQWo15IyWoeKOXl/hKxA9KPuDHngXX022b1ly+5ZOZbxBAZZMod4y4b4FiRUhRI97r9l CxsP/EPHuuTIZ82BYhrhbtab8HuRo2ofne2TfAWY2BlA7ExM8XShMd9bRPZrNTokPQPUCWCg CdIATQIDAQABo4IBzzCCAcswPAYIKwYBBQUHAQEEMDAuMCwGCCsGAQUFBzABhiBodHRwOi8v b2NzcC5jdWRhc3lzdGVtcy5uZXQ6ODg4ODAJBgNVHRMEAjAAMBEGCWCGSAGG+EIBAQQEAwIF oDAOBgNVHQ8BAf8EBAMCBeAwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMEMDMGCWCG SAGG+EIBDQQmFiRPcGVuU1NMIEdlbmVyYXRlZCBDbGllbnQgQ2VydGlmaWNhdGUwHQYDVR0O BBYEFLElmNWeVgsBPe7O8NiBzjvjYnpRMIHKBgNVHSMEgcIwgb+AFF3AXsKnjdPND5+bxVEC GKtc047PoYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UE BwwJTmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRh IFN5c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYITAORIioIQ zl6738WMYyE12A3YSDAdBgNVHREEFjAUgRJrYXJsQGRlbm5pbmdlci5uZXQwDQYJKoZIhvcN AQELBQADggIBAJXboPFBMLMtaiUt4KEtJCXlHO/3ZzIUIw/eobWFMdhe7M4+0u3te0sr77QR dcPKR0UeHffvpth2Mb3h28WfN0FmJmLwJk+pOx4u6uO3O0E1jNXoKh8fVcL4KU79oEQyYkbu 2HwbXBU9HbldPOOZDnPLi0whi/sbFHdyd4/w/NmnPgzAsQNZ2BYT9uBNr+jZw4SsluQzXG1X lFL/qCBoi1N2mqKPIepfGYF6drbr1RnXEJJsuD+NILLooTNf7PMgHPZ4VSWQXLNeFfygoOOK FiO0qfxPKpDMA+FHa8yNjAJZAgdJX5Mm1kbqipvb+r/H1UAmrzGMbhmf1gConsT5f8KU4n3Q IM2sOpTQe7BoVKlQM/fpQi6aBzu67M1iF1WtODpa5QUPvj1etaK+R3eYBzi4DIbCIWst8MdA 1+fEeKJFvMEZQONpkCwrJ+tJEuGQmjoQZgK1HeloepF0WDcviiho5FlgtAij+iBPtwMuuLiL shAXA5afMX1hYM4l11JXntle12EQFP1r6wOUkpOdxceCcMVDEJBBCHW2ZmdEaXgAm1VU+fnQ qS/wNw/S0X3RJT1qjr5uVlp2Y0auG/eG0jy6TT0KzTJeR9tLSDXprYkN2l/Qf7/nT6Q03qyE QnnKiBXWAZXveafyU/zYa7t3PTWFQGgWoC4w6XqgPo4KV44OMYIFBzCCBQMCAQEwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBglghkgBZQMEAgMFAKCCAkUw GAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMjIwNjI2MjM0NjQx WjBPBgkqhkiG9w0BCQQxQgRAFuFgqIK1gtLij066ErbrSHE7QjHZd/+Q88Q6HSnYp1VtJPNN CWU4kHscUm86BChRq34yz0UiIyYTvVSwMqjeOTBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFl AwQBKjALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3 DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGjBgkrBgEEAYI3EAQxgZUwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTCBpQYLKoZIhvcNAQkQAgsxgZWg gZIwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lz dGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0 ZW1zIExMQyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBgkqhkiG9w0BAQEF AASCAgCkb1B7gXo/I4OfXqA/zMmoKIeXmZ94QpP9e7XP4CbrzSJKOsZbqh0hItNEaKoklGU6 119SMPMC+rUc3L5CLGB+Fxkmk6njR7/RTrJGB0txJ8OV08rt4zwStySVHBJeQDklDxAGvBku 2rFaleMSdpfgcZ8flqmL9yJJHIfJLJ8e5nmqFzZJYYOmE1aiOJt0ek6Uzwq0wqnJfCd41Wud vMR1ZyPl83Z/VcHE5lJGCZsrIPgpqgZ3/TQlemP0WaoUHun9UzVrUycY+mSoDUBK3bVJynP2 r85cUeujN5LI/tPS0YEFGYEJ5ihDI9xuOcLsuRRZZQkYRAOObpophNAxu2q5+T7FNMfkfNZC 02wSQMGeU9q33/ErPXW1fZvwadu3FoaRYi18xZ4kK8TUhyOY7+zDZBCyx8dbajIGgljDDgom L/V70a1jAOOUZfM7e3AoTLyDzmFWuPIGGzCW2p1T/3hz4zh8B2AI6JmGA4S7WG8YTH0SFzQF Bwm3WvbaNNRzwWhoLDd5ppgLdZsaUnkd7SasWLLPPCY3TXEdijpNSsZCw28v//xKo8SeRyGw C68LTYGSvym3bjHirgDm/2/fG2TmkK8dcPWF6bN6Rpml22sih7rPF53g1nItWiWieFWR10CZ y68Z0ioxteF2bsTKIZgi3FtltKol8i65ANghZfELmAAAAAAAAA== --------------ms080304010900090100080303--