From owner-freebsd-perl@FreeBSD.ORG Fri Jan 2 16:31:05 2009 Return-Path: Delivered-To: freebsd-perl@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 67AF41065676 for ; Fri, 2 Jan 2009 16:31:05 +0000 (UTC) (envelope-from fernan.aguero@gmail.com) Received: from qw-out-2122.google.com (qw-out-2122.google.com [74.125.92.26]) by mx1.freebsd.org (Postfix) with ESMTP id 170408FC08 for ; Fri, 2 Jan 2009 16:31:04 +0000 (UTC) (envelope-from fernan.aguero@gmail.com) Received: by qw-out-2122.google.com with SMTP id 9so3833058qwb.7 for ; Fri, 02 Jan 2009 08:31:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:cc:mime-version:content-type; bh=xGsrgY6SCZh9G85bPpLhBypmI1lPUBVFL703cU93eTI=; b=vHRaMNHx8iSSBqc3Z9Mo9mMBM5M1AnSLvffea2KcUF52ajKdh0nbDi8bQSA1SiJPUL DMNRHfie1lam8SExN4waBBEpRsLpDB8IQdWJ0lfOoaIeJI+w03AGLHv36u3ZHks43G7Y 2WZ4VpZH9S+QLSaIbhXqW3ywPnurs1Y56bRxQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:mime-version:content-type; b=iXvp14bVH3jrBiHlsR+jJAvpGC6ZT5zCksTkEo12LV49IPEt3qSnWqbmAuqrNoCpuK j3PmL6yLzQR8bdYsuGk+7tIyDu6+GYLqKO8O+cge3HCtFyFEcO5D4cXump0UYNfb007s xXIKkVT3gfv8aKrlNVFYLZmoZesmpux6c/3Gc= Received: by 10.214.80.21 with SMTP id d21mr15006796qab.267.1230911886861; Fri, 02 Jan 2009 07:58:06 -0800 (PST) Received: by 10.214.45.3 with HTTP; Fri, 2 Jan 2009 07:58:06 -0800 (PST) Message-ID: <520894aa0901020758o79bb1233teee539f6d599d10a@mail.gmail.com> Date: Fri, 2 Jan 2009 13:58:06 -0200 From: "Fernan Aguero" To: freebsd-perl@freebsd.org MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_85910_20528024.1230911886862" Cc: Subject: Storable byteorder incompatibilities in different FreeBSD installations X-BeenThere: freebsd-perl@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: maintainer of a number of perl-related ports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Jan 2009 16:31:06 -0000 ------=_Part_85910_20528024.1230911886862 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Hi, I've been bitten by binary incompatibilitiess (bytorder, and/or byte sizes of ints, longs, and whatnot) between two perl installations (both on FreeBSD, using latest perl5.8 from ports). Upon investigation, it does not seem clear to me why the two installations should be different. And it prompted me to ask a few questions :) The context: I have a perl web application running under mod_perl in FreeBSD that dumps Perl data structures using Storable (binary format) and stores them, after encoding them with Base64, into a MySQL database. This application is under production and we have just deployed a test server for development in another FreeBSD box. Binary incompatibilities when reading the Storable binary data have been observed in the past when running the web application from Linux boxes (Ubuntu, i386 or amd64). Now, they have been also observed under FreeBSD in the new test server. This prompted us to do an investigation of the issue, because up to now we were assuming we would have no problems if we kept our production servers running FreeBSD. Below is the description of each box and the output of: perl -MConfig -le "print @Config{byteorder}" perl -MConfig -le "print @Config{shortsize}" perl -MConfig -le "print @Config{intsize}" perl -MConfig -le "print @Config{longsize}" perl -MConfig -le "print @Config{longlongsize}" perl -V (I'm not including the full output, due to its size, only diffs between selected cases) 1. Production server: FreeBSD-6.3, i386, perl5.8 from ports byteorder: 12345678 shortsize: 2 intsize: 4 longsize: 4 lonlongsize: 8 2. Development/test server: FreeBSD-7.1-RC1, amd64, perl5.8 from ports byteorder: 12345678 shortsize: 2 intsize: 4 longsize:8 lonlongsize: 8 3. Yet another server where we did some tests: FreeBSD-6.3, i386, perl5.8 from ports, perl5.8 standalone build perl5.8 from ports byteorder: 12345678 shortsize: 2 intsize: 4 longsize: 4 lonlongsize: 8 perl5.8, standalone build cd /usr/ports/lang/perl5.8 && make extract cd work/perl-5.8.8 && ./Configure (answering all questions with the default values except for the prefix installation) make && make test && make install (PREFIX is /home/fernan/perl) byteorder: 1234 shortsize: 2 intsize: 4 longsize: 4 lonlongsize: 8 As you can see, a Perl built outside of the ports system, has a different binary configuration (and should produce incompatible Storable images) as the Perl built from ports. As a sidenote, the binary configuration is similar to the one found in ubuntu pre-compiled binary perl packages (Linux, see below). Attached to this message is a diff of the output produced by 'perl -V' both for the perl built from the lang/perl5.8 port, and from the perl built from sources, as explained above. 4. Other test boxes: Ubuntu-8.10, i386, perl5.8 and perl5.10 byteorder: 1234 shortsize: 2 intsize: 4 longsize: 4 lonlongsize: 8 Up to now the questions that come to mind are: i) why is it that the perl built from ports produces binary dumps (using Storable) with incompatible byteorder with respect to the one built from Perl sources? (both perls were built on the same platform, same gcc) ii) Why does the Perl built from ports have an unusual byteorder? iii) Why does Perl built from ports in different platforms produce incompatible binary dumps (this time because of a difference in longsize)? Is it because it was compiled with a different gcc (3.4.6 in FBSD-6.3 vs 4.2.1 in FBSD-7.1)? Or because it was compiled in a different platform (amd64 vs i386)? Reading the Configure script that comes with perl-5.8.8, and regarding the byteorder, it says that: "A big-endian machine like a Pyramid or a Motorola 680?0 chip will come out to 4321. A little-endian machine like a Vax or an Intel 80?86 chip would be 1234. Other machines may have weird orders like 3412. A Cray will report 87654321, an Alpha will report 12345678. If the test program works the default is probably right. I'm now running the test program..." So why is it that building Perl outside of the ports system makes the test program report '1234' which is what I would expect from an i386 box, while it gives '12345678' when running from within the ports system? Why this unusual byteorder? And now for a pragmatic and urgent question: iv) how can I use the ports system to build a Perl in different FBSD boxes that would result in the same binary configuration (byteorder, shortsize, intsize, etc.) to guarantee it would produce compatible (i.e. readable from other FBSD boxes) dumps of Perl data structures? Sorry for the long post, and have yourselves a happy and peaceful 2009! -- fernan ------=_Part_85910_20528024.1230911886862 Content-Type: text/x-patch; name=perl.gama.diff Content-Transfer-Encoding: base64 X-Attachment-Id: f_fph0qahy0 Content-Disposition: attachment; filename=perl.gama.diff LS0tIHBlcmwuc3RhbmRhbG9uZS5nYW1hCTIwMDgtMTItMzEgMTI6NDM6MDQuMDAwMDAwMDAwIC0w MzAwCisrKyBwZXJsLnBvcnRzLmdhbWEJMjAwOC0xMi0zMSAxMjo0MzoxMi4wMDAwMDAwMDAgLTAz MDAKQEAgLTEsNDIgKzEsNDYgQEAKIFN1bW1hcnkgb2YgbXkgcGVybDUgKHJldmlzaW9uIDUgdmVy c2lvbiA4IHN1YnZlcnNpb24gOCkgY29uZmlndXJhdGlvbjoKICAgUGxhdGZvcm06Ci0gICAgb3Nu YW1lPWZyZWVic2QsIG9zdmVycz02LjMtcmVsZWFzZS1wNSwgYXJjaG5hbWU9aTM4Ni1mcmVlYnNk Ci0gICAgdW5hbWU9J2ZyZWVic2QgZ2FtYS5paWIudW5zYW0uZWR1LmFyIDYuMy1yZWxlYXNlLXA1 IGZyZWVic2QgNi4zLXJlbGVhc2UtcDUgIzA6IHNhdCBvY3QgMTEgMTM6MTQ6MTQgYXJ0IDIwMDgg ZmVybmFuQGdhbWEuaWliLnVuc2FtLmVkdS5hcjp1c3JvYmpmcmVlYnNkZnJlZWJzZC02LjNzcmNz eXNnYW1hIGkzODYgJwotICAgIGNvbmZpZ19hcmdzPScnCisgICAgb3NuYW1lPWZyZWVic2QsIG9z dmVycz02LjMtcmVsZWFzZS1wMSwgYXJjaG5hbWU9aTM4Ni1mcmVlYnNkLTY0aW50CisgICAgdW5h bWU9J2ZyZWVic2QgZ2FtYS5paWIudW5zYW0uZWR1LmFyIDYuMy1yZWxlYXNlLXAxIGZyZWVic2Qg Ni4zLXJlbGVhc2UtcDEgIzA6IHdlZCBhcHIgMiAxODozNzoxNCBhcnQgMjAwOCBmZXJuYW5AZ2Ft YS5paWIudW5zYW0uZWR1LmFyOnVzcm9iamZyZWVic2RmcmVlYnNkLTYuM3NyY3N5c2dhbWEgaTM4 NiAnCisgICAgY29uZmlnX2FyZ3M9Jy1zZGUgLURwcmVmaXg9L3Vzci9sb2NhbCAtRGFyY2hsaWI9 L3Vzci9sb2NhbC9saWIvcGVybDUvNS44LjgvbWFjaCAtRHByaXZsaWI9L3Vzci9sb2NhbC9saWIv cGVybDUvNS44LjggLURtYW4zZGlyPS91c3IvbG9jYWwvbGliL3Blcmw1LzUuOC44L3BlcmwvbWFu L21hbjMgLURtYW4xZGlyPS91c3IvbG9jYWwvbWFuL21hbjEgLURzaXRlYXJjaD0vdXNyL2xvY2Fs L2xpYi9wZXJsNS9zaXRlX3BlcmwvNS44LjgvbWFjaCAtRHNpdGVsaWI9L3Vzci9sb2NhbC9saWIv cGVybDUvc2l0ZV9wZXJsLzUuOC44IC1Ec2NyaXB0ZGlyPS91c3IvbG9jYWwvYmluIC1Ec2l0ZW1h bjNkaXI9L3Vzci9sb2NhbC9saWIvcGVybDUvNS44LjgvbWFuL21hbjMgLURzaXRlbWFuMWRpcj0v dXNyL2xvY2FsL21hbi9tYW4xIC1VaV9tYWxsb2MgLVVpX2ljb252IC1VaW5zdGFsbHVzcmJpbnBl cmwgLURjYz1jYyAtRHVzZXNocnBsaWIgLURjY2ZsYWdzPS1EQVBQTExJQl9FWFA9Ii91c3IvbG9j YWwvbGliL3Blcmw1LzUuOC44L0JTRFBBTiIgLURvcHRpbWl6ZT0tTzIgLWZuby1zdHJpY3QtYWxp YXNpbmcgLXBpcGUgIC1VZF9kb3N1aWQgLVVpX2dkYm0gLUR1c2V0aHJlYWRzPW4gLUR1c2VteW1h bGxvYz15IC1EdXNlNjRiaXRpbnQnCiAgICAgaGludD1yZWNvbW1lbmRlZCwgdXNlcG9zaXg9dHJ1 ZSwgZF9zaWdhY3Rpb249ZGVmaW5lCiAgICAgdXNldGhyZWFkcz11bmRlZiB1c2U1MDA1dGhyZWFk cz11bmRlZiB1c2VpdGhyZWFkcz11bmRlZiB1c2VtdWx0aXBsaWNpdHk9dW5kZWYKICAgICB1c2Vw ZXJsaW89ZGVmaW5lIGRfc2Zpbz11bmRlZiB1c2VsYXJnZWZpbGVzPWRlZmluZSB1c2Vzb2Nrcz11 bmRlZgotICAgIHVzZTY0Yml0aW50PXVuZGVmIHVzZTY0Yml0YWxsPXVuZGVmIHVzZWxvbmdkb3Vi bGU9dW5kZWYKLSAgICB1c2VteW1hbGxvYz1uLCBiaW5jb21wYXQ1MDA1PXVuZGVmCisgICAgdXNl NjRiaXRpbnQ9ZGVmaW5lIHVzZTY0Yml0YWxsPXVuZGVmIHVzZWxvbmdkb3VibGU9dW5kZWYKKyAg ICB1c2VteW1hbGxvYz15LCBiaW5jb21wYXQ1MDA1PXVuZGVmCiAgIENvbXBpbGVyOgotICAgIGNj PSdjYycsIGNjZmxhZ3MgPSctREhBU19GUFNFVE1BU0sgLURIQVNfRkxPQVRJTkdQT0lOVF9IIC1m bm8tc3RyaWN0LWFsaWFzaW5nIC1waXBlIC1XZGVjbGFyYXRpb24tYWZ0ZXItc3RhdGVtZW50IC1J L3Vzci9sb2NhbC9pbmNsdWRlJywKLSAgICBvcHRpbWl6ZT0nLU8nLAotICAgIGNwcGZsYWdzPSct REhBU19GUFNFVE1BU0sgLURIQVNfRkxPQVRJTkdQT0lOVF9IIC1mbm8tc3RyaWN0LWFsaWFzaW5n IC1waXBlIC1XZGVjbGFyYXRpb24tYWZ0ZXItc3RhdGVtZW50IC1JL3Vzci9sb2NhbC9pbmNsdWRl JworICAgIGNjPSdjYycsIGNjZmxhZ3MgPSctREFQUExMSUJfRVhQPSIvdXNyL2xvY2FsL2xpYi9w ZXJsNS81LjguOC9CU0RQQU4iIC1ESEFTX0ZQU0VUTUFTSyAtREhBU19GTE9BVElOR1BPSU5UX0gg LWZuby1zdHJpY3QtYWxpYXNpbmcgLXBpcGUgLVdkZWNsYXJhdGlvbi1hZnRlci1zdGF0ZW1lbnQg LUkvdXNyL2xvY2FsL2luY2x1ZGUnLAorICAgIG9wdGltaXplPSctTzIgLWZuby1zdHJpY3QtYWxp YXNpbmcgLXBpcGUgJywKKyAgICBjcHBmbGFncz0nLURBUFBMTElCX0VYUD0iL3Vzci9sb2NhbC9s aWIvcGVybDUvNS44LjgvQlNEUEFOIiAtREhBU19GUFNFVE1BU0sgLURIQVNfRkxPQVRJTkdQT0lO VF9IIC1mbm8tc3RyaWN0LWFsaWFzaW5nIC1waXBlIC1XZGVjbGFyYXRpb24tYWZ0ZXItc3RhdGVt ZW50IC1JL3Vzci9sb2NhbC9pbmNsdWRlJwogICAgIGNjdmVyc2lvbj0nJywgZ2NjdmVyc2lvbj0n My40LjYgW0ZyZWVCU0RdIDIwMDYwMzA1JywgZ2Njb3NhbmR2ZXJzPScnCi0gICAgaW50c2l6ZT00 LCBsb25nc2l6ZT00LCBwdHJzaXplPTQsIGRvdWJsZXNpemU9OCwgYnl0ZW9yZGVyPTEyMzQKKyAg ICBpbnRzaXplPTQsIGxvbmdzaXplPTQsIHB0cnNpemU9NCwgZG91Ymxlc2l6ZT04LCBieXRlb3Jk ZXI9MTIzNDU2NzgKICAgICBkX2xvbmdsb25nPWRlZmluZSwgbG9uZ2xvbmdzaXplPTgsIGRfbG9u Z2RibD1kZWZpbmUsIGxvbmdkYmxzaXplPTEyCi0gICAgaXZ0eXBlPSdsb25nJywgaXZzaXplPTQs IG52dHlwZT0nZG91YmxlJywgbnZzaXplPTgsIE9mZl90PSdvZmZfdCcsIGxzZWVrc2l6ZT04Cisg ICAgaXZ0eXBlPSdsb25nIGxvbmcnLCBpdnNpemU9OCwgbnZ0eXBlPSdkb3VibGUnLCBudnNpemU9 OCwgT2ZmX3Q9J29mZl90JywgbHNlZWtzaXplPTgKICAgICBhbGlnbmJ5dGVzPTQsIHByb3RvdHlw ZT1kZWZpbmUKICAgTGlua2VyIGFuZCBMaWJyYXJpZXM6Ci0gICAgbGQ9J2NjJywgbGRmbGFncyA9 Jy1XbCwtRSAgLUwvdXNyL2xvY2FsL2xpYicKKyAgICBsZD0nY2MnLCBsZGZsYWdzID0nIC1XbCwt RSAtTC91c3IvbG9jYWwvbGliJwogICAgIGxpYnB0aD0vdXNyL2xpYiAvdXNyL2xvY2FsL2xpYgot ICAgIGxpYnM9LWxnZGJtIC1sbSAtbGNyeXB0IC1sdXRpbCAtbGMKLSAgICBwZXJsbGlicz0tbG0g LWxjcnlwdCAtbHV0aWwgLWxjCi0gICAgbGliYz0sIHNvPXNvLCB1c2VzaHJwbGliPWZhbHNlLCBs aWJwZXJsPWxpYnBlcmwuYQorICAgIGxpYnM9LWxnZGJtIC1sbSAtbGNyeXB0IC1sdXRpbAorICAg IHBlcmxsaWJzPS1sbSAtbGNyeXB0IC1sdXRpbAorICAgIGxpYmM9LCBzbz1zbywgdXNlc2hycGxp Yj10cnVlLCBsaWJwZXJsPWxpYnBlcmwuc28KICAgICBnbnVsaWJjX3ZlcnNpb249JycKICAgRHlu YW1pYyBMaW5raW5nOgotICAgIGRsc3JjPWRsX2Rsb3Blbi54cywgZGxleHQ9c28sIGRfZGxzeW11 bj11bmRlZiwgY2NkbGZsYWdzPScgJworICAgIGRsc3JjPWRsX2Rsb3Blbi54cywgZGxleHQ9c28s IGRfZGxzeW11bj11bmRlZiwgY2NkbGZsYWdzPScgIC1XbCwtUi91c3IvbG9jYWwvbGliL3Blcmw1 LzUuOC44L21hY2gvQ09SRScKICAgICBjY2NkbGZsYWdzPSctRFBJQyAtZlBJQycsIGxkZGxmbGFn cz0nLXNoYXJlZCAgLUwvdXNyL2xvY2FsL2xpYicKIAogCiBDaGFyYWN0ZXJpc3RpY3Mgb2YgdGhp cyBiaW5hcnkgKGZyb20gbGlicGVybCk6IAotICBDb21waWxlLXRpbWUgb3B0aW9uczogUEVSTF9N QUxMT0NfV1JBUCBVU0VfTEFSR0VfRklMRVMgVVNFX1BFUkxJTworICBDb21waWxlLXRpbWUgb3B0 aW9uczogTVlNQUxMT0MgUEVSTF9NQUxMT0NfV1JBUCBVU0VfNjRfQklUX0lOVAorICAgICAgICAg ICAgICAgICAgICAgICAgVVNFX0xBUkdFX0ZJTEVTIFVTRV9QRVJMSU8KKyAgTG9jYWxseSBhcHBs aWVkIHBhdGNoZXM6CisJZGVmaW5lZC1vcgogICBCdWlsdCB1bmRlciBmcmVlYnNkCi0gIENvbXBp bGVkIGF0IERlYyAyMyAyMDA4IDIwOjI2OjQzCisgIENvbXBpbGVkIGF0IE1heSAyMCAyMDA4IDEw OjMzOjA1CiAgIEBJTkM6Ci0gICAgL2hvbWUvZmVybmFuL3BlcmwvbGliLzUuOC44L2kzODYtZnJl ZWJzZAotICAgIC9ob21lL2Zlcm5hbi9wZXJsL2xpYi81LjguOAotICAgIC9ob21lL2Zlcm5hbi9w ZXJsL2xpYi9zaXRlX3BlcmwvNS44LjgvaTM4Ni1mcmVlYnNkCi0gICAgL2hvbWUvZmVybmFuL3Bl cmwvbGliL3NpdGVfcGVybC81LjguOAotICAgIC9ob21lL2Zlcm5hbi9wZXJsL2xpYi9zaXRlX3Bl cmwKKyAgICAvdXNyL2xvY2FsL2xpYi9wZXJsNS81LjguOC9CU0RQQU4KKyAgICAvdXNyL2xvY2Fs L2xpYi9wZXJsNS9zaXRlX3BlcmwvNS44LjgvbWFjaAorICAgIC91c3IvbG9jYWwvbGliL3Blcmw1 L3NpdGVfcGVybC81LjguOAorICAgIC91c3IvbG9jYWwvbGliL3Blcmw1L3NpdGVfcGVybAorICAg IC91c3IvbG9jYWwvbGliL3Blcmw1LzUuOC44L21hY2gKKyAgICAvdXNyL2xvY2FsL2xpYi9wZXJs NS81LjguOAogICAgIC4K ------=_Part_85910_20528024.1230911886862--