From owner-freebsd-arch Thu Oct 25 16:36:10 2001 Delivered-To: freebsd-arch@freebsd.org Received: from peter3.wemm.org (c1315225-a.plstn1.sfba.home.com [24.14.150.180]) by hub.freebsd.org (Postfix) with ESMTP id C5EA737B409 for ; Thu, 25 Oct 2001 16:36:02 -0700 (PDT) Received: from overcee.netplex.com.au (overcee.wemm.org [10.0.0.3]) by peter3.wemm.org (8.11.0/8.11.0) with ESMTP id f9PNa2M41082 for ; Thu, 25 Oct 2001 16:36:02 -0700 (PDT) (envelope-from peter@wemm.org) Received: from wemm.org (localhost [127.0.0.1]) by overcee.netplex.com.au (Postfix) with ESMTP id 587C63808 for ; Thu, 25 Oct 2001 16:36:02 -0700 (PDT) (envelope-from peter@wemm.org) X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 To: arch@freebsd.org Subject: 64 bit times revisited.. Date: Thu, 25 Oct 2001 16:36:02 -0700 From: Peter Wemm Message-Id: <20011025233602.587C63808@overcee.netplex.com.au> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I know I am going to regret this, but the subject came up again where we got burnet by code that "knows" that time_t is a long, and on machines where long != int, this can get ugly when pointers to long and int are mixed up. It was pointed out that *all* the linux 64 bit arches have 64 bit time_t, so obviously it isn't too hard a problem to solve. I did a scan around our tree looking for things where size of time_t matters. Here's what I found initially: sizeof(time_t) exposed: /etc/pwd.db (pw_change, pw_expire) /var/run/utmp (ut_time) /var/log/lastlog (ll_time) dump/restore tape format (spcl. c_data, d_ddate, etc) /var/db/acct (ac_btime - begin time of a process) notable exceptions where times are explicitly sized: ufs inode format (has ufs_time_t == 32 bits explicitly) nfs (uses 32 bit timestamps in v3, and 64 bit on v4 (I believe)). notable exeptions where times are *ambiguous*: rpc on-the-wire (both int, u_int, long and u_long timestamps (!)) yp/nis (uses u_long ctime, mtime - see struct nis_oid) There are a couple of other interesting things on 64 bit machines (alpha, ia64, sparc64) as things stand right now: struct timeval { long tv_sec; /* 64 bit */ long tv_usec; /* 64 bit */ } struct timespec { time_t tv_sec; /* 32 bit */ long tv_nsec; /* 64 bit */ } Anywhere that struct timeval or struct timespec are exposed presently leads to a 32 / 64 bit incompatability. Also, struct timespec has structure alignment issues so that tv_sec actually uses 64 bits of space anyway in order to preserve the 64 bit alignment of tv_nsec. This stuff is well exposed in struct stat. I was (foolishly?) optimistic in wondering if we may be able to share the freebsd system call sysvec tables stuff between i386 and ia64 applications but sadly, it is not to be. There are too many "long"'s in enough syscalls to break it. And then we have to swab struct stat and a bunch of other things. The same thing will apply to x86-64 which runs both 32 and 64 bit compiled applications which will have a different sizeof(long). I am starting to wonder now if we're doing ourselves a long-term disservice by locking time_t into 32 bit. People may say "but we can resolve it closer to the time", but they also said that about 32 bit file sizes. Look at the trouble that Linux is still having with applications and libraries due to the transition from off_t being 32 bits to 64 bits, and recall how little pain we had because 4.4BSD did it right from the start. I am also wondering if we should reconsider time_t == int32_t given that we have at least two new 64 bit platforms coming online as we speak, and now is the next best chance that we have to make a clean break like 4.4BSD did with file sizes. Intel are planning on the IA64 architecture to last something like 20-30 years before it runs out of steam.. that is awfully close to Y2038. Maybe I'm being optomistic in thinking that FreeBSD or even unix will be around in another 20 years, but I'd sure like to think that we dont help contribute to our demise by not having a long term solution to the time_t wraparound problem. Possible solutions: 1) Do nothing. (This worked well for Y2K consultants between 1995-2000) 2) Use 64 bit time_t on new 64 bit platforms. 2) Switch to 64 bit time_t on 64 bit platforms. 3) Switch to 64 bit time_t on everything including i386. Pros/cons for each: 1) Pro: No effort required. Con: As time goes by, we're going to see more and more time calculation bugs and eventually shooting ourselves in the foot. 2) Pro: Clean break, no cost for new platforms. Con: There are still some hard coded 32 bit timestamps around that will eventually require magic number / version / format bumps. eg: UFS inodes etc. Dual-mode cpus (ia64, x86-64) may have fun. Different types (int on alpha, long in x86-64, ia64 and sparc64, int on ppc and i386 - printf format hell). Doesn't solve hard coded timestamp sizes in exposed disk and wire structures. 3) Pro: Clean break for new platforms. We could even use the same type (long) across the board, including i386, and that saves trouble with printf etc. (%ld everywhere) Con: This hurts the alpha, our only established 64 bit platform. Doesn't solve dual-mode cpu problems Doesn't solve hard coded timestamp sizes in exposed disk and wire structures. 4) Pro: All wire and on-disk formats mentioning time_t will be compatable across the entire freebsd range. The Pentium21 (to be released in year 2021) will be Y2038 safe. :-) Con: Break i386 *.o compatability. We would be subjecting ourselves to the same sort of pain that the rest of the unix world went through (and is still going through with the 64 bit filesize transition). Doesn't solve things like "int32_t start_time;" in exposed disk and on-the-wire structures. Printf format hell (%qd on i386, %ld on the 64 bit platforms) where it causes real screwups if there is a mistake. Personally, I feel that we should (at the least) do #2 (64 bit times for new 64 bit platforms), and maybe even #3 (change the alpha too) if we have the willpower. The alpha problem can be solved with version bumps etc and keep old syscall interfaces around like we do with compat_43. #3 has the advantage of being able to use a constant printf format and have the same type everywhere (long). I'm somewhat disappointed that we didn't make a clean break with time_t == long on the alpha. Even DECpaq realized that was a mistake and tried to switch to 64 bit time_t in Tru64 v5.0. We still have to deal with some on-disk and on-wire exposure: /etc/pwd.db (pw_change, pw_expire) This can be fixed fairly painlessly. The .db files have typed records so we can create new record types and generate old ones for compatability with 32 bit applications. The old record type could be changed from time_t to int32_t pw_change etc. Only pwd_mkdb ever writes to this, so it can be done in a safe, upward compatable fashion so that old apps will continue to run for years. /var/run/utmp (ut_time) /var/log/lastlog (ll_time) ouch! we learned the hard way on this stuff with the change in size of login names etc. We could explicitly size these timestamps while waiting for the next reason to change the format or switching to a real utmp API (and changing the timestamps at the same time). dump/restore tape format (spcl. c_data, d_ddate, etc) create a new record type for 64 bit timestamps and switch the current record specification to int32_t (thats all that is in UFS right now after all). /var/db/acct (ac_btime - begin time of a process) Only system tools read this. This should be harmless to change at any time, but should probably be switched to int32_t for now. In summary.. I think we've been shortsighted and made a mistake. I dont believe it is too late to fix - in fact, we have some excellent opportunities to fix it coming up. My way of thinking is that we should take the opportunity, but the bigger question is whether or not to disturb the alpha.. I suspect we can, and that we can make it painless enough for what the (relatively few) people are using them for. So.. What do people think? Let the bikeshedding begin... Cheers, -Peter -- Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message