From owner-freebsd-questions@FreeBSD.ORG Wed Jul 13 07:58:22 2005 Return-Path: X-Original-To: questions@freebsd.org Delivered-To: freebsd-questions@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id ACC5616A41C for ; Wed, 13 Jul 2005 07:58:22 +0000 (GMT) (envelope-from sysjo@hades.yoafrica.com) Received: from cortizone.yoafrica.com (cortizone.yoafrica.com [196.44.176.6]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6D9F343D4C for ; Wed, 13 Jul 2005 07:58:21 +0000 (GMT) (envelope-from sysjo@hades.yoafrica.com) Received: from hades.yoafrica.com ([196.44.177.50] ident=postfix) by cortizone.yoafrica.com with esmtp (Exim 4.51 (FreeBSD)) id 1Dsc8I-0009RD-D0; Wed, 13 Jul 2005 07:58:19 +0000 Received: by hades.yoafrica.com (Postfix, from userid 1001) id 9DD376184; Wed, 13 Jul 2005 09:58:01 +0200 (CAT) Date: Wed, 13 Jul 2005 09:58:01 +0200 From: John Oxley To: Nikolas Britton Message-ID: <20050713075801.GC8552@yoafrica.com> References: <200507120941.12027.ean@hedron.org> <42D3D25A.9040107@speechpro.com> <200507121110.01535.ean@hedron.org> Mime-Version: 1.0 Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg=sha1; boundary="gDGSpKKIBgtShtf+" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.9i Cc: questions@freebsd.org Subject: Re: Whats up with KCalc? X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jul 2005 07:58:22 -0000 --gDGSpKKIBgtShtf+ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Jul 13, 2005 at 01:13:24AM -0500, Nikolas Britton wrote: > On 7/12/05, Nikolas Britton wrote: > Ok... this thing is WAY OFF... if you type in 85.49 and then hit the + > key the thing automatically jumps to > 85.48999999999999488409230252727866172791!!! My SuSE box shows "85.49" > but if you copy and paste the number into a text editor it shows > 85.48999... too. IANA Mathematician, but my guess would be that there is a rounding error. KCalc as you have all said uses 96bit long double precision. This means that floating point numbers are stored as 96bit doubles. And yes 46 is a floating point (46.000). My varsity days are a bit hazy, but I believe a double is stored as X bits ^ Y bits. Not sure of the ratios of X and Y though. This means you can only get a certain accuracy. If X is 8 for example, the accuracy is 1/(268435456) =3D~ 3.725290e-09. =20 Therefore your smallest unit number is that and any other number will be a multiple of that, raised to a power which is where the Y bits comes in. Oh I forgot to mention that X+Y =3D 96. This is probably a very bad explanation and hopefully someone can do it better :) -John --gDGSpKKIBgtShtf+ Content-Type: application/x-pkcs7-signature Content-Disposition: attachment; filename="smime.p7s" Content-Transfer-Encoding: base64 MIIIkgYJKoZIhvcNAQcCoIIIgzCCCH8CAQExCzAJBgUrDgMCGgUAMAsGCSqGSIb3DQEHAaCC BhYwggLPMIICOKADAgECAgMO0a0wDQYJKoZIhvcNAQEEBQAwYjELMAkGA1UEBhMCWkExJTAj BgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQ ZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA1MDUzMTA2NTEzMloXDTA2MDUzMTA2 NTEzMlowQzEfMB0GA1UEAxMWVGhhd3RlIEZyZWVtYWlsIE1lbWJlcjEgMB4GCSqGSIb3DQEJ ARYRam9obkB5b2FmcmljYS5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDa ZqyI7h6jGtwHQhbqtJlwQy9isJivpp+Cju1sorwJG0FE7Irqoc4impUG6aBGVFjunlMwA76A d6NInU9+KIwr1SRJO4d6qZQhUq810QMpj3vrn7TOtPbnrqKiK1tU//86WVuJHGVc3/4pYpmP jSSilAV8Gj+12YJlKP2ClbnkUF0PwDvEVlJxRjIqrW6Crl5SeCqyJ+JxFK7WQVQ6+nusuO37 0qj0FytU2IE8kxqveKstObVUwd0lQhNTS29e2ytvcNgXk2TrqlBhLpxV9xh/l52jYlRXk8aP HYxnWU/iJmXCgLl9NADpwaV3xajGyFMNSK1Plsm/r4GthtA5vdPXAgMBAAGjLjAsMBwGA1Ud EQQVMBOBEWpvaG5AeW9hZnJpY2EuY29tMAwGA1UdEwEB/wQCMAAwDQYJKoZIhvcNAQEEBQAD gYEAV+aYxxGXhdvzxI4uT4EAKvxdqGzZ1ZVAjoszmpNp2nLTMZ+juzSoqT+SiwoaLQia/T+m BIInP+JnRelE5up8zd1JXHxCjB8StssZNusEE8Hz2c/hZyAIyEdmRRY/mwlHntBu7nkNnmuq LuSeBR8g+ChtyOHZ+IyUnYDlWtUrLM4wggM/MIICqKADAgECAgENMA0GCSqGSIb3DQEBBQUA MIHRMQswCQYDVQQGEwJaQTEVMBMGA1UECBMMV2VzdGVybiBDYXBlMRIwEAYDVQQHEwlDYXBl IFRvd24xGjAYBgNVBAoTEVRoYXd0ZSBDb25zdWx0aW5nMSgwJgYDVQQLEx9DZXJ0aWZpY2F0 aW9uIFNlcnZpY2VzIERpdmlzaW9uMSQwIgYDVQQDExtUaGF3dGUgUGVyc29uYWwgRnJlZW1h aWwgQ0ExKzApBgkqhkiG9w0BCQEWHHBlcnNvbmFsLWZyZWVtYWlsQHRoYXd0ZS5jb20wHhcN MDMwNzE3MDAwMDAwWhcNMTMwNzE2MjM1OTU5WjBiMQswCQYDVQQGEwJaQTElMCMGA1UEChMc VGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNvbmFs IEZyZWVtYWlsIElzc3VpbmcgQ0EwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBAMSmPFVz VftOucqZWh5owHUEcJ3f6f+jHuy9zfVb8hp2vX8MOmHyv1HOAdTlUAow1wJjWiyJFXCO3cnw K4Vaqj9xVsuvPAsH5/EfkTYkKhPPK9Xzgnc9A74r/rsYPge/QIACZNenprufZdHFKlSFD0gE f6e20TxhBEAeZBlyYLf7AgMBAAGjgZQwgZEwEgYDVR0TAQH/BAgwBgEB/wIBADBDBgNVHR8E PDA6MDigNqA0hjJodHRwOi8vY3JsLnRoYXd0ZS5jb20vVGhhd3RlUGVyc29uYWxGcmVlbWFp bENBLmNybDALBgNVHQ8EBAMCAQYwKQYDVR0RBCIwIKQeMBwxGjAYBgNVBAMTEVByaXZhdGVM YWJlbDItMTM4MA0GCSqGSIb3DQEBBQUAA4GBAEiM0VCD6gsuzA2jZqxnD3+vrL7CF6FDlpSd f0whuPg2H6otnzYvwPQcUCCTcDz9reFhYsPZOhl+hLGZGwDFGguCdJ4lUJRix9sncVcljd2p nDmOjCBPZV+V2vf3h9bGCE6u9uo05RAaWzVNd+NWIXiC3CEZNd4ksdMdRv9dX2VPMYICRDCC AkACAQEwaTBiMQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0 eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIElzc3VpbmcgQ0EC Aw7RrTAJBgUrDgMCGgUAoIGxMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcN AQkFMQ8XDTA1MDcxMzA3NTgwMVowIwYJKoZIhvcNAQkEMRYEFHsHxR8Fz/yfpDOwsjXGOeRw sn1nMFIGCSqGSIb3DQEJDzFFMEMwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqG SIb3DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMA0GCSqGSIb3DQEBAQUABIIBAMwm SrYeNcRzjSfO4JzO0trF+Ult76vFqG64YT0LCIkHx2FdeKMEqE8oIOia8pCUt/ssBU0v0EVW xsa0niLMmH+wryC8l0k0MjIYgQf5XqgeiP3OivCIV0+fA5lvjR3lMZJKSaoFwIbjSIguDgzl AxFMgimiqqzBR8Iae4A8rF26kEStrzYUhnlgqZYsj+NjfZfrEGlUyg3FLx97Rk/TW22ULEdA eZ0/fUXL9+uDk+lVngB6CxZpahfoSLUbsoOz9UUtoelvrL4AP9h9WOgMOeex4E4qQtFlICh4 W94lNfSYj4xJsal0rkCO4j9hTyqAWi9oK8faCk3SzILL7YkW7gI= --gDGSpKKIBgtShtf+--