Date: Wed, 13 Jul 2005 09:58:01 +0200 From: John Oxley <john@yoafrica.com> To: Nikolas Britton <nikolas.britton@gmail.com> Cc: questions@freebsd.org Subject: Re: Whats up with KCalc? Message-ID: <20050713075801.GC8552@yoafrica.com> In-Reply-To: <ef10de9a05071223136515c381@mail.gmail.com> References: <ef10de9a05071203165b346ce1@mail.gmail.com> <200507120941.12027.ean@hedron.org> <42D3D25A.9040107@speechpro.com> <200507121110.01535.ean@hedron.org> <ef10de9a05071213443f407987@mail.gmail.com> <ef10de9a05071223136515c381@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--gDGSpKKIBgtShtf+ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Jul 13, 2005 at 01:13:24AM -0500, Nikolas Britton wrote: > On 7/12/05, Nikolas Britton <nikolas.britton@gmail.com> wrote: > Ok... this thing is WAY OFF... if you type in 85.49 and then hit the + > key the thing automatically jumps to > 85.48999999999999488409230252727866172791!!! My SuSE box shows "85.49" > but if you copy and paste the number into a text editor it shows > 85.48999... too. IANA Mathematician, but my guess would be that there is a rounding error. KCalc as you have all said uses 96bit long double precision. This means that floating point numbers are stored as 96bit doubles. And yes 46 is a floating point (46.000). My varsity days are a bit hazy, but I believe a double is stored as X bits ^ Y bits. Not sure of the ratios of X and Y though. This means you can only get a certain accuracy. If X is 8 for example, the accuracy is 1/(268435456) =3D~ 3.725290e-09. =20 Therefore your smallest unit number is that and any other number will be a multiple of that, raised to a power which is where the Y bits comes in. Oh I forgot to mention that X+Y =3D 96. This is probably a very bad explanation and hopefully someone can do it better :) -John --gDGSpKKIBgtShtf+ Content-Type: application/x-pkcs7-signature Content-Disposition: attachment; filename="smime.p7s" Content-Transfer-Encoding: base64 MIIIkgYJKoZIhvcNAQcCoIIIgzCCCH8CAQExCzAJBgUrDgMCGgUAMAsGCSqGSIb3DQEHAaCC BhYwggLPMIICOKADAgECAgMO0a0wDQYJKoZIhvcNAQEEBQAwYjELMAkGA1UEBhMCWkExJTAj BgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQ ZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA1MDUzMTA2NTEzMloXDTA2MDUzMTA2 NTEzMlowQzEfMB0GA1UEAxMWVGhhd3RlIEZyZWVtYWlsIE1lbWJlcjEgMB4GCSqGSIb3DQEJ ARYRam9obkB5b2FmcmljYS5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDa ZqyI7h6jGtwHQhbqtJlwQy9isJivpp+Cju1sorwJG0FE7Irqoc4impUG6aBGVFjunlMwA76A d6NInU9+KIwr1SRJO4d6qZQhUq810QMpj3vrn7TOtPbnrqKiK1tU//86WVuJHGVc3/4pYpmP jSSilAV8Gj+12YJlKP2ClbnkUF0PwDvEVlJxRjIqrW6Crl5SeCqyJ+JxFK7WQVQ6+nusuO37 0qj0FytU2IE8kxqveKstObVUwd0lQhNTS29e2ytvcNgXk2TrqlBhLpxV9xh/l52jYlRXk8aP HYxnWU/iJmXCgLl9NADpwaV3xajGyFMNSK1Plsm/r4GthtA5vdPXAgMBAAGjLjAsMBwGA1Ud EQQVMBOBEWpvaG5AeW9hZnJpY2EuY29tMAwGA1UdEwEB/wQCMAAwDQYJKoZIhvcNAQEEBQAD gYEAV+aYxxGXhdvzxI4uT4EAKvxdqGzZ1ZVAjoszmpNp2nLTMZ+juzSoqT+SiwoaLQia/T+m BIInP+JnRelE5up8zd1JXHxCjB8StssZNusEE8Hz2c/hZyAIyEdmRRY/mwlHntBu7nkNnmuq LuSeBR8g+ChtyOHZ+IyUnYDlWtUrLM4wggM/MIICqKADAgECAgENMA0GCSqGSIb3DQEBBQUA MIHRMQswCQYDVQQGEwJaQTEVMBMGA1UECBMMV2VzdGVybiBDYXBlMRIwEAYDVQQHEwlDYXBl IFRvd24xGjAYBgNVBAoTEVRoYXd0ZSBDb25zdWx0aW5nMSgwJgYDVQQLEx9DZXJ0aWZpY2F0 aW9uIFNlcnZpY2VzIERpdmlzaW9uMSQwIgYDVQQDExtUaGF3dGUgUGVyc29uYWwgRnJlZW1h aWwgQ0ExKzApBgkqhkiG9w0BCQEWHHBlcnNvbmFsLWZyZWVtYWlsQHRoYXd0ZS5jb20wHhcN MDMwNzE3MDAwMDAwWhcNMTMwNzE2MjM1OTU5WjBiMQswCQYDVQQGEwJaQTElMCMGA1UEChMc VGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNvbmFs IEZyZWVtYWlsIElzc3VpbmcgQ0EwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBAMSmPFVz VftOucqZWh5owHUEcJ3f6f+jHuy9zfVb8hp2vX8MOmHyv1HOAdTlUAow1wJjWiyJFXCO3cnw K4Vaqj9xVsuvPAsH5/EfkTYkKhPPK9Xzgnc9A74r/rsYPge/QIACZNenprufZdHFKlSFD0gE f6e20TxhBEAeZBlyYLf7AgMBAAGjgZQwgZEwEgYDVR0TAQH/BAgwBgEB/wIBADBDBgNVHR8E PDA6MDigNqA0hjJodHRwOi8vY3JsLnRoYXd0ZS5jb20vVGhhd3RlUGVyc29uYWxGcmVlbWFp bENBLmNybDALBgNVHQ8EBAMCAQYwKQYDVR0RBCIwIKQeMBwxGjAYBgNVBAMTEVByaXZhdGVM YWJlbDItMTM4MA0GCSqGSIb3DQEBBQUAA4GBAEiM0VCD6gsuzA2jZqxnD3+vrL7CF6FDlpSd f0whuPg2H6otnzYvwPQcUCCTcDz9reFhYsPZOhl+hLGZGwDFGguCdJ4lUJRix9sncVcljd2p nDmOjCBPZV+V2vf3h9bGCE6u9uo05RAaWzVNd+NWIXiC3CEZNd4ksdMdRv9dX2VPMYICRDCC AkACAQEwaTBiMQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0 eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIElzc3VpbmcgQ0EC Aw7RrTAJBgUrDgMCGgUAoIGxMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcN AQkFMQ8XDTA1MDcxMzA3NTgwMVowIwYJKoZIhvcNAQkEMRYEFHsHxR8Fz/yfpDOwsjXGOeRw sn1nMFIGCSqGSIb3DQEJDzFFMEMwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqG SIb3DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMA0GCSqGSIb3DQEBAQUABIIBAMwm SrYeNcRzjSfO4JzO0trF+Ult76vFqG64YT0LCIkHx2FdeKMEqE8oIOia8pCUt/ssBU0v0EVW xsa0niLMmH+wryC8l0k0MjIYgQf5XqgeiP3OivCIV0+fA5lvjR3lMZJKSaoFwIbjSIguDgzl AxFMgimiqqzBR8Iae4A8rF26kEStrzYUhnlgqZYsj+NjfZfrEGlUyg3FLx97Rk/TW22ULEdA eZ0/fUXL9+uDk+lVngB6CxZpahfoSLUbsoOz9UUtoelvrL4AP9h9WOgMOeex4E4qQtFlICh4 W94lNfSYj4xJsal0rkCO4j9hTyqAWi9oK8faCk3SzILL7YkW7gI= --gDGSpKKIBgtShtf+--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20050713075801.GC8552>