[AccessD] Math in VBA

Gustav Brock Gustav at cactus.dk
Thu Jul 19 14:25:42 CDT 2007


Hi Mark

Chris is right. You have to keep the _numeric_ expressions within that of a Long if you wish to maintain accuracy.

Thus:
? CDec(36 ^10)
 3656158440062980 
? CDec(36 ^ 5) * CDec(36 ^ 5)
 3656158440062976 

because CDec(36 ^ 5) = 60466176.

/gustav

>>> cjeris at fas.harvard.edu 19-07-2007 20:37 >>>
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Mark A Matte wrote:
> ttt = CDec(36 * 36 * 36)

6^6 = 46656 overflows a 16-bit signed integer.  Now we can all line up
and kick someone in the head for leaving a 16-bit signed integer type in
a program written after 1995.

Try
ttt = CDec(36) * CDec(36) * CDec(36)

Also, be careful!
> tt = CDec(36 ^ 10)

That ^ is the _floating_point_ exponentiation operator!  The result is
not the integer 6^20; if you look at tt, you will see that it ends in a
0, which no power of 6 does.  What you get there is the decimal
conversion of the floating-point exponentiation.

peace, Chris Jeris

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org 

iD8DBQFGn69k5ICCNV0oGWARAjPHAJ9dz3+coNp2KVcjqwRK0FtFVFj9bACfW+oY
JR0dThbFLOYR8UefedT5X48=
=KxwX
-----END PGP SIGNATURE-----





More information about the AccessD mailing list