[Tutor] Controlling the number of decimal places "Representation Error"
Kent Johnson
kent37 at tds.net
Fri Oct 5 18:49:44 CEST 2007
Carnell, James E wrote:
> Question:
> Is it a bad practice to avoid 0.1 representation errors
> (0.100000000000001) by just doing the following?
I'm afraid I don't understand either the problem you are trying to solve
or your proposed solution. Can you say more about it?
Kent
>
> #NOTE: all the myVariableName stuff is because Outlook keeps changing
> everything I type.
>
> #I need 2 decimal places (my input number shouldn't be over 255)
>
> myNum = 1
> myDiv = 3
>
> #10000/3 = 3333 and I remember that the last 3 digits are decimals the
> very last can have int() problems
>
> myResult = (myNum * 1000)/myDiv
>
> myArray = []
>
> For I in range( wookie loads ):
> myArray.append(myResult) #this thing can get big, but I don't
> need that much precision
> #but I do need to keep it small since
> it might be running on
> #an embedded system
>
>
> Sincerely,
>
> James Carnell
>
> _______________________________________________
> Tutor maillist - Tutor at python.org
> http://mail.python.org/mailman/listinfo/tutor
>
More information about the Tutor
mailing list