Math

[Adron]

Calculate 8000 * 0.3 - 8000 * 3 / 10

Comments?

[Explicit]

0?

[Adron]

0?

What version of VB did you use to come to that conclusion? If you did not, consider why this was posted to the VB forum...

[Warrior]

I got 0 (VB6)

[QwertyMonster]

I got 0 too (I used Visual basic 6).

[Explicit]

0?

What version of VB did you use to come to that conclusion? If you did not, consider why this was posted to the VB forum...

Pffffft, who needs VB when you have your fingers?  I counted out to 8000, and chopped off a finger to attain the three-tenths fraction, then smashed my hands together and found nothing was left.

[Adron]

Hmm, weird. I did that in VB6, and did not get 0. Just typed it in the debug window,

Code Select Expand


?8000 * 0.3 - 8000 * 3 / 10


[Ringo]

I got -8.88178419700125E-14
[Edit]: and -1500 in a string

[MyndFyre]

Hmm, weird. I did that in VB6, and did not get 0. Just typed it in the debug window,

Code Select Expand


?8000 * 0.3 - 8000 * 3 / 10


Perhaps the debug window doesn't follow order of operations?

[Hdx]

Wow another reason ive droped VB.
But one question. WTF!?!?!
~-~(HDX)~-~

[Adron]

Perhaps the debug window doesn't follow order of operations?

Well, it originally came from a larger program. So it's not just the debug window.

[l)ragon]

Calculate 8000 * 0.3 - 8000 * 3 / 10

Comments?

Code Select Expand

'VB6
Private Sub Form_Load()
Dim a As Double
Dim b As Double
Dim x As Double
Dim y As Double
    a = 8000 * 0.3     '2400
    b = 8000 * 3 / 10  '2400
    x = a - b
    y = 8000 * 0.3 - 8000 * 3 / 10
Debug.Print x 'Real answer
'= 0
Debug.Print y 'Real Problem
'= -8.88178419700125E-14
End Sub
Is this problem only with subtraction? (Addition works fine.)

I guess the solution to this is to create your own calculation processor.

[Ringo]

Currency can do it all in one:
Const Zero As Currency = 8000 * 0.3 - 8000 * 3 / 10

[Adron]

Currency can do it all in one:
Const Zero As Currency = 8000 * 0.3 - 8000 * 3 / 10

Currency can't handle the size of the answer so it rounds to zero. I'll give you another test for that though:

Const Answer As Currency = Fix(8000 * 0.3)

[Ringo]

hmm, what about CDec(CDbl(8000 * 0.3) - CDbl(8000 * 3 / 10))?