Print Thread
Bug on FPU calculation
#48325 12/10/13 08:56 PM
Joined: Aug 2003
Posts: 101
Chesapeake Bay
Member
OP Offline
Member
Joined: Aug 2003
Posts: 101
Chesapeake Bay
There is a bug on calculation of large amount of numbers with 2 decimal numbers.
PLD - gives: .10 SQL+: .05

Table defined as:
SQL> desc dropme_test
Name Type Nullable Default Comments
---- ------------ -------- ------- --------
COL1 NUMBER(22,2) Y

SQL+
[Linked Image]
PLD [Linked Image]
Setting [Linked Image]

windows 7
ora 10.2 64bit
pld 10.0.05.1710

Re: Bug on FPU calculation
null0000 #48326 12/10/13 08:58 PM
Joined: Aug 2003
Posts: 101
Chesapeake Bay
Member
OP Offline
Member
Joined: Aug 2003
Posts: 101
Chesapeake Bay
Data sample:
+000000420854067.50
+000000000000000.00
+000000000408640.63
+000000000001762.50
+000000000000000.00
+000000000034974.00
+000000000016843.91
+000000000000000.00
+000000000000000.00
-000000026293472.67
-000000036958788.40
-000000004267889.58
-000000055527262.86
-000000001585748.78
-000000000003168.00
-000000004265526.66
-000000001174908.54
-000000012318261.16
-000000000490548.21
-000000006070603.28
-000000923009320.85
-000000000063496.20

it is a financial data (signed), so precision is important

Re: Bug on FPU calculation
null0000 #48327 12/10/13 09:45 PM
Joined: Aug 2003
Posts: 101
Chesapeake Bay
Member
OP Offline
Member
Joined: Aug 2003
Posts: 101
Chesapeake Bay
Gentlemen, I've done some debugging for you...
[Linked Image]
[Linked Image]

I am dealing with finance -- so need to rely on tool 100%. Can you please expedite solution.
Thanks.

Re: Bug on FPU calculation
null0000 #48332 12/11/13 10:33 AM
Joined: Aug 1999
Posts: 22,173
Member
Offline
Member
Joined: Aug 1999
Posts: 22,173
To keep the full Oracle precision, go to Tools > Preferences > SQL Window and enable the "Number fields to_char" option. Now all number fields will be returned as character strings with full Oracle precision.


Marco Kalter
Allround Automations

Moderated by  support 

Link Copied to Clipboard
Powered by UBB.threads™ PHP Forum Software 7.7.4
(Release build 20200307)
Responsive Width:

PHP: 7.1.33 Page Time: 0.034s Queries: 15 (0.007s) Memory: 2.5092 MB (Peak: 3.0378 MB) Data Comp: Off Server Time: 2024-03-28 15:26:10 UTC
Valid HTML 5 and Valid CSS