ABL DECIMAL and .NET System.Double or System.Single
- Last Updated: June 19, 2019
- 1 minute read
- OpenEdge
- Version 13.0
- Documentation
The ABL DECIMAL is the ABL data type that
can represent the widest range of .NET System.Double and
System.Single values, and the reverse is also true.
However, the ABL DECIMAL cannot represent all values of the
.NET System.Double, and the .NET System.Single cannot represent all values of the ABL DECIMAL, as the examples in this topic show.
The following example raises a run-time
error because the maximum System.Double value represents
a whole number of over 300 digits that is too large to assign to
an ABL DECIMAL that can hold a maximum of 50 digits:
|
The following example raises a run-time error
because the negative value of the ABL DECIMAL (50
whole-number digits), is too large to pass to a System.Single method
parameter, which can represent whole-number values of no more than
38 digits:
|
The following example returns a value for dResult of 0.0000003469 (with
rounding), because an ABL DECIMAL can only represent
10 digits to the right of the decimal point. Any remaining digits
are truncated:
|
Note that If the ABL calculation and DECIMAL data type
maintained the full precision of the .NET System.Double,
the value of dResult would be 0.00000034687985 (with rounding).