Numeric Operations

Numbers may be stored internally in several different ways, which can have a noticeable effect on the performance of operations requiring numeric parameters. These are:

Binary numbers are stored together with a scale value. This scale value can range from zero to the maximum scaling factor, or precision, in force when the number is generated. Precision is dynamic and can be changed at any point in the program (via the PRECISION statement). The scale value of a variable is more difficult to predict than previous releases. However, in general, numbers are less likely to overflow as the scale value is only as high as is required to hold the number.

The scale value of the result of a calculation depends both on the scale values of the operands and the type of operation:

Operations that require integer parameters are able to use numbers with a zero scale value directly, but numbers with a non-zero scale factor have to be unscaled first. It is therefore preferable in this case to avoid operations that generate numbers with decimal places.

Note

  • Literals and constants with decimal places are not truncated to the current PRECISION, so a formula may contain values to a greater accuracy than the final result.

  • Assignment is not affected by PRECISION, so VAR2=VAR1  assigns the exact value of VAR1  to VAR2  even if it has a scale factor larger than the current PRECISION.

  • The PRECISION can be dynamically changed while a program is running. Changing the PRECISION only affects the results of subsequent calculations.

  • Programs and CALLed subroutines need not have the same PRECISION. A RETURN causes the PRECISION to be reset to its value at the CALL.

If the value of a binary number, or the result of binary arithmetic, is of a magnitude that cannot be stored in 48 bit two's complement binary (approximately 14 decimal digits), it is converted to a string number and calculations are performed using string number math routines. If the final result of an operation performed using string number math routines can be stored in 48 bit two's complement binary within the limits of the current precision, it is converted and stored as binary.

Considerations for Efficiency

Note

Constant values are compiled into literals in the object code every time they are used; assigning to a variable only compiles the literal where it is assigned. Therefore using EQUATE increases object code size.