PRECISION Statement

Sets the degree of precision to which values are calculated until another precision statement is encountered. A default of 4 is used if no PRECISION statement is executed.

Syntax

PRECISION precision-value

Syntax elements

precision-value The number (from 0 to 127) of decimal places to which values are calculated (other than those returned by some mathematical functions) and to which all values are truncated.

Operation

A program can have multiple PRECISION statements. A PRECISION statement remains in force until another PRECISION statement is executed or the program ends. Values passed via the COMMON statement and all called subroutines need not have the same precision as the calling program.

If a literal is specified, its precision will be used until a result is truncated. The operations that will truncate the result, and possibly intermediate values, to precision digits are Multiply, Divide, SQRT and run-time conversion of a string to a number. Mathematical functions COS, EXP, LN, PWR, SIN, and TAN calculate results to fixed precision of five decimal places, but the result calculated is truncated if the current precision is set to a value less than 5.

If a variable is assigned a value with more decimal places than are specified in the PRECISION statement, the extra places are truncated. Rounding does not take place.

Examples

PRECISION 3
PRINT SQRT(5)

Prints 2.236, because the PRECISION statement specifies 3 decimal places.

PRECISION 3
X = 9.123456
CRT X
CRT X + 0
CRT X * 1

Does not truncate X initially, as X is set to a literal, and thus displays 9.123456. Then displays the same value again, as an addition uses the larger of the precision of the two numbers. X is then temporarily truncated (due to the multiply operation) to the precision generally in effect (3), and thus displays 9.123. The value of X is unchanged, however, until an operation like X = X * 1 is executed, when X is truncated to the precision generally in effect.

PRECISION 3
Y = "4.723428"
X = Y + 1
Z = Y * 1
PRINT X, Y, Z

Prints 5.723 as the value of X, 4.723428 as the value of Y and 4.723 as the value of Z. The value of Y is printed to 6 decimal places as Y is set to a literal. The value of X is truncated to 3 decimal places as, in this case, the addition of 1 is effectively a conversion of a string to a number (Y was assigned a literal string value because of the double quotes). The value of Z is calculated via a multiply and is therefore truncated to 3 decimal places because of the PRECISION statement.