Display Precision

Customization ›› Geometry ››
GeoCrew

GeoView performs all coordinate computations using double precision. All numbers are stored internally using the long format specified by the IEEE floating point standard. However, all numbers are displayed differently depending on the context and precision. Displaying full precision numbers to users may render displaying information useless, as floating-point numbers have a finite precision of roughly 16 significant decimal digits.


Application enables following coordinate display precision customization (based on specified coordinate dimensionality):



Please note that display precision does not affect rounding numbers - values simply get truncated. For example:


XY precision set to "0.0000" and Z precision set to "0.000" (3D coordinate dimensionality):

Actual value    = 100.4534932878, 200.2233402, 34.9949998

Displayed value = 100.4534, 200.2233, 34.994


XY precision set to "0.00" and Z precision set to "0" (3D coordinate dimensionality):

Actual value    = 100.4534932878, 200.2233402, 34.9949998

Displayed value = 100.45, 200.22, 34


XY precision set to "0.0000" and Z precision set to "0.000" (2D coordinate dimensionality):

Actual value    = 100.4534932878, 200.2233402, 34.9949998

Displayed value = 100.4534, 200.2233