When the limit of linearity (Correlation Coeffiecient R2) is ≥0.999 & the result 0.99856 can considered acceptable or not? i.e. Is it ok to round off 0.99856 to 0.999?
Hi, as the specification for linearity is stated to 3 decimal places the calculated result should be reported to 3 decimal places also. So in your eg a result of 0.99856 would meet the specification of >= 0.999