Rounding rules - Openlab 2.5

Hi ludivineb

I've had a look at some data where this should be replicated and I see the value displayed in a report with all three decimal places.  Where are you seeing this?  Can you share a screenshot?

/Andy

Hello Andy,

Thank you for your reply. Here is a screenshot to visualize the rounding problem:

When "Signal" is selected in the "Scope" column, the rounding is wrong while when "Peak and groups" is selected, the rounding is right.

Thank you in advance for your answer.

Ludivine

Hi ludivineb

Ah, I see.  I had only tested it in a table, and not in a custom calculation with a scope like signal.  The rounding, I would argue, is correct in both cases, but the formatting of the results is different when the value is displayed in a Results Table.

Change the format of the result for the calculation with the scope Signal:

These are the results for three different calculations:

The calculations being

Min("Compound_Concentration",CurrentInjection.AllIdentifiedPeaks) - in my data this gives me a number like yours with the third decimal place being zero

Round(CC1,3) - CC1 is the result of the calculation above, and shows the behaviour that you described

Format(CC2,"F3") - CC2 is the result of the rounding calculation, and, IF TYPE ASSOCIATED WITH THIS CALCULATION IS STRING, gives you the behaviour you want to see

Hope that answers your question.

Regards

Andy

Hi ludivineb

Ah, I see.  I had only tested it in a table, and not in a custom calculation with a scope like signal.  The rounding, I would argue, is correct in both cases, but the formatting of the results is different when the value is displayed in a Results Table.

Change the format of the result for the calculation with the scope Signal:

These are the results for three different calculations:

The calculations being

Min("Compound_Concentration",CurrentInjection.AllIdentifiedPeaks) - in my data this gives me a number like yours with the third decimal place being zero

Round(CC1,3) - CC1 is the result of the calculation above, and shows the behaviour that you described

Format(CC2,"F3") - CC2 is the result of the rounding calculation, and, IF TYPE ASSOCIATED WITH THIS CALCULATION IS STRING, gives you the behaviour you want to see

Hope that answers your question.

Regards

Andy

Hello,

I think this has more to do with the display controls in injection results between peaks and the summary values. In the peak results you can force numeric precision on a column which will give you the remain zeros. This control is not available in in the summary results which may be why you see the difference.

Marty Adams