Contents

Calculations for Assessing Data Quality

Effective Date: 07/2008
Point of Contact: Quality Engineer for Regulatory Programs

This exhibit provides formulas that are typically used to quantify measures for assessing data quality. All relevant calculations performed must be applied as presented in this exhibit.

Calculations are provided for:

• precision
• accuracy
• measures of agreement
• detection limits/sensitivity
• uncertainty
• control charts.

Precision

Precision is a measure of the reproducibility of measurements taken under specified conditions. Sample precision is typically assessed through the use of sample duplicates, sample replicates, and/or matrix spike duplicates. Analytical precision is typically assessed by analyzing standards repeatedly or, longer-term, comparing previous performance to current performance.

Precision is expressed as either the percent relative standard deviation (%RSD) or the relative percent difference (RPD).

%RSD

%RSD is used when there are at least three measurements and is calculated as follows:

where:

• s = standard deviation with n - 1 degrees of freedom (n = total number of observed values)
• = mean of observed values.

RPD

The RPD is used when two measurements exist. The RPD expresses the precision of duplicates and is calculated as follows:

where:

• x₁,x₂ = observed values
• = mean of observed values

Accuracy

Accuracy describes the correctness of a measurement. It reflects the degree to which the measured value approaches the true or expected value of the constituent of concern.

Sample and Standard Accuracy

If the measured value contains contribution from the sample, such as in the case of a MS or MSD, accuracy is expressed as the percent recovery (%R) and is calculated as follows:

where:

• SSR = measured value of spiked sample
• SR = measured value of sample
• SA = known value of spike added.
• If there is no sample contribution to the measured value, such as in the case of a LCS or BS, the measured value of the sample is zero (i.e., SR = 0), and accuracy (%R) is expressed as the measured value divided by the known value, multiplied by 100.

Yield Recovery

Yield percent recovery (%Y) of a tracer or carrier is typically used in radiochemical analysis and is a measure of the effectiveness of separation methods for some radionuclides. It is expressed as the percent recovery and is generally used to correct the analyte recovery in the sample for radiochemical analysis. Yield percent recovery is calculated as follows:

where:

• T_m = final measured value of the tracer or carrier
• T_k = added value of the tracer or carrier.

Measures of Agreement

The following approaches are used to determine the degree of agreement that exists between two measurements.

Percent Difference (%D)

The percent difference is used to compare one reference point to another in cases where they would be expected to behave similarly. The %D is used in organic and inorganic (e.g., in organic to compare average RRF from calibration to RRF from continuing calibration, in inorganic to compare serial dilution analyses). The %D is calculated as follows:

where:

• I = original result
• C = comparison result.
• Note: %D is typically reported in absolute terms.

Bias

Bias is a systematic error inherent in a method or caused by some artifact or idiosyncrasy of the measurement system. Bias can be assessed by comparing a measured value to an accepted reference value in a sample of known concentration or by determining the recovery of a known amount of contaminant spiked into a sample. Thus the bias, caused by matrix effects, as reflected by the MS is calculated as follows:

where:

• X_s = measured value of spiked sample
• X_u = sample or miscellaneous contribution
• K = known value of spike added.

If no sample or miscellaneous contributions exist, X_u would be zero.

Mean Difference

MD may be used to compare two radiochemical determinations which are reported with a total propagated uncertainty. The MD determines if the results are statistically different at the 95% confidence level. The MD is calculated using:

where:

• R₁ = first sample result
• R₂ = second sample result
• a₁ = two sigma total propagated uncertainty of first result
• a₂ = two sigma total propagated uncertainty of second result.

If the MD result is greater than or equal to 1.96, there is 95% confidence that the two results are not equal.

Detection Limits/Sensitivity

Method Detection Limit/Instrument Detection Limit for Inorganic and Organic Methods

The method detection limit (MDL) is "the minimum concentration that can be measured and reported with 99% confidence that the value is above zero" (SW-846 1987, consistent with the requirements specified in 40 CFR 136, Appendix B, 1990). The MDL is described briefly in the following text.

When the MDL is determined by spiking reagent water with each analyte of concern and not subjected to preliminary preparation, it is considered the equivalent of an IDL. The IDL is determined for most inorganic techniques (ICP, Graphite Furnace Atomic Absorption [GFAA], and Flame Atomic Absorption [FLAA]). The following considerations apply to the selection of the IDL standard:

• Concentration of the IDL standard should be at least equal to or in the same concentration range as the estimated IDL.
• Concentration of the IDL standard should fall in the region of the standard curve where there is significant change in sensitivity.

A minimum of seven aliquots of the IDL standard are required to calculate the IDL. Each must be processed through the entire analytical method (in cases such as organic analysis and mercury and cyanide determinations, the IDL standard should be subjected to preliminary extraction/digestion/distillation). Determination of the standard deviation(s) of the replicate measurements for each analyte shall be calculated based on n-1 degrees of freedom.

The IDL for each analyte is calculated as follows:

• IDL = t (n-1, alpha = .99) (s)
• where:
• t (n-1, alpha = .99) = the one-sided t-statistic appropriate for the number of observations (n) used to determine the standard deviation, at the 99% confidence level (e.g., n = 7, t = 3.14).

At a minimum, the IDL shall be determined annually

The term MDL is only used when the detection limit is determined in the sample matrix and subjected to sample preparation and analysis. The MDL is determined as above for IDL; however, only three replicates are required.

EQL for Inorganic and Organic Methods

The EQL has been defined by the USEPA as the lowest level that can be reliably achieved within specified limits of precision and accuracy during routine operating conditions. The EQL should be calculated at 5 to10 times the IDL; however, in limited applications, it is more appropriate to use the lowest standard in a calibration curve. This decision must take into account the data quality requirements from the end-user of the data. All sample processing steps of the analytical method shall be included in the final determination of the EQL.

Detection Limit Calculations for Radiochemistry Methods

Two concepts of detection are presented in this section, the Decision Level Count Rate (DLR) and the MDA. These are calculated according to equations in ANSI N42.2, Quality Assurance for Radioassay Laboratories, final draft dated February 9, 1994.

Decision Level Count Rate

The decision-level count rate (DLR) is defined as a 95% confidence limit for a critical decision level. This level is used for making a decision as to whether a sample emits radiation above the appropriate blank background level. The decision should be based solely on whether the net count rate observed for that sample exceeds this DLR. The DLR is calculated as shown below:

where:

• R_b = Background count rate
• T = Sample count time
• T_b = Background count time

Note: The DLR calculation referenced makes the assumption that the background count rate and the sample blank count rate are equal. Alternate DLR calculations can be used when they have been clearly defined and documented. The radio-analytical counting technique(s) where the alternate DLR calculations are applied shall also be identified.

When T_b is assumed to be equal to T the DLR can be simplified as shown below:

where:

• S_b = Standard deviation of background (or appropriate blank) count rate for the counting time (T).

For the purpose of interpreting whether an individual sample measurement is different from its appropriate blank, it is recommended to compare the net sample count rate with a DLR calculated using the sample specific "appropriate" blank. The "appropriate" blank should include measurement interferences from impurities (e.g., elevated Compton continuum, channel crosstalk from higher energy alpha particles measured by liquid scintillation) that are not typically known a priori or included in the nominal a priori DLR limit. This "true" decision level for the sample is different from the nominal a priori decision limit. For some measurement processes, the determination of the "true" appropriate blank for each sample may be impractical. However, every effort should be taken to properly assess the parameters of the appropriate blank.

Minimum Detectable Activity

The MDA is calculated based on Currie's (1968) formula and is simplified to the following two equations when the counting time in the sample is the same as in the background.

or

where:

• T = Sample count time
• K = Detector calibration factor (e.g., count rate/disintegration rate)
• S_b = Standard deviation of background count rate for the counting time (T).

When T_b is not equal to T, MDA is calculated as shown below.

where:

• R_b = Background count rate
• T_b = Background count time
• T = Sample count time
• ξ = Counting efficiency
• b = Abundance
• k = Conversion factor to convert to desired units.

The MDC is the MDA divided by additional experimental factors specific to the sample analyzed. The MDC is dependent on several experimental factors [e.g., sample aliquot size, yield (matrix effects), isotope half-life, length of time between sample collection and sample counting]. MDC is calculated using:

where:

• V = sample aliquot size (g or mL)
• Y = chemical yield
• D = decay factor (correction for radioactive decay to reference date).

Software provided by vendors may have variations of the above formula. A vendor-provided software or data reduction package is adequate for data calculation.

Uncertainty

Uncertainty is defined as the range of values within which the true value is estimated to reside. The uncertainty associated with a result shall be reported. Uncertainty is commonly used in the radiochemical analyses to express method and counting error. The estimate of possible uncertainty consists of systematic and random variability. Each contributing source of uncertainty is expected to be distributed over its range.

Each systematic component can be estimated in terms of the measurement result for the contributing source of uncertainty. Random uncertainties are derived from an analysis of replicate observations of a random process. The total random uncertainty is obtained by propagating the individual random uncertainty components. However, the typical methods used are not sufficient to separate systematic and random uncertainties such that biases can be corrected. Therefore, total uncertainty is calculated as the square root of the sum of the squares of all uncertainties, both systematic and random. Uncertainty will be measured or estimated if it cannot be measured. Total uncertainty is generally calculated using:

where:

• u_x = total uncertainties
• s_x = total random uncertainty
• = systematic uncertainty
• q = the number of systematic uncertainty components.

All components of uncertainty used in this calculation must be expressed based on the same units (e.g., concentration or percentage). Uncertainties shall only be rounded up prior to reporting, when rounding is appropriate.

Control Charts

QC performance trending provides the analyst with early warning of impending problems in a preparative or analytical method. BS/LCS performance for all routine preparations shall be monitored via control charts. The laboratory also monitors calibration verification standards (i.e., counter control standard) for radiochemistry. In those cases where the analytical technique involves a large number of analytes [e.g., inductively-coupled plasma (ICP), GC/MS], the laboratory selects a subset representative of the total for control charting. Control charts are the preferred method for trending QC; however, tabulation of results is acceptable as well.

Control charts may be prepared according to the following generic example or by equivalent methods selected to address specific variables:

• Obtain a group of repeated measurements from the analysis of a QC standard for which a control chart is to be prepared. A group of no less than seven and preferably 15 measurements group is needed to obtain initial estimates for control chart statistics. These measurements shall be performed on different days and at a minimum of half-day intervals.
• Calculate the average () for the group.
• Calculate the standard deviation(s) for the series of measurements
• Calculate the upper and lower warning and control limits as follows:
• Warning Limits = ± 2s
• Control Limits = ± 3s

Prepare a control chart with control lines corresponding to the average and the upper and lower warning and control limits. Warning and control limits shall not be updated on a continuous basis.

• Plot each measurement in the order obtained beginning with the original measurements used to calculate the control chart limits.
• Control chart limits shall be recalculated when major system changes occur.

For these control charts, there is a chance that 1 result out of 20 will exceed the warning limits and only 3 results out of 1000 will randomly exceed the control limits. If results exceed the warning limits more frequently than 1 in 20, then a systematic error exists (provided only the upper or lower limit has been crossed) or the random error has increased if both warning limits have been exceeded haphazardly. Investigation shall take place to determine the source of the increase in error. Corrective action shall take place to reduce the error. All corrective actions taken shall be documented.

In the case of analytical QC, corrective action may be recalibration and/or reanalysis. The standard that originally failed shall be run once more and perform acceptably for those analytes which originally failed in order for reanalysis to be considered valid. This allowance does not apply to CCV and CCB quality control since the failure affects the previous samples analyzed since the last valid CCV/CCB. A reanalysis can, however, be used to reinstate the rest of the run. This does not apply when recalibration is first performed. In the case of sample related QC, corrective action may include re-sampling and/or re-preparation and subsequent reanalysis.

If a QC result falls outside the statistically derived limits but inside the limits specified within this QA Plan, then the sample results may be reported; however, performance of the system shall be monitored.