Passing the Test: Understanding Proficiency Testing

Understanding PTs

PTs are characterized materials created to represent the types of samples, matrices, and analyte targets being evaluated in laboratories. PTs are assigned reference values which are not disclosed to participants. PT samples are treated like blind samples; in which the nature and quantities are unknown. PT tests may provide basic information regarding the target identity, quantitative range and other information which directs the analyst in how to perform sample preparation or analyses. PT samples can come in many different forms from solid or liquid matrices to extracted oils, or solid dosage formulations like capsules and tablets. Analysts are expected to prepare and treat PTs in the same way similar types of samples would be routinely processed.

PT participants confidentially share their results with the PT provider for final evaluation and grading. In this way, PTs serve as an indicator for the competency of a laboratories staff and analytical performance. Results reported to the PT provider are compared to the established reference values for that PT. A reference laboratory obtains established reference values or averages the values reported by the PT participants (consensus value).

Results can be reported individually, meaning each analyst reports a result, or by calculating an average from all analysts performing the PTs. The individual result entry method is often preferred because it serves to document passing or not passing results at an individual level. This is most helpful for assessing and attributing competency of each lab analyst performing the PT. Participants who achieve passing PT results are ensuring the validity and reliability of their lab’s test results.

PTs should be an integral part of all testing labs’ quality system. When evaluating PT programs, laboratories must consider some critical points such as: the qualifications of the PT administrator, the quality and qualifications of the PT provider, and the accessibility of data. Within QMS systems based on ISO guidelines, laboratories are required to use certified providers for their analytical standards, methods, and PTs. ISO 17025 laboratories must use PT providers accredited to ISO 17043 and CRM providers to ISO 17034.

Methods used for PTs should have previously been validated by the laboratories or standards organizations which have issued those methodologies (i.e., AOAC, ASTM). In cases that standardized methods are not available; the laboratory should validate or verify their methods that will be used for the PT before the study begins. PTs are not a means of methods validation. A suitable QC-known, certified reference material or primary reference standard is employed for validating an analytical method. If a certified reference material is not available; a suitably validated and characterized material (usually by consensus or round-robin testing) canbe validated for use as standard in lieu of a CRM.

Acceptable PT results can serve as method verification when PTs are performed by the lab’s validated method. After the in-house methods have been validated, the methods can be verified by successfully passing a PT from an accredited third-party provider. In this example, PT is integral to the QMS of the participant lab.

PTs serves to measure the ongoing proficiency of independent laboratories through interlaboratory comparison of test results for the same sample. Some regulatory bodies stipulate the frequency of PT participation. Testing laboratories participate in accordance with their accreditation audit schedule. Some accreditation audits only occur once every other year. Thus, the labs would participate in PTs once every two years.

When determining the schedule and budget for PT participation, it is best to consider the accreditation goals and regulatory requirements for the laboratory when determining the schedule and budget for PT participation. It is best for each analyst who is reporting data to perform PT at least annually to monitor performance during a twelve-month period. Longer stretches without PT participation could negatively impact timely identification of non-conformance and delay implementation of effective corrective action. Laboratories that do not achieve acceptable results on their PT should examine analyst training and method procedures to check for deficiencies.

PTs calculations and results

Once participants analyze the PT, the participants report their results to the PT provider for review and evaluation. PT providers use customized software programs to evaluate participants’ results using programmed algorithms and statistics that meet the accrediting bodies guidelines. ISO guidelines (ISO 13528) have two basic methods for statistical evaluation of proficiency tests: the En-value and the z-score. An En-value is most commonly used in interlaboratory comparisons where the laboratories report their uncertainty calculations. (Equation 1).

Equation 1. En-Value Equation:Xi is lab reported value versus the Xref value reported by PT provider divided by the square root of the combination of uncertainties from the lab value and the reference value expressed using k=2 with a 95% coverage factor. Results with En-value between 1 and -1 are acceptable and agree with the reference values. Results with En-values greater than one or less than -1 are not acceptable.

The most practical of the statistical functions for chemical and biological analyses is the z-score. (Equation 2).

Equation 2. Z-score equation where µ is the mean subtracted by the Xi lab reported value; divided by the standard deviation, s.

The z-score can be used within interlaboratory comparisons without an uncertainty calculation; The z-score assumes all reported samples are from the same population, batch, or sample set and have the same uncertainty.

The results are calculated after the outliers are removed or are obtained from robust statistics. Z-scores less than two are considered acceptable. Scores between two and three are considered suspect while scores greater than three are unacceptable.

Once participant results are compared to reference values and assigned acceptable or unacceptable scores, those scores are provided to the laboratories. Results of a particular PT scheme may be deemed suspect if a clear bias or trend is apparent. Even if the results themselves are passing, the PT results can be considered unsatisfactory if there is a clear and measured bias. All failures or unsatisfactory results should be directed towards root cause analysis and corrective actions indicated by the lab’s QMS.

Corrective actions

In the event that analysts or laboratories fails PTs, there should be a procedure for review along with a plan of action to correct the problems found. First, a root cause analysis is conducted to identify and document the problems. If a root cause analysis finds deficiencies within a system, then corrective action is devised to remedy issues. A retesting of a corrected system must be conducted to determine if the correction was successful.

Points of review should include laboratory processes and all internal quality control data. Points to reexamine during a system and procedure review for a PT failure include:

• Preparation – Did the preparation process differ between routine samples and the PT samples?
• Instrumentation and equipment - Does the deficiency lie within the system, and can it be corrected with replacement, repair, or recalibration?
• Environment – Where are all materials and equipment maintained at the correct temperature?
• Examine all processes, standards, and calculations:Preparation – Did the preparation process differ between routine samples and the PT samples?
• Instrumentation and equipment - Does the deficiency lie within the system, and can it be corrected with replacement, repair, or recalibration?
• Environment – Where are all materials and equipment maintained at the correct temperature?
• Examine all processes, standards, and calculations:
  • Were all the calculations correct and documented including unit conversions? (Figure 1)
  • Are all the dilutions correct and within the correct dynamic range for the system? (Figure 2)
  • Were the standards, reagents, QC samples and controls within expiration? Were samples and standards prepared fresh or new standards opened?

Figure 1. Unit conversions

Figure 2. Dilution tips

Best practices for PTs Preparation

The proper handing and processing of PT samples can be critical in passing PTs. There are other additional areas which play key roles in passing these tests including instrument calibration, maintenance, and updating procedures and methodologies. Here is a list of helpful points to assist laboratories and analysts in passing their PTs. (Figure 2)

Figure 3. Best practices for PTs

PTs limitations

PTs are a necessary and helpful tool in the QMS arsenal of a laboratory. But they do have some limitations and misuses. In PT schemes where interlaboratory tests are used to establish values, there is often a retrospective aspect to the tests. There are often routine samples or analyses not reflected by the PT schemes either because the PT is offered in a different sample format or matrix than what the lab routinely tests.

For example, if a laboratory tests pesticides in cannabis extracts in ethanol but the PT is pesticides in plant material, the PT doesn’t match the preparation routine of that laboratory. In this case, the laboratory must document all the steps taken to prepare the PT sample and the ways in which it could differ from the preparation of actual samples. The best practice is to try and match the PT to the types of routine samples run by the laboratory. If a commercial PT program does not offer an obviousmatch in an established PT program; the laboratory can work with the PT provider to create the matrix-matched PT suitable for that laboratory’s needs.

An unacceptable PT only indicates a problem exists. It does not identify the root cause of the problem. Conversely, an acceptable PT result does not indicate competence in all areas of QMS. PTs are not substitutes for using quality controls and reference standards. They all work collectively within the QMS.

Conclusions - Tying it all together

PTs serve to measure the ongoing proficiency of independent laboratories through interlaboratory comparison of test results for the same characterized material. PTs participation is often stipulated by the regulatory authority overseeing laboratories operations or an accrediting body to which laboratories comply, or often both authorities concurrently. It is best to consider the laboratories accreditation goals and regulatory requirements when determining the schedule and budget for PTs participation. It is best to perform PTs at least annually to monitor lab performance during a twelve-month period. Longer stretches without PTs participation could negatively impact timely identification of non-conformance and delay implementation of effective corrective action. PTs are useful as tools to verify continuous improvement, resolve or monitor effective corrective actions, and demonstrate training and competency for lab analysts. PTs are a multi-purpose external quality assessment tool and should be an integral part of the lab’s QMS.

Further Reading

• ISO/IEC: 17043:2010 – Conformity assessment – general requirements for proficiency
  testing
• ISO/IEC: 17025- General requirements for the competence of testing and calibration
  laboratories
• USP Proficiency Testing Program:www.usp.org/proficiency-testing
• NSI Laboratory Solutions PT Program:www.nsilabsolutions.com/proficiency-testing/
• Spex Knowledge Base for white papers on dilutions, calibrations, and standards:www.spex.com/KnowledgeBase/AppNotesWhitepaper