The Ongoing Myth of Potency Accuracy in Cannabis Analysis

Accuracy measures how far away a measurement is from its true value. In the absence of a matrix reference materials with known cannabinoid concentrations, it is impossible to obtain potency accuracy in cannabis analysis. This leads to all sorts of problems, including inter-laboratory variation, and people always assuming their laboratory is right and everyone else’s is wrong with resultant arguments. Here, we suggest some solutions to the problem.

I’m a laid-back guy—really, I am. But I get very frustrated when I see bad science threatening the health of cannabis patients and the viability of our industry. Early on in this column series I spent a lot of time railing against the lack of scientific rigor in cannabis analysis (1). I then promised you, dear reader, to shut up and see if my columns made any difference. After spending a few years writing about the basics of the use of spectroscopy and chromatography in cannabis potency analysis, I am afraid to report that I have seen little progress on the issues I have brought up previously. It appears I have been screaming into the void. Since I am a glutton for punishment, I will continue to scream into the void with this column in the hopes that someone will hear. I wrote a previous column on cannabis potency calibration problems (1-3). Those columns are the starting point and expand on some of the topics from those columns.

Accuracy Versus Precision

I have written on this topic previously (4), but as I like to say, a review never hurt anybody. The concepts of accuracy and precision are shown visually in Figure 1.

Figure 1 makes clear that accuracy and precision are not the same thing. Imagine we are measuring the mass of a standard on a balance seven times. The bullseye on the target represents the known mass and the dots represent its measurements. Precision is a measure of the spread, scatter, variance, or dispersion in a set of measured values. Accuracy measures how far away you are from the true value.

The leftmost target in Figure 1 shows data that are imprecise and inaccurate. This is because there is a wide scatter in the data meaning the data are imprecise, and the data points are far away from the center of the bullseye meaning the data are inaccurate. The center target in Figure 1 shows data that is precise, but inaccurate. The data are precise because they are tightly clustered with respect to each other. However, the data are inaccurate because this cluster is far from the true value as represented by the bullseye. The data in the rightmost target in Figure 1 are what we want, data that are precise and accurate. These data are precise because the data points are tightly clustered together. These data are accurate because they are tightly clustered around the bullseye, the true value. Please note, and this is important, that to calculate accuracy we must have a known value of the quantity being measured.

Where Do We Find True Values for Cannabis Potency Analysis?

A commonly used technique used to determine potency values in cannabis samples is chromatography. Both gas and liquid chromatography have been used, however, for several reasons high performance liquid chromatography (HPLC) is used most often these days. As I have discussed in the past, my favorite HPLC potency method is that of Giese and colleagues (5).

For the sake of example, let's imagine we want to calibrate an HPLC for tetrahydrocannabinol (THC) analysis of cannabis plant material. Any number of companies, some of whom advertise in this esteemed publication, sell THC standard samples. These typically consist of a solvent with a known concentration of THC dissolved in it. They can be made, for example, by weighing out a known amount of pure THC and dissolving it in a known volume of solvent. The standard samples are then injected into the HPLC and the peak height or area (assume we are using peak area going forward) of THC’s corresponding peak is measured. Figure 2 shows an HPLC chromatogram for a sample containing cannabinoids. Peak #5 is for THC.

Then, using the principles of calibration science (to be covered in an upcoming column series) and a series of standards of differing known concentration, a mathematical model relating HPLC peak area to THC concentration is computed. To determine the concentration of THC in an unknown sample, it is injected into the HPLC, the THC peak area is determined, then the calibration model and the measured peak area are used to predict the concentration of THC in the unknown sample.

There are number of places where this process can go wrong. Firstly, we found out the hard way soon after the legalization of cannabis and the initial sale of cannabis HPLC standards that some cannabinoids are not stable in certain solvents! So, when buying cannabinoid HPLC standards please make sure you know what solvent your cannabinoids are dissolved in, double check with the manufacturer to make sure the standards are stable, and make sure to abide by all instructions for storing your standards and what their expiration dates might be. In many cases, your standards will need to be stored in a refrigerator and may only be good for a limited amount of time once opened.

Let’s assume that we have miraculously solved all the problems with cannabinoid HPLC standards and that we have achieved a perfect separation and calibration. Is this sufficient to give us a true THC potency accuracy measurement? Afraid not. The problem is that the standard we inject into the HPLC is in idealized form, pure THC in solvent. Unfortunately, that is not the state of the actual sample being analyzed. In reality, our samples can be plant material, extracts, distillates, hashish, or final products such as tinctures or edibles. These samples are nothing like the final solution of pure cannabinoids that are injected into an HPLC. Also, all these samples require significant amounts of sample preparation, including extraction, mixing, filtering, dilution, and injection into a chromatograph. As I have discussed previously (2), based on my observations at dozens of cannabis laboratories, one of the biggest sources of variation between laboratories is the extraction process. A goal of any chromatographic extraction process should be to get all the analyte into the solution that will be injected into the chromatograph; if any analyte is left behind, we will not measure the true value of the concentration of the analyte in the sample actually being analyzed, which means this measurement will be inaccurate. An extraction method that consistently removes a given percentage of the analyte, let’s say 80%, will be precise because the results may cluster tightly around a given value, but this data will still be inaccurate because we will be 20% off from the true value. Most, if not all, published cannabis potency methods say nothing about whether they tested to see whether they got all of the analyte out or not. In other words, we have no way of validating chromatographic extraction methods in the cannabis industry!

To solve this problem we need cannabis flower, extract, distillate, and final products with known amounts of our analytes of interest such as cannabinoids and terpenes in these matrices. Only then will we have a known value of our analyte in our sample matrix and have a way of determining whether our extraction technique is correct or not. The National Institutes of Standards and Technology (NIST) call these types of standards matrix reference materials (6).

Where Can One Find Cannabis Matrix Reference Materials?

In the US, true values for analytical work are typically provided by NIST, and they are perfectly capable of producing good matrix reference materials. For example, for the analysis of green tea there exists a matrix reference material of known composition that can be used to check an instrument’s calibration (7). That is, green tea plant material samples exist with known amounts of the analytes of interest for the green tea industry. These samples can be used to validate extraction methods to make sure they work correctly.

I chose green tea as an example because it is a good mimic for cannabis plant material; if NIST can put together good green tea matrix reference materials it should be able to issue good cannabis matrix reference materials such as for cannabis plant material. Except that they can’t. Because of the perversion in the way cannabis is regulated in the US, anything with more than 0.3% total THC is illegal at the federal level. It would be illegal for NIST, being a federal agency, to ship matrix reference materials with greater than 0.3% total THC in it across state lines. Hence, the lack of NIST issued cannabis matrix reference materials. This means there are no true values available for cannabis potency analysis. This is why I say that accuracy in cannabis potency measurements is a myth. It will take rationalization of cannabis regulation, and legalization at the federal level, for this problem to be solved.

You Cannot Compare Potency Measurements Between Different Instruments or Methods!

Again, accuracy is a measure of how far off you are from the true value, and in cannabis potency analysis those true values must be supplied by appropriate matrix reference materials. Currently, there are no appropriate cannabis reference materials available, for example, there are no known true values. And again, this is because we have no way of validating sample extraction methods. How then can we calculate accuracies in cannabis potency analyses? We can’t! The proof that varying methods cause variations in potency analyses of the same sample has been well documented (8).

I realize that any number of research groups and professional organizations are working on or have published cannabis potency methods, but no two of these methods are the same, and lacking appropriate matrix reference materials we have no way of knowing whether some, none, or all of these methods are accurate. I strongly encourage these groups and organizations to stop work on all their different methods, wait until there are appropriate cannabis matrix reference materials available, and then work together to promulgate a single cannabis potency method that our industry can use.

At the end of the day, the lack of appropriate cannabis matrix reference materials means that we have no way of measuring potency accuracy in the cannabis industry. This means that potency measurements made with different methods or with different instruments cannot be compared to each other. Think about that for a little bit. Measurements made by different third-party laboratories cannot be compared to each other, and even measurements made by the same laboratory using the same method but on different instruments cannot be compared to each other. This can lead to any number of problems.

As many of you know, I have done extensive research on the application of mid-infrared spectroscopy to cannabis analysis (8-14). The lack of appropriate cannabis matrix reference materials has given me no end of trouble in my attempt to gain converts to this method. Here are two stories:

The Perfect Tincture

I am in the process of trying to develop a spectroscopic tincture analysis method for a potential customer. We of course need standard samples with known concentrations of the analytes to be able to obtain a calibration. The accuracy of the concentration determination of the standards is probably the biggest contributor to the accuracy of predicted concentrations. So, we measured the spectra of the tincture standards, and I was provided with the “true” analyte values by the potential customer. When I asked about the accuracy of the true values I was told, “The method used to analyze these samples was certified by government agency XYZ.” I said, “That’s nice, what potency analysis method was used?” Of course, the response was chromatography. Then I asked, “What was the accuracy of the chromatographic method?” The response, “I don’t know, the method was certified by government agency XYZ.” A perfect example of circular logic. The customer then went on to criticize the accuracy of my spectroscopic method with no knowledge of whether the reference data were any good or not.

To bless any analysis by saying it was done by spectroscopy or chromatography is meaningless. We are analytical chemists, and we need numbers to provide meaning to our analyses. It is never scientifically correct to accept the results of a method on blind faith. Analytical data lacking accuracy determinations is meaningless. A set of tincture matrix reference materials would solve this problem.

Your Results Don’t Match My Laboratory!

It is a rule of thumb that a properly calibrated spectrometer will be 1.5x to 2x less accurate than the reference method used to calibrate it, which in cannabis analysis is typically chromatography. I have written extensively about the problem of inter-laboratory variation in the cannabis industry (8). A large part of this problem is the lack of proper matrix reference materials for cannabis potency analyses as discussed above, not to mention the lack of a standardized cannabis potency method. Now, science knows how to compensate for variations across methods, it’s called a crossover study. We simply run a set of standards using method 1, the same standards using method 2, and then compare the results. The problem with this approach in the cannabis potency business is that because of the lack of matrix reference materials we lack the proper standards for a crossover study! This means we cannot compare cannabis potency data across instruments or laboratories until the proper matrix reference materials appear. Why this makes me pull my hair out, is that in my work all mid-infrared spectrometers are calibrated using reference HPLC values from a state licensed, International Organization for Standardization (ISO) certified laboratory (15).

All too often I will be contacted by someone saying that their pet laboratory’s numbers using HPLC do not match those obtained by mid-infrared spectroscopy. The problem here isn’t the mid-infrared method, it is that the two chromatographic methods in question are not the same, and we have no way of knowing which one is correct because of the lack of appropriate cannabis matrix reference materials.

The solution to this problem is for the customer’s spectrometer to be calibrated using values from their preferred laboratory. Of course, customers bristle at having to do this and end up thinking there is something wrong with spectroscopy potency methods rather than understanding that the problem is lack of appropriate matrix reference materials.

What we can aspire to have here is precision and a well implemented cannabis potency method should be able to achieve this goal. Given the problems discussed in this column, we may not get a true value, but we can at least aspire to reproducibility giving us precise but inaccurate data. This will at least allow cannabis businesses to use potency analyses to run their operations. For example, if we tweak a distillation and the measured potency goes up, that increase is probably real, and the adjustment is worth keeping. Where the lack of accuracy is dangerous is in the final products we sell to cannabis patients. It is essential that we know what the exact amount of THC, cannabidiol (CBD), and other analytes are in products put into or onto a human being so we can ensure cannabis patients are getting the right dose of medicine.

If Cannabis Matrix Reference Materials Were Ever to Magically Appear

In a perfect future alternative universe where cannabis is regulated rationally and appropriate cannabis matrix reference materials are available, what would we do then? I believe a mass inter-laboratory variation study is needed. All 3rd party laboratories that want to be legally certified to provide cannabis potency measurements to the public need to be required to be involved in this study. Once these laboratories run samples with known true values and then compare their own results, they can ascertain whether their methods are good or not. Only once we have this vital information, can we take cannabis potency measurements out of the realm of mythology and into science.

Conclusions

The lack of appropriate cannabis potency matrix reference materials means that accuracy measurements in these analyses cannot be computed. This means we cannot compare potency measurements across instruments, methods, or laboratories. This lack of accuracy makes adaptation of new technologies for potency measurements difficult, and the resultant error in cannabis dosage measurements threatens patient health. If, and when, appropriate cannabis matrix reference materials appear, a massive round-robin study involving all laboratories that wish to provide certified potency measurements to the public needs to be performed.

References

1. Smith, B.C., Dispelling Cannabis Analysis Calibration Myths, Cannabis Science and Technology, 2020, 3(1), 10-14.
2. Smith, B.C., How Standardized Sample Preparation Can Solve the Inter-Laboratory Variation Problem, Cannabis Science and Technology, 2020, 3(2), 10-15.
3. Smith,B.C., Dispelling Cannabis Analysis Myths, Part I, Cannabis Science and Technology, 2021, 4(5), 10-12.
4. Smith, B.C., Error, Accuracy, and Precision, Cannabis Science and Technology, 2018, 1(4), 12-16.
5. Giese, M.W., Lewis, M.A., Giese, L., and Smith, K.M., J. AOAC Int., 2015, 98(6), 1503.
6. NIST Tools for Cannabis Laboratory Quality Assurance, https://www.nist.gov/programs-projects/nist-tools-cannabis-laboratory-quality-assurance.
7. Green Tea Standard Reference Material (SRM), https://www.nist.gov/image/greenteastandardreferencematerialsrmjpg.
8. Smith, B.C., Lessard, P., and Pearson, R., Inter-Laboratory Variation in Cannabis Analysis: Pesticides and Potency in Distillates, Cannabis Science and Technology, 2019, 2(1), 14-19.
9. Smith, B.C., Lewis M., and Mendez, J., Optimization of Cannabis Grows Using Fourier Transform Mid-Infrared Spectroscopy, PerkinElmer Application Note, 2016.
10. Smith, B.C., Quantitation of Cannabinoids in Dried Ground Hemp by Mid-Infrared Spectroscopy, Cannabis Science and Technology, 2019, 2(6), 10-14.
11. Smith, B.C., A Proposed Representative Sampling Plan for Hemp Grows, Cannabis Science and Technology, 2020, 3(6), 10-13.
12. Smith, B.C., Terpenes and Testing, Jan.-Feb. 2018., pg. 32.
13. Smith, B.C., Terpenes and Testing, Nov.-Dec. 2017, pg. 48.
14. Smith, B.C., Fucetola, C.A., Ehrmantraut, K., and Hagan, T., Terpenes & Testing, Sept./Oct. 2020, pg. 19-24.
15. https://www.sclabs.com/.

About the Columnist

Brian C. Smith, PhD, is Founder, CEO, and Chief Technical Officer of Big Sur Scientific. He is the inventor of the BSS series of patented mid-infrared based cannabis analyzers. Dr. Smith has done pioneering research and published numerous peer-reviewed papers on the application of mid-infrared spectroscopy to cannabis analysis, and sits on the editorial board of Cannabis Science and Technology. He has worked as a laboratory director for a cannabis extractor, as an analytical chemist for Waters Associates and PerkinElmer, and as an analytical instrument salesperson. He has more than 30 years of experience in chemical analysis and has written three books on the subject. Dr. Smith earned his PhD on physical chemistry from Dartmouth College.
Direct correspondence to: brian@bigsurscientific.com.

How to Cite This Article

Smith, B., The Ongoing Myth of Potency Accuracy in Cannabis Analysis, Cannabis Science and Technology, 2023, 6(8), 6-12.