Inter-Lab Variation in the Cannabis Industry, Part I: Problem and Causes

Too often, cannabis laboratories get different results on the same sample. There are a number of causes and solutions. In this installment, the problem is presented along with its causes. In the second part of this series, potential solutions will be discussed.

In the first installment of this column series, I stated one of my goals was to teach the basics of chemical analysis for those in the cannabis industry that are new to the field (1). I started by writing columns on precision and accuracy (2) and representative sampling (3). Normally at this point in a course on analytical chemistry I would proceed onto a topic such as the basics of chromatography. However, one of my other goals for this column is to address important issues facing the cannabis analysis industry. There now exists an issue so important in this industry that I feel the need to interrupt your regularly scheduled cannabis analysis column for this important message: inter-laboratory variation is a serious problem in our industry. I feel it is so important that I will be devoting this and the next column to this topic.

Cannabis is Medicine…Test it Like Medicine!

The U.S. Food and Drug Administration (FDA) is now on record stating that cannabidiol (CBD) has medicinal uses (4). Here is an excerpt from their announcement: “Marketing unapproved products, with uncertain dosages and formulations can keep patients from accessing appropriate, recognized therapies to treat serious and even fatal diseases.” Thus, the FDA has set itself the goal of insuring cannabis medicines contain correct dosages.

One of my mantras is, “Cannabis is medicine . . . test it like medicine!” (This phrase was in the running to be the name of this column series, but I thought “Cannabis Analysis” was easier to say and remember.) The point is that the cannabis industry needs to adopt the testing regimen of the pharmaceutical industry to produce safe and effective medicines, and the FDA agrees with me. This means performing what I call cradle-to-grave testing: Analyzing raw materials for identity and purity, monitoring every step of the chemical manufacturing process to make sure it is done according to standard operating procedures; testing final products for correct amounts of active pharmaceutical ingredients (APIs) and excipients; and testing for compliance with any applicable laws and regulations. Many cannabis businesses are dependent upon third party cannabis analysis laboratories for some or all their testing.

Inter-laboratory variation is the problem of different laboratories obtaining statistically different results on the same sample. It is a well documented phenomenon in the cannabis industry (5–8). One of the best papers on this topic is surprisingly found in the pages of the prestigious Journal of the American Medical Association (JAMA) (5). The authors purchased 84 CBD containing products over the internet from 31 different companies, and had their potencies analyzed by an independent laboratory. More than 69% of these products were mislabeled. One of the likely explanations for this problem is inter-laboratory variation-the laboratories who analyzed these products for label claims incorrectly measured the amount of CBD in the samples. Based on this study, it sounds like the FDA has its work cut out for it trying to insure cannabis patients get the right dosage of CBD.

In addition to the JAMA article, local news teams have been on the hunt investigating inter-laboratory variation in the cannabis industry (6,7). In one study (7), the same homogenized cannabis oil sample was delivered by hand to 10 different cannabis testing laboratories in California. The range of measured potencies was from 64% to 83%. There is no other explanation for these results but inter-laboratory variation. An internet search will turn up other such studies with similar results.

Some colleagues and I recently performed a round-robin study of five California cannabis laboratories (8). We took a sample of distillate known to be free of pesticides, spiked it with known amounts of six different pesticides, and hand delivered the samples in duplicate to the laboratories on the same day. The false negative rate was 78%. That is, 78% of the time the laboratories failed to detect pesticides known to be present in the distillate. Our results probably mean pesticide tainted products have made it into the hands of California cannabis patients and consumers.

Additionally, we submitted homogenized distillate samples from the same batch in quadruplicate to the five different laboratories (8). The range of the 20 potency measurements received was 77% to 94%. Again, the cause here must be inter-laboratory variation. We proved via mid-infrared (IR) spectroscopy that the sample was homogeneous (8). I am sure many of you have heard of or experienced the inter-laboratory variation problem yourselves.

Let’s review the results here:

• 69% of CBD products mislabeled with the wrong dosage.

• Measured potency range of 64% to 83% measured on the same sample by 10 laboratories.

• Pesticide false negative rate of 78%.

• Measured potency range of 77% to 94% measured on same sample by five laboratories.

These results are shocking and unacceptable. Can you imagine the uproar if 69% of a company’s pills sold to the public were found to have the wrong amount of API? Can you imagine the controversy that would result if we found out that 78% of the food we eat contained previously undetected pesticides? There would be reams of bad press, a raft of lawsuits, congressional investigations, companies going bankrupt, and maybe even people going to jail. This will happen to our industry if we don’t fix the inter-laboratory variation problem.

Press reports of this problem shake the public’s confidence in our industry and make it look like we don’t know what we are doing, that we might be selling tainted or dangerous products, or, worst case scenario, that we are frauds and crooks. I am sure it is a long cherished goal of many in this industry to see medical or recreational cannabis legalized at the federal level in the United States. To achieve this, we must convince the public that we sell a safe, effective product, and that our business practices are above board and beyond repute. The inter-laboratory variation problem and its associated bad publicity makes achieving our goal that much harder, to the detriment of all in the cannabis industry.

The inter-laboratory variation problem makes it hard for cannabis businesses, our clients, to make rational business decisions. For example, I have seen a distillate batch worth hundreds of thousands of dollars be declared pesticide free by a cannabis laboratory hired by the seller but tested above legal limits by another laboratory hired by the potential buyer. The buyer nixed the deal, causing the seller significant financial heartache. What is a cannabis business to do in this situation? Who are we to believe? How can a cannabis business stay in business, when whether or not they can legally sell their product is a crapshoot?

Causes of the Problem

As with any complex problem, there are multiple causes to the cannabis industry inter-laboratory variation problem. Here are my observations.

Lack of Standard Methods

In the U.S., the analysis of pharmaceuticals is regulated by the FDA. This industry can also turn to the United States Pharmacopeia (USP) for standard methods and reference materials. In the food industry, the Association of Official Analytical Chemists (AOAC) promulgates standard methods for things like testing the fat and protein content in what we eat. Because cannabis is still illegal at the federal level in the U.S., this precludes our FDA from getting involved in regulating the cannabis industry. I have heard that there are people at the USP working on cannabis analysis standard methods, and I know there are hard working volunteers at the AOAC that have formed committees and working groups, and have issued calls for cannabis analysis methods. However, all of this takes time and, at the moment (in my observation), there has yet to be promulgated clear guidance on methods that all cannabis analysis laboratories can use. I believe the day of standard cannabis analysis methods will come. But in the meantime, the problems caused by inter-laboratory variation will continue. I hope my pointing out the problem in this column will cause more people to become involved in obtaining a solution.

Sample Inhomogeneity

I addressed this issue in detail in my last column (3). The problem is that cannabis samples, particularly plant material, can be so inhomogeneous that different aliquots of the same material batch can have different chemical compositions. When these samples, assumed to be the same, are tested by different laboratories, and they get different results, we are all left scratching our heads.

Lack of Appropriate Standards

Another industry that deals with the analysis of inhomogeneous plant material is the tea industry. In the U.S., the National Institute of Standards and Technology (NIST), a part of the federal government, issues standard reference materials (SRM) for industry. In the case of green tea, they have issued a standard reference material (9) that can be used for method development and laboratory round-robin studies to measure inter-laboratory variation. In the U.S., cannabis standard reference materials issued by NIST do not yet exist because of the federal illegality problem. To the best of my knowledge, there does not yet exist a standard cannabis bud or standard reference cannabis oil, extract, or distillate. We can’t develop standard methods across laboratories and track inter-laboratory variation if we can’t guarantee that every laboratory is analyzing the same sample.

Sample Preparation Variability

I have been involved in cannabis analysis since soon after recreational cannabis became legal in Colorado, and have visited dozens of cannabis laboratories. In my observation, the biggest variable from laboratory to laboratory is sample preparation. Variances in amounts, grinding, solvents, shaking, vortexing, centrifuging and so forth, can lead laboratories to extract analytes from samples with different efficiencies causing part of the inter-laboratory variation problem. The most common ways of determining potency and pesticides in cannabis samples are gas chromatography (GC), high performance liquid chromatography (HPLC), mass spectrometry (MS), and combinations thereof (10–12). To calibrate these standard solutions with known amounts of cannabinoids, terpenes, and pesticides that are sold and analyzed, instruments that are capable of amazing feats of accuracy and precision are used. However, our clients do not submit solutions of pure compounds to us for analysis. They submit inhomogeneous plant material, gooey brown stuff, or complex matrices like brownies and cakes. To prepare samples for these analyses they must be prepared in such a way that the analytes end up in the right concentration range in solution. You may have an instrument whose calibration line constructed using pure standards has a correlation coefficient of 0.999 and accuracy to the second decimal place. But if the sample is not prepared properly or does not have reproducibly, that accuracy is meaningless.

Sample Instability

It has been shown that cannabis samples of various types lose potency over time (13–17). One study using mid-IR spectroscopy found that cannabis oils lose total tetrahydrocannabinol (THC) via a first order mechanism with a half-life of 7.9 months. Because of this mechanism, potency loss is the greatest early on (17). Thus, when doing a round robin study, if laboratories get the same sample on different dates, they may be analyzing samples that are measurably different from each other. This means that for cannabis oils, and perhaps for other cannabis samples, potency values can measurably change in as little as a week. As a result, laboratories analyzing the same sample at different times may get different results.

Large Factor Dilutions

Modern chromatographs and mass spectrometers are incredibly sensitive, capable of measuring pesticides in the parts per billion level. However, when we need to measure potencies some samples contain THC amounts greater than 90%. Therefore, concentrated samples must be diluted by significant amounts to keep from overwhelming our sensitive instruments. Dilutions by factors of 100 times or more are not uncommon. When making these large factor dilutions, a small amount of analyte containing solution is dissolved in large amounts of solvent. It is not always easy to measure out small amounts of liquid reproducibly. If you literally have to dissolve one drop of analyte solution in 100 mL of solvent, getting that drop size exactly right each time is nontrivial, and small error is magnified when diluting by a large amount. I believe part of the explanation for the distillate potency variation discussed above (8) may be problems with large level dilutions.

Treating Analytical Methods as Intellectual Property

In my own attempts to establish standard methods in our industry, I ask laboratory proprietors to share their methods with other laboratories. Too often the answer is “no way.” These laboratories are under the mistaken belief that their in-house developed method has some important difference than all other laboratories, and to disclose that difference would put them at a competitive disadvantage. I can assure all laboratory owners that I have observed enough methods being run at enough laboratories to tell you that no one has a “magic trick” that will set them head and shoulders above the rest. Almost all laboratories are filled with smart people and state-of-the art equipment. We are all trying to solve the same problem and, chances are, we all will arrive at essentially the same solution. There is no need to be secretive. Worse yet, by not being forthright with methods, it makes it impossible to develop and implement standard methods across the industry, meaning the inter-laboratory variation problem will never be solved.

Human Error

Last but not least is good old-fashioned human error. Despite training, adherence to standard operating procedures, and our best intentions, we are all human and we make mistakes. For example, I once had a laboratory report back to me that a sample contained 234% tetrahydrocannabinolic acid (THCA). That mistake should have been recognized by the analyst, or their supervisor, and never allowed to leave the laboratory. Undoubtedly, similar screw ups contribute to the inter-laboratory variation problem.

Conclusions

The inter-laboratory variation problem in the cannabis analysis industry is real and threatens the health and safety of consumers and the future of our industry. It has a number of causes, including a lack of standard methods and reference materials. These problems are exacerbated by the lack of marijuana legality at the federal level in the United States. Potential solutions to this problem will be offered in the next column.

References:

1. B.C. Smith, Cannabis Science and Technology 1(3), 10–12 (2018).
2. B.C. Smith, Cannabis Science and Technology 1(4), 12–16 (2018).
3. B.C. Smith, Cannabis Science and Technology 2(1), 14–19 (2019).
4. https://www.fda.gov/newsevents/newsroom/pressannouncements/ucm611046.htm.
5. M.O. Bonn-Miller, M.J.E. Loflin, B.F. Thomas, J.P. Marcu, T. Hyke, and V. Ryan, Journal of the American Medical Association 318, 1708 (2017).
6. B. Young, The Seattle Times, January 5, 2016. https://www.seattletimes.com/seattle-news/marijuana/some-pot-labs-in-state-failed-no-pot-at-all-says-scientist/.
7. https://www.nbcbayarea.com/investigations/Industry-Insiders-Warn-of-Fraud-at-Marijuana-Testing-Labs-458125743.html?_osource=SocialFlowFB_BAYBrand.
8. B.C. Smith, P. Lessard, and R. Pearson, Cannabis Science and Technology 2(1), 48–53 (2019).
9. https://www.nist.gov/publications/development-and-certification-green-tea-containing-standard-reference-materials.
10. A. Hazekamp, A. Peltenburg, R. Verpoorte, and C. Giroud, J. Liquid Chrom. & Related Techniques 28, 2361 (2005).
11. B. De Backer, B. Debrus, P. Lebrun, L. Theunis, N. Dubois, L. Decock, A Verstraete, P. Hubert, and C. Charlier, J. Chromatrography B 877, 4115 (2009).
12. M. Giese, M. Lewis, L. Giese, and K. Smith, J. AOAC Intl. 98, 1503 (2015).
13. M. Starks, Marijuana Chemistry (Ronin Publishing, Oakland, California, 1977).
14. J. Fairbairn, J. Liebmann, and M. Rowan, Journal of Pharmacy and Pharmcacology 28, 1 (1976).
15. I. Trofin, G. Dabija, D. Vaireanu, and L. Filipescu, Revista de Chimie (Bucharest) 63, 293 (2012).
16. C. Lindholst, Australian Journal of Forensic Sciences 42, 181 (2010).
17. B.C. Smith, Terpenes & Testing Magazine, Nov./Dec.(6), 48–51 (2017).

Brian C. Smith, PhD, is Founder, CEO, and Chief Technical Officer of Big Sur Scientific in Capitola, California. Dr. Smith has more than 40 years of experience as an industrial analytical chemist having worked for such companies as Xerox, IBM, Waters Associates, and Princeton Instruments. For 20 years he ran Spectros Associates, an analytical chemistry training and consulting firm where he taught thousands of people around the world how to improve their chemical analyses. Dr. Smith has written three books on infrared spectroscopy, and earned his PhD in physical chemistry from Dartmouth College.

How to Cite This Article

B.C. Smith, Cannabis Science and Technology 2(2), 12-17 (2019).