Something that’s been receiving a lot of attention these days is the legitimacy and effectiveness of cannabis competitions, with some cups being caught in the line of fire more than others. Through the grapevine I’ve heard a lot of people lamenting all sides of this issue—from event organizers to judges to attendees, all the way down to the person who bought that 1/8th because it was an “award winning” strain.
What’s become overwhelmingly clear is that we as an industry can and need to do better. Specifically, we need to work on facilitating transparency, and creating a setting where as many confounding variables are controlled as possible. This not only will increase the efficacy and truth of what we judge, but also help bring more legitimacy to competitions as a whole.
Previously I’ve written about the lack of standards in the industry, and I hope this piece can be a constructive follow-up on how we can apply some of those standards where they’re arguably needed most: the competitions responsible for proclaiming who’s best of the best. Many of my thoughts and recommendations aren’t universal and might be implemented in one event but not another; please keep in mind I’m not referring to any one cup specifically, but rather using this as my open letter to all future competition organizers, judges, and attendees, for whichever respective event they choose to partake in.
Variables and Bias
In my intro I mention confounding variables. For anyone unfamiliar with the term, it’s when you have a subject being tested that includes some factor that might influence the consistency of outcomes between testing subjects. To control for these variables, a scientific approach must be taken to try to analyze the potential factors causing influence.
One such example of a variable in cannabis judging is the delivery method: if not every joint/vape/glass-piece/etc. follows a strict code of uniformity, this variable will ultimately affect the outcome of judging—even if inadvertently. Let’s take a pipe. If the pipe isn’t cleaned the same way between each use, it’s going to affect judgment. There’s no way a dirty pipe will ever taste the same as a clean one.
Biases are the second major issue at hand; that is, when the judges’ opinions might be influenced by external sources. One of the best examples of this is authority bias, where a known professional’s opinion inevitably outweighs and influences those around him. So let’s say out of a group of judges, one has a larger IG following and greater notoriety than the others: it’s safe to say some of the other judges may look to that individual for guidance on their own scoring metrics. This would lead to biased judging among those influenced that may misrepresent what the best quality cannabis coming out of the cup might be.
Suggested Guidelines
Now that I’ve briefly gone over what I believe to be the two major sources for judgment issues within competitions, I want to provide some suggestions as to what can be done better. I’m going to have a utopian mindset here, and probably nearly unrealistic standards at points. But this is just food for thought to hopefully inspire anyone who plans to run a cannabis competition to think about how they might be able to adopt and implement some of these guidelines.
The main reason for my disclaimer is due to the financial burden of some of the rules I’d like to see implemented. The purer the competition, the more resources will be needed to properly establish fairness. At a certain point those costs might prove prohibitive, ultimately leading to accessibility issues. Some might say, of course, let only the best who can afford it join the competition, but if a cannabis award is meant to help seek out the heat, not all those growing the heat (especially beginners or smaller farmers) have the resources to take part in something that might cost so much.
Guidelines
Place – My first guideline which I think will help with the issue of bias is to ensure that judges are never allowed in the same room together. The reason is pretty obvious: when judges are smoking next to each other, talking to each other, and making decisions together, the competition is de facto flawed from the start. People are easily swayed by majority opinion, especially when in-person in a group and influence becomes a chain reaction. My suggestion here would be to have judging done outside of the competition, with scores being submitted before the event even takes place on a completely anonymous basis.
Randomization – The second guideline, relatedly, would be randomization of judges. Ultimately to get a real, impartial score for the consumer, which is really what these competitions are about, it’s important that the people judging have no direct affiliation with or affinity for the product at hand. If judges are familiar with the product they’re judging, there will likely be biases, whether in the form of brand loyalties, business ties, or any other factor that could make someone want to vote for (or even against) a brand they’re familiar with. This is why it would be important that judges are drawn from a random sample in a completely randomized way—think something along the lines of a raffle. Although this may sound like I’m talking about the judge kits we see given out nowadays for certain competitions, there’s major flaws there. What’s to stop a deep-pocketed MSO from having people buy judge kits and vote for them?
Delivery – The third guideline would be delivery method. Every cup should have an agreed-upon way in which consumption takes place. There are, of course, those that would say they can judge best through their preferred method of intake, be it a joint, vape, or Backwoods. But one method might bring out something another won’t, ultimately affecting the score differently among judges. This means, for example, that if one strain for whatever reason might have a unique interaction with a tobacco leaf that alters its flavor, it may taste better or worse than when consumed in a joint. The same logic can be applied from joint to vape, etc.
The solution is a clearly delineated standard for consumption per event or award. If it’s a blunt competition, all the judges should only be smoking the same blunt wraps. If it’s a vape cup, everyone should be using the same vape—same with rolling papers and any other potential consumption method. I’d even go as far as to say in edible competitions the type of oil being used could very well be in separate categories, one for distillate-based edibles, one for ice water hash-based edibles, etc.
Time gating – The fourth guideline, which is one I feel extremely strongly about, is also one of the hardest to achieve from both a financial and organizational perspective. That would be consumption in a set timeframe. What we see a lot nowadays is judges being overburdened with entries to the point they must sit there smoking joint after joint, attempting to differentiate in a situation that makes differentiating nearly impossible. It’s not to say, of course, that judges might not find one strain more flavorful than the next, but the most important metric to be measured (at least in my opinion) is the effect, and there’s no way to properly discern what strain is causing what effect within such a compressed timeline.
The simple solution is breaking up consumption: to have a set time every day to smoke, with that smoke always being the first smoke of the day. There should also be a set amount of smoking each judge commits to not exceed throughout the day following the judging period, and that ideally should be time-gated as well to avoid a previous night’s debauchery affecting tomorrow’s palate and sensitivity. Hell, if we really want to go down the rabbit hole, judges should be on that same time-related consumption diet even a week prior to judging, to get their bodies used to a schedule that provides a consistency for them tolerance-wise—in turn optimizing their skills of discernment. We can go deeper and deeper, but properly ensuring any of this becomes ridiculously expensive. But alas, it’s important. Whichever organizer solves this one, my hat goes off to you!
Some Final Thoughts
I was sitting down to dinner with David Goldman and Michael Koehn a few weeks ago, two good friends who run the San Francisco Brownie Mary Democratic Club (the very same one Dennis Peron was active in during the advent of the 215 eras). We were discussing this very issue of competitions and their legitimacy, and I got a blast from the past as they schooled me on the cannabis competitions they used to run in SF back when doing such a thing would warrant some enforcement, even in the Bay.
I was surprised to hear, as they explained the minutiae in which they held these competitions, that even back then people were attempting to avoid a lot of the issues we’ve discussed here. These competitions were done out of living rooms, keep in mind, so they were extremely limited in the resources they had access to and what they could ultimately do. But even then, consistency mattered: the same person rolling all the joints, a scoring card with different metrics, an attempt at limited consumption (taking just a couple of hits as opposed to finishing each joint), complete anonymity, and no talking amongst judges.
I was impressed. If even back then they managed to provide such consistency—even with such limited resources—there really isn’t much excuse for us, in this new recreationally legal world we live in, to permit such laissez-faire judgment parameters within our competitions. Again, I want to reiterate that many cups do great jobs in some metrics, but certainly not all. This piece is an open letter to discuss as a community and figure out ways we can realistically ensure we are getting the end-consumer what they deserve, and ultimately help the awards willing to facilitate these measures to amass the most loyal of followings.
Edited by Lori Arden
