Calibrating a Global Cupping Team: Labtime with Joel

At Red Fox, our coffee program relies from start to finish on a team of globally-distributed cuppers working simultaneously on shared standards. From the initial purchasing process, to the quality control work that goes into getting a coffee from port of origin to port of destination in its original condition, to the ongoing work of effectively communicating current quality over the life and sales cycle of a coffee within the team and outward, calibration of every member at all times is essential to us, our vendors, and our customers making good decisions. In other words, making great coffee happen is only possible when our whole team knows exactly what we’re looking for and how to communicate about it. 

So how do we do that? I talked with global lab manager and QC veteran Joel Edwards on calibration and how to train and calibrate a global cupping team.

What is calibration?

According to Joel, calibration generally means “basically just that all cuppers (coffee tasters) are on the same page.” More specifically for our program, it means “that the qualities we are looking for are clearly defined for all cuppers.” So, not just that we all know what we’re looking for, but that our assessments have specific quantitative and qualitative meanings that are documented in an official way that we can all access. 

A distributed team

Our team is globally distributed in that we don’t only have just one location where we all work together, and that team members move frequently from place to place according to our sourcing needs. We’re a small staff, but since we have headquarters in Berkeley, Lima, and Oaxaca as well as a remote team throughout the US, tasters could be in as many as 7-15 different locations at once and still need to communicate along a shared standard. 

“In a sense, everyone on our team cups,” Joel says. “The core sourcing and quality control team is about 7 that we rely on across various countries, but everyone cups and needs to be calibrated.” The reason, he says, is that even if your main role is sales or marketing, you need to be able to communicate a coffee’s quality to customers, “that if a customer asks for a sweet/round coffee for blending or a super bright, floral single origin offering, the sales reps can identify not just those broad categories but nuances among them to find the perfect coffee.” 

Signal detection cupping

Bringing on and training new members into the core cupping team is a challenge Joel and others have been working for years to systematize. One of the key ways we quantify our quality standards and cup high volumes consistently is through a cupping method called signal detection, first introduced to us by coffee quality control specialist Paul Songer. 

In signal detection:

• Cuppers taste 3 cups of each coffee in double-blind randomized placement along a table.
• Cuppers score coffees on a scale of 1-6 with 1 representing “defective” and 6 representing “outstanding.”
• Coffees we buy score a 4 or higher.
• Signal scores allow us to more effectively construct lots and separate coffee. 

With signal detection, we reduce bias by double-blinding the coffee and subjectivity by focusing on numeric scores over notes. We aren’t assigning an SCA score to coffees via signal detection, we’re simply making purchasing decisions. That makes this method not only a great cupping system for purchasing, but also a great tool on which to train new cuppers and train past opinion and bias as much as possible. 

All of our new hires have cupping experience, but signal is usually new to them. “It’s the main way we purchase coffee, so training new team members on signal detection is critical,” says Joel, who prefers the clarity of the 1-6 scoring system to debating whether a coffee is an 86 or an 86.25. “We’re using that 6 point system and really drilling down on “this is a 3, it’s out, this is a 4, it’s regional quality, this is a 5, it’s a coffee we’ll separate as a producer lot.” 

It’s most important to know exactly what we’re looking for when we designate a coffee a 4 (regional quality, clean/sweet/bright) and a 5 (separate out, in addition to clean/sweet/bright it has distinct flavors and added complexity). Below that, we aren’t purchasing the coffee, and above that, we’re just determining a higher premium. “The key is the score differences that lead to different actions on the buying side—to buy or not buy, to separate or not separate. With those we also determine the correct premiums to pay partners based on those preestablished systems.”

Freezing samples

As part of the calibration process, the Berkeley lab regularly chooses samples to vacuum seal and freeze for later examples of various attributes. 

Good educational samples to select:

• Clear examples of coffees that represent signal scores 3, 4, and 5.
• Defects of all kinds. 
• In other words: clear examples of both what we want to buy and what we want to avoid. 

In bringing educational samples with defects, “one thing we look for is any hint of age or fade,” a flavor that often presents like paper or cardboard. This is a major thing to avoid in our buying, especially because if a coffee shows this at the offer stage, it’s bound to get worse down the line. “When I go to recalibrate down to the different offices, I’ll bring a coffee that has a hint of fade, one with medium fade, and one with the highest level of fade,” he says. “It’s a great way to really demonstrate that flavor and how to start looking for it in even minute quantities.”   

Harder to guarantee in a sample but just as important to learn is phenol, a defect that presents with a chlorine-like taste. Unlike fade, phenol varies from bean to bean and can present in just one cup of a sample. 

Another important and challenging defect they train on is ferment defect, which can present with an overripe/rotten fruit taste or a vinegar-like acidity. “It’s potentially even more challenging to train on because it’s subjective,” says Joel. “The industry’s palate has changed so much over the years that coffees that a decade ago would have been considered defective are now desirable for many.” Perception of fermentation flavors can vary not just based on calibration but based on individuals’ actual perception, plus some people do like the intensity of flavor in heavily fermented coffees and seek it out. But, it’s still key to set clear standards and train on them, so we bring samples from rejected lots that are over the line for us and taste them together.

“We all have different thresholds, even for defects,” says Joel. “For example, for phenol, I have a fairly high threshold whereas team member Rachel has a very low threshold and can taste phenol at very low levels. On the other hand, I’m extremely sensitive to fade. We all just taste differently—no one can assume that just because you’re tasting certain things in a coffee, everyone else is.” And that’s exactly why calibration within a balanced team is so important. 

Regular recalibration

In addition to initial training, Joel leads a 1-3 day recalibration before Peru and Mexico seasons to bring the team together and taste coffee. “Some of the cuppers will have a few months not cupping before the season starts, so it’s important to get that muscle back in shape. Along with bringing defects down, I’ll bring specific examples of coffees in each quality tier. We’lll use the samples I bring as well as any early offers or harvest samples that are at the lab.”

But at this point, our team is very experienced. “Fabian and Carina are fantastic cuppers, and they’re traveling between Peru and Mexico so they stay in calibration and can help guide everyone else so that they get and stay on the same page,” he says. 

One rule he follows is to always have a lead cupper, “someone who’s been on the team longer and can lead the team through discussions and help settle scores when they don’t agree. Sometimes a signal detection comes out to a 3.5 when everyone’s scores are averaged together, and they can help break the tie and communicate their reasoning.”

“The biggest challenge is keeping everyone on the same page: as the team grows, so does the number of individual preferences” says Joel, making it harder to home in on our specific quality standards. The antidote is spending more time with each team. 

Bringing in sourcing partners

“In the past we’ve brought in key partners like the lead cuppers from cooperatives we work with in Peru for calibration.” says Joel. “We show them signal detection and our number system and what we’re looking for. It’s been extremely valuable and is something we want to do more of.” 

His fear is that producers may see their coffee separated out one year and not the next, and won’t necessarily know why. “We don’t want it to feel arbitrary, so it’s helpful to show people that cupping really is blind, to show them the specific flavors our customers will pay those premiums for. We want to be as transparent as possible, to pull back the curtain so they can see it’s not arbitrary at all.”

Cupping doesn’t have to be complex or mysterious

“There’s a segment of the specialty coffee industry that likes to promote a degree of secrecy around cupping. It’s often seen as this very complex thing you should pay thousands to learn,” says Joel. “Really it’s just like anything else: you just have to practice.” He wants to demystify the process, especially the minutiae of hyperspecific notes. “The key is paying attention to what you are tasting. How do you perceive sweetness in coffee? What does lack of sweetness or dryness taste like? Is the coffee bright? How well integrated is that acidity? What is the body like? Is it round and weighty or thin and quick? Are there any unique flavors? These are the building blocks of cupping coffee.”

So in other words, if any part of this feels mystifying to you, get in touch and we’ll talk about it!