As the Specialty Coffee Association brings a more subjective and qualitative form of coffee quality analysis to the forefront of the coffee industry with its Coffee Value Assessment, we at Red Fox continue to rely directly on large-scale double-blind quantitative analysis (among other tools) to bring in great coffees we and our clients can count on. The quality of the supply chains that bring coffee to our cupping table is primary to us, but to truly assess a coffee’s comprehensive value to our clients, we need to be able to measure its sensory quality and projected longevity with speed and accuracy as well.
To quickly and objectively assess coffee’s value, we employ a technique called signal detection cupping for all offer samples.
What is Signal Detection Cupping
Signal detection cupping, which we developed with intensive training from sensory expert Paul Songer in the early days of Red Fox, is mission critical to our work. It allows us to cup massive quantities of coffee with as much objectivity as possible by double-blinding the coffees. It also allows us to focus on the broader attributes of each coffee by evaluating them on a 1-6 scale, which keeps us from falling into the trap of hyperfocusing on coffee’s individualized flavor notes before we’ve even purchased them.
During signal detection, we:
- Cup three separate samples from each lot sample
- Assign random numbers to each cup (scrambled via spreadsheet)
- Place these cups around the table at random as dictated by their numbers
- Cup the table in order of cup number, rather than by lot
- Score each coffee on a scale of 1-6
- Enter the numbers into the spreadsheet
After each cupping, we:
- Unscramble the numbers via the same spreadsheet
- Assess our aggregated scores to determine the coffees’ scores
- Discuss and revisit coffees that have scores in a gray area or a high disparity between cups or cuppers
- Communicate producer and lot separations to producer and supply chain partners based on scores
We brew the coffees without assessing fragrance or aroma, then move through the table quickly once the coffees are completely cool, taking one or two tastes of each and assigning signal scores. While this process would be overly simple if used by a solo cupper, it works perfectly with a team, ensuring that all cuppers taste all coffees in a uniform and brisk fashion and don’t second-guess their first instincts.
After the data is unscrambled, we have the ability to look at it in a more aggregated form—not focusing on whose coffee we cupped or which score any individual cupper assigned—but also the traceability to see those details if we want or need.
Finding the Signal
The signal detection system uses a 1-6 scale that essentially groups coffees by how we do or do not want to use them. We know that at Red Fox we specifically look for coffees that are clean, sweet, and bright for larger regional lots, and coffees that have those attributes plus added complexity for community ID or producer ID lots.
Here’s what the numbers mean:
- Defective
- Not Red Fox quality (maybe not defective, but falls short of clean and sweet)
- Probably not Red Fox quality (might be clean but not sweet or even clean and sweet but lacking any brightness)
- Probably Red Fox quality (clean, sweet, and bright)
- Yes Red Fox quality (clean, sweet, and bright with added elements of complexity)
- Definitely Red Fox quality (not just clean, sweet, and bright but very complex and distinct)
Each panel participant cups the table a single time at their desired temperature and we come back to the table to taste and discuss coffees in question after the data is entered into the system and analyzed. We rely on the group’s aggregated scores and calibration to generate accurate and comprehensive results rather than on individual opinions.
Signal detection allows us to make sound decisions on large quantities of offers by helping us avoid getting stuck in the weeds on the specific attributes of each coffee and our history with it before we have a quantified analysis.
Double-Blinding to Reduce Bias
One of the core attributes of signal detection cupping is that we use a spreadsheet to double-blind the coffees on the table. This allows us to reduce bias when cupping.
For example, if we’ve purchased a coffee from a specific producer in the past, it’s important for us to have an objective quality assessment of their current offering before we make decisions about their coffee. It not only allows us to make better decisions, but also have better communication and feedback with the producer if needed.
Double-blinding and randomizing coffees across the table also helps avoid any effects of modulation across a single lot. When you taste one coffee divided into three samples, all next to each other, your second and third impressions might be heavily informed by your first impression. By breaking lots up, signal detection helps us truly evaluate each coffee three separate times. Each sample has three unique data points per cupper, and with an average of three cuppers per session, that’s nine data points—a pretty objective evaluation of each sample.
Lot Construction & Separation
One of the best parts of signal detection for our operations is that the scoring system lets us separate coffee into broader tranches, which we can then organize into bespoke lots and separations.
We primarily work with communities of smallholder farmers around the world, some of whose annual offering is under 1 bag per year grown on under a single hectare of land. We cup and score each producer’s lots individually rather than making selections based on large regional type samples, a time- and labor-intensive process that requires a more efficient process like signal detection.
Signal detection’s bulk scoring lets us compose lots intentionally out of all these tiny samples—whether into larger regional lots that represent subregions or larger communities, community ID lots that are specifically designed to represent small subcommunities, neighborhoods, and even families, and individual producers whose coffee we want to separate and sell individually.
We take signal scores and move directly into lot construction, communicating the necessary next steps right away, often within minutes.
Sensory & Note-Focused Cupping Later
Once we’ve used a given signal detection session to purchase coffees, compose lots, and give instruction for separation and transit, the coffee is milled and a sample is taken as a “preship,” which we then receive and cup outside of the signal detection framework.
When it comes to offer samples, we have to use signal detection, but once we have preships, we’re able to move toward a more standard sensory and note-focused cupping. We still typically conceal the names of the coffees while we cup them, though, and avoid discussion until all cuppers have finished with initial evaluation.
We use the same process for arrival samples and regular quality check-ins on SPOT lots.
Who Needs Signal Detection
We at Red Fox need signal detection because we have to rapidly cup through a large volume of small-lot samples as objectively as possible, and organize the results into ready-to-ship lots that best represent each producers’ work. We don’t use signal detection later in the process when we need cup scores and flavor notes, but rather to make speedy, accurate purchasing decisions.
While signal detection might not be the right cupping procedure for every roaster, it does rely on principles that underpin any solid cupping protocol: ensuring consistency, eliminating bias, and relying on group calibration more than individual opinions.
Interested in sourcing coffee with us? Reach out at info@redfoxcoffeemerchants.com. To learn more about our work, check out our journal and follow us on Instagram @redfoxcoffeemerchants.