Thoughts on Espresso Evaluation

There has been a lot of debate going on, especially at wunder-sites CoffeeGeek and Coffeed, concerning what methodologies, boundaries, limits, variances, etc. are appropriate for evaluating espresso blends much like single origins and cupping. For what it is worth, here's my take.

Coffee, whether green, roasted, ground, or brewed, is an extremely volatile product. The guys at Barismo would be the first to tell you that if the pre-roasted green is skanked, then no level of roasting mastery will make a good cup of coffee. The crop, the level of moisture, the storage, the transportation from farm to processing to warehouse to wholesaler to roaster, the depulping method, the geographical location (both regional and down to the individual farm), and much more! all affect the way that the coffee will roast and taste down the line.

The roast can happen in too humid of conditions, too dry of conditions, type of roaster affects taste, length of time, change in air stability, cleanliness of machine, and whether the roaster has a cold all affect the final product.

Time from roaster to grinder, whether too long or too short, affect it, although I've never seen any hard data on this, just a lot of assertion.

Time from grinder to brewing device (in this case the overly complicated, Gnostic-like-initiation-needed espresso machine) affects it, although everyone's opinion on this seems to differ (30 seconds to staling, 3 minutes, 3 hours, 3 weeks; I've never actually seen any hard data on this, just my own experience and, you guessed it, assertion).

Time in said brewing device, temperature, tamping, leveling, dosing, distributing, barista competence, all affect the final product.

Not to mention those overly subjective taste buds--they just won't bow down to the canons of modern science!

One weak in the chain makes a weak product. The question is, can any entire company or roaster be judged on one lot of coffee? Many variables are beyond the roaster's or the barista's control. Even the best can have bad days. Or not "dial it in" well enough. Let's not forget transportation from one place to another. Yes, coffee from Italy is going to stale before it reaches the states, but probably also coffee that has sat (sitten?) in a hot UPS truck for one day is going to be adversely affected or one that has flown in a cold hulled FedEx plane. How much difference? It is hard to tell, but there is a difference, even in color.

More than any possible (and possibly uncontrollable) weak link in the chain, it is this transportation issue that bothers me the most (green storage is the second, but the barismo.com guys are pretty tight-lipped about their experiments). If we know that transportation affects the beans "adversely", why do we continue to send them long distances for evaluation and expect a fair hearing? The quality of the product has changed and the score has dropped, possibly plummeted. One reason, money. If you ship your beans farther, more people can buy them, which turns into cha-ching. Two reason, branding and furtherance of your name/philosophy. Intelligentsia is a great example of this, and please notice that I don't think any of these reasons are bad reasons, I respect Intelli and have had great coffee from their providers in the Pittsburgh area. Three reason, everybody's doing it. I have yet to meet a roaster that refuses to ship farther than a day away "as the UPS drives". I'm sure they are out there; that is going to be my policy when I start roasting (getting ever closer!), although that might be more from my regional bias than anything else. I'm sure there are other reasons, also, all of which make a formidable barrier to the idea of not shipping. But what about the quality?

I realize that my own Achilles-heel is showing; our drip provider, Grounds for Change, is located way far away (more than a day, that's for sure). However, I'm not using them for evaluation, I'm using them for the taste that I get, even after transportation (which is pretty good, otherwise I wouldn't serve it).

But I don't know if evaluation quality can be maintained through transportation. Imagine if the slighted Italian roasters (just check out their scores!) would agree that the American/Canadian "upstarts" are better than them. They might be right, but only locationally. If an Italian CoffeeGeek had the same test, with the same provisions and methodologies followed by Mark Prince, then I assure you the scores would be reversed. You just can't compare things that have had such divergent histories. It is like comparing a McCormick Reaper (which a great-grandpa of mine had a patent on) to a modern John Deere corn-harvesting-zip-code-having-eat-all-massive-soil-compacting-behemoth tractor in how efficiently they harvest corn. Well, what are the factors? Are we talking large, industrial field, or small, tight, hilly field? Etc. Etc. Etc. (Yul Brenner cameo). So, the results there, at least the Italian stuff, doesn't mean a whole lot. Plus, the initial impetus, the infamous Ken David's review, had a less transparent methodology, so we aren't sure why the Italians fared better: did he have access to fresher Italian roasts or older American roasts; was that batch of BlackCat an off-batch, etc. etc.?

Do I think it is possible to accurately review and evaluated espresso? Sure, but the methodology should be tightened up significantly. Here are my thoughts (with, of course, a provisio that should have been placed at the beginning perhaps: I'm not an accomplished cupper and I'm still working on "dialing in" my shop's espresso, so I'm no expert):

Batches with similar histories since roast should be evaluated, not ones that might have been roasted weeks or months apart.

Anonymous batches should be selected, that is, no roaster should have the opportunity to dress up their roast for the competition while they offer their customers something different.

A disclaimer should proceed all such evaluations of the volatile nature of said reviews. Roasts change from batch to batch, taste buds change, etc. In other words, this is an art, not a science. You cannot, I know, make people actually believe or listen to this disclaimer, but it should be there anyway.

As for the rest, I generally agree with Mark Prince's evaluation criteria and methodology, although I think more than three tasters, and preferably less than half of them related by blood or marriage, should be the rule.

Well, if you've made it this far, I thank you and would greatly appreciate your feedback. Bottom's up!