jump to navigation

A Matter of Taste Tests March 28, 2007

Posted by caveblogem in coffee, Other, statistical analysis.

Back a few weeks ago, just before the Academy Awards, I was listening to this interview on the radio about how sound effects are done for the movies.  This guy in the business explained to the interviewer that you couldn’t always use the sounds that people might actually hear-the actual, authentic sounds.  Sometimes you had to produce the sounds that people expected to hear.  I think that one of his examples was from Blood Diamond or perhaps Babel.  He was saying that part of the film was shot outside of the United States and some of the sounds of birds would not even have been recognized as birds by U.S. moviegoers, so he substituted them for more common sounds in the film–chickens, maybe.  I don’t know.

I bring this up because I have serious doubts about taste tests, particularly where coffee is concerned.  I saw the results of one such a few months ago and, if I remember correctly (and my memory is what experts call capricious, unpredictable, and merely adequate) Dunkin’ Donuts coffee was rated at the top, followed by Green Mountain, with Eight O’ Clock and then maybe Seattle’s Best all coming in ahead of Starbucks.

Now there is no accounting for taste, I know.  But the sound guy interview made me wonder a little.  Isn’t it possible that people doing taste tests for coffee are judging not which tastes best, but which tastes most like coffee?  And if their experience of coffee is growing up with Folgers or Maxwell House or institutional coffee from cafeteria urns, then the ones that remind them of that taste, but slightly less bitter, will rise to the top in a test?

I wonder about this because I didn’t really take my coffee drinking seriously at all until I moved to the Pacific Northwest.  There I went to a school where Starbucks was pretty much the only coffee served.  And it was relatively cheap.  For a small fee the vendor supplied people with a thermos mug (see below, click picture to enlarge).  Refills for the mug pictured below (20 oz, I think, which was the third largest size available–and this was long before people started getting iced coffee drinks) were 74 cents.


To me, that stuff was coffee.  You didn’t need to put cream or sugar or anything in it.  It was sweet and not bitter, and had a lot of flavor. 

A couple of years ago my Mom was out visiting and we decided to take the train into Boston.  There was a Dunk’s at the train station here in Lowell and we stopped to get some coffee and then rushed onto the train. The Dunk’s had a foul smell, but I didn’t think much about it at the time.  Mom took one sniff of her coffee and threw it in the trash before we even got on the train.  I kept mine, and tried several times to drink some, because I really needed the caffeine.  I just couldn’t do it.  When we got to the North Station I dumped it into the trash.  It smelled foul.  But the odd thing is, it smelled just like the kind of coffee I used to get at a cafeteria in San Francisco, which was made by an immigrant couple who drank only tea.



1. strugglingwriter - March 28, 2007

This is a good question, and I’m not sure of the answer. Everyone brings personal biases to these things and what is “best” is subjective anyhow. The key is to get a good (diverse) mix of people so as to counteract this bias. I do like that these test are blind, so as to eliminate “brand biases”

2. itsy - March 28, 2007

i like how you segued from the sounds to the coffee… very slick.

on sounds – i once saw a neat documentary on how nature films are made. naively i assumed all the sounds were taken from the actual moments filled but no, all the sounds are manufactured in studios- some guy with “bear feet” on a stick and a tub of styrofoam estimating the footsteps of a bear cub in snow; breaking broccoli stems to evoke the snap of bones, etc.

about the coffee- i admit i’m a bit surprised at the results. i’ve often heard market analysts and such say that starbucks raised the bar for the average american’s concept of good coffee, but the results seem to indicate the contrary.

3. kaitlyn - March 28, 2007

This is good thinking, about expectations vs reality and how influenced we are by habit vs actual taste. Your segue is graceful. I always think its funny when I see someone order a brown liquid that is simultaneously bubbling and fizzing and steaming, and which contains more salt and sugar than is recommended in a week by dieticians. They’re not bothered–because it’s “Coke”. So “Coke” is a habit, it’s a practice, its an expectation than erases all natural experience. Weird.
Incidentally, did you ever hear that story about the sound effect of the giant ball rolling down the chute toward Indiana Jones in the first movie of that name? The guys at Lucas took a mike out to the parking lot of the studio and recorded someone’s Honda driving slowly over gravel. Wha-Lah.
Also incidentally, I have a peeve about Peet’s, a great independent coffee company here in the Bay Area. Their coffee is burnt. I mean it. It tastes over roasted, deeply acidic–it even gives me that wrinkle in the middle of my forehead normally reserved for tequila! But no one seems to notice this but me! Once I noticed a couple get into a spat in the parking lot outside a Peet’s after both had just ingested their rocket fuel. Now, I’m a Coffee Drinker, no joke. But that stuff?? I’m just grateful they don’t serve it at the UN.

4. caveblogem - March 29, 2007


I suspect that there is a factor involved in the testing that makes these taste tests not only subjective but totally worthless. I knew a guy once who told me about the rigorous training that Starbucks barristas undergo. I suspect, based on personal experience, that the main problem with Dunks’ coffee is not the actual beans or the way that they are processed but the horrible things that are done to it in the brewing process, right there at the store.

I can’t count the number of times I have had incredibly bitter coffee that somebody in the office prepared from excellent beans. Oftentimes it is their mistaken notion that the less coffee you use in the basket the less bitter the coffee will be, when mostly it has the opposite effect.

5. caveblogem - March 29, 2007


Thanks. I suspected that stuff about nature documentaries. It is particularly obvious when you think about Jacques Cousteau. Underwater microphones, anyone? In the 1970s? Hmmm.

As a survey researcher myself I wasn’t so surprised at the results, just interested and somewhat disgusted–these things are so easy to manipulate. I do think that Starbucks has certainly raised the bar for the service they provide, which is a clean, pseudo-intellectual environment and reasonably consistent coffee. But the price. . . .

6. caveblogem - March 29, 2007


Thanks and thanks. The Coke thing is pretty disgusting, I guess. And it (and most other major cola brands) contains phosphoric acid, which is incredibly powerful, stronger than hydrochloric acid, responsible for more cases of acid reflux than anyone really wants to admit. Makes you look at the fizzing and the layer of fog over the top of the glass in a different way, for sure.

I hadn’t heard that about Indiana Jones. You know the radio show, Prairie Home Companion? People go to see it live and marvel at the sound effects guy. What I’d like to see, perhaps as an added thing on DVDs, instead of the insipid interviews and such, is a little window in the upper left hand corner of the screen where you can watch the sound effects guys do all of the noises of the film, jumping around, rattling chains, pounding on chicken carcasses, snapping broccoli stalks, throwing jelly on the wall, smashing crockery. I’d pay good money for that.

Peets was an interesting phenomenon for me. I was living in Washington State in the 1990s, but going home to California, the central valley and bay area for holidays and such. Sometimes people would make presents of Peets wonderful coffee. I tried to seem grateful.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: