Karl Storchmann and Doug Frost on Wine Judging

Karl Storchmann, economist at Whitman, has an article here on the reliability of wine judges in California State Fair Commercial Wine Competition and concludes that at this competition, at least, the average wine judge is very unreliable. This is a conclusion that by and large I subscribe to...I hazard a guess that at a very large wine competition, it's extremely hard to select judges that are skilled and knowledgeable to judge wines on a consistent basis (and I hazard a guess that both knowledge and skill are key determinants of that ability).

I emailed Doug Frost, Master of Wine and Master Sommelier (and one of only three people in the world with that dual distinction) for a comment. Here is his response:

Eapen - yep, I saw it and there's no question that differing judges have differing responses to wines. And that some judges aren't consistent. For one of the competitions I run, we have a qualifying test in which the potential judges are given nine wines to taste and rank. Then they are given the same wines two more times (in a different order each time) and told to rank them again. Selected judges either get it right 90% of the time or more, or some have a score of perfect. Those are the judges we like.

Doug

I think both Karl and Doug are worth listening to...Hopefully the publicity surrounding Karl's work will be very useful in changing how wine competitions select their judges.