You guys aren't being fair! The judging is biased! You're playing favorites!
As the 2011 Battle of the Burgers unfolded, we heard all the familiar slams -- especially after readers' beloved burger joints went down. But throughout the process, we strove for fairness, precision, evenhandedness. Slip-ups were inevitable, especially when you're eating 100 burgers in six weeks. But overall, we think we succeeded.
Here's how it worked: Eight judges were assigned to the matchups. For the first three rounds, no judges were allowed to repeat a place that they had previously judged. For the Final Four and the Final, all of the judges ate together. (The Final Four was also judged by Megan Hall, winner of our Guest Taster contest.) For each matchup, judges were required to travel to the first establishment, consume no more than half of the burgers they ordered (We didn't want them feeling too full), and then travel directly to the next burger establishment and repeat the process. At each place, judges were encouraged to order the burger that would allow each place to shine, whether the house special or a server's recommendation or just something that sounded really delicious. Burgers were considered on their individual components -- the flavor of the patties, the freshness of the veggies, the effectiveness of the toppings and cheeses -- as well as their overall flavor profile.
Anonymity was key -- we didn't want any of the burger joints being judged knowing what we were up to. Or at least we tried. In the first round, at Pop's, our judge used his corporate credit card and was quickly "made" by the eagle-eyed owner, Russell House. We instantly ordered a redo with a different judge. (Didn't much matter, Pop's ably beat Red Robin in both matchups.) We also ordered a redo when one of the judges decided to go through the drive-through window at In-N-Out. The parking lot was too crowded, he protested. All burgers must be eaten on premises, we sternly countered. (Again, it didn't much matter -- In-N-Out lost in both matchups to the vastly underrated Grumps.)
And, of course, sometimes we forgot our own rules. Judges were reminded repeatedly to do their research before showing up, in order to figure out the best burger to order. But when two of us showed up at Maple and Motor, we didn't at first realize that the add-ons -- a fried egg, jalapeños, etc. -- are what people say makes their burgers so terrific. Until, that is, Top Chef star Tiffany Derry showed up for lunch on the same day. We decided to eavesdrop on what she was ordering. That's when we started to realize that we might have ordered wrong. And that's when we ordered another do-over. (In both of those matchups, OC Burger maintained a very slight edge over the laudable but sometimes messy Maple and Motor.)
Most importantly, though, we demanded consistency from our hamburgers. What we think distinguishes our Burger Battle from most other "best burger" surveys is that we return to places again and again, asking them to compete head-to-head against a range of burgers. To win this thing, you can't get lucky. You have to deliver a string of ace performances.
And if you think our judges were biased or bit into their burgers with any preconceived notions, trust us when we tell you this: There was the fierce Charley's partisan whose face collapsed in disappointment when he realized that they put too much mayo on his burger, and there was no way he could argue for them over Chop House. There was the judge who dines regularly on his off time at H2 Burger Co., and always comes back raving -- except for the day he kept back from judging them against Keller's and said, "H2 really screwed that one up." And there was the judge who had never before thought much of Pop's, who insisted there was no freaking way they could beat his beloved Fred's, and who took one bite of that crazy Belly Buster and had his entire perception of the burger world shaken.
