A peek into our testing process

Here at King Arthur Flour's Baker's Hotline, we notice when a particular recipe starts to garner attention – good or not so good. "Did you see that review on [recipe X]?" "I just had my second call about [recipe X]."

Over the last year, we noticed something was brewing with our Golden Vanilla Cake recipe.

We carefully recorded customer feedback from phone calls and emails, and forwarded the feedback along to our test kitchen team. Bakers from the hotline tested the recipe to see if they could mimic customers' results. We chatted in the break room, at our desks, and via email. Our research and development team was brought in as well.

The two issues readers had noted were dryness in the crumb, and lack of a strong vanilla flavor. Enter R & D team member Jonathan, and his eye for detail.

And so it begins: Testing, testing, testing

Jon tested the Golden Vanilla Cake recipe as originally written and submitted it to our tasting panel. The panel decided that yes, we could make it better.

More milk? Maybe. Beef up the butter? Will sour cream be a spoiler? Cookbook author and high altitude baking expert Susan Purdy has a saying, "You can only change a pie so much before you end up with pizza."  Were we looking to modify – or start fresh, from the ground up?

Jon baked cake after cake, keeping careful notes on the adjustments. As much as we all love to bake, believe me, the fifth, sixth, and seventh rounds of cake really did begin to lose their charm. Who knew you could grow weary of the scent of baking cake?

Not everything tested in the kitchen makes it past the cooling rack. Some experiments go straight to our compost bins. Others make it farther, going to a small group of initial testers for a thumbs-up or down. Eventually, we'll have a version we're ready to move to the next stage... group tastings.

Moving forward: Time for tasting

Once a batch is baked, the finished cakes are sent out for taste-testing. One cake will be the standard recipe, the other the modified version.

Did you know there's actually a science to labeling test samples? I didn't either, for a long time. But we are wired to prefer certain numbers and letters. You'll hardly ever see us test A against B. Why? Well, who doesn't like getting an A rather than a B? A is better in our minds. A+, A #1, A for effort.

For testing purposes, three-digit numbers are the way to go, beginning with at least a 4. The lowest numbers – 1, 2 and 3 – are just like A & B: everyone likes #1, no matter what.

Today's test subjects are #793 and #427. Again, one is the control and one has been modified. Side by side before cutting, they look nearly identical. Good browning, good height, no sinking.

They've both passed the appearance test, but what about flavor and texture?

Perfect for slicing and layering, our goal is butter cake just like what Nana used to make. Forget insipid and flimsy boxed mixes, we want a cake with both flavor and substance. So a Golden Vanilla Cake that's golden without enough vanilla flavor just won't make the cut; nor will one whose texture is off: too dense, "sawdust-y," or dry.

No free snacking: If you bite, you must write!

When we send out samples for a "Bite and Write," they're cut the same and arranged the same way on the tray. We eat with our eyes first, so any differences in appearance may change the way tasters view the product before they even pick it up.

In the olden days (you know, 5 years ago or so), we were a lot more casual about feedback. We'd put a product out and take any comments that came our way. These days we're much more sophisticated, and ask our testers to indicate a clear preference with supporting reasons for the choice. "Soooo yummy" might make the recipe writer feel terrific, but it doesn't go a long way towards aiding in research.

Did you notice the "too close to call?" Yes, sometimes the two samples will be the exact same product, no changes; and sometimes we're hoping a replacement ingredient will so closely match the original that testers will consider them impossible to tell apart.

Almost there: Assessing feedback

So, how did our tasters perceive these cakes? Overall, they both had good flavor and texture. #427 emerged as the moister of the two cakes, with the flavor being just about too close to call. A few tasters preferred #793, but after all the comments were in, #427 emerged as the preferred cake.

Golden Vanilla Cake, perfected!

In the end, two small changes were made to the recipe: the milk was increased by 1/4 cup to create a moister crumb, and the vanilla was increased by 1 teaspoon to bump up the flavor. This cake proves the adage, "A little goes a long way."

We hope you've enjoyed this peek into our testing process. Have you ever made our Golden Vanilla Cake? Let us know about your cake-baking adventures in the comments below.